qsdy
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,277 Commits 18 Branches 22 Tags 3.6 GiB
Commit Graph

97 Commits

Author SHA1 Message Date
Cathy Wong 78392e0d66 dataset: Pybind change for data transforms 2021-04-06 10:10:14 -04:00
mindspore-ci-bot 0cfd8b7c9e !13364 Add WordpieceTokenizer and AddDict to Jieba 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-04-06 05:04:13 +08:00
hesham 0d86cb1423 Remove rows_per_buffer 2021-04-04 00:22:16 -04:00
ms_yan b6ef88c168 add stop_dataset_profiler option 2021-03-27 20:36:40 +08:00
alex-yuyue 2d6d608fae Add WordpieceTokenizer to API and IR. Add AddDict to C++ API 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-18 16:42:31 -04:00
Eric 4be0ffb9ba removing type_id.h from header files 2021-03-16 16:51:10 -04:00
ms_yan 92e86804e1 init add acltdt handle create and destory 
add hostpush part modify

optimize previous code

provide aclhandle access method

modify CMakeList format

add device_id parameter into TransferNode

update acltdt api
 2021-03-15 12:45:09 +08:00
Eric 82f7c6f2c7 Moving data_helper 
exposing json

WIP fix for removing templates

Removing templates part 1

replacing core/constants.h

removed template part 2

Changed fake data to filler data

Fixing CI

Fix CI part 2

CI fix part 2

Fix CI 3

Fixed lizard for data_util

Fixed clang

Fixing CI part 4
 2021-03-12 15:30:34 -05:00
alex-yuyue 2457f52596 Fix CI warning and test team issues 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-11 13:34:51 -05:00
Zirui Wu 8022001450 Remove datasets.h dependency on tensor.h and other related dependencies 
Replace std::function<TensorRow(TensorRow)> with std::function<MSTensorVec(MSTensor)> by using std::bind

Replace pad_info with MSTensor

Switch TensorShape/dataset::DataType to vector<int64> /mindspore::DataType

Change TypeId to mindspore::DataType in SchemaObj

address review cmts, add CreateFromMSTensor() in tensor.h/.cc
 2021-03-05 16:26:58 -05:00
Eric d328d6eb54 Adding affine 
Need to move Affine to image_utilC

Refactored affine

Added affine API

Added API class impl

Removed default from vision ir

Compiling

removed extra file

Spell check
 2021-02-25 16:55:02 -05:00
mindspore-ci-bot ced5575387 !12353 Move sampler IR code to engine/ir 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-02-13 06:44:47 +08:00
mohammad 74bc54b694 push down sampler IR files 2021-02-11 17:26:13 -05:00
mindspore-ci-bot ed7fef5d5e !12349 [MD] Push down IR files for transforms and text 
From: @tina_mengting_zhang
Reviewed-by: 
Signed-off-by:
 2021-02-12 00:01:13 +08:00
TinaMengtingZhang 462422acc7 Move transforms and text ir down 2021-02-10 15:04:07 -05:00
mindspore-ci-bot 806b639f02 !9720 restructure python bindings 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-02-11 02:21:32 +08:00
mohammad 52334c8e4d restructure python bindings 2021-02-09 19:38:01 -05:00
Cathy Wong fc8b9fc85e dataset: Push down IR vision files 2021-02-09 16:27:08 -05:00
hesham 37add20783 Fix caching problem after cleanup 2021-02-08 20:39:23 -05:00
mindspore-ci-bot e6dcb0c31d !12239 [MD] fix bugs of execute & update doc description in minddata 
From: @luoyang42
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
 2021-02-08 10:15:08 +08:00
luoyang cd86788cf3 fix bugs of execute & update doc description in minddata 2021-02-07 22:52:52 +08:00
gongxiaoqing 7f538b51e7 回退 'Pull Request !11074 : replace tdt with acltdt interface' 2021-02-06 14:42:14 +08:00
mindspore-ci-bot c2582dcab9 !11074 replace tdt with acltdt interface 
From: @ms_yan
Reviewed-by: @lilongfei15
Signed-off-by:
 2021-02-06 13:22:39 +08:00
hesham 4bbc3445f1 datasets.py cleanup 2021-02-05 09:50:27 -05:00
ms_yan 293f81128d init add acltdt handle create and destory 
add hostpush part modify

optimize previous code

provide aclhandle access method

modify CMakeList format

add device_id parameter into TransferNode
 2021-02-03 11:23:27 +08:00
luoyang fa4f67cbcd LookupOp: default value of unknown_token support NoneType 2021-02-02 16:17:02 +08:00
lixian 7d2fd6e76c cxx api refactor: tensor/status/model 2021-02-02 09:15:56 +08:00
mindspore-ci-bot 320ea51308 !11853 Add call for decoupled image and text ops 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-02-01 19:47:00 +08:00
alex-yuyue 6fd58dc580 Add call for decoupled image and text ops 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-01-29 21:36:31 -05:00
Mahdi 8f34faeb7a Changed bindings to SamplerObj 2021-01-29 14:20:05 -05:00
mindspore-ci-bot 20c1727b2c !11631 Add SubsetSampler 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-01-29 03:48:45 +08:00
heleiwang 2dc9ba761c gnn support weight sampling neighbors. 2021-01-28 15:28:44 +08:00
hesham 7120c66998 Add SubsetSampler 2021-01-27 15:00:18 -05:00
mindspore-ci-bot 029ea5d1f3 !11301 Push sampler to GeneratorOp c++ 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-01-26 01:31:43 +08:00
hesham c3278c983d Move GeneratorOp sampler to c++ 2021-01-22 12:03:40 -05:00
alex-yuyue a8675f8227 TextOp decoupling 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-01-20 21:52:45 -05:00
mindspore-ci-bot 0e03630503 !11139 [MD][BugFix] MindData remove the compile link of _c_dataengine and numa 
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
 2021-01-15 09:12:55 +08:00
mindspore-ci-bot a2d9c5914a !10944 Tensor op decoupling stage 2 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-01-15 05:28:50 +08:00
mindspore-ci-bot ee3cc09b22 !11259 Fix CI issues 
From: @ezphlow
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @robingrosman
 2021-01-15 05:15:10 +08:00
alex-yuyue b54e01bfb4 Tensor op decoupling stage 2 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-01-14 11:51:16 -05:00
Eric f758bfa79f CI fix part 1 
Added CI fix
 2021-01-13 22:03:58 -05:00
Cathy Wong 4b1ccf9dd5 dataset: Use int32_t for text's vocab_size 2021-01-13 18:12:50 -05:00
TinaMengtingZhang 4812fece80 [Part I] Push down json save logic to IR and add getter to each IR node 2021-01-12 11:17:45 -05:00
xiefangqi f09fdb2679 change md numa link to dllload 2021-01-12 14:59:09 +08:00
mohammad f48ab2b5c9 make MD public headers standalone 2021-01-11 06:35:53 -05:00
Eric be46ccf721 Started vision decoupling 
map validate params

Changing bindings

Change behavior of vision.cc

Added signature and changed Cmake

Compiling

Need to fix compile

Added compiling random transforms

python changes for decoupling

two failed test case remaining

Compiling random choice

passes all ut

Changed assert

review comments

Added validate params

add back return value

Fix lint

py lint fix

pylint 2

Addressing comments
 2021-01-08 09:42:20 -05:00
mindspore-ci-bot bccf04d36c !8378 [MD][GPU] minddata device_que push optimizer 
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
 2020-12-15 17:31:47 +08:00
mindspore-ci-bot c4f284b928 !9910 [MD][Perf] MindData add NormalizePad for GPU performance 
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
 2020-12-15 16:51:22 +08:00
xiefangqi ae36071fc8 add push opt logic 2020-12-15 15:35:42 +08:00
xiefangqi 3c6cc3c58c Add NormalizePadOp 2020-12-15 11:45:41 +08:00