Col_lin
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
23,188 Commits 18 Branches 22 Tags 3.6 GiB
Commit Graph

646 Commits

Author SHA1 Message Date
jonyguo 9452142b50 enhance: enable shuffle infile / files when mindrecord 2021-05-14 17:41:18 +08:00
mindspore-ci-bot b5cddcb8cc !16036 Improve performance of multi-processing option for dataset 
From: @robingrosman
Reviewed-by: 
Signed-off-by:
 2021-05-13 01:41:16 +08:00
RobinGrosman 9a7d1fc034 Optimize multi-process in minddata by passing data via shared memory instead of multiprocessing.queue 2021-05-11 15:16:29 -07:00
mindspore-ci-bot ebed057e72 !16196 If Imagefolder input directory is not right, will occur out of memory error 
From: @xiefangqi
Reviewed-by: @ms_yan,@heleiwang,@jonyguo
Signed-off-by: @jonyguo
 2021-05-11 17:36:28 +08:00
mindspore-ci-bot d3a175e30f !16078 Modify Normalize to accept images with more than 3 channels 
From: @hfarahat
Reviewed-by: @mikef,@robingrosman
Signed-off-by: @robingrosman
 2021-05-11 02:03:34 +08:00
xiefangqi e0148f7181 fix minddata directory path problem 2021-05-10 22:16:37 +08:00
mindspore-ci-bot b86747b9a6 !15813 Convert some pipeline Ops to be inlined 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-05-08 05:30:39 +08:00
hesham fd943bbdc0 Modify normalize to support multiple of channels 2021-05-07 17:06:26 -04:00
mindspore-ci-bot e0422e972e !16019 If dataset_dir is not a dir or something else validation is not right somehow 
From: @xiefangqi
Reviewed-by: @ms_yan,@liucunwei,@jonyguo
Signed-off-by: @liucunwei
 2021-05-07 11:57:31 +08:00
mindspore-ci-bot b85bf60eb8 !15727 VOCDataset read xml will filter invalid bbox automationally 
From: @xiefangqi
Reviewed-by: @heleiwang,@jonyguo
Signed-off-by: @jonyguo
 2021-05-07 09:50:53 +08:00
mindspore-ci-bot eb08d2e952 !16024 dataset: modify some error msg in dataset module 
From: @ms_yan
Reviewed-by: @jonyguo,@heleiwang
Signed-off-by: @jonyguo
 2021-05-07 09:12:46 +08:00
ms_yan 34dcd2b463 modify log of dataset part 2021-05-06 23:33:17 +08:00
xiefangqi b6e1fceedc minddata modify dir validation 2021-05-06 21:35:31 +08:00
Lixia Chen 5967cb2cd7 Unblock mindrecord under cache 2021-05-05 13:28:13 -04:00
hesham e4c0bd51a5 Modify concat to be an inlined Op 
Modify zip to be an inlined Op
Modify take to be an inlined Op
Modify skip to be an inlined Op
Modify rename to be an inlined Op
Modify filter to consume child's rows using one thread
 2021-05-04 12:45:18 -04:00
mindspore-ci-bot cefcf6c2ae !15931 Add eager tests for transform ops, and add a shape comparison in the EXPECT_MSTENSOR_EQ Macro 
From: @alexyuyue
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @robingrosman
 2021-04-30 06:15:12 +08:00
alex-yuyue 9ef4331f56 Add eager tests for transform ops 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-04-29 13:19:24 -04:00
liyong 587247e793 add support save dynamic shape tensor 2021-04-27 19:15:10 +08:00
xiefangqi 458ff5fed9 md fix voc and manifest error scenario 2021-04-27 14:24:57 +08:00
dinglinhe 54fb72e2b3 Update normalization description at nn/layer/normalization.py 2021-04-20 12:30:37 +08:00
Cathy Wong 636809c255 dataset: C++ API ToNumber & Lookup: change data_type parm from string to DataType 2021-04-19 12:35:37 -04:00
mindspore-ci-bot f9c5060cf2 !14930 Handle ignored returned Status 
From: @hfarahat
Reviewed-by: @robingrosman,@pandoublefeng
Signed-off-by: @robingrosman
 2021-04-17 03:52:41 +08:00
hesham 6e6eaebb77 Fix unused return value in main code 2021-04-16 11:33:35 -04:00
alex-yuyue 97e3232b6d Add more ut for transform ops and subset sampler 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-04-15 16:51:41 -04:00
mindspore-ci-bot 727203a498 !14911 dataset: Fill op: C++ API support, UTs and Pybind decoupling 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-04-14 05:32:11 +08:00
Cathy Wong 7e6a03487e dataset: Fill op: C++ API support, UTs and Pybind decoupling 2021-04-13 10:02:39 -04:00
mindspore-ci-bot e07a69d8f8 !14874 GNN dataset new feature 
From: @lizhenglong1992
Reviewed-by: 
Signed-off-by:
 2021-04-13 10:44:27 +08:00
Zhenglong Li f3895f2b09 Add get edges from nodes function for gnn network 2021-04-10 18:31:13 +08:00
mindspore-ci-bot f0f7bf1ed1 !14885 fix: wrong input order 
From: @jonyguo
Reviewed-by: @liucunwei,@heleiwang
Signed-off-by: @liucunwei
 2021-04-10 16:52:41 +08:00
jonyguo a5cc81dc97 fix: wrong input order in test_paddeddataset.py 2021-04-09 17:22:31 +08:00
Lixia Chen 1f2b178a16 Reset CacheClient's status for a new run 
Turn off lock only after all data is cached
 2021-04-08 15:30:48 -04:00
mindspore-ci-bot a9ba9b90f6 !14680 MindData Codespell correction 
From: @luoyang42
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
 2021-04-07 18:28:55 +08:00
luoyang 624511337c MindData Codespell correction 2021-04-07 10:20:29 +08:00
hesham 8535e89e6c Remove DataBuffer 2021-04-05 17:55:59 -04:00
hesham 0d86cb1423 Remove rows_per_buffer 2021-04-04 00:22:16 -04:00
mindspore-ci-bot c99fe1e412 !12841 DataBuffer removal from DE 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-04-04 03:28:13 +08:00
luoyang ec21d3f748 Fix validation of RGB2HSV/HSV2RGB/RandomResizedCrop in py_transform 
Fix error msg of pythontokenizer
 2021-04-01 19:09:07 +08:00
hesham a9a80693be Replace Databuffers with TensorRows 2021-03-31 17:00:36 -04:00
xulei2020 4122d1dcce fix code format bug 2021-03-25 09:18:36 +08:00
YangLuo f99204b292 fix python tokenizer 2021-03-18 15:47:09 +08:00
Lixia Chen 30017ea5a0 Fix codex issues 2021-03-15 11:15:15 -04:00
mindspore-ci-bot ac18da3770 !13061 [MD][bugfix] Fix to_json error: can't pickle pyfunc 
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
 2021-03-13 12:48:26 +08:00
Xiao Tianci 4a7100c7f9 fix RandomAffine degree error and some examples in docs 2021-03-12 17:39:38 +08:00
mindspore-ci-bot 682a8926bd !13176 fix minddata transform issue 
From: @luoyang42
Reviewed-by: 
Signed-off-by:
 2021-03-12 17:01:56 +08:00
xiefangqi 41f3e02e87 fix pyfunc pickle issue 2021-03-12 15:17:08 +08:00
YangLuo 74d7bb41a6 fix minddata transform issue 2021-03-12 11:37:14 +08:00
mindspore-ci-bot f7040ca134 !13198 Fix get_dataset_size error with PKSampler 
From: @ziruiwu
Reviewed-by: 
Signed-off-by:
 2021-03-12 06:11:26 +08:00
mindspore-ci-bot 1bb789278d !13200 Fix CI warnings and test team issues 
From: @alexyuyue
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @robingrosman
 2021-03-12 05:38:01 +08:00
Zirui Wu 2692e3cc3d fix pk sampler's get_dataset error due to num_class unavaiable at pre-runtime 2021-03-11 15:43:02 -05:00
alex-yuyue 2457f52596 Fix CI warning and test team issues 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-11 13:34:51 -05:00