Commit Graph

690 Commits

Author SHA1 Message Date
i-robot 8508c9eb45 !18179 [MD] add slice patches op in c++ and python
Merge pull request !18179 from liyong126/slice_patches_op
2021-07-05 07:04:45 +00:00
liyong 9a1de0af65 add slice patches in python and c++ 2021-07-05 09:16:47 +08:00
Xiao Tianci 8cdd0d8d90 fix param check for Rotate and GaussianBlur 2021-07-01 20:31:33 +08:00
YangLuo 17c121b087 Give concreted error when mappable dataset lack __len__ method 2021-06-29 11:13:12 +08:00
Lixia Chen 1f03b16e0e Fix cache_admin server start stderr output
Fix TFRecord get_dataset_size() when cached
2021-06-23 18:32:54 -04:00
Harshvardhan Gupta 6845a16b44 fix coredump with large max_rowsize and add arg validations 2021-06-23 09:31:11 -04:00
YangLuo 4e9f6d54f7 update cpp api & doc 2021-06-23 11:14:52 +08:00
chenx2ovo 4b98099dcb [feat][assistant][I3CEGQ] add op rgb2bgr 2021-06-21 16:16:50 +08:00
i-robot ab3259c43a !18292 MD Profiling UT: Check CPU util file. Add sequential pipeline UTs
Merge pull request !18292 from cathwong/ckw_mon_py_analyze_ut4
2021-06-17 21:47:10 +08:00
liyong 2989a5a23b fix some bugs 2021-06-17 11:56:10 +08:00
Cathy Wong c37eb350f6 MD Profiling UT: Check CPU util file. Add sequential pipeline UTs 2021-06-16 15:30:59 -04:00
Xiao Tianci a54e65a7ad add Crop Python API 2021-06-10 09:46:57 +08:00
i-robot e60f5f12f5 !17836 Add Rotate API for CV data processing
Merge pull request !17836 from xiaotianci/rotate
2021-06-10 09:29:18 +08:00
i-robot 50f11d5439 !17503 dataset: enhance the expression of error msg
Merge pull request !17503 from ms_yan/log_modify
2021-06-08 22:25:04 +08:00
Xiao Tianci 9c4c1c6a77 add Rotate Python API 2021-06-08 22:19:15 +08:00
i-robot 3a63a66d64 !17807 MD Profiling UT: Add MD Analyze UT for MinddataProfilingAnalyzer
Merge pull request !17807 from cathwong/ckw_mon_py_analyze_ut3
2021-06-08 21:52:55 +08:00
ms_yan be65ebcb28 complete dataset log 2021-06-08 16:45:47 +08:00
mindspore-ci-bot 8934e4a513 !17879 dynamic min max shape phase 2
From: @luoyang42
Reviewed-by: @liucunwei,@jonyguo
Signed-off-by: @liucunwei
2021-06-08 14:49:13 +08:00
Cathy Wong b7342efe12 MD Profiling UT: Add MD Analyze UT. Add more pipeline profiling UTs. 2021-06-07 11:00:34 -04:00
luoyang dad289c679 dynamic min max shape phase 2 2021-06-07 20:06:26 +08:00
Zhenglong Li 2c7d1bf603 Implement new gnn feature 2021-06-07 10:20:06 +08:00
mindspore-ci-bot b52f9ddb08 !17060 Add HorizontalFlip and VerticalFlip for CV data processing
From: @tiancixiao
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
2021-06-02 17:48:28 +08:00
mindspore-ci-bot c4757f0692 !17461 MD Profiling Analyze: Handle cpu util with missing ops. Enhance UTs.
From: @cathwong
Reviewed-by: @robingrosman
Signed-off-by: @robingrosman
2021-06-02 11:12:59 +08:00
mindspore-ci-bot ae2e7c1288 !17541 Fix MD Cache code check warnings [master]
From: @lixiachen
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @robingrosman
2021-06-02 04:53:54 +08:00
Lixia Chen 8c1b561149 Fix cache code check warnings 2021-06-01 10:02:38 -04:00
Cathy Wong f8beb933ad MD Profiling Analyze: Handle cpu util with missing ops. Enhance UTs. 2021-05-31 20:33:19 -04:00
Xiao Tianci 9c74d55a49 add HorizontalFlip and VerticalFlip APIs 2021-05-31 15:55:21 +08:00
hesham 3a9695313b files_need_cleanup fixes 2021-05-30 15:44:08 -04:00
mindspore-ci-bot 2433778dac !15963 Support getting dynamic_min_max_shape of data
From: @luoyang42
Reviewed-by: 
Signed-off-by:
2021-05-26 09:41:02 +08:00
mindspore-ci-bot 730e211e52 !16702 Add GaussianBlur API for CV data processing
From: @tiancixiao
Reviewed-by: @liucunwei,@jonyguo
Signed-off-by: @liucunwei
2021-05-26 09:09:41 +08:00
mindspore-ci-bot d0978f709a !16167 CocoDataset add a new column according a new arg extra_metadata
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
2021-05-25 19:55:41 +08:00
mindspore-ci-bot e513786709 !16565 Bug in Normalize when num_channels > 128
From: @hfarahat
Reviewed-by: @robingrosman,@pandoublefeng
Signed-off-by: @robingrosman
2021-05-25 10:24:25 +08:00
xiefangqi 53891dbdf7 coco add img_id column 2021-05-24 17:19:58 +08:00
luoyang 13ef5b61f3 support getting dynamic_min_max_shape of data 2021-05-24 17:04:50 +08:00
Xiao Tianci 3f495bbae1 add GaussianBlur API 2021-05-24 16:27:05 +08:00
mindspore-ci-bot 05bbd1d8fa !16426 Change the permission of cache shared memory to 600 for security
From: @lixiachen
Reviewed-by: @robingrosman,@pandoublefeng
Signed-off-by: @robingrosman
2021-05-20 02:36:26 +08:00
mindspore-ci-bot 1cb2d31642 !16509 GNN dataset new feature for invalid edge or node id case
From: @lizhenglong1992
Reviewed-by: @liucunwei,@heleiwang
Signed-off-by: @liucunwei
2021-05-19 09:20:24 +08:00
hesham bc92011b5f Bug in Normalize when num_channels > 128 2021-05-18 12:23:11 -04:00
Zhenglong Li 9284b6d6af Add special case returned value for get feature function 2021-05-18 15:53:01 +08:00
mindspore-ci-bot ec9fc5fb3b !16384 Fix python multiprocessing with shared memory for edge case
From: @robingrosman
Reviewed-by: @mikef
Signed-off-by: @mikef
2021-05-18 10:30:19 +08:00
Lixia Chen 8812111f94 change shared mem perm to 600 & minor other fixes 2021-05-17 17:19:55 -04:00
mindspore-ci-bot 601322356b !15638 VOCDataset need to add img_id for evaluation
From: @xiefangqi
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
2021-05-17 19:33:24 +08:00
xiefangqi 9b846c7873 voc add image_id column 2021-05-17 16:09:04 +08:00
RobinGrosman ca88d7d76a fix edge case and docstring in minddata multiprocessing shared memory code 2021-05-16 06:46:44 -07:00
jonyguo 9452142b50 enhance: enable shuffle infile / files when mindrecord 2021-05-14 17:41:18 +08:00
mindspore-ci-bot b5cddcb8cc !16036 Improve performance of multi-processing option for dataset
From: @robingrosman
Reviewed-by: 
Signed-off-by:
2021-05-13 01:41:16 +08:00
RobinGrosman 9a7d1fc034 Optimize multi-process in minddata by passing data via shared memory instead of multiprocessing.queue 2021-05-11 15:16:29 -07:00
mindspore-ci-bot ebed057e72 !16196 If Imagefolder input directory is not right, will occur out of memory error
From: @xiefangqi
Reviewed-by: @ms_yan,@heleiwang,@jonyguo
Signed-off-by: @jonyguo
2021-05-11 17:36:28 +08:00
mindspore-ci-bot d3a175e30f !16078 Modify Normalize to accept images with more than 3 channels
From: @hfarahat
Reviewed-by: @mikef,@robingrosman
Signed-off-by: @robingrosman
2021-05-11 02:03:34 +08:00
xiefangqi e0148f7181 fix minddata directory path problem 2021-05-10 22:16:37 +08:00