timekiller
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
42,204 Commits 19 Branches 0 Tags 564 MiB
Commit Graph

30 Commits

Author SHA1 Message Date
xiefangqi f81c7967ec fix config interface comments 2022-03-22 10:04:12 +08:00
xiefangqi 2de6c8908b add a clean up subprocess to clean subprocess when main process exit 2022-03-05 17:24:11 +08:00
RobinGrosman c90efc0403 add check for shm available when num_devices=1 to avoid bus errors 
-Required modifying tests to reduce memory usage to check doesn't fail during CI
-Also required fixing max_rowsize to be externalized from map+batch() function
 2021-09-24 10:24:31 -07:00
ms_yan 36a8886ca2 Revert "[feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset" 
This reverts commit b077aa1cab.

Revert "[feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset"

This reverts commit 4e6f7dc97d.

delete pass_registry_test.cc

comment  hiai_nlu_model_multi.pb related  line
 2021-08-23 01:46:38 +08:00
djc 4e6f7dc97d [feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset 2021-08-22 13:39:37 +08:00
YangLuo 9352935ff6 Fix probability core dump in MindData coverage test 2021-08-19 14:34:57 +08:00
Harshvardhan Gupta 6845a16b44 fix coredump with large max_rowsize and add arg validations 2021-06-23 09:31:11 -04:00
hesham 0d86cb1423 Remove rows_per_buffer 2021-04-04 00:22:16 -04:00
RobinGrosman a0e75b845a Increase default level of parallelism in mindddata pipeline to 8. 
Adjust memory usage to not increase as parallelism increases.  it will
stay at same level it would be with 4 parallelism
 2021-03-10 18:38:11 -08:00
Zirui Wu 4ae7102fd3 address 2 tickets 2020-12-18 15:03:32 -05:00
Zirui Wu d6df1b0832 Implemented AutoNumWorker Pass which sets num_workers of selected parallel ops automatically if enabled 2020-12-08 17:49:05 -05:00
xiefangqi 9b3c33e157 minddata iterator output ms_tensor 2020-09-11 11:35:50 +08:00
nhussain 92e99ff224 change map calls 2020-09-10 10:25:05 -04:00
nhussain 3bac9d3713 switch input columns and operation 
change ImagefolderDV2 name

change ds.transforms.vision to ds.vision

change batch api to match map api more closely

compose op changes

test_pylint

remove compose op from vision, move to transform module, refactor map and batch to use column_order
 2020-09-08 10:26:20 -04:00
Lixia Chen 27d5281641 Change epoch count to 1 for python testcases 2020-09-01 11:24:26 -04:00
tony_liu2 70bfd506a1 add function 
pr comment fix

revert graphengine
 2020-08-27 16:50:19 -04:00
tinazhang 340d98a4d1 added test case to cifar_op 
update cifar10 dataset
fixing missing error handling code in validator
 2020-07-14 11:22:38 -04:00
yanghaitao 365c901ee0 get default value if num_parallel_worker is None 2020-06-30 21:10:37 +08:00
mindspore-ci-bot c70e6987ba !2624 Fix config test 
Merge pull request !2624 from EricZ/fix_config_test
 2020-06-30 01:11:12 +08:00
yanghaitao fd701b8504 set monitor sampling interval through json file 2020-06-28 15:18:21 +08:00
Eric a4f5802924 Added Fix for test config 2020-06-27 15:20:48 -04:00
Cathy Wong 277aba5326 dataset: Fixup docs; remove pylint disabled messages in UT 2020-06-26 12:26:41 -04:00
Yang 9b2a778d94 !1478 [Dataset] clean pylint. 2020-05-28 14:06:36 +08:00
Yang abca62f407 10:00 26/5 clean pylint 2020-05-26 10:05:12 +08:00
Cathy Wong f891e1755c Cleanup dataset UT: restore config support 2020-05-25 15:10:03 -04:00
jinyaohui 5a914994ba clean pylint 2020-05-18 16:42:35 +08:00
jinyaohui bcfaff97f9 clean pylint 2020-05-18 10:31:46 +08:00
eric 26cb3e8a5f Added test function to show that seed doesn't work. 
Added testcase to show that c image aug don't use seed properly

Added passing test cases

Added working testcases for using seed

Added additional test cases to show seed use

Added test case for seed
 2020-04-30 18:57:59 -04:00
eric cd94518769 X# This is a combination of 2 commits. 
Initial commit for dataset op python

Added signature to barrier

Adde compiling barrier code

Rebasing, fixed new compile errors

Final fix for make_unique

Added pybind API for barrier

Fixed pyfunc invocation

python interface - sync_wait

!1 sync_wait python interface
* python interface - sync_wait

fix test

update test

update test

Added new test case

add test case

test for shuffle + batch

Added two-sync test case

Restrited that no shuffle after sync

Added sync to pipeline info

block first databuffer as well

Intelligently get batch size

Fix default case

Lock Pair shares among all iterators

Added fix for empty character

Fixed up test case formatting

Fix end of epoch in sync_wait

Fixing CI
 2020-04-21 17:42:13 -04:00
zhunaipan 930a1fb0a8 initial version 
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
 2020-03-27 22:54:54 +08:00