e5f62013f4
dataset Python UT: Add Sampler Chain Tests, with get_dataset_size
2020-11-17 15:24:44 -05:00
mindspore-ci-bot
d2427b6e89
!8538 fix get_num_class in imagefolder
...
From: @ziruiwu
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @nsyca
2020-11-13 05:07:11 +08:00
Zirui Wu
3555d19b07
fix get num_class in imagefolder
...
address cmt
review cmts
2020-11-12 14:26:54 -05:00
Jesse Lee
c795879313
CacheOp delivery for HQ tester
2020-11-12 08:48:29 -05:00
Xiao Tianci
4a5e287b92
add filter c api and ut
2020-11-06 15:49:23 +08:00
jonyguo
1b78e34310
add ANTIALIAS support for python progress
2020-11-03 11:03:46 +08:00
82cc7c3f63
[MD] Revert changes to weighted_random_sampler in PR7866
2020-10-31 12:56:17 +08:00
mindspore-ci-bot
cacebd1211
!7933 Cache performance updates
...
Merge pull request !7933 from Jamie/CacheOp_dev
2020-10-31 04:25:47 +08:00
0d40abf5dd
fix md padding bug
2020-10-30 12:00:44 +08:00
Lixia Chen
572c5e5f29
Rebase up to 542a52fbf8
2020-10-29 20:34:09 -04:00
Zirui Wu
ea97197311
fix tfrecord_op core dump
...
add test case
fix ci
fix ci round 2
address review cmts
2020-10-28 17:38:43 -04:00
Xiao Tianci
823e52f1dc
add params checking of offset in distributed_sampler
...
add related comment to explain
add related UT
2020-10-28 16:46:12 +08:00
mindspore-ci-bot
c5867d2d7e
!7676 [MD] fix arm compile error
...
Merge pull request !7676 from liyong126/fix_arm_env
2020-10-24 14:43:39 +08:00
1e7f46c998
fix unorder_map hash
2020-10-24 11:03:06 +08:00
ee042b90f7
fix num_sample in sequentialSampler and randomSampler
2020-10-23 19:26:10 +08:00
8f392204be
minddata build with mindspore-lite train
2020-10-22 21:34:56 +08:00
fb2bd156c9
add unique op
2020-10-22 09:12:37 +08:00
Zirui Wu
c42a1b4a2f
fix batch core dump when col name doesn't exist
...
fix ci
2020-10-13 10:50:07 -04:00
mindspore-ci-bot
6f77ec45f1
!6905 [MD] Enable cache for more leaf datasets
...
Merge pull request !6905 from lixiachen/CacheOp_dev
2020-09-30 16:49:23 +08:00
mindspore-ci-bot
2a799fe90e
!6018 Set the number of epochs in mode.train() non-sink Mode
...
Merge pull request !6018 from h.farahat/ms_nonsink_epochs
2020-09-28 22:15:54 +08:00
82bc278e26
minddata use new ps api
2020-09-27 17:38:22 +08:00
jonyguo
19ca9762db
skip graphdata distributed test case when asan mode
2020-09-27 16:24:20 +08:00
Lixia Chen
477808e622
Enable cache for other leaf ops
2020-09-25 13:11:04 -04:00
mindspore-ci-bot
0d8015b043
!5300 [MD] Add nD Slice Support
...
Merge pull request !5300 from nhussain/slice_op
2020-09-25 20:38:19 +08:00
mindspore-ci-bot
b785b7d0ff
!6813 dataset UT: check PIL version, delete test_callback_2maps
...
Merge pull request !6813 from cathwong/ckw_dataset_cleanup8
2020-09-25 20:34:55 +08:00
hesham
9cee0d2143
Add num_epochs to non-sink training
2020-09-24 23:24:43 -04:00
f2b07d907a
dataset UT: check PIL version, delete test_callback_2maps
2020-09-24 17:01:06 -04:00
mindspore-ci-bot
4c0b3c1bd3
!6017 per_batch_map needs to support x input columns and y output columns
...
Merge pull request !6017 from ZiruiWu/per_batch_map_multi_col
2020-09-25 04:08:01 +08:00
mindspore-ci-bot
e88e114a50
!5930 Cache server phase 2 single node
...
Merge pull request !5930 from Jamie/CacheOp_dev
2020-09-25 03:54:55 +08:00
77d507279c
first working implementation
...
add string implementation, can handle partial string slices
finish core implementation, added Index object
fix up the logic
add in new SliceOption object, cleaning up tests
2020-09-24 15:39:28 -04:00
Lixia Chen
983827ec5c
Rebase up to 88ded11f59
2020-09-24 14:09:08 -04:00
Zirui Wu
0cab373223
per_batch_map supports x num_columns to y num_columns
...
decided to use project after batch as well
per_batch_map 1->x colunm works
add more test case
add test cases, project after map
col order hahaah
fix ci
add test cases
addr review cmts
ci
fix build err
fix test case failure
ci
2020-09-24 12:01:22 -04:00
mindspore-ci-bot
2f3add4acd
!6301 [MD] Combine c++ and python ops in map
...
Merge pull request !6301 from nhussain/c_py_compose
2020-09-24 23:10:47 +08:00
jonyguo
1c6c54ae0f
fix: testcase test_cutmix_batch_op.py slowly
2020-09-23 10:24:39 +08:00
fda9462682
embed python compose op
...
Leaf ops do self-reset.
embed python compose op
add implicit compose op
more tests
2020-09-21 11:24:55 -04:00
shenwei41
f2e34b2eaf
Clean up the view code
2020-09-19 19:02:04 +08:00
mindspore-ci-bot
53a82fa6e0
!6472 fix get_dataset_size error in CelebaDataset when usage is not 'all'
...
Merge pull request !6472 from yanghaitao/yht_celeba_get_dataset_size
2020-09-19 18:07:25 +08:00
mindspore-ci-bot
92aecf128f
!6549 [MD] minddata add iterator parameter validation
...
Merge pull request !6549 from xiefangqi/add_output_numpy_valid
2020-09-19 14:30:10 +08:00
yanghaitao1
4ff4c17632
fix get_dataset_size in CelebADataset when usage is not all
2020-09-19 02:16:29 -04:00
a6360cb2e4
add output_numpy validation to iterator
2020-09-19 11:56:24 +08:00
yanghaitao1
dd0d9fe9ab
fix ci error
2020-09-18 23:49:20 -04:00
yanghaitao1
da15495d7b
fix bucket_batch ci error
2020-09-18 08:16:14 -04:00
mindspore-ci-bot
bef2118253
!6300 [MD] fix get_dataset_size bug when set num_samples in DistributedSampler
...
Merge pull request !6300 from liyong126/fix_md_num_samples
2020-09-16 09:56:53 +08:00
mindspore-ci-bot
0d486adef1
!3974 Eliminate the pause for each cycle of the Repeat
...
Merge pull request !3974 from lixiachen/repeat_task2
2020-09-16 09:35:39 +08:00
930e85ed5c
fix get_dataset_size in distributedSampler & num_samples
2020-09-15 22:53:57 +08:00
f7adf648e9
dataset API docstring: Update datasets, samplers, graphdata and text
2020-09-14 18:10:44 -04:00
Lixia Chen
b5b333726f
Leaf ops do self-reset.
2020-09-14 09:48:14 -04:00
653c79b4af
md fix per_batch_map issue
2020-09-14 09:55:51 +08:00
16147669a6
fix num_samples in concatDataset
2020-09-12 15:06:18 +08:00
mindspore-ci-bot
eb049b0e93
!5842 [Dataset] dataset op log changes
...
Merge pull request !5842 from luoyang/pylint
2020-09-12 09:42:42 +08:00
Cathy Wong
mindspore-ci-bot
Zirui Wu
Jesse Lee
Xiao Tianci
jonyguo
Lixia Chen
hesham
shenwei41
yanghaitao1