Commit Graph

242 Commits

Author SHA1 Message Date
luoyang 4f98ecb433 Fix bugs in c-api: rename, concat, take, sampler, duplicate column &
Change docstring of OneHot
2020-08-28 14:22:56 +08:00
mahdi a5228c75c7 Fixed 2D one-hot label problems in CutMix and MixUp 2020-08-27 22:03:39 +08:00
mindspore-ci-bot e3b0ae75ae !5286 [MD] minddata gpu add circular_memory to device_queue
Merge pull request !5286 from xiefangqi/xfq_add_circular_gpu_r0.7
2020-08-27 10:12:23 +08:00
xiefangqi 6d0ae1794f add circular_memory function to gpu 2020-08-26 21:49:21 +08:00
avakh 6300058cb6 fixing a bug in random sharpness 2020-08-26 21:36:52 +08:00
qianlong 6b6409d910 DecodeAndCropAndResizeJepg do not support odd size 2020-08-26 15:32:18 +08:00
mindspore-ci-bot fd8ad73689 !5194 fix: padded dataset when no div and with repeat op for br:r0.7
Merge pull request !5194 from guozhijian/fix_padded_with_no_div_repeat_r0.7
2020-08-26 10:47:37 +08:00
qianlong bc8aec007f fix softdvpp coredump 2020-08-26 09:07:35 +08:00
jonyguo d262c63214 fix: padded dataset with non div & repeat 2020-08-26 08:58:24 +08:00
luoyang a75ac9c445 Add type check for RandomPosterize & Add Float tensor support for CutMixBatch 2020-08-25 15:10:24 +08:00
mindspore-ci-bot 9b503e4f38 !4955 Fixes for Dynamic Augmentation Ops
Merge pull request !4955 from MahdiRahmaniHanzaki/dynamic-ops-fix
2020-08-22 10:27:50 +08:00
mindspore-ci-bot 8ee136db18 !4953 C++ API: Minor fixes for dataset parameters
Merge pull request !4953 from cathwong/ckw_c_api_fixes1
2020-08-22 09:14:08 +08:00
Mahdi a5f9b8f92e Added fix for MixUpBatch and CutMixBatch and for RandomAffine
updated c color op descriptions
2020-08-21 17:40:08 -04:00
Cathy Wong 93810a0dc8 C++ API: Minor fixes for dataset parameters 2020-08-21 17:35:39 -04:00
mindspore-ci-bot 9d7250c483 !4776 Introduce 2 extra ctrl flags to DataBuffer in dataset, address remaining cmts to PR4632
Merge pull request !4776 from ZiruiWu/map_callback_follow_up
2020-08-22 03:31:31 +08:00
Zirui Wu 74c1e6da60 introducing pause and quit flags to DataBuffer
fix review cmts

fix ci

fix Ci

fixci

address ci

ci

- add timeout
- add more test cases

fix CI

address review cmts
2020-08-21 14:33:26 -04:00
Jesse Lee ebd4cc5c0a Fix glog off compilation bug 2020-08-21 09:37:22 -04:00
mindspore-ci-bot 82c888f065 !4930 Fix CI cifar hang issue
Merge pull request !4930 from xiefangqi/fix_cifar_nofile_issue
2020-08-21 20:49:40 +08:00
xiefangqi e3e7820413 fix cifar stuck problem 2020-08-21 19:35:07 +08:00
mindspore-ci-bot b2a5440548 !4883 Fix compilation warnings
Merge pull request !4883 from zhoufeng/xiu-ba-ge
2020-08-21 12:33:20 +08:00
zhoufeng 22ec976ef7 fix compilation warnings
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-08-21 09:44:04 +08:00
mindspore-ci-bot 11e670c54b !4863 Fix bugs in c-api text
Merge pull request !4863 from luoyang/c-api-pyfunc
2020-08-21 09:34:14 +08:00
mindspore-ci-bot 4ee0bf22c3 !4820 Alarm modification
Merge pull request !4820 from shenwei41/sw_master
2020-08-21 09:26:25 +08:00
mindspore-ci-bot 5a0fe979ab !4871 GE backend does not use glog (-g off) and is causing linking error on linking cache_server exe
Merge pull request !4871 from JesseKLee/tdt
2020-08-21 02:07:19 +08:00
Jesse Lee c9d48817ef glog 2020-08-20 12:33:27 -04:00
mindspore-ci-bot 9a3baf4f6c !4314 [MD] remove defaults used in binding code, and remove bilinear resize op
Merge pull request !4314 from nhussain/clean_up_bindings
2020-08-20 22:16:56 +08:00
mindspore-ci-bot 4fe2892b89 !4771 Fixes Comment style
Merge pull request !4771 from EricZ/md_tensor_from_mem
2020-08-20 22:08:18 +08:00
luoyang 536cc335b2 Fix bugs in c-api text 2020-08-20 21:27:38 +08:00
mindspore-ci-bot 98052f9d06 !4847 [MD] Add some data validation
Merge pull request !4847 from xiefangqi/xfq_add_numworker_validate
2020-08-20 20:22:51 +08:00
shenwei41 a4e7a75891 Modify alarm 2020-08-20 19:28:50 +08:00
xiefangqi 7dc7990d75 add NumWorkers validate 2020-08-20 17:25:30 +08:00
limingqi107 5b76e8f3d7 gpu add format transform pass 2020-08-20 16:08:21 +08:00
Jesse Lee 8a08d0c37b Phase 2 of CacheOp 2020-08-19 13:45:55 -04:00
Zirui Wu 78c1aa1d96 Implemented Callback for Dataset
implment pause in MapOp, added more to callback

add ds_callback

- Initial drop of Python DSCallback

- Pybind DSCallback

- Pybind DSCallback

added callback to mapOp

- de_pipeline DSCallback

- de_pipeline DSCallback

add test case, segfault for now

fix seg fault

- de_pipeline DSCallback

remove 1 line

update callback test case, now works

use builder class for mapOp callback

- de_pipeline DSCallback

- de_pipeline DSCallback

- de_pipeline DSCallback

better test case

minor fix

add comments and minor clean ups

get rid of nullptr in MapOp, use other flag instead

fix a bug ParseMapOp only takes 1 callback

- Added WaitedDSCalabck

refactor callback param

fix text case incorrect number

- added testing

fix cpp test case

- added testing

- revert back lenet changes

- cleanup test_callbacks.py

- cleanup test_callbacks.py

fix CI stage I

fix CI stage II

fix CI and update epoch counter

- add validation
- add more testing  test_callbacks.py

use random data op to do tests

adjust when to call EpochBegin/End

- add repeat with callback

- addressing reviewers' comments

- docstring and CI fixes

- docstring and CI fixes

- docstring and CI fixes

- rebase with upstream/master

fix cpp test case

fix review comments

addr review cmts, add test case
2020-08-19 12:31:04 -04:00
Eric d799373950 Added created from memory 2020-08-19 12:06:04 -04:00
YangLuo b50ae27c72 !4597 [Dataset] C++ API Support for build_vocab 2020-08-19 22:47:47 +08:00
nhussain eb9a611041 remove old defaults 2020-08-19 10:11:19 -04:00
mindspore-ci-bot cda930f9e4 !4759 Fix duplicate column names of CSV Dataset
Merge pull request !4759 from jiangzhiwen/dataset/fix_duplicate_column_names
2020-08-19 21:14:31 +08:00
jiangzhiwen 74be92d698 fix duplicate column names 2020-08-19 20:12:07 +08:00
mindspore-ci-bot d541e261a0 !4696 C++ API Support for CSV Dataset
Merge pull request !4696 from jiangzhiwen/jzw/c_api_csv
2020-08-19 17:08:33 +08:00
mindspore-ci-bot 0fa0cd7bfe !4710 Fix tfreader multi-thread issue
Merge pull request !4710 from xiefangqi/xfq_fix_tfreader_issue
2020-08-19 16:10:31 +08:00
mindspore-ci-bot 256dccc651 !4498 Gnn data processing supports distributed scenarios
Merge pull request !4498 from heleiwang/gnn_distributed
2020-08-19 16:09:05 +08:00
jiangzhiwen 39df25f803 c++ api for CSV Dataset 2020-08-19 16:02:11 +08:00
heleiwang 8ee4d8e92d Gnn data processing supports distributed scenarios 2020-08-19 11:09:01 +08:00
mindspore-ci-bot 6868b9b6fa !4422 Minddata new dataset api ==> RandomData
Merge pull request !4422 from xiefangqi/xfq_c++api_randomdata
2020-08-19 10:48:53 +08:00
xiefangqi 8e0a7bfbd3 fix tfreader WaitPost register failed 2020-08-19 09:24:11 +08:00
mindspore-ci-bot 58523a41fe !4392 use builtin float16 for arm
Merge pull request !4392 from xychow/use-float16-in-arm-neon
2020-08-19 09:17:09 +08:00
xiefangqi b91e56375e add randomdataset and schema 2020-08-19 09:13:43 +08:00
nhussain 738ae2c78d change api to use std_vector
try to display image
2020-08-18 15:41:21 -04:00
islam_amin 979111d0ef Added Posterize Op 2020-08-18 13:55:47 -04:00