Commit Graph

641 Commits

Author SHA1 Message Date
Cathy Wong 46e9d79a50 C++ API: Lexicographical order support for CLUE, CSV & TextFile Datasets 2020-08-25 14:11:19 -04:00
luoyang b26f6b3d5e fix bug in c-api: rename, concat, take 2020-08-25 16:25:00 +08:00
mindspore-ci-bot 0bbce9367e !4772 Adding AlbumDataset for device training
Merge pull request !4772 from EricZ/ms-album
2020-08-24 08:27:14 +08:00
Eric c79db93c48 Initial commit for album
Added linter fix for album dataset

Added testDataset

Adding signature

Added JsonDataset example API

Example dataset

Resolving format

More fixing

Refactor

Small fix

Added compiling album dataset

Running tests

Added linter fix #1

Passing UT

Added dataset API

Addressing clang

Clang part 2

Fixing pass

Fixed tree check

lint fix

Added lint fix part 2
2020-08-22 11:07:10 -04:00
mindspore-ci-bot 9b503e4f38 !4955 Fixes for Dynamic Augmentation Ops
Merge pull request !4955 from MahdiRahmaniHanzaki/dynamic-ops-fix
2020-08-22 10:27:50 +08:00
Mahdi a5f9b8f92e Added fix for MixUpBatch and CutMixBatch and for RandomAffine
updated c color op descriptions
2020-08-21 17:40:08 -04:00
mindspore-ci-bot 9d7250c483 !4776 Introduce 2 extra ctrl flags to DataBuffer in dataset, address remaining cmts to PR4632
Merge pull request !4776 from ZiruiWu/map_callback_follow_up
2020-08-22 03:31:31 +08:00
Zirui Wu 74c1e6da60 introducing pause and quit flags to DataBuffer
fix review cmts

fix ci

fix Ci

fixci

address ci

ci

- add timeout
- add more test cases

fix CI

address review cmts
2020-08-21 14:33:26 -04:00
mindspore-ci-bot b2a5440548 !4883 Fix compilation warnings
Merge pull request !4883 from zhoufeng/xiu-ba-ge
2020-08-21 12:33:20 +08:00
zhoufeng 22ec976ef7 fix compilation warnings
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-08-21 09:44:04 +08:00
mindspore-ci-bot 9a3baf4f6c !4314 [MD] remove defaults used in binding code, and remove bilinear resize op
Merge pull request !4314 from nhussain/clean_up_bindings
2020-08-20 22:16:56 +08:00
mindspore-ci-bot bfa3cd900e !4843 remove ccsrc/common.h by explicit dependent header file
Merge pull request !4843 from xychow/remove-ccsrc-common-h
2020-08-20 22:02:13 +08:00
zhousiyi d0e58dd765 remove ccsrc/common.h
replace frontend/operator/ops.h in backend with base/core_ops.h as
backend should not use any frontend-only primitive
2020-08-20 09:34:07 +00:00
xiefangqi 7dc7990d75 add NumWorkers validate 2020-08-20 17:25:30 +08:00
mindspore-ci-bot 7b0a4e5929 !4796 Fix randomdata ut core dump problem
Merge pull request !4796 from xiefangqi/fix_randomdata_ut
2020-08-20 14:11:55 +08:00
xiefangqi c0d18d2517 fix randomdata ut problem 2020-08-20 11:13:02 +08:00
Jesse Lee 8a08d0c37b Phase 2 of CacheOp 2020-08-19 13:45:55 -04:00
Zirui Wu 78c1aa1d96 Implemented Callback for Dataset
implment pause in MapOp, added more to callback

add ds_callback

- Initial drop of Python DSCallback

- Pybind DSCallback

- Pybind DSCallback

added callback to mapOp

- de_pipeline DSCallback

- de_pipeline DSCallback

add test case, segfault for now

fix seg fault

- de_pipeline DSCallback

remove 1 line

update callback test case, now works

use builder class for mapOp callback

- de_pipeline DSCallback

- de_pipeline DSCallback

- de_pipeline DSCallback

better test case

minor fix

add comments and minor clean ups

get rid of nullptr in MapOp, use other flag instead

fix a bug ParseMapOp only takes 1 callback

- Added WaitedDSCalabck

refactor callback param

fix text case incorrect number

- added testing

fix cpp test case

- added testing

- revert back lenet changes

- cleanup test_callbacks.py

- cleanup test_callbacks.py

fix CI stage I

fix CI stage II

fix CI and update epoch counter

- add validation
- add more testing  test_callbacks.py

use random data op to do tests

adjust when to call EpochBegin/End

- add repeat with callback

- addressing reviewers' comments

- docstring and CI fixes

- docstring and CI fixes

- docstring and CI fixes

- rebase with upstream/master

fix cpp test case

fix review comments

addr review cmts, add test case
2020-08-19 12:31:04 -04:00
YangLuo b50ae27c72 !4597 [Dataset] C++ API Support for build_vocab 2020-08-19 22:47:47 +08:00
nhussain eb9a611041 remove old defaults 2020-08-19 10:11:19 -04:00
mindspore-ci-bot d541e261a0 !4696 C++ API Support for CSV Dataset
Merge pull request !4696 from jiangzhiwen/jzw/c_api_csv
2020-08-19 17:08:33 +08:00
mindspore-ci-bot 2db4bda219 !4622 serving restful init
Merge pull request !4622 from hexia/http_init
2020-08-19 16:15:58 +08:00
mindspore-ci-bot 256dccc651 !4498 Gnn data processing supports distributed scenarios
Merge pull request !4498 from heleiwang/gnn_distributed
2020-08-19 16:09:05 +08:00
jiangzhiwen 39df25f803 c++ api for CSV Dataset 2020-08-19 16:02:11 +08:00
heleiwang 8ee4d8e92d Gnn data processing supports distributed scenarios 2020-08-19 11:09:01 +08:00
mindspore-ci-bot 6868b9b6fa !4422 Minddata new dataset api ==> RandomData
Merge pull request !4422 from xiefangqi/xfq_c++api_randomdata
2020-08-19 10:48:53 +08:00
xiefangqi b91e56375e add randomdataset and schema 2020-08-19 09:13:43 +08:00
nhussain 738ae2c78d change api to use std_vector
try to display image
2020-08-18 15:41:21 -04:00
islam_amin 979111d0ef Added Posterize Op 2020-08-18 13:55:47 -04:00
hexia 75d116b5db serving http init 2020-08-18 20:15:47 +08:00
mindspore-ci-bot 4f46c4277a !4614 Code compile performance optimization
Merge pull request !4614 from zhoufeng/code-compile-performance
2020-08-18 16:45:15 +08:00
mindspore-ci-bot cc077de9e3 !4425 fix if statement with no return coredump
Merge pull request !4425 from fary86/fix_if_no_return_coredump
2020-08-18 16:09:31 +08:00
zhoufeng 663278112f optimize code compile performance
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-08-18 14:33:24 +08:00
mindspore-ci-bot 26de4540ef !4639 rename param_value to param_info
Merge pull request !4639 from vlne-v1/rename_param_value
2020-08-18 14:16:23 +08:00
fary86 38083e055a Fix coredump missing return statement after while loop 2020-08-18 14:16:21 +08:00
Wei Luning c1c30a44f1 rename param_value -> param_info 2020-08-18 11:16:28 +08:00
jiangzhiwen a88273bd3f c++ api support for Manifest Dataset 2020-08-18 09:48:55 +08:00
mindspore-ci-bot 113ff6ca24 !4367 CutMixBatch Augmentation Op
Merge pull request !4367 from MahdiRahmaniHanzaki/cutmix
2020-08-18 03:27:33 +08:00
mahdi 3ecc53fb4e Added CutMix 2020-08-17 13:01:47 -04:00
mindspore-ci-bot dda4c26e7c !4423 add reshape type of tensor
Merge pull request !4423 from lianliguang/add-reshape-type-to-tensor
2020-08-17 11:34:11 +08:00
WilliamLian 6760d9976d add reshape type to tensor 2020-08-17 09:34:38 +08:00
mindspore-ci-bot f875bf21bc !2948 fix control flow
Merge pull request !2948 from amongo/FixControlFlow
2020-08-17 08:56:22 +08:00
huangdongrun 2a6d346d2f support if by if grad parameter
add join for ref

adjust env eliminate to eliminate all env ops

add partial app cache

resolve while endless

fix env eliminate

support for "for while" cases

fix join shape error
2020-08-15 09:47:10 +08:00
Cathy Wong 9c8af0d1cf C++ API: Provide validate param functions 2020-08-14 17:54:38 -04:00
mindspore-ci-bot 7d70fb4dc4 !3769 add RandomColorOp
Merge pull request !3769 from Alexey_Shevlyakov/ColorOp
2020-08-14 23:43:08 +08:00
mindspore-ci-bot ebdea65f70 !4402 [Dataset] Fix c_api bugs
Merge pull request !4402 from luoyang/c-api-pyfunc
2020-08-14 17:40:32 +08:00
mindspore-ci-bot e9629f95e1 !4418 Add UniqueWithPad cpu kernel
Merge pull request !4418 from huanghui/unique-with-pad-cpu-kernel
2020-08-14 09:46:22 +08:00
Alexey Shevlyakov 8526d5414d RandomColor 2020-08-13 16:10:21 -04:00
mindspore-ci-bot 2953720169 !4322 Adding RGBA color conversion
Merge pull request !4322 from EricZ/bitmap_support
2020-08-14 01:58:33 +08:00
mindspore-ci-bot 7a10e07fc6 !4383 Add distributedSampler to concatOP
Merge pull request !4383 from genglishuai/genglshOK
2020-08-14 00:06:06 +08:00