Commit Graph

174 Commits

Author SHA1 Message Date
mindspore-ci-bot dceb3c2be4 !2026 Change C++ behavior in RandomCropAndResize
Merge pull request !2026 from MahdiRahmaniHanzaki/I1K3FL-random-resized-crop
2020-06-16 05:27:36 +08:00
Mahdi add29f95f5 resolved issues in the comments 2020-06-15 16:45:18 -04:00
mindspore-ci-bot 2cab44a637 !2028 FillOp for BERT
Merge pull request !2028 from nhussain/fillOp_learning
2020-06-16 02:45:45 +08:00
nhussain dd9bf09f0a added FillOp for #119 - special Ops 2020-06-15 11:30:43 -04:00
Peilin Wang 848e07d022 initial commit, start of BucketBatchByLengthOp
c implementation done, just need to call batch/pad

added python api and validator

added pybind/de_pipeline stuff, fixed some compile errors, figure out how null py::function works

added tiny bit of doc

integrated with static batch methods

fixed some bugs

some more bug fixes and cleanup

ci fix

fix ci

ci fix

fix ci

added test_cases and debugged

addressed code review comments

addressed code review comments

ci fix

ci fix

addressed code review comments

addressed code review comments
2020-06-15 10:20:01 -04:00
mindspore-ci-bot a8cf83acf2 !1932 Add CLUE dataset
Merge pull request !1932 from jiangzhiwen/dataset/clue
2020-06-15 11:48:35 +08:00
jiangzhiwen e0e167a000 add CLUE dataset 2020-06-15 10:40:46 +08:00
hesham a22763b612 Bug in CIFAR after removing GetMutabble buffer
Fixes # I1KIPC
2020-06-14 13:27:17 -04:00
jonyguo bb2082daec fix: MindDataset padded log error 2020-06-13 19:46:24 +08:00
mindspore-ci-bot 3085e51e45 !1922 check parameter num_samples of sampler
Merge pull request !1922 from yanghaitao/yht_check_num_samples_sampler
2020-06-12 10:22:14 +08:00
mindspore-ci-bot 1b5d9a1194 !1981 Implemented from_dataset, a way to build a vocab from existing dataset
Merge pull request !1981 from ZiruiWu/from_dataset_dev
2020-06-12 08:08:19 +08:00
Zirui Wu 880ce5ea26 implemented from_dataset
fix complie error

more tests

address CI complains

fix ci

adress review comments

address review cmts
2020-06-11 16:20:19 -04:00
Alexey Shevlyakov 4e333a2f22 MindData profiler infrastructure 2020-06-11 14:52:36 -04:00
mindspore-ci-bot 0a95223f25 !1910 RandomColorAdjust error for grayscale images
Merge pull request !1910 from MahdiRahmaniHanzaki/I1J9SQ-random-color-adjust-bug
2020-06-11 23:15:19 +08:00
yanghaitao 8f16cff12e add para check for sampler 2020-06-11 22:26:06 +08:00
jonyguo 1de7271afc add floatxx test case 2020-06-11 19:26:54 +08:00
ms_yan bc22c172b8 add TensorDataset and its ut 2020-06-11 15:24:12 +08:00
mindspore-ci-bot 251a6667a5 !1989 fix MindDataset distribute shuffle error
Merge pull request !1989 from guozhijian/fix_distribute_shuffle
2020-06-11 14:51:17 +08:00
mindspore-ci-bot b3da41bd7a !1913 CocoDataset implementation
Merge pull request !1913 from xiefangqi/xfq_support_coco
2020-06-11 14:37:44 +08:00
xiefangqi 097cbcc0cb CocoDataset implementation 2020-06-11 11:52:17 +08:00
jonyguo 96859f44b6 fix: MindDataset distribute shuffle bug 2020-06-11 11:06:34 +08:00
Mahdi ba2e2dced4 fixed some warnings and typos 2020-06-10 17:18:56 -04:00
Tinazhang 2812cd57db fix pylint error 2020-06-10 15:55:11 -04:00
mindspore-ci-bot fe0c3e9a8f !1908 Add UT to six data vision transforms ops
Merge pull request !1908 from Tinazhang/ut-normalize
2020-06-11 00:43:57 +08:00
heleiwang 3ece8dd090 1. support get_all_edges, get_nodes_from_edge, get_sampled_neighbors, get_neg_sampled_neighbors and graph_info API
2. mod cora and citeseer conversion script
2020-06-10 15:52:26 +08:00
mindspore-ci-bot 5eb95599f6 !1874 Updates to string tensor
Merge pull request !1874 from h.farahat/string_tensor_updates
2020-06-10 00:59:17 +08:00
Tinazhang c3de5c7a54 adding new TCs to six ops and skip md5 case in RandomPerspective 2020-06-09 10:51:45 -04:00
hesham f837ddc956 - Bug when empty strings sent to Python
- Support accepting Numpy of str as input
- Support batching strings
- Core logic of batch&pad is static
- Make Pad a utility function
2020-06-09 10:34:38 -04:00
liyong feff8899ac support padding samples 2020-06-05 09:14:42 +08:00
mindspore-ci-bot 5c21616293 !1807 Implemented Ngram TensorOp for dataset
Merge pull request !1807 from ZiruiWu/ngram_dev
2020-06-05 01:04:27 +08:00
Zirui Wu dbf9936ec4 Implemented n-gram for dataset TensorOp 2020-06-04 09:57:44 -04:00
Jamie Nisbet 51bc0c0460 consistent design for num_samples
updates

more update

more work

more fixin

post rebase updates

clang formatting

code review recovery

ci fixes

updates

update

update

update
2020-06-03 15:01:18 -04:00
Tinazhang 33fe66fd1e fix error handling for bigger crop size case 2020-06-02 11:06:41 -04:00
Yang 5522edc4b5 !1713 [Dataset] Clean pylint. 2020-06-02 09:02:19 +08:00
liuwenhao4 7b911886ec Fixing some tiny faults about Pylint in my code(ops) 2020-05-30 16:55:37 +08:00
mindspore-ci-bot 7878743400 !1638 fixed rounding edge case in split
Merge pull request !1638 from Peilin/splitOp-after-testing
2020-05-30 05:31:33 +08:00
mindspore-ci-bot 0e3dd8149a !1703 Adding sync_wait input check
Merge pull request !1703 from EricZ/master
2020-05-30 05:29:38 +08:00
mindspore-ci-bot c366b3fb18 !1704 fix num rows bug in sampler
Merge pull request !1704 from Peilin/fix-random-sampler-bug
2020-05-30 05:27:20 +08:00
Eric 8e1b2107ce Added example for sync_wait bad batch_num
Added broken test case for cv

Fixed error handling in sync_wait
2020-05-29 16:30:12 -04:00
mindspore-ci-bot 976b4793d1 !1544 Python Unit Test for TenCrop
Merge pull request !1544 from MahdiRahmaniHanzaki/I1IHNG-ten-crop
2020-05-30 03:02:29 +08:00
Peilin Wang 3ef3d1433d fixed num_samples bug for RandomSampler 2020-05-29 13:09:26 -04:00
Tinazhang f4a61fdbb2 Adding new UT cases and missing TCs in PY Ops. 2020-05-29 12:12:17 -04:00
Peilin Wang 5469be2a97 fixed bug for split, RandomSampler and some other cleanup
add another test case

typo

merge conflict

another PR changed testing behavior, updated test cases in this commit

added input check for use_sampler

addressed code review comments

fixed pylint, not related to my changes

fixed edge case of rounding in getting split sizes

fix pylint
2020-05-29 11:27:32 -04:00
Mahdi dfc097019b Added TenCrop test
Added unit tests for both testing the functinality of the TenCrop and
its error messages. Due to the similarity of this method to FiveCrop the
test cases are similar to FiveCrop test cases.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>

added error_msg function call in the main method

refactored the test and added visual representation of the results

Separated the two error cases into two different functions and used the
visualize function in util.py to plot the result of TenCrop.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>

Added new test cases

Added new test cases including test case for checking the error message
when the size variable is not a positive integer, test case for
rectangle crop, test case for vertical flip setting, and testing for
similarity of the result of TenCrop for the same input data in different runs.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>

changed visualize in test_five_crop

Changed the visualize function in test_five_crop to use the already
existing function in util.py
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>

made generate_golden variable global
2020-05-28 17:52:56 -04:00
mindspore-ci-bot 21da86b393 !1591 add get_dataset_size for CelebADataset
Merge pull request !1591 from yanghaitao/yht_celeba_get_dataset_size
2020-05-28 22:21:43 +08:00
yanghaitao cc6c7a3f60 add get_dataset_size to celebadataset 2020-05-28 17:00:14 +08:00
mindspore-ci-bot 0f4dc13e8e !1478 [Dataset] Fix CI warnings, clean pylint.
Merge pull request !1478 from luoyang/pylint
2020-05-28 15:54:52 +08:00
Yang 9b2a778d94 !1478 [Dataset] clean pylint. 2020-05-28 14:06:36 +08:00
xiefangqi 5e4728c50f add voc support split 2020-05-28 11:18:32 +08:00
Peilin Wang d4c93575e7 fixed bug for split, RandomSampler and some other cleanup
add another test case

typo

merge conflict

another PR changed testing behavior, updated test cases in this commit

added input check for use_sampler

addressed code review comments

fixed pylint, not related to my changes
2020-05-26 16:29:52 -04:00