Commit Graph

226 Commits

Author SHA1 Message Date
yanghaitao f3423208c4 set monitor sampling interval through json file 2020-06-28 15:30:37 +08:00
mindspore-ci-bot beb436f457 !2398 Bug in Slice when multiple rows are used
Merge pull request !2398 from h.farahat/slice_bug
2020-06-22 09:23:44 +08:00
mindspore-ci-bot a9c309da4c !2404 [Dataset]rename input variable name to children and output variable to parent
Merge pull request !2404 from xulei/input_output
2020-06-22 09:22:33 +08:00
mindspore-ci-bot 1ea38eb60c !2375 Add Python Tokenizer
Merge pull request !2375 from h.farahat/python_tokenizer
2020-06-22 09:20:33 +08:00
liyong 2412ee09ce fix distributedSampler reshuffle and fix random_device failed 2020-06-21 18:23:47 +08:00
mindspore-ci-bot 78a8bc302d !2380 Fix CocoDataset issue
Merge pull request !2380 from xiefangqi/xfq_fix_coco_issue_01
2020-06-20 18:41:07 +08:00
xulei2020 3b71bd0d5d rename input to children, output to parent 2020-06-20 16:40:10 +08:00
hesham 68030e6a4b Slice Bug 2020-06-20 00:47:13 -04:00
qianlong 980ddd32a2 change output of WordpieceTokenizer and BertTokenizer to 1-D string tensors 2020-06-20 12:08:43 +08:00
xiefangqi 5703a10b8b fix CocoDataset issue 2020-06-20 09:51:10 +08:00
mindspore-ci-bot bbf69912be !2363 fix TextFildDataset and CLUEDataset does not support to_device
Merge pull request !2363 from yanghaitao/yht_fix_textfiledataset
2020-06-20 09:47:01 +08:00
hesham e981c67acd Python Tokenizer
!38 Synchronize with latest Ascend software suite 17 Jun 2020
Merge pull request !38 from yanghaoran/master
2020-06-19 19:57:11 -04:00
hesham 674415f7be Cleanup work for Concate, Mask, Slice, PadEnd and TruncatePair 2020-06-19 19:11:50 -04:00
mindspore-ci-bot bc4b1c2460 !2373 Comments + improved casting/parameters + UT tests for 3 AugOps (Augmentation Ops RandomCropWithBBox, Random Vertical FlipWithWithBBox and Random CropAndResizeWithWithBBox)
Merge pull request !2373 from danishnxt/AugOpsD1
2020-06-20 05:19:03 +08:00
mindspore-ci-bot 7bdcc319b2 !2325 Addressing comments from PR2314
Merge pull request !2325 from imaaamin/object_ops_pr2
2020-06-20 04:50:56 +08:00
Danish Farid 6442a85b75 comments added, ptrs to consts, UT tests for BBOps
updated comments for 3 BB AugOps

addressing comments, void -> status, macros added, fixed test cmt

changed types -> value, fixed ut test funcs, grammar

bug type fix

fixed condition in tests for viz
2020-06-19 16:09:16 -04:00
islam_amin edc42c5b85 addressed comments from reviews 2020-06-19 14:53:46 -04:00
nhussain d89101b95f add missing test 2020-06-19 11:44:41 -04:00
yanghaitao 9f336bb735 fix TextFileDataset and CLUEDataset failed with to_device 2020-06-19 17:28:05 +08:00
peilinwang 1e36b0649f remove graphengine changes
remove graphengine changes

concat op

Truncate Pair

concat_op

remove graph engine changes

ToNumberOp implementation almost done

ToNumberOp complete

ci fix

ci fix

ci fix

ci fix

ci fix

ci fix

ci fix

ci fix

ci fix

ci fix

merge conflicts
2020-06-18 20:07:35 -04:00
hesham 11826fb256 DuplicateOp 2020-06-18 19:47:27 -04:00
mindspore-ci-bot 90bb9320aa !2314 Implemented RandomHorizontalFlipWithBBox and BoundingBoxAugment C++ Ops
Merge pull request !2314 from imaaamin/object_ops_pr
2020-06-19 07:08:42 +08:00
mindspore-ci-bot a83baafbf6 !2253 Add ConcatOp to Dataset
Merge pull request !2253 from nhussain/concat_op
2020-06-19 06:32:15 +08:00
mindspore-ci-bot 3784220056 !2317 rework on dataset.text.vocab to support any user special_tokens
Merge pull request !2317 from ZiruiWu/vocab_rework
2020-06-19 05:25:31 +08:00
nhussain 5515016dba remove graphengine changes
remove graphengine changes

concat op

Truncate Pair

concat_op

remove graph engine changes
2020-06-18 17:17:50 -04:00
mindspore-ci-bot ffc8a3c362 !2274 add PadEndOp
Merge pull request !2274 from xunxue/padend
2020-06-19 05:05:14 +08:00
mindspore-ci-bot e4322baece !2316 [Dataset] Move python tree transformation into cpp pass
Merge pull request !2316 from JunhanHu/alter_node_cpp
2020-06-19 05:02:03 +08:00
islam_amin 0af156d169 added two ops for object detection, random horizontal flip with bbox and bouding box augment. along with their unit tests 2020-06-18 16:55:31 -04:00
xunxue 87aa9c8f7a PadEndOp 2020-06-18 16:07:34 -04:00
Junhan Hu 57f3732ac3 Move python tree transformation into cpp pass. 2020-06-18 15:52:38 -04:00
Zirui Wu b6e9504b31 phase I of Vocab rework
phase II vocab rework

added more test cases

fix api doc string

address review cmts and fix CI

address ci complains

fix review cmts

ci
2020-06-18 15:36:28 -04:00
Zirui Wu 8f2674850b address API doc style and content
addr review cmts

revert

ci

private class

ci

fix ci

address ci

address review cmts

fix style err
2020-06-18 12:23:42 -04:00
mindspore-ci-bot 5850b99161 !2211 Add TruncatePair Op to dataset
Merge pull request !2211 from h.farahat/pair_truncate
2020-06-18 20:58:40 +08:00
mindspore-ci-bot d0c5071cab !1993 [Dataset] Fix codedex.
Merge pull request !1993 from luoyang/pylint
2020-06-18 18:59:31 +08:00
ms_yan 8d1dae46ac Throw error when load config failed 2020-06-18 13:47:32 +08:00
hesham b9495a9ccc Truncate Pair 2020-06-17 23:11:27 -04:00
mindspore-ci-bot ea829e89c3 !2198 Add MaskOp to dataset
Merge pull request !2198 from h.farahat/mask_op
2020-06-18 02:50:16 +08:00
mindspore-ci-bot 808380a9f9 !2142 Cleanup dataset UT: refactor visualize()
Merge pull request !2142 from Tinazhang/reorg-visualize
2020-06-18 01:30:36 +08:00
ms_yan fdafc690f8 repair problem in NumpySlicesDataset 2020-06-17 21:49:02 +08:00
Tinazhang 169eaf73c7 reorganize util function visualize() 2020-06-17 09:30:05 -04:00
Yang dee8471d23 !1993 [Dataset] Fix codedex. 2020-06-17 16:25:38 +08:00
qianlong 4f16f036be Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp
add CaseFold, NormalizeUTF8

add RegexReplace

add RegexTokenizer

add BasicTokenizer

add WordpieceTokenizer

add BertTokenizer
2020-06-17 15:47:04 +08:00
hesham f2462bb00d Mask Op 2020-06-16 23:51:46 -04:00
mindspore-ci-bot ca94916ef9 !2184 CocoDataset add category_id check
Merge pull request !2184 from xiefangqi/xfq_fix_coco_issue
2020-06-17 10:06:41 +08:00
mindspore-ci-bot 625f2421b5 !1977 random walk in gnn node2vec
Merge pull request !1977 from JonathanY/randomwalk
2020-06-17 03:22:21 +08:00
mindspore-ci-bot 3adbde56b2 !2162 SliceOp
Merge pull request !2162 from h.farahat/slice_op
2020-06-17 01:22:47 +08:00
mindspore-ci-bot 898b2fdecd !2187 Fix comment display issues in BuildVocabDataset
Merge pull request !2187 from ZiruiWu/vocab_rework
2020-06-17 00:51:37 +08:00
Zirui Wu 2794883644 fix selected minor issues
fix review comments
2020-06-16 11:07:49 -04:00
xiefangqi aab190fcdb CocoDataset add category_id validation 2020-06-16 21:37:22 +08:00
liyong d4f8f57c7e add split in minddataset 2020-06-16 19:50:41 +08:00