timekiller
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
42,204 Commits 19 Branches 0 Tags 564 MiB
Commit Graph

27 Commits

Author SHA1 Message Date
liu-yongqi-63 457ee03082 Vocab C++ Interface Alignment and SentencePieceVocab C++ interface alignment and Python interface refactoring 2022-03-10 11:33:00 +08:00
tophand 797cd52c96 [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 2022-01-28 02:26:20 -08:00
zhangyuqwer e934381b08 [feat][assistant][I3ZSQV] add new data operator CharNGram 2021-12-04 22:09:34 +08:00
zy 8f9a09f79c [fix] [assistant] [I3ZSQR] add new data operator GloVe 2021-12-01 17:12:20 +08:00
zy 5ca98a494a [fix] [assistant] [I3ZSQS] add new data operator FastText 2021-12-01 11:12:08 +08:00
zhangyuqwer 6a6e73ef82 [fix] [assistant] [I3ZSQM] add new data operator Vectors 2021-11-30 14:39:42 +08:00
YangLuo bf48400c10 unify include file of minddata 2021-04-22 10:59:36 +08:00
Cathy Wong 636809c255 dataset: C++ API ToNumber & Lookup: change data_type parm from string to DataType 2021-04-19 12:35:37 -04:00
Cathy Wong a2ee0cb3e5 dataset UT: Update UTs to check for return Status 2021-04-14 17:52:31 -04:00
mindspore-ci-bot 0cfd8b7c9e !13364 Add WordpieceTokenizer and AddDict to Jieba 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-04-06 05:04:13 +08:00
alex-yuyue 2d6d608fae Add WordpieceTokenizer to API and IR. Add AddDict to C++ API 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-18 16:42:31 -04:00
Cathy Wong 1775c4e83d dataset: Reinstate INFO logging and data verification - part 3 
Updates: GetItemAt, CreateFromVector, CreateScalar.
Add TEST_MS_LOG_MSTENSOR.
Wrap ASSERT_OK around Tensor function calls.
 2021-03-17 13:48:04 -04:00
Zirui Wu 8022001450 Remove datasets.h dependency on tensor.h and other related dependencies 
Replace std::function<TensorRow(TensorRow)> with std::function<MSTensorVec(MSTensor)> by using std::bind

Replace pad_info with MSTensor

Switch TensorShape/dataset::DataType to vector<int64> /mindspore::DataType

Change TypeId to mindspore::DataType in SchemaObj

address review cmts, add CreateFromMSTensor() in tensor.h/.cc
 2021-03-05 16:26:58 -05:00
TinaMengtingZhang c5aa3eeef2 Add API class for data transform ops except Compose, RandomApply, RandomChoice 
Add API class for text transforms ops
 2021-02-18 15:58:44 -05:00
Eric 9953757ff4 Fixed tenor compile error for MD 2021-02-09 14:56:31 +08:00
Cathy Wong 3e6e35d749 dataset: delete minddata/dataset/include/status.h 2021-02-04 10:19:08 -05:00
lixian 7d2fd6e76c cxx api refactor: tensor/status/model 2021-02-02 09:15:56 +08:00
alex-yuyue a8675f8227 TextOp decoupling 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-01-20 21:52:45 -05:00
mohammad f48ab2b5c9 make MD public headers standalone 2021-01-11 06:35:53 -05:00
YangLuo da6de519ae Add ut for dataset to_number 2020-12-21 20:09:03 +08:00
Xiao Tianci 0659473535 add TruncateSequencePair, ToNumber C++ API and enable three test cases 2020-12-08 11:12:16 +08:00
mindspore-ci-bot 6b5626634c !9530 add BasicTokenizer and BertTokenizer C++ API 
From: @tiancixiao
Reviewed-by: @mikef,@nsyca
Signed-off-by: @nsyca
 2020-12-08 04:09:40 +08:00
shenwei41 206ae1a7c3 Add Three text C++ API 2020-12-05 10:32:15 +08:00
Xiao Tianci 290cc72887 add BasicTokenizer and BertTokenizer C++ API 2020-12-04 21:51:49 +08:00
shenwei41 3bea84d0f7 add four new text API 2020-12-01 10:21:47 +08:00
mohammad 9cfeb08c4d port name() to TensorOperation and its derived classes 2020-11-27 19:00:56 -05:00
shenwei41 4e56618d18 Add text C++ API 2020-11-27 11:41:35 +08:00