Col_lin
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,898 Commits 18 Branches 22 Tags 3.6 GiB
Commit Graph

138 Commits

Author SHA1 Message Date
YangLuo 0a4da64e13 Fix minddata issues 2021-05-10 11:12:00 +08:00
mindspore-ci-bot 34f4260cd1 !15979 Officially support caching over MindRecord dataset 
From: @lixiachen
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @robingrosman
 2021-05-07 03:39:12 +08:00
ms_yan ae436cb454 descrease num_worker when mechine has less worker then default 2021-05-06 09:12:58 +08:00
Lixia Chen 5967cb2cd7 Unblock mindrecord under cache 2021-05-05 13:28:13 -04:00
mohammad 9ce28cd34c split sampler vision IR files 2021-04-29 16:30:18 -04:00
mindspore-ci-bot 1e2e554db3 !15380 Unify minddata include files(except for datasets.h) in scenario of cloud and lite 
From: @luoyang42
Reviewed-by: @tiancixiao
Signed-off-by:
 2021-04-28 09:57:46 +08:00
YangLuo bf48400c10 unify include file of minddata 2021-04-22 10:59:36 +08:00
Xiao Tianci 5a9622387c add folders and CMakeLists for audio operators 2021-04-19 20:30:00 +08:00
mindspore-ci-bot f9c5060cf2 !14930 Handle ignored returned Status 
From: @hfarahat
Reviewed-by: @robingrosman,@pandoublefeng
Signed-off-by: @robingrosman
 2021-04-17 03:52:41 +08:00
hesham 98f0f36a20 Removed unused include 2021-04-16 11:40:56 -04:00
hesham 6e6eaebb77 Fix unused return value in main code 2021-04-16 11:33:35 -04:00
Zirui Wu 8ebc512d25 plugin infrastruture for mindddata 
add single element string tensor support for plugin tensor
 2021-04-15 21:11:16 -04:00
mindspore-ci-bot 727203a498 !14911 dataset: Fill op: C++ API support, UTs and Pybind decoupling 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-04-14 05:32:11 +08:00
Cathy Wong 7e6a03487e dataset: Fill op: C++ API support, UTs and Pybind decoupling 2021-04-13 10:02:39 -04:00
mindspore-ci-bot e07a69d8f8 !14874 GNN dataset new feature 
From: @lizhenglong1992
Reviewed-by: 
Signed-off-by:
 2021-04-13 10:44:27 +08:00
ms_yan ecb8d6bc08 add pil version cubic resize 2021-04-12 21:56:14 +08:00
Zhenglong Li f3895f2b09 Add get edges from nodes function for gnn network 2021-04-10 18:31:13 +08:00
alex-yuyue 83aaed09bb TensorOp decoupling stage 3 (transform ops) 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-04-07 18:46:23 -04:00
Cathy Wong 78392e0d66 dataset: Pybind change for data transforms 2021-04-06 10:10:14 -04:00
mindspore-ci-bot 0cfd8b7c9e !13364 Add WordpieceTokenizer and AddDict to Jieba 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-04-06 05:04:13 +08:00
hesham 0d86cb1423 Remove rows_per_buffer 2021-04-04 00:22:16 -04:00
ms_yan b6ef88c168 add stop_dataset_profiler option 2021-03-27 20:36:40 +08:00
alex-yuyue 2d6d608fae Add WordpieceTokenizer to API and IR. Add AddDict to C++ API 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-18 16:42:31 -04:00
Eric 4be0ffb9ba removing type_id.h from header files 2021-03-16 16:51:10 -04:00
ms_yan 92e86804e1 init add acltdt handle create and destory 
add hostpush part modify

optimize previous code

provide aclhandle access method

modify CMakeList format

add device_id parameter into TransferNode

update acltdt api
 2021-03-15 12:45:09 +08:00
Eric 82f7c6f2c7 Moving data_helper 
exposing json

WIP fix for removing templates

Removing templates part 1

replacing core/constants.h

removed template part 2

Changed fake data to filler data

Fixing CI

Fix CI part 2

CI fix part 2

Fix CI 3

Fixed lizard for data_util

Fixed clang

Fixing CI part 4
 2021-03-12 15:30:34 -05:00
alex-yuyue 2457f52596 Fix CI warning and test team issues 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-11 13:34:51 -05:00
Eric ed373f61ac Adding inheritance to user defined sampler 
what if we just do nothing

trying wrapper approach

Fix yolo
 2021-03-09 18:14:49 -05:00
Zirui Wu 8022001450 Remove datasets.h dependency on tensor.h and other related dependencies 
Replace std::function<TensorRow(TensorRow)> with std::function<MSTensorVec(MSTensor)> by using std::bind

Replace pad_info with MSTensor

Switch TensorShape/dataset::DataType to vector<int64> /mindspore::DataType

Change TypeId to mindspore::DataType in SchemaObj

address review cmts, add CreateFromMSTensor() in tensor.h/.cc
 2021-03-05 16:26:58 -05:00
Eric d328d6eb54 Adding affine 
Need to move Affine to image_utilC

Refactored affine

Added affine API

Added API class impl

Removed default from vision ir

Compiling

removed extra file

Spell check
 2021-02-25 16:55:02 -05:00
mindspore-ci-bot ced5575387 !12353 Move sampler IR code to engine/ir 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-02-13 06:44:47 +08:00
mohammad 74bc54b694 push down sampler IR files 2021-02-11 17:26:13 -05:00
mindspore-ci-bot ed7fef5d5e !12349 [MD] Push down IR files for transforms and text 
From: @tina_mengting_zhang
Reviewed-by: 
Signed-off-by:
 2021-02-12 00:01:13 +08:00
TinaMengtingZhang 462422acc7 Move transforms and text ir down 2021-02-10 15:04:07 -05:00
mindspore-ci-bot 806b639f02 !9720 restructure python bindings 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-02-11 02:21:32 +08:00
mohammad 52334c8e4d restructure python bindings 2021-02-09 19:38:01 -05:00
Cathy Wong fc8b9fc85e dataset: Push down IR vision files 2021-02-09 16:27:08 -05:00
hesham 37add20783 Fix caching problem after cleanup 2021-02-08 20:39:23 -05:00
mindspore-ci-bot e6dcb0c31d !12239 [MD] fix bugs of execute & update doc description in minddata 
From: @luoyang42
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
 2021-02-08 10:15:08 +08:00
luoyang cd86788cf3 fix bugs of execute & update doc description in minddata 2021-02-07 22:52:52 +08:00
gongxiaoqing 7f538b51e7 回退 'Pull Request !11074 : replace tdt with acltdt interface' 2021-02-06 14:42:14 +08:00
mindspore-ci-bot c2582dcab9 !11074 replace tdt with acltdt interface 
From: @ms_yan
Reviewed-by: @lilongfei15
Signed-off-by:
 2021-02-06 13:22:39 +08:00
hesham 4bbc3445f1 datasets.py cleanup 2021-02-05 09:50:27 -05:00
ms_yan 293f81128d init add acltdt handle create and destory 
add hostpush part modify

optimize previous code

provide aclhandle access method

modify CMakeList format

add device_id parameter into TransferNode
 2021-02-03 11:23:27 +08:00
luoyang fa4f67cbcd LookupOp: default value of unknown_token support NoneType 2021-02-02 16:17:02 +08:00
lixian 7d2fd6e76c cxx api refactor: tensor/status/model 2021-02-02 09:15:56 +08:00
mindspore-ci-bot 320ea51308 !11853 Add call for decoupled image and text ops 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-02-01 19:47:00 +08:00
yanghaitao1 83cb87bcd1 cpu profiling 
add operator profiling part

add op_id param when create task

repair bugs in op cpu sampling

fix wrong order of write and parse cpu_info

add process cpu profiling part

add multiprocessing part(part 1)

adapter cpu_sampling c layer

fix little error in sampling

descrease cirle complex in voc

descrease cirle complex in device_que
 2021-02-01 13:18:54 +08:00
alex-yuyue 6fd58dc580 Add call for decoupled image and text ops 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-01-29 21:36:31 -05:00
Mahdi 8f34faeb7a Changed bindings to SamplerObj 2021-01-29 14:20:05 -05:00