yankf21
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,217 Commits 18 Branches 22 Tags 3.6 GiB
Commit Graph

249 Commits

Author SHA1 Message Date
hesham a9e9266149 Deepcopy problem when pyfunc cannot be pickled 2020-04-22 11:55:27 -04:00
liuxiao b6e77e5178 Add ReluV2/ReluGradV2/ConfusionMulGrad for VM 2020-04-22 21:09:37 +08:00
mindspore-ci-bot 1a0d2666b7 !559 Add Fill/Erf op for VM. 
Merge pull request !559 from liuxiao/fill-op
 2020-04-22 19:22:04 +08:00
mindspore-ci-bot 818acd46d4 !569 fix bug in checkpoint when save scaler 
Merge pull request !569 from wangnan39/master
 2020-04-22 18:55:02 +08:00
lichenever c78630d737 support multiple subgraphs 2020-04-22 17:40:56 +08:00
wangnan39@huawei.com f38d18c665 fix bug in checkpoint when save scaler 2020-04-22 16:04:55 +08:00
mindspore-ci-bot 24a9f4974c !536 support ellipsis and bool for tensor slice 
Merge pull request !536 from zhangbuxue/support_elipis_for_tensor_slice
 2020-04-22 15:49:33 +08:00
liuxiao 5b6c8fade8 Add Erf\FillD operator for VM 2020-04-22 15:32:42 +08:00
buxue 437bb8c27c support ellipsis and bool for tensor slice 2020-04-22 15:00:04 +08:00
mindspore-ci-bot 67057d1309 !541 add average pooling 1D 
Merge pull request !541 from JichenZhao/avgpooling
 2020-04-22 14:13:04 +08:00
mindspore-ci-bot 4516b8feff !437 Add L2Loss op for VM. 
Merge pull request !437 from liuxiao/add-l2-loss
 2020-04-22 10:27:04 +08:00
mindspore-ci-bot 3b083d07bb !550 support one node communication group 
Merge pull request !550 from yao_yf/support_one_node_communication_group
 2020-04-22 10:17:01 +08:00
liuxiao c874e2d484 Add L2Loss op for VM 2020-04-22 09:31:14 +08:00
mindspore-ci-bot aad5771a62 !524 Added support for UA augmentation ops with tests 
Merge pull request !524 from alashkari/ua-ops
 2020-04-22 08:28:47 +08:00
Amir Lashkari 56e7a7deb5 Added UniformAugment + Python Augmentation Ops 2020-04-21 19:54:32 -04:00
mindspore-ci-bot dc0491caf9 !508 [Dataset] Adding sync_wait operator for dataset 
Merge pull request !508 from EricZ/master
 2020-04-22 06:15:24 +08:00
eric cd94518769 X# This is a combination of 2 commits. 
Initial commit for dataset op python

Added signature to barrier

Adde compiling barrier code

Rebasing, fixed new compile errors

Final fix for make_unique

Added pybind API for barrier

Fixed pyfunc invocation

python interface - sync_wait

!1 sync_wait python interface
* python interface - sync_wait

fix test

update test

update test

Added new test case

add test case

test for shuffle + batch

Added two-sync test case

Restrited that no shuffle after sync

Added sync to pipeline info

block first databuffer as well

Intelligently get batch size

Fix default case

Lock Pair shares among all iterators

Added fix for empty character

Fixed up test case formatting

Fix end of epoch in sync_wait

Fixing CI
 2020-04-21 17:42:13 -04:00
Junhan Hu 78001ac9e6 Add multiprocessing support for Mindspore.Dataset.GeneratorDataset 2020-04-21 16:03:43 -04:00
mindspore-ci-bot fb18671b28 !506 [Dataset] Multiprocessing support for Pyfunc 
Merge pull request !506 from JunhanHu/multiprocess_pyfunc
 2020-04-22 03:57:26 +08:00
Junhan Hu b13e7bc31a Add python multiprocessing support for Mindspore.dataset 2020-04-21 14:56:25 -04:00
yao_yf 371b3338b6 support one node communication group 2020-04-21 21:16:22 +08:00
mindspore-ci-bot 72f42fc37c !170 Add prim name to error message for operators in nn_ops.py 
Merge pull request !170 from fary86/add_prim_name_to_error_message_for_nn_ops
 2020-04-21 21:14:44 +08:00
wangnan39@huawei.com b812b18c02 support update parameter for vm 2020-04-21 20:19:46 +08:00
fary86 6dd72f654a Add prim name to error message for nn_ops.py 2020-04-21 19:25:30 +08:00
qianlong db80f4ff92 The num_samples and numRows in schema for TFRecordDataset are conflict 2020-04-21 17:08:59 +08:00
zhaojichen 94c99998ae add AvgPooling layer 2020-04-21 03:17:24 -04:00
mindspore-ci-bot d9e4dcc33b !483 Optimize skip dataset op 
Merge pull request !483 from jiangzhiwen/dataset/skip_opt
 2020-04-21 15:10:23 +08:00
mindspore-ci-bot 862d23fe90 !481 Move args_type_check function to _checkparam.py 
Merge pull request !481 from leonwanghui/rm-code
 2020-04-21 11:23:05 +08:00
mindspore-ci-bot 7aac50804a !514 [MD] add Pk Sampler in minddataset 
Merge pull request !514 from liyong126/mindrecord_pk_sampler_lee
 2020-04-21 11:10:19 +08:00
leonwanghui 24b26ee1a8 Move args_type_check function to _checkparam.py 2020-04-21 10:20:09 +08:00
liyong f1542a90a3 add pk sampler 2020-04-21 10:08:38 +08:00
candanzg 0de05b3949 [bug] fixed bool check for cast op 
Signed-off-by: candanzg <zhangshucheng@huawei.com>
 2020-04-21 09:48:34 +08:00
mindspore-ci-bot 27a88a6bc3 !456 Auto cast case for NotEqual op 
Merge pull request !456 from candanzg/not_equal_auto_cast
 2020-04-21 09:20:40 +08:00
jiangzhiwen e1b109e8b8 optimize skip dataset op 2020-04-21 09:16:37 +08:00
Cathy Wong 60df369100 Fixup py Normalize doc: takes input CHW 2020-04-20 15:17:30 -04:00
mindspore-ci-bot 6369cf27bd !406 added first row crc check for when reading tfrecord files 
Merge pull request !406 from Peilin/first-row-crc-check
 2020-04-21 00:10:18 +08:00
mindspore-ci-bot 98fbd30a5b !460 [Data]Add filter operation 
Merge pull request !460 from xulei/filter_master
 2020-04-20 23:33:44 +08:00
mindspore-ci-bot 822a3160e4 !404 [Dataset] Add Python Sampler support for CPP dataset 
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
 2020-04-20 22:45:51 +08:00
xulei2020 c705ea5e5b add filterOp code 2020-04-20 22:32:19 +08:00
Peilin Wang 9bc2134cb7 added checking of first row crc to find invalid tfrecord files 
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file

fixed clang format

fixed cppcheck

fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before

fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments

more clang whitespace fixes...

changed print to logger.info
 2020-04-20 10:25:26 -04:00
mindspore-ci-bot d8176a77f4 !425 New api "TextFileDataset" 
Merge pull request !425 from yanghaitao/yht_textfiledatasetv2
 2020-04-20 22:14:26 +08:00
mindspore-ci-bot 5434a96d85 !493 support python func print and != for list with none 
Merge pull request !493 from zhangbuxue/support_python_func_print
 2020-04-20 20:32:18 +08:00
mindspore-ci-bot 0cdb1171d5 !87 Take AllToAll as a virtual operator in cost model 
Merge pull request !87 from gziyan/dev_alltoall
 2020-04-20 20:13:15 +08:00
mindspore-ci-bot c3ec9712e0 !403 add cell class name to error message 
Merge pull request !403 from fary86/add_cell_name_to_error_message_for_nn_layer
 2020-04-20 20:06:12 +08:00
buxue 7c233a57fa support python func print and != for list with none 2020-04-20 19:55:50 +08:00
mindspore-ci-bot 679dbd27b3 !478 fix save_graphs_path and log file mode related issues 
Merge pull request !478 from fary86/fix_log_and_ir_issues
 2020-04-20 19:07:38 +08:00
fary86 7e23a1a475 Fix issues of save_graphs_path, Type/Value error message and log file mode 2020-04-20 16:23:40 +08:00
mindspore-ci-bot 3ba31ec15a !419 Tensor assign with bool Tensor 
Merge pull request !419 from candanzg/tensor_assign_bool_index
 2020-04-20 15:00:55 +08:00
yanghaitao 2795e492ff TextFileDataset 2020-04-20 11:18:55 +08:00
mindspore-ci-bot 18580a7867 !261 Add summary module python code to extract histogram info from tensor 
Merge pull request !261 from wenkai/histogram_dev3cp
 2020-04-20 11:08:34 +08:00
mindspore-ci-bot 3822b4837f !340 Add a HistogramSummary ops to record tensor value 
Merge pull request !340 from ougongchang/master
 2020-04-20 10:54:04 +08:00
Ziyan 0d208e00bd Model ALLTOALL as a single operator in cost model; scale the ALLTOALL, 
ALLGATHER, and REDUCESCATTER with different factors; change the BETA and
GAMMA value in cost model.
 2020-04-20 09:11:03 +08:00
candanzg 3f087dba1a Tensor assign syntax: 
1) A[B]=U
2) A[A>n]=U
    A.shape == B.shape
    U is a scalar or Tensor(size==1)
    B is Tensor(dtype=bool)
    n is a Number

Signed-off-by: candanzg <zhangshucheng@huawei.com>
 2020-04-20 08:44:34 +08:00
Junhan Hu 43a2e99833 Add python sampler support for CPP dataset 2020-04-19 20:36:37 -04:00
candanzg 79c4312e89 NotEqual op auto cast 
Signed-off-by: candanzg <zhangshucheng@huawei.com>
 2020-04-19 11:41:47 +08:00
ms_yan f0c07c3fa6 Realize take op and add ut 2020-04-18 18:33:57 +08:00
mindspore-ci-bot cc08fdfd9a !420 Fix mixed precision usage when const value passed as argument 
Merge pull request !420 from amongo/FixMixedPrecisionForConstParam
 2020-04-18 10:31:33 +08:00
ougongchang 0ed6d9178e add Histogram summary operator 
clean clang format errors and cpplint errors

add some test cases for histogram summary operator
 2020-04-18 09:56:45 +08:00
mindspore-ci-bot 80333e9f55 !435 Fix dataset serialize and deserialize for MindDataset 
Merge pull request !435 from anthonyaje/fix_serdes_dataset
 2020-04-18 09:37:22 +08:00
mindspore-ci-bot d34afbd61b !400 Support math operation between float and int for scalar 
Merge pull request !400 from amongo/FixMathOps
 2020-04-18 09:34:42 +08:00
mindspore-ci-bot 40f0a4a4f4 !333 Add skip op to Dataset 
Merge pull request !333 from jiangzhiwen/dataset/skip
 2020-04-18 09:19:07 +08:00
mindspore-ci-bot 9e1b5efd1d !434 Bug in cleaning dataset iterators 
Merge pull request !434 from h.farahat/multi_itr_bug
 2020-04-18 05:17:31 +08:00
anthonyaje ea297c0889 Fix dataset serdes for MindDataset 2020-04-17 13:24:02 -04:00
hesham 3c02c82771 Bug in weak reference. 
Add new test cases
 2020-04-17 11:36:32 -04:00
jzw 3f7054dccb add skip dataset op 2020-04-17 19:48:25 +08:00
huangdongrun 7fae76d10d add support for calculation between float and int 
format code

remove test case for fp64

add back scalar manipulation

remove save graph
 2020-04-17 19:13:52 +08:00
huangdongrun 9cb665e634 add suport for parameter of const value pass as mixed precision args 
fix pylint
 2020-04-17 16:27:46 +08:00
mindspore-ci-bot 7214c04114 !402 Add parallel op for Square 
Merge pull request !402 from yangzhenzhang/square
 2020-04-17 15:06:40 +08:00
fary86 8cbbbd950e Add cell name to error message 2020-04-17 12:58:11 +08:00
yangzhenzhang 36ffb66782 add parallel op for square 2020-04-17 10:42:52 +08:00
mindspore-ci-bot 2d31ae97e8 !383 add cell psnr 
Merge pull request !383 from zhaozhenlong/cell/psnr
 2020-04-17 10:23:30 +08:00
mindspore-ci-bot 08985a1e56 !286 resolve pynative operator unsupport issue 
Merge pull request !286 from wangqiuliang/resolve-pynative-operator-issue
 2020-04-17 10:04:54 +08:00
wenkai cce61d462c histogram python dev 2020-04-17 09:42:02 +08:00
zhaozhenlong aa8fbcc06e add cell psnr 2020-04-17 09:15:51 +08:00
mindspore-ci-bot cf026096a6 !183 Mindspore.dataset CPP sampler for GeneratorDataset 
Merge pull request !183 from JunhanHu/cpp_sampler
 2020-04-16 22:30:59 +08:00
kingfo 75fec82b52 resolve pynative operator issue 2020-04-16 21:02:22 +08:00
mindspore-ci-bot d3e4dd06b0 !386 modify test_cell_bprop.py ut test_with_param 
Merge pull request !386 from changzherui/modify-ut
 2020-04-16 20:15:15 +08:00
mindspore-ci-bot 549bfb97ad !343 Fix usage of variable arguments in test framework 
Merge pull request !343 from amongo/FixGradUsingInterfaceExtend
 2020-04-16 19:42:07 +08:00
mindspore-ci-bot 31621467a4 !217 support operator ** // % for scalar and tensor, and "in" "not in" key word, and string concat 
Merge pull request !217 from zhangbuxue/support_pow_operator
 2020-04-16 18:53:47 +08:00
mindspore-ci-bot 0d0517c22d !379 Add Group normalization 
Merge pull request !379 from JichenZhao/groupnorm
 2020-04-16 17:30:12 +08:00
buxue 80978cf3cc support operator ** // % for scalar and tensor, and in not in for dict, ang str concat 2020-04-16 17:03:31 +08:00
huangdongrun efedcfb530 add test case of Bprop and tester 
add test case for grad concat

fix usage in test framework

fix testcase

format code
 2020-04-16 16:59:18 +08:00
chang zherui 8bdcd8c267 modify test_cell_bprop.py ut 2020-04-16 16:20:24 +08:00
yangzhenzhang 57cd9f8188 add parallel op for sigmoidloss 2020-04-16 14:33:24 +08:00
zhaojichen 04c522d0c6 Add Group Normalization 2020-04-16 01:59:32 -04:00
zhaojichen 0b7de6968f Add Group Normalization 2020-04-16 01:56:12 -04:00
zhaojichen ebe6efff71 Add Group Normalization 2020-04-16 01:42:32 -04:00
yangzhenzhang 6d522f0a4f add parallel op for layernorm 2020-04-15 19:30:07 +08:00
mindspore-ci-bot ba55a8ed0b !304 [Auto parallel] Change 'NOT_FULLY_USE_DEVICES' to 'FULLY_USE_DEVICES' and make ALL-1 user-specified-strategy valid in auto-parallel 
Merge pull request !304 from Xiaoda/modify-not-fully-use-devices-and-strategy-checking
 2020-04-15 17:21:45 +08:00
buxue 62807da0c4 Develop operator Unfold，take the ge backend，dock with tbe's ExtractImagePatches operator 2020-04-15 15:26:44 +08:00
Xiaoda Zhang ffb2cb03a4 Change 'NOT_FULLY_USE_DEVICES' to 'FULLY_USE_DEVICES' and make ALL-1 user-specified-strategy valid in auto-parallel 2020-04-15 10:14:46 +08:00
Junhan Hu 9739d3b048 Add CPP sampler support for GeneratorDataset 2020-04-14 12:09:03 -04:00
mindspore-ci-bot 30de261c3c !243 Support nested repeat 
Merge pull request !243 from h.farahat/nested_repeat
 2020-04-14 23:07:32 +08:00
mindspore-ci-bot fe4cde6b1e !294 Add cell ssim 
Merge pull request !294 from zhaozhenlong/cell/ssim
 2020-04-14 22:34:03 +08:00
mindspore-ci-bot 150024f089 !312 Implemention of new dataset ops: apply 
Merge pull request !312 from xiefangqi/xfq_apply_branch
 2020-04-14 22:29:07 +08:00
hesham 0fc23eee0f Support nested repeat 2020-04-14 10:13:18 -04:00
zhaozhenlong 6a2cf4b6e6 ssim impl code 2020-04-14 21:53:29 +08:00
xiefangqi 1a1cbc6814 implemention of new api: apply 2020-04-14 21:43:53 +08:00
Wei Luning e1c8f248e0 Fix the output is not tuple, when eval 2020-04-14 16:14:41 +08:00
mindspore-ci-bot f1fa2a9941 !273 [MD] update subset random sampler in minddataset 
Merge pull request !273 from liyong126/mindrecord_subset_sampler_python
 2020-04-14 11:18:36 +08:00