Xiaoda Zhang
df67e74eaf
making sharding_propagation smooth, add a reshape justification:
...
1) when propagate sharding strategy from one op to another, try to find the strategy with zero communication cost;
2) if there is no such strategy, find the strategy with minimum communication cost, and raise a warning;
2021-11-18 15:55:50 +08:00
i-robot
9f52343a6a
!26350 add check for resizenearestneighbor parallel op
...
Merge pull request !26350 from yangzhenzhang/add-check-for-resize-op
2021-11-18 05:45:57 +00:00
ba99e4c505
add check for resize op
2021-11-16 15:39:52 +08:00
ttudu
33ac1de062
fix bug
2021-11-16 11:14:58 +08:00
i-robot
7a73bae5c3
!26036 add output strategy for matmul operator
...
Merge pull request !26036 from yangzhenzhang/add-output-strategy-for-op-init
2021-11-12 01:22:22 +00:00
Xiaoda Zhang
a772767265
support reshape in sharding propagation:
...
1) using 'swc index of strategy_cost_' as reshape's selected strategy;
2) when encountering reshape in BFS, select the 'swc index' with zero communication cost;
3) when encountering a reshape that is already visited, check whether there exists communication between reshape and current operator. It is OK if communication happens between two configured operators;
4) currently, two consecutive reshapes are not supported;
5) adjusting BFS structure in graph_costmodel.cc;
6) adjusting some code in step_auto_parallel.cc to avoid cyclomatic complexity.
2021-11-11 11:35:43 +08:00
8431ba616c
add output strategy for op init
2021-11-10 15:35:45 +08:00
f354ab22a3
add pipeline shard interface
...
Add support for no pipeline accugradient
Add delay tag for fusion op
Optimizer the visite order
add mirror for mini step control
Move the group to attributes
Add gradient_shard control for the mini step
Fix code stype
Fix ut description
Add interface
2021-11-09 09:11:13 +08:00
i-robot
ded1c77bbf
!25765 neighborExchangeV2 & grad
...
Merge pull request !25765 from TuDouNi/neighborExchangeV2
2021-11-05 09:31:03 +00:00
ttudu
e953c15cd2
NeighborExchangeV2 & Grad
2021-11-05 11:00:46 +08:00
6ad6304b77
add output strategy
2021-11-04 16:28:54 +08:00
c42081619e
add parallel op for resizenearestneighbor
2021-10-28 11:31:00 +08:00
4fbc59a98a
utfixs
2021-10-18 18:06:37 +08:00
i-robot
f83070728d
!24790 support user define strategy gen method under auto parallel context
...
Merge pull request !24790 from zhuyuxiao/master
2021-10-14 09:03:36 +00:00
zhuyuxiao
1907246931
change api
2021-10-13 10:28:30 +08:00
i-robot
3fd94000c5
!24568 Apply batch parallel in auto_parallel mode when strategies are not specified
...
Merge pull request !24568 from zhuyuxiao/master
2021-10-13 01:20:15 +00:00
zhuyuxiao
cf76c76745
apply batch parallel in auto_parallel mode when strategies are not specified
2021-10-12 15:44:10 +08:00
b303d6001c
parallel ut refactor
2021-10-12 15:18:51 +08:00
i-robot
7cde7731b0
!23537 Update pangu reshape and softmax.
...
Merge pull request !23537 from linqingke/pangu
2021-09-24 08:12:13 +00:00
i-robot
e7cb505e68
!23569 Produce parallel operators for ResizeBilinear and ResizeNearestNeighbor
...
Merge pull request !23569 from Bert0108/resizebilinear_parallel_ops
2021-09-24 03:05:23 +00:00
i-robot
d37fccc56f
!23544 remove deprecated gather op
...
Merge pull request !23544 from zhuyuxiao/master
2021-09-24 01:20:53 +00:00
2d3d0b673e
parallel operators for ResizeBilinear and ResizeNearestNeighbor
2021-09-24 09:07:49 +08:00
acde7febef
update pangu reshape and softmax performance.
...
Add layer norm judge
Fix layer norm name error
Fix input tyoe check
Fix ut test
Add 3d supports
2021-09-24 09:06:16 +08:00
zhoufeng
1f934bd782
check neighbor attr type
...
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2021-09-17 21:15:37 +08:00
zhuyuxiao
79d99323a2
rename gather
2021-09-16 16:07:05 +08:00
0b89d5c9c4
fix batch size error
2021-09-16 10:56:40 +08:00
i-robot
bbdacd41f4
!20585 add security isolate for save_graphs
...
Merge pull request !20585 from huanghui/add-security-isolate-for-DumpIR
2021-09-15 01:25:46 +00:00
i-robot
db19a40280
!23365 add print op security
...
Merge pull request !23365 from fangzehua/print_sec
2021-09-14 09:01:31 +00:00
huanghui
ba66c0d491
add security isolate for save_graphs
2021-09-14 10:24:12 +08:00
4ccc635a07
add print security
2021-09-13 17:45:36 +08:00
1b8eb283e4
modify batch parallel info
2021-09-13 16:28:56 +08:00
e02f553010
Fix spell error and add mode check
2021-09-09 18:50:53 +08:00
Xiaoda Zhang
5613c0b974
add a moe implementation:
...
1) extend the Liner cell for including BatchMatMul implementation, in
which the first dimension indicates the expert number;
2) implement a Switch (top1) router;
3) implement a MoE cell, which extends the FeedForward cell.
2021-09-09 11:53:42 +08:00
i-robot
77424eaad5
!23004 Add args Check for Transformer
...
Merge pull request !23004 from huangxinjing/args_check
2021-09-08 08:24:06 +00:00
6cea07f749
Add args check
2021-09-08 09:12:35 +08:00
3ef26288a2
parallel_sparse_attention_ops_fix_repeated_cal
2021-09-06 19:57:22 +08:00
i-robot
fa12d62d4d
!21776 set device_id master 0813
...
Merge pull request !21776 from mindspore_ding/set_device_id_master_0813
2021-09-06 03:02:58 +00:00
dingpeifei
b4bc6000dc
set device id master 0813
2021-09-02 20:51:50 +08:00
i-robot
d87d0e07c2
!22255 recompute_interface_modify
...
Merge pull request !22255 from yao_yf/recompute_interface_modify
2021-09-01 08:09:36 +00:00
39055af6e4
recompute interface modify
2021-09-01 10:59:59 +08:00
i-robot
cc8d614b25
!22650 fixed sparse attention modify
...
Merge pull request !22650 from yao_yf/fixed_sparse_attention_modify
2021-09-01 01:29:45 +00:00
i-robot
389f3a6b6c
!21835 make alltoall and neighborexchange to be interface && revert pr 21395
...
Merge pull request !21835 from zhoufeng/revert-same-input-to-comm-op
2021-08-31 08:53:46 +00:00
82889ec56b
fixed sparse attention
2021-08-31 16:11:56 +08:00
68dd138462
add parallel sparse attention ops: dsd_matmul
2021-08-30 18:46:52 +08:00
zhoufeng
ecae690a19
Revert "fix same node is used by two comm op"
...
This reverts commit b09d411dc4
2021-08-30 16:54:38 +08:00
b8a9cbe2a3
add cus_matmul_dds parallel ops
2021-08-30 09:44:48 +08:00
Zhang Qinghua
a137fa1d0b
Optimize the Executors routines.
...
- Fix the key generating.
- Distinguish the executors.
2021-08-28 17:36:12 +08:00
zhihenghu
ce12c02343
Add Sparse Attention
...
adjut the file structure and name
Deleted extra information
Do some formatting work
Add test case and fix some document
fix imports
2021-08-28 11:50:58 +08:00
i-robot
e6e1f37ae4
!22346 [Core] Fix the bug of scope setting when cloning nodes
...
Merge pull request !22346 from Xiaoda/86-fix-the-fullname-scope-bug
2021-08-26 13:04:15 +00:00
i-robot
8d00a8d803
!22360 Fix Transformer Mirror Error
...
Merge pull request !22360 from huangxinjing/fix_transformer_mirror_error
2021-08-26 08:16:33 +00:00
Xiaoda Zhang
i-robot
yangzhenzhang
ttudu
zhuyuxiao
zhoufeng
huanghui
dingpeifei
Zhang Qinghua
zhihenghu