Commit Graph

59 Commits

Author SHA1 Message Date
mindspore-ci-bot 5b95409022 !13512 add some expander ops
From: @zengzitao
Reviewed-by: 
Signed-off-by:
2021-03-19 15:00:23 +08:00
mindspore-ci-bot 2fadad0875 !13121 expander lamb_apply_optimizer_assign
From: @wenfangpei
Reviewed-by: 
Signed-off-by:
2021-03-19 14:59:37 +08:00
wenfangpei 043a558ae2 expander lamb_apply_optimizer_assign 2021-03-19 09:54:10 +08:00
zengzitao d0a656f3cd add some expander ops 2021-03-18 19:03:36 +08:00
dayschan a2967330ea Normalize the Reduce nodes' axis in GraphKernel
changed all reduce axis to non-negative value
2021-03-16 15:55:50 +08:00
tronzhang 7252ffb66b pipeline optimization for parallel fusion 2021-03-09 17:07:24 +08:00
mindspore-ci-bot d285692217 !12852 Change maketuple in coreops
From: @liangzhibo
Reviewed-by: 
Signed-off-by:
2021-03-05 19:13:42 +08:00
l00591931 680324f225 Change make tuple in core.ops 2021-03-05 10:54:33 +08:00
dayschan 454500309c add OpInfer for op Select 2021-03-05 10:21:28 +08:00
dayschan 7beca18f3c Refactor GraphKernelExpander (3rd submission)
Rewrited the OpInfer as a class with functions "infer_shape","infer_type"
and "infer_format". Used op name to find the subclass that have implemented
these functions. Two common class "_Elemwise" and "_Reduce" was provided.

Op BiasAddGrad supported "FractalNZ" format.
2021-02-25 19:14:46 +08:00
dayschan 9d572f3963 Refactor GraphKernelExpander (2nd submission)
Changed the expander to class, and used the class name to identify the operator.
Moved the original op logic into the `_expand` function; added a `_check` function to check op inputs.

Use decorator to register the whitelist formats that operator supports,
and the decorator will change the `_check` function to check the formats.
2021-02-22 14:49:21 +08:00
dayschan e0e6c39eae Refactor GraphKernelExpander (1st submission)
Decoupled from the front-end interfaces.
1. Removed the call to "Renormalize".
   Completed the infer-format in model_builder.
   Only used the device shape and device format to
   infer an abstract shape without considering padding.
2. Removed the call to python's Primitive interfaces.
   The "Renormalize" relies on the PrimitivePy, so they can be
   removed together. After that, the functions "ConstAttrToInput",
   "DeleteAttrInInput" and related can be removed.
3. Reuse the AkgKernelJsonGenerator in GraphKernelExpander.
   1) set the attribute "extract_opinfo_from_anf" to true, so that
   the generator can handle the basic operator with anf info.
   2) added a function "extract_expand_info" in expander.py
   to convert the json into a more friendly format. The attrs
   was converted to a dict instead of a list.
4. Scalars only support DefaultFormat.
   Removed the argument "format" from graph_builder.value
5. Moved the expander op list from graph_kernel_helper.cc to graph_kernel_expander.cc
2021-02-07 10:48:23 +08:00
jinyaohui 30a27b2adb modify Gelu、FastGelu to GeLU and FastGeLU 2021-02-05 17:19:52 +08:00
mindspore-ci-bot 0ff27ef3b4 !11930 【GraphKernel】Replace Assign with InplaceAssign
From: @dayschan
Reviewed-by: @gaoxiong1,@dylangeng
Signed-off-by: @gaoxiong1
2021-02-03 11:04:07 +08:00
mindspore-ci-bot a24ff36d9c !11777 stitch fusion
From: @r1chardf1d0
Reviewed-by: 
Signed-off-by:
2021-02-02 10:49:04 +08:00
l00591931 9ec100d069 Change TensorAdd to Add, from r1.1 to master 2021-02-01 17:53:52 +08:00
dayschan 08345c54ea [GraphKernel] Replace Assign with InplaceAssign
1. added a pass to replace Assign with InplaceAssign.
2. bugfix in eliminate_redundant_output. the side-effect node should not be eliminated.
3. bugfix in graph_kernel/splitter.py, the kernel includes InplaceAssign should be a composite node.
4. added two tool functions GetAllInputDeviceTypes and GetAllOutputDeviceTypes into AnfAlgo.
5. do not fuse a single Assign in pass BasicOpsFusion.
2021-02-01 17:15:35 +08:00
r1chardf1d0 9d6392c5c5 stitch info 2021-02-01 10:20:55 +08:00
tronzhang d078cbfa99 support parallel fusion 2021-01-27 17:27:13 +08:00
tronzhang ad59129c07 if input is none, make it a null list 2021-01-07 09:37:28 +08:00
Gaoxiong cc89c3f896 set default mode for output reshape op 2021-01-04 21:07:58 +08:00
Gaoxiong 32e19e83da update graph kernel split model for Ascend 2021-01-03 17:17:02 +08:00
mindspore-ci-bot 639e0c5fbd !10257 【GraphKernel】Enhance the fusion capacity for getitem nodes
From: @dayschan
Reviewed-by: @gaoxiong1,@ckey_dou
Signed-off-by: @ckey_dou
2020-12-22 09:50:41 +08:00
dayschan 26ac9167f8 Enhance the fusion capacity for getitem nodes.
fixbug in ReplaceNewFuseCNode
add a pass to eliminate repeated output after cse
fixbug in graph_kernel_splitter
do not fuse reshape op as output in costmodel.
2020-12-21 20:22:26 +08:00
looop5 8bbe723603 add Tile infer shape function 2020-12-21 16:50:12 +08:00
looop5 56fa56b173 add graph kernel testcases 2020-12-12 14:56:21 +08:00
mindspore-ci-bot 037a121e05 !9691 expand ClipByNormNoDivSum in graph kernel
From: @looop5
Reviewed-by: @gaoxiong1,@ckey_dou
Signed-off-by: @ckey_dou
2020-12-09 16:07:50 +08:00
dayschan 85b69bf91f Add a float16 restriction in the solution of reduction op's precision problem in graph splitter. 2020-12-09 11:56:03 +08:00
looop5 fa519433ef expand ClipByNormNoDivSum 2020-12-09 09:25:56 +08:00
dayschan 297f075dca Fix precision problem 2020-12-09 00:16:20 +08:00
zhupuxu 4f569677b7 redundant_codes
Signed-off-by: zhupuxu <zhupuxu@huawei.com>
2020-12-08 19:27:04 +08:00
mindspore-ci-bot 7b311f7d2a !9570 Modifications for GraphKernel
From: @dayschan
Reviewed-by: @gaoxiong1,@ckey_dou
Signed-off-by: @ckey_dou
2020-12-08 09:29:04 +08:00
dayschan 6be3cc6f0d consider atomic_add strategy in graph splitter; fixbugs; fuse and inline single op 2020-12-07 15:53:32 +08:00
looop5 848be9b07c add tile to expand list
add tile expander

add BroadcastTo in model

fix BroadcastTo op calling error and infer shape

rewrite tile expander

not split broadcast_to

add SqrtGrad expander
2020-12-05 10:57:55 +08:00
dayschan e5306b913d GraphKernel Fuser
Refactor the BasicOpsFusion and CompositeOpsFusion to one pass.
Add a pass to eliminate the redundant output.

TODO: rename the file basic_ops_fusion and delete the file composite_ops_fusion
2020-12-04 11:02:36 +08:00
tronzhang 2190da9946 support atomic clean and change package for akg. 2020-12-02 16:26:41 +08:00
mindspore-ci-bot ebef1df00b !8994 split dropout op and expand dropout
From: @zengzitao
Reviewed-by: 
Signed-off-by:
2020-11-26 20:31:51 +08:00
zengzitao 3ef0e9f053 substitute dropout by cudnnuniformreal and dropout 2020-11-26 10:36:02 +08:00
Gaoxiong e4c3d3e0e9 update graph kernel split model 2020-11-25 14:26:00 +08:00
mindspore-ci-bot 232dff3598 !8685 [GraphKernel] For fp16 value, declare fp32 firstly and than cast to fp16 in expander
From: @tronzhang
Reviewed-by: 
Signed-off-by:
2020-11-18 14:28:52 +08:00
mindspore-ci-bot 3b946d4eb2 !8678 expand logsoftmax and grad, delete cast in softmax and fix layernorm compute dsl
From: @zengzitao
Reviewed-by: @gaoxiong1,@ryanww
Signed-off-by: @ryanww
2020-11-18 14:27:37 +08:00
tronzhang 80f071e9fa declare fp32 and than cast to fp16 in expander 2020-11-18 09:55:34 +08:00
tronzhang 9d7494f4df split shape ops for more fusion pportunity. 2020-11-17 15:22:51 +08:00
zengzitao 266bfa50bf expand logsoftmax and logsoftmax_grad, delete softmax's cast and fix layernorm op 2020-11-17 09:57:58 +08:00
zengzitao 326540cbbd expand layernorm_grad op 2020-11-16 09:03:13 +08:00
zengzitao 28f1db74dd expand maximum_grad minimum_grad dropout_grad op 2020-11-12 10:31:03 +08:00
dayschan 195b1fe8d5 Add Transpose into fusible list. 2020-11-11 11:02:06 +08:00
zengzitao db27783d54 expand tanh_grad and reduce_mean, fix bug and add test_case in ci 2020-11-10 10:37:38 +08:00
zengzitao 53043ae18f support expand fused_adam and fused_adam_weight_decay op 2020-11-05 20:56:49 +08:00
zengzitao 5cfa172720 expand gelu and gelugrad op 2020-11-02 11:35:01 +08:00