i-robot
dfd71f668b
!26556 modify parallel api note for master
...
Merge pull request !26556 from lilei/modify_parallel_API_note
2021-11-25 03:02:55 +00:00
i-robot
8c58c2dc1d
!26730 [Auto parallel] [MoE] Changing large transpose for performace in MoE
...
Merge pull request !26730 from Xiaoda/109-change-moe-transpose-for-performance
2021-11-25 02:22:54 +00:00
f8827a09fb
modify parallel API note for master
2021-11-24 17:44:35 +08:00
Xiaoda Zhang
23387e154c
using multiple operators to implement the sematic of a large transpose
2021-11-24 15:13:47 +08:00
i-robot
519f14a909
!26006 slice recompute activation
...
Merge pull request !26006 from yao_yf/add_transformer_slice_activation_config
2021-11-24 06:14:28 +00:00
55efaabe2a
Fix api doc error
2021-11-24 09:24:40 +08:00
Xiaoda Zhang
2c25b277d0
adding constraints checking and examples for MoEConfig
2021-11-22 16:59:22 +08:00
188d39da83
slice_activation_in_recompute
...
slice recompute activation
2021-11-19 10:51:54 +08:00
8c2dec7fe2
Fix coo layers
2021-11-12 16:06:02 +08:00
i-robot
781b2ace39
!26039 Fix Merge Error For the Linear in Transformer API
...
Merge pull request !26039 from huangxinjing/fix_linear_merger_error
2021-11-10 01:06:56 +00:00
d1123c9b20
fix linear merge error for pangu model
2021-11-09 11:58:07 +08:00
f354ab22a3
add pipeline shard interface
...
Add support for no pipeline accugradient
Add delay tag for fusion op
Optimizer the visite order
add mirror for mini step control
Move the group to attributes
Add gradient_shard control for the mini step
Fix code stype
Fix ut description
Add interface
2021-11-09 09:11:13 +08:00
Xiaoda Zhang
7165c2158d
adjusting symbols
2021-11-05 10:02:12 +08:00
i-robot
b8ec196d6e
!25708 Add mode check
...
Merge pull request !25708 from huangxinjing/add_mode_check
2021-11-04 07:09:06 +00:00
i-robot
27be612516
!25719 Add Device number check
...
Merge pull request !25719 from huangxinjing/add_device_number_check
2021-11-04 01:34:22 +00:00
bd5cfafe2c
Add mode check
...
Add parallel mode check
2021-11-04 09:05:34 +08:00
741e3604eb
Add Device Number Check
2021-11-01 15:36:50 +08:00
i-robot
1cd6573bff
!25406 Fix bug for GPU Pipeline
...
Merge pull request !25406 from Cononlly/gpu-moe
2021-11-01 06:12:12 +00:00
b00518648
ef715a54a0
clean code
2021-10-30 09:23:23 +08:00
e0674b37d3
Remove the comments
...
fix matmul name
2021-10-28 11:18:01 +08:00
7bd77e549c
Fix number of the input params error
2021-10-26 12:04:12 +08:00
cc83b8064f
Fix code for GPU Pipeline
2021-10-26 10:41:51 +08:00
i-robot
6ca06c2502
!25176 Fix dropout strategy
...
Merge pull request !25176 from huangxinjing/origin/master
2021-10-21 06:07:22 +00:00
4dc8e99d7c
Fix transformer dropout error
2021-10-20 14:27:04 +08:00
i-robot
ae87f31dea
!25088 Optimize the comment and log descriptions for the python source code
...
Merge pull request !25088 from zhunaipan/master
2021-10-20 06:26:56 +00:00
8ce4e62725
optimize the comment and log description
...
修改： ops/operations/_inner_ops.py
修改： ops/operations/_quant_ops.py
修改： ops/operations/array_ops.py
修改： ops/operations/comm_ops.py
修改： ops/operations/math_ops.py
修改： ops/operations/quantum_ops.py
修改： ops/operations/rl_ops.py
修改： ops/operations/sponge_ops.py
修改： ops/operations/sponge_update_ops.py
修改： train/__init__.py
修改： common/tensor.py
修改： train/serialization.py
修改： ccsrc/pipeline/jit/parse/parse.h
修改： explainer/benchmark/_attribution/metric.py
修改： ops/composite/multitype_ops/_constexpr_utils.py
修改： ops/operations/comm_ops.py
修改： RELEASE.md
修改： mindspore/_extends/parse/standard_method.py
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/concat_offset_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/dynamic_shape_cpu_kernel.cc
修改： mindspore/ccsrc/frontend/parallel/ops_info/reshape_info.cc
修改： mindspore/ccsrc/frontend/parallel/ops_info/tile_info.cc
修改： mindspore/ccsrc/frontend/parallel/ops_info/transpose_info.cc
修改： mindspore/ccsrc/frontend/parallel/strategy.h
修改： mindspore/common/tensor.py
修改： mindspore/core/abstract/prim_arrays.cc
修改： mindspore/core/abstract/prim_nn.cc
修改： mindspore/core/ops/conv2d.cc
修改： mindspore/core/ops/logical_and.h
修改： mindspore/core/ops/logical_not.h
修改： mindspore/core/ops/logical_or.h
修改： mindspore/core/ops/reduce_all.h
修改： mindspore/core/ops/reduce_any.h
修改： mindspore/lite/src/runtime/kernel/arm/fp32_grad/sgd.cc
修改： mindspore/nn/layer/quant.py
修改： mindspore/nn/optim/sgd.py
修改： mindspore/nn/sparse/sparse.py
修改： mindspore/numpy/array_creations.py
修改： mindspore/numpy/array_ops.py
修改： mindspore/numpy/logic_ops.py
修改： mindspore/numpy/math_ops.py
修改： mindspore/ops/operations/_inner_ops.py
修改： mindspore/ops/operations/array_ops.py
修改： mindspore/ops/operations/rl_ops.py
修改： mindspore/train/_utils.py
修改： tests/ut/python/model/test_lenet_core_after_exception.py
修改： mindspore/_extends/parse/standard_method.py
修改： mindspore/ops/operations/rl_ops.py
修改： mindspore/core/abstract/prim_nn.cc
修改： mindspore/core/ops/conv2d.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/ctcloss_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/fl/fused_pull_weight_kernel.h
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/fl/fused_push_weight_kernel.h
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/mkldnn/conv2d_grad_filter_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/mkldnn/conv2d_grad_input_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/ps/sparse_apply_ftrl_ps_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/ps/sparse_apply_lazy_adam_ps_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/rolling_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/scatter_arithmetic_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/split_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/update_cache_cpu_kernel.cc
修改： mindspore/ccsrc/backend/kernel_compiler/gpu/arrays/split_gpu_kernel.h
修改： mindspore/ccsrc/backend/kernel_compiler/gpu/math/broadcast_gpu_kernel.h
修改： mindspore/ccsrc/backend/kernel_compiler/gpu/nn/conv2d_grad_input_gpu_kernel.h
修改： mindspore/ccsrc/fl/server/server.cc
修改： mindspore/ccsrc/frontend/optimizer/ad/kpynative.cc
修改： mindspore/ccsrc/frontend/optimizer/irpass/incorporate_getitem.h
修改： mindspore/ccsrc/frontend/optimizer/irpass/inline.h
修改： mindspore/ccsrc/minddata/dataset/core/device_tensor.cc
修改： mindspore/ccsrc/minddata/dataset/core/tensor.cc
修改： mindspore/ccsrc/minddata/dataset/engine/datasetops/source/emnist_op.cc
修改： mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc
修改： mindspore/ccsrc/minddata/dataset/engine/datasetops/source/qmnist_op.cc
修改： mindspore/ccsrc/minddata/dataset/engine/ir/datasetops/dataset_node.cc
修改： mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc
修改： mindspore/ccsrc/minddata/dataset/kernels/image/lite_image_utils.cc
修改： mindspore/ccsrc/pipeline/jit/action.cc
修改： mindspore/ccsrc/pipeline/jit/static_analysis/evaluator.cc
修改： mindspore/ccsrc/runtime/device/ascend/executor/tiling/op_tiling_adapter.cc
修改： mindspore/compression/quant/quant_utils.py
修改： mindspore/core/abstract/prim_nn.cc
修改： mindspore/dataset/engine/validators.py
修改： mindspore/lite/micro/coder/opcoders/nnacl/fp32/affine_fp32_coder.cc
修改： mindspore/lite/micro/coder/opcoders/nnacl/int8/affine_int8_coder.cc
修改： mindspore/lite/src/runtime/kernel/ascend310/src/custom_kernel.cc
修改： mindspore/lite/src/runtime/kernel/opencl/kernel/matmul.cc
修改： mindspore/lite/src/runtime/kernel/opencl/kernel/strassen.cc
修改： mindspore/lite/tools/common/graph_util.h
修改： mindspore/lite/tools/optimizer/fisson/fisson_util.cc
修改： mindspore/ops/composite/math_ops.py
修改： mindspore/ops/operations/_inner_ops.py
修改： mindspore/ops/operations/array_ops.py
修改： mindspore/ops/operations/math_ops.py
修改： mindspore/ops/operations/other_ops.py
修改： mindspore/boost/boost_cell_wrapper.py
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/update_cache_cpu_kernel.cc
修改： mindspore/ccsrc/common/trans.cc
修改： mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc
修改： mindspore/ccsrc/frontend/parallel/ops_info/gather_info.cc
修改： mindspore/lite/src/common/log_util.h
修改： mindspore/nn/wrap/loss_scale.py
修改： mindspore/parallel/nn/moe.py
修改： tests/mindspore_test_framework/mindspore_test.py
修改： mindspore/ccsrc/backend/kernel_compiler/cpu/split_cpu_kernel.cc
修改： mindspore/lite/tools/common/graph_util.h
修改： mindspore/ccsrc/frontend/parallel/ops_info/gather_info.cc
修改： mindspore/core/ops/conv2d.cc
修改： tests/ut/python/model/test_lenet_core_after_exception.py
2021-10-20 12:02:40 +08:00
Xiaoda Zhang
50e0fa79af
adjust indent
2021-10-18 18:25:00 +08:00
i-robot
94ed3d9f97
!24665 [Auto parallel] Refine the comments of parallel-related interfaces
...
Merge pull request !24665 from Xiaoda/95-add-comments-of-parallel-interfaces
2021-10-14 03:56:43 +00:00
Xiaoda Zhang
88b6a58462
refine the comments of parallel-related interfaces
2021-10-12 19:35:24 +08:00
ebdad2e450
Fix api comment
2021-10-11 09:11:16 +08:00
i-robot
ab5c15074a
!24486 modify parallel api note for master
...
Merge pull request !24486 from lilei/code_docs_modify_parallel_note
2021-10-09 02:36:31 +00:00
36596cc7e6
modify parallel API note
2021-09-30 17:42:24 +08:00
a3ee85ede4
Fix reshape error
2021-09-30 11:06:46 +08:00
827597f6fe
Add transformer example
2021-09-29 21:08:27 +08:00
9ddd2b7669
Add past none check
...
Add if else for past none
2021-09-26 09:13:29 +08:00
acde7febef
update pangu reshape and softmax performance.
...
Add layer norm judge
Fix layer norm name error
Fix input tyoe check
Fix ut test
Add 3d supports
2021-09-24 09:06:16 +08:00
b787c5c8c8
Fix spell error
2021-09-22 14:21:41 +08:00
31b3b46852
Replace TensorAdd with Add
2021-09-16 21:25:15 +08:00
i-robot
cdbe9b9a64
!23596 [Auto parallel] Move the MoE-related staff to an isolated file
...
Merge pull request !23596 from Xiaoda/89-moe-adaption
2021-09-16 09:57:04 +00:00
Xiaoda Zhang
615be06ec8
move moe-related staff to an isolated file
2021-09-16 14:26:19 +08:00
0b89d5c9c4
fix batch size error
2021-09-16 10:56:40 +08:00
i-robot
be10bb7af6
!23517 fix parallel opt recompute pass warning
...
Merge pull request !23517 from yao_yf/parallel_opt_recompute_pass_warning_fix
2021-09-15 08:18:14 +00:00
b647bbf444
fix warning
2021-09-15 11:57:58 +08:00
c3a98bab2b
Add code check
...
fix pangu error
fix leakyrelu error
2021-09-14 19:43:51 +08:00
i-robot
7828fb0028
!23328 Fix error message
...
Merge pull request !23328 from huangxinjing/fix_error_msg
2021-09-13 08:38:16 +00:00
3d1f74b9d0
fixed sparse atten add strategy for div
2021-09-13 10:03:41 +08:00
7932c88aaf
fix message error
2021-09-13 09:00:20 +08:00
110008e868
fix input error
2021-09-10 10:51:13 +08:00
e02f553010
Fix spell error and add mode check
2021-09-09 18:50:53 +08:00
Xiaoda Zhang
5613c0b974
add a moe implementation:
...
1) extend the Liner cell for including BatchMatMul implementation, in
which the first dimension indicates the expert number;
2) implement a Switch (top1) router;
3) implement a MoE cell, which extends the FeedForward cell.
2021-09-09 11:53:42 +08:00
i-robot
lilei
Xiaoda Zhang
b00518648