yankf21
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

!30973 fix some bugs in parallel 

Merge pull request !30973 from yao_yf/communication_group_fix
This commit is contained in:
i-robot 2022-03-09 06:42:05 +00:00 committed by Gitee
parent aec8ab9c75 a87bbea034
commit 46cc578dcf
No known key found for this signature in database
GPG Key ID: 173E9B9CA92EEF8F
4 changed files with 23 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/api/api_python/nn/mindspore.nn.AdaSumByDeltaWeightWrapCell.rst
View File

@ -27,7 +27,7 @@ mindspore.nn.AdaSumByDeltaWeightWrapCell
**输入：**
- **gradients** (tuple[Tensor]) - `params` 的梯度，形状（shape）与 `params` 相同，与所传优化器的输入一致。
- **grads** (tuple[Tensor]) - `params` 的梯度，形状（shape）与 `params` 相同，与所传优化器的输入一致。
**异常：**

2
docs/api/api_python/nn/mindspore.nn.AdaSumByGradWrapCell.rst
View File

@ -27,7 +27,7 @@ mindspore.nn.AdaSumByGradWrapCell
**输入：**
- **gradients** (tuple[Tensor]) - `params` 的梯度，形状（shape）与 `params` 相同，与所传优化器的输入一致。
- **grads** (tuple[Tensor]) - `params` 的梯度，形状（shape）与 `params` 相同，与所传优化器的输入一致。
**异常：**

3
mindspore/ccsrc/frontend/parallel/device_manager.cc
View File

@ -170,8 +170,7 @@ Status IsFeasibleDeiveListOneServer(const RankList &rank_list) {
if (rank_list.size() == 4 && (rank_list[3] - rank_list[0] == 3) && (rank_list[0] == 0 || rank_list[3] == 7)) {
return SUCCESS;
}
if (rank_list.size() == 4 && (rank_list[3] % 2 == rank_list[2] % 2) && (rank_list[2] % 2 == rank_list[1] % 2) &&
(rank_list[1] % 2 == rank_list[0] % 2)) {
if (rank_list.size() == 4 && (rank_list[3] % 4 == rank_list[1] % 4) && (rank_list[2] % 4 == rank_list[0] % 4)) {
return SUCCESS;
}
if (rank_list.size() == 2) {

22
mindspore/python/mindspore/nn/optim/adasum.py
View File

@ -392,7 +392,16 @@ class AdaSumByGradWrapCell(Cell):
requires only one input.
Inputs:
- **grads** (Tuple(Tensor)) - Tuple of gradients.
- **grads** (Tuple(Tensor)) - Tuple of gradients, same with the input of passed optimizer.
Raises:
RuntimeError: If `parallel_mode` uses `stand_alone` mode, AdaSum only supports use in distributed scenarios.
RuntimeError: If the optimizer parallel is used when using AdaSum.
RuntimeError: If the pipeline parallel is used when using AdaSum.
RuntimeError: If `device_num` is not a power of 2, or less than 16.
Supported Platforms:
``Ascend`` ``GPU``
Examples:
>>> from mindspore import nn
@ -436,7 +445,16 @@ class AdaSumByDeltaWeightWrapCell(Cell):
requires only one input.
Inputs:
- **grads** (Tuple(Tensor)) - Tuple of gradients.
- **grads** (Tuple(Tensor)) - Tuple of gradients, same with the input of passed optimizer.
Raises:
RuntimeError: If `parallel_mode` uses `stand_alone` mode, AdaSum only supports use in distributed scenarios.
RuntimeError: If the optimizer parallel is used when using AdaSum.
RuntimeError: If the pipeline parallel is used when using AdaSum.
RuntimeError: If `device_num` is not a power of 2, or less than 16.
Supported Platforms:
``Ascend`` ``GPU``
Examples:
>>> from mindspore import nn