Col_lin
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
42,205 Commits 18 Branches 22 Tags 3.6 GiB
Commit Graph

99 Commits

Author SHA1 Message Date
臧庆香 061ce5df20 parallel strategy pb to json 2022-04-02 19:00:35 +08:00
臧庆香 4349a5e375 parallel strategy remove graph name 2022-03-30 09:37:04 +08:00
liuchuting d15ce09cd1 fix some warning of ms. 2022-03-20 23:47:05 +08:00
zhoufeng b7bb53ff73 build libmindspore_backend.so 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-03-17 19:14:45 +08:00
臧庆香 1dd489096e profiler pynative python code 2022-03-09 12:38:39 +08:00
i-robot 476ec8007b
!30775 Support GPU cluster step trace. 
Merge pull request !30775 from liuchuting/Q1
 2022-03-05 06:37:24 +00:00
i-robot e0de84afec
!30787 profiler supports pynative mode 
Merge pull request !30787 from zangqx/zangqx_5
 2022-03-04 09:43:27 +00:00
liuchuting cf2877d86e Support cluster step trace. 2022-03-04 14:28:06 +08:00
臧庆香 1fdd76c50f profiler pynative cpp 2022-03-04 12:14:00 +08:00
zhoufeng f49b195c39 extract common as an independent shared library 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-02-26 10:55:51 +08:00
fangzehua e64055d2ca add parallel profiling 2022-02-22 14:38:24 +08:00
marui d47e79b04c Refactor ccsrc directories and CMakeLists files 2022-02-13 14:37:51 +08:00
liuchuting 2a9a968f7f Profiler memory could not collect data. 2021-12-30 16:12:19 +08:00
yanghaitao1 c9d2bfad7b report cann error msg if failed to call cann api 2021-12-21 15:45:54 +08:00
liuchuting 6f6f21126d Fixed an issue where GPU multi-card network iteration tail time was negative. 2021-12-21 10:03:04 +08:00
liuchuting 0686651e9b Fixed an issue where GPU multi-card network iteration tail time was negative. 2021-12-19 19:57:35 +08:00
yuximiao 5e494d71e4 report cann msg if cann api return error 2021-12-16 15:18:42 +08:00
yuximiao 393d3621a3 fix no paralle stratege if start profiler in the process of trainging 2021-12-13 10:44:07 +08:00
臧庆香 1c82f79320 Add ACL_PROF_HCCL_TRACE Switch 2021-12-06 20:33:47 +08:00
i-robot 812dba7e21 !24771 add compile option H to visible hidden interface 
Merge pull request !24771 from Healing/hidden_visible
 2021-12-03 03:57:55 +00:00
yuximiao e99c0a48e6 support start profiler in the minddle of training. 2021-11-30 10:42:00 +08:00
He Wei 41dcac9c49 Replace std::unordered_map/set with robin-hood-hashing 
Robin-hood-hashing (https://github.com/martinus/robin-hood-hashing)
is considered faster then std::unordered_map/set,
so we use it to improve mindspore performance.

1. robin_hood head file in `third_party/robin_hood/include`;
2. In `utils/hash_map.h` and `utils/hash_set.h`, we define:
 - mindspore::HashMap as an alias of robin_hood::unordered_map;
 - mindspore::HashSet as an alias of robin_hood::unordered_set;
3. Replace:
 - `#include <unordered_map>` --> `#include "utils/hash_map.h"`;
 - `#include <unordered_set>` --> `#include "utils/hash_set.h"`;
 - `std::unordered_map` --> `mindspore::HashMap`;
 - `std::unordered_set` --> `mindspore::HashSet`;
 - `map.insert(std::pair(key, value))` --> `map.emplace(key, value)`;
 - `[] (const std::pair<K, V> &p) {..} ` --> `[] (const auto &p) {..} `;
4. Fix issues found by switch to robin_hood:
 - AnfNodeConfig hash and equal;
 - Fix a bug in `Slice::operator==()`;
 - Fix a bug in `CNode::HasPrimalAttr()`;
 - Fix map.erase() usage bugs: `map.erase(iter++)` --> `iter = map.erase(iter)`;
 - Fix some iterator invalidated problem;
5. Some std::unordered_map/set can not replace by robin_hood:
 - As parameter of functions that exposed to python by pybind11;
 - Use bad hash that cause robin_hood::map over_flow, such as AbstractBasePtrListHasher;
6. Update cpp unit tests;
7. Add build option '-F' to enable robin_hood, default on.
 2021-11-24 10:47:40 +08:00
ougongchang c01dbf25c9 Change the parallel strategt data to JSON format 
Because Protobuf parses the pb file data slowly, in a matter of seconds, it is changed to JSON data
 2021-11-23 21:42:04 +08:00
i-robot 8d603ffc41 !26468 Change profiler parallel strategy file mode to 600, keep same with the other files 
Merge pull request !26468 from ougongchang/fix_mode
 2021-11-19 01:43:05 +00:00
i-robot ce00ee1ad1 !25367 use acl api to control profiling 
Merge pull request !25367 from yanghaitao/yht_condation_start_profiler
 2021-11-18 12:36:14 +00:00
ougongchang 744e291982 Change profiler parallel strategy file mode to 600, keep same with the 
other files
 2021-11-18 09:44:19 +08:00
yanghaitao1 c94aa6b872 use profiler acl api instead 2021-11-17 09:33:31 -05:00
ougongchang a748d2c8af fix profiling parallel strategy can not get data in SaveCompileGraph phase 2021-11-17 11:52:00 +08:00
liuchuting bb5bf3f289 repair warning of memcpy,memset 2021-11-15 14:46:40 +08:00
ougongchang 9229f1c1ff profiler support to collect parallel strategy info 
If SetNodeOutputType functions forcibly splits into multiple functions, the readability decreases, so it blocks lizard scans
 2021-11-11 20:11:42 +08:00
yelihua cfa8c7a0e8 update OWNERS 2021-11-08 17:23:35 +08:00
healing 56ae399228 open gpu profiling interface 2021-11-03 21:19:00 +08:00
yanghaitao1 a30b940ad3 add commiter to OWNERS files 2021-09-29 03:48:58 -04:00
i-robot 9f92ac51b1 !24250 fix some review issues in profiler 
Merge pull request !24250 from yanghaitao/yht_fix_review_issue_0927
 2021-09-28 01:53:07 +00:00
yanghaitao1 09e2d4bb88 fix some review issues in profiler 2021-09-27 05:53:04 -04:00
yanghaitao1 6a92b2c538 fix reviews issues in profiler 2021-09-26 02:41:45 -04:00
i-robot a009c8e8b4 !23542 fix pclint in profiler 
Merge pull request !23542 from yanghaitao/yht_fix_issue_0915
 2021-09-16 12:02:35 +00:00
yanghaitao1 56fd6f57e7 fix pclint in profiler 2021-09-15 04:07:27 -04:00
yanghaitao1 177f3f75bf remove profiler if compiled with -s on 2021-09-14 05:29:18 -04:00
i-robot 94f7cba97c !23219 remove profiling parameters in set_context function 
Merge pull request !23219 from yanghaitao/yht_remove_set_context_from_profiling_3
 2021-09-11 07:02:00 +00:00
zhangyihui e121fb2e8c Heterogeneous cpu performance in GPU 2021-09-10 20:11:42 +08:00
yanghaitao1 83302ad23c remove profiling parameters in set_context function 2021-09-10 04:35:35 -04:00
i-robot 18078a6457 !22917 If users need GPU profiling, they need to use the root account to start the process 
Merge pull request !22917 from zangqx/profiling_gpu_permission
 2021-09-08 01:49:09 +00:00
臧庆香 2bc81c48e0 profiling GPU permission 2021-09-07 17:23:53 +08:00
zhangyihui 6a36171a0e transform device_id to rank_id for cpu_profiler 2021-09-07 14:50:31 +08:00
zhangyihui 3d19949eb4 device_id to rank_id 2021-08-25 09:52:23 +08:00
ms_yan 36a8886ca2 Revert "[feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset" 
This reverts commit b077aa1cab.

Revert "[feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset"

This reverts commit 4e6f7dc97d.

delete pass_registry_test.cc

comment  hiai_nlu_model_multi.pb related  line
 2021-08-23 01:46:38 +08:00
djc b077aa1cab [feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset 2021-08-22 16:26:45 +08:00
djc 4e6f7dc97d [feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset 2021-08-22 13:39:37 +08:00
lvchangquan e8d9803258 add profiling codes based on ascend and gpu in pynative mode 2021-08-10 20:46:51 +08:00