Col_lin
/
mindspore2022
forked from huawei/mindspore2022
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
27,162 Commits 18 Branches 22 Tags 3.6 GiB
Commit Graph

53 Commits

Author SHA1 Message Date
Parastoo Ashtari df425aa027 fixed core dump issue in bert net 2021-07-12 17:55:44 -04:00
Parastoo Ashtari e2a4172560 make gpu dump truly async 2021-06-28 16:44:35 -04:00
Parastoo Ashtari 6ed17d52b1 Add debugger to new unified GPU runtime 2021-06-22 18:23:56 -04:00
John Tzanakakis ac1847ffac fix iter 0 and iter 1 being dumped in dir 0, make op_debug_mode optional for sync mode, read input files for offline debugger 2021-06-09 21:57:07 -04:00
jiangshuqiang 6154f957fb fix static check 2021-05-31 16:47:21 +08:00
TinaMengtingZhang da6e068ed7 fix ci codecheck alarm in master 2021-05-25 15:47:47 -04:00
Parastoo Ashtari e9e18d2253 add suspend for last node in the graph 
removed double stopping for the last kernel

moved the ResetLoadedTensors to the correct place for GPU
 2021-05-12 16:25:34 -04:00
mindspore-ci-bot 963eece5d9 !16034 use device type in dump file name when trans_flag=false 
From: @john_tzanakakis
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @robingrosman
 2021-05-07 21:10:30 +08:00
John Tzanakakis b5e70f7231 use device type in dump file name when trans_flag=false 2021-05-06 14:07:20 -04:00
mindspore-ci-bot ca0ca3d2c2 !15793 fix open source static check 
From: @jiang-shuqiang
Reviewed-by: @yelihua,@yelihua,@lilongfei15
Signed-off-by: @lilongfei15
 2021-05-06 20:51:22 +08:00
jiangshuqiang e90697801e fix open source static check 2021-05-06 11:23:12 +08:00
mindspore-ci-bot 32c8a1529e !15981 Address ci alarm in master 
From: @tina_mengting_zhang
Reviewed-by: @john_tzanakakis,@robingrosman
Signed-off-by: @john_tzanakakis
 2021-05-04 21:21:19 +08:00
tinazhang 6b84149e2f Address ci alarm for master branch: pclint, CppCodeStyle and codex 2021-05-03 16:46:14 -04:00
Parastoo Ashtari 7b9a73fb1b Fixing multi graph suspend for debugger in GPU 
removed the suspend from preExecute function to avoid double stopping in multigraph models

replaced else if with else in postExecute

add else if to check the smoke ascend test

improve the format
 2021-04-28 22:32:42 -04:00
John Tzanakakis ddae425e0c use host instead of device shape for debugger 2021-04-22 17:54:51 -04:00
Islam Amin 0fa5a443be fix dataset_sink_mode=True issue with GPU 2021-03-15 11:57:23 -04:00
Amir Lashkari d83e7c41c8 Fixed log issue 
Fixed cpplint
 2021-03-09 14:23:34 -05:00
maning202007 b437c0d418 Change debugger version from 1.1.0 to 1.2.0 2021-02-08 14:52:34 +08:00
mindspore-ci-bot 59d2affa97 !10021 Fixed double stutter issue at the end of step for debugger 
From: @adelshafiei
Reviewed-by: @john_tzanakakis,@wangyue01
Signed-off-by: @wangyue01
 2021-01-06 10:08:55 +08:00
jiangshuqiang 0411ffa922 fix env param 
add true/false/0 for ENABLE_MS_DEBUGGER
 2020-12-19 14:42:29 +08:00
Adel Shafiei 1d795a4f9d Fixed the double stutter problem at the end of step for debugger 2020-12-15 13:59:58 -05:00
Adel Shafiei 809c060ae1 fixed recheck issue for operator overflow 2020-12-11 13:07:44 -05:00
Adel Shafiei f4f8ff7d9a support operation overflow watchpoint for multi-graph 2020-12-09 10:12:16 -05:00
Harshvardhan Gupta dd0084c52b improve perf, keep consistent tensor state, fix recheck, check weights at step end 2020-12-08 15:22:40 -05:00
mindspore-ci-bot 95573571f0 !9511 Codedex change for tensor_loader 
From: @liangzhibo
Reviewed-by: 
Signed-off-by:
 2020-12-07 12:47:25 +08:00
l00591931 1d1cab986d Codedex change for tensor_loader 2020-12-07 09:45:27 +08:00
Adel Shafiei 870062770b raise an exception if debugger PORT and/or IP are not set correctly. 2020-12-03 13:54:16 -05:00
Adel Shafiei a322184057 debugger backend compability support 2020-11-27 11:43:25 -05:00
yelihua 5026ef1d46 load tensors for all graphs 2020-11-15 19:11:41 +08:00
lichen_101010 1b6265fa43 Debugger multi-graph support implementation 
Other Contributor: Adel Shafiei, John Tzanakakis
 2020-11-14 06:41:42 -05:00
Harshvardhan Gupta 727d424553 support watchpoints on weights and bias, add support advanced watchpoints 2020-11-03 13:23:52 -05:00
Harshvardhan Gupta e6451adc50 generate appropriate error when debugger is used with dataset_sink_mode=True 2020-10-29 15:39:29 -04:00
Harshvardhan Gupta 81e90c5ece add recheck 2020-10-20 10:40:02 -04:00
Harshvardhan Gupta 744355a005 remove dbg args from runtime and remove needless argument from IsWatchpoint 2020-10-19 11:22:22 -04:00
lichen_101010 678962d718 implementation for dumping const values for each step 
fix no output address error

address clang-format error

address some comments

clang_format check
 2020-10-15 10:34:04 -04:00
Harshvardhan Gupta e77e816f1d fix overflow dump path 2020-10-08 12:14:06 -04:00
Harshvardhan Gupta 7c5e0541ba load inputs before suspending execution in dbg 2020-10-05 10:40:32 -04:00
maning202007 ec631997b0 fix some safe warnings in debugger 2020-09-21 14:41:14 +08:00
maning202007 82bad1ab84 Fix unsafe codes in Debugger 2020-09-18 10:09:21 +08:00
lichen_101010 dffa61b228 send info when training is done 2020-09-11 18:40:54 -04:00
John Tzanakakis b0a7ebdeb0 enable debugger by default and set correct log message severity 2020-09-10 21:42:00 -04:00
maning202007 af9c9b5129 Delete the openssl related code to restore debugger 2020-09-10 11:14:11 +08:00
Harshvardhan Gupta 3226e8407a fix gpu wp 2020-08-27 17:41:50 -04:00
maning202007 303bec43d4 Add secure mechanism for debugger 
Add certificate validaty check
 2020-08-21 10:45:56 +08:00
mindspore-ci-bot 3449abd741 !4629 Support sending large graph and large tensors for mindspore debugger 
Merge pull request !4629 from lichen_101010/sendgraph_sendtensor_issue
 2020-08-21 10:22:13 +08:00
lichen_101010 2a8bf9f00b addressed sendgraph issue 
change buffer type and chunk size

addressed sendtensor issue for large tensors

fix sendtensor part 2

fix bugs for sendtensor issue

fix CI errors

removes a print line

address comments
 2020-08-20 11:34:05 -04:00
Adel Shafiei 4834a3378b Op Overflow Watchpoint support for D-chip debugger 
Other Authors: Harshvardhan Gupta, Li Chen
 2020-08-17 18:06:47 -04:00
lichen_101010 7499c72d54 mindspore grpc implementation 
fix bugs for grpc implementation

addressed peer review comments

delete device_target code from Adel

add checksinglewatchpoint function for node level debugger

set the device target when sending metadata

add current node name

fix bugs for current node name

fix run_level_ bug

fix bugs for CheckSingleWatchpoint

fix multi-outputs node issue

fix num_step_ bug

fix continue_to previous node issue

fix run_level issue

fix merge conflict

smart kernel read, watch hit stop mid-sep, fix step number, read input tensors

cleanup the code and isolate UpdataStepNum function

do cpplint, Cppcheck and clang-format check

recover CMakeList.txt

mindspore grpc implementation

fix bugs for grpc implementation

addressed peer review comments

delete device_target code from Adel

add checksinglewatchpoint function for node level debugger

set the device target when sending metadata

add current node name

fix bugs for current node name

fix run_level_ bug

fix bugs for CheckSingleWatchpoint

fix multi-outputs node issue

fix num_step_ bug

fix continue_to previous node issue

fix run_level issue

fix merge conflict

smart kernel read, watch hit stop mid-sep, fix step number, read input tensors

cleanup the code and isolate UpdataStepNum function

do cpplint, Cppcheck and clang-format check

recover CMakeList.txt

only update step_num in one place

fix clang-format error

fix CI errors part2

update graphengine version

addressed comments
 2020-08-04 12:51:45 -04:00
John Tzanakakis b3c0eb61d5 GPU debugger - milestone 1 and GPU dump 
Additonal Authors: Adel Shafiei, Harshvardhan Gupta
 2020-07-22 15:58:36 -04:00
lichen_101010 6bb2182134 Add partial memory reuse support to debugger 
move pre-execution of debugger from rungraph to build/compile graph

support partial mem reuse for a scope of nodes

set default mem reuse to be true for debugger

remove some redundant lines

remove redundant code and fix a bug for supporting partial no mem reuse a scope of nodes

resolve CI errors

Solve CI errors

solve cpplint errors

solve CI build error

manually fix the CI compile UT error

Optimize code for mem reuse support

Debug optimization of debugger memory reuse

debug code for debugger memory reuse part2

address clang-format errors

Switch memory reuse on and off based on environment variable

Fix typo

Fix typo

Load watchpoint value only

fix bugs

Addressed comments from lupengcheng

fix typo

Fix typo

fix CI errors

refactor some code

fix typo

addressed comments from canadian teamates

remove locking from TensorLoader

fix CI errors

add lock to tensor_loader

fix rebase-to-master conflict

fix rebase conflicts

fix rebase conflicts part 2

fix rebase conflicts part 3
 2020-07-14 14:47:25 -04:00