16ac3638d3
[ME][Fallback] Add some fallback numpy test cases
2021-11-24 11:20:06 +08:00
i-robot
c6821bde0a
!21648 [assistant][ops] Add new data loading operator YesNoDataset
...
Merge pull request !21648 from 杨旭华/YesNoDataset
2021-11-19 09:05:07 +00:00
f465cd0242
[feat][assistant][I3T96L]add new dataset operator YesNo
2021-11-19 13:36:24 +08:00
i-robot
f38df5c888
!26234 New dataset config option, auto_offload
...
Merge pull request !26234 from markuskunej/offload_config
2021-11-19 03:44:56 +00:00
i-robot
7559d5b798
!26494 [Auto parallel] Adjusting sharding propagation
...
Merge pull request !26494 from Xiaoda/102-adjusting-sharding-propagation
2021-11-19 02:00:46 +00:00
i-robot
3c39afad11
!24547 [MS][crowdfunding]New operator implementation, AutoAugment
...
Merge pull request !24547 from yangwm/autoaugment
2021-11-19 01:00:51 +00:00
29cf442521
Added dataset.config.set_auto_offload(), which will automatically attempt to offload dataset map operations if set to True.
2021-11-18 21:13:20 +00:00
i-robot
117df75afc
!26126 MD Profiling - Add Start Stop Python UT
...
Merge pull request !26126 from cathwong/ckw_ut_prof_startstop
2021-11-18 14:05:00 +00:00
Xiaoda Zhang
df67e74eaf
making sharding_propagation smooth, add a reshape justification:
...
1) when propagate sharding strategy from one op to another, try to find the strategy with zero communication cost;
2) if there is no such strategy, find the strategy with minimum communication cost, and raise a warning;
2021-11-18 15:55:50 +08:00
i-robot
9f52343a6a
!26350 add check for resizenearestneighbor parallel op
...
Merge pull request !26350 from yangzhenzhang/add-check-for-resize-op
2021-11-18 05:45:57 +00:00
despicablemme
09897ce7ec
[feat] [assistant] [I40GZP] add new data op AutoAugment
2021-11-18 11:38:18 +08:00
i-robot
707e7fbbf9
!26453 [Fallback] Add some fallback numpy test cases
...
Merge pull request !26453 from Margaret_wangrui/fallback_tests
2021-11-18 03:01:47 +00:00
c6582b9390
[Fallback] Add some fallback numpy test cases
2021-11-18 10:18:13 +08:00
b1094b9d81
MD Profiling - Add Start Stop Python UT
2021-11-17 13:49:43 -05:00
i-robot
4e88f2e5e8
!26345 add examples of function grad, jvp and vjp
...
Merge pull request !26345 from chenzhuo/jvp
2021-11-17 08:35:38 +00:00
i-robot
36517de69c
!24519 [assistant][ops] Add new audio operator SlidingWindowCmn.
...
Merge pull request !24519 from Rainfor/SlidingWindowCmn
2021-11-17 06:40:43 +00:00
i-robot
9f2e1edc00
!26279 neighborexchangev2 fix bug
...
Merge pull request !26279 from TuDouNi/neighborexchangev2_bug
2021-11-17 02:29:21 +00:00
wangkc123
706f9e2cbb
[feat] [assistant] [I3CKEH] add new audio operator SlidingWindowCmn
2021-11-16 22:00:09 +08:00
ba99e4c505
add check for resize op
2021-11-16 15:39:52 +08:00
i-robot
58b69a05ee
!20576 [assistant][ops] add new dataset loading operator LJSpeechDataset
...
Merge pull request !20576 from 杨旭华/LJSpeechDataset
2021-11-16 07:00:20 +00:00
chenzhuo
d07f24462c
add examples of grad, jvp and vjp
2021-11-16 14:15:26 +08:00
ttudu
33ac1de062
fix bug
2021-11-16 11:14:58 +08:00
9663884db3
fix occasional failed in random sampler
2021-11-16 09:53:38 +08:00
i-robot
5af8572cd7
!25488 add the set_dump API
...
Merge pull request !25488 from wenkai/wk1027setdump1
2021-11-16 01:47:32 +00:00
i-robot
0113b3eee1
!26236 Add MD Profiler save()
...
Merge pull request !26236 from Mohammad Motallebi/add_profiler_save
2021-11-16 01:04:53 +00:00
mohammad
5c8ab5f60c
add MD Profiler Save()
2021-11-15 13:18:36 -05:00
d2eca80d40
add the set_dump API, see I4AUIR for details
2021-11-15 21:28:40 +08:00
i-robot
9dfe08b805
!25909 [Fallback] Add global Py param when processing global namespace info
...
Merge pull request !25909 from huangbingjian/update_global_params
2021-11-15 11:43:35 +00:00
i-robot
9522ee9686
!25703 [assistant][ops] Add math operator LpNorm
...
Merge pull request !25703 from 孟权令/LpNorm
2021-11-15 02:36:54 +00:00
6fe2d35ae2
[feat][assistant][I3T96D] add new dataset loading operator LJSpeechDataset
2021-11-14 14:52:15 +08:00
i-robot
e154dca4b1
!22193 [assistant][ops] Add new loader operator DBpedia
...
Merge pull request !22193 from 杨旭华/DBpedia
2021-11-12 09:02:28 +00:00
i-robot
e773f0621e
!25918 add Function jvp and vjp
...
Merge pull request !25918 from chenzhuo/jvp
2021-11-12 06:37:46 +00:00
i-robot
5869e02fa1
!26135 Enable tuple input of optimizer as variable
...
Merge pull request !26135 from LiangZhibo/broaden3
2021-11-12 06:29:35 +00:00
0d26c38693
[feat][assistant][I40GXT] add new loader DBpedia
2021-11-11 19:02:22 -08:00
i-robot
7a73bae5c3
!26036 add output strategy for matmul operator
...
Merge pull request !26036 from yangzhenzhang/add-output-strategy-for-op-init
2021-11-12 01:22:22 +00:00
i-robot
e03544685b
!26049 [Auto parallel] Support reshape in sharding propagation
...
Merge pull request !26049 from Xiaoda/99-ad-reshape-adjusting-in-sharding-propagation
2021-11-12 01:14:18 +00:00
chenzhuo
f36edec3ff
add function jvp
2021-11-12 08:44:41 +08:00
i-robot
1792d5f8a4
!26131 Offload Hardware Acceleration Computing
...
Merge pull request !26131 from alashkari/hw_accel_move_copy
2021-11-11 13:18:07 +00:00
i-robot
b910870ecc
!19986 [assistant][ops] Add new dataset loading operator AGNEWS
...
Merge pull request !19986 from 杨旭华/AGNEWS
2021-11-11 06:22:25 +00:00
Xiaoda Zhang
a772767265
support reshape in sharding propagation:
...
1) using 'swc index of strategy_cost_' as reshape's selected strategy;
2) when encountering reshape in BFS, select the 'swc index' with zero communication cost;
3) when encountering a reshape that is already visited, check whether there exists communication between reshape and current operator. It is OK if communication happens between two configured operators;
4) currently, two consecutive reshapes are not supported;
5) adjusting BFS structure in graph_costmodel.cc;
6) adjusting some code in step_auto_parallel.cc to avoid cyclomatic complexity.
2021-11-11 11:35:43 +08:00
l00591931
9353bd10a4
Enable tuple as variable in optimizer
2021-11-11 11:31:05 +08:00
Megalomania
c34bbabed9
[feat] [assistant] [I3T96M] add new loader AGNEWS
2021-11-09 23:51:19 -08:00
8431ba616c
add output strategy for op init
2021-11-10 15:35:45 +08:00
22e03ab12e
Adding offload hardware accelerator.
2021-11-10 07:03:43 +00:00
i-robot
f1f7731fb2
!25964 Fix serdes multithreading bug
...
Merge pull request !25964 from zetongzhao/serdes_multithreading
2021-11-09 15:48:06 +00:00
i-robot
14c3517b9a
!25452 Add Gradient Sharding
...
Merge pull request !25452 from huangxinjing/pipeline_shard_mini_step
2021-11-09 08:15:23 +00:00
d0ca18b921
Add global Py param when processing global namespace info.
2021-11-09 09:34:30 +08:00
f354ab22a3
add pipeline shard interface
...
Add support for no pipeline accugradient
Add delay tag for fusion op
Optimizer the visite order
add mirror for mini step control
Move the group to attributes
Add gradient_shard control for the mini step
Fix code stype
Fix ut description
Add interface
2021-11-09 09:11:13 +08:00
i-robot
d89a39c661
!25851 MD Profiling Start and Stop Support
...
Merge pull request !25851 from cathwong/ckw_prof_startstop4
2021-11-08 16:16:42 +00:00
i-robot
527abb8ede
!26003 [MD] Added restrictions on shuffle and num_samples
...
Merge pull request !26003 from liyong126/fix_shuffle_and_num_samples
2021-11-08 11:58:08 +00:00
Margaret_wangrui
i-robot
Xiaoda Zhang
despicablemme
wangkc123
chenzhuo
ttudu
mohammad
l00591931
Megalomania