natas_hw
/
dolphinscheduler0701
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,251 Commits 28 Branches 59 Tags 445 MiB
Commit Graph

7251 Commits

Author SHA1 Message Date
Jiajie Zhong d3f087e29b
[feat][ci] Add pre-commit for easy spotless config (#12192) 
* Use pre-commit to handle spotless check
* Migrate exists python pre-commit-config into dir root one
* Change related document
 2022-09-29 10:10:53 +08:00
Eric Gao d1e409e9a2
[Improvement][Test] Remove powermock in service and server modules (#12164) 
* Remove powermock and refactor some related code in dolphinscheduler-service and dolphinscheduler-server modules
* Remove redundant comments
* Add null check
 2022-09-29 08:16:23 +08:00
darkz 995c1480f9
[Document][Improvement] Polish docs for standalone deployment (#12181) 
* Polish docs for standalone deployment
 2022-09-28 20:32:22 +08:00
JieguangZhou 856083d591
fix ml group (#12172) 2022-09-28 18:30:52 +08:00
TyrantLucifer 4973b30efb
[Feature][doc] Add a guide to nominate new committer (#12175) 
Co-authored-by: Kirs <ACM_China@yeah.net>
 2022-09-28 14:23:09 +08:00
Stalary aa5526ef5d
[Improvement][Test] Remove Powermock in dolphinscheduler-task-plugin (#12153) 
* Remove the usage of powermock in task-plugin module
 2022-09-28 14:13:23 +08:00
Jiajie Zhong ad4f344283
[fix][test] Fix flaky test in CI (#12017) 2022-09-28 14:05:26 +08:00
Wenjun Ruan d526dd89f6
[hotfix][UI] Remove SURVEIL and NEXT_LOOP in ui (#12160) 
* Remove SURVEIL and NEXT_LOOP in ui

* Fix UT
 2022-09-28 09:17:12 +08:00
Chris 7ed52c3ecb
[improve][python] Validate version of Python API at launch (#11626) 2022-09-27 21:28:20 +08:00
xuhhui 6466cc7c41
[Improvement][Master] Construct processInstance may NPE when master handling command (#12056) 
* [Improvement][Master] Construct processInstance may NPE when master handling command

* use an enpty map

Co-authored-by: xuhaihui <xuhaihui@cmss.chinamobile.com>
 2022-09-27 09:13:48 +08:00
Jiajie Zhong b95db44130
[fix][doc] Do not change docsdev.js during releasing (#12151) 
ref: https://github.com/apache/dolphinscheduler/pull/12146#discussion_r979708629
and https://github.com/apache/dolphinscheduler-website/pull/829
 2022-09-26 21:46:18 +08:00
Devosend 87490a34cc
fix env config space bug (#12147) 2022-09-26 19:06:56 +08:00
Eric Gao cc82206c33
Remove powermock in dolphinscheduler-master module (#12143) 
* Remove powermock related code in dolphinscheduler-master module
* Remove redundant comments, useless code and unnecessary stubbings
 2022-09-26 17:02:14 +08:00
Kerwin 4269fa7ea7
[release][doc] Add release 3.0.1 (#12139) 
* add release 3.0.1
* add 30x version in the github bug report
 2022-09-26 16:18:31 +08:00
冯剑 779a0b9c75
[Improvement-#11608][task-plugin] New time variables are added to facilitate business development (#11667) 
Add new time expressions for users' convenience

Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
Co-authored-by: David <dailidong66@gmail.com>
 2022-09-26 16:01:32 +08:00
caishunfeng 151c2d2c70
fix upgrade sql (#12141) 2022-09-26 15:21:52 +08:00
Tq cf5a8894e2
[BUG][BE-12113]fix htrace dependency conflicts (#12126) 
* fix htrace dependency conflicts

* replace htrace-core with hbase-noop-htrace
 2022-09-26 13:49:08 +08:00
zhuxt2015 e6832220c3
[Improvement][SQL Task]use default sql segment separator (#10869) 2022-09-26 11:08:57 +08:00
labbomb 64f4cb4f3e
refactor: dag-menu refactor (#11965) 
* refactor: dag-menu refactor

* feat: add i18n support for the dag-menu

* feat: add e2e class

Co-authored-by: devosend <devosend@gmail.com>
Co-authored-by: Tq <tianqitobethefirst@gmail.com>
 2022-09-26 11:06:39 +08:00
Yann Ann 289e1ecdc6
[Bug] [Common] update IPv4 Pattern (#11762) 
* [Bug] [Common] IPv4 Pattern is wrong. #11758
 2022-09-26 09:10:22 +08:00
Yann Ann ada7cf71d5
[fix#12000]Cannot remove the WorkerGroup from the master service (#12050) 
* [Bug] [Master] Cannot remove the WorkerGroup from the master service. #12000

* remove unnecessary locks

* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-09-24 18:57:20 +08:00
Eric Gao 064224696f
[Doc][Improvement] Add instructions for process execution type and data complement (#12121) 
* Add instructions for process execution type and data complement

* Fix and polish the docs
 2022-09-24 09:59:02 +08:00
Jiajie Zhong 82ddd72e4a
[Feature][API] New restful API for workflow and schedule (#11912) 
* [feat] New restful API for workflow and schedule

CURD for workflow and schedule, different with exists
API, this new restful api only operate single resource
in each request, and return the latest. For example,
previous workflow should also need to post tasks definition
and tasks relation definition, but this patch will allow
you to create workflow without task relate information

* use checkProjectAndAuthThrowException, and fix CI error

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ScheduleV2Controller.java

* change method name
from createProcessDefinitionV2 to createSingleProcessDefinition
from updateProcessDefinitionV2 to updateSingleProcessDefinition

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
 2022-09-23 22:25:19 +08:00
Jiajie Zhong a86f4e2693
[fix][python] Task switch branch not show in webui (#12120) 
when create task switch in python api, switch branch not show correctly,
due to we add some unnecessary attribute to switch task, this patch we
add `_task_ignore_attr` in class `Task` to ignore those unnecessary
attribute from `_task_default_attr`
 2022-09-23 19:41:05 +08:00
Jiajie Zhong 2531c222de
[Fix] Also auto formatter workflow instance if location is null (#12080) 
* [fix] Also auto formatter workflow instance if location is null

In #11535(096fae77) and #11681(4dca488c), we already formatter
workflow definition, but I find out we forget formatter workflow
instance with definition's location is null, this patch also
formatter workflow instance.

* remove console log
 2022-09-23 18:20:39 +08:00
JieguangZhou 6eb1eb722a
fix python home conflict (#12112) 2022-09-23 09:40:18 +08:00
Amy0104 e11e129cda
[Improvement][UI] Add filterable for `src_table` and `target_table` in the data quality task. (#12099) 2022-09-22 20:43:34 +08:00
chenrj b1b57c8fb7
[feat][python] new resource plugin GitHub and gitlab (#11831) 
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
Co-authored-by: JieguangZhou <jieguang_zhou@163.com>
Co-authored-by: Devosend <devosend@gmail.com>
Co-authored-by: juzimao <578961953@qq.com>
 2022-09-22 14:43:00 +08:00
jackfanwan 6e03a7e557
[BUG]Delete Setting ID (#12096) 
* Delete Setting ID

* modify id

Co-authored-by: jackfanwan <fanwanlong@kezaihui.com>
 2022-09-22 12:12:43 +08:00
Lyle Shaw cc492c3e13
[Feature] Add CURD to the project/tenant/user section of the python-DS (#11162) 
- Add CURD in project
- Add CURD in tenant
- Add CURD in user
- Add test in user

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
 2022-09-22 11:03:27 +08:00
Wenjun Ruan fba5a8eaa0
Fix insert command error due to the id is not null (#12092) 2022-09-22 09:52:32 +08:00
冯剑 9b3b4e22b4
[Bug-#12057][task-plugin] fix when the sql query result is empty, the email fails to send the attachment, and an exception will be reported (#12059) 
* [Bug-#12057][task-plugin] fix when the sql query result is empty, the email fails to send the attachment, and an exception will be reported

Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
 2022-09-22 09:39:02 +08:00
rickchengx 08a4c7981f
[improve] Remove the spark version of spark task (#11860) 2022-09-21 20:27:28 +08:00
JieguangZhou cf522e2fc0
[improve] Optimize MLFlow task plugin for easy of use (#12071) 
optimize code
 2022-09-21 10:45:05 +08:00
Eric Gao b52da64001
[Improvement][UT] Upgrade jUnit to 5.+ (#10976) (#11332) 
* [Improvement][UT] Upgrade jUnit to 5.+ (#10976)

* Refactor AlertServerTest with jUnit-5 as an example
 2022-09-21 10:15:21 +08:00
jackfanwan 7d80ea34ac
[Improvement] Add Set cluster name (#12074) 2022-09-21 10:05:15 +08:00
limaiwang ed1d1e8856
add(dependent node's add unprivileged project list): dependent node's add unprivileged project list (#12073) 2022-09-21 09:28:22 +08:00
Wenjun Ruan cb4ef54a0e
Script cannot contains ''' in params (#12067) 2022-09-21 08:52:51 +08:00
Eric Gao 66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708) 
* Add support for Alibaba Cloud OSS as storage of resource center

* Add dependency licenses

* Refactor code, add unit tests and related docs
 2022-09-20 22:44:06 +08:00
Wenjun Ruan 26dad5e2a8
Fix update TaskDefinition error (#12060) 2022-09-20 22:23:16 +08:00
jackfanwan 0bf639b851
[improvement]Add Set cluster name (#12058) 
* Add Set cluster name

* add unit test

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
 2022-09-20 21:26:05 +08:00
insist777 955e8ebe0f
[Improve] Fix support test tasks UT (#12062) 2022-09-20 15:46:29 +08:00
金鑫 649015bfff
[Bug][ApiServer] When modifying the task definition, inserting the task definition log table will report the bug of primary key conflict #12037 
Co-authored-by: 金鑫 <jinyanhui@huansi.net>
 2022-09-20 14:42:21 +08:00
Eric Gao e1b55db72a
[Improvement][Test] Give an example on replacing Powermock with Mockito (#11588) 
* Give an example on replacing powermock with mockito

* Remove redundant comments

* Refactoring related UTs for better readability.
 2022-09-20 09:48:02 +08:00
rickchengx b3bbf69b2e
[Improvement-11291][sql] Improvement the log output while executing multiple sql statements (#11292) 2022-09-20 09:30:27 +08:00
songjianet 8cddb10773
[Feat][UI] Update package. (#12046) 
* [Feat][UI] Update package.

* [Feat][UI] Update package.
 2022-09-19 20:58:41 +08:00
Kerwin 89d55bc9b7
Fix link errors in release documentation. (#12049) 2022-09-19 20:01:23 +08:00
juzimao c8884e4f2f
and task instance date range search (#11963) 2022-09-19 17:50:24 +08:00
JieguangZhou ad683c3c42
[feat][python] Support MLflow task in python api (#11962) 2022-09-18 16:28:18 +08:00
limfff c24ad9adb6
[Fix] Fixed an error importing project Maven (#12026) 2022-09-18 10:11:20 +08:00