Commit Graph

187 Commits

Author SHA1 Message Date
zhuangchong a3a9cb9d46 [maven-release-plugin] prepare for next development iteration 2022-12-07 17:23:58 +08:00
zhuangchong f1aefae5e2 [maven-release-plugin] prepare release 3.1.2 2022-12-07 17:23:57 +08:00
Kerwin d4c6ec4fa1 [Bug-12963] [Master] Fix dependent task node null pointer exception (#12965)
* Fix that there are both manual and scheduled workflow instances in dependent nodes, and one of them will report a null pointer exception during execution.
2022-11-24 19:03:49 +08:00
Kerwin 6ffac27dbc fix NPE while retry task (#12903) 2022-11-16 11:05:13 +08:00
zhuangchong 48ad9ff412 [maven-release-plugin] prepare for next development iteration 2022-11-08 11:18:29 +08:00
zhuangchong efc0367999 [maven-release-plugin] prepare release 3.1.1 2022-11-08 11:18:28 +08:00
zhuangchong 145c5ac8cb [maven-release-plugin] prepare for next development iteration 2022-11-07 20:43:01 +08:00
zhuangchong 525a6846ac [maven-release-plugin] prepare release 3.1.1 2022-11-07 20:43:00 +08:00
Kerwin 9126e1d1fc
[3.1.1-prepare][Test] test ut (#12729) 2022-11-07 10:57:48 +08:00
Wenjun Ruan 9957df1c41 Fix the waiting strategy cannot recovery if the serverstate is already in running (#12651) 2022-11-02 21:50:59 +08:00
Wenjun Ruan 0c3ab4335e cherry-pick Add worker-group-refresh-interval in master config #12601 2022-10-31 13:51:52 +08:00
Kerwin 94ef02e9de
cherry-pick [Improvement] Merge spi.utils into common.utils (#12607)
#12502

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-29 21:30:28 +08:00
liqiquan 826ca39a31 [Bug][Master] Add the aws-java-sdk-s3 jar package to the master module (#12259) (#12512)
Co-authored-by: qiquan.li <qiquan.li@linkflowtech.com>
2022-10-27 14:47:12 +08:00
kezhenxu94 eb7b48596c cherry-pick Reorganize some classes in common module, remove duplicate classes
#12321
2022-10-27 13:53:41 +08:00
sgw d742994e62 [DS-12131][master] Optimize the log printing of the master module acc… (#12152)
* [DS-12131][master] Optimize the log printing of the master module according to the log specification.
2022-10-27 13:22:48 +08:00
Wenjun Ruan a17d0cc5d2 cherry-pick [Improvement][WorkerGroup]Remove workerGroup in registry #12217 2022-10-27 13:21:24 +08:00
Kerwin 58ef3cccd7
[3.1.1-prepare]cherry-pick [Improvement] remove log-server and server module #12206 (#12562) 2022-10-27 12:54:38 +08:00
Kerwin f3277277f0
cherry-pick Refactor LogServiceClient Singleton to avoid repeat creation of NettyClient #11777 (#12542)
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2022-10-26 16:20:13 +08:00
Kerwin 344293102d
[3.1.1-prepare][cherry-pick][Improvement] Add remote task model #11767 (#12541) 2022-10-26 15:44:13 +08:00
Yann Ann 35608becac make sure all failed task will save in errorTaskMap (#12424) 2022-10-19 10:49:47 +08:00
Kerwin 2402b8a6ef
Use temurin Java docker images instead of deprecated ones (#12334) (#12421)
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-10-18 18:51:14 +08:00
caishunfeng 7e39396a76 [maven-release-plugin] prepare for next development iteration 2022-09-26 21:50:03 +08:00
caishunfeng ae33ba5947 [maven-release-plugin] prepare release 3.1.0 2022-09-26 21:49:27 +08:00
Tq 3efe4bc308 [BUG][BE-12113]fix htrace dependency conflicts (#12126)
* fix htrace dependency conflicts

* replace htrace-core with hbase-noop-htrace
2022-09-26 13:49:11 +08:00
Yann Ann 3e899bee06 [fix#12000]Cannot remove the WorkerGroup from the master service (#12050)
* [Bug] [Master] Cannot remove the WorkerGroup from the master service. #12000

* remove unnecessary locks

* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-09-25 23:18:09 +08:00
caishunfeng 780a509f67 fix workflow keep running when task fail (#11930) 2022-09-19 17:39:18 +08:00
Daniel Y 2a437607ae [Improvement-11834] Upgrade docker base image to support python3.9 (#11835)
* use openjdk:8-jre-slim-bullseye instead of buster to support python3.9

* use openjdk:8-jre-slim-bullseye instead of buster across all components
2022-09-19 15:50:25 +08:00
LiuBodong 0f3b42925f [Fix][Monitor]Monitor UI not show DisakAvailable and MemoryUsage correctly (#11870) 2022-09-19 15:30:49 +08:00
kezhenxu94 89b4192d3d Remove htrace dependencies (#11881)
* Remove htrace dependencies

* Update pom.xml
2022-09-19 15:27:41 +08:00
Kengo Seki 216ceea641 [Improve] Upgrade Hadoop to 3.2.4 (#11832) 2022-09-19 15:26:58 +08:00
kezhenxu94 fcc75ef1c6 Bump up dependencies to fix cves (#11765) 2022-09-19 15:10:56 +08:00
Kengo Seki 52b79b017e [Improvement] Replace commons-lang 2 function invocations with commons-lang3 (#11810)
* [Improvement] Replace commons-lang 2 function invocations with commons-lang3
2022-09-19 14:48:17 +08:00
caishunfeng 27b69e608a
[Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 (#11864) (#11950)
* [Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838

Co-authored-by: Yann Ann <2993643785@qq.com>
2022-09-15 10:39:00 +08:00
caishunfeng f8d46a26c1
[Feature-11530] add state history for process instance (#11757)
* [Feature] add state history for process instance (#97)

* add state history for process instance

* upsertProcessInstance

* remove unuse method

* fix UT

Co-authored-by: caishunfeng <534328519@qq.com>
2022-09-03 10:14:05 +08:00
kezhenxu94 277f137358
Add Kubernetes configmap reload to all components (#11730) 2022-09-02 12:03:19 +08:00
caishunfeng f95cdd350a
reduce task dependencies (#11731) 2022-09-01 17:04:07 +08:00
Wenjun Ruan 67e7f88d8b
Refactor heart beat task, use json to serialize/deserialize (#11702)
* Refactor heart beat task, use json to serialize/deserialize
2022-08-31 16:20:23 +08:00
Wenjun Ruan b96d69701a
[Improvement-#11613] Add spi priority factory (#11614)
* Add spi priority factory

* Add doc

* Add override log

* Use lombok

* Add comment
2022-08-24 19:51:47 +08:00
JinYong Li 3f2ca7bca3
[Fix-9980] [Server] fix heartBeatTaskCount bug (#11232)
* fix heartBeat bug

* modify class name

* fix conflict

Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
2022-08-23 11:30:13 +08:00
insist777 abfef1a929
[improvement] Add two parameters in workergroup, and support the application of description display and other parameters (#11542)
* [Improvement] Improvement default worker group

* [Improvement] Improvement default worker group

* [Improvement] Improvement default worker group

* [Improvement] Improvement default worker group

* [Improvement] Improvement default worker group

* [improvement] The default worker grouping supports editing

* [improvement] The default worker grouping supports editing

* Update WorkerGroupServiceImpl.java

* Update WorkerGroupServiceImpl.java

* The default worker grouping supports editing

* update sql file

* update

* update

* update

* update
2022-08-23 10:54:17 +08:00
Wenjun Ruan 4d13a5104b
Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469) 2022-08-23 10:05:03 +08:00
Wenjun Ruan a41c6824fc
Add dependent task instance log (#11541)
* Add dependent task instance log

* Optimize log

* Fix dependent task initialize failed will throw exception
2022-08-23 09:55:38 +08:00
Wenjun Ruan 8886d5a126
Fix quartz threadPriority config name error (#11596)
* Fix quartz threadPriority config name error

* Add batchTriggerAcquisitionMaxCount config
2022-08-23 09:32:05 +08:00
Wenjun Ruan 3516533017
Remove logger header in task log file (#11555) 2022-08-19 14:01:52 +08:00
kezhenxu94 f6ef628030
Support reading application properties from kubernetes config map (#11402) 2022-08-16 20:25:17 +08:00
Wenjun Ruan 4362fb8448
If the task processor is not found need to throw error rather than exception (#11461) 2022-08-13 22:01:39 +08:00
Wenjun Ruan 05589606a2
Set master's task running status in `runTask` to avoid the task group acquire failed, but the task status is in running (#11451) 2022-08-13 14:19:13 +08:00
Wenjun Ruan 7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368)
* Add waiting strategy to support master/worker can recover from registry lost

* throw exception when zookeeper registry start failed due to interrupted
2022-08-13 09:52:03 +08:00
Wenjun Ruan 0ca3086296
Fix workflow instance may failover many times due to doesn't check the restart time (#11445) 2022-08-13 00:05:37 +08:00
Wenjun Ruan a2b5219315
Fix find last process instance in dependent task is not correct (#11424) 2022-08-12 14:11:28 +08:00