natas_hw
/
dolphinscheduler0701
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,913 Commits 28 Branches 59 Tags 445 MiB
Commit Graph

142 Commits

Author SHA1 Message Date
zhuangchong 5f85235462 [maven-release-plugin] prepare for next development iteration 2022-09-22 16:10:18 +08:00
zhuangchong cb03ca8166 [maven-release-plugin] prepare release 3.0.1 2022-09-22 16:10:17 +08:00
caishunfeng 54a19b3cb7
[Bug][Timezone][3.0.1] fix timezone when complement data and preview (#12093) 
* fix timezone in complement data and schedule preview

* fix timezone in complement data and schedule preview

* code improvement
 2022-09-22 14:32:24 +08:00
zhuangchong 364cafe138 [maven-release-plugin] prepare for next development iteration 2022-09-19 13:36:46 +08:00
zhuangchong 17e36b7060 [maven-release-plugin] prepare release 3.0.1 2022-09-19 13:36:45 +08:00
JinYong Li ab8f878631
fix heartBeatTaskCount bug (#12024) 2022-09-18 11:49:21 +08:00
Wenjun Ruan bc629f8f58
[3.0.1-prepare]cherry-pick Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469) (#12013) 
* Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469)

(cherry picked from commit 4d13a5104b)
 2022-09-18 09:37:57 +08:00
Wenjun Ruan 041e1fd3e6
Set master's task running status in `runTask` to avoid the task group acquire failed, but the task status is in running (#11451) (#12011) 
(cherry picked from commit 05589606a2)
 2022-09-17 22:55:40 +08:00
Wenjun Ruan 8fefb9ecfd
Add dependent task instance log (#11541) (#12014) 
* Add dependent task instance log

* Optimize log

* Fix dependent task initialize failed will throw exception

(cherry picked from commit a41c6824fc)
 2022-09-17 18:32:28 +08:00
Wenjun Ruan 84876b9f15
If the task processor is not found need to throw error rather than exception (#11461) (#12012) 
(cherry picked from commit 4362fb8448)
 2022-09-17 18:25:11 +08:00
Wenjun Ruan 41a6d09c4f
Fix workflow instance may failover many times due to doesn't check the restart time (#11445) (#12010) 
(cherry picked from commit 0ca3086296)
 2022-09-17 17:51:40 +08:00
Wenjun Ruan 0237654e96
Fix find last process instance in dependent task is not correct (#11424) (#12009) 
(cherry picked from commit a2b5219315)
 2022-09-17 17:50:56 +08:00
Wenjun Ruan a615e6ee3d
Fix quartz threadPriority config name error (#11596) (#12015) 
* Fix quartz threadPriority config name error

* Add batchTriggerAcquisitionMaxCount config

(cherry picked from commit 8886d5a126)
 2022-09-17 14:25:20 +08:00
Wenjun Ruan fc72f8bae1
[Fix-11413] Cannot set task status to kill if the task is not in running(#11414) (#12007) 
(cherry picked from commit 496c2d4bfa)
 2022-09-17 13:55:20 +08:00
Wenjun Ruan 36135d4151
Clear thread name in BaseTaskProcessor (#11422) (#12008) 
(cherry picked from commit 47b50067ef)
 2022-09-17 11:30:57 +08:00
Kerwin d94797dfbc
Remove logger header in task log file (#11555) (#11968) 2022-09-16 13:03:22 +08:00
caishunfeng 43d9c00259
[Cherry-pick][Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 #11864 (#11949) 
Co-authored-by: Yann Ann <2993643785@qq.com>
 2022-09-15 10:36:34 +08:00
Jiajie Zhong 889fd580e3 [maven-release-plugin] prepare for next development iteration 2022-08-05 18:16:57 +08:00
Jiajie Zhong 9efd1ace78 [maven-release-plugin] prepare release 3.0.0 2022-08-05 18:16:57 +08:00
Wenjun Ruan 230adbf621
Fix TaskGroup cannot work and will cause master dead loop (#11254) (#11318) 
* Fix TaskGroup cannot work and will cause master dead loop

* Remove acquireTaskGroupAgain in ProcessServiceImpl

(cherry picked from commit acd3d3fab1)
 2022-08-05 17:40:22 +08:00
xiangzihao a05d7cc8f4 fix_10514 (#10568) 
(cherry picked from commit 0f38217b12)
 2022-08-03 19:41:14 +08:00
xiangzihao 5e129d478e add datasource health check to the healthcheck endpoint (#10588) 
(cherry picked from commit 1111371c9a)
 2022-08-03 19:41:14 +08:00
xiangzihao b0113a9b64 dependent node log output optimization and fix checkstyle (#10593) 
(cherry picked from commit 397e51fa63)
 2022-08-03 19:41:14 +08:00
Wenjun Ruan 072672a127 Fix recovery from failed task will dead loop (#11239) 
(cherry picked from commit 04f3aa9713)
 2022-08-02 19:07:08 +08:00
caishunfeng 034475bfbe [Bug-11101] fix task failover NPE (#11168) 
(cherry picked from commit 6c7e00c4aa)
 2022-07-28 11:14:33 +08:00
caishunfeng cec5e88319
[Bug-11071] Fix task local params NPE (#11166) 
* fix local param NPE

* delete empty files
 2022-07-28 09:43:58 +08:00
caishunfeng b9e3187ad5 [Feature-10871] add workflow executing data query (#10875) 
* add workflow executing data query
* fix sonar check for interrupted
 2022-07-19 16:10:42 +08:00
Wenjun Ruan 5a2ea0b76b Fix compile error 2022-07-19 12:30:26 +08:00
Wenjun Ruan 648d469b62 Return status 2022-07-19 12:27:49 +08:00
Wenjun Ruan f3250bf5fa Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979) 
* Use try-with-resource to close resource, and add heart error threshold to avoid worker cannot close due to heart beat check failed

* Move heartbeat error threshold to applicaiton.yml

(cherry picked from commit 2be1d4bf0a)
 2022-07-19 12:23:20 +08:00
Wenjun Ruan 17f0be5bfb Kill task when we do master failover to avoid task lost (#10997) 
(cherry picked from commit 998e4d74dd)
 2022-07-19 12:22:26 +08:00
Wenjun Ruan 44ddb6908e Fix kill yarn job error when failover caused by doesn't set ProcessDefinition (#10948) 
(cherry picked from commit b245e7c973)
 2022-07-19 12:22:13 +08:00
Wenjun Ruan 812d7a8f26 Fix taskInstance's host is not worker nettyServer address (#10926) 
* Fix taskInstance's host is not worker nettyServer address

* Remove unnecessary mock

(cherry picked from commit df0416c193)
 2022-07-19 12:21:59 +08:00
Wenjun Ruan 527ee472fb Catch exception when check state in StateWheelExecuteThread (#10908) 
* Catch exception when check state

(cherry picked from commit 2a67866718)
 2022-07-19 12:21:20 +08:00
Wenjun Ruan 9ee20cffef [Fix-10827] Fix network error cause worker cannot send message to master (#10886) 
* Fix network error cause worker cannot send message to master

(cherry picked from commit cade66a9b6)
 2022-07-19 12:20:18 +08:00
Wenjun Ruan b259deb196 [Fix-10854] Fix database restart may lost task instance status (#10866) 
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status

(cherry picked from commit f639a2eed4)
 2022-07-19 12:12:57 +08:00
Wenjun Ruan 71edaf41a2 [Fix-10842] Fix master/worker failover will cause status incorrect (#10839) 
* Fix master failover will not update task instance status
* Add some failover log
* Fix worker failover will rerun task more than once
* Fix workflowInstance failover may rerun already success taskInstance

(cherry picked from commit 3f69ec8f28)
 2022-07-19 12:08:32 +08:00
Wenjun Ruan 4fc9bce444 [Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event

(cherry picked from commit 67d14fb7b3)
 2022-07-19 12:03:15 +08:00
Wenjun Ruan 04c47034d4 Add task prepare metrics 2022-07-19 11:52:18 +08:00
Wenjun Ruan 7500e99682 [Fix-10785] Fix state event handle error will not retry (#10786) 
* Fix state event handle error will not retry

* Use state event handler to deal with the event

(cherry picked from commit 67d14fb7b3)
 2022-07-19 11:52:06 +08:00
Wenjun Ruan 3b923e5933 [Fix-10666] Workflow submit failed will still in memory and never retry (#10667) 
* Workflow submit failed will still in memory and never retry

(cherry picked from commit 35a10d092f)
 2022-07-19 11:47:04 +08:00
Wenjun Ruan 6c83967ebe [Improvement-10617] Add comment in slot check (#10618) 
(cherry picked from commit 247ca4ae8a)
 2022-07-19 11:42:31 +08:00
Wenjun Ruan 4b224ae2e5 Validate master/worker config (#10649) 
(cherry picked from commit 35b25da863)
 2022-07-19 11:42:17 +08:00
Wenjun Ruan db31deb54f [Bug] [Master] Worker failover will cause task cannot be failover (#10631) 
* fix worker failover may lose event

(cherry picked from commit 66624c5c86)
 2022-07-19 11:41:25 +08:00
Wenjun Ruan fc1c1f6ad1 add CMDPARAM_COMPLEMENT_DATA_SCHEDULE_DATE_LIST 2022-07-19 11:30:47 +08:00
Wenjun Ruan 3ab9ee13fc Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516) 
* Optimize master log, add workflow instance id and task instance id in log

* Use MDC to set the workflow info in log4j

* Add workflowInstanceId and taskInstanceId in MDC

(cherry picked from commit db595b3eff)
 2022-07-19 11:29:41 +08:00
Wenjun Ruan 9a28d32057 Remove the schedule thread in LowerWeightHostManager (#10310) 
(cherry picked from commit b100f6c489)
 2022-07-19 11:23:59 +08:00
Wenjun Ruan 90c87f0121 [Fix-10413] Fix Master startup failure the server still hang (#10500) 
* Fix Master startup failure the server still hang

(cherry picked from commit 117f78ec4b)
 2022-07-19 11:22:53 +08:00
Wenjun Ruan 9a59054655 Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check (#10491) 
(cherry picked from commit 0bdfa0cff9)
 2022-07-19 11:22:33 +08:00
Wenjun Ruan 9a4c7f876a Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479) 
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe

(cherry picked from commit ad2646ff1f)
 2022-07-19 11:22:16 +08:00