Commit Graph

333 Commits

Author SHA1 Message Date
Jay Chung 8a4f111fd4 [maven-release-plugin] prepare release 3.2.1 2024-02-07 11:09:56 +08:00
Jay Chung d5c543f467
Revert some incompatible change for 3.2.1 (#15580)
* Revert "Fix k8sTaskExecutionContext setting configYaml (#15116)"

This reverts commit ce11674668.

* Revert "[Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977)"

This reverts commit c532fea00e.
2024-02-07 10:41:04 +08:00
Jay Chung 7a1be3f329 [maven-release-plugin] prepare for next development iteration 2024-02-07 00:15:26 +08:00
Jay Chung ba5ec7375c [maven-release-plugin] prepare release 3.2.1 2024-02-07 00:15:25 +08:00
Jay Chung fa7b4c1c8f fix: start param for wf not work (#15544)
* fix: start param for wf not work

fix: #15280

* fix test

(cherry picked from commit 01eb8f834f)
2024-02-06 18:24:29 +08:00
caishunfeng ca93aa8e26 fix switch js (#15487)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit ef9ed3db55)
2024-02-06 18:22:55 +08:00
Jay Chung acb1e0a09f fix: data quality can not use (#15551)
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit 8efaa9fa1f)
2024-02-06 18:22:51 +08:00
Wenjun Ruan c8f120a60a Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time (#15528)
(cherry picked from commit 89742332a8)
2024-02-06 18:22:34 +08:00
Wenjun Ruan be58ba69a9 Exit JVM when OOM (#15538)
(cherry picked from commit 25810a81dc)
2024-02-06 18:21:56 +08:00
Jay Chung 898b001608
cp: Reduce the size of tarball to continue ASF release (#15004) (#15540)
* Reduce the size of tarball to continue ASF release

for more detail you can see https://lists.apache.org/thread/rmp7fghlj0n7h9y2v3p8gkw9f9qbo6qt
2024-02-06 18:15:57 +08:00
sleo 431414729f
【Master】【Bug】Fix a bug, When the worker service offline, workerNodeInfo cache in master cannot delete the offline worker (#15459)
* remove work from workerNodeInfo when work server down

* modify log msg

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* add unit test

* add unit test

* modify the code style

* add license header

---------

Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 18:10:36 +08:00
Wenjun Ruan 64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception (#15496)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Wenjun Ruan 44c356db4d
TaskGroupPriority only compare When TaskGroup is same (#15486) 2024-01-18 10:59:54 +08:00
Wenjun Ruan 603dfa2e4b
Fix notify failover WorkflowInstance will cause NPE (#15499) 2024-01-17 10:24:41 +08:00
Wenjun Ruan 7b9c9e0eb2
Enable set ServerLoadProtection fot Master/Worker (#15439) 2024-01-14 22:17:58 +08:00
Gallardot a405abec9e
[Bug][Master]serial_wait strategy workflow unable to wake up (#15270)
* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 17:53:24 +08:00
Gallardot 575b89e2f9
[Bug][Registry] Optimizing waiting strategy (#15223)
* [Improvement][Registry] Optimizing waiting strategy

Signed-off-by: Gallardot <gallardot@apache.org>
2024-01-02 22:41:50 +09:00
Wenjun Ruan d6dea4633c
Optimize server startup log (#15362) 2023-12-27 20:20:21 +08:00
Wenjun Ruan 5523a62825
Remove taskQueue and looper in worker (#15292) 2023-12-27 19:51:45 +08:00
Wenjun Ruan 5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355) 2023-12-23 22:47:59 +08:00
Gallardot f8a44ff719
[Bug][Master] send ACK event timeout (#15346) 2023-12-21 18:54:44 +08:00
ll 0c470ffe66
[Fix #15129] [Dependent] The date rules of the dependent node are ambiguous. (#15289)
* [Fix-15129][Dependent] Fix the ambiguity in date rules for dependent node.

* [fix #15129] Revert ddl

* restore findLastProcessInterval

* update: mvn spotless:apply

---------

Co-authored-by: 李乐 <lile@boimc.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-12-08 16:18:33 +08:00
Wenjun Ruan 14272dafab
Fix failover Master might not release taskGroup (#15287) 2023-12-06 13:31:46 +08:00
Wenjun Ruan 2119e41800
[Improvement] Move delay calculation to Master (#15278) 2023-12-06 09:37:14 +08:00
Wenjun Ruan 43f5f24529
Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00
caishunfeng b5eddc0ce8
fix switch condition (#15228)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-11-27 14:03:37 +08:00
Wenjun Ruan 1d23e4fe5d
Remove spring cache for dao (#15184) 2023-11-24 14:54:05 +08:00
Dyqer cca1c4a172
fix can't stop bug (#15191)
Co-authored-by: Leoric Yue <leoric.yue@zoom.us>
2023-11-20 21:02:44 +08:00
Wenjun Ruan d6175f3bbf
Exclude DataSourceAutoConfiguration in worker server (#15169) 2023-11-16 19:17:39 +08:00
旺阳 5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue (#14992)
* change data quality field name

* fix-dao

* fix-dao

* fix-dao

* fix-dao

* update

* test

* update

* update
2023-11-13 11:04:30 +08:00
fuchanghai 2622685784
[Bug] Fix endless loop (#15092)
* [fix-#13045] fix endless loop

---------

Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-11-08 11:03:55 +08:00
Eric Gao d675d32771
Fix confusing constant string for unit convertor (#15126) 2023-11-06 21:32:36 +08:00
Wei Xiaonan c0ed68121d
[Feature-14832][Listener]Implementation of Listener Mechanism (#14981)
* first commit

* 1. sql: sync ddl
2. front-end: change to ternary expression
3. back-end: correct license header in ListenerEvent.java

* test case

* frontend: remove unnecessary console

* fix unit test

* remove log depends on user-provided value

* fix dolphinscheduler_postgresql.sql

* sync database schema

* fix unit test

* fix unit test

* fix some NIT.

* extract GLOBAL_ALERT_GROUP_ID into variable

* fix ddl bug

* add column task_type in t_ds_fav_task in upgrade/3.2.0_schema

* add unit test
2023-10-29 20:28:05 +08:00
Wenjun Ruan a5284e4024
Remove mapper usage in tools (#15073) 2023-10-25 22:58:47 +08:00
Orange Summer 688f84452d
[Feature-14678][Master][UI]Dependent task parameter passing (#14702)
* feat: dependent task parameter passing

* style: code format

* refactor: remove inappropriate log

* refactor: Modify unreasonable code structure

* refactor: Modify unreasonable code structure

* refactor: Solve conflicts caused by merge

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: BaoLiang <29528966+lenboo@users.noreply.github.com>
2023-10-25 16:24:30 +08:00
caishunfeng c59b2d5b8c
remove sub workflow finish notify (#15057)
Co-authored-by: xiangzihao <460888207@qq.com>
2023-10-23 10:19:57 +08:00
Wenjun Ruan 33084d281f
Add dolphinscheduler-dao-plugin module (#15019) 2023-10-14 17:58:40 +08:00
dbac d683553324
[Improvement][Master] Calculate the remainTime then we set the delay execution. (#15012)
* [#15007][fix][worker-server] fix bug task delay execution

* [#15007][fix][worker-server] fix bug task delay execution
    mvn spotless:apply

* [#15007][fix][worker-server] fix bug task delay execution
    worker fix  task delay logic

---------

Co-authored-by: 旺阳 <wang@lqwang.net>
2023-10-11 02:32:07 -05:00
Wenjun Ruan 629fced9ef
Fix task log appender will not be closed immediate after throwing exception (#14989) 2023-10-07 20:19:02 -05:00
Aaron Wang 73e846d03e
[Improvement-14884][Master] Add overload state in master heartbeat to trigger slot change (#14887) 2023-09-14 10:36:42 +08:00
sgw 76bbcbeb30
[Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent (#14824)
* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- delete useless output

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- add log
- run spotless check

---------

Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
2023-09-09 17:23:34 +08:00
eye-gu 7ae4fb3787
[Fix-14149][master] task finished so not dispatch (#14161) 2023-08-31 20:08:09 +08:00
eye-gu 33525a17c3
[Fix-14828][master] check task id not null to avoid NPE (#14829)
Co-authored-by: eye <eye.gu@aloudata.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-08-31 18:34:31 +08:00
Wenjun Ruan 742f0c3e60
Add workflowInstanceHost and taskInstanceHost in ITaskInstanceExecutionEvent (#14827) 2023-08-29 13:32:17 +08:00
Wenjun Ruan d223d654cc
Add rpc benchmark test (#14797) 2023-08-27 20:59:55 +08:00
sgw 9c2838ccc9
fix the status of logical task is not be correctly changed when task is finished. (#14820) (#14822) 2023-08-27 13:20:37 +08:00
sgw ac04288e2b
fix the full log path of logical task is not be set when task is dispatched (#14819) (#14821) 2023-08-27 12:24:04 +08:00
Wenjun Ruan 55cc77226f
[RPC] Refactor the remote module. (#14776)
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
caishunfeng 59998f4c75
[Improvement] add dispatch fail times when compare (#14761)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2023-08-18 15:26:01 +08:00
JieguangZhou 8e64026e93
fix fault tolerant enviroment missing (#14735) 2023-08-11 14:35:31 +08:00