dff98f0492
mvn spotless:apply
2024-06-08 23:24:51 +08:00
063d0ad81d
move sparkParameters from spark to api
2024-06-08 23:23:42 +08:00
d5df5ccda5
try to fix worker E2E-K8S-EXECUTE
2024-06-08 22:54:11 +08:00
19a73c8630
add registry-all to alert、api、master、worker pom
2024-06-08 22:02:15 +08:00
e6245b0999
remove registry pure
2024-06-08 21:39:36 +08:00
3ad53c5200
3.2.2 release add task-pure delete some task module to reduce the size of the deployment package ( #16094 )
...
* add task-pure delete some task module to reduce the size of the deployment package
* add version of 3.2.2
* Update Chart.yaml Rollback APPversion
* remove task pure plugin
* fix ci
* add pure datasource plugin
* remove ck，sr from datasource pure
* remove task plugin chunjun ，dms ，datafactory from pure
* add api pom dependency
* spotless:apply
* add dependency license
* [maven-release-plugin] prepare release 3.2.2
* [maven-release-plugin] prepare for next development iteration
* remove datasource-pure：
dolphinscheduler-datasource-sqlserver
dolphinscheduler-datasource-oracle
remove task-pure：
dolphinscheduler-task-dinky
dolphinscheduler-task-pytorch
dolphinscheduler-task-datasync
dolphinscheduler-task-kubeflow
dolphinscheduler-task-emr
dolphinscheduler-task-pigeon
dolphinscheduler-task-flink-stream
* add register pure remove jdbc and etcd
* add dolphinscheduler-storage-pure remove task of
dolphinscheduler-storage-obs
dolphinscheduler-storage-abs
dolphinscheduler-storage-gcs
dolphinscheduler-storage-oss
* mvn spotless:apply
---------
Co-authored-by: wangxj959 <wangxj959@chinaunicom.cn>
Co-authored-by: wangxj <wangxj31>
Co-authored-by: wangxj3 <wangxj@apache.org>
2024-06-08 17:05:20 +08:00
31e79d10ea
Fix task nodes under switch task will not be skipped ( #16109 )
...
(cherry picked from commit f687a7d851c206ae30377ac3f9dc025cec35ffb2)
2024-06-05 23:41:52 +08:00
50c434e6c4
Fix task instance which status is dispatch will not be failover ( #16110 )
...
(cherry picked from commit a59ecf2aa19f4acd8d9981d14abe4d0c0d8e7fec)
2024-06-05 13:31:29 +08:00
fd864bf351
Revert "3.2.2 release cherry-pick cp: Reduce the size of tarball to continue …" ( #16068 )
...
This reverts commit 646bbf4881
2024-05-25 10:50:37 +08:00
646bbf4881
3.2.2 release cherry-pick cp: Reduce the size of tarball to continue ASF release ( #16042 )
...
* add version of 3.2.2
* Fix rpc channel leak due to concurrent operation (#16021 )
* Fix rpc channel leak due to concurrent operation
* Throw channel create failed exception
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
* Fix WorkerTaskExecutorThreadPool#isOverload is incorrect (#16027 )
* cp: Reduce the size of tarball to continue ASF release (#15004 ) (#15540 )
* Reduce the size of tarball to continue ASF release
for more detail you can see https://lists.apache.org/thread/rmp7fghlj0n7h9y2v3p8gkw9f9qbo6qt
# Conflicts:
# tools/dependencies/known-dependencies.txt
* Increase block-until-connected in ZookeeperRegistryTestCase (#16041 )
* [Fix][CI] fix the ci error of Values.datasource.profile (#16031 )
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
* Correction of Typos in the Chinese Document Appendix for Task Parameters (#16033 )
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
* [DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type (#15967 )
* [Improvement][parameter] New data types and type filtering
* [Improvement][parameter] Improvement startup parameters/global parameters data type
* fix api interfaces compatible
* add project parameter data type default value
* [Improvement][parameter] New data types and type filtering
* [Improvement][parameter] Improvement startup parameters/global parameters data type
* fix api interfaces compatible
* add project parameter data type default value
* improvement project code
* remove useless imports
* remove method onClearSearchTaskType
* add parameter doc
* optimisation logic
* code conflict resolution
* code conflict resolution
* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978 )
* update
* test
* add monitor enhance ui
* update
* update
* update doc
* fix spotless
* update
* update
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* update
* fix spotless
* update
---------
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* [Improvement][Monitor] Add UT for montor (#15998 )
* formatting Code
* add pom
* Fix NettyRemotingClient might throw IllegalMonitorStateException (#16038 )
* Removed unused StateEventHandleFailure (#16052 )
* Remove OmitStackTraceInFastThrow in start.sh (#16054 )
* Update Chart.yaml Rollback APPversion
* modify log msg (#16062 )
Co-authored-by: xiangzihao <460888207@qq.com>
* [DS-16046][fix] Set PreparedStatement parameter type (#16050 )
Fix the PreparedStatement parameter is TIME, set it to java.sql.Time
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
* Update pom.xml
* [DSIP-42] Add dolphinscheduler-aws-authentication module (#16043 )
---------
Co-authored-by: wangxj <wangxj31>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: TianXinCoord <tianxincoord@163.com>
Co-authored-by: 小可耐 <46134044+sdhzwc@users.noreply.github.com>
Co-authored-by: sleo <97011595+alei1206@users.noreply.github.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: yinxiaolog <43245392+yinxiaolog@users.noreply.github.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2024-05-25 10:16:18 +08:00
61915a2d5d
Fix workflow instance restart failed due to duplicate key in varpool ( #16001 )
2024-05-16 11:32:27 +08:00
ba5de75829
Add tenantCode propagation to DynamicCommandUtils.createCommand ( #15956 )
2024-05-09 13:06:22 +08:00
bbca37d03e
[FIX] Completed tasks cannot be re-executed in a workflow instance ( #15884 )
...
* fix bug: Failed to resume stopped workflow instance
* Revert "fix bug: Failed to resume stopped workflow instance"
This reverts commit 1546e9d5a5
2024-05-09 11:36:56 +08:00
99d8276be7
Optimizing the scope of RPC base classes ( #15946 )
...
* Optimizing the scope of RPC base classes
* Fix UT
2024-05-08 11:46:09 +08:00
d218b021e7
[DSIP-25][Remote Logging] Split remote logging configuration ( #15826 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-06 11:33:16 +08:00
647cbae400
[DSIP-32][Master] Add command fetcher strategy for master fetch command ( #15900 )
2024-04-29 16:14:23 +08:00
b29965bdce
[Fix] In updateWorkerNodes method, the workerNodeInfoWriteLock should be used. #15898 ( #15903 )
...
Co-authored-by: answerliao <answerliao@tencent.com>
2024-04-26 11:24:05 +08:00
b3b8c0784d
Fix kill dynamic task doesn't kill the wait to run workflow instances ( #15896 )
2024-04-25 11:05:51 +08:00
e2c8b080f9
[DSIP-31] Reduce the thread pool size of hikari ( #15890 )
2024-04-22 17:05:24 +08:00
285c5a8eb5
[DSIP-28] Donnot scan whole bean under classpath ( #15874 )
2024-04-19 18:12:40 +08:00
163f8f01f0
Fix jdbc registry cannot work ( #15861 )
2024-04-18 15:43:28 +08:00
883848f6ee
Remove unused caffeine cache ( #15830 )
2024-04-11 19:28:00 +08:00
66df5d4b90
Split cpuUsage to systemCpuUsage and jvmCpuUsage ( #15803 )
2024-04-09 14:04:24 +08:00
c0435e5bd8
fix dependent task logic ( #15795 )
...
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
2024-04-03 13:56:41 +08:00
dc4dad135c
Fix TaskGroupCoordinator might cause OOM when there is a lot of waiting TaskGroupQueue ( #15773 )
2024-03-30 09:43:42 +08:00
dcc9d64ef6
Fix Switch logic task doesn't check the branch exist ( #15755 )
2024-03-24 20:52:00 +08:00
2e169076dc
Fix delay execution task will never be dispatched ( #15751 )
...
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-03-22 16:23:55 +08:00
99a2461785
[Improvement] change js engine ( #15747 )
...
* change js engine
* add dependent jar
2024-03-22 11:31:47 +08:00
4a255fd523
Fix switch js, add black key ( #15680 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-03-18 20:24:54 +08:00
eab71f1071
[Improvement-15707][Master] Work out the issue that the workflow with a task dependency couldn't work well. ( #15712 )
2024-03-14 17:57:13 +08:00
738da1cda3
[Fix][Master] Fix Potential danger in the event of a worker failover ( #15689 )
...
* clean unused import
* fix style check
* fix when path is null or empty, it will cause serverhost is null,
* fix UT test (#15684 )
* [Fix-15639] parameterPassing is null case NPE (#15678 )
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* fix when path is null or empty, it will cause serverhost is null,
2024-03-11 21:50:21 +08:00
04a6b0d281
Fix task mighe be dispatched even if it has been killed ( #15662 )
2024-03-04 17:30:43 +08:00
2b4c1e06b6
Fix recover pause workflowInstance may generate failover taskInstance ( #15611 )
2024-02-22 15:57:08 +08:00
8a35e8b2f8
Fix switch task will throw exception when there is not parameters ( #15594 )
2024-02-19 11:12:22 +08:00
3526b42df6
Add Server HealthIndicator ( #15274 )
2024-02-07 14:07:30 +08:00
01eb8f834f
fix: start param for wf not work ( #15544 )
...
* fix: start param for wf not work
fix : #15280
* fix test
2024-02-05 16:17:30 +08:00
ef9ed3db55
fix switch js ( #15487 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 10:23:47 +08:00
8efaa9fa1f
fix: data quality can not use ( #15551 )
...
* fix: data quality can not use
fix : #15468 , #15249 , #14858
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
89742332a8
Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time ( #15528 )
2024-02-01 17:13:58 +08:00
25810a81dc
Exit JVM when OOM ( #15538 )
2024-01-31 13:59:30 +08:00
431414729f
【Master】【Bug】Fix a bug, When the worker service offline, workerNodeInfo cache in master cannot delete the offline worker ( #15459 )
...
* remove work from workerNodeInfo when work server down
* modify log msg
Co-authored-by: xiangzihao <zihaoxiang@apache.org>
* add unit test
* add unit test
* modify the code style
* add license header
---------
Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 18:10:36 +08:00
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception ( #15496 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
44c356db4d
TaskGroupPriority only compare When TaskGroup is same ( #15486 )
2024-01-18 10:59:54 +08:00
603dfa2e4b
Fix notify failover WorkflowInstance will cause NPE ( #15499 )
2024-01-17 10:24:41 +08:00
7b9c9e0eb2
Enable set ServerLoadProtection fot Master/Worker ( #15439 )
2024-01-14 22:17:58 +08:00
a405abec9e
[Bug][Master]serial_wait strategy workflow unable to wake up ( #15270 )
...
* fix: serial_wait strategy workflow unable to wake up
Signed-off-by: Gallardot <gallardot@apache.org>
* fix: serial_wait strategy workflow unable to wake up
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 17:53:24 +08:00
575b89e2f9
[Bug][Registry] Optimizing waiting strategy ( #15223 )
...
* [Improvement][Registry] Optimizing waiting strategy
Signed-off-by: Gallardot <gallardot@apache.org>
2024-01-02 22:41:50 +09:00
d6dea4633c
Optimize server startup log ( #15362 )
2023-12-27 20:20:21 +08:00
5523a62825
Remove taskQueue and looper in worker ( #15292 )
2023-12-27 19:51:45 +08:00
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) ( #15355 )
2023-12-23 22:47:59 +08:00
wangxj959