Commit Graph

8362 Commits

Author SHA1 Message Date
sleo 26f2aadb06
Rename variable QUEUE_MAX_SIZE to QUEUE_INITIAL_SIZE (#16040) 2024-05-24 20:09:10 +08:00
Wenjun Ruan f754611509
[DSIP-42] Add dolphinscheduler-aws-authentication module (#16043) 2024-05-24 14:15:03 +08:00
yinxiaolog baa252a0ed
[DS-16046][fix] Set PreparedStatement parameter type (#16050)
Fix the PreparedStatement parameter is TIME, set it to java.sql.Time

Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-24 13:12:16 +08:00
sleo 581a7edad8
modify log msg (#16062)
Co-authored-by: xiangzihao <460888207@qq.com>
2024-05-24 11:28:49 +08:00
Wenjun Ruan 5037f0a5e3
Remove OmitStackTraceInFastThrow in start.sh (#16054) 2024-05-24 10:34:10 +08:00
Wenjun Ruan 6862fe7548
Removed unused StateEventHandleFailure (#16052) 2024-05-23 11:35:46 +08:00
Wenjun Ruan 56ecec779c
Fix NettyRemotingClient might throw IllegalMonitorStateException (#16038) 2024-05-22 10:26:04 +08:00
小可耐 0fc2fe971a
[DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type (#15967)
* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* improvement project code

* remove useless imports

* remove method onClearSearchTaskType

* add parameter doc

* optimisation logic

* code conflict resolution

* code conflict resolution

* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978)

* update

* test

* add monitor enhance ui

* update

* update

* update doc

* fix spotless

* update

* update

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* update

* fix spotless

* update

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* [Improvement][Monitor] Add UT for montor (#15998)

* formatting Code
2024-05-21 15:39:43 +08:00
TianXinCoord cf400b86ba
Correction of Typos in the Chinese Document Appendix for Task Parameters (#16033)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-21 14:00:52 +08:00
Rick Cheng dbd790ddac
[Fix][CI] fix the ci error of Values.datasource.profile (#16031)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-05-21 11:16:30 +08:00
Wenjun Ruan e4df01a033
Increase block-until-connected in ZookeeperRegistryTestCase (#16041) 2024-05-21 10:32:15 +08:00
Wenjun Ruan b184364561
Fix WorkerTaskExecutorThreadPool#isOverload is incorrect (#16027) 2024-05-20 19:13:53 +08:00
Wenjun Ruan 5463d0255a
Fix rpc channel leak due to concurrent operation (#16021)
* Fix rpc channel leak due to concurrent operation

* Throw channel create failed exception

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-20 12:10:33 +08:00
jiabei 7b7a33337f
docs: fix type in contribute.md (#16022)
Co-authored-by: jxbwhe <iswanghe@aliyun.com>
2024-05-18 21:34:08 +08:00
Wenjun Ruan cb0b4fa52f
Deprecated 2.0.9/3.0.6 upgrade test (#16012) 2024-05-17 19:17:54 +08:00
Wenjun Ruan 4e73fc28b0
Use random port in registry IT (#16014) 2024-05-17 17:35:44 +08:00
JohnHuang 79f90a87f8
[Feature][Helm] make api/master/worker/alert application.yaml configurable (#15922) 2024-05-17 15:23:21 +08:00
Wenjun Ruan 61915a2d5d
Fix workflow instance restart failed due to duplicate key in varpool (#16001) 2024-05-16 11:32:27 +08:00
旺阳 68a190674f
[Improvement][Monitor] Add UT for montor (#15998) 2024-05-15 14:44:04 +08:00
旺阳 0e5cb664bb
[Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978)
* update

* test

* add monitor enhance ui

* update

* update

* update doc

* fix spotless

* update

* update

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* update

* fix spotless

* update

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2024-05-15 13:58:59 +08:00
Gee 4f4a13da02
[Fix-15992][server] maintain slot index by iterate MasterPriorityQueue orderly (#15993) 2024-05-15 11:44:27 +08:00
Wenjun Ruan 1a4537ecbd
Fix database schema change check failed with pg in ci (#15995)
Co-authored-by: xiangzihao <460888207@qq.com>
2024-05-15 10:36:27 +08:00
John Bampton 8d6e9eecfc
docs: fix spelling (#15996) 2024-05-15 09:41:02 +08:00
Eric Gao dc306bfa1d
[Improvement] Use safe constructor with snake yaml (#15758) 2024-05-14 18:04:04 +08:00
licc6 680d2e0b05
Repair resource configuration exist typo (#15988)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-05-13 19:47:47 +08:00
Wenjun Ruan 7c8fa9b48c
Add IntegretionTest for registry module (#15981) 2024-05-13 16:35:37 +08:00
Wenjun Ruan 3446fd8ab1
EMR task support replace params placeholder (#15975)
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-05-10 17:34:26 +08:00
Gallardot ace20f96c2
[Bug] [Helm] No DB Nodes Exist (#15970) 2024-05-10 15:43:00 +08:00
JohnHuang ad1a6af4fb
Add link to ETCD/JDBC Registry Guideline (#15597) 2024-05-09 16:52:37 +08:00
cntiger 60b019b729
[Improvement] Fix the git url command injection in pytorch task(#15873) (#15950)
* fix the git url command injection danger(#15873)

* [Improvement] Fix the git url command injection in pytorch,format code style task(#15873)

---------

Co-authored-by: cntigers <Xiaohu4321@>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 14:50:27 +08:00
Zzih96 5c569b705c
[fix-15907] Fix get remote shell exit code is incorrect (#15911)
* ./mvnw spotless:apply

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-remoteshell/src/main/java/org/apache/dolphinscheduler/plugin/task/remoteshell/RemoteExecutor.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

---------

Co-authored-by: 詹子恒 <ziheng.zhan@longbridge.sg>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 14:15:06 +08:00
刘阳阳(liuyangyang) ba5de75829
Add tenantCode propagation to DynamicCommandUtils.createCommand (#15956) 2024-05-09 13:06:22 +08:00
Wenjun Ruan 8d336def61
[DSIP-35][Alert] Refactor the alert thread model (#15932) 2024-05-09 12:23:01 +08:00
privking bbca37d03e
[FIX] Completed tasks cannot be re-executed in a workflow instance (#15884)
* fix bug: Failed to resume stopped workflow instance

* Revert "fix bug: Failed to resume stopped workflow instance"

This reverts commit 1546e9d5a5.

* fix bug : Completed tasks cannot be re-executed in a workflow instance

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-09 11:36:56 +08:00
xiangzihao 8426d2346c
[HotFix] [CI] Temporary skipping mergeable check (#15958)
* temporary skipping mergeable check
2024-05-09 11:01:05 +08:00
Wenjun Ruan 99d8276be7
Optimizing the scope of RPC base classes (#15946)
* Optimizing the scope of RPC base classes

* Fix UT
2024-05-08 11:46:09 +08:00
JohnHuang d218b021e7
[DSIP-25][Remote Logging] Split remote logging configuration (#15826)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-05-06 11:33:16 +08:00
Evan Sun fa6ea8ba7b
[TEST] increase coverage of project workergroup relation service (#15944)
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
2024-05-06 10:02:04 +08:00
Evan Sun ebcdaeb9ac
[TEST] increase coverage of project preference service test (#15939) 2024-04-29 21:03:01 +08:00
Wenjun Ruan 0a11cd21bd
Fix jar path is not correct in java task (#15906) 2024-04-29 18:07:05 +08:00
Wenjun Ruan 647cbae400
[DSIP-32][Master] Add command fetcher strategy for master fetch command (#15900) 2024-04-29 16:14:23 +08:00
calvin 5a6c6c37f0
[Improvement-15910][UI] Supposed to provide a default value for the custom parallelism when using the mode of parallel execution. (#15912)
* worked out the issue

* imrpove the parallism strategy

* imrpove the parallism strategy

* merge from dev
2024-04-28 18:47:20 +08:00
小可耐 7a55adeae9
[Improvement-15919][datasource] Improvement datasource get name (#15920) 2024-04-28 11:02:44 +08:00
DaqianLiao b29965bdce
[Fix] In updateWorkerNodes method, the workerNodeInfoWriteLock should be used. #15898 (#15903)
Co-authored-by: answerliao <answerliao@tencent.com>
2024-04-26 11:24:05 +08:00
Wenjun Ruan d1135eabc7
[DSIP-34] Change required_approving_review_count to 2 (#15914) 2024-04-25 19:39:56 +08:00
Wenjun Ruan 446f6ba72b
Fix auto create tennat concurrently will cause the task failed (#15909) 2024-04-25 18:14:48 +08:00
Wenjun Ruan b3b8c0784d
Fix kill dynamic task doesn't kill the wait to run workflow instances (#15896) 2024-04-25 11:05:51 +08:00
Evan Sun 1d13ef09c3
[TEST] increase coverage of project parameter service test (#15905)
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
2024-04-24 23:31:56 +08:00
旺阳 59f060e278
[Improvement] Fix alert code smell 2024-04-23 15:10:08 +08:00
Wenjun Ruan e9d85914d7
Fix queryByTypeAndJobId might error due to multiple result (#15883) 2024-04-22 17:41:17 +08:00