Commit Graph

8416 Commits

Author SHA1 Message Date
Rick Cheng 718a01bb42
[Improvement][UT] Add UT for end time check when force success task (#15562) 2024-02-08 09:19:56 +08:00
Wenjun Ruan 3526b42df6
Add Server HealthIndicator (#15274) 2024-02-07 14:07:30 +08:00
Jay Chung bd83631955
fix: Resource relate path invalid when tenant change (#15581)
* fix can not get correct resource related path when
  user run workflow with differnet tenant of resource
  created
* also fix can not get correct related path when we
  use `resource.storage.type=LOCAL`
2024-02-06 23:56:03 +08:00
seckiller 89da67ddeb
[fix][worker]:Fix the issue of missing disk usage reporting information in the worker (#15565)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-06 16:43:29 +08:00
Wenjun Ruan 1b32e28cc8
Set the workflow instance ready state to running in failover (#15572) 2024-02-06 15:25:50 +08:00
旺阳 73a5a77db2
[Fix] Fix WorkflowInstance batch start failed will throw incorrect exception. (#15577) 2024-02-06 13:16:25 +08:00
Wenjun Ruan 4541e74e53
Fix create parent directory will cause FileAlreadyExistsException (#15576)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-06 11:59:07 +08:00
John Huang 69676b445c
[Improvement][UT] Improve Worker registry coverage (#15380)
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-06 10:45:53 +08:00
Wenjun Ruan 43a06525a2
Fix Recover WorkflowInstance will casue workflow Instance state is success but task insatnce is killed/paused (#15574) 2024-02-06 10:09:30 +08:00
Jay Chung 91d56f4860
fix: data quality may fail in docker mode (#15563) 2024-02-05 17:14:59 +08:00
Jay Chung 01eb8f834f
fix: start param for wf not work (#15544)
* fix: start param for wf not work

fix: #15280

* fix test
2024-02-05 16:17:30 +08:00
小可耐 e5a208f363
[Bug][force-success] force success add end time (#15144)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-05 14:23:16 +08:00
Jay Chung d8e820c4b9
fix: ddl without drop exists (#14128)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-05 12:20:07 +08:00
dada Sun 4d6af516b9
[Bug][Task Api] fix 'MACPATTERN' in ProcessUtils and cover all cases on MacOS in ProcessUtilsTest (#15480) (#15529)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-05 11:41:21 +08:00
caishunfeng ef9ed3db55
fix switch js (#15487)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 10:23:47 +08:00
Jay Chung 8efaa9fa1f
fix: data quality can not use (#15551)
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
Wenjun Ruan 4d15932ede
Fix createFile with permission will not work (#15556) 2024-02-03 18:00:33 +08:00
John Huang 1b42d45fcd
[Doc][Docker] fix typo on start with docker (#15534)
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-02 14:17:11 +08:00
Wenjun Ruan 89742332a8
Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time (#15528) 2024-02-01 17:13:58 +08:00
John Huang 2f66a667ba
refactor comments & function name for confuse (#15546)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-02-01 10:39:54 +08:00
Wenjun Ruan 25810a81dc
Exit JVM when OOM (#15538) 2024-01-31 13:59:30 +08:00
Gallardot 3640a28348
[Improvement][HTTP] support custom rendering of http body (#15531)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-01-31 11:02:48 +08:00
Wenjun Ruan 86ef9666c4
Fix exception occur in RpcServer side, it will not be sent to RpcClient (#15536) 2024-01-30 17:54:29 +08:00
clovelll edbf5cd3af
front: When you edit a task in the task definition list, the front task list is displayed (#12819)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-29 13:40:21 +08:00
BaiJv e20e066076
[Fix] [Bug] Change default version of Workflow/TaskDefinition to `1` (#15498) 2024-01-26 20:51:17 +08:00
Wenjun Ruan 419cb49311
Change download url in backend.yml (#15526) 2024-01-26 15:07:05 +08:00
sleo 431414729f
【Master】【Bug】Fix a bug, When the worker service offline, workerNodeInfo cache in master cannot delete the offline worker (#15459)
* remove work from workerNodeInfo when work server down

* modify log msg

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* add unit test

* add unit test

* modify the code style

* add license header

---------

Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 18:10:36 +08:00
tangjiaolong feb302335b
修复工作流导入时update_time时间一样,导致分页查询存在乱序问题 (#14810)
Co-authored-by: David Zollo <dailidong66@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 17:36:12 +08:00
zhihuasu d80a29c48c
[BUG] #15013 Fix retryInterval in RetryPolicy will never be used in RetryUtils (#15014)
* Update RetryUtils.java

assigned

* UT

* Update RetryUtilsTest.java

---------

Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 15:07:11 +08:00
John Huang 8abb5e55b7
[Doc][K8S] Add DS K8S Operator into k8s deployment character (#15516)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-24 11:10:58 +08:00
sunsongyang 95ae0036f6
[improvement][api] Fix typo for controllers (#15438)
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-24 10:39:35 +08:00
Wenjun Ruan 5a7deb28f7
Throw IllegalArgumentException if parse time placeholder error (#15514)
(cherry picked from commit 5fa31e036d77698f3732a42664bfac3e7edcb8e4)
2024-01-22 10:21:54 +08:00
Wenjun Ruan 308e4fb4a6
Directly Throw exception when taskInstancy log path is empty which log need to be queried (#15511) 2024-01-20 16:40:30 +08:00
John Huang e00121b44b
Add guideline link into DolphinScheduler mail list (#15447)
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-19 15:33:53 +08:00
Wenjun Ruan dcf69cad20
Fix PostgresqlDatabaseContainerProvider get Image is incorrect (#15434) 2024-01-19 10:29:46 +08:00
Wenjun Ruan 0dc373df2a
Set TaskGroupQueue updateTime when force start (#15510) 2024-01-18 22:11:50 +08:00
silenceland 16c97ef44d
[DS-15489][style]rename the vo object suffix (#15504)
- the naming suffix of the objects under the package dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo is not unified, such as AlertPluginInstanceVO and ScheduleVO, etc.
- this issue solve it, and uniformly modified to the VO suffix

Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 17:13:53 +08:00
Wenjun Ruan 64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception (#15496)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Wenjun Ruan 44c356db4d
TaskGroupPriority only compare When TaskGroup is same (#15486) 2024-01-18 10:59:54 +08:00
sunsongyang e1fcd4e485
[Bug][Api] Fix NPE when deleting a workflow instance (#15485) (#15488) 2024-01-17 22:03:33 +08:00
Gallardot c7a11ce6ed
[Improvement][K8S]Optimize MDC for K8S tasks (#15390)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-17 17:56:11 +08:00
Rick Cheng e4de06b5af
[Doc-15500][Task] Update cli opts of spark and flink (#15501) 2024-01-17 14:22:27 +08:00
Wenjun Ruan 603dfa2e4b
Fix notify failover WorkflowInstance will cause NPE (#15499) 2024-01-17 10:24:41 +08:00
aiwenmo ace0860d3d
[Feature-15475][DinkyTask] DinkyTask supports Dinky-1.0.0 and common sql (#15479) 2024-01-16 22:47:22 +08:00
Wenjun Ruan 0b873e6870
[HotFix] Fix createTaskInstanceWorkingDirectory failed if the old path exist (#15377) 2024-01-15 11:53:55 +08:00
AliceXiaoLu cb5317f7ca
[bug] Exception when using host in ipv6 format (#14040)
* DB2 edit again, no username.

* [bug] db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"

* Update Db2DataSourceProcessor.java

* Update Db2DataSourceProcessorTest.java

* 【Bug】OracleSource edit again, ConnectionType is null

* when user inputs ipv6,checkhost will return "!IPV4_PATTERN.matcher(host).matches()";
I think we should use isInetAddress to check the host is isValid

* Update AbstractDataSourceProcessor.java

* Update DatabendDataSourceProcessorTest.java

last week other modified this

* Update MySQLDataSourceProcessorTest.java

* Update DataSourceUtilsTest.java

* Update MySQLDataSourceProcessorTest.java

* Update KyuubiDataSourceProcessorTest.java

* Update AbstractDataSourceProcessor.java

if host is ipv6 or ipv4 will OK; if not check other

* Update AbstractDataSourceProcessor.java

* Update AbstractDataSourceProcessor.java

* Update DataSourceUtilsTest.java

增加ipv6数据源验证

* [Bug] [dolphinscheduler-api] queryDataSource'password should be hidden.

---------

Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-15 09:33:08 +08:00
Wenjun Ruan 7b9c9e0eb2
Enable set ServerLoadProtection fot Master/Worker (#15439) 2024-01-14 22:17:58 +08:00
Gallardot a405abec9e
[Bug][Master]serial_wait strategy workflow unable to wake up (#15270)
* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 17:53:24 +08:00
lizhenglei 3c7a77c90e
[Improvement][E2E] add e2e javatask case (#15469)
* add java task

* fix

* support jdk

* fix java

* fix  java_home

* add ut

* recover

---------

Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2024-01-12 17:20:40 +08:00
lizhenglei bd48c99178
[BUG][Task] fix java task classpath (#15470)
* fix javatask bug

* cover

---------

Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 14:39:59 +08:00