Commit Graph

8381 Commits

Author SHA1 Message Date
Wenjun Ruan 0dc373df2a
Set TaskGroupQueue updateTime when force start (#15510) 2024-01-18 22:11:50 +08:00
silenceland 16c97ef44d
[DS-15489][style]rename the vo object suffix (#15504)
- the naming suffix of the objects under the package dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo is not unified, such as AlertPluginInstanceVO and ScheduleVO, etc.
- this issue solve it, and uniformly modified to the VO suffix

Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 17:13:53 +08:00
Wenjun Ruan 64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception (#15496)
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
Wenjun Ruan 44c356db4d
TaskGroupPriority only compare When TaskGroup is same (#15486) 2024-01-18 10:59:54 +08:00
sunsongyang e1fcd4e485
[Bug][Api] Fix NPE when deleting a workflow instance (#15485) (#15488) 2024-01-17 22:03:33 +08:00
Gallardot c7a11ce6ed
[Improvement][K8S]Optimize MDC for K8S tasks (#15390)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-17 17:56:11 +08:00
Rick Cheng e4de06b5af
[Doc-15500][Task] Update cli opts of spark and flink (#15501) 2024-01-17 14:22:27 +08:00
Wenjun Ruan 603dfa2e4b
Fix notify failover WorkflowInstance will cause NPE (#15499) 2024-01-17 10:24:41 +08:00
aiwenmo ace0860d3d
[Feature-15475][DinkyTask] DinkyTask supports Dinky-1.0.0 and common sql (#15479) 2024-01-16 22:47:22 +08:00
Wenjun Ruan 0b873e6870
[HotFix] Fix createTaskInstanceWorkingDirectory failed if the old path exist (#15377) 2024-01-15 11:53:55 +08:00
AliceXiaoLu cb5317f7ca
[bug] Exception when using host in ipv6 format (#14040)
* DB2 edit again, no username.

* [bug] db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"

* Update Db2DataSourceProcessor.java

* Update Db2DataSourceProcessorTest.java

* 【Bug】OracleSource edit again, ConnectionType is null

* when user inputs ipv6,checkhost will return "!IPV4_PATTERN.matcher(host).matches()";
I think we should use isInetAddress to check the host is isValid

* Update AbstractDataSourceProcessor.java

* Update DatabendDataSourceProcessorTest.java

last week other modified this

* Update MySQLDataSourceProcessorTest.java

* Update DataSourceUtilsTest.java

* Update MySQLDataSourceProcessorTest.java

* Update KyuubiDataSourceProcessorTest.java

* Update AbstractDataSourceProcessor.java

if host is ipv6 or ipv4 will OK; if not check other

* Update AbstractDataSourceProcessor.java

* Update AbstractDataSourceProcessor.java

* Update DataSourceUtilsTest.java

增加ipv6数据源验证

* [Bug] [dolphinscheduler-api] queryDataSource'password should be hidden.

---------

Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-15 09:33:08 +08:00
Wenjun Ruan 7b9c9e0eb2
Enable set ServerLoadProtection fot Master/Worker (#15439) 2024-01-14 22:17:58 +08:00
Gallardot a405abec9e
[Bug][Master]serial_wait strategy workflow unable to wake up (#15270)
* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

* fix: serial_wait strategy workflow unable to wake up

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 17:53:24 +08:00
lizhenglei 3c7a77c90e
[Improvement][E2E] add e2e javatask case (#15469)
* add java task

* fix

* support jdk

* fix java

* fix  java_home

* add ut

* recover

---------

Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2024-01-12 17:20:40 +08:00
lizhenglei bd48c99178
[BUG][Task] fix java task classpath (#15470)
* fix javatask bug

* cover

---------

Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-12 14:39:59 +08:00
liyou 0f7081be10
[Feature] timed scheduler Improvement (#15449) (#15450) 2024-01-11 17:25:31 +08:00
lgcareer 309c8c97a9
[Enhancement][API]Enhance mysql connection properties (#15433) 2024-01-06 19:25:08 +08:00
Wenjun Ruan 6c1e001edf
Add config for defaultTenantEnabled (#15391) 2024-01-06 18:26:22 +08:00
旺阳 e5a512803e
[Bug] [Audit log] Fix Audit log UI query error (#15427) 2024-01-05 13:25:54 +08:00
lizhenglei 4c31e19e11
[Improvement][E2E]e2e improve add workflow httpTask e2e case (#15420)
* e2e case

* Create local.yml

* local

* fix

* fix

* fix blank

---------

Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2024-01-05 12:11:00 +08:00
dalong 3f21a7a577
doc write wrong,should be MinIO it's not MinION (#15395)
Co-authored-by: 86157 <mapengfei@lnxjit.commapengfei@lnxjit.com>
2024-01-03 14:25:05 +08:00
Gallardot 4fe59a7ae8
[Doc]remove temporary markdown comments (#15385)
Signed-off-by: Gallardot <gallardot@apache.org>
2024-01-03 10:48:07 +08:00
Gallardot 575b89e2f9
[Bug][Registry] Optimizing waiting strategy (#15223)
* [Improvement][Registry] Optimizing waiting strategy

Signed-off-by: Gallardot <gallardot@apache.org>
2024-01-02 22:41:50 +09:00
liyou fd74cf16cb
adjust the sequence of alarm group and add validate (#15382) (#15384) 2024-01-02 17:36:30 +08:00
Wenjun Ruan bf2cdc5300
Use Druid to split sql (#15367) 2024-01-02 12:00:53 +08:00
liyou c3ffb1330d
optimize add select filter (#15378) (#15379) 2023-12-29 13:29:09 +08:00
Gallardot 1c1d4bd592
[Improvement][Helm] using helm-docs to generate docs automatically (#15299) 2023-12-29 11:24:37 +08:00
Gallardot 93ea5f64c3
[Improvement][K8S] Custom label of a K8S task can be passed to the pod (#15369)
Signed-off-by: Gallardot <gallardot@apache.org>
2023-12-28 14:49:29 +09:00
Wenjun Ruan d6dea4633c
Optimize server startup log (#15362) 2023-12-27 20:20:21 +08:00
Wenjun Ruan 5523a62825
Remove taskQueue and looper in worker (#15292) 2023-12-27 19:51:45 +08:00
lizhenglei 74fd24b091
[Improvement][E2E] support e2e compose v2 fix code style (#15325)
* support e2e compose v2 fix code stystle

* fix

* fix

* fix version

* Revert "fix version"

This reverts commit 4f1c1963

* fix version

* fix blank

---------

Co-authored-by: 80597928 <673421862@qq.com>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-27 15:35:05 +08:00
liyou c849501f3b
Display the resource file doesn't exist message in task create page (#15350) 2023-12-26 11:07:43 +08:00
Kerwin 4e069991c7
Add 3.1.9 release doc (#15361) 2023-12-25 23:37:05 +08:00
Wenjun Ruan 0e88ea3ac8
Recreate new TaskInstance Working Directory when exist in worker (#15358) 2023-12-25 16:00:41 +08:00
Wenjun Ruan 921253109a
[Bug] Close SSH session after remote shell finish (#15348)
* Close SSH session after remote shell finish

---------

Co-authored-by: David Zollo <davidzollo365@gmail.com>
2023-12-25 13:04:45 +08:00
Wenjun Ruan b73194bd35
Fix check value rather than key in AbstractDataSourceProcessor#checkOther (#15351) 2023-12-25 10:00:07 +08:00
Wenjun Ruan 5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355) 2023-12-23 22:47:59 +08:00
David Zollo a0bc778baf
[Improvement] Ensure that HttpUtils can only get result from certification URL (#15288)
* make httputils just can get Http request not Https
* add test
2023-12-23 19:08:02 +08:00
Devosend b7ffa74537
delete debugger (#15316) 2023-12-21 22:32:09 +08:00
Wenjun Ruan f921457a55
Set maxRows in SqlTask (#15342)
(cherry picked from commit b6b88e3e62d8245e08408fb60c48d23c0bd0bb46)
2023-12-21 20:51:49 +08:00
Gallardot f8a44ff719
[Bug][Master] send ACK event timeout (#15346) 2023-12-21 18:54:44 +08:00
chenrj ce11674668
Fix k8sTaskExecutionContext setting configYaml (#15116)
* fixed the issue of obtaining kubeConfig of k8s tasks

* removed excess files

* removed excess files

* formatted the code

---------

Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-12-14 21:34:42 +08:00
xujiaqiang 159179ac95
[Feature-15146][dolphinscheduler-task-sqoop] add sqoop source/target type (#15146)
* task list: sgoop node params  optimize

* security.ts  add  alarm_instance params

* 1 add SqoopTask params
2 add alert plugin aliyun-voice

* add license header

* commit sqhoop optimize

* pnpm-locl.yaml supplement annotation

* remove irrelevent commit.

* Code specification optimization

* optimize sqoop task ui

* Merge Code

* add the license header to  pnpm-locl.yaml

* format the code

* format the code

* Fix sqoop task echo error

---------

Co-authored-by: xujiaqiang <xujiaqiang@aimatech.com>
Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-13 17:12:05 +08:00
xujiaqiang d144f48ed7
[Feature-15248][dolphinscheduler-alert-plugins] add alert plugin aliyun-voice (#15248)
* add alert aliyun-voice

* VoiceParam replace these with lombok @Data

* security.ts add alarm_instance params

* optimize alert aliyun vocie code

* Merge Code

* rollback pnpm-lock.yaml to branch :origin/dev version

* add the doc about the new plugin

* add com.aliyun.dyvmsapi20170525 version 2.14

* completed

* run mvn spotless:apply

* Code optimization

* Change to @Data

* Code specification optimization

* add com.aliyund.yvmsapi20170525.jar license

* modify com.aliyund.yvmsapi20170525.jar license

* modify com.aliyund.yvmsapi20170525.jar license

* add aliyun-vocie licenses

* add aliyun-voice link to docsdev.js

* modify vocie module to aliyunVoice

* add lisense

---------

Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-13 15:56:37 +08:00
xujiaqiang 0feb144180
[Improvement-15260][dolphinscheduler-datasource-hana] add hana related dependencies (#15260)
* add hana related dependencies

* optimizing HANA data source bugs

* run mvn spotless:apply

* Fix hana datasource getValidationQuery()

* Fix hana datasource testGetJdbcUrl()

---------

Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
2023-12-12 17:11:27 +08:00
ll 0c470ffe66
[Fix #15129] [Dependent] The date rules of the dependent node are ambiguous. (#15289)
* [Fix-15129][Dependent] Fix the ambiguity in date rules for dependent node.

* [fix #15129] Revert ddl

* restore findLastProcessInterval

* update: mvn spotless:apply

---------

Co-authored-by: 李乐 <lile@boimc.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-12-08 16:18:33 +08:00
Dyqer 0bb48f3429
fail-fast for dependent check (#15197)
Co-authored-by: Leoric Yue <leoric.yue@zoom.us>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-12-07 11:06:10 +08:00
Wenjun Ruan 14272dafab
Fix failover Master might not release taskGroup (#15287) 2023-12-06 13:31:46 +08:00
Wenjun Ruan 2119e41800
[Improvement] Move delay calculation to Master (#15278) 2023-12-06 09:37:14 +08:00
Wenjun Ruan 43f5f24529
Add dolphinscheduler-extract-common module (#15266) 2023-12-05 21:06:39 +08:00