Commit Graph

8054 Commits

Author SHA1 Message Date
Chenshuai 3a4f28546e fix: filter (#14770)
Co-authored-by: Jim Chen <chenshuai19950725@gmail.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 502bd36fde)
2023-08-30 17:10:01 +08:00
Aaron Wang f3affa0d75 fix: null app-info-path cause NPE (#14752)
(cherry picked from commit a491e3a849)
2023-08-30 17:09:57 +08:00
caishunfeng 5d9d6061fd [Improvement] add dispatch fail times when compare (#14761)
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
(cherry picked from commit 59998f4c75)
2023-08-30 17:09:52 +08:00
wangzheyuan cdc32b3dc3 Fix a typo in development-environment-setup.md (#14767)
(cherry picked from commit 34d2f67052)
2023-08-30 17:09:28 +08:00
KingsleyY 22eac768da [Fix-14646][Worker] Fix match wrong pid on linux server (#14652)
* [DS-14646][Worker] Fix match wrong pid on linux server
This closes #14646

---------

Co-authored-by: yangxin3 <yangxin3@newhope.cn>
(cherry picked from commit b3af374fe6)
2023-08-30 17:09:25 +08:00
Gallardot 9094423635 fix: kubectl version update (#14762)
Signed-off-by: Gallardot <gallardot@apache.org>
(cherry picked from commit f7ba26f030)
2023-08-30 17:09:18 +08:00
calvin 477a8bdf42 [Improvement][UI] Support to disable or enable the project preferences. (#14756)
(cherry picked from commit e2b97c026e)
2023-08-30 17:09:14 +08:00
Gallardot 47e1bddd4d [Improvement][security] get-user-info API endpoint should not return the user's password in MD5 format (#14680)
* [Improvement][security] get-user-info API endpoint should not return the user's password in MD5 format

Signed-off-by: Gallardot <gallardot@apache.org>

* [Improvement][security] list-paging API endpoint should not return the user's password in MD5 format

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
(cherry picked from commit a826ed2c1a)
2023-08-30 17:08:50 +08:00
Edison Catto 3e0569b4fa [Feature][Resource Center] Add support for Huawei Cloud OBS (#14643)
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center

* add license and add doc

* add 3-party dependency license

* Update LICENSE

* fix

* Update pom.xml

* fix

* fix

---------

Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
(cherry picked from commit 25480ae9e7)
2023-08-30 17:08:39 +08:00
eye-gu 0cb0e76ed3 [Fix-13972][dao] set default value for command (#14612)
(cherry picked from commit dde45dbdfb)
2023-08-30 17:08:34 +08:00
Aaron Wang 6617a00b9b [Fix-14721] [K8S Task] Handle job delete event when user manually delete the job in k8s cluster (#14722)
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
(cherry picked from commit cb55476235)
2023-08-30 17:08:24 +08:00
JieguangZhou 8e4d022692 fix fault tolerant enviroment missing (#14735)
(cherry picked from commit 8e64026e93)
2023-08-30 17:08:18 +08:00
Wenjun Ruan c49d58300d Set tenant as the the resource file owner (#13832)
(cherry picked from commit 93b0283dfe)
2023-08-30 17:08:10 +08:00
Aaron Wang 988aaf880c [Fix-14715] [Resource Center] Modify default fs of local storage plugin (#14716)
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 2314b67870)
2023-08-30 17:07:24 +08:00
Wenjun Ruan 3bc82e7973 Upgrade JDK17 for Sonar (#14726)
(cherry picked from commit 27745a8a84)
2023-08-30 17:06:11 +08:00
Wenjun Ruan 8905f75b02 Fix message in MessageRetryRunner might disorder (#14725)
(cherry picked from commit 3efcf0c5ac)
2023-08-30 17:06:07 +08:00
eye-gu 82f75623a8 [Fix-14714][master] package task instance before put into cache (#14718)
Co-authored-by: eye <eye.gu@aloudata.com>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 7c37a7f9fe)
2023-08-30 17:06:01 +08:00
Gallardot 8e16834836 [Improvement][helm] configmap changes automatically trigger restart (#14657)
* [Improvement][helm] configmap changes automatically trigger restarts

Signed-off-by: Gallardot <gallardot@apache.org>

* feat: add an optional parameter

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit 11d1df2f32)
2023-08-30 17:05:50 +08:00
Kerwin ad9d04c974 Add 3.1.8 release document (#14707)
(cherry picked from commit 420bcd69a6)
2023-08-30 17:05:42 +08:00
Wenjun Ruan 29791122aa Fix serial mode will cause NPE in Workflow bootstrap (#14703)
(cherry picked from commit db62ce0e47)
2023-08-30 17:05:39 +08:00
eye-gu 49a984b368 [Fix-14683][pom] upgrade mysql driver version (#14684)
(cherry picked from commit 4bae7b1ae2)
2023-08-30 17:05:30 +08:00
Gallardot f157c7bbd1 [Doc]fix typo (#14691)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 57b9da145e)
2023-08-30 17:05:11 +08:00
xiangzihao fdfe0a5c65 use wildcard import (#14697)
(cherry picked from commit 85f6a6e73c)
2023-08-30 17:05:07 +08:00
Gallardot 24bab934e0 [Bug][Alert]batchInsert execption #14686 (#14687)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit a5f5aacb5d)
2023-08-30 17:04:44 +08:00
xiangzihao ce5c8f530b [CI][E2E] Try to fix e2e timeout error (#14690)
* try to fix e2e timeout error

* remove use file

(cherry picked from commit dec5228d62)
2023-08-30 17:04:03 +08:00
Aaron Wang 3499c25f84 fix: do not override k8sConfigYaml in ShShellInterceptorBuilder (#14682)
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 5ec9085113)
2023-08-30 17:04:01 +08:00
sleo f3fe454c67 [Fix-14668][dolphinscheduler-tools] Modify the imported jar package. Use org.apache.commons.lang3.StringUtils. (#14669)
(cherry picked from commit 139f23040a)
2023-08-30 17:03:54 +08:00
Tengting Xu 0c68761096 [Workflow] Print WorkflowInstance detail/ TaskInstance detail when finished (#14677)
* [Workflow] Print WorkflowInstance detail/ TaskInstance detail when finished

* [Workflow] minor fix

* [Workflow] add ut

* [Workflow] add header

---------

Co-authored-by: tengting.xu <xtt@dipeak.com>
(cherry picked from commit 9963bfc612)
2023-08-30 17:03:43 +08:00
Gallardot df955e3919 [Bug][Alert]batchInsert execption #14675 (#14676)
Signed-off-by: Gallardot <gallardot@apache.org>
(cherry picked from commit 37ecd262ee)
2023-08-30 17:03:10 +08:00
Gallardot 9af39ea893 [Doc] fix built-in typo (#14667)
Signed-off-by: Gallardot <gallardot@apache.org>
(cherry picked from commit 2f3ab18b29)
2023-08-30 17:02:34 +08:00
Gallardot 5fb947659b [Improvement][UI] Image pull policy in the K8S task provides default values (#14650)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
(cherry picked from commit 150d9e04b0)
2023-08-30 17:02:31 +08:00
eye-gu bddde77d98 [Fix-14008][registry] Fix etcd memory leak due to leaseId (#14034)
(cherry picked from commit d87a0d831c)
2023-08-30 17:02:26 +08:00
Wenjun Ruan 92146ddc2d Support get property from env/jvm/common.properties (#14645)
(cherry picked from commit 9ce8871158)
2023-08-30 17:01:06 +08:00
Rick Cheng 5052720fd8 [Improvement][Parameter] Improve the order of project parameter (#14581)
(cherry picked from commit 79cf785eca)
2023-08-30 17:00:33 +08:00
Wenjun Ruan 4398ab9136 Support filter restrict network interface (#14638)
(cherry picked from commit 2b99451ccd)
2023-08-30 17:00:19 +08:00
Gallardot 40fb3555b9 [Improvement][CI] deploy to k8s (#14555)
* [Improvement][CI] deploy to k8s

Signed-off-by: Gallardot <gallardot@apache.org>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
(cherry picked from commit 5a550dd628)
2023-08-30 16:59:53 +08:00
amao 5ab3f53297 [Improvement-4375][api] cannot delete yarn queue (#13046)
* [Bug-12833][api]
delete api

* [Bug-12833][api]
delete yarn queue

* [Bug-12833][api]
add Unit Test

* Repair the failed CI

* Repair the failed UT

* Repair the failed Docs

* Repair the failed Doc

---------

Co-authored-by: amao <Guoqing.Mao@Yumchina.com>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 5ccc4cc889)
2023-08-30 16:59:50 +08:00
KingsleyY f65dab2e21 [Fix-10795][plugin] sqoop task query columns bug (#10776)
* [Improvement][Api] add @Override on the Override method

* [FIX]sqoop task query columns bug

* merge two if

* update checkstyle problem

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/SqoopColumnType.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/generator/sources/MySQLSourceGenerator.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* code format

* code format

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 512f4c7021)
2023-08-30 16:59:43 +08:00
Wenjun Ruan 5f48b01591 Use AdHoc datasource client in sqlTask (#14631)
* Use AdHoc datasource client in sqlTask

* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient

(cherry picked from commit 4aab0b234f)
2023-08-30 16:59:38 +08:00
xiangzihao 8bdaa0229a [CI][E2E] Upgrade e2e dependency version (#14634)
* upgrade e2e dependency version

* upgrade e2e dependency version

* optimize imports

* remove duplicated awaitility

* remove duplicated awaitility

(cherry picked from commit c6696683e5)
2023-08-30 16:59:16 +08:00
KingsleyY 85fa99c5f8 [DS-14566][master] fix serial_wait command_param does not replace global_params (#14606)
This closes #14566

Co-authored-by: yangxin3 <yangxin3@newhope.cn>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 4a80e3a021)
2023-08-30 16:59:12 +08:00
xiangzihao 9556a7c463 add e2e optional for ci (#14633)
(cherry picked from commit 164154488a)
2023-08-30 13:59:29 +08:00
Wenjun Ruan e13466b343 Revert "datasource test and sql task Remove connection pool issues is #14179 (#14193)" (#14626)
This reverts commit e4fb5b30a4.

(cherry picked from commit 6617e3fb72)
2023-08-30 13:59:27 +08:00
xiangzihao 775b276129 [CI] [E2E] Increase e2e containers timeout (#14628)
* increase e2e containers timeout

(cherry picked from commit 930d2f05b7)
2023-08-30 13:59:25 +08:00
Gallardot c927d657f3 [Improvement][HELM] add driver-class-name (#14614)
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit 0188f40b84)
2023-08-30 13:59:21 +08:00
Aaron Wang f666fd34f9 [Improvement-14269][API] Bind task group with project (#14392)
* [Improvement-14269][API] Bind task group with project

* remove comment

* add project permission check for task group operation

* add doc

(cherry picked from commit 759756ae80)
2023-08-30 13:59:17 +08:00
Aaron Wang 1bc266a202 [Fix-14512][BOM] kubernetes-client version conflict (#14561)
(cherry picked from commit ca760aabd4)
2023-08-30 13:58:27 +08:00
xuhui1231 24857751ed datasource test and sql task Remove connection pool issues is #14179 (#14193)
* datasource test and sql task Remove connection pool issues is #14179

* datasource test and sql task Remove connection pool issues is #14179 uniform style

* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720

* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720

---------

Co-authored-by: xuhui <xuhui@3h1china.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
(cherry picked from commit e4fb5b30a4)
2023-08-30 13:58:22 +08:00
Jay Chung 023b8fb912 ci: Mergeable must have ready-to-merge and not require version (#14600)
you can see our init discuss in https://lists.apache.org/thread/xgcd3wndfst88ornvdknd64nol76k89n

(cherry picked from commit d92e2b360e)
2023-08-30 13:57:34 +08:00
Eric Gao e6e012091d fix process utils comment typo (#14613)
(cherry picked from commit d6a4e99e19)
2023-08-30 13:57:29 +08:00