e3fdb1a3ff
[Fix-15639] parameterPassing is null case NPE ( #15678 )
...
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2024-03-08 17:52:24 +08:00
24ff70dc88
Fix out parameter may loss ( #15617 )
2024-02-23 11:11:06 +08:00
91d56f4860
fix: data quality may fail in docker mode ( #15563 )
2024-02-05 17:14:59 +08:00
4d6af516b9
[Bug][Task Api] fix 'MACPATTERN' in ProcessUtils and cover all cases on MacOS in ProcessUtilsTest ( #15480 ) ( #15529 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-05 11:41:21 +08:00
8efaa9fa1f
fix: data quality can not use ( #15551 )
...
* fix: data quality can not use
fix : #15468 , #15249 , #14858
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-05 09:54:06 +08:00
4d15932ede
Fix createFile with permission will not work ( #15556 )
2024-02-03 18:00:33 +08:00
d80a29c48c
[BUG] #15013 Fix retryInterval in RetryPolicy will never be used in RetryUtils ( #15014 )
...
* Update RetryUtils.java
assigned
* UT
* Update RetryUtilsTest.java
---------
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-25 15:07:11 +08:00
5a7deb28f7
Throw IllegalArgumentException if parse time placeholder error ( #15514 )
...
(cherry picked from commit 5fa31e036d77698f3732a42664bfac3e7edcb8e4)
2024-01-22 10:21:54 +08:00
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception ( #15496 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
c7a11ce6ed
[Improvement][K8S]Optimize MDC for K8S tasks ( #15390 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-17 17:56:11 +08:00
93ea5f64c3
[Improvement][K8S] Custom label of a K8S task can be passed to the pod ( #15369 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
2023-12-28 14:49:29 +09:00
d6dea4633c
Optimize server startup log ( #15362 )
2023-12-27 20:20:21 +08:00
5523a62825
Remove taskQueue and looper in worker ( #15292 )
2023-12-27 19:51:45 +08:00
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) ( #15355 )
2023-12-23 22:47:59 +08:00
ce11674668
Fix k8sTaskExecutionContext setting configYaml ( #15116 )
...
* fixed the issue of obtaining kubeConfig of k8s tasks
* removed excess files
* removed excess files
* formatted the code
---------
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-12-14 21:34:42 +08:00
0bb48f3429
fail-fast for dependent check ( #15197 )
...
Co-authored-by: Leoric Yue <leoric.yue@zoom.us>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-12-07 11:06:10 +08:00
2119e41800
[Improvement] Move delay calculation to Master ( #15278 )
2023-12-06 09:37:14 +08:00
db3d84b73d
[HotFix] Fix TaskOutputParameterParser might OOM if meed a bad output param expression ( #15264 )
2023-12-04 18:38:38 +08:00
ef2e9c9b67
Set the tenant as the owner in final stage ( #15256 )
2023-12-01 21:49:28 +08:00
71ee1f0c3d
Support parse task output params under multiple log ( #15244 )
2023-12-01 10:16:36 +08:00
df656a7671
Fix home page workflow instance miss status ( #15193 )
2023-11-22 10:33:24 +08:00
5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue ( #14992 )
...
* change data quality field name
* fix-dao
* fix-dao
* fix-dao
* fix-dao
* update
* test
* update
* update
2023-11-13 11:04:30 +08:00
c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center ( #14977 )
2023-10-31 17:07:24 +08:00
5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks ( #14976 )
...
* Refactoring `Sagemaker` task plugin with connections managed in connection center.
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-10-31 10:07:05 +08:00
688f84452d
[Feature-14678][Master][UI]Dependent task parameter passing ( #14702 )
...
* feat: dependent task parameter passing
* style: code format
* refactor: remove inappropriate log
* refactor: Modify unreasonable code structure
* refactor: Modify unreasonable code structure
* refactor: Solve conflicts caused by merge
---------
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: BaoLiang <29528966+lenboo@users.noreply.github.com>
2023-10-25 16:24:30 +08:00
049c1511fa
[Improvement][K8S] K8S task support passing context parameters to downstream task ( #14934 )
...
* [Improvement][K8S] K8S task support passing context parameters to downstream task
Signed-off-by: Gallardot <gallardot@apache.org>
* [Improvement][K8S] K8S task support passing context parameters to downstream task
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: update doc img
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: update doc
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: add UT
Signed-off-by: Gallardot <gallardot@apache.org>
* merge suggestion
Co-authored-by: 旺阳 <wang@lqwang.net>
* merge suggestion
Co-authored-by: 旺阳 <wang@lqwang.net>
* merge suggestion
Co-authored-by: 旺阳 <wang@lqwang.net>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: David Zollo <dailidong66@gmail.com>
Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-10-08 17:27:10 +08:00
f90eadd40f
[feature-#14449] add secret in k8s Task ( #14930 )
...
* [feature-#14449] add secret
* [feature-#14449] upgrade UT and rename parameter ```secret```
* [feature-#14449] fix UT
* [feature-#14449] fix UT
* [feature-#14449] improvement
* [feature-#14449] format
---------
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-09-28 11:06:35 +08:00
6de1e2c1b3
Fix the error of using shell task to obtain Home variable in Ubuntu system. ( #14964 )
2023-09-26 17:18:38 +08:00
f6a6794289
[Bug][Worker] Global and local parameters are not replaced in SQL components ( #14926 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
2023-09-15 10:31:14 +08:00
55cc77226f
[RPC] Refactor the remote module. ( #14776 )
...
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
a491e3a849
fix: null app-info-path cause NPE ( #14752 )
2023-08-19 13:28:45 +08:00
59998f4c75
[Improvement] add dispatch fail times when compare ( #14761 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2023-08-18 15:26:01 +08:00
b3af374fe6
[Fix-14646][Worker] Fix match wrong pid on linux server ( #14652 )
...
* [DS-14646][Worker] Fix match wrong pid on linux server
This closes #14646
---------
Co-authored-by: yangxin3 <yangxin3@newhope.cn>
2023-08-17 19:17:16 +08:00
25480ae9e7
[Feature][Resource Center] Add support for Huawei Cloud OBS ( #14643 )
...
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center
* add license and add doc
* add 3-party dependency license
* Update LICENSE
* fix
* Update pom.xml
* fix
* fix
---------
Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2023-08-14 13:25:48 +08:00
cb55476235
[Fix-14721] [K8S Task] Handle job delete event when user manually delete the job in k8s cluster ( #14722 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-08-11 17:15:07 +08:00
5ec9085113
fix: do not override k8sConfigYaml in ShShellInterceptorBuilder ( #14682 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
2023-08-02 18:01:34 +08:00
9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
2023-07-28 14:41:58 +08:00
d6a4e99e19
fix process utils comment typo ( #14613 )
2023-07-21 13:18:10 +08:00
04800a48d1
[Feature-13763][K8S Task] collect real-time log ( #14379 )
...
* [Improvement-13763][K8S Task] collect real-time log
* fix codesmell
* get pod watcher until pod is ready
* fix codesmell
* specify container name & loop waiting pod creation
* sleep when pod is not initialized
---------
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-07-20 18:38:25 +08:00
c30cca9d9a
Support execute shell in different interceptor ( #14582 )
2023-07-19 14:50:04 +08:00
e6d94632b9
Add WorkflowExecuteContext ( #14544 )
2023-07-14 18:40:01 +08:00
09afa75a3d
[Improvement][Worker] No need to kill remote app when appid is empty ( #14534 )
2023-07-13 15:40:58 +08:00
c1cffba349
[Improvement] Remove the unused newDaemonSingleThreadExecutor ( #14520 )
2023-07-12 21:03:41 +08:00
175d976cd2
[Feature][Data Quality] Data Quality Support Choose Database ( #14406 )
...
* add dataquality database api
* change ui
* api change
* update
* fix spotless
* fix h2
* fix pg
* fix-dead-line
* update
* fix-spotless
* update pg sql
* add ut
* fix ut
2023-07-12 09:00:28 +08:00
0b69236b04
[Feature-14251][Task] Support yarn queue definition in yarn task ( #14310 )
2023-07-07 14:24:47 +08:00
742a2dc780
[Fix-14447][K8S Task] Fix IndexOutofBoundsException for pod name without dash ( #14448 )
2023-07-07 12:53:16 +08:00
cff53feaeb
[Improvement-14464][Task] Remove the useless taskResultString ( #14465 )
2023-07-06 17:31:33 +08:00
65b6a4b097
[Feature-14421][K8S Task] Configurable image pull policy ( #14426 )
2023-07-05 09:17:47 +08:00
b49b91c45d
[Improvement-14331][common] Remove the duplicate ParameterUtils ( #14332 )
2023-06-14 15:00:54 +08:00
a55612e4f7
Fix switch task use boolean type error ( #14326 )
2023-06-13 13:26:05 +08:00
lch