c9066e8de9
Use MDC to filter task instance log ( #13673 )
...
* Use MDC to collect task instance log
* Use MDCAutoClosableContext to remove the MDC key
2023-03-06 17:44:23 +08:00
fd720880c7
[Improvement] Add missing placeholder ( #13663 )
...
* Add missing placeholder
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/KubernetesApplicationManager.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
---------
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2023-03-03 18:02:46 +08:00
ca9df868f1
Fix task instance log order might be wrong due to async print ( #13660 )
2023-03-02 22:37:05 +08:00
43d79e45b4
Fix data quality pwd error ( #13643 )
2023-02-28 18:50:44 +08:00
7586710d49
[fix-13625] Modify systemd-run from MemoryMax to MemoryLimit for more generality ( #13627 )
...
Co-authored-by: xiecao <haolong.zhl@alibaba-inc.com>
2023-02-28 14:17:54 +08:00
4351a25f2a
Add execute function to handle the workflow instance operation ( #13610 )
2023-02-24 13:47:41 +08:00
79d3ed0744
Update DataxUtils.java ( #13604 )
2023-02-22 19:01:43 +08:00
7bf3e3cdd6
[improve-#13201] update pid during running ( #13206 )
2023-02-22 14:22:49 +08:00
047fa2f65e
[Feature-13511] Submit Spark task directly on Kubernetes ( #13550 )
2023-02-21 23:26:21 +08:00
2bd65fb2df
[Feature][Remote Logging] Add support for writing task logs to OSS ( #13332 )
2023-02-13 16:38:26 +08:00
37939add0c
[fix-13492]add single quotes ( #13534 )
...
* [fix-13492]add single quotes
* [improve-#13201] fix test error
---------
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-02-09 19:02:27 +08:00
16b193454b
[Improvement][Task Plugin] Support Task Plugin output response as a parameter ( #13529 )
...
* support http output response as param
* add UT
* fix sqoop UT
2023-02-08 23:03:21 +08:00
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2023-02-07 20:32:53 +08:00
8fa90c2d8f
feature flinksql support standalone deploy mode ( #13495 )
2023-02-05 20:37:56 +08:00
5a9cb7ee64
Fix task log might delay until task finished due to batch flush ( #13502 )
2023-02-02 21:48:03 +08:00
78ba71bdbd
[Feature][task]add azure datafactory task type ( #13417 )
...
* init data factory
* add task-type-config.yaml
* add data factory ui
* submit doc
* submit license
---------
Co-authored-by: devosend <devosend@gmail.com>
2023-01-30 11:15:20 +08:00
a1f5675012
[Bug][TaskPlugin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string ( #13162 )
...
* [Bug] [TaskPlugin-Zeppelin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string. #13161
2023-01-06 19:29:05 +08:00
ba0a253f09
fix k8s task run fail ( #13348 )
2023-01-06 09:44:12 +08:00
6634ff98af
Add source env list in common.properties ( #13317 )
2023-01-04 20:53:52 +08:00
df491f78cc
fix sagemaker sync status ( #13264 )
2022-12-22 20:17:17 +08:00
14ec4a2398
Remove dao module in worker ( #13242 )
2022-12-22 12:25:29 +08:00
b2336b0ce9
Remove datasource in procedure task ( #13198 )
2022-12-19 17:14:59 +08:00
042ec74a24
[refactor] cache compiled regex pattern ( #13208 )
...
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
2022-12-18 11:24:11 +08:00
43e6ac32dd
[Fix-13187] refactor OpenmldbParameters ( #13190 )
2022-12-14 20:13:45 +08:00
838a0df783
Remove logic task plugin module ( #13171 )
2022-12-13 16:30:44 +08:00
70ccffeee2
Format task parameter as pretty json ( #13173 )
2022-12-13 16:30:21 +08:00
ab96a3d0c0
Fix datasource jdbc url param doesn't work ( #13098 )
2022-12-12 11:46:40 +08:00
560d232cbd
[Bug][Worker] Fix duplicated key bug when have out parameters of same name ( #12930 )
2022-12-12 11:07:23 +08:00
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 ( #13002 )
2022-12-10 23:50:19 +08:00
169168ef34
Add plugin-all module ( #13079 )
2022-12-02 23:19:08 +08:00
c2ae76c0bd
Expand all static imports and add CI check to block wildcard static imports ( #13067 )
2022-12-01 14:15:28 +08:00
32a8241ef2
Don't source env file in task, this will override tenant's environment ( #13028 )
2022-11-28 21:09:50 +08:00
ae670e6e1c
[Bug-13010] [Task] The Flink SQL task page selects the pre-job deployment mode, but the task executed by the worker is the Flink local mode ( #13011 )
2022-11-28 10:09:49 +08:00
d480c271ec
[Bug-12994] [Worker] Fix kill process does not take effect ( #12995 )
2022-11-27 14:27:48 +08:00
a2ff140f43
Fix sql task will send alert if we don't choose the send email ( #12984 )
2022-11-24 21:26:52 +08:00
38b876733c
[Feature-10498] Mask the password in the log of sqoop task ( #11589 )
2022-11-24 14:54:54 +08:00
6e224d08a2
[fix] Python task can not pass the parameters to downstream task. ( #12961 )
2022-11-22 16:11:45 +08:00
7c90bf01bc
[Improvement][Task Plugin] Improvement Kubeflow task plugin ( #12928 )
...
add example
check phase in status:conditions
2022-11-17 15:59:44 +08:00
2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis ( #12693 )
2022-11-17 10:14:36 +08:00
92a9b7e243
Use System#lineSeparator to replace /n ( #12889 )
2022-11-15 13:57:45 +08:00
229c554912
[feature][task] Add Kubeflow task plugin for MLOps scenario ( #12843 )
2022-11-11 16:08:38 +08:00
43d69ae157
Upgrade clickhouse jdbc driver ( #12639 )
2022-11-11 13:38:58 +08:00
c4fb555883
Modify shell env to bash ( #12742 )
2022-11-07 11:21:11 +08:00
2c3cc31a4c
Fix flink sql cannot run due to missing main jar ( #12705 )
2022-11-04 23:02:00 +08:00
eaef68856b
Change command file permission to 755 ( #12678 )
...
(cherry picked from commit 95484d930762712f4ab68ece27ca3b65a2073ab2)
2022-11-03 22:51:37 +08:00
a47b4d0672
[Feature][Task] Transfer files between tasks ( #12552 )
...
* add data transfer between tasks
* add delete DATA_TRANSFER API
* convert Result to DeleteDataTransferResponse
* add api UT
* fix final
* fix doc
2022-11-03 16:33:39 +08:00
b71d4983dd
[Feature-6586][Server]add some ds process definition demo when init ( #11759 )
...
* [Feature-6586][Server]add some ds process definition demo when init
1.add some ds process definition demo when init, to display what task type can run and make user easy to
use ds.
2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service
3.need modify the tenant information in it
2022-10-31 22:35:11 +08:00
3c31ddfd9f
[Improvement-12536][k8s] Support the command for the container in k8s task plugin ( #12538 )
2022-10-31 19:06:41 +08:00
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 13:53:44 +08:00
Wenjun Ruan