a55612e4f7
Fix switch task use boolean type error ( #14326 )
2023-06-13 13:26:05 +08:00
1cd5070b5d
[Improvement-14329][common] Remove the duplicate BusinessTimeUtils ( #14330 )
2023-06-13 08:30:34 +08:00
4fa0ab4d7e
[feature][Resource] Add support for Azure Blob Storage in the resource center ( #14082 )
2023-06-12 12:05:49 +08:00
de2cc0e235
[bug][plugin]Fix: Correct the way to determine the yarn queue in Flink CommandLine and SQL mode ( #14237 )
...
* Fix: Correct the way to determine the yarn queue in Flink CommandLine
* fix the yarn queue in sql mode && refine the code
* refine code
* remove unnecessary comment
* fix yarn queue properties
* remove redundant variable
2023-06-12 11:04:39 +08:00
b7fb393eed
fix: trim node selector requirement value ( #14307 )
2023-06-09 18:06:52 +08:00
66e09dc66e
[Improvement-14295][common] Remove the duplicate FileUtils ( #14301 )
2023-06-08 08:59:53 +08:00
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters ( #14127 )
...
* add dynamic task plugin
* fix ui
* update code
* add DOC
* ADD UT
* fix doc
* fix sanity check
* add upgrade ddl
* fix sql
2023-06-07 16:01:06 +08:00
2c057d6a8f
[Improvement-14292][Common] Remove the duplicate OSUtils ( #14293 )
2023-06-07 14:59:25 +08:00
4d798e774e
[Improvement-14280][Task] update the task execution status check ( #14281 )
2023-06-06 15:25:57 +08:00
d587b5f03d
[Bug][Master][Fixes 14216]workflow state keep READY_STOP when sub_process state … ( #14221 )
...
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes #14216
* [Bug] [Master] workflow state keep READY_STOP when sub_process state STOP
Fixes mvn spotless:apply#14216
* Update testFile.txt.crc
* Update testFile.txt.crc
* Delete testFile.txt.crc
2023-06-06 11:04:32 +08:00
9e0991145e
[Improvement-14146][K8S Task] print complete error logs in task log file ( #14147 )
2023-05-27 17:39:03 +08:00
1011852ab2
[Improvement-14195][Parameter] remove misleading comment of combining local and global parameter ( #14196 )
2023-05-26 09:29:09 +08:00
4e78248238
[Feature][K8S Task] support node selector ( #14126 )
...
* feat: support K8S node selector
* fix CI
2023-05-26 09:21:23 +08:00
af7da0630e
[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables ( #14099 )
...
* [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables
2023-05-25 11:02:39 +08:00
606b56403e
[Fix-13381][plugin] fix error while parsing hive load sql ( #13378 )
...
* [fix][task plugin][task sql] fix parsing error while using variables sometimes parsing exception happened if task sql contains a variable followed by \" or \' .
for example:
input sql: load inpath '/tmp/test_table/dt=${dt}'
into table test_table partition(dt=${dt})
after replace: preparing : load inpath '/tmp/test_table/dt=?
into table test_table partition(dt=?)
2023-05-22 11:19:30 +08:00
8315279620
[Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes ( #14150 )
...
* Added a dependent task failure policy in the dependent task node
2023-05-19 14:49:50 +08:00
56b0f91f6e
fix: Save task seatunnel error ( #14129 )
2023-05-19 10:47:16 +08:00
67f50e3904
[fix-#13940] add status of ``` stop``` in class of TaskExecutionStatus ( #14016 )
2023-05-18 13:31:02 +08:00
c3809fbecf
[Improvement-13868][K8S Task] Support the custom labels for k8s task ( #14067 )
...
* [Improvement-13868] Support the custom labels for k8s task
* fix front-end type mismatched
2023-05-16 12:04:33 +08:00
a3b2cc0275
[CI] Try fix api test error ( #14094 )
...
* try fix api test error
* add workergroupapitest to ci and fix log output
* fix log output
* skip remote shell UT
---------
Co-authored-by: SbloodyS <460888207@qq.com>
2023-05-12 21:39:00 +08:00
3eb4c0ed1d
[Improvement-14054][Worker] Kill multiple yarn apps at the same time ( #14055 )
...
* [Improvement-14054][Worker] Kill multiple yarn apps at the same time
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/YarnApplicationManager.java
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-05-06 19:13:12 +08:00
84edea7593
[Improvement-14052][Log] Remove the useless filter in getAppIdsFromLogFile ( #14053 )
2023-05-06 17:00:53 +08:00
3bad68a942
[Bug-13929][K8S Task] Set command contains single quote is invalid ( #13930 )
...
* [Fix-13929][K8S Task] Set command contains single quote is invalid
* fix UT
2023-05-01 11:45:03 +08:00
cf91544b99
Refactor logic task ( #13948 )
2023-04-26 10:01:51 +08:00
db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin ( #13866 )
...
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-04-24 21:28:14 +08:00
ec29220977
[improve-#13966] mask datax log ( #13969 )
...
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-04-21 14:51:37 +08:00
34f5b009e1
[Fix 13941]Fix replace time variable ( #13975 )
...
* fix replace time variable
Co-authored-by: <ziheng.zhan@longbridge.sg>
2023-04-21 14:37:52 +08:00
45586f71d5
[BUG][Resource Center] Task can not select main JAR ( #13536 )
2023-04-18 18:38:16 +08:00
35bb533a50
[Improvement][Task Plugin] Integrate zeppelin password authentication with DS zeppelin task plugin ( #13861 )
...
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-04-18 12:18:51 +08:00
119f08d2ba
[Bug-13882][Worker] Upgrade kubernetes-client version to 6.0.0 ( #13891 )
2023-04-17 12:04:36 +08:00
4439c1ec90
[Feature-13052][Task Plugin] Support Apache Seatunnel Connector-V2 ( #13086 )
2023-04-08 22:02:19 +08:00
d772f2492b
fix: when sudo enable is false, execute command will failed ( #13860 )
2023-04-07 17:13:22 +08:00
d6fcf60534
[Improvement-13751][Worker] Support real-time pod log collection ( #13752 )
...
* Support real-time pod log collection
* datax task test have to mock ShellCommandExecutor with whole dependencies
* replace \n with EMPTY_STRING
* modify error log msg
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* throw exception in pod log collection thread
---------
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2023-04-05 16:13:45 +08:00
a1861bc2df
Use DATAX_PYTHON to specify a datax python version ( #13849 )
...
(cherry picked from commit 752452ecbd2f8094b819eae219f3eec0f6a6427b)
2023-04-03 08:47:32 +08:00
273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin ( #13801 )
2023-03-30 20:17:28 +08:00
9df89cf0cf
Use bootstrap user as the default tenant ( #13825 )
2023-03-30 14:11:23 +08:00
85605bbef9
[Improvement-13653][Remote Logging] Support api-server to get task log from remote target ( #13654 )
2023-03-30 12:07:58 +08:00
d1b6e6f02c
Take over task instance in master failover ( #13798 )
2023-03-28 16:02:57 +08:00
b98566e070
[Bug-13782] match ')' in setValue regular expression ( #13787 )
2023-03-23 22:56:22 +08:00
2dceb6b8ca
[Feature-13428][Resource Center] Add support for Google Cloud Storage in the resource center ( #13562 )
2023-03-15 19:29:27 +08:00
aef5524ee7
Clearer task runnable. ( #13689 )
...
* Clean unused method in AbstractTask
* Kill task by process.destroy
* wait 5/s after destroy process
2023-03-09 20:46:29 +08:00
6218f818c0
[improve-#13665] improve datax parser if sql contain * ( #13680 )
...
* [improve-#13665] improve datax parser if sql contain *
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxTask.java
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-03-07 11:16:53 +08:00
c9066e8de9
Use MDC to filter task instance log ( #13673 )
...
* Use MDC to collect task instance log
* Use MDCAutoClosableContext to remove the MDC key
2023-03-06 17:44:23 +08:00
fd720880c7
[Improvement] Add missing placeholder ( #13663 )
...
* Add missing placeholder
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/am/KubernetesApplicationManager.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
---------
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2023-03-03 18:02:46 +08:00
ca9df868f1
Fix task instance log order might be wrong due to async print ( #13660 )
2023-03-02 22:37:05 +08:00
43d79e45b4
Fix data quality pwd error ( #13643 )
2023-02-28 18:50:44 +08:00
7586710d49
[fix-13625] Modify systemd-run from MemoryMax to MemoryLimit for more generality ( #13627 )
...
Co-authored-by: xiecao <haolong.zhl@alibaba-inc.com>
2023-02-28 14:17:54 +08:00
4351a25f2a
Add execute function to handle the workflow instance operation ( #13610 )
2023-02-24 13:47:41 +08:00
79d3ed0744
Update DataxUtils.java ( #13604 )
2023-02-22 19:01:43 +08:00
7bf3e3cdd6
[improve-#13201] update pid during running ( #13206 )
2023-02-22 14:22:49 +08:00
047fa2f65e
[Feature-13511] Submit Spark task directly on Kubernetes ( #13550 )
2023-02-21 23:26:21 +08:00
2bd65fb2df
[Feature][Remote Logging] Add support for writing task logs to OSS ( #13332 )
2023-02-13 16:38:26 +08:00
37939add0c
[fix-13492]add single quotes ( #13534 )
...
* [fix-13492]add single quotes
* [improve-#13201] fix test error
---------
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-02-09 19:02:27 +08:00
16b193454b
[Improvement][Task Plugin] Support Task Plugin output response as a parameter ( #13529 )
...
* support http output response as param
* add UT
* fix sqoop UT
2023-02-08 23:03:21 +08:00
8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger ( #13509 )
2023-02-07 20:32:53 +08:00
8fa90c2d8f
feature flinksql support standalone deploy mode ( #13495 )
2023-02-05 20:37:56 +08:00
5a9cb7ee64
Fix task log might delay until task finished due to batch flush ( #13502 )
2023-02-02 21:48:03 +08:00
78ba71bdbd
[Feature][task]add azure datafactory task type ( #13417 )
...
* init data factory
* add task-type-config.yaml
* add data factory ui
* submit doc
* submit license
---------
Co-authored-by: devosend <devosend@gmail.com>
2023-01-30 11:15:20 +08:00
a1f5675012
[Bug][TaskPlugin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string ( #13162 )
...
* [Bug] [TaskPlugin-Zeppelin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string. #13161
2023-01-06 19:29:05 +08:00
ba0a253f09
fix k8s task run fail ( #13348 )
2023-01-06 09:44:12 +08:00
6634ff98af
Add source env list in common.properties ( #13317 )
2023-01-04 20:53:52 +08:00
df491f78cc
fix sagemaker sync status ( #13264 )
2022-12-22 20:17:17 +08:00
14ec4a2398
Remove dao module in worker ( #13242 )
2022-12-22 12:25:29 +08:00
b2336b0ce9
Remove datasource in procedure task ( #13198 )
2022-12-19 17:14:59 +08:00
042ec74a24
[refactor] cache compiled regex pattern ( #13208 )
...
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
* [refactor] cache compiled regex pattern
2022-12-18 11:24:11 +08:00
43e6ac32dd
[Fix-13187] refactor OpenmldbParameters ( #13190 )
2022-12-14 20:13:45 +08:00
838a0df783
Remove logic task plugin module ( #13171 )
2022-12-13 16:30:44 +08:00
70ccffeee2
Format task parameter as pretty json ( #13173 )
2022-12-13 16:30:21 +08:00
ab96a3d0c0
Fix datasource jdbc url param doesn't work ( #13098 )
2022-12-12 11:46:40 +08:00
560d232cbd
[Bug][Worker] Fix duplicated key bug when have out parameters of same name ( #12930 )
2022-12-12 11:07:23 +08:00
6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 ( #13002 )
2022-12-10 23:50:19 +08:00
169168ef34
Add plugin-all module ( #13079 )
2022-12-02 23:19:08 +08:00
c2ae76c0bd
Expand all static imports and add CI check to block wildcard static imports ( #13067 )
2022-12-01 14:15:28 +08:00
32a8241ef2
Don't source env file in task, this will override tenant's environment ( #13028 )
2022-11-28 21:09:50 +08:00
ae670e6e1c
[Bug-13010] [Task] The Flink SQL task page selects the pre-job deployment mode, but the task executed by the worker is the Flink local mode ( #13011 )
2022-11-28 10:09:49 +08:00
d480c271ec
[Bug-12994] [Worker] Fix kill process does not take effect ( #12995 )
2022-11-27 14:27:48 +08:00
a2ff140f43
Fix sql task will send alert if we don't choose the send email ( #12984 )
2022-11-24 21:26:52 +08:00
38b876733c
[Feature-10498] Mask the password in the log of sqoop task ( #11589 )
2022-11-24 14:54:54 +08:00
6e224d08a2
[fix] Python task can not pass the parameters to downstream task. ( #12961 )
2022-11-22 16:11:45 +08:00
7c90bf01bc
[Improvement][Task Plugin] Improvement Kubeflow task plugin ( #12928 )
...
add example
check phase in status:conditions
2022-11-17 15:59:44 +08:00
2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis ( #12693 )
2022-11-17 10:14:36 +08:00
92a9b7e243
Use System#lineSeparator to replace /n ( #12889 )
2022-11-15 13:57:45 +08:00
229c554912
[feature][task] Add Kubeflow task plugin for MLOps scenario ( #12843 )
2022-11-11 16:08:38 +08:00
43d69ae157
Upgrade clickhouse jdbc driver ( #12639 )
2022-11-11 13:38:58 +08:00
c4fb555883
Modify shell env to bash ( #12742 )
2022-11-07 11:21:11 +08:00
2c3cc31a4c
Fix flink sql cannot run due to missing main jar ( #12705 )
2022-11-04 23:02:00 +08:00
eaef68856b
Change command file permission to 755 ( #12678 )
...
(cherry picked from commit 95484d930762712f4ab68ece27ca3b65a2073ab2)
2022-11-03 22:51:37 +08:00
a47b4d0672
[Feature][Task] Transfer files between tasks ( #12552 )
...
* add data transfer between tasks
* add delete DATA_TRANSFER API
* convert Result to DeleteDataTransferResponse
* add api UT
* fix final
* fix doc
2022-11-03 16:33:39 +08:00
b71d4983dd
[Feature-6586][Server]add some ds process definition demo when init ( #11759 )
...
* [Feature-6586][Server]add some ds process definition demo when init
1.add some ds process definition demo when init, to display what task type can run and make user easy to
use ds.
2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service
3.need modify the tenant information in it
2022-10-31 22:35:11 +08:00
3c31ddfd9f
[Improvement-12536][k8s] Support the command for the container in k8s task plugin ( #12538 )
2022-10-31 19:06:41 +08:00
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 13:53:44 +08:00
dedff70f90
[Improvement] Merge spi.utils into common.utils ( #12502 )
...
* improve JSONUtil
* merge spi.utils into common.utils
* split constants && remove some copy utils method
* remove StringUtils
2022-10-23 21:23:54 +08:00
f0a34cddec
[improvement-#12416] Use CollectionUtils.isEmpty judge the collection isEmpty to avoid NPE ( #12449 )
2022-10-20 14:26:56 +08:00
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2022-10-19 11:08:04 +08:00
1e11034049
[Fix][Task] Fix dependent task can not predicate the status of the corresponding task correctly ( #12253 )
...
* [Fix][Task] dependent task can not predicate the status of the corresponding task correctly
2022-10-19 10:47:08 +08:00
ba538067f2
[fix] Change the default unix shell executor from sh to bash ( #12180 )
...
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-10-18 12:57:37 +08:00
3bef85f546
fix datax NPE issue ( #12388 )
2022-10-18 09:03:16 +08:00
55004bebe0
[Improvement-12391][API] Workflow definitions that contain logical task nodes support the copy function ( #12392 )
...
* Workflow definitions that contain logical task nodes support the copy function.
2022-10-17 10:28:58 +08:00
b7bd8d7803
[Improvement] add presto in datax ( #12371 )
2022-10-16 17:19:29 +08:00
7aa8a77fb2
[Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @Ignore to @Disable
2022-10-16 12:49:11 +08:00
7b44612f28
[Improvement-12372][k8s] Update the deprecated k8s api ( #12373 )
2022-10-14 16:18:35 +08:00
2f37da0dbc
[Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom ( #12311 )
...
* move mockito dependencies from sub-modules to root pom
* Add check in CI to block the usage of powermock
2022-10-14 12:06:51 +08:00
db1d319159
Reorganize some classes in common module, remove duplicate classes ( #12321 )
2022-10-13 10:23:27 +08:00
05099bd8f7
[Improvement-12296][Test] Close static mock ( #12300 )
2022-10-11 16:13:05 +08:00
875682d267
[Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example ( #12299 )
...
* Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example
2022-10-11 11:54:38 +08:00
a865be09b6
Remove powermock from DatasyncTaskTest ( #12270 )
2022-10-09 12:38:03 +08:00
c7435a51ac
[Feature][task-plugins] aws datasync task plugins ( #11865 )
...
* submit datasync task
* add aws software package license
2022-10-09 10:47:01 +08:00
0687697869
[Improvement][Task Plugin] Modify the comment of 'deployMode'. ( #12163 )
2022-10-07 21:21:13 +08:00
c87f6d89a4
[fix#12195] Change date to timestamp to support cross diff time zones ( #12239 )
...
* fix DS support cross time zone, use timestamp replace date
2022-10-01 12:50:42 +08:00
308f632fcc
[Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) ( #11868 )
...
* add aws dms
2022-09-30 10:16:16 +08:00
aa5526ef5d
[Improvement][Test] Remove Powermock in dolphinscheduler-task-plugin ( #12153 )
...
* Remove the usage of powermock in task-plugin module
2022-09-28 14:13:23 +08:00
779a0b9c75
[Improvement-#11608][task-plugin] New time variables are added to facilitate business development ( #11667 )
...
Add new time expressions for users' convenience
Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
Co-authored-by: David <dailidong66@gmail.com>
2022-09-26 16:01:32 +08:00
e6832220c3
[Improvement][SQL Task]use default sql segment separator ( #10869 )
2022-09-26 11:08:57 +08:00
6eb1eb722a
fix python home conflict ( #12112 )
2022-09-23 09:40:18 +08:00
9b3b4e22b4
[Bug-#12057][task-plugin] fix when the sql query result is empty, the email fails to send the attachment, and an exception will be reported ( #12059 )
...
* [Bug-#12057][task-plugin] fix when the sql query result is empty, the email fails to send the attachment, and an exception will be reported
Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
2022-09-22 09:39:02 +08:00
08a4c7981f
[improve] Remove the spark version of spark task ( #11860 )
2022-09-21 20:27:28 +08:00
cf522e2fc0
[improve] Optimize MLFlow task plugin for easy of use ( #12071 )
...
optimize code
2022-09-21 10:45:05 +08:00
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center ( #11708 )
...
* Add support for Alibaba Cloud OSS as storage of resource center
* Add dependency licenses
* Refactor code, add unit tests and related docs
2022-09-20 22:44:06 +08:00
e1b55db72a
[Improvement][Test] Give an example on replacing Powermock with Mockito ( #11588 )
...
* Give an example on replacing powermock with mockito
* Remove redundant comments
* Refactoring related UTs for better readability.
2022-09-20 09:48:02 +08:00
b3bbf69b2e
[Improvement-11291][sql] Improvement the log output while executing multiple sql statements ( #11292 )
2022-09-20 09:30:27 +08:00
bbc228e93f
[Feature][PyDolphinScheduler] Support DVC task in pyds #11922 ( #11941 )
...
* add dvc task in pyds
* add BaseDVC class
2022-09-17 16:55:51 +08:00
5b384f3fab
[Feature-11473][Task]Support test task ( #11670 )
...
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* Update TaskExecuteThread.java
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update DataSource.java
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages_zh_CN.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages.properties
* Update messages_en_US.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
2022-09-17 13:40:56 +08:00
511149f644
[fix-10938]: use dot to replace source and make the default env work in shell ( #11625 )
...
* [fix-10938]: use the dot to replace source and make the default env work in the sh
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-09-15 15:05:09 +08:00
d3a77c68e6
Make LogServiceClient Singleton ( #11777 )
2022-09-14 16:18:08 +08:00
0deafba12c
fix procedure task param pass ( #11919 )
2022-09-14 13:44:01 +08:00
25b78a8003
[Bug] [spark-sql] In spark-sql, select both SPARK1 and SPARK2 versions and execute ${SPARK_HOME2}/bin/spark-sql ( #11721 )
...
select different versions of spark-sql to execute different versions of spark-sql
2022-09-12 10:35:27 +08:00
147e0ae8d7
Remove duplicate Spring version management ( #11899 )
2022-09-11 20:50:07 +08:00
1164cb776d
Add chunjun in dolphinscheduler-task-all/pom.xml ( #11375 )
2022-09-11 19:14:05 +08:00
5d9331f4ee
[Feature-10683][Task Plugin] Add Java Task Plugin. ( #10689 )
2022-09-09 10:37:24 +08:00
e101a5cb2c
[fix-#11815] fix ck column names contain special characters ( #11818 )
...
Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
2022-09-08 15:09:17 +08:00
2e61c76c22
[Improvement] Add remote task model ( #11767 )
2022-09-08 13:41:36 +08:00
efedc9a53c
Bump up dependencies to fix cves ( #11765 )
2022-09-08 09:05:05 +08:00
444c7e6eb7
[Improvement] Replace commons-lang 2 function invocations with commons-lang3 ( #11810 )
...
* [Improvement] Replace commons-lang 2 function invocations with commons-lang3
2022-09-07 13:48:09 +08:00
f8b9aad239
[feature-#11689] change replaceAll to replace in method of tryExecuteSqlResolveColumnNames ( #11696 )
2022-09-05 09:54:18 +08:00
f95cdd350a
reduce task dependencies ( #11731 )
2022-09-01 17:04:07 +08:00
03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task ( #11701 )
...
* Fix kill job may cause worker oom
2022-08-31 15:25:01 +08:00
4a37345436
[Feature][Task Plugin] Support hive cli task plugin ( #11651 )
...
* Add hive cli task plugin
* Add docs for hive-cli task plugin
* Fix hive-cli task related front-end nits and checkstyle
* Fix exception handling logic
2022-08-30 10:45:51 +08:00
1b120e3a59
Refactor worker execute task process ( #11540 )
...
* Refactor worker execute task process
2022-08-26 13:33:51 +08:00
b96d69701a
[Improvement-#11613] Add spi priority factory ( #11614 )
...
* Add spi priority factory
* Add doc
* Add override log
* Use lombok
* Add comment
2022-08-24 19:51:47 +08:00
a41c6824fc
Add dependent task instance log ( #11541 )
...
* Add dependent task instance log
* Optimize log
* Fix dependent task initialize failed will throw exception
2022-08-23 09:55:38 +08:00
13e4720aa2
Upgrade spring boot and swagger ( #11574 )
2022-08-22 10:27:40 +08:00
d8d5d392a7
[feature-11408] add hive option ( #11493 )
2022-08-20 08:25:53 +08:00
a974ba74ab
[Feature] New task plugin Pytorch ( #11498 )
...
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-08-18 10:20:37 +08:00
11d74fd7e8
[test-11432][HttpTask] Add unit tests to http task ( #11453 )
...
* [test][HttpTask] Add unit tests to http task
* [test][HttpTask] fix http task test code format problem
* [test][HttpTask] fix http task test code replace http code to constants
2022-08-17 22:02:52 +08:00
e5d06c40a5
[Feature-10182][task-datax] ( #11387 )
...
In the process of realizing datax synchronization, use custom json for better connection to enable kerberos-authenticated clusters, such as using plug-ins such as hdfs and hbase
2022-08-17 10:01:20 +08:00
f6ef628030
Support reading application properties from kubernetes config map ( #11402 )
2022-08-16 20:25:17 +08:00
896fef6c98
[Feature][Style] Enable spotless to manage imports ( #11458 )
2022-08-13 10:05:03 +08:00
b79bad19e5
[Fix-11051][Task]Fix the process exitValue exception when the process times out and th… ( #11099 )
2022-08-11 09:28:40 +08:00
5fa3e7b1ed
Add pause to task instance status ( #11390 )
2022-08-11 09:25:53 +08:00
Wenjun Ruan