Commit Graph

254 Commits

Author SHA1 Message Date
zhuangchong 172ebdf04e [maven-release-plugin] prepare for next development iteration 2023-01-04 23:18:59 +08:00
zhuangchong 4a3299ac8c [maven-release-plugin] prepare release 3.1.3 2023-01-04 23:18:57 +08:00
zhuangchong 0691d6a887 [maven-release-plugin] prepare for next development iteration 2023-01-04 15:50:57 +08:00
zhuangchong 3d605ded03 [maven-release-plugin] prepare release 3.1.3 2023-01-04 15:50:55 +08:00
zhuangchong f83b08ac01 [maven-release-plugin] prepare for next development iteration 2023-01-04 12:51:57 +08:00
zhuangchong 450e8ee1b6 [maven-release-plugin] prepare release 3.1.3 2023-01-04 12:51:55 +08:00
zhuangchong a3a9cb9d46 [maven-release-plugin] prepare for next development iteration 2022-12-07 17:23:58 +08:00
zhuangchong f1aefae5e2 [maven-release-plugin] prepare release 3.1.2 2022-12-07 17:23:57 +08:00
zhuangchong 48ad9ff412 [maven-release-plugin] prepare for next development iteration 2022-11-08 11:18:29 +08:00
zhuangchong efc0367999 [maven-release-plugin] prepare release 3.1.1 2022-11-08 11:18:28 +08:00
zhuangchong 145c5ac8cb [maven-release-plugin] prepare for next development iteration 2022-11-07 20:43:01 +08:00
zhuangchong 525a6846ac [maven-release-plugin] prepare release 3.1.1 2022-11-07 20:43:00 +08:00
Kerwin 94ef02e9de
cherry-pick [Improvement] Merge spi.utils into common.utils (#12607)
#12502

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-29 21:30:28 +08:00
kezhenxu94 eb7b48596c cherry-pick Reorganize some classes in common module, remove duplicate classes
#12321
2022-10-27 13:53:41 +08:00
sgw d742994e62 [DS-12131][master] Optimize the log printing of the master module acc… (#12152)
* [DS-12131][master] Optimize the log printing of the master module according to the log specification.
2022-10-27 13:22:48 +08:00
Kerwin f3277277f0
cherry-pick Refactor LogServiceClient Singleton to avoid repeat creation of NettyClient #11777 (#12542)
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2022-10-26 16:20:13 +08:00
caishunfeng 7e39396a76 [maven-release-plugin] prepare for next development iteration 2022-09-26 21:50:03 +08:00
caishunfeng ae33ba5947 [maven-release-plugin] prepare release 3.1.0 2022-09-26 21:49:27 +08:00
Wenjun Ruan 03e1e6fe45
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11701)
* Fix kill job may cause worker oom
2022-08-31 15:25:01 +08:00
Wenjun Ruan 5fa3e7b1ed
Add pause to task instance status (#11390) 2022-08-11 09:25:53 +08:00
caishunfeng 0464123c2b
[Feature-11223] support stream task (#11350)
* add task execute type

* update task definition list paging

* update task instance list paging

* stream task start

* [Feature][UI] Some changes to execute task.
    * Set the connection edge to dashed line.
    * Add FLINK_STREAM task.

* add stream task

* flink savepoint and cancel

* fix query bug

* add stream task definition

* add task instance for stream task

* delete stream task definition state

* update api for stream task definition edit

* modify search for stream task instance

* add language

* delete task type search for stream task definition

* change task type search for stream task instance

* add jump button

* add savepoint

* add down log for stream task instance

* ui test

* stream task start

* run DAG

* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.

* [Feature][UI] Remove some fields for FLINK_STREAM.

* add start modal

* add dryRun column for stream task instance

* fix duration

* fix pon

* fix build error

* Add success tip

* add auto sync for stream task instance

* remove forgien key for task instance

* license header

* UT fix

* modify locales

* recover common config

* fix UT

* add doc

Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2022-08-10 21:44:43 +08:00
Wenjun Ruan 8774415197
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340) 2022-08-10 11:00:23 +08:00
zhuxt2015 3701a24d15
[Improvement][Task Log] Task status log print description instead of code (#11009)
* use execution status instead of status code
2022-07-22 13:34:31 +08:00
Wenjun Ruan 5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
Wenjun Ruan 998e4d74dd
Kill task when we do master failover to avoid task lost (#10997) 2022-07-15 18:44:42 +08:00
Wenjun Ruan cade66a9b6
[Fix-10827] Fix network error cause worker cannot send message to master (#10886)
* Fix network error cause worker cannot send message to master
2022-07-12 14:08:42 +08:00
caishunfeng aa8b88a875
[Feature-10871] add workflow executing data query (#10875)
* add workflow executing data query
* fix sonar check for interrupted
2022-07-11 16:55:55 +08:00
Wenjun Ruan f639a2eed4
[Fix-10854] Fix database restart may lost task instance status (#10866)
* Fix database update error doesn't rollback the task instance status

* Fix database error may cause workflow dead with running status
2022-07-11 09:57:00 +08:00
Wenjun Ruan 67d14fb7b3
[Fix-10785] Fix state event handle error will not retry (#10786)
* Fix state event handle error will not retry

* Use state event handler to deal with the event
2022-07-06 14:53:28 +08:00
Wenjun Ruan 66624c5c86
[Bug] [Master] Worker failover will cause task cannot be failover (#10631)
* fix worker failover may lose event
2022-06-28 16:08:35 +08:00
WangJPLeo f46faa02c3
[Fix-10452] Serial wait for policy recovery (#10453)
* Serial wait for policy recovery

* processInstance state check null

* add sendresult(Host host, Command command) method and replace the original.
2022-06-22 14:28:50 +08:00
Wenjun Ruan ad2646ff1f
Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479)
* Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe
2022-06-16 21:46:18 +08:00
Wenjun Ruan b0d9d3f9ab
Add some warning log in master (#10383)
* Add some warn log in master

* fix may skip sleep
2022-06-10 19:38:20 +08:00
Wenjun Ruan e21d7b1551
[Feature][metrics] Add master, worker metrics (#10326)
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT
2022-06-09 10:55:39 +08:00
Wenjun Ruan 516757cc74
[Fix] [Worker] Fix worker will hang if fails to start (#10342)
* Fix worker will hang if fails to start

* Add .run to ignore
2022-06-06 10:49:23 +08:00
kezhenxu94 d80cf21456
Clean up unused dependencies and packaging issues (#9944) 2022-05-31 15:22:41 +08:00
JinYong Li 49979c658e
[Fix-8828] [Master] Assign tasks to worker optimization (#9919)
* fix 9584

* master recall

* fix ut

* update logger

* update delay queue

* fix ut

* remove sleep

Co-authored-by: 进勇 <lijinyong@cai-inc.com>
Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
2022-05-31 11:49:54 +08:00
guoshupei ca95d2f928
[Fix-9221] [alert-server] optimization and gracefully close (#9246)
* [Fix-9221] [alert-server] optimization and gracefully close

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unnecessary Mockito stubbings

* [Fix-9221] [alert-server] init AlertPluginManager in AlertServer

* [Fix-9221] [alert-server] AlertServerTest add AlertPluginManager installPlugin

* [Fix-9221] [alert-server] replace @Eventlistener with @PostConstruct

* [Fix-9221] [alert-server] sonar check solution

* [Improvement-9221] [alert] update constructor injection and replace IStoppable with Closeable

Co-authored-by: guoshupei <guoshupei@lixiang.com>
2022-04-08 10:02:10 +08:00
Eric Gao fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
caishunfeng ddf1ff98fa
[Improvement-7697][Master/Worker] Change the task ack to runnning callback (#8719)
* rebase dev

* change task state to dispatch if dispatch success

* update task host when dispatch

* add dispatch task event

* test

* check tenant after enable auto create

* handle dispatch state

Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-30 17:35:05 +08:00
Kerwin a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584)
* Task plug-in optimization, task-related classes are migrated to the task plugin module

* remove tasktype enum class.

* fix code style.

* fix code style

* fix code style

* fix ProcessServiceTest.testUpdateResourceInfo test class

* fix code style

* fix test task plugin manager check parameters null

* fix test ut

* update aws license

* change TaskTimeoutStrategy class
2022-03-15 15:16:36 +08:00
wangyang 0d8079a816
[DS-7992][feat] Alert module judging strategy (#8636) 2022-03-14 14:25:27 +08:00
yimaixinchen a72742b9de
remove unuse import jsonFormat (#8548) 2022-02-25 22:05:05 +08:00
caishunfeng 84f2cfec39
[Feature][Server] Custom timezone - Add timezone handler unified (#8477)
* date convert of timezone

* remove @JsonFormat

* add unit test

* fix time preview in scheduler

* optimization & add env config

Co-authored-by: caishunfeng <534328519@qq.com>
2022-02-24 18:20:34 +08:00
Wenjun Ruan 66dcf2376a
Optimize some code (#8324) 2022-02-10 19:07:53 +08:00
Wenjun Ruan 093815785b
Remove unused class (#8299) 2022-02-08 19:50:19 +08:00
Kerwin 103ccb01da
change version to 2.0.4-SNAPSHOT (#8188) 2022-01-25 15:17:17 +08:00
xuhhui 0cfb2043cf
Improvement: improve Pong.java mistake (#7822) 2022-01-12 01:20:13 +08:00
kezhenxu94 67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
wind a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187)
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 15:55:22 +08:00