Commit Graph

910 Commits

Author SHA1 Message Date
Kirs 67508d8d1b
update project version to 2.0.0-SNAPSHOT (#6383) 2021-09-28 12:00:43 +08:00
百岁 34352c1bf8
Enhance using experience of DataX by introduce TIS (#6229)
* Enhance using experience of DataX by introduce TIS [Feature-5992]

* Enhance using experience of DataX by introduce TIS [Feature-5992]

* fix stylecheck error

* make testCase pass

* make dev testCase pass

* add new java dependency Java-WebSocket

* modfiy TISParameters.java avoid to name confliction

* add InterruptedException checking

* reAdd async-http-client annotation

* in order to staisfy the coverage degree add test which has been removed

* make testCase pass

* make testCase pass

* add jacoco dependency

* make code duplications be more lower

* add Java-WebSocket dependency in root pom

* remove useless code comment

* make tis http apply path and post body configurable ,the params save in config.properites

* Remove the dangerous instance of double-checked locking
2021-09-26 18:40:41 +08:00
HanayoZz 4625783479
[Fix-6072][common] Remove Generic method JsonUtils.toMap (#6283)
* [Fix][common] Remove Generic method JsonUtils.toMap, Replace by parseObj(String json, TypeReference type)
2021-09-23 23:06:53 +08:00
LiuBodong c4375a54c1
[Fix][Task] Task lost resource information (#6292) (#6295)
Co-authored-by: liubodong <liubodong>
2021-09-23 22:23:17 +08:00
JinyLeeChina 07d6542feb
[Fix-6139][API] fix workFlowLineage relation (#6217)
* fix bug of view-tree api

* fix bug of view-tree api

* fix ut

* fix ut

* fix master buildFlowDag error

* fix workFlowLineage relation

* fix ut

* fix ut

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-09-15 17:42:21 +08:00
OS 8694b9c8f9
fix bug-6208 json parse error in sub process (#6211) 2021-09-15 10:29:26 +08:00
Jiajie Zhong 170280fab2
Simplify constants variables (#6183)
* Combine `alter` and `registry` properties,
  bese on https://github.com/apache/dolphinscheduler/pull/6180
* Remove duplicate variable `SLASH` and `SINGLE_SLASH`
2021-09-13 22:03:09 +08:00
JinyLeeChina 14e4d7086d
[Fix-6139][API] fix bug of view-tree api (#6188)
* fix bug of view-tree api

* fix bug of view-tree api

* fix ut

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-09-13 20:40:52 +08:00
kezhenxu94 ea2a9dbcb2
Clean up third-party files that are not mentioned in LICENSE (#6165) 2021-09-11 11:27:42 +08:00
kezhenxu94 a632be73c0
Clean up unused util classes... (#6154) 2021-09-10 00:35:39 +08:00
JinyLeeChina e8ddc9103d
[Bug][API] fix genTaskCodeList return same code and save proces error (#6150)
* fix mysql create sentence bug

* fix mysql create sentence bug

* fix genTaskCodeList return same code and save proces error

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-09-09 22:01:22 +08:00
CalvinKirs dbb4adf6c3 Merge remote-tracking branch 'upstream/dev' into task-end
# Conflicts:
#	.asf.yaml
#	dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/FuncUtils.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/AlertGroupControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/StateEventType.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Command.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ErrorCommand.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ScheduleMapper.xml
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionLogMapper.xml
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/ConditionsTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/DependentTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/CommonTaskProcessor.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/ConditionTaskProcessor.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/DependentTaskProcessor.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/ConditionsTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/DependentTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/MasterBaseTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/MasterTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/SubProcessTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/WorkerManagerThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/python/PythonTask.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTask.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/master/WorkflowExecuteThreadTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/processor/TaskKillProcessorTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThreadTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/runner/WorkerManagerThreadTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/PythonCommandExecutorTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/http/HttpTaskTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/worker/task/sql/SqlTaskTest.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/quartz/cron/CronUtils.java
#	dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java
#	dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/DolphinSchedulerPlugin.java
#	dolphinscheduler-standalone-server/pom.xml
#	dolphinscheduler-standalone-server/src/main/java/org/apache/dolphinscheduler/server/StandaloneServer.java
#	dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/ShellCommandExecutor.java
#	dolphinscheduler-task-plugin/dolphinscheduler-task-procedure/src/main/java/org/apache/dolphinscheduler/plugin/task/procedure/ProcedureTaskChannelFactory.java
#	dolphinscheduler-task-plugin/dolphinscheduler-task-python/src/main/java/org/apache/dolphinscheduler/plugin/task/python/PythonCommandExecutor.java
#	dolphinscheduler-task-plugin/dolphinscheduler-task-shell/src/main/java/org/apache/dolphinscheduler/plugin/task/shell/ShellTask.java
#	dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java
#	dolphinscheduler-task-plugin/dolphinscheduler-task-sqoop/src/main/java/org/apache/dolphinscheduler/plugin/task/sqoop/SqoopTask.java
#	dolphinscheduler-ui/package.json
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.scss
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.vue
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/formModel/formModel.vue
#	sql/dolphinscheduler_h2.sql
2021-09-08 16:44:07 +08:00
Kirs 9508d1e978
[SPI][TASK]TaskPlugin (#6122)
* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)

* [Bug][WorkerServer] SqlTask NullPointerException #5549

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428

* fix the readme typing issue (#5998)

* Fix unchecked type conversions

* Use indentation level reported by checkstyle

* Reorganize CI workflows to fasten the wasted time and resources (#6011)

* Add standalone server module to make it easier to develop (#6022)

* Task node of SWITCH (#5939)

* [Feature-#5273][server-master] Task node of SWITCH (#5922)



Co-authored-by: wangxj <wangxj31>

* remove description of bonecp (#6030)

Co-authored-by: shaojwu <shaojwu@ebay.com>

* [Improvement][Api Module]split alert group list-paging interface (#5941)

* [Improvement][Api Module]split alert group list-paging interface

* [FIX-#6007]Wrong complement date (#6026)

* [FIX-#6007]Wrong complement date

* [style]Wrong complement date

* [Improvement-6024][dist] Remove useless packaging commands (#6029)

·Remove useless packaging commands in dolphinscheduler-bin.xml

This closes #6024

Co-authored-by: mask <liuhu@zhiyoutec.com>

* [FIX-5908][MasterServer] When executing an compensation task, the execution thread would have a NPE (#5909)

* fix the npe in MasterExec

* fix the compile error

* Add `.asf.yaml` to easily set the GitHub metadata (#6035)

* fix dead server cannot stop (#6046)

* Enhancement Translation (#6042)

* replaced Loading... with i18n

* modified Edit zh_CN translation

* Delete zh_CN.js

Co-authored-by: David <dailidong66@gmail.com>

* fix bug #6053 zh_CN.js is lost

* [Fix-6038][ui] width of "SQL Statement" in Dag FormLineModal will be shrunk if sql line is too long (#6040)

This closes #6038

* [Improvement] Fix inefficient map iterator (#6004)

* Fix inefficient map iterator

* Use forEach and remove call to valueOf

* Modify AbstractParameters

* Enhance `StandaloneServer` so that we don't need to update the version number manually (#6074)

* Remove invalid character in `.asf.yaml` (#6075)

* Remove invalid character `\n` in `.asf.yaml` (#6077)

It turns out that the invalid character is `\n`

* Add alert server into standalone-server as well and some minor polish (#6087)

* Support starting standalone server in Docker image (#6102)

Also remove unused class

* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095)

* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)

* master refactor:
   1. spi for task submit and other actions(pause, kill)
   2. remove threads for process instance and task instance.
   3. add events for process instance and task instance

* ut npe

* add try catch

* code style

* fix critical bugs

* fix critical bugs

* fix critical bugs

* fix critical bugs

* Remove unused params in SwitchTaskTest (#6109)

* [Feature-5987][Server] Support to set multiple environment configs for a certain worker. (#6082)

* support multi environments

* add some test cases

* add an environment vue component

* improve environment form

* improve environment form

* add environment worker group relation

* add environment worker group relation

* add the environment choice for formModel

* set an environment for the task

* modify the modal form of starting process

* add the environment config to TaskExecutionContext

* add the environment config to the timing form

* fix conflicts

* fix issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some bugs in the code review

* add the same table and columns to support H2

* fix some bugs

* [Plugin][Task]Task Spi

* [Plugin][Task]Task Spi

* [Plugin][Task]Task Spi

* [Plugin][Task]Task Spi

* [Plugin][Task]Task Spi

* [Plugin][Task]Task Spi

Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: myangle1120 <942542838@qq.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: shaojwu <shaojwu@ebay.com>
Co-authored-by: Shukun Zhang <60541766+andream7@users.noreply.github.com>
Co-authored-by: linquan <1175687813@qq.com>
Co-authored-by: mask <39329477+Narcasserun@users.noreply.github.com>
Co-authored-by: mask <liuhu@zhiyoutec.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: RichardStark <49977764+RichardStark@users.noreply.github.com>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: lenboo <baoliang.leon@gmail.com>
Co-authored-by: lilyzhou <lj_zhou@outlook.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
Co-authored-by: junfan.zhang <zuston.shacha@gmail.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
2021-09-08 10:40:40 +08:00
lenboo b661ec9675 Merge remote-tracking branch 'upstream/dev' into json2
# Conflicts:
#	.github/actions/translate-on-issue
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AccessTokenController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertGroupController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AlertPluginInstanceController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataSourceController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/LoggerController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessInstanceController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProjectController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/QueueController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ResourcesController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/SchedulerController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TaskInstanceController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/TenantController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UiPluginController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/UsersController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/WorkerGroupController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/EnvironmentWorkerGroupRelationService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/SchedulerService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskInstanceService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/DataSourceServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/SchedulerServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskInstanceServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/utils/exportprocess/TaskNodeParamFactory.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/AbstractControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/AlertGroupControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessInstanceControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/SchedulerControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/DataAnalysisServiceTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ExecutorService2Test.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionServiceTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/TaskInstanceServiceTest.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/Constants.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/model/TaskNode.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Command.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ErrorCommand.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ScheduleMapper.xml
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionLogMapper.xml
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/task/DependentTaskProcessFactory.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/ShellCommandExecutor.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/master/SwitchTaskTest.java
#	dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/master/WorkflowExecuteThreadTest.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
#	dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.js
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.scss
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/dag.vue
#	dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/formModel/formModel.vue
#	dolphinscheduler-ui/src/js/conf/home/pages/projects/pages/definition/pages/list/_source/start.vue
#	dolphinscheduler-ui/src/js/conf/home/store/security/actions.js
#	dolphinscheduler-ui/src/js/module/i18n/locale/en_US.js
#	dolphinscheduler-ui/src/js/module/i18n/locale/zh_CN.js
#	sql/dolphinscheduler_h2.sql
2021-09-07 23:52:03 +08:00
JinyLeeChina 33ea3eddb9
[Feature][JsonSplit-api] merge code from dev to json2 (#6115)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)



* add a new init method.

* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)

* add the necessary parameters

* openapi improve

* fix code style error

* [FIX-#5721][master-server] Global params parameter missing (#5757)



Co-authored-by: wangxj <wangxj31>

* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move`  doesn't work. (#5739)

* Update relatedItems.vue

* Update relatedItems.vue

* [Improvement#5741][Worker] Improve task process status log  (#5776)

* [Improvement-5773][server] need to support two parameters related to task (#5774)

* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>

* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE

* [Improvement][Worker] Task log may be lost #5775 (#5783)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file
  Upgrade checkstyle.xml to support checkstyle version 8.24+

* change ci checkstyle version

* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected.  (#5796)

* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.

* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)

·the list of workflow instances is sorted by start time and end time
·This closes #5621

* fix (#5803)

Co-authored-by: shuangbofu <fusb@tuya.com>

* fix: Remove duplicate "registryClient.close" method calls (#5805)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Improvement][SPI] support load single plugin (#5794)

change load operation of 'registry.plugin.dir'

* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)

* fix: refactor registry client, remove spring annotation

* fix UT

* fix UT

* fix checkstyle

* fix UT

* fix UT

* fix UT

* fix: Rename RegistryCenterUtils method name

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)

* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)

* enhance the validation in UI

* enchance form verifaction

* simplify disable condition

* fix: Remove unused class (#5833)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [fix-5737] [Bug][Datasource] datsource other param check error (#5835)

Co-authored-by: wanggang <wanggy01@servyou.com.cn>

* [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

[Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

* [Fix-5825][BUG][WEB] the resource tree in the process definition of latest dev branch can't display correctly (#5826)

* resoures-shows-error

* fix codestyle error

* add license header for new js

* fix codesmell

* [Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867)

* provide two system parameters to support the rest of type of tasks

* provide two system parameters to support the rest of type of tasks

* improve test conversion

* [Improvement][Fix-5769][UI]When we try to delete the existing dag, the console in web browser would shows exception (#5770)

* fix bug

* cache the this variable

* Avoid self name

* fix code style compile error

* [Fix-5781][UT] Fix test coverage in sonar (#5817)

* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781

* [Fix-5808][Server]  When we try to transfer data using datax between  different types of data sources, the worker will exit with ClassCastException (#5809)

* bug fix

* fix bug

* simplify the code format

* add a new parameter to make it easier to understand.

* [Fix-5830][Improvement][UI] Improve the selection style in dag edit dialog (#5829)

* improve the selection style

* update another file

* remove unnecessary css part.

* [Fix-5904][upgrade]fix dev branch upgrade mysql sql script error (#5821)

* fix dev branch upgrade mysql sql script error.

* Update naming convention.

* [Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832)

* fix: refactor api utils class, remove spring annotation.

* fix: Optimization comments

Co-authored-by: wen-hemin <wenhemin@apache.com>

* correct the wrong annotion from zk queue implemented to java priority blocking queue (#5906)

Co-authored-by: ywang46 <ywang46@paypal.com>

* Add a Gitter chat badge to README.md (#5883)

* Add Gitter badge

* Update README.md

Co-authored-by: David <dailidong66@gmail.com>

* ci: improve maven connection in CI builds (#5924)

issue: #5921

* [Improvement][Master]fix typo (#5934)

·fix typo in MasterBaseTaskExecThread

* [Fix-5886][server] Enhanced scheduler delete check (#5936)

* Add:Name verification remove the first and last spaces.

* Update: wrong word: 'WAITTING' ->'WAITING'

* Add: Strengthen verification

Co-authored-by: Squid <2824638304@qq.com>

* [Improvement-5880][api] Optimized data structure of pagination query API results (#5895)

* [5880][refactor]Optimized data structure of pagination query API results
- refactor PageInfo and delete returnDataListPaging in API
- modify the related Controller and Service and the corresponding Test

* Merge branch 'dev' of github.com:apache/dolphinscheduler into dev

 Conflicts:
	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java

Co-authored-by: 蔡泽华 <sorea1k@163.com>

* [IMPROVEMENT]fix mysql comment error (#5959)

* [Improvement][Api]fix typo (#5960)

* [Imporvement #5621][job instance] start-time and end-time (#5621)
·the list of workflow instances is sorted by start time and end time
·This closes #5621

* [FIX-5975]queryLastRunningProcess sql in ProcessInstanceMapper.xml (#5980)

* [NEW FEATURE][FIX-4385] compensation task add the ability to configure parallelism  (#5912)

* update

* web improved

* improve the ui

* add the ability to configure the parallelism

* update variables

* enhance the ut and add necessary note

* fix code style

* fix code style issue

* ensure the complation task in parallel mode can run the right numbers of tasks.

* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)

* [Bug][WorkerServer] SqlTask NullPointerException #5549

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428

* fix the readme typing issue (#5998)

* Fix unchecked type conversions

* Use indentation level reported by checkstyle

* Reorganize CI workflows to fasten the wasted time and resources (#6011)

* Add standalone server module to make it easier to develop (#6022)

* Task node of SWITCH (#5939)

* [Feature-#5273][server-master] Task node of SWITCH (#5922)



Co-authored-by: wangxj <wangxj31>

* remove description of bonecp (#6030)

Co-authored-by: shaojwu <shaojwu@ebay.com>

* [Improvement][Api Module]split alert group list-paging interface (#5941)

* [Improvement][Api Module]split alert group list-paging interface

* [FIX-#6007]Wrong complement date (#6026)

* [FIX-#6007]Wrong complement date

* [style]Wrong complement date

* [Improvement-6024][dist] Remove useless packaging commands (#6029)

·Remove useless packaging commands in dolphinscheduler-bin.xml

This closes #6024

Co-authored-by: mask <liuhu@zhiyoutec.com>

* [FIX-5908][MasterServer] When executing an compensation task, the execution thread would have a NPE (#5909)

* fix the npe in MasterExec

* fix the compile error

* Add `.asf.yaml` to easily set the GitHub metadata (#6035)

* fix dead server cannot stop (#6046)

* Enhancement Translation (#6042)

* replaced Loading... with i18n

* modified Edit zh_CN translation

* Delete zh_CN.js

Co-authored-by: David <dailidong66@gmail.com>

* fix bug #6053 zh_CN.js is lost

* [Fix-6038][ui] width of "SQL Statement" in Dag FormLineModal will be shrunk if sql line is too long (#6040)

This closes #6038

* [Improvement] Fix inefficient map iterator (#6004)

* Fix inefficient map iterator

* Use forEach and remove call to valueOf

* Modify AbstractParameters

* Enhance `StandaloneServer` so that we don't need to update the version number manually (#6074)

* Remove invalid character in `.asf.yaml` (#6075)

* Remove invalid character `\n` in `.asf.yaml` (#6077)

It turns out that the invalid character is `\n`

* Add alert server into standalone-server as well and some minor polish (#6087)

* Support starting standalone server in Docker image (#6102)

Also remove unused class

* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095)

* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)

* master refactor:
   1. spi for task submit and other actions(pause, kill)
   2. remove threads for process instance and task instance.
   3. add events for process instance and task instance

* ut npe

* add try catch

* code style

* fix critical bugs

* fix critical bugs

* fix critical bugs

* fix critical bugs

* Remove unused params in SwitchTaskTest (#6109)

* fix ut

* fix ut

* fix ut

* fix ut

* fix server module ut

* fix ut

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
Co-authored-by: AzureCN <colorazure@163.com>
Co-authored-by: 深刻 <tsund@qq.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Yao WANG <Yao.MR.CN@gmail.com>
Co-authored-by: ywang46 <ywang46@paypal.com>
Co-authored-by: The Gitter Badger <badger@gitter.im>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: Squidyu <1297554122@qq.com>
Co-authored-by: Squid <2824638304@qq.com>
Co-authored-by: soreak <60459867+soreak@users.noreply.github.com>
Co-authored-by: 蔡泽华 <sorea1k@163.com>
Co-authored-by: yimaixinchen <yimaixinchen@163.com>
Co-authored-by: atai-555 <74188560+atai-555@users.noreply.github.com>
Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: myangle1120 <942542838@qq.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: shaojwu <shaojwu@ebay.com>
Co-authored-by: Shukun Zhang <60541766+andream7@users.noreply.github.com>
Co-authored-by: linquan <1175687813@qq.com>
Co-authored-by: mask <39329477+Narcasserun@users.noreply.github.com>
Co-authored-by: mask <liuhu@zhiyoutec.com>
Co-authored-by: RichardStark <49977764+RichardStark@users.noreply.github.com>
Co-authored-by: lenboo <baoliang.leon@gmail.com>
Co-authored-by: lilyzhou <lj_zhou@outlook.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
Co-authored-by: junfan.zhang <zuston.shacha@gmail.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-09-07 17:15:05 +08:00
Kerwin 36d60a564a
[Feature][Task]Task plugin spi (#6116) 2021-09-07 15:04:18 +08:00
Hua Jiang d7af95f98c
[Feature-5987][Server] Support to set multiple environment configs for a certain worker. (#6082)
* support multi environments

* add some test cases

* add an environment vue component

* improve environment form

* improve environment form

* add environment worker group relation

* add environment worker group relation

* add the environment choice for formModel

* set an environment for the task

* modify the modal form of starting process

* add the environment config to TaskExecutionContext

* add the environment config to the timing form

* fix conflicts

* fix issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some issues of the code style

* fix some bugs in the code review

* add the same table and columns to support H2

* fix some bugs
2021-09-07 10:22:17 +08:00
OS e34c65d5a6
[Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6095)
* [Feature-4355][Master-Worker-API] improvements of master and scheduler module (#6085)

* master refactor:
   1. spi for task submit and other actions(pause, kill)
   2. remove threads for process instance and task instance.
   3. add events for process instance and task instance

* ut npe

* add try catch

* code style

* fix critical bugs

* fix critical bugs

* fix critical bugs

* fix critical bugs
2021-09-06 16:57:02 +08:00
Kirs 4715de11fc
[Feature][Plugin]Task (#6105)
* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)

* [Bug][WorkerServer] SqlTask NullPointerException #5549

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428

* fix the readme typing issue (#5998)

* Fix unchecked type conversions

* Use indentation level reported by checkstyle

* Reorganize CI workflows to fasten the wasted time and resources (#6011)

* Add standalone server module to make it easier to develop (#6022)

* Task node of SWITCH (#5939)

* [Feature-#5273][server-master] Task node of SWITCH (#5922)



Co-authored-by: wangxj <wangxj31>

* remove description of bonecp (#6030)

Co-authored-by: shaojwu <shaojwu@ebay.com>

* [Improvement][Api Module]split alert group list-paging interface (#5941)

* [Improvement][Api Module]split alert group list-paging interface

* [FIX-#6007]Wrong complement date (#6026)

* [FIX-#6007]Wrong complement date

* [style]Wrong complement date

* [Improvement-6024][dist] Remove useless packaging commands (#6029)

·Remove useless packaging commands in dolphinscheduler-bin.xml

This closes #6024

Co-authored-by: mask <liuhu@zhiyoutec.com>

* [FIX-5908][MasterServer] When executing an compensation task, the execution thread would have a NPE (#5909)

* fix the npe in MasterExec

* fix the compile error

* Add `.asf.yaml` to easily set the GitHub metadata (#6035)

* fix dead server cannot stop (#6046)

* Enhancement Translation (#6042)

* replaced Loading... with i18n

* modified Edit zh_CN translation

* Delete zh_CN.js

Co-authored-by: David <dailidong66@gmail.com>

* fix bug #6053 zh_CN.js is lost

* [Feature][Plugin]Task

* fix license head

* fix code style

* fix code style

* fix build err

Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: myangle1120 <942542838@qq.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: shaojwu <shaojwu@ebay.com>
Co-authored-by: Shukun Zhang <60541766+andream7@users.noreply.github.com>
Co-authored-by: linquan <1175687813@qq.com>
Co-authored-by: mask <39329477+Narcasserun@users.noreply.github.com>
Co-authored-by: mask <liuhu@zhiyoutec.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: RichardStark <49977764+RichardStark@users.noreply.github.com>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: lenboo <baoliang.leon@gmail.com>
2021-09-06 09:38:29 +08:00
JinyLeeChina 2f0a277206
[Feature][JsonSplit-api] update method of ProcessDefinition (#6089)
* refactor method of task save

* fix ut

* fix ut

* update method of processDefinition

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-09-02 15:55:50 +08:00
Anton Lyxell ca93b8a40c
[Improvement] Fix inefficient map iterator (#6004)
* Fix inefficient map iterator

* Use forEach and remove call to valueOf

* Modify AbstractParameters
2021-08-31 21:49:10 +08:00
JinyLeeChina ee4e64a9a0
[Feature][JsonSplit-api] refactor method of task save (#6067)
* refactor method of task save

* fix ut

* fix ut

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-08-31 15:17:06 +08:00
JinyLeeChina 920140028b
[Feature][JsonSplit-api] update of processInstance (#6061)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)



* add a new init method.

* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)

* add the necessary parameters

* openapi improve

* fix code style error

* [FIX-#5721][master-server] Global params parameter missing (#5757)



Co-authored-by: wangxj <wangxj31>

* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move`  doesn't work. (#5739)

* Update relatedItems.vue

* Update relatedItems.vue

* [Improvement#5741][Worker] Improve task process status log  (#5776)

* [Improvement-5773][server] need to support two parameters related to task (#5774)

* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>

* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE

* [Improvement][Worker] Task log may be lost #5775 (#5783)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file
  Upgrade checkstyle.xml to support checkstyle version 8.24+

* change ci checkstyle version

* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected.  (#5796)

* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.

* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)

·the list of workflow instances is sorted by start time and end time
·This closes #5621

* fix (#5803)

Co-authored-by: shuangbofu <fusb@tuya.com>

* fix: Remove duplicate "registryClient.close" method calls (#5805)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Improvement][SPI] support load single plugin (#5794)

change load operation of 'registry.plugin.dir'

* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)

* fix: refactor registry client, remove spring annotation

* fix UT

* fix UT

* fix checkstyle

* fix UT

* fix UT

* fix UT

* fix: Rename RegistryCenterUtils method name

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)

* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)

* enhance the validation in UI

* enchance form verifaction

* simplify disable condition

* fix: Remove unused class (#5833)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [fix-5737] [Bug][Datasource] datsource other param check error (#5835)

Co-authored-by: wanggang <wanggy01@servyou.com.cn>

* [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

[Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

* [Fix-5825][BUG][WEB] the resource tree in the process definition of latest dev branch can't display correctly (#5826)

* resoures-shows-error

* fix codestyle error

* add license header for new js

* fix codesmell

* [Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867)

* provide two system parameters to support the rest of type of tasks

* provide two system parameters to support the rest of type of tasks

* improve test conversion

* [Improvement][Fix-5769][UI]When we try to delete the existing dag, the console in web browser would shows exception (#5770)

* fix bug

* cache the this variable

* Avoid self name

* fix code style compile error

* [Fix-5781][UT] Fix test coverage in sonar (#5817)

* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781

* [Fix-5808][Server]  When we try to transfer data using datax between  different types of data sources, the worker will exit with ClassCastException (#5809)

* bug fix

* fix bug

* simplify the code format

* add a new parameter to make it easier to understand.

* [Fix-5830][Improvement][UI] Improve the selection style in dag edit dialog (#5829)

* improve the selection style

* update another file

* remove unnecessary css part.

* [Fix-5904][upgrade]fix dev branch upgrade mysql sql script error (#5821)

* fix dev branch upgrade mysql sql script error.

* Update naming convention.

* [Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832)

* fix: refactor api utils class, remove spring annotation.

* fix: Optimization comments

Co-authored-by: wen-hemin <wenhemin@apache.com>

* correct the wrong annotion from zk queue implemented to java priority blocking queue (#5906)

Co-authored-by: ywang46 <ywang46@paypal.com>

* Add a Gitter chat badge to README.md (#5883)

* Add Gitter badge

* Update README.md

Co-authored-by: David <dailidong66@gmail.com>

* ci: improve maven connection in CI builds (#5924)

issue: #5921

* [Improvement][Master]fix typo (#5934)

·fix typo in MasterBaseTaskExecThread

* [Fix-5886][server] Enhanced scheduler delete check (#5936)

* Add:Name verification remove the first and last spaces.

* Update: wrong word: 'WAITTING' ->'WAITING'

* Add: Strengthen verification

Co-authored-by: Squid <2824638304@qq.com>

* [Improvement-5880][api] Optimized data structure of pagination query API results (#5895)

* [5880][refactor]Optimized data structure of pagination query API results
- refactor PageInfo and delete returnDataListPaging in API
- modify the related Controller and Service and the corresponding Test

* Merge branch 'dev' of github.com:apache/dolphinscheduler into dev

 Conflicts:
	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java

Co-authored-by: 蔡泽华 <sorea1k@163.com>

* [IMPROVEMENT]fix mysql comment error (#5959)

* [Improvement][Api]fix typo (#5960)

* [Imporvement #5621][job instance] start-time and end-time (#5621)
·the list of workflow instances is sorted by start time and end time
·This closes #5621

* [FIX-5975]queryLastRunningProcess sql in ProcessInstanceMapper.xml (#5980)

* [NEW FEATURE][FIX-4385] compensation task add the ability to configure parallelism  (#5912)

* update

* web improved

* improve the ui

* add the ability to configure the parallelism

* update variables

* enhance the ut and add necessary note

* fix code style

* fix code style issue

* ensure the complation task in parallel mode can run the right numbers of tasks.

* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)

* [Bug][WorkerServer] SqlTask NullPointerException #5549

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428

* fix the readme typing issue (#5998)

* Fix unchecked type conversions

* Use indentation level reported by checkstyle

* Reorganize CI workflows to fasten the wasted time and resources (#6011)

* Add standalone server module to make it easier to develop (#6022)

* fix ut

* update of processInstance

* fix ut

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
Co-authored-by: AzureCN <colorazure@163.com>
Co-authored-by: 深刻 <tsund@qq.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Yao WANG <Yao.MR.CN@gmail.com>
Co-authored-by: ywang46 <ywang46@paypal.com>
Co-authored-by: The Gitter Badger <badger@gitter.im>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: Squidyu <1297554122@qq.com>
Co-authored-by: Squid <2824638304@qq.com>
Co-authored-by: soreak <60459867+soreak@users.noreply.github.com>
Co-authored-by: 蔡泽华 <sorea1k@163.com>
Co-authored-by: yimaixinchen <yimaixinchen@163.com>
Co-authored-by: atai-555 <74188560+atai-555@users.noreply.github.com>
Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-08-29 21:43:38 +08:00
wangxj3 2e1768aae6
[Feature-#5273][server-master] Task node of SWITCH (#5922)
Co-authored-by: wangxj <wangxj31>
2021-08-24 00:35:18 +08:00
JinyLeeChina 3b495d2fb0
[Feature][JsonSplit-api] merge code from dev to json2 (#6023)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)



* add a new init method.

* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)

* add the necessary parameters

* openapi improve

* fix code style error

* [FIX-#5721][master-server] Global params parameter missing (#5757)



Co-authored-by: wangxj <wangxj31>

* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move`  doesn't work. (#5739)

* Update relatedItems.vue

* Update relatedItems.vue

* [Improvement#5741][Worker] Improve task process status log  (#5776)

* [Improvement-5773][server] need to support two parameters related to task (#5774)

* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>

* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE

* [Improvement][Worker] Task log may be lost #5775 (#5783)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file
  Upgrade checkstyle.xml to support checkstyle version 8.24+

* change ci checkstyle version

* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected.  (#5796)

* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.

* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)

·the list of workflow instances is sorted by start time and end time
·This closes #5621

* fix (#5803)

Co-authored-by: shuangbofu <fusb@tuya.com>

* fix: Remove duplicate "registryClient.close" method calls (#5805)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Improvement][SPI] support load single plugin (#5794)

change load operation of 'registry.plugin.dir'

* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)

* fix: refactor registry client, remove spring annotation

* fix UT

* fix UT

* fix checkstyle

* fix UT

* fix UT

* fix UT

* fix: Rename RegistryCenterUtils method name

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)

* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)

* enhance the validation in UI

* enchance form verifaction

* simplify disable condition

* fix: Remove unused class (#5833)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [fix-5737] [Bug][Datasource] datsource other param check error (#5835)

Co-authored-by: wanggang <wanggy01@servyou.com.cn>

* [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

[Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

* [Fix-5825][BUG][WEB] the resource tree in the process definition of latest dev branch can't display correctly (#5826)

* resoures-shows-error

* fix codestyle error

* add license header for new js

* fix codesmell

* [Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867)

* provide two system parameters to support the rest of type of tasks

* provide two system parameters to support the rest of type of tasks

* improve test conversion

* [Improvement][Fix-5769][UI]When we try to delete the existing dag, the console in web browser would shows exception (#5770)

* fix bug

* cache the this variable

* Avoid self name

* fix code style compile error

* [Fix-5781][UT] Fix test coverage in sonar (#5817)

* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781

* [Fix-5808][Server]  When we try to transfer data using datax between  different types of data sources, the worker will exit with ClassCastException (#5809)

* bug fix

* fix bug

* simplify the code format

* add a new parameter to make it easier to understand.

* [Fix-5830][Improvement][UI] Improve the selection style in dag edit dialog (#5829)

* improve the selection style

* update another file

* remove unnecessary css part.

* [Fix-5904][upgrade]fix dev branch upgrade mysql sql script error (#5821)

* fix dev branch upgrade mysql sql script error.

* Update naming convention.

* [Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832)

* fix: refactor api utils class, remove spring annotation.

* fix: Optimization comments

Co-authored-by: wen-hemin <wenhemin@apache.com>

* correct the wrong annotion from zk queue implemented to java priority blocking queue (#5906)

Co-authored-by: ywang46 <ywang46@paypal.com>

* Add a Gitter chat badge to README.md (#5883)

* Add Gitter badge

* Update README.md

Co-authored-by: David <dailidong66@gmail.com>

* ci: improve maven connection in CI builds (#5924)

issue: #5921

* [Improvement][Master]fix typo (#5934)

·fix typo in MasterBaseTaskExecThread

* [Fix-5886][server] Enhanced scheduler delete check (#5936)

* Add:Name verification remove the first and last spaces.

* Update: wrong word: 'WAITTING' ->'WAITING'

* Add: Strengthen verification

Co-authored-by: Squid <2824638304@qq.com>

* [Improvement-5880][api] Optimized data structure of pagination query API results (#5895)

* [5880][refactor]Optimized data structure of pagination query API results
- refactor PageInfo and delete returnDataListPaging in API
- modify the related Controller and Service and the corresponding Test

* Merge branch 'dev' of github.com:apache/dolphinscheduler into dev

 Conflicts:
	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java

Co-authored-by: 蔡泽华 <sorea1k@163.com>

* [IMPROVEMENT]fix mysql comment error (#5959)

* [Improvement][Api]fix typo (#5960)

* [Imporvement #5621][job instance] start-time and end-time (#5621)
·the list of workflow instances is sorted by start time and end time
·This closes #5621

* [FIX-5975]queryLastRunningProcess sql in ProcessInstanceMapper.xml (#5980)

* [NEW FEATURE][FIX-4385] compensation task add the ability to configure parallelism  (#5912)

* update

* web improved

* improve the ui

* add the ability to configure the parallelism

* update variables

* enhance the ut and add necessary note

* fix code style

* fix code style issue

* ensure the complation task in parallel mode can run the right numbers of tasks.

* [Improvement][dao]When I search for the keyword description, the web UI shows empty (#5952)

* [Bug][WorkerServer] SqlTask NullPointerException #5549

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword Modify User, the web UI shows empty #5428

* [Improvement][dao]When I search for the keyword description, the web UI shows empty #5428

* fix the readme typing issue (#5998)

* Fix unchecked type conversions

* Use indentation level reported by checkstyle

* Reorganize CI workflows to fasten the wasted time and resources (#6011)

* Add standalone server module to make it easier to develop (#6022)

* fix ut

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
Co-authored-by: AzureCN <colorazure@163.com>
Co-authored-by: 深刻 <tsund@qq.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Yao WANG <Yao.MR.CN@gmail.com>
Co-authored-by: ywang46 <ywang46@paypal.com>
Co-authored-by: The Gitter Badger <badger@gitter.im>
Co-authored-by: David <dailidong66@gmail.com>
Co-authored-by: Squidyu <1297554122@qq.com>
Co-authored-by: Squid <2824638304@qq.com>
Co-authored-by: soreak <60459867+soreak@users.noreply.github.com>
Co-authored-by: 蔡泽华 <sorea1k@163.com>
Co-authored-by: yimaixinchen <yimaixinchen@163.com>
Co-authored-by: atai-555 <74188560+atai-555@users.noreply.github.com>
Co-authored-by: didiaode18 <563646039@qq.com>
Co-authored-by: Roy <yongjuncao1213@gmail.com>
Co-authored-by: lyxell <alyxell@kth.se>
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-08-23 16:23:40 +08:00
kezhenxu94 bf90106d0c
Add standalone server module to make it easier to develop (#6022) 2021-08-23 13:25:33 +08:00
lyxell db6db88b6f Use indentation level reported by checkstyle 2021-08-19 17:21:39 +02:00
lyxell 0206a0e9f1 Fix unchecked type conversions 2021-08-18 14:49:42 +02:00
Kerwin 79952d926f
[Feature][SPI] Task (#5996) 2021-08-18 10:32:19 +08:00
Kerwin 0344b21dbc
[json_split_two]cherry pick [Improvement-5880][api] Optimized data structure of pagination query API results (#5895). (#5966)
* cherry pick [Improvement-5880][api] Optimized data structure of pagination query API results (#5895).

* check code style.

* update test class.

* Comment merge adjustment.

Co-authored-by: soreak <60459867+soreak@users.noreply.github.com>
2021-08-10 17:18:47 +08:00
soreak f567517031
[Improvement-5880][api] Optimized data structure of pagination query API results (#5895)
* [5880][refactor]Optimized data structure of pagination query API results
- refactor PageInfo and delete returnDataListPaging in API
- modify the related Controller and Service and the corresponding Test

* Merge branch 'dev' of github.com:apache/dolphinscheduler into dev

 Conflicts:
	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java

Co-authored-by: 蔡泽华 <sorea1k@163.com>
2021-08-03 12:30:55 +08:00
JinyLeeChina 7427dc6fd3
[Feature][JsonSplit-api]merge code from dev to json_split_two (#5923)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)



* add a new init method.

* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)

* add the necessary parameters

* openapi improve

* fix code style error

* [FIX-#5721][master-server] Global params parameter missing (#5757)



Co-authored-by: wangxj <wangxj31>

* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move`  doesn't work. (#5739)

* Update relatedItems.vue

* Update relatedItems.vue

* [Improvement#5741][Worker] Improve task process status log  (#5776)

* [Improvement-5773][server] need to support two parameters related to task (#5774)

* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>

* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE

* [Improvement][Worker] Task log may be lost #5775 (#5783)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file
  Upgrade checkstyle.xml to support checkstyle version 8.24+

* change ci checkstyle version

* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected.  (#5796)

* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.

* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)

·the list of workflow instances is sorted by start time and end time
·This closes #5621

* fix (#5803)

Co-authored-by: shuangbofu <fusb@tuya.com>

* fix: Remove duplicate "registryClient.close" method calls (#5805)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Improvement][SPI] support load single plugin (#5794)

change load operation of 'registry.plugin.dir'

* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)

* fix: refactor registry client, remove spring annotation

* fix UT

* fix UT

* fix checkstyle

* fix UT

* fix UT

* fix UT

* fix: Rename RegistryCenterUtils method name

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)

* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)

* enhance the validation in UI

* enchance form verifaction

* simplify disable condition

* fix: Remove unused class (#5833)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [fix-5737] [Bug][Datasource] datsource other param check error (#5835)

Co-authored-by: wanggang <wanggy01@servyou.com.cn>

* [Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

[Fix-5719][K8s] Fix Ingress tls: got map expected array On TLS enabled On Kubernetes

* [Fix-5825][BUG][WEB] the resource tree in the process definition of latest dev branch can't display correctly (#5826)

* resoures-shows-error

* fix codestyle error

* add license header for new js

* fix codesmell

* [Improvement-5852][server] Support two parameters related to task for the rest of type of tasks. (#5867)

* provide two system parameters to support the rest of type of tasks

* provide two system parameters to support the rest of type of tasks

* improve test conversion

* [Improvement][Fix-5769][UI]When we try to delete the existing dag, the console in web browser would shows exception (#5770)

* fix bug

* cache the this variable

* Avoid self name

* fix code style compile error

* [Fix-5781][UT] Fix test coverage in sonar (#5817)

* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781

* [Fix-5808][Server]  When we try to transfer data using datax between  different types of data sources, the worker will exit with ClassCastException (#5809)

* bug fix

* fix bug

* simplify the code format

* add a new parameter to make it easier to understand.

* [Fix-5830][Improvement][UI] Improve the selection style in dag edit dialog (#5829)

* improve the selection style

* update another file

* remove unnecessary css part.

* [Fix-5904][upgrade]fix dev branch upgrade mysql sql script error (#5821)

* fix dev branch upgrade mysql sql script error.

* Update naming convention.

* [Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832)

* fix: refactor api utils class, remove spring annotation.

* fix: Optimization comments

Co-authored-by: wen-hemin <wenhemin@apache.com>

* correct the wrong annotion from zk queue implemented to java priority blocking queue (#5906)

Co-authored-by: ywang46 <ywang46@paypal.com>

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
Co-authored-by: AzureCN <colorazure@163.com>
Co-authored-by: 深刻 <tsund@qq.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Yao WANG <Yao.MR.CN@gmail.com>
Co-authored-by: ywang46 <ywang46@paypal.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-31 11:19:41 +08:00
JinyLeeChina 93abae94e4
[Feature][JsonSplit-api] export/import of ProcessDefinition api (#5910)
* modify viewTree of ProcessDefiniton

* modify ut

* modify ut

* modify viewTree of ProcessDefiniton

* export/import of ProcessDefinition api

* fix ut

* fix ut

* fix codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-29 16:49:52 +08:00
wen-hemin 72d68041e3
[Improvement][Api Module] refactor DataSourceParam and DependentParam, remove spring annotation (#5832)
* fix: refactor api utils class, remove spring annotation.

* fix: Optimization comments

Co-authored-by: wen-hemin <wenhemin@apache.com>
2021-07-29 11:36:10 +08:00
深刻 4c0993cdf4
[Fix-5781][UT] Fix test coverage in sonar (#5817)
* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781
2021-07-26 22:04:36 +08:00
geosmart 2df6ee1efb
[fix-5737] [Bug][Datasource] datsource other param check error (#5835)
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
2021-07-18 21:46:58 +08:00
JinyLeeChina 13c607eeb9
[Feature][JsonSplit-api]merge code from dev to json_2 (#5827)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

* [Fix-5699][UI] Fix update user error in user information (#5700)

* [Improvement] the automatically generated spi service name in alert-plugin is wrong (#5676)

* bug fix

the auto generated spi service can't be recongized



* include a  new method

* [Improvement-5622][project management] Modify the title (#5723)

* [Fix-5714] When updating the existing alarm instance, the creation time should't be updated (#5715)



* add a new init method.

* [Fix#5758] There are some problems in the api documentation that need to be improved (#5759)

* add the necessary parameters

* openapi improve

* fix code style error

* [FIX-#5721][master-server] Global params parameter missing (#5757)



Co-authored-by: wangxj <wangxj31>

* [Fix-5738][UI] The cancel button in the pop-up dialog of `batch copy` and `batch move`  doesn't work. (#5739)

* Update relatedItems.vue

* Update relatedItems.vue

* [Improvement#5741][Worker] Improve task process status log  (#5776)

* [Improvement-5773][server] need to support two parameters related to task (#5774)

* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>

* [FIX-5786][Improvement][Server] When the Worker turns down, the MasterServer cannot handle the Remove event correctly and throws NPE

* [Improvement][Worker] Task log may be lost #5775 (#5783)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file (#5789)

* [Imporvement #5725][CheckStyle] upgrade checkstyle file
  Upgrade checkstyle.xml to support checkstyle version 8.24+

* change ci checkstyle version

* [Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected.  (#5796)

* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.

* [Imporvement #5621][job instance] start-time and end-time (#5621) (#5797)

·the list of workflow instances is sorted by start time and end time
·This closes #5621

* fix (#5803)

Co-authored-by: shuangbofu <fusb@tuya.com>

* fix: Remove duplicate "registryClient.close" method calls (#5805)

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Improvement][SPI] support load single plugin (#5794)

change load operation of 'registry.plugin.dir'

* [Improvement][Api Module] refactor registry client, remove spring annotation (#5814)

* fix: refactor registry client, remove spring annotation

* fix UT

* fix UT

* fix checkstyle

* fix UT

* fix UT

* fix UT

* fix: Rename RegistryCenterUtils method name

Co-authored-by: wen-hemin <wenhemin@apache.com>

* [Fix-5699][UI] Fix update user error in user information introduced by #5700 (#5735)

* [Fix-5726] When we used the UI page, we found some problems such as parameter validation, parameter update shows success but actually work (#5727)

* enhance the validation in UI

* enchance form verifaction

* simplify disable condition

* fix conflicts

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: Shiwen Cheng <chengshiwen0103@gmail.com>
Co-authored-by: Jianchao Wang <akingchao@qq.com>
Co-authored-by: Tanvi Moharir <74228962+tanvimoharir@users.noreply.github.com>
Co-authored-by: Hua Jiang <jianghuachinacom@163.com>
Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
Co-authored-by: Wenjun Ruan <861923274@qq.com>
Co-authored-by: Tandoy <56899730+Tandoy@users.noreply.github.com>
Co-authored-by: 傅双波 <786183073@qq.com>
Co-authored-by: shuangbofu <fusb@tuya.com>
Co-authored-by: wen-hemin <39549317+wen-hemin@users.noreply.github.com>
Co-authored-by: wen-hemin <wenhemin@apache.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-16 14:55:08 +08:00
kyoty 7bffe0ac85
[Fix-5795][Improvement][Server] The starttime field in the HttpTask log is not displayed as expected. (#5796)
* improve timestamp format

make the startime in the log of httptask to be easier to read.


* fix bad code smell and update the note.
2021-07-12 14:31:48 +08:00
Hua Jiang 9fd5145b66
[Improvement-5773][server] need to support two parameters related to task (#5774)
* add some new parameter for task

* restore official properties

* improve imports

* modify a variable's name

Co-authored-by: jiang hua <jiang.hua@zhaopin.com.cn>
2021-07-09 17:00:32 +08:00
JinyLeeChina 14035c03e2
[Feature][JsonSplit-api]move of processDefinition, the projectCode instead of projectName (#5744)
* select-by-code of processDefinition

* fix ut

* prcessDefinition copy

* move of processDefinition api and the projectCode instead of projectName

* fix ut

* fix ut

* fix review

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-05 18:04:17 +08:00
JinyLeeChina c386b64500
[Feature][JsonSplit-api] copy of processDefinition (#5740)
* select-by-code of processDefinition

* fix ut

* prcessDefinition copy

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-07-03 14:10:16 +08:00
JinyLeeChina 57414c4df7
[Feature][JsonSplit-api] merging from dev to json_split_two (#5712)
* [BUG-#5678][Registry]fix registry init node miss (#5686)

* [Improvement][UI] Update the update time after the user information is successfully modified (#5684)

* improve

edit the userinfo success, but the updatetime is not the latest.

* Improved shell task execution result log information, adding process.waitFor() and process.exitValue() information to the original log (#5691)

Co-authored-by: shenglm <shenglm840722@126.com>

* [Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)

* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>

* Issue robot translation judgment changed to Chinese (#5694)



Co-authored-by: chenxingchun <438044805@qq.com>

* the update function should use post instead of get (#5703)

* enhance form verify (#5696)

* checkState only supports %s not {} (#5711)

* [Fix-5701]When deleting a user, the accessToken associated with the user should also be deleted (#5697)

* update

* fix the codestyle error

* fix the compile error

* support rollback

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: kyoty <echohlne@gmail.com>
Co-authored-by: ji04xiaogang <ji04xiaogang@163.com>
Co-authored-by: shenglm <shenglm840722@126.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-06-29 22:55:01 +08:00
wangxj3 6df58e8d78
[Feature-#5565][Master Worker-Server] Global Param passed by sense dependencies (#5603)
* add globalParams new plan with varPool

* add unit test

* add python task varPoolParams


Co-authored-by: wangxj <wangxj31>
2021-06-25 11:37:23 +08:00
Squidyu 664206cb81
wrong word: 'WAITTING' ->'WAITING' (#5682)
* Add:Name verification remove the first and last spaces.

* Update: wrong word: 'WAITTING' ->'WAITING'

Co-authored-by: Squid <2824638304@qq.com>
2021-06-22 21:56:21 +08:00
kyoty 75813e7866
[bug fix] Some password attributes can be viewd directly in the log (#5638)
* fix the password display incorrectly

* update

* update

* fic code style

fic code style

* Update SensitiveDataConverterTest.java

* Update SensitiveDataConverter.java

* Update SensitiveDataConverter.java

* Update SensitiveDataConverterTest.java

* Update pom.xml
2021-06-18 19:10:57 +08:00
zhuangchong 3026f04d85
[Improvement-5452][Task] ds flink task support submit a PyFlink job via the CLI (#5453)
* flink task support submit a PyFlink job via the CLI.

* optimize attribute name.

* Modify pyflink parameter judgment logic
2021-06-17 15:19:25 +08:00
kyoty 67711442d5
[Improvement][SQL] Query return number In SQL should be configurable (#5632)
* [Improvement][SQL] Query return number should be configurable #5580

* Update SqlTaskTest.java

* Update sql.vue
2021-06-16 09:40:21 +08:00
kyoty 71a44f8a57 [Fix-5483] [Bug][API] Can't view variables in the page of Process Instance (#5631) 2021-06-15 10:30:26 +08:00
blackberrier 053e548bf3 [Improvement-5539][Master] Check status of taskInstance from cache (#5572)
* improvement:check status of taskInstance from cache

* issue5572 use timer instead of while&sleep; consider concurrent modification

* use computeifpresent instead of lock

* simplify getByTaskInstanceId function

* add ut for TaskInstanceCacheManagerImpl; fix bug in TaskInstanceCacheManagerImpl

* add Apache license header;add test class in root pom
2021-06-15 10:30:26 +08:00
Kirs 30c88032d3 [Feature-#3961][Registry]Registry-SPI (#5562)
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-15 10:30:26 +08:00
Shiwen Cheng d8f2aa3f20 [Fix-5468][Common] Fix obtaining IP is incorrect (#5594) 2021-06-15 10:30:26 +08:00
kyoty 0d5037e7c3
[Fix-5483] [Bug][API] Can't view variables in the page of Process Instance (#5631) 2021-06-13 11:43:53 +08:00
blackberrier 79eb2e85d7
[Improvement-5539][Master] Check status of taskInstance from cache (#5572)
* improvement:check status of taskInstance from cache

* issue5572 use timer instead of while&sleep; consider concurrent modification

* use computeifpresent instead of lock

* simplify getByTaskInstanceId function

* add ut for TaskInstanceCacheManagerImpl; fix bug in TaskInstanceCacheManagerImpl

* add Apache license header;add test class in root pom
2021-06-10 09:39:12 +08:00
Kirs e2243d63be
[Feature-#3961][Registry]Registry-SPI (#5562)
[Feature#3961]Registry SPI

All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
2021-06-09 15:21:02 +08:00
Shiwen Cheng 281b5aea6b
[Fix-5468][Common] Fix obtaining IP is incorrect (#5594) 2021-06-07 22:12:49 +08:00
Shiwen Cheng b537f4eb26
[Improvement][Docker/K8s] Support more configurations and service accesses, reduce the duplication configurations, improve image for python and the compatibility of python home (#5393)
* [Improvement][Docker] Support more configurations and improve image for python

* [Improvement][K8s] Support more configurations, more service accesses, and reduce the duplication configurations

* [Improvement][Python] Improve the compatibility of python home

* [Improvement][Install] Fix install config and install.sh

* [Improvement][Install] Fix workflow ut

* [Improvement][Docker] Optimize docker-swarm/check script

* [Improvement][DB] Update default username and password of database

* Update comments in master/worker.properties

* Specify version for all images

* [Improvement][Docker] Optimize PS1 and WORKDIR

* [Improvement][SQL] Reduce heap size to 64m in create-dolphinscheduler.sh and upgrade-dolphinscheduler.sh

* [Fix-5431][K8s] Fix master and worker cannot get the right address with custom DNS
2021-05-24 00:03:35 +08:00
zhuangchong 9ba4ffbe48
[Improvement-5527][api-server] failed find any kerberos (#5533)
* fix failed find any kerberos.

* increase code coverage.

* update common utils test.

* update common utils test.

* update common utils test.

* fix exception type.
2021-05-23 23:43:59 +08:00
QuakeWang 46660b58ed
[Test] Add SparkParameters test (#5478)
* Create SparkparametersTest.java

Add a test case

* Update and rename SparkparametersTest.java to SparkParametersTest.java

* Update pom.xml

add SparkParametersTest.java in pom file
2021-05-22 23:56:39 +08:00
zhuangchong 1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization (#5441)
* Optimize dependencies between modules.

* update module-alert dependency.

* add repository in the pom xml.

* update Central Repository in the pom.xml.
2021-05-19 10:05:00 +08:00
Yao WANG bc22ae7c91
[Improvement][Task] Remove deprecated TaskRecordDao and simply the after() method in the AbstractTask class (#5492)
* remove the depreated data quality module

* fix lint

Co-authored-by: Yao <ywang46@paypal.com>
2021-05-18 17:00:03 +08:00
ruanwenjun a925f64571
[Feature][api] schedule add timezone support #5259 (#5465)
* [Feature][api] schedule add timezone support #5259

* import moment-timezone

* fix code smell
2021-05-17 21:08:20 +08:00
ruanwenjun 68301db6b9
[Improvement][Api] Upload resource to remote failed, the local tmp file need to be cleared #5475 (#5476) 2021-05-17 11:13:14 +08:00
lcdrs d04f4b6053
[Implement][Common] Modify the plug-in loading log to print messages
Co-authored-by: caoping <caoping@staff.weibo.com>
2021-05-13 16:26:21 +08:00
baoliang 156dadac69 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
2021-05-11 10:08:28 +08:00
zhuangchong 9c77faa8ac
[Improvement-5058][Master] The master-server creates the DB data source twice (#5139)
* reduce the number of data sources created twice by master server.

* update ProcessAlertManagerTest test class to increase coverage.

* add DolphinSchedulerPluginLoaderTest test ignore.

Co-authored-by: zhuang chong
2021-05-10 09:56:17 +08:00
JinyLeeChina f7b71fb9ca
[Feature][JsonSplit] Fix codeStyle for json_split (#5432)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

* fix process lineage bug

* fix codeStyle for json_split

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-05-09 00:01:09 +08:00
baoliang 1646188027 fix code style 2021-04-27 14:21:30 +08:00
bao liang 86dcddb32b
[Feature-4417]fix code style (#5395)
* fix code style

* fix ut

* remove unused ut

* remove unused ut

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2021-04-27 14:04:13 +08:00
JinyLeeChina 2a44cb9de6
[Feature][JsonSplit] update codeStyle for merge to dev (#5392)
* update taskParams/add task delayTime/fix conditionType bug

* update codeStyle for merge to dev

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-26 22:30:35 +08:00
baoliang 746d9eca1d Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionControllerTest.java
#	dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessDefinitionVersionServiceTest.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/model/TaskNode.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterBaseTaskExecThread.java
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/processdure/ProcedureTask.java
#	dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
2021-04-26 14:14:05 +08:00
JinyLeeChina c3558965c1
update taskParams/add task delayTime/fix conditionType bug (#5385)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-26 11:28:35 +08:00
ruanwenjun e6d8da484f
[Improvement][API] Create Datasource #5083 (#5116)
* [Improvement][API] Create Datasource #5083

  * Refact create/ update datasource API

* fix Code Smells

* fix code smell

* resolve code smell

* use ConnectionParam to replace BaseDatasource

* solve license

* add ut

* code check

* add ut

* fix ut coverage

* fix ut
2021-04-25 13:51:59 +08:00
bao liang 75d1ad645c
fix bug: 5349, work flow would running if task is killed by manual. (#5362)
Co-authored-by: baoliang <baoliang@analysys.com.cn>
2021-04-23 14:29:26 +08:00
luoyuan e57ee658e8
[Improvement] Optimized close of IO (#5336)
* [improvement]modify all methods that refer to LogClientService in the code.

* delete unused code

* [Improvement] Optimized close of IO
2021-04-21 22:47:16 +08:00
JinyLeeChina feffd972b5
[Feature][JsonSplit] Fix task logger path (#5331)
* update SnowFlake

* update processDefinite from processInstance

* update processDefinite from processInstance

* Fix task logger path

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-21 17:19:09 +08:00
Wenjun 5f1fc462c9
[Bug][server-api] copy the node ,the preTasks is error #5261 (#5264) 2021-04-20 15:17:17 +08:00
JinyLeeChina 49b234609a
update SnowFlake (#5321)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-19 23:12:09 +08:00
roi 837c9ba2cc
fix bug:when shell task raw script has $[${day} - 1],throw java.lang.IllegalArgumentException::Pattern includes reserved character:'{' (#5313) 2021-04-19 17:50:29 +08:00
Shiwen Cheng 83519bdfc3
[Improvement][Server] Optimize log display (#5311) 2021-04-19 10:29:32 +08:00
Shiwen Cheng e92e29ef9a
[Fix-5309][Common] Fix the wrong memory (#5312)
* [Fix-5309][Common] Fix the wrong memory

* [Fix-5309][Common] Fix unit test and remove unused code
2021-04-19 10:24:54 +08:00
JinyLeeChina f7a06a033d
[Feature][JsonSplit] Fix master/worker about shellTask (#5300)
* Fix quoted bug about processDefineId of processInstance

* Fix master/worker about shellTask

* codeStyle

* codeStyle

* update taskType

* codeStyle

* update timeoutNotifyStrategy

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-18 12:45:30 +08:00
wenjun f3dfaacca2
[Improvement][Service] Optimize query log (#5205)
* [Improvement][Service] Optimize query log

* remove IPUtils
2021-04-15 20:25:01 +08:00
Shiwen Cheng 1c1f6663a6
[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select (#5182)
* [Improvement][Server] Optimize and reduce database and ZooKeeper query

* [Improvement][WorkerGroup] Replace worker address input with multiple-select

* [Improvement][WorkerGroup] Fix unit test

* [Improvement][API&UI] Optimize long host to short host and long host display in k8s
2021-04-14 21:39:39 +08:00
baoliang 0166aebf71 Merge remote-tracking branch 'upstream/dev' into json_split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/enums/Status.java
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/StringUtils.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.java
#	dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectMapper.xml
#	dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/process/ProcessServiceTest.java
2021-04-13 23:10:19 +08:00
didiaode18 0eb8c0c234
[Improvement][common]Task status error 2021-04-12 19:32:57 +08:00
JinyLeeChina 73f5a8e0a7
[Feature][JsonSplit] Fix master/worker server (#5253)
* Fix dependency failure

* Fix taskInstance

* task instance list page

* code review

* fix logger path

* Fix master/worker server

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-12 10:53:22 +08:00
JinyLeeChina 31188ffca0
[Feature][JsonSplit] Fix taskInstance (#5246)
* Fix dependency failure

* Fix taskInstance

* task instance list page

* code review

* fix logger path

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-11 16:52:19 +08:00
zhuangchong 73e917d3c1
Clean up useless files. (#5252) 2021-04-11 14:07:58 +08:00
JinyLeeChina 8c88bfa222
Fix dependency failure (#5239)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-09 11:25:18 +08:00
xiaoyu 2f1f193ba8
Update common.properties (#5223) 2021-04-07 16:17:21 +08:00
bao liang 86ce8f133a
[JsonSplit-4417][master/worker/api]json split- remove json in task instance. (#5160)
* [json split] refactor json in task instance

* add task_pramas in task_instance

* feature #4417 refactor json split in task instance

* code style

* code style

* code style

* code style

* code style

* update

* update

* update unit test

* update
2021-04-03 17:58:04 +08:00
JinyLeeChina dd36da8d1f
[Feature][JsonSplit] fix process bug (#5193)
* transform taskCode from long to string

* fix process bug

* code review

* code review

* code review

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-04-01 17:45:44 +08:00
zhuangchong e53369318b
[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal (#4722)
* fix the shell starts the yarn task in the background

* update StringUtils code style.

* solve code smell.

* add method comment in StringUtils class.

* update AlertGroupMapper code style.

* update AlertGroupMapperTest

* update sql script test.
2021-03-30 22:33:49 +08:00
lenboo c8ac6f99fe Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-common/src/main/resources/common.properties
2021-03-30 11:31:42 +08:00
JinyLeeChina 689106d114
transform taskCode from long to string (#5174)
Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-30 11:07:28 +08:00
JinyLeeChina a467fffd7b
[Feature][JsonSplit] fix taskId in json (#5165)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

* fix taskId in processDefinitionJson

* fix json taskId

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-29 14:22:40 +08:00
guohaozhang a6a1b94df4
[Feature-#5128]Support turning off sudo permissions (#5129)
* [Feature-#5128]Support turning off sudo permissions

* [Feature-#5128] without sudo permission do not create tenant user
2021-03-29 11:07:39 +08:00
Shiwen Cheng 492b318bd3
[Fix][Server] Fix clear task execute path is related to master (#5123) 2021-03-26 10:11:56 +08:00
lenboo 0914699fc5 Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into spilit 2021-03-26 00:06:26 +08:00
lenboo f05ba84968 Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.java
#	dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapperTest.java
2021-03-25 23:54:01 +08:00
Simon cd19507271
[feature][jsonsplit]fix copy/export/import processDefinition bugs (#5146)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style

* repalce project_id with code

* conflicts solve

* conflicts solve

* conflicts solve

* bugfix

* modify listResources mothod and remove getResourceIds mothod

* 1

* conflicts solve

* modify listResources mothod and remove getResourceIds mothod

* modify listResources mothod and remove getResourceIds mothod

* replace processDefinitionVersion with processDefinitionLog

* codestyle

* codestyle

* add mapper module ut

* codestyle

* fix ProcessInstanceMapperTest

* codestyle

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* conflicts solve

* fix ProcessInstanceMapperTest

* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest

* fix ProjectMapperTest/ProjectUserMapperTest/ScheduleMapperTest

* fix TaskInstanceMapperTest

* add TaskDefinitionLogMapperTest/TaskDefinitionMapperTest and bugfix

* codestyle

* codestyle

* Refactor process lineage

* Refactor process lineage

* codestyle

* codestyle

* Refactor process lineage

* Refactor process lineage

* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest

* add WorkFlowLineageMapperTest/WorkFlowLineageServiceTest

* add ProcessServiceTest

* 1

* 1

* fix copy/export/import processDefinition bugs

* 1

* codestyle
2021-03-25 18:27:47 +08:00
JinyLeeChina 1eaf1f6234
[Feature][JsonSplit] fix taskId in processDefinitionJson (#5136)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

* fix taskId in processDefinitionJson

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-24 11:44:14 +08:00
wenjun 5d264c9f20
[Improvement][Api] Optimize query processInstanceList #5134 (#5135)
* Batch query user rather than several query
2021-03-23 23:48:13 +08:00
Shiwen Cheng 0d065c0712
[Feature-5062][*] Create/Edit/Delete Worker Group in Web UI (#5113)
* [Improvement][Server] Rename MASTER_PREFIX to MASTER_TYPE and WORKER_PREFIX to WORKER_TYPE

* [Feature][Api&Dao] Support api and dao of worker group

* [Feature][UI] Support to create, edit and delete worker group

* [Feature][API&UI] Support work groups from database and zookeeper to display together in worker group management

* [Feature][API&Dao] Support long ip list and check request parameters when saving worker group

* [Feature][*] Refactor ipList to addrList

* [Feature][Server] Support host manager to select worker host by worker group in database

* [Improvement][Server] Improve the config of master and worker

* [Improvement][*] Rename zkRegistered to systemDefault

* [Improvement][Test] Fix unit test and improve ui

* [Improvement][Server] Improve getServerMaps

Co-authored-by: dailidong <dailidong66@gmail.com>
2021-03-23 23:47:41 +08:00
JinyLeeChina d99934d0da
[Feature][JsonSplit] Fix api bug (#5102)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

* Fix api bug"

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-23 10:44:25 +08:00
lenboo 010b49c933 merge from dev 2021-03-22 15:18:16 +08:00
Shiwen Cheng 8ac72e80e6
[Feature-5087][SqlTask] Add a switch to send mail and print head logs in SqlTask (#5088)
* [Feature-5087][SqlTask] Add the switch to send mail and print head logs in SqlTask

* [Feature-5087][SqlTask] Add SqlParametersTest
2021-03-19 15:07:39 +08:00
wenjun 086e716441
[Improvement][Dao] SQL optimization - check exist #5069 (#5070) 2021-03-18 22:49:12 +08:00
Shiwen Cheng f94cfc620d
[Feature][Worker] Add a configuration item to set whether the tenant is automatically created on Worker (#5007) 2021-03-18 18:34:42 +08:00
wangxj3 29d42fd92d
[Feature-#3805][server-master] Task parameter transfer (#5077)
* fix out param format bug

Co-authored-by: wangxj <wangxj31>
2021-03-18 11:50:10 +08:00
Shiwen Cheng 5856a12855
[Improvement-4984][Worker] Refactor and Improve worker load balance (#4996)
* [Improvement][Remote] Improve unit tests

* [Improvement][Worker] Improve worker load balance

* [Improvement][Worker] Fix code smells

* [Improvement][Worker] Rename weight to hostWeight
2021-03-18 10:03:58 +08:00
Ts686 6216817861
[Improvement-4971][alert|dao|service]Remove redundant PropertyUtils.class (#5020)
* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.

* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.

* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.

* [Improvement][alert|dao|service]
1. Remove redundant PropertyUtils.class (#4971)
2. Remove useless import package.
2021-03-17 15:28:07 +08:00
Shiwen Cheng 1cd62b4a5e
[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py (#5019) 2021-03-17 14:38:04 +08:00
lenboo cf63fc1fee Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-16 15:50:06 +08:00
JinyLeeChina d98c2c5cfe
[Feature][JsonSplit] Fix bug in interface of queryProcessDefinitionList (#5043)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

* Fix logical bug in saving process definition

* codeSytle

* Fix bug in interface of  queryProcessDefinitionList

* codeSytle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-16 15:44:03 +08:00
Kirs 6bf5031f45
[FIX] fix MySQL datasource jdbc connect parameters (#4851)
* [FIX-CVE-2020-11974] fix MySQLDataSource Security

* remove useless code

* add ut

* A more reasonable way

* code style

* fix test error

* code style

* update exception msg
2021-03-16 14:40:07 +08:00
lenboo a574d2e38c Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-12 10:44:43 +08:00
Shiwen Cheng ae608e024a
[Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping (#5038)
* [Fix-5037][Server] Fix that both the master and the worker is hanging after restarting and stopping

* [Improvement][*] Replace commons.lang.StringUtils with common.utils.StringUtils
2021-03-12 09:33:30 +08:00
zhuangchong c8e31ad58e
[Improvement-4884][Common] fix the subsequent processing of getting the YARN ResourceManager address to be empty (#4885)
* fix yarn application url generate empty.

* removing Excess Processing

* update hadoop utils test class .

* add base exception class.

* update hadoop utils class code style.
2021-03-11 22:14:52 +08:00
wenjun ea6b1de120
[Improvement][worker] Optimize KillYarnJob (4939) (#4943) 2021-03-11 22:13:44 +08:00
lenboo 6ef54ebe9b Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-11 17:01:29 +08:00
Shiwen Cheng beb5adb985
[Feature][MR] Support mapreduce name (#5029) 2021-03-11 12:35:08 +08:00
lenboo 81dcd6873d Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-09 18:19:08 +08:00
Shiwen Cheng e74932cfc2
[Feature-4960][Spark] Add name input for Spark and improve Spark & MR args (#4959)
* [Improvement][Spark] Improve spark args and add name input

* [Improvement][MR] Improve map reduce args

* [Improvement][Spark] Fix check style

* [Improvement][Spark] Fix check style for spark, flink and mr
2021-03-09 12:51:35 +08:00
JinyLeeChina 7f0678d3be
[Feature][JsonSplit] Fix save process bug (#5001)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

* fix save process bug

* codeStyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-08 17:01:02 +08:00
lenboo eeaa495661 Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into spilit 2021-03-08 10:53:02 +08:00
lenboo bb7106d100 Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-08 10:23:16 +08:00
zhuangchong e68e994d46
when Kerberos authentication is enabled,cannot get application status. (#4980) 2021-03-08 08:54:03 +08:00
JinyLeeChina 39a9c0c392
[Feature][jsonspilt]modify process instance for project home page (#4991)
* modify checkDAGRing and ProcessService method

* merge

* modify dagRing

* modify process instance for project home page

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-03-07 18:12:01 +08:00
lenboo 5bda7c344c Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-05 22:11:55 +08:00
dddyszy 01b71f1a0d
[Fix-4839][Server] Process cannot stop when killed with retry (#4841)
* [Fix-4839][Server] Process cannot stop when killed with retry

* [Fix-4839][Server] Process cannot stop when killed with retry
2021-03-04 21:10:11 +08:00
Shiwen Cheng d667525913
[Fix-4905][K8s] Fix incorrect host problem in minikube (#4906)
* [Fix][K8s] Fix incorrect host problem in minikube

* [Improvement][K8s] Rename MASTER_WORKER_STS_PATTERN to STS_PATTERN
2021-03-04 10:05:24 +08:00
Shiwen Cheng 24fda3fbb1
[Feature][Flink] Support parallelism input (#4937)
* [Feature][Flink] Support parallelism input

* [Feature][Flink] Fix check style

* [Feature][Flink] Remove FLINK_DETACH
2021-03-03 21:15:20 +08:00
lenboo 2603d52044 Merge remote-tracking branch 'upstream/dev' into spilit 2021-03-01 15:20:34 +08:00
Simon 6d087ccce0
[Feature][JsonSplit] fix processDefinitionService ut and bug (#4894)
* Modify Project and ProjectUser Mapper

* Modify Project and ProjectUser Mapper

* project_code is bigint(20)

* modify ERROR name

* modify saveProcessDefine, remove the duplicate code with createTaskAndRelation

* modify import/export processdefinition, add genProcessData

* fix ut and bug

* code style
2021-03-01 15:01:17 +08:00
zhuangchong a388b6853d
Optimize HashMap Initial Capacity. (#4896) 2021-02-27 21:36:03 +08:00
lenboo ccc5aa91d8 merge from dev 2021-02-25 18:09:32 +08:00
Kirs a9e1d316f5
[Improvement][SPI]Migrate related classes loaded by SPI classes to public modules (#4586)
* Migrate related classes loaded by SPI classes to public modules

* Migrate related classes loaded by SPI classes to public modules

* remove guava

* update guava

* update guava

* move to common model

* code style

* code style

* resolved conflicting file

* remove ignore test

* add jar

* fix ut error

* fix test

* fix error

* test

* exclude asm、resolver、aether-api

* code style

* remove default plugin jar config

* add LICENSE file

* add version

* add version

* remove ignore ut

* fix ut error

* add ut

* code style

* add license
2021-02-25 11:39:46 +08:00
guohaozhang c64d73b075
[Fix-#4866][common] Fix loading hdfs-site.xml (#4867) 2021-02-25 09:58:34 +08:00
JinyLeeChina a32b40ee95
[Feature][JsonSplit] add gen taskNodeList method (#4861)
* add gen taskNodeList method

* modify code style

* modify code style

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-02-24 17:36:01 +08:00
lenboo 2afc9f3bfc merge from dev 2021-02-24 12:10:07 +08:00
Shiwen Cheng 149b0dab4d
[Improvement][Docker] Sync docker conf templates to the latest conf properties (#4772) 2021-02-23 14:47:10 +08:00
bao liang 62e961e3f1
[Feature][JsonSplit]refactor remove the json in process instance and definition (#4828)
* #4417 [JsonSplit] refactor process definition json

* [#4177] refactor json split. remove json in process definition and process instance

* [#4177] refactor json split. copy process definition need set task code 0L.

* code style

* code style

* code style

* code style

* update

* update
2021-02-22 10:13:32 +08:00
lenboo 4c9c41933d Merge remote-tracking branch 'upstream/dev' into spilit
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionVersionServiceImpl.java
2021-02-21 14:54:18 +08:00
Shiwen Cheng 8879d8ff60
[Improvement][*] Define version with revision globally and update version to 1.3.6-SNAPSHOT (#4773)
* [Improvement][*] Update version to 1.4.0-SNAPSHOT

* [Improvement][*] Update tab to space under checkstyle

* [Improvement][*] Define version with revision globally

* [Improvement][*] Update version to 1.3.6-SNAPSHOT
2021-02-20 14:22:49 +08:00
felix.wang 3110f023af
[BUG][API]Fix format2Duration when parameter has one is null (#4802)
* fix format2Duration when parameter has one is null
2021-02-19 23:18:11 +08:00
Shiwen Cheng 61397b338a
[Improvement][api] Introduce base service interface for clear code and unify all controllers, services and tests of api (#4798)
* [Improvement][api] Introduce base service interface for clear code

* [Improvement][api] Unify all controllers, services and tests of api

* [Fix][api] Fix unit tests left over from the past

* [Improvement][api] Use RegexUtils.escapeNRT instead of StringUtils.replaceNRTtoUnderline for vulnerable check
2021-02-19 11:52:42 +08:00
Shiwen Cheng 9ae29a756f
[Improvement-3369][api] Introduce resources, scheduler and taskinstance service interface for clear code (#4766)
* [Improvement-3369][api] Introduce resources, scheduler and taskinstance service interface for clear code
2021-02-18 23:17:01 +08:00
Shiwen Cheng dc55b5ba6f
[Improvement-3369][api] Introduce monitor, processinstance and queue service interface for clear code (#4765)
* [Improvement-3369][api] Introduce monitor, processinstance and queue service interface for clear code
2021-02-18 19:27:13 +08:00
Shiwen Cheng 070424fc78
[Improvement-3369][api] Introduce taskrecord, udffunc, workflowlineage and workergroup service interface for clear code (#4768)
* [Improvement-3369][api] Introduce taskrecord, udffunc, workflowlineage and workergroup service interface for clear code
2021-02-18 19:18:05 +08:00
lenboo 1b9ab88e78 Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-18 17:28:58 +08:00
Shiwen Cheng a53195fa15
[Improvement-3369][api] Introduce alert group and users service interface for clear code (#4758) 2021-02-18 15:05:49 +08:00
Shiwen Cheng 8ebddc1f5b
[Fix-3298][K8s] Fix wrong host of task instance in k8s (#4786) 2021-02-18 11:41:58 +08:00
lenboo f7ca571f7d Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-18 11:16:31 +08:00
zhuangchong 23591b71a2
[Improvement-4751][API] Improvement the duration field modify the display to a time-based minute-second format (e.g. 1d 10h 20m 1s) (#4752)
* imp ui instance duration field display content.

* update DateUtils format2Duration method.
2021-02-18 10:08:23 +08:00
lenboo 3571de972e Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-17 10:39:43 +08:00
zhuangchong 18e4087c7e
[Improvement][JSONUtils] Improvement JSONUtils parse object method (#4781)
* Improvement JSONUtils parse object method.

* update JSONUtils toMap method.

* rerun ut test.
2021-02-16 20:22:01 +08:00
lenboo 9f809de693 merge from dev 2021-02-15 10:42:10 +08:00
Shiwen Cheng 53598fdd18
[Fix-3298][K8s] Fix task log disappear after recreating or upgrading helm release (#4755)
* [Improvement][K8s] Alter column host varchar(15) to varchar(135) for long host

* [Improvement][K8s] Improve getHost and getAddr in NetUtils

* [Improvement][K8s] Replace getHost with getAddr
2021-02-14 21:33:45 +08:00
lenboo 9141840bbe merge from dev 2021-02-07 14:51:48 +08:00
zhuangchong a5443f1173
[fix-4128][worker] Dolphin executes the command using sudo to specify the -u parameter (#4217)
* update sudo cmd.

* update sudo cmd.

* update OSUtils code style.

* add OSUtils test method.
2021-02-06 22:20:25 +08:00
zhuangchong 03a2c6b7b7
[Improvement-4450][datasource] Hive/Spark data sources do not support multi-tenancy when Kerberos authentication is enabled (#4593)
* hive/spark jdbc add kerberos paramter.

* update datasource vue paramter placeholder.

* update code style.

* add datasource ApiImplicitParam in the kerberos paramter.

* update constants class code style.

* add code test and code style.

* update BaseDataSourceTest code style.

* update BaseDataSourceTest class code style.

* update DataSourceController class code style.

* update DataSourceController ApiImplicitParam

* update ui create data source vue.
2021-02-06 16:57:59 +08:00
lenboo 5073510119 Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-06 14:27:50 +08:00
Shiwen Cheng 773909efe2
[Improvement][Common] Add more resource suffix (#4683) 2021-02-05 22:17:51 +08:00
lenboo 6844f67957 Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-02 16:53:28 +08:00
zt-1997 14d49bc584
[FIX-2101]It solves the problem that the name of the branch will not change after the name of the branch is changed in the workflow page and instance page. (#4555)
Co-authored-by: zt-1997 <“18841012545@163.com”>
2021-02-02 16:07:37 +08:00
lenboo 988649a1b2 Merge remote-tracking branch 'upstream/dev' into spilit 2021-02-02 11:31:46 +08:00
enoughwell c1875e80d5
[Feature-4596] Add a description for the system env path in common.properties (#4638)
* add a description for the system env path in common.properties

* Update common.properties

Co-authored-by: dailidong <dailidong66@gmail.com>
2021-01-31 22:13:52 +08:00
lenboo c8d5454741 merge form dev 2021-01-31 16:45:54 +08:00
Segun Ogundipe e083e28720
[Improvement][service] Refactor service module to fix code smell (#4513)
* chore: Refactor code to fix code smell in service module

* chore: Add licence header to new files

* chore: Fix comment from code review

* chore[service]: Reduce the number of custom runtime exception to one
2021-01-29 19:44:33 +08:00
lenboo 1d72a43680 Merge remote-tracking branch 'upstream/dev' into spilit 2021-01-26 10:34:27 +08:00
cooper a5f31b75ab
[Improvement-4506][LICENSE] upgrade the version of the commons-beanutils (#4525)
* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env

* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env

* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env

* [Fix-3233][api-server] rollback

* [Fix-3233][api-server] delete release-docs/licenses/LICENSE-jsp-api-2.1-6.1.14.txt

* [Fix-3233][api-server] delete ant-1.6.5.jar core-3.1.1.jar jsp-api-2.1-6.1.14.jar

* [Fix-3233][api-server] delete LICENSE-ant-1.6.5.txt,LICENSE-core-3.1.1.txt

* [Fix-3233][api-server] delete maven repository address of ant-1.6.5.jar ,core-3.1.1.jar ,jsp-api-2-1-6.1.14.jar

* [DS-4506][LICENSE] upgrade the version of the commons-beanutils

* [DS-4506][LICENSE] upgrade the version of the commons-beanutils

* [DS-4506][LICENSE] upgrade the version of the commons-beanutils

Co-authored-by: 李长福 <changfu.li@dmall.com>
2021-01-25 13:43:06 +08:00
JinyLeeChina 648bc412a1
[Feature][JsonSplit] add task and relation save code (#4546)
* add task and relation save code

* modify codestyle

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-01-25 10:44:15 +08:00
lenboo 59a542d68f Merge branch 'json_split' of https://github.com/apache/incubator-dolphinscheduler into json-split
# Conflicts:
#	dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProcessInstanceService.java
#	dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java
2021-01-25 10:38:28 +08:00
Shiwen Cheng 0b8fcac37b
[Fix-4439][*] Fix cors problem (#4490)
* [Fix][UI] Fix warning in npm install

* [Fix-4439][*] Fix CORS problem

* [Improvement][API] Improve locale change interceptor

* [Improvement][API] Add test for locale change interceptor

* [Improvement][API] Fix code style

* [Improvement][Build] Update .gitignore
2021-01-24 15:41:58 +08:00
crazycarry 5bae9336b0
update get system load (#4524)
Co-authored-by: hujinhui <hujinhui@shein.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2021-01-24 14:53:27 +08:00
Kirs 1c5abbb2ae
Merge branch 'dev' into alert_plugin_design 2021-01-22 18:16:23 +08:00
snail_fly b290754f83
[Improvement-#4503][dolphinscheduler-dist]Jackson version upgraded to 2.10.5 from 2.9.10 (#4522)
* [Improvement-4503][dolphinscheduler-dist]jackson 2.9.10,upgrade to 2.10.5

* [BUG][dolphinscheduler-common]MissingNode can't be casted to ObjectNode when JSONUtils parseObject

Co-authored-by: 吕丽粉 <lvlf@hsmap.com>
2021-01-22 18:09:27 +08:00
CalvinKirs 64b4715b73 reslove 2021-01-22 18:00:34 +08:00
Kirs ab95b16d21
Revert "[Feature-#3049] Alert Plugin Design(merge alert_plugin_design to dev) (#4495)" (#4535)
This reverts commit 0c8d08cb
2021-01-22 17:35:02 +08:00
Kirs 73b0eaa999
merge dev to alert_plugin_design (#4530)
* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* fix #3900 kill multi yarn app in one job

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* add ProcessUtils UT

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* [Draft][Merge][133-dev]133 merge dev (#4031)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>

* Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)

This reverts commit ad2d9f99d0.

* [Fix][API] Condition task null pointer exception (#4056)

* add ProcessUtils UT

* modify ProcessUtils & ProcessUtilsTest

* Enhance user experience, add close button, return to the previous page (#4006)

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067)

* [Fix][api] Fix build parameter error of sqlserver when create. (#4015)

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog().

* [Fix][api] Fix build parameter error of sqlserver when create.

* [Feature-3985][Datax] Datax supports setting up running memory (#3986)

* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit

* [Improvement-3767][api] Task instance supports query by process instance name (#3825)

* Task instance supports query by process instance name.

* add test code checkstyle.

* add test param.

* resolve the sonar duplication check.

* solve logger single-line string exceeds 200 characters.

* resolve the sonar check.

* Resolve code conflicts.

Co-authored-by: zhuangchong <zhuangchong8@163.com>

* deal with magic value

* [Feature-4050][server] Spark task support udv inject (#4061)

* #4050 spark task support udv inject

* modify spark task UT

* modify sparkTaskExecutionCtx

* add exp for spark task get main jar method

* deal with magic value

* [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481)

* JSONUtils.toMap call improvement.

* [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error

* 解决单独执行子节点空指针的问题

* using OSUtils.execCmd when kill yarn app

* 解决单独执行子节点空指针的问题(增加checkstyle)

* 解决单独执行子节点空指针的问题(增加checkstyle)

* [FIX-#4083][server]fix taskInstance state change error
Concurrent processing of ack message and result message causes the execution sequence to be wrong

# this close # 4083

* code style

* fix replaceNRTtoUnderline NullPointerException #4098 (#4100)

* fix replaceNRTtoUnderline NullPointerException

* add  unit Test

* add taskResponseTest

* add taskResponseTest

* code smell

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* [FIX-4034][server] fix sqoop import fail (#4036)

* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* test

* remove assert

* test

* test

* fix task instance status judgment error

* fix: security page disappear delay problem when force refresh under GENERAL_USER

* improvement: resovle download url with resolveURL to prevent change of apiPrefix

* fix sqoop task jdbc string contains special char (#4105)

* [Bug][Common] read file garbled (#3479)

* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [Improvement-3933][db operation] Improve the performance of sql query (#3940)

* optimize select * case

* emove redundancy

* bug fixed

* Update en_US.js

* Update startup.sh

* optimize

* optimize code

* optimize

* bug fixed

* add ut

* bug fixed

* bug fixed

* bug fixed

* bug fixed

* Delete WorkFlowLineageMapper.xml

* Delete createTenement.vue

* recove wrongly deleted file

* Update WorkFlowLineageMapper.xml

* Update createTenement.vue

* [Feature][API]enable response resources gzip compression (#4121)

* enable response compression

* add server.compression.mime-types with default value explicitly

* [Improvement][API] ignore noNodeException when get worker groups (#4120)

* ignore noNodeException when get worker groups

* add ut

* [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)

* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* [FIX#4033] $[] conflicts with mysql keywords (#4111)

* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error

* split sqoop import hive database and table (#4141)

* upgrade quartz version to 2.3.0

* add HikariCP-java6,c3p0,mchange-commons-java license

* upgrade jackson version to 2.9.10

* start from force success

* correct param name

* correct names

* remove c3p0,mchange-commons-java license

* update Check code style

* update  code style

* Update pom.xml

* Update pom.xml

* Update LICENSE

not need to add HikariCP-java6

* Delete LICENSE-HikariCP-java6.txt

* Update known-dependencies.txt

* Update TaskInstanceMapper.xml

optimize page of [TaskInstance] load data slow

* [FIX-PR-4097][server-master]task ack miss (#4189)

When the message of successful execution arrives earlier than
the message of ack,
the message of ack will be discarded,
resulting in some information missing

* fix bug #4125 (#4127)

* fix bug #4125

* code style!

* code style.

Co-authored-by: chengp <chengp@chengp.net>

* [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201)

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-3457][flink] fix flink args build problem (#4166)

* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local

* [common]del windows file( not support windows ) (#4204)

* del support win

* del support win

* [Feature-4138][Master] dispatch workgroup error add sleep time (#4139)

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* fix code smell & code style

* fix code smell & code style

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX-#4172][server-worker] kill task NPE (#4182)

* [FIX-#4172][server-worker] kill task NPE

The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear
Modification method: write into the cache when the task is received, and mark it as preData
If the task is killed before the Process is generated, delete the cache directly at this time
It will be judged before the process is generated. If the task has been killed, it will not be executed.
After the new process is created, write it into the cache, and judge again, if kill, then kill the process.

this closes #4172

* Delete the commented out code
Add spring beans

* code smell

* add test

* add test

* fix error

* test

* test

* revert

* fix error

* [Feature-3878]Replace the page with element-ui (#4065)

* [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui

* Change node version

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Replace the page with element-ui

* Replace user management with elment-ui

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Repair the last Sunday of each month

* support auto eslint for .js, .vue file on save

* update .eslintrc.yml and license check exclude .eslintignore

* fix eslint: fix syntax by npm run lint automatically

* fix eslint: fatal syntax errors

* fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '=='

* fix eslint: Unexpected side effect in 'cacheParams' computed property

* fix eslint: assigned a value but never used

* fix eslint: component has been registered but not used

* fix eslint: unexpected mutation of prop

* fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219)

* deleted invalid code as assigned in issue 4215 (#4221)

deleted the code at following two places:
org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue
org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo

* [FIX-#3177]Task time parameter parsing error (#4224)

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* fix sql error (#4227)

* [FIX-3177]Task time parameter parsing error (#4228)

* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import

* fix date_convert null

* fix time cover

* fix time cover

* reformat

* add ut

* [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162)

* fix(ui): re-login problem in new tab

* refresh page in new tab automatically

* support to reload router view in lightweight

* optimize visibility code

* [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225)

Add no-var rule for eslint, and fix related files
Add npm run lint:fix command, which distinguished from the default command npm run lint
The behavior of command npm run lint and command npm run lint:fix should be distinguished
The first command is used to lint only, and the second one is used to lint and fix problems automatically

* [FIX][UI ]  security user state  tenantCode User Type Display abnormal (#4255)

* fix user state error

* fix userType  error

* fix tenantCode  error

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250)

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Remove useless import

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest

* [Fix-4222][Master]Fix code smell

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Fix code smell

Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* update bug template (#4260)

* add pasue icon button and adjust operation button style

* Align each item one by one in locale

* match frontend and backend for force success

* [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)

* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.

* remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262)

* remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters()

* remove unused imports in AbstractTask.java

* reformat

* fix code style and test

* [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272)

* fix version funcation delete. (#4265)

* [improvement][config] Update datasource.properties,add mysql meta data template (#4266)

* Update datasource.properties

[新增]元数据mysql连接模版

* Update datasource.properties

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix taskInstance submitTime is empty. (#4274)

* [Improvement-3878]Tenant list delete user name (#4278)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html

* Fix checkstyle error

* Update swagger-models in known-dependencies.txt

* [Improvement-3878][ui]Fix the list style (#4280)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [FIX] [UI] fix create project cancel button invalid (#4282)

* fix create project cancel button invalid

* [Improvement] Refactor code to support distributed tracing (#4270)

* Refactor code to support tracing

* Extension network protocol, support context and version
* Extension master asynchronous queue support context
* Extract scan task method from MasterSchedulerService for tracing

* fix

* fix

* add test case

* fix

* fix

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Improvement] Add Flink job name (#4285)

* [Improvement] Add Flink job name

* fix typo

* [Improvement][ui]List vacancy optimization and icon icon repair (#4286)

* [Improvement][ui] List vacancy optimization and icon icon repair

* adjust locale for new pr

* [FEATURE-736] integrate ldap authentication (#3743)

* [FEATURE-736] integrate ldap authentication

* add ldap authentication type
* refactor authentication with ldap and password
* add  createUser for ldap in user service
* remove duplicate password authenticator

* [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290)

* fix flink name not display in process definition editor

* fix flink name not escape problem

* simplify escape method

* fix database is the mysql DB keyword. (#4295)

* fix the token management list does not display the user name. (#4302)

* Dev imp server process utils (#4263)


* Adds the comment on the WINDOWSATTERN

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>

* fix version close function does not take effect. (#4307)

* fix user management authorization operation exceptions do not prompt exception information. (#4292)

* [Fix][UI] Fix ui style problem and refactor form style (#4329)

* rename from-model to form-model

* [UI] remove duplicated css style code

* [UI] refactor css style of form model

* fix form model overflow problem in ie

* Align each item one by one in locale

* fix el-dialog width is too wide

* fix locale

* [Improvement][UI] Improve script box and dialog css style (#4331)

* [Fix-4335][UI] Fix IE 9-11 not supported

* [FIx-4338][UI] Fix invalid date problem in IE

* [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341)

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Fix][common] only two yarns can be selected(#4314) (#4344)

* modify_Status (#4352)

Co-authored-by: zt-1997 <“18841012545@163.com”>

* fix the page does not refresh after clicking the delete function.. (#4360)

* [FIX-4326][UI] Add description field to data source list (#4362)

* [Feature-4318][API] Rename tenant code (#4320)

Rename tenant code
Rename tenant code (English version)
Rname tenant name in ui label
Rname TENANT_CODE_EXIST to OS_TENANT_CODE_EXIST
Rname enum  TENANT_CODE_* to OS_TENANT_CODE_*

* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env (#4312)

* [Fix-3233][api-server] Fix the bug when compile the code in Java9 env
 delete release-docs/licenses/LICENSE-jsp-api-2.1-6.1.14.txt
 delete ant-1.6.5.jar core-3.1.1.jar jsp-api-2.1-6.1.14.jar
 delete LICENSE-ant-1.6.5.txt,LICENSE-core-3.1.1.txt
 delete maven repository address of ant-1.6.5.jar ,core-3.1.1.jar ,jsp-api-2-1-6.1.14.jar

Co-authored-by: 李长福 <changfu.li@dmall.com>

* [Improvement][ui] Introduce remixicon, delete ans-ui license and font-awesome (#4313)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [Improvement][ui] List vacancy optimization and icon icon repair

* Add font-awesome license and delete ans-ui license

* Introduce remixicon

* [Improvement-4257][api]  Increase "queryProcessDefinitionByName" API (#4390)

* 1. Increase "queryProcessDefinitionByName" API.
2. Will queryProcessDefinitionById api "PROCESS_INSTANCE_EXIST" state is modified to "PROCESS_DEFINE_NOT_EXIST".

* Correct spelling mistakes in words

* Logging should not be vulnerable to injection attacks

Co-authored-by: yinyong <yinyong@netposa.com>

* remove the operation resume_from_forced_success

* refactor code styles

* readme file fixed (#4374)

* readme fix

* readme fix

* readme fix

* readme fix

* readme fix

Co-authored-by: TinaPang <pangtianyang@analysys.com.cn>

* [Fix-4293][UI] Fix data loss in switched tab (#4415)

* [Fix-4293][UI] Fix data loss in switched tab

* The disclaimer should be same with the website

* [Improvement][UI] Rename disabled to enabled in menu (#4414)

* [Feature-#130] pass global param values when starting new process instance (#4372)

* [DS-130][feat] pass global param values when starting new process instance
    add optional param for start-process-instance api
    reuse command_param in command table for persistence
    overload curingGlobalParams function in ParameterUtils
    not adapt the UI code yet

* change import order

* support datetime expression

* print start params

* (fix) avoid npe when cmdParam is null

Change-Id: I3b4c4b5fa1df316ff221e27146e45d7d4d3a404e

* fix alert-spi error
todo sql script

* [DS-4396][feat] Add project information to email alert message (#4413)

* [DS-4396][feat] Add project information to email alert message
- build find project and user sql to ProjectMapper
- add project information to alert db.

* [DS-4396][style] change style

* add page query plugin instance
add plugin instance check name

* add page query plugin instance
add plugin instance check name

* add props field

* ding talk some filed is not required

* fix bug

* [Improvement][UI] Correct spelling mistakes (#4440)

* fix error

* revert

* [FEATURE-4451][API]After the workflow editing page is submitted, the user chooses whether to go online or not. 0 means not to go online, 1 means online. (#4437)

* vvvv

* vvvvv

* vvvvv

* testReleaseProcessDefinition

Co-authored-by: zt-1997 <“18841012545@163.com”>

* [Feature-4428][dao、UI] It is recommended to add an owner in the data source center and the Resource Center to make it easier to manage (#4442)

* add owner to datasource list and resource list

* add owner to datasource list and resource list

Co-authored-by: bobqiu <xiaoqiu369@gmail.com>

* [Fix-4455][common] fix parse shell output (#4456)

* add name field provide to ui

* [FIX][UT]testVerifyTenantCode assert error (#4460)

* [FIX][UT]testVerifyTenantCode assert error

* [FIX][UT]testVerifyTenantCode assert error

* code smell

* fix the exception of update null point of workflow instance due to no… (#4459)

* fix the exception of update null point of workflow instance due to not add global parameter when create workflow definition

* optimization of the leading package

Co-authored-by: yaoshui <huangqitao@accesscorporate.com.cn>

* chore: set up license-eye to check license headers (#4453)

* [Improvement][alert] Refactor alert module to fix code smell (#4434)

* chore: Refactore dolphinscheduler-alert to fix code smell

* chore: Refactor code to fix codestyle error

* fix: use shellformat(shfmt) and shellcheck fix mvnw (#3692)

use shellformat(shfmt) and shellcheck fix mvnw

* [Feature-4451][UI]: add an option to the UI page for saving the workflow definition: whether to go online? online by default (#4474)

Co-authored-by: linyanbin <linyanbin@lizhi.fm>

* [FIX-4247][Api]Fixed the problem that the global parameters are not updated after the workflow instance click rerun

Co-authored-by: zt-1997 <“18841012545@163.com”>

* [Improvement-4435][datasource] the datasource tests the connection and returns details when the connection is wrong (#4436)

* add datasource test connection return result message.

* add code checkstyle.

* spark/hive datasoure test connection add loadKerberosConf.

* solve sonarcloud coverage.

* [Improvement-4480][API] change method checkAdmin to isNotAdmin (#4483)

* change method 'checkAdmin' to 'isNotAdmin', for better understand.

Co-authored-by: 蒙强 <mengqiang@zhongan.com>

* fix send error

* add ui

* delete useless constant definitions
code style

* code style

* fix license head error

* [Improvement-#4481][API] ProcessDefinitionController save API optimization 

Co-authored-by: lei.zou <lei.zou@rootcloud.com>

* fix ut error

* fix ut error

* del unused jar

* [Feature-4423][UI] When the process definition is running, the pop-up box can manually set the global parameters. (#4433)

* [DS-130][feat] pass global param values when starting new process instance
    add optional param for start-process-instance api
    reuse command_param in command table for persistence
    overload curingGlobalParams function in ParameterUtils
    not adapt the UI code yet

* change import order

* support datetime expression

* print start params

* (fix) avoid npe when cmdParam is null

Change-Id: I3b4c4b5fa1df316ff221e27146e45d7d4d3a404e

* [Feature-4423][UI] When the process definition is running, the pop-up box can manually set the global parameters
(frontend) adapt the UI code
(backend-fix) add empty string check for param

Change-Id: I710db55f5059f8bd324c79f4494f2798d58e7b19

* add Startup parameters label

Change-Id: I5ac82031ea1b64abec330ee8cf2991477a28fcaa

* reuse i18n label

Change-Id: I5f322cb1dd8e2cade0c679bd025fc984e31bf3ae

* import alert-plugin

* upgrade jboss-logging

* upgrade jboss-logging

* fix ut error

* fix Vulnerabilities

* fix Vulnerabilities

* fix Vulnerabilities

* code smell

* [fix-4472][DataSource] Hive JDBC partial permission parameter concatenation JDBC URL error (#4473)

* Solve the problem of parameter splicing of permission part in Hive JDBC URL.

* update code style.

* add test check in pom.xml.

* [FIX-#4468] [UI] Workflow relationships Name filtering invalidated #4508

Co-authored-by: 花花 <huahua@thinkingdata.cn>

* fix Vulnerabilities

* add  public constructors

* fix  Vulnerabilities

* [NOTICE Update]Update year to 2021 (#4521)

* [feat-4496][server]  Add to! {} is used to mark the custom parameters to be output as-is in sql (#4497)

* feat([server]): Add to! {} is used to mark the custom parameters to be output as-is in sql

Before pre-compiling sql, replace the custom parameters marked with !{}
to prevent the parameters in the hive plus partition path from being
replaced with single quotes

Closes This closes #4496

* feat([server]): Add to! {} is used to mark the custom parameters to be output as-is in sql

Before pre-compiling sql, replace the custom parameters marked with !{}
to prevent the parameters in the hive plus partition path from being
replaced with single quotes

Closes This closes #4496

* feat([server]): Add to! {} is used to mark the custom parameters to be output as-is in sql

Before pre-compiling sql, replace the custom parameters marked with !{}
to prevent the parameters in the hive plus partition path from being
replaced with single quotes

Closes This closes #4496

* resolve conflicts

* add eg

* add eg

* code style

* revert mysql config

Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: yinancx <mockibird@live.cn>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: 曹聚阳 <yinancx@live.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>
Co-authored-by: zt-1997 <41983395+zt-1997@users.noreply.github.com>
Co-authored-by: zt-1997 <“18841012545@163.com”>
Co-authored-by: xiaojingXU <68894048+xxjingcd@users.noreply.github.com>
Co-authored-by: guohaozhang <zhishengqianjun@gmail.com>
Co-authored-by: cooper <1322849632@qq.com>
Co-authored-by: 李长福 <changfu.li@dmall.com>
Co-authored-by: yy0812 <32183971+yy0812@users.noreply.github.com>
Co-authored-by: yinyong <yinyong@netposa.com>
Co-authored-by: ptyp <30622951+ptyp@users.noreply.github.com>
Co-authored-by: TinaPang <pangtianyang@analysys.com.cn>
Co-authored-by: Dean Wong <wangding85@gmail.com>
Co-authored-by: K.O <okoahin@gmail.com>
Co-authored-by: qiubo <38316980+qiubo369@users.noreply.github.com>
Co-authored-by: bobqiu <xiaoqiu369@gmail.com>
Co-authored-by: Hxssssss <huangqt1992@163.com>
Co-authored-by: yaoshui <huangqitao@accesscorporate.com.cn>
Co-authored-by: Zhenxu Ke <kezhenxu94@apache.org>
Co-authored-by: Segun Ogundipe <davephenoms@gmail.com>
Co-authored-by: xiagw <fxiaxiaoyu@gmail.com>
Co-authored-by: linyanbin666 <35388422+linyanbin666@users.noreply.github.com>
Co-authored-by: linyanbin <linyanbin@lizhi.fm>
Co-authored-by: qmengss <qmengss@users.noreply.github.com>
Co-authored-by: 蒙强 <mengqiang@zhongan.com>
Co-authored-by: 青年 <1043706593@qq.com>
Co-authored-by: lei.zou <lei.zou@rootcloud.com>
Co-authored-by: 小二黑 <784445949@qq.com>
Co-authored-by: 花花 <huahua@thinkingdata.cn>
Co-authored-by: liuxuedongcn <46193861+liuxuedongcn@users.noreply.github.com>
2021-01-22 16:26:52 +08:00
lenboo 49204a6b13 merge from dev 2021-01-22 11:29:26 +08:00
Kirs 0c8d08cbaa
[Feature-#3049] Alert Plugin Design(merge alert_plugin_design to dev) (#4495)
* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui]Alert plugin design (#3734)

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui] Add alarm instance page

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601)

* DS SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* add TODO

* delete

* compile

* spi commit

* Plugin Alert

* fix some bug

* add todo

* change web ui from alpacajs to form-create

* remove module

* add plugin schema

* add license header

* update alert and spi module version

* update the alert plugin sub module version

* comment the maven.local.repository param

* move utils from spi to common module

* add license header

* add license header and delete some chinese comment

* update spi packages

* delete no use alert_xx.properties

* update mysql.connector.version back to 5.1.34

* delete no use comment in pom.xml

* update email stmp password

* add license

* add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file

* format the code style

* format new clase file with checkstyle

* update plugin params to Builder model

* move JSONUtils to SPI because plugin can not dependency common module

* move JSONUtils to SPI because plugin can not dependency common module

* delete collection dependency

* replace PluginParamsTransfer to spi PluginParamsTransfer

* update dolphinscheduler-maven-plugin to 1.0.0

* update license

* update apache-rat-plugin add exclude '.iml' file

* check license

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* default datasource should be postgresql

* add license files

* add license files

* postgresql port should be 5432

* postgresql test

* mv show_type to spi

add license header to AlertConstants

* check style fix

* copy check style file from branch dev

* alert show_type set by plugin

* alert show_type set by plugin

* add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java

* add Bean to TaskCallbackServiceTestConfig

* add Bean to TaskCallbackServiceTestConfig

* fix check style

* check style fix

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* fix check style

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* check style fix

* rollback test change

* rollback test change

* rollback dao pom change

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* add ut to pom.xml

* add upgrade schema to global schema

* fix ut failed

* fix ut failed

* fix ut failed

* fix ut failed

* add test EmailAlertPluginTest to pom.xml

* fix ut failed

* fix ut failed

* fix check style

* update license header to presto license header

* presto license header not check

* fix ut coverage

* fix ut coverage

* fix ut

* fix ut

* fix ut

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

Co-authored-by: break60 <790061044@qq.com>

* Replace the tenant management page ui with element-ui (#3891)

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui] Add alarm instance page

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui]Alert plugin design (#3734)

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui] Add alarm instance page

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601)

* DS SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* add TODO

* delete

* compile

* spi commit

* Plugin Alert

* fix some bug

* add todo

* change web ui from alpacajs to form-create

* remove module

* add plugin schema

* add license header

* update alert and spi module version

* update the alert plugin sub module version

* comment the maven.local.repository param

* move utils from spi to common module

* add license header

* add license header and delete some chinese comment

* update spi packages

* delete no use alert_xx.properties

* update mysql.connector.version back to 5.1.34

* delete no use comment in pom.xml

* update email stmp password

* add license

* add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file

* format the code style

* format new clase file with checkstyle

* update plugin params to Builder model

* move JSONUtils to SPI because plugin can not dependency common module

* move JSONUtils to SPI because plugin can not dependency common module

* delete collection dependency

* replace PluginParamsTransfer to spi PluginParamsTransfer

* update dolphinscheduler-maven-plugin to 1.0.0

* update license

* update apache-rat-plugin add exclude '.iml' file

* check license

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* default datasource should be postgresql

* add license files

* add license files

* postgresql port should be 5432

* postgresql test

* mv show_type to spi

add license header to AlertConstants

* check style fix

* copy check style file from branch dev

* alert show_type set by plugin

* alert show_type set by plugin

* add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java

* add Bean to TaskCallbackServiceTestConfig

* add Bean to TaskCallbackServiceTestConfig

* fix check style

* check style fix

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* fix check style

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* check style fix

* rollback test change

* rollback test change

* rollback dao pom change

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* add ut to pom.xml

* add upgrade schema to global schema

* fix ut failed

* fix ut failed

* fix ut failed

* fix ut failed

* add test EmailAlertPluginTest to pom.xml

* fix ut failed

* fix ut failed

* fix check style

* update license header to presto license header

* presto license header not check

* fix ut coverage

* fix ut coverage

* fix ut

* fix ut

* fix ut

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

Co-authored-by: break60 <790061044@qq.com>

* The tenant management module ui is replaced with element-ui

* fix

* fix

* fix

* fix

* fix

* modify

* fix

Co-authored-by: gaojun2048 <32193458+gaojun2048@users.noreply.github.com>

* the alert module support service.

* the alert module support service.

* Delete blank lines.

* add code checkstyle.

* Solve the sonar bug.

* Re-execute build.

* add test.

* update ut.

* add test class.

* update test.

* [feature#3356] alert-spi support DingTalk&WeChat (#3869)

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui]Alert plugin design (#3734)

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [feature][ui] Add alarm instance page

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601)

* DS SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* add TODO

* delete

* compile

* spi commit

* Plugin Alert

* fix some bug

* add todo

* change web ui from alpacajs to form-create

* remove module

* add plugin schema

* add license header

* update alert and spi module version

* update the alert plugin sub module version

* comment the maven.local.repository param

* move utils from spi to common module

* add license header

* add license header and delete some chinese comment

* update spi packages

* delete no use alert_xx.properties

* update mysql.connector.version back to 5.1.34

* delete no use comment in pom.xml

* update email stmp password

* add license

* add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file

* format the code style

* format new clase file with checkstyle

* update plugin params to Builder model

* move JSONUtils to SPI because plugin can not dependency common module

* move JSONUtils to SPI because plugin can not dependency common module

* delete collection dependency

* replace PluginParamsTransfer to spi PluginParamsTransfer

* update dolphinscheduler-maven-plugin to 1.0.0

* update license

* update apache-rat-plugin add exclude '.iml' file

* check license

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* default datasource should be postgresql

* add license files

* add license files

* postgresql port should be 5432

* postgresql test

* mv show_type to spi

add license header to AlertConstants

* check style fix

* copy check style file from branch dev

* alert show_type set by plugin

* alert show_type set by plugin

* add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java

* add Bean to TaskCallbackServiceTestConfig

* add Bean to TaskCallbackServiceTestConfig

* fix check style

* check style fix

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* fix check style

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* check style fix

* rollback test change

* rollback test change

* rollback dao pom change

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* add ut to pom.xml

* add upgrade schema to global schema

* fix ut failed

* fix ut failed

* fix ut failed

* fix ut failed

* add test EmailAlertPluginTest to pom.xml

* fix ut failed

* fix ut failed

* fix check style

* update license header to presto license header

* presto license header not check

* fix ut coverage

* fix ut coverage

* fix ut

* fix ut

* fix ut

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

Co-authored-by: break60 <790061044@qq.com>

* [feature#3356] alert-spi support DingTalk

this closes # 3356

* add test

* code style

* we chat alert

* support we chat alert

* support we chat alert

* support we chat alert,update ding talk alert

* code style

* add test

* code style

* clean old code

* clean old code

* code smell

* code style

* add test

* simple config

* code style

* code style

* code style

* delete old file

* fix log content error

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: gaojun2048 <32193458+gaojun2048@users.noreply.github.com>

* [FEATURE#4029] alert spi support script alert

this closes #4029

* code style

* code smell

* add test

* code style

* code smell

* code style

* fix name error

* init sms plugin

* modify http type

* delete dolphinscheduler-alert-sms plugin

* dolphinscheduler plugin need add <package>dolphinscheduler-plugin</package> in the pom.xml file

* delete alert http plugin (#4152)

delete alert http plugin (#4152)
This part of the code was merged by mistake, so I need to delete it

* [Feature-3749][Alert-SPI] SqlTask should send notifications by alert server api (#4080)

* add sqltask send sync alert server.

* update alert-sms license.

* update AlertServer test.

* remote EmailAlertPluginTest.

* update sqltask.

* update test class.

* [Alert plugin design] add http alert plugin (#4165)

* add http alert plugin

* [Feature-3311] Add alert plugin instance function (#4038)

* [Feature-#3252][build]plugins need put in plugins dir when install (#4259)

Update maven build, put the alert plugins in the ${install_dir}/lib/plugin/alert/${plugin_module_name} directory
(assembly build and rpm build).
Note: When adding a new plug-in, it must be reflected in provisio/dolphinscheduler.xml.

this closes #3252

* [Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages (#4311)

* [Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages

*Some plugins (such as alert plugin) need to provide UI interfaces to users.
*We use from-creat to dynamically generate UI interfaces. Related parameters are mainly provided by pluginParams.
*From-create can generate dynamic ui based on this parameter.

this closes #4310

* add license head

* rename

* add ut

* add license

* add query plugin detail interface

* fix error

* [Feature-#3828][server] Delete recipients and CCs in the process defi… (#4303)

* [Feature-#3828][server] Delete recipients and CCs in the process definition and select the associated alert group instead

* The specific alarm instance associated with the alert group
* The process definition only associates the alert group, and does not care about the specific alarm instance.

this closes #3828

* remove test cc

* remove test cc

* fix test error

* add sql upgrade script
delete alert column(show_type、alert_type、re、cc)

* code style

* update test
Note the statement to delete the column, and provide related instructions

* delete AlertGroup field(alert_type)

* fix ut error

* fix ut error

* fix ut error

* code style

* delete api-doc receiverCC

* delete UserAlertGroupMapper

* delete UserAlertGroup

* delete UserAlertGroupApi

* alert group add coulmn user_id

* alter user_id to create_user_id

* userinfo remove alert group info

* Replace the front-end code of the plugin branch with the front-end code of the dev branch (#4353)

* Replace the front-end code of the plugin branch with the front-end code of the dev branch

* Merge dev to alert_plugin_design (#4351)

* fixed queryProcessInstanceListPaging query slow (#3893)

because queryProcessInstanceListPaging has contain longtext type field by instance.*,so queryProcessInstanceListPaging query slow , but longtext field not used.

* fixed  queryTaskInstanceListPaging query slow (#3892)

because queryTaskInstanceListPaging  has contain longtext  type field  by instance.*,so  queryTaskInstanceListPaging query slow , but longtext  field not used.

* add verify tenant name cannot contain special characters. (#3865)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes. (#3895)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes.(#3894)

Signed-off-by: YoungMan <bushengquan@gmail.com>

* Update Dockerfile

update the  comment
and delete the maridb client

Co-authored-by: dailidong <dailidong66@gmail.com>

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* [CodeClean][DAO]Remove redundant comments (#3939)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [Fix][api] Add queryProjectCreatedAndAuthorizedByUser function (#3658)

* fix getLoginUserCreated bug

* fix getLoginUserCreated bug 1

* fix checkstyle

* modify function name

* modify comment

* fix unit test error

* rollback sqoop task hive target file

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog(). (#3963)

* Enhance user experience, add close button to file details page

* [Test-3557][API] full cover DataAnalysisServiceImpl (#3605)

* generate equals and hashCode method

* make up for the state of oblivion

* simplify countQueueState method

* delete try catch, because this never throw exception

* full cover DataAnalysisServiceImpl

* improve test coverage

* adjust code style

* simplify countCommandState method

* reduce duplication

* simplify countTaskDtos method

* Update DataAnalysisServiceImpl.java

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* the resources API document does not match the interface parameters

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* fix bug

* reset code

* Update udfs.vue

* [Feature-3941][ui] Remove tenant name  (#3949)

* FeatureUI-3941: Removed Tenant

* FeatureUI-3941: Removed tenantName in the suggested pages

* FeatureUI-3941: Updated the suggested files

* FeatureUI-3941: Updated the TenantController

* FeatureUI-3941: Updated the TenantController and the pom.xml

* FeatureUI-3941: Updated the TenantControllerTest

* [Improvement][script] daemon script usage (#3968)

* Improvement daemon script usage

* don't remove status command

don't remove status command

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix #3688, Optimization of table building statements (#3689)

* merge from 1.3.3-release

* merge from 1.3.3-release

* [Improvement-4012][common/remote] Json util code integration,remove the remote module json util (#4013)

* Json util code integration,remove the remote module json util.

* update code checkstyle.

* update code checkstyle.

* add code checkstyle.

* add test class.

* update JSONUtils class.

* update JSONUtils class.

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* fix #3900 kill multi yarn app in one job

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* add ProcessUtils UT

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* [Draft][Merge][133-dev]133 merge dev (#4031)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>

* Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)

This reverts commit ad2d9f99d0.

* [Fix][API] Condition task null pointer exception (#4056)

* add ProcessUtils UT

* modify ProcessUtils & ProcessUtilsTest

* Enhance user experience, add close button, return to the previous page (#4006)

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067)

* [Fix][api] Fix build parameter error of sqlserver when create. (#4015)

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog().

* [Fix][api] Fix build parameter error of sqlserver when create.

* [Feature-3985][Datax] Datax supports setting up running memory (#3986)

* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit

* [Improvement-3767][api] Task instance supports query by process instance name (#3825)

* Task instance supports query by process instance name.

* add test code checkstyle.

* add test param.

* resolve the sonar duplication check.

* solve logger single-line string exceeds 200 characters.

* resolve the sonar check.

* Resolve code conflicts.

Co-authored-by: zhuangchong <zhuangchong8@163.com>

* deal with magic value

* [Feature-4050][server] Spark task support udv inject (#4061)

* #4050 spark task support udv inject

* modify spark task UT

* modify sparkTaskExecutionCtx

* add exp for spark task get main jar method

* deal with magic value

* [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481)

* JSONUtils.toMap call improvement.

* [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error

* 解决单独执行子节点空指针的问题

* using OSUtils.execCmd when kill yarn app

* 解决单独执行子节点空指针的问题(增加checkstyle)

* 解决单独执行子节点空指针的问题(增加checkstyle)

* [FIX-#4083][server]fix taskInstance state change error
Concurrent processing of ack message and result message causes the execution sequence to be wrong

# this close # 4083

* code style

* fix replaceNRTtoUnderline NullPointerException #4098 (#4100)

* fix replaceNRTtoUnderline NullPointerException

* add  unit Test

* add taskResponseTest

* add taskResponseTest

* code smell

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* [FIX-4034][server] fix sqoop import fail (#4036)

* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* test

* remove assert

* test

* test

* fix task instance status judgment error

* fix: security page disappear delay problem when force refresh under GENERAL_USER

* improvement: resovle download url with resolveURL to prevent change of apiPrefix

* fix sqoop task jdbc string contains special char (#4105)

* [Bug][Common] read file garbled (#3479)

* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [Improvement-3933][db operation] Improve the performance of sql query (#3940)

* optimize select * case

* emove redundancy

* bug fixed

* Update en_US.js

* Update startup.sh

* optimize

* optimize code

* optimize

* bug fixed

* add ut

* bug fixed

* bug fixed

* bug fixed

* bug fixed

* Delete WorkFlowLineageMapper.xml

* Delete createTenement.vue

* recove wrongly deleted file

* Update WorkFlowLineageMapper.xml

* Update createTenement.vue

* [Feature][API]enable response resources gzip compression (#4121)

* enable response compression

* add server.compression.mime-types with default value explicitly

* [Improvement][API] ignore noNodeException when get worker groups (#4120)

* ignore noNodeException when get worker groups

* add ut

* [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)

* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* [FIX#4033] $[] conflicts with mysql keywords (#4111)

* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error

* split sqoop import hive database and table (#4141)

* upgrade quartz version to 2.3.0

* add HikariCP-java6,c3p0,mchange-commons-java license

* upgrade jackson version to 2.9.10

* remove c3p0,mchange-commons-java license

* update Check code style

* update  code style

* Update pom.xml

* Update pom.xml

* Update LICENSE

not need to add HikariCP-java6

* Delete LICENSE-HikariCP-java6.txt

* Update known-dependencies.txt

* Update TaskInstanceMapper.xml

optimize page of [TaskInstance] load data slow

* [FIX-PR-4097][server-master]task ack miss (#4189)

When the message of successful execution arrives earlier than
the message of ack,
the message of ack will be discarded,
resulting in some information missing

* fix bug #4125 (#4127)

* fix bug #4125

* code style!

* code style.

Co-authored-by: chengp <chengp@chengp.net>

* [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201)

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-3457][flink] fix flink args build problem (#4166)

* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local

* [common]del windows file( not support windows ) (#4204)

* del support win

* del support win

* [Feature-4138][Master] dispatch workgroup error add sleep time (#4139)

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* fix code smell & code style

* fix code smell & code style

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX-#4172][server-worker] kill task NPE (#4182)

* [FIX-#4172][server-worker] kill task NPE

The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear
Modification method: write into the cache when the task is received, and mark it as preData
If the task is killed before the Process is generated, delete the cache directly at this time
It will be judged before the process is generated. If the task has been killed, it will not be executed.
After the new process is created, write it into the cache, and judge again, if kill, then kill the process.

this closes #4172

* Delete the commented out code
Add spring beans

* code smell

* add test

* add test

* fix error

* test

* test

* revert

* fix error

* [Feature-3878]Replace the page with element-ui (#4065)

* [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui

* Change node version

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Replace the page with element-ui

* Replace user management with elment-ui

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Repair the last Sunday of each month

* support auto eslint for .js, .vue file on save

* update .eslintrc.yml and license check exclude .eslintignore

* fix eslint: fix syntax by npm run lint automatically

* fix eslint: fatal syntax errors

* fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '=='

* fix eslint: Unexpected side effect in 'cacheParams' computed property

* fix eslint: assigned a value but never used

* fix eslint: component has been registered but not used

* fix eslint: unexpected mutation of prop

* fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219)

* deleted invalid code as assigned in issue 4215 (#4221)

deleted the code at following two places:
org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue
org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo

* [FIX-#3177]Task time parameter parsing error (#4224)

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* fix sql error (#4227)

* [FIX-3177]Task time parameter parsing error (#4228)

* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import

* fix date_convert null

* fix time cover

* fix time cover

* reformat

* add ut

* [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162)

* fix(ui): re-login problem in new tab

* refresh page in new tab automatically

* support to reload router view in lightweight

* optimize visibility code

* [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225)

Add no-var rule for eslint, and fix related files
Add npm run lint:fix command, which distinguished from the default command npm run lint
The behavior of command npm run lint and command npm run lint:fix should be distinguished
The first command is used to lint only, and the second one is used to lint and fix problems automatically

* [FIX][UI ]  security user state  tenantCode User Type Display abnormal (#4255)

* fix user state error

* fix userType  error

* fix tenantCode  error

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250)

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Remove useless import

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest

* [Fix-4222][Master]Fix code smell

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Fix code smell

Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* update bug template (#4260)

* [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)

* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.

* remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262)

* remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters()

* remove unused imports in AbstractTask.java

* reformat

* [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272)

* fix version funcation delete. (#4265)

* [improvement][config] Update datasource.properties,add mysql meta data template (#4266)

* Update datasource.properties

[新增]元数据mysql连接模版

* Update datasource.properties

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix taskInstance submitTime is empty. (#4274)

* [Improvement-3878]Tenant list delete user name (#4278)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html

* Fix checkstyle error

* Update swagger-models in known-dependencies.txt

* [Improvement-3878][ui]Fix the list style (#4280)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [FIX] [UI] fix create project cancel button invalid (#4282)

* fix create project cancel button invalid

* [Improvement] Refactor code to support distributed tracing (#4270)

* Refactor code to support tracing

* Extension network protocol, support context and version
* Extension master asynchronous queue support context
* Extract scan task method from MasterSchedulerService for tracing

* fix

* fix

* add test case

* fix

* fix

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Improvement] Add Flink job name (#4285)

* [Improvement] Add Flink job name

* fix typo

* [Improvement][ui]List vacancy optimization and icon icon repair (#4286)

* [Improvement][ui] List vacancy optimization and icon icon repair

* [FEATURE-736] integrate ldap authentication (#3743)

* [FEATURE-736] integrate ldap authentication

* add ldap authentication type
* refactor authentication with ldap and password
* add  createUser for ldap in user service
* remove duplicate password authenticator

* [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290)

* fix flink name not display in process definition editor

* fix flink name not escape problem

* simplify escape method

* fix database is the mysql DB keyword. (#4295)

* fix the token management list does not display the user name. (#4302)

* Dev imp server process utils (#4263)


* Adds the comment on the WINDOWSATTERN

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>

* fix version close function does not take effect. (#4307)

* fix user management authorization operation exceptions do not prompt exception information. (#4292)

* [Fix][UI] Fix ui style problem and refactor form style (#4329)

* rename from-model to form-model

* [UI] remove duplicated css style code

* [UI] refactor css style of form model

* fix form model overflow problem in ie

* Align each item one by one in locale

* fix el-dialog width is too wide

* fix locale

* [Improvement][UI] Improve script box and dialog css style (#4331)

* [Fix-4335][UI] Fix IE 9-11 not supported

* [FIx-4338][UI] Fix invalid date problem in IE

* [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341)

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Fix][common] only two yarns can be selected(#4314) (#4344)

* Handling conflicts manually

* code style

* ut skip ui

* fix ut error

* fix ut error

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* ci skip ui

* ignore AlertPluginManagerTest.java and DolphinPluginLoaderTest.java

* fix ut error

* fix ut error

* test

* code style

* replace ui

* replace ui

* replace ui

* replace ui

* fix license

* fix license

Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: BoYiZhang <zhangboyi_mx@163.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: 孙继峰 <sun.jifeng@outlook.com>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com>
Co-authored-by: huzekang <1040080742@qq.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>

* fix alert-spi error
todo sql script

* add page query plugin instance
add plugin instance check name

* add page query plugin instance
add plugin instance check name

* add props field

* ding talk some filed is not required

* fix bug

* fix error

* revert

* add name field provide to ui

* fix send error

* add ui

* delete useless constant definitions
code style

* code style

* fix license head error

* fix ut error

* fix ut error

* del unused jar

* import alert-plugin

* upgrade jboss-logging

* upgrade jboss-logging

* fix ut error

* fix Vulnerabilities

* fix Vulnerabilities

* fix Vulnerabilities

* code smell

* fix Vulnerabilities

* add  public constructors

* fix  Vulnerabilities

* resolve conflicts

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: gaojun2048 <32193458+gaojun2048@users.noreply.github.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: samz406 <samz406@foxmail.com>
Co-authored-by: gaojun2048 <540957506@qq.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: BoYiZhang <zhangboyi_mx@163.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: 孙继峰 <sun.jifeng@outlook.com>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com>
Co-authored-by: huzekang <1040080742@qq.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>
2021-01-22 11:01:46 +08:00
lenboo 4a9099ccc7 Merge remote-tracking branch 'upstream/dev' into spilit 2021-01-21 11:00:46 +08:00
zhuangchong 734b188bd1
[fix-4472][DataSource] Hive JDBC partial permission parameter concatenation JDBC URL error (#4473)
* Solve the problem of parameter splicing of permission part in Hive JDBC URL.

* update code style.

* add test check in pom.xml.
2021-01-20 17:20:24 +08:00
lenboo db5c3bb7f8 Merge remote-tracking branch 'upstream/dev' into json-split 2021-01-18 16:50:42 +08:00
zhuangchong 91dfe0913f
[Improvement-4435][datasource] the datasource tests the connection and returns details when the connection is wrong (#4436)
* add datasource test connection return result message.

* add code checkstyle.

* spark/hive datasoure test connection add loadKerberosConf.

* solve sonarcloud coverage.
2021-01-17 23:02:02 +08:00
JinyLeeChina c7038a087c
[Feature][JsonSplit] task and relation entity (#4475)
* task and relation entity

* task and relation entity

* task and relation entity

* task and relation entity

* task and relation entity

* add timeout flag

Co-authored-by: JinyLeeChina <297062848@qq.com>
2021-01-16 23:57:59 +08:00
lenboo c3d18127be Merge remote-tracking branch 'upstream/dev' into json-split 2021-01-15 10:11:09 +08:00
guohaozhang fb5501f590
[Fix-4455][common] fix parse shell output (#4456) 2021-01-14 18:41:34 +08:00
JinyLeeChina 74712a4448
[Feature][JsonSplit] snowFlakeUtils (#4432)
* snowFlakeUtils

* snowFlakeUtils

* snowFlakeUtils

* fix instance

* 'codestyle'

* 'codestyle'

* 'codestyle'

* 'codestyle'

* 'codestyle'

Co-authored-by: 文蛤 <wenge@myweimai.com>
2021-01-13 14:23:19 +08:00
Dean Wong 83d53505de
[Feature-#130] pass global param values when starting new process instance (#4372)
* [DS-130][feat] pass global param values when starting new process instance
    add optional param for start-process-instance api
    reuse command_param in command table for persistence
    overload curingGlobalParams function in ParameterUtils
    not adapt the UI code yet

* change import order

* support datetime expression

* print start params

* (fix) avoid npe when cmdParam is null

Change-Id: I3b4c4b5fa1df316ff221e27146e45d7d4d3a404e
2021-01-12 09:13:27 +08:00
Kirs 4d553603ee
Merge dev to alert_plugin_design (#4351)
* fixed queryProcessInstanceListPaging query slow (#3893)

because queryProcessInstanceListPaging has contain longtext type field by instance.*,so queryProcessInstanceListPaging query slow , but longtext field not used.

* fixed  queryTaskInstanceListPaging query slow (#3892)

because queryTaskInstanceListPaging  has contain longtext  type field  by instance.*,so  queryTaskInstanceListPaging query slow , but longtext  field not used.

* add verify tenant name cannot contain special characters. (#3865)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes. (#3895)

* [Fix][Docker] fix startup.sh variables cannot reference in single quotes.(#3894)

Signed-off-by: YoungMan <bushengquan@gmail.com>

* Update Dockerfile

update the  comment
and delete the maridb client

Co-authored-by: dailidong <dailidong66@gmail.com>

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* [CodeClean][DAO]Remove redundant comments (#3939)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

* Remove redundant comments
ProcessInstanceMapper# queryProcessInstanceListPaging

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [Fix][api] Add queryProjectCreatedAndAuthorizedByUser function (#3658)

* fix getLoginUserCreated bug

* fix getLoginUserCreated bug 1

* fix checkstyle

* modify function name

* modify comment

* fix unit test error

* rollback sqoop task hive target file

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* When running a task, the resource file is lost, which results in an error

* When running a task, the resource file is lost, which results in an error

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog(). (#3963)

* Enhance user experience, add close button to file details page

* [Test-3557][API] full cover DataAnalysisServiceImpl (#3605)

* generate equals and hashCode method

* make up for the state of oblivion

* simplify countQueueState method

* delete try catch, because this never throw exception

* full cover DataAnalysisServiceImpl

* improve test coverage

* adjust code style

* simplify countCommandState method

* reduce duplication

* simplify countTaskDtos method

* Update DataAnalysisServiceImpl.java

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* the resources API document does not match the interface parameters

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* fix bug

* reset code

* Update udfs.vue

* [Feature-3941][ui] Remove tenant name  (#3949)

* FeatureUI-3941: Removed Tenant

* FeatureUI-3941: Removed tenantName in the suggested pages

* FeatureUI-3941: Updated the suggested files

* FeatureUI-3941: Updated the TenantController

* FeatureUI-3941: Updated the TenantController and the pom.xml

* FeatureUI-3941: Updated the TenantControllerTest

* [Improvement][script] daemon script usage (#3968)

* Improvement daemon script usage

* don't remove status command

don't remove status command

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix #3688, Optimization of table building statements (#3689)

* merge from 1.3.3-release

* merge from 1.3.3-release

* [Improvement-4012][common/remote] Json util code integration,remove the remote module json util (#4013)

* Json util code integration,remove the remote module json util.

* update code checkstyle.

* update code checkstyle.

* add code checkstyle.

* add test class.

* update JSONUtils class.

* update JSONUtils class.

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* fix #3900 kill multi yarn app in one job

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* add ProcessUtils UT

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* [Draft][Merge][133-dev]133 merge dev (#4031)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>

* Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)

This reverts commit ad2d9f99d0.

* [Fix][API] Condition task null pointer exception (#4056)

* add ProcessUtils UT

* modify ProcessUtils & ProcessUtilsTest

* Enhance user experience, add close button, return to the previous page (#4006)

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX_BUG][server-test] dismiss of server warm-up time in RoundRobinSelectorTest (#4067)

* [Fix][api] Fix build parameter error of sqlserver when create. (#4015)

* [fix-#3962][api] Avoid ClassCastException for LoggerService.queryLog().

* [Fix][api] Fix build parameter error of sqlserver when create.

* [Feature-3985][Datax] Datax supports setting up running memory (#3986)

* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit

* [Improvement-3767][api] Task instance supports query by process instance name (#3825)

* Task instance supports query by process instance name.

* add test code checkstyle.

* add test param.

* resolve the sonar duplication check.

* solve logger single-line string exceeds 200 characters.

* resolve the sonar check.

* Resolve code conflicts.

Co-authored-by: zhuangchong <zhuangchong8@163.com>

* deal with magic value

* [Feature-4050][server] Spark task support udv inject (#4061)

* #4050 spark task support udv inject

* modify spark task UT

* modify sparkTaskExecutionCtx

* add exp for spark task get main jar method

* deal with magic value

* [Improvement-3471][common] JSONUtils.toMap It is not necessary to check whether the JSON method is empty again. #3471 (#3481)

* JSONUtils.toMap call improvement.

* [Fix-4054][Api] Fix The last week of the month for adding/editing timing, preview and save timing will report an error

* 解决单独执行子节点空指针的问题

* using OSUtils.execCmd when kill yarn app

* 解决单独执行子节点空指针的问题(增加checkstyle)

* 解决单独执行子节点空指针的问题(增加checkstyle)

* [FIX-#4083][server]fix taskInstance state change error
Concurrent processing of ack message and result message causes the execution sequence to be wrong

# this close # 4083

* code style

* fix replaceNRTtoUnderline NullPointerException #4098 (#4100)

* fix replaceNRTtoUnderline NullPointerException

* add  unit Test

* add taskResponseTest

* add taskResponseTest

* code smell

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* [FIX-4034][server] fix sqoop import fail (#4036)

* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants

* Time is too small and the task is not finished

* Time is too small and the task is not finished

* test

* remove assert

* test

* test

* fix task instance status judgment error

* fix: security page disappear delay problem when force refresh under GENERAL_USER

* improvement: resovle download url with resolveURL to prevent change of apiPrefix

* fix sqoop task jdbc string contains special char (#4105)

* [Bug][Common] read file garbled (#3479)

* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [Improvement-3933][db operation] Improve the performance of sql query (#3940)

* optimize select * case

* emove redundancy

* bug fixed

* Update en_US.js

* Update startup.sh

* optimize

* optimize code

* optimize

* bug fixed

* add ut

* bug fixed

* bug fixed

* bug fixed

* bug fixed

* Delete WorkFlowLineageMapper.xml

* Delete createTenement.vue

* recove wrongly deleted file

* Update WorkFlowLineageMapper.xml

* Update createTenement.vue

* [Feature][API]enable response resources gzip compression (#4121)

* enable response compression

* add server.compression.mime-types with default value explicitly

* [Improvement][API] ignore noNodeException when get worker groups (#4120)

* ignore noNodeException when get worker groups

* add ut

* [Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)

* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* [FIX#4033] $[] conflicts with mysql keywords (#4111)

* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error

* split sqoop import hive database and table (#4141)

* upgrade quartz version to 2.3.0

* add HikariCP-java6,c3p0,mchange-commons-java license

* upgrade jackson version to 2.9.10

* remove c3p0,mchange-commons-java license

* update Check code style

* update  code style

* Update pom.xml

* Update pom.xml

* Update LICENSE

not need to add HikariCP-java6

* Delete LICENSE-HikariCP-java6.txt

* Update known-dependencies.txt

* Update TaskInstanceMapper.xml

optimize page of [TaskInstance] load data slow

* [FIX-PR-4097][server-master]task ack miss (#4189)

When the message of successful execution arrives earlier than
the message of ack,
the message of ack will be discarded,
resulting in some information missing

* fix bug #4125 (#4127)

* fix bug #4125

* code style!

* code style.

Co-authored-by: chengp <chengp@chengp.net>

* [FIX-4190][DAO] When the amount of json data is large, the process list page display slowly. (#4201)

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

* fix 4190: When the amount of json data is large, process list page display slowly

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-3457][flink] fix flink args build problem (#4166)

* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local

* [common]del windows file( not support windows ) (#4204)

* del support win

* del support win

* [Feature-4138][Master] dispatch workgroup error add sleep time (#4139)

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* When there are tasks with assignment failure and the number of tasks in the current task queue is less than 10, sleep for 1 second

* fix code smell & code style

* fix code smell & code style

Co-authored-by: zhanglong <zhanglong@ysstech.com>

* [FIX-#4172][server-worker] kill task NPE (#4182)

* [FIX-#4172][server-worker] kill task NPE

The cache task will be sent when the Process is generated. Before that, if a kill task appears, then NPE will appear
Modification method: write into the cache when the task is received, and mark it as preData
If the task is killed before the Process is generated, delete the cache directly at this time
It will be judged before the process is generated. If the task has been killed, it will not be executed.
After the new process is created, write it into the cache, and judge again, if kill, then kill the process.

this closes #4172

* Delete the commented out code
Add spring beans

* code smell

* add test

* add test

* fix error

* test

* test

* revert

* fix error

* [Feature-3878]Replace the page with element-ui (#4065)

* [Feature-3878]Introduce elment-ui and replace the security center module page with elment-ui

* Change node version

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Replace the page with element-ui

* Replace user management with elment-ui

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Repair the last Sunday of each month

* support auto eslint for .js, .vue file on save

* update .eslintrc.yml and license check exclude .eslintignore

* fix eslint: fix syntax by npm run lint automatically

* fix eslint: fatal syntax errors

* fix eslint: expected '!==' but instead saw '!=', expected '===' but instead saw '=='

* fix eslint: Unexpected side effect in 'cacheParams' computed property

* fix eslint: assigned a value but never used

* fix eslint: component has been registered but not used

* fix eslint: unexpected mutation of prop

* fix bug: start from the setting nodes with NODE_PRE would be NPE. (#4219)

* deleted invalid code as assigned in issue 4215 (#4221)

deleted the code at following two places:
org.apache.dolphinscheduler.service.process.ProcessService#checkTaskExistsInTaskQueue
org.apache.dolphinscheduler.service.process.ProcessService#taskZkInfo

* [FIX-#3177]Task time parameter parsing error (#4224)

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* fix sql error (#4227)

* [FIX-3177]Task time parameter parsing error (#4228)

* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import

* fix date_convert null

* fix time cover

* fix time cover

* reformat

* add ut

* [Fix][UI]: fix re-login problem in new tab and state synchronization problem in multiple tabs (#4162)

* fix(ui): re-login problem in new tab

* refresh page in new tab automatically

* support to reload router view in lightweight

* optimize visibility code

* [Improvement][UI] Add no-var rule for eslint and add 'npm run lint:fix' command (#4225)

Add no-var rule for eslint, and fix related files
Add npm run lint:fix command, which distinguished from the default command npm run lint
The behavior of command npm run lint and command npm run lint:fix should be distinguished
The first command is used to lint only, and the second one is used to lint and fix problems automatically

* [FIX][UI ]  security user state  tenantCode User Type Display abnormal (#4255)

* fix user state error

* fix userType  error

* fix tenantCode  error

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority. (#4250)

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Add the priority queue to ensure that tasks are submitted according to priority.

* [Fix-4222][Master]Remove useless import

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]add PeerTaskInstancePriorityQueueTest

* [Fix-4222][Master]Fix code smell

* [Fix-4222][Master]Reformat code style

* [Fix-4222][Master]Fix code smell

Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* update bug template (#4260)

* [Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)

* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.

* remove AbstractTask.getCurTaskParamsClass()and replace with TaskParametersUtils.getParameters() (#4262)

* remove getCurTaskParamsClass() in AbstractTask.java and replace it with TaskParametersUtils.getParameters()

* remove unused imports in AbstractTask.java

* reformat

* [Fix-4271][server] Fix IOException or NoSuchFileException in logger server (#4272)

* fix version funcation delete. (#4265)

* [improvement][config] Update datasource.properties,add mysql meta data template (#4266)

* Update datasource.properties

[新增]元数据mysql连接模版

* Update datasource.properties

Co-authored-by: dailidong <dailidong66@gmail.com>

* fix taskInstance submitTime is empty. (#4274)

* [Improvement-3878]Tenant list delete user name (#4278)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html (#4269)

* [Fix-4268] Fix NumberFormatException when visiting a doc.html or swagger-ui.html

* Fix checkstyle error

* Update swagger-models in known-dependencies.txt

* [Improvement-3878][ui]Fix the list style (#4280)

* [Improvement-3878]Tenant list delete user name

* [Improvement-3878][ui]Fix the list style

* [FIX] [UI] fix create project cancel button invalid (#4282)

* fix create project cancel button invalid

* [Improvement] Refactor code to support distributed tracing (#4270)

* Refactor code to support tracing

* Extension network protocol, support context and version
* Extension master asynchronous queue support context
* Extract scan task method from MasterSchedulerService for tracing

* fix

* fix

* add test case

* fix

* fix

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Improvement] Add Flink job name (#4285)

* [Improvement] Add Flink job name

* fix typo

* [Improvement][ui]List vacancy optimization and icon icon repair (#4286)

* [Improvement][ui] List vacancy optimization and icon icon repair

* [FEATURE-736] integrate ldap authentication (#3743)

* [FEATURE-736] integrate ldap authentication

* add ldap authentication type
* refactor authentication with ldap and password
* add  createUser for ldap in user service
* remove duplicate password authenticator

* [Fix-4289][*] Flink name with disappeared and unescaped problem (#4290)

* fix flink name not display in process definition editor

* fix flink name not escape problem

* simplify escape method

* fix database is the mysql DB keyword. (#4295)

* fix the token management list does not display the user name. (#4302)

* Dev imp server process utils (#4263)


* Adds the comment on the WINDOWSATTERN

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>

* fix version close function does not take effect. (#4307)

* fix user management authorization operation exceptions do not prompt exception information. (#4292)

* [Fix][UI] Fix ui style problem and refactor form style (#4329)

* rename from-model to form-model

* [UI] remove duplicated css style code

* [UI] refactor css style of form model

* fix form model overflow problem in ie

* Align each item one by one in locale

* fix el-dialog width is too wide

* fix locale

* [Improvement][UI] Improve script box and dialog css style (#4331)

* [Fix-4335][UI] Fix IE 9-11 not supported

* [FIx-4338][UI] Fix invalid date problem in IE

* [Improvement] Use environment variable $DOLPHINSCHEDULE_OPTS as daemon startup arguments (#4341)

Co-authored-by: hailin0 <hailin0@yeah.net>

* [Fix][common] only two yarns can be selected(#4314) (#4344)

* Handling conflicts manually

* code style

* ut skip ui

* fix ut error

* fix ut error

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* delete SchedulerController Unused field

* ci skip ui

* ignore AlertPluginManagerTest.java and DolphinPluginLoaderTest.java

* fix ut error

* fix ut error

* test

* code style

* replace ui

* replace ui

* replace ui

* replace ui

* fix license

* fix license

Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: spring-bu <37682426+spring-bu@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: BoYiZhang <zhangboyi_mx@163.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: zhanglong <zhanglong@ysstech.com>
Co-authored-by: 孙继峰 <sun.jifeng@outlook.com>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: IamMujuziMoses <49279842+IamMujuziMoses@users.noreply.github.com>
Co-authored-by: huzekang <1040080742@qq.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: Eights-LI <yelli.eights@gmail.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: hailin0 <hailin0@yeah.net>
Co-authored-by: yangquan <iyeeku@qq.com>
Co-authored-by: liliang1991 <liliang_68@126.com>
Co-authored-by: liang.li.c <liang.li.c@17zuoye.com>
Co-authored-by: felix.wang <59079269+felix-thinkingdata@users.noreply.github.com>
Co-authored-by: chengshiwen <chengshiwen0103@gmail.com>
Co-authored-by: Yarlung <turingdojo@163.com>
Co-authored-by: karlsun <karlsun@tencent.com>
Co-authored-by: wangxj3 <857234426@qq.com>
Co-authored-by: chengp <5058557@qq.com>
Co-authored-by: chengp <chengp@chengp.net>
Co-authored-by: Saksham Gupta <saksham1319@gmail.com>
Co-authored-by: Tq <36755957+Tianqi-Dotes@users.noreply.github.com>
Co-authored-by: Jatham <59549328+Jatham922@users.noreply.github.com>
Co-authored-by: hailin0 <hailin0@foxmail.com>
Co-authored-by: zh0122 <zh0122@gmail.com>
Co-authored-by: geosmart <geosmart@hotmail.com>
Co-authored-by: 李丛阳 <liby-1987@163.com>
Co-authored-by: 0002939 <licongyang@mininglamp.com>
Co-authored-by: kamisamak <1057372918@qq.com>
2021-01-08 15:56:19 +08:00
chengshiwen e3547a37cb refactor code styles 2021-01-07 15:32:01 +08:00
chengshiwen 5f39792c66 remove the operation resume_from_forced_success 2021-01-07 14:53:34 +08:00
dailidong 325bc2e9e5
Merge branch 'dev' into dev 2021-01-05 16:13:03 +08:00
kamisamak ccd06fec16
[Fix][common] only two yarns can be selected(#4314) (#4344) 2020-12-30 11:03:36 +08:00
Kirs 17d44216a4
[Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages (#4311)
* [Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages

*Some plugins (such as alert plugin) need to provide UI interfaces to users.
*We use from-creat to dynamically generate UI interfaces. Related parameters are mainly provided by pluginParams.
*From-create can generate dynamic ui based on this parameter.

this closes #4310

* add license head

* rename

* add ut

* add license

* add query plugin detail interface

* fix error
2020-12-28 11:25:00 +08:00
xingchun-chen 55447348ed
Merge pull request #4237 from CalvinKirs/fix_date_convert_null
[FIX-3177]Task time parameter parsing error
2020-12-21 15:54:40 +08:00
chengshiwen ae0ee3fe32 fix code style and test 2020-12-20 23:41:03 +08:00
zhuangchong 3bcd3fbcd3
[Improvement-4069][server] When the tenant does not exist, the task execution should throw an exception (#4108)
* when  the tenant does not exist, the task execution should throw an exception

* remote method createWorkDirAndUserIfAbsent

* set the task status failed when the tenant code does not exist.

* add taskLog.

* update check os user exists

* update TaskExecuteThreadTest test method.

* solving sonar fail.
2020-12-20 21:17:39 +08:00
chengshiwen af720d3ef1 match frontend and backend for force success 2020-12-20 20:00:55 +08:00
chengshiwen c3b5c5fe40 merge force task success from legen618/dev-FTS 2020-12-20 15:45:34 +08:00
CalvinKirs 9d6ad4d883 fix date_convert null 2020-12-16 10:40:40 +08:00
Kirs d7b3936205
[FIX-3177]Task time parameter parsing error (#4228)
* [FIX-3177]Task time parameter parsing error
when system.datetime !=null $[datetime] = system.datetime
else $[datetime] = current time

* remove unused import
2020-12-15 16:24:31 +08:00
Kirs 8919e3838f
[FIX-#3177]Task time parameter parsing error (#4224)
* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177

* [FIX-#3177]Task time parameter parsing error

rerun schedule time error

this closes #3177
2020-12-15 10:30:36 +08:00
zhuangchong 169a4ace77
[Feature-3749][Alert-SPI] SqlTask should send notifications by alert server api (#4080)
* add sqltask send sync alert server.

* update alert-sms license.

* update AlertServer test.

* remote EmailAlertPluginTest.

* update sqltask.

* update test class.
2020-12-10 22:41:11 +08:00
Kirs 0039b1bfcb
[common]del windows file( not support windows ) (#4204)
* del support win

* del support win
2020-12-10 22:39:20 +08:00
Shiwen Cheng cbc30b4900
[Fix-3457][flink] fix flink args build problem (#4166)
* [Fix][Flink] fix flink args build problem

* [Fix][Flink] fix FlinkArgsUtilsTest

* [Improvement][UI] hide version and cluster input when deployMode is local
2020-12-10 22:37:21 +08:00
Kirs 3ceb612467
[FIX#4033] $[] conflicts with mysql keywords (#4111)
* [FIX#4033] $[] conflicts with mysql keywords
We currently only use this symbol for dates, so I filtered out the number type.
this close #4033

* test

* fix error
2020-12-01 14:34:03 +08:00
felix.wang 3e411d075f
[Improvement][Code style] FIX SPELL WAITTING TO WAITING , etc. (#4118)
* FIX SPELL

* FIX SPELL AND  Optimizing code conventions

* add ut  cannot construct process instance, return null;

* add ut testExportProcessMetaData

* add ut testExportProcessMetaData

* add ut testImportProcessSchedule

* add ut MasterExecThreadTest

* add ut MasterExecThreadTest

* add ut testSubProcessViewTree

* add ut testComplementWithStartNodeList

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId

* add ut testRecurseFindSubProcessId
2020-12-01 08:58:55 +08:00
BoYiZhang bb13f2eae7
[Bug][Common] read file garbled (#3479)
* fix bug : Random code problem

Co-authored-by: zhanglong <zhanglong@ysstech.com>
2020-11-27 21:55:28 +08:00
Yelli 145314b782
[FIX-4034][server] fix sqoop import fail (#4036)
* fix #4043, sqoop import query fail

* fix #4043, sqoop task hard code & code style

* add license for SqoopConstants

* add private constructor for SqoopConstants

* fixed sqoop mysql pwd have special character

* fix checkstyle

* fix sqoop task log

* remove unused constants
2020-11-25 11:11:44 +08:00
felix.wang 8f5da09d3e
fix replaceNRTtoUnderline NullPointerException #4098 (#4100)
* fix replaceNRTtoUnderline NullPointerException

* add  unit Test
2020-11-24 18:32:50 +08:00
Eights-LI c24dbc448d deal with magic value 2020-11-17 22:25:25 +08:00
BoYiZhang fe3026627f
[Feature-3985][Datax] Datax supports setting up running memory (#3986)
* Datax supports setting up running memory

* Datax supports setting up running memory

* Datax supports setting up running memory

* When running a task, the resource file is lost, which results in an error

* add unit test

* add unit test

* add unit test

* add test unit

* add test unit

* add test unit

* fix code smell

* add test unit

* add test unit
2020-11-16 10:55:20 +08:00
dailidong 46091bb870
Merge pull request #4058 from apache/133-merge-dev
[Merge][133-dev] merge from 1.3.3-release to branch  dev
2020-11-12 11:42:29 +08:00
hailin0 95374c30b9
[Fix][API] Condition task null pointer exception (#4056) 2020-11-12 11:36:34 +08:00
bao liang 02dbd3ee06
Revert "[Draft][Merge][133-dev]133 merge dev (#4031)" (#4057)
This reverts commit ad2d9f99d0.
2020-11-12 11:33:46 +08:00
bao liang ad2d9f99d0
[Draft][Merge][133-dev]133 merge dev (#4031)
* [Feture-3327][ui]Add the function of re-uploading files in the resource center

* [Feture-3327][ui]Add the function of re-uploading files in the resource center (#3394)

* Before creating a workflow, clear the canvas

* [Fix-3256][ui] herry pick commit from dev for Fix admin user info update error (#3306)

* [Feture-3327][ui]Add the function of re-uploading files in the resource center

Co-authored-by: wuchunfu <319355703@qq.com>

* [Improvement-3327][api]support re-upload the resource file (#3395)

* [Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version

* hive UDF function to modify the background color

* fix

* fix bug: Fix master task dependency check bug

* cancel spark task version check (#3406)

Co-authored-by: Eights-Li <yelli.hl@gmail.com>

* [Bug][ui]Fix front-end bug #3413

* [Feature][ambari_plugin]support one worker can belongs different worker groups when execute install script (#3410)

* Optimize dag

* Update actions.js (#3401)

* [Fix-3256][ui] Fix admin user info update error (#3425) (#3428)

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [PROPOSAL-3139] Datasource selection changes from radio to select

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [BUG FIX] issues #3256

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* [Fix-3256][ui] Fix admin user info update error

* reset createUser.vue

* [Fix-3256][ui] Fix admin user info update error

Co-authored-by: dailidong <dailidong66@gmail.com>

Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [Fix-3433][api]Fixed that release the imported process definition which version is below 1.3.0 will be failure

* dag connection add check

* fix

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x (#3454)

* Remove node deep monitoring

* If worker group id is null,don't need to set the value of the worker group (#3460)

* [Fix-3423][dao][sql]Fixed that the resource file of the task node can't be found when upgrade from 1.2.0 to 1.3.x

* [Fix-3423][dao]If worker group id is null,don't need to set the value of the worker group

* [ui]Code optimization

* fix

* fix

* [fix-3058][ui]Move rtTargetArr to jsPlumbHandle.js

* [optimization][ui]Prevent the shell script input box from being empty

* [Fix-3462][api]If login user is admin,need list all udfs (#3465)

* [Fix-3462][api]If login user is admin,need list all udfs

* [Fix-3462][api]add the test on the method of QueryUdfFuncList

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3462][api]fix the code smell

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource (#3482)

* [fixBug-3058][ui]Fix connection abnormalities in historical workflow instance data

* [Feture-3327][ui]Add the function of re-uploading files in the udf subdirectory

* fix bug: Fix master task dependency check bug (#3473)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ci_e2e fail (#3497)

* [Fix-3469][api]Should filter the resource by the different program type (#3498)

* [Fix-3463][api]Fixed that run the sql task will be failure after rename the udf resource

* [Fix-3469][api]Should list python file and jar file

* [Fix-3469][api]Should filter the resource by the different program type

* [Fix-3469][api]fix the code smell

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3499)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* [Fix-3469][ui]The value of maintenance resources and the filtering of resources according to different program types

* fix

* Revert "fix ci_e2e fail (#3497)"

This reverts commit e367f90bb7.

* test

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback (#3503)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* test release 1.3.2 version rollback (#3504)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* fix ds muti-level directory in zk, which lead to fail to assign work

* add login user check some actions in api

* [Hotfix][ci] Fix e2e ci docker image build error

* modify tag 1.3.0 to HEAD

* modify tag 1.3.0 to HEAD (#3525)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* remove OGNL part of the mybaits notice (#3526)

* [maven-release-plugin] prepare release 1.3.2

* [maven-release-plugin] prepare for next development iteration

* release 1.3.2 version rollback (#3527)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Script input box to modify the delay loading time

* fix

* fix

* fix

* fix

* modify general user can't create token

* [ui]It is forbidden to select non-existent resources and modify the tree display data format

* modify general user can't create token (#3533)

* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

* modify general user can't create token

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* if task is null , set task type is null instead of "null"

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE (#3537)

* [Fix-3536][api]If user didn't have tenant,create resource will NPE

* [Fix-3536][api]If user didn't have tenant,create resource directory will NPE

* modify general user can't create,delete,update token (#3538)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [ui]Resource delete OK button to increase loading, change the number of homepage display cursor

* fix

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3748)

* [fixbug][ui]Repair the master and worker management instrument display

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode (#3741)

* [fixbug][ui]Remove non-existent or deleted resources disabled

* [fixBug-3621][ui]If the workflow instance status is executing status, it is forbidden to select

* [fix-3553][ui]Repair click workflow connection, select the entire path

* fix

* fix

* [Fix-3238][docker]Fix that can not create folder in docker with standalone mode

* [Fix-3616][Server] when worker akc/response master exception , async retry (#3776)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Improvement-3720][ui] js mailbox verification fix

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3784)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3124][docker]Fix that can not build a docker image on windows (#3765)

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect (#3786)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>

* [Fix-3258][Security][Worker group manage] Connot get create time and update time,report DateTimeParseException (#3787)

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* [Fix-3616][Server] when worker akc/response master exception , async retry

* The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected.

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [Fix-3549] [Server][sqlTask]The alias column in the query SQL does not take effect

* [BugFixed] issue #3258 (#3265)

* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* [fixBug-3792][ui]Click on the sidebar to adapt the width of the pie chart on the project homepage

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

* [Fix-#3713][common]Fix that catfile method Stream not closed (#3810)

* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3487][api、dao] cherry pick from dev to fix that create folder duplicate name under multithreading

* [Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)

* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>

* Set up JDK 11 for SonarCloud in github action. (#3052)

* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.

* [fixBug-3621][ui]Select the batch checkbox to unfilter the instances in the executing state

* add verify tenant name cannot contain special characters.

* [fixBug-3840][ui]The tenant code only allows letters or a combination of letters and numbers

* fix

* fix

* fix

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it need replace the origin resource file. (#3862)

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [Fix-#3702][api] When re-upload the resource file but don't change the name or desc,it will not replace the origin resource file.

* [fixbug-3621][ui]Workflow instance ready to stop and ready to suspend state prohibits checking

* [fixbug-3887][ui]Fix missing English translation of re-upload files

* add process define name verify. (#3879)

* Revert "[1.3.3-release][fix-3835][ui] When the tenantName contains "<", the tenant drop-down list is blankadd verify tenant name cannot contain special characters."

* revert pr 3872

* [FIX-3617][Service]after subtask fault tolerance, 2 task instances are generated (#3830)

* fix bug(#3617): after subtask fault tolerance, 2 task instances are generated.

* delete unused code

* update code smell

* refactor sub work command process

* add process service ut

* add license header

* fix some code smell

* chang ut java8 to java11

* update sonar to java11

* copy ut config from dev

* remove checkstyle

* revert to 1.3.3

* change proess service test to executor service

* add process service test

* add process service test

* revert

* revert

* add comments

* change dev to 1.3.3-release

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3487][sql] add dolphinscheduler_dml.sql under 1.3.3_schema (#3907)

* [FIX-3836][1.3.3-release-API] process definition validation name interface prompt information error  (#3899)

* fix bug : error message

* fix code smell

* fix code smell

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat (#3913)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

Co-authored-by: Kirs <acm_master@163.com>

* Repair check box cannot be canceled

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly (#3918)

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [FIX_#3789][remote]cherry pick from dev to support netty heart beat

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

* [fix-3843][api] When update workflow definition,if name already exists, the prompt is not friendly

Co-authored-by: Kirs <acm_master@163.com>

* [Fix-#3487][sql] update uc_dolphin_T_t_ds_resources_un

* Workflow definition name re-modified and added check

* [fix-#3843][api]When update workflow definition,if name already exists, the prompt is not friendly.

* update code.

* [#3931][ui]Field name optimization for spark, flink, and mr

* change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934)

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [ambari-plugin]change version 1.3.2 to 1.3.3 (#3935)

* fix bug:3615 After the task is executed successfully, but the next task has not been submitted, stop the master

* [fixBug-3964][ui]Switch back and forth over timeout alarm, the selected value is empty

* solve too many files, close logClientService (#3971)

* fix #3966 sub process doesnot send alert mail after process instance ending. (#3972)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [Fix-#3618][server] resolve task executed finished but not release the file handle (#3975)

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3618][server] resolve task executed finished but not release the file handle

* [Fix-#3958][api] files should not be created successfully in the directory of the authorized file

* [FIX-3966] The timeout warning does not take effect in sub_process (#3982)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix import dolphinscheduler_mysql.sql insert admin user data

* [FIX-3929] condition task would post wrong tasks when failover. (#3999)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [FIX-3929]  because of no lock, start up failover would dispatch two same tasks. (#4004)

* fix #3966 sub process doesnot send alert mail after process instance ending.

* fix bug 3964: sub_process The timeout warning does not take effect
add timeout warning for sub_process/dependent task.

* fix code smell

* fix code smell

* fix code smell

* update worker group inherit from parent

* remove stdout in logback configuration

* fix bug #3929 condition task would post error when failover.

* remove unused test

* add comments

* add skip node judge

* fix bug 3929: because of no lock, start up failover would dispatch two same tasks.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* revert pom version to 1.3.3-release

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* [release]revert pom version to 1.3.3-release

* fix bug 4010: remove failed condition tasks from error-task-list. (#4011)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [maven-release-plugin] prepare release 1.3.3

* [maven-release-plugin] prepare for next development iteration

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut test

* refactor ut test

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor ut

* merge from 1.3.3-release

* refactor ut

* refactor ut

* refactor

* refactor

* refactor code style

* refactor code style

* refactor code style

* refactor code style

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* merge from 1.3.3-release

* refactor code style

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: wuchunfu <319355703@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: Yelli <amarantine@my.com>
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
Co-authored-by: JinyLeeChina <42576980+JinyLeeChina@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: XiaotaoYi <v-xiayi@hotmail.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
Co-authored-by: zhuangchong <zhuangchong8@163.com>
Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: Jave-Chen <baicai.chen@gmail.com>
Co-authored-by: zhuangchong <zhuangchong6@163.com>
Co-authored-by: zhuangchong <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: lgcareer <lgcareer@apache.org>
Co-authored-by: wulingqi <wulingqi@baijiahulian.com>
2020-11-12 11:27:05 +08:00
bao liang 7239979167
Merge branch 'dev' into 133-merge-dev 2020-11-09 10:08:19 +08:00
zhuangchong 4b502e361f
[Improvement-4012][common/remote] Json util code integration,remove the remote module json util (#4013)
* Json util code integration,remove the remote module json util.

* update code checkstyle.

* update code checkstyle.

* add code checkstyle.

* add test class.

* update JSONUtils class.

* update JSONUtils class.
2020-11-07 19:37:48 +08:00
baoliang 403a4a20f2 merge from 1.3.3-release 2020-11-06 11:44:34 +08:00
baoliang 381d23ea4f merge from 1.3.3-release 2020-11-06 00:03:20 +08:00
lgcareer be07cb62db [maven-release-plugin] prepare for next development iteration 2020-10-29 17:49:28 +08:00
lgcareer 55242916b2 [maven-release-plugin] prepare release 1.3.3 2020-10-29 17:49:18 +08:00
lgcareer b01898a8eb [release]revert pom version to 1.3.3-release 2020-10-29 17:29:38 +08:00
lgcareer 1076a2349c [maven-release-plugin] prepare for next development iteration 2020-10-28 19:19:46 +08:00
lgcareer ed016499aa [maven-release-plugin] prepare release 1.3.3 2020-10-28 19:19:35 +08:00
lgcareer 0d21fa9e3b revert pom version to 1.3.3-release 2020-10-28 19:11:35 +08:00
Zhou.Z 897ce2d183
Merge branch 'dev' into dev-FTS 2020-10-25 12:24:15 +08:00
zhuangchong 30d75f2185 the alert module support service. 2020-10-23 11:58:28 +08:00
BoYiZhang 7dd717af16 When running a task, the resource file is lost, which results in an error 2020-10-21 23:32:22 +08:00
BoYiZhang 48177b650e When running a task, the resource file is lost, which results in an error 2020-10-20 20:26:09 +08:00
BoYiZhang ef6cc62b96 When running a task, the resource file is lost, which results in an error 2020-10-20 20:20:01 +08:00
BoYiZhang aace551782 When running a task, the resource file is lost, which results in an error 2020-10-20 20:15:23 +08:00
BoYiZhang fc565813ef When running a task, the resource file is lost, which results in an error 2020-10-20 19:25:14 +08:00
lgcareer 92a793288f [maven-release-plugin] prepare for next development iteration 2020-10-17 18:14:23 +08:00
lgcareer c144d81579 [maven-release-plugin] prepare release 1.3.3 2020-10-17 18:14:13 +08:00
lgcareer ae85116fa5
change version from 1.3.2-SNAPSHOT to 1.3.3-SNAPSHOT (#3934) 2020-10-17 17:50:02 +08:00
dailidong 787845578b Merge remote-tracking branch 'origin/dev' into dev 2020-10-16 16:02:37 +08:00
dailidong f430fed8f9 Merge remote-tracking branch 'upstream/dev' into dev 2020-10-16 16:01:12 +08:00
dailidong 15436cfcdd
Delete ThreadUtilsTest.java
not useful test
2020-10-16 13:49:57 +08:00
dailidong ed3f08916f
Update ThreadUtilsTest.java 2020-10-16 13:16:03 +08:00
dailidong 29a083ca8f
Update ThreadUtilsTest.java 2020-10-16 10:43:46 +08:00
Zhou.Z 37762c29ea
Merge branch 'dev' into dev-FTS 2020-10-13 13:49:34 +08:00
gaojun2048 1d254347df [Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601)
* DS SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* add TODO

* delete

* compile

* spi commit

* Plugin Alert

* fix some bug

* add todo

* change web ui from alpacajs to form-create

* remove module

* add plugin schema

* add license header

* update alert and spi module version

* update the alert plugin sub module version

* comment the maven.local.repository param

* move utils from spi to common module

* add license header

* add license header and delete some chinese comment

* update spi packages

* delete no use alert_xx.properties

* update mysql.connector.version back to 5.1.34

* delete no use comment in pom.xml

* update email stmp password

* add license

* add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file

* format the code style

* format new clase file with checkstyle

* update plugin params to Builder model

* move JSONUtils to SPI because plugin can not dependency common module

* move JSONUtils to SPI because plugin can not dependency common module

* delete collection dependency

* replace PluginParamsTransfer to spi PluginParamsTransfer

* update dolphinscheduler-maven-plugin to 1.0.0

* update license

* update apache-rat-plugin add exclude '.iml' file

* check license

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* default datasource should be postgresql

* add license files

* add license files

* postgresql port should be 5432

* postgresql test

* mv show_type to spi

add license header to AlertConstants

* check style fix

* copy check style file from branch dev

* alert show_type set by plugin

* alert show_type set by plugin

* add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java

* add Bean to TaskCallbackServiceTestConfig

* add Bean to TaskCallbackServiceTestConfig

* fix check style

* check style fix

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* fix check style

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* check style fix

* rollback test change

* rollback test change

* rollback dao pom change

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* add ut to pom.xml

* add upgrade schema to global schema

* fix ut failed

* fix ut failed

* fix ut failed

* fix ut failed

* add test EmailAlertPluginTest to pom.xml

* fix ut failed

* fix ut failed

* fix check style

* update license header to presto license header

* presto license header not check

* fix ut coverage

* fix ut coverage

* fix ut

* fix ut

* fix ut

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

Co-authored-by: break60 <790061044@qq.com>
2020-10-12 15:28:11 +08:00
dailidong 8e0fb35366
Update dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/threadutils/ThreadUtilsTest.java
Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-10-08 23:27:36 +08:00
guyinyou 3b581455fc
[Feature]Add Python task "task variable / result transfer" implementation (#3659)
* 增加Python Task的“任务变量/结果传递”实现
Signed-off-by: 古崟佑

* add two files license
Signed-off-by: 古崟佑

* fix 'server/worker/task/AbstractCommandExecutor.java' code style
Signed-off-by: 1941815847Cy4 <1941815847cy4@kuaishou.com>

* update DB
Signed-off-by: 古崟佑

* update DB -- 2
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codestyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* add VarPoolUtils Test
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest codeStyle
Signed-off-by: 古崟佑

* add test config for VarPoolUtilsTest
Signed-off-by: 古崟佑

* fix unit test
Signed-off-by: 古崟佑

* fix codeStyle
Signed-off-by: 古崟佑

* fix VarPoolUtilsTest.java
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

* change the test class path
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

* fix "print the error message"
Signed-off-by: 古崟佑

* fix bug
Signed-off-by: 古崟佑

* fix
Signed-off-by: 古崟佑

Co-authored-by: 1941815847Cy4 <1941815847cy4@kuaishou.com>
2020-09-25 13:20:42 +08:00
lgcareer 2e4f9f19ce
[Fix-#3713][common]Fix that catfile method Stream not closed (#3810)
* [Bug-3713][HadoopUtils] catfile method Stream not closed  (#3715)

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>

* [Fix-#3713][common]Fix that catfile method Stream not closed

Co-authored-by: BoYiZhang <39816903+BoYiZhang@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
2020-09-24 15:28:02 +08:00
BoYiZhang 375ec125c5
[Feature-3773][processInstance, taskInstance] Change title name : Replacement System.currentTimeMillis () (#3774)
* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* Replacement System.currentTimeMillis () to DateUtils.getCurrentTimeStamp()

* update checkstyle

* update checkstyle

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
2020-09-22 17:38:17 +08:00
zhuangchong 36d5349610
[fix-3745][server] server get tasktype NPE exception (#3746) 2020-09-17 13:58:43 +08:00
qiaozhanwei 29f1123d1d
[Fix-3616][Server] when worker akc/response master exception , async retry (#3748) 2020-09-17 10:06:59 +08:00
zhuangchong fad2852768
[Improvement][server] WATERDROP task plug-in optimization in switch case code cleaning. (#3652)
* WATERDROP switch case code checkstyle.

* add TaskManagerTest.

* TaskManagerTest checkstyle.

* TaskManagerTest checkstyle.

* TaskManagerTest add pom maven-surefire

* TaskManagerTest update.
2020-09-14 15:59:56 +08:00
felix.wang 565b8d3b66
[Improvement-3690][common] Get the native IP policy problem (获取本机ip策略问题 ) (#3695)
* Get the Intranet IP

* Get the Intranet IP

* fix code smell

* fix code smell

* fix code smell

* fix code smell

* support ipv6  --x:--x:--x:--x:--x:--x:--x:--x

* Update dolphinscheduler-common/src/main/resources/common.properties

Co-authored-by: Yichao Yang <1048262223@qq.com>

Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-09-14 10:39:37 +08:00
BoYiZhang 7af20ca3af
[Bug-3713][HadoopUtils] catfile method Stream not closed (#3715)
* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* catfile method Stream not closed

* catfile method Stream not closed

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
2020-09-12 23:42:40 +08:00
Zhou Zheng cf9576b74a Merge branch 'dev' into dev-FTS 2020-09-08 11:58:06 +08:00
dailidong 27139913c0 Merge remote-tracking branch 'upstream/dev' into dev 2020-09-07 15:56:04 +08:00
小清 ba3adbd34b
[FIXBUG][common]fix yarn applications status cause NPE(#3645) (#3648)
* fix #3645

fix get yarn applications status from yarn server,http code return 404, get yarn applications status cause NullPointerException

* Update HadoopUtils.java

* Update HadoopUtils.java
2020-09-07 10:36:43 +08:00
Yichao Yang ac4ed94061
[Improvement][common] Add UnsupportedOperationException for utils construct (#3381)
* [Improvement][common] Add UnsupportedOperationException for utils construct

* Fix checkstyle
2020-09-02 15:59:47 +08:00
Zhou Zheng 7e4bf1902c Merge branch 'dev' into dev-FTS 2020-08-26 22:54:11 +08:00
Zhou Zheng de19fabbc4 change back to original import order 2020-08-26 22:30:02 +08:00
zshit 26184de905
fix bug#3583 (#3585) 2020-08-25 11:15:43 +08:00
felix.wang 2ba529a96c
[Improvement][alert] Optimize alarm information (#3559)
* #3299  Json string parsing problem caused by non-standard json format.

* #3299  Json string parsing problem caused by non-standard json format.

* #3299  Json string parsing problem caused by non-standard json format. fix  code style

* #3299  Json string parsing problem caused by non-standard json format. fix  code style

* Optimize alarm information

* fix code style

* add license

* fix code smell

* Update dolphinscheduler-alert/src/main/java/org/apache/dolphinscheduler/alert/utils/EnterpriseWeChatUtils.java

Co-authored-by: Yichao Yang <1048262223@qq.com>

* fix code smell

* fix code smell

* 修改ut 和replace问题。

* 修改ut

Co-authored-by: wangjianda <Felix@thinkingdata.com>
Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-08-25 11:10:44 +08:00
Zhou Zheng 31cd1f6cb2 adapt to code style 2020-08-23 15:14:18 +08:00
Zhou Zheng 295796a5c5 fix 2020-08-22 22:47:59 +08:00
Zhou Zheng 6bdd11da51 Merge branch 'dev' into dev-FTS 2020-08-22 22:31:07 +08:00
vanilla111 a4ee351a3a
delay execution of tasks and improve some designs (#3427) 2020-08-19 11:36:30 +08:00
dengc367 ad89f433f1
[Feature]modify some cases from rockxsj:Feature-presto to add presto datasource support (#3468)
* Feature presto (#1)

* * add presto datasource support
update .gitigonre to igonre some files

* * use another presto driver

* * add LICENSE files about presto-jdbc

* * just for test sonar

Co-authored-by: rockxsj <junjun@mgtv.com>

* modify the io.prestosql.jdbc.PrestoDriver to com.facebook.presto.jdbc.PrestoDriver

* add presto connection in sql node

Co-authored-by: rockxsj <junjun@mgtv.com>
2020-08-17 17:28:46 +08:00
qiaozhanwei 917a4b20b5
release 1.3.2 version rollback (#3527)
* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-08-17 14:21:13 +08:00
qiaozhanwei 32edd19ac5 [maven-release-plugin] prepare for next development iteration 2020-08-17 14:03:19 +08:00
qiaozhanwei 4d2f0b0578 [maven-release-plugin] prepare release 1.3.2 2020-08-17 14:03:08 +08:00
qiaozhanwei 11d3c07e46
modify tag 1.3.0 to HEAD (#3525)
* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

* modify tag 1.3.0 to HEAD

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-08-17 11:46:35 +08:00
qiaozhanwei f3f895751b [maven-release-plugin] prepare for next development iteration 2020-08-14 19:20:27 +08:00
qiaozhanwei f19b125374 [maven-release-plugin] prepare release 1.3.2 2020-08-14 19:20:17 +08:00
qiaozhanwei 4ba6b35b0f
test release 1.3.2 version rollback (#3504)
* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-08-14 18:55:03 +08:00
qiaozhanwei f50353f8a6 [maven-release-plugin] prepare for next development iteration 2020-08-14 18:26:09 +08:00
qiaozhanwei afd227a179 [maven-release-plugin] prepare release 1.3.2 2020-08-14 18:25:59 +08:00
qiaozhanwei 660eb5a1e6
test release 1.3.2 version rollback (#3503)
* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

* test

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-08-14 18:19:19 +08:00
qiaozhanwei b5f0f23d85 [maven-release-plugin] prepare for next development iteration 2020-08-14 17:37:33 +08:00
qiaozhanwei 88b3ef5d30 [maven-release-plugin] prepare release 1.3.2 2020-08-14 17:37:23 +08:00
qiaozhanwei 45424ec265
test release 1.3.2 version rollback (#3499)
* [Feature] JVM parameter optimization , related issue #3370

* [Feature] JVM parameter optimization , related issue #3370

* test release 1.3.2 version rollback

* test release 1.3.2 version rollback

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-08-14 17:31:09 +08:00
qiaozhanwei c6f0c8901c [maven-release-plugin] prepare for next development iteration 2020-08-14 16:04:49 +08:00
qiaozhanwei 8a6732ae4c [maven-release-plugin] prepare release 1.3.2 2020-08-14 16:04:38 +08:00
rockxsj 2b46db0bf7
[Feature]add presto datasource support (#3308)
* * add presto datasource support
update .gitigonre to igonre some files

* * use another presto driver

* * add LICENSE files about presto-jdbc

* * just for test sonar

Co-authored-by: rockxsj <junjun@mgtv.com>
2020-08-14 08:52:31 +08:00
zixi0825 46ab86d6df
batch copy or move process #2753 (#2884)
* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* Roll back the package.json file

* rerun test

* revert blank

* resolve conflicts

* resolve conflicts

* resolve ut error

* resolve sonar error

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-08-13 15:37:46 +08:00
felix.wang 5584f0cb4d
[Fix-3262][common] When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled (#3264)
* fix bug #3165   get resource.storage.type value   toUpperCase

* fix bug #3176  optimize Gets the value of this property “resource.storage.type”

* fix When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled

* fix When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled

* fix When you request the URL through applicationID to get the application status, you cannot get it if Kerberos authentication is enabled

* fix miss

* add KerberosHttpClient fix  kerberos bug

* fix map init

* add juint test

* Extraction of public methods

* Extraction of public methods

* Fix code quality

* Fix code quality

* Fix code quality

* Fix code quality

* Fix code quality

* Fix code quality

* Fix code quality

* rebuild test

* rebuild test

* check style

* check style

* check style

* revert UT POM

* Kerberos judgment goes up to hadoopUtils

* fix merge

* Remove connection pool shutdown

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-08-10 15:18:01 +08:00
Yichao Yang 3ed0afa6ac
[Improvement-3369][api] Introduce data analysis service interface for clear code (#3439)
* [Improvement][api] Introduce data analysis service interface for clear code

* Remove the bad smell

* Remove the bad smell
2020-08-10 14:47:45 +08:00
Zhou Zheng 90a060128f 修改与新增commandType相关的代码
1. dataAnalysis
2. alertManager
3. businessTimeUtils的schedule选择
2020-08-07 15:39:48 +08:00
Yelli e4097a488c
cancel spark task version check (#3406)
Co-authored-by: Eights-Li <yelli.hl@gmail.com>
2020-08-05 10:20:34 +08:00
dailidong d64127b90a optimize TheadUtils test 2020-08-04 17:57:23 +08:00
lgcareer a3793be7da
[Fix-3390][server]Running hive sql task need find the hdfs path correctly (#3396)
* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* [Fix-3390][api]Running hive sql task need find the hdfs path correctly

* update soft version
2020-08-04 15:04:56 +08:00
CalvinKirs 6ecc95a338
[FIX_BUG][*] fix some bugs from sonar analysis (#3322)
* [FIX_BUG][*] fix some bugs from sonar analysis

* remove invalid single test

* code smell

* remove exception catches that never happen

* delete invalid exception catch

* remove excess locks

* fix e2e test error

* recovery related exception removal
2020-08-04 02:23:50 +00:00
lgcareer 3744167d52
[Fix-3364]Running the spark task fail because the can't find the jar (#3378) 2020-08-03 10:31:27 +08:00
BoYiZhang d70d8a6054
[Feature-3222][datasource] Store password in ciphertext instead (#3330)
* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Store password in ciphertext instead

* Store password in ciphertext instead

* Store password in ciphertext instead

* Store password in ciphertext instead

* Store password in ciphertext instead

* update test unit

* add test unit

* add test unit

* update e2e test

* update unit test

* update unit test

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
2020-08-01 14:28:12 +08:00
xsbai 87eb7c8211
adds waterdrop plugin (#3073)
* add plugin waterdrop

* add plugin waterdrop

* add plugin waterdrop

* fix deploy mode error

* Update en_US.js

fix conflict

* Update en_US.js

fix conflict

* Update en_US.js

Add missing commas

* Update zh_CN.js

Add missing commas

* Update waterdrop.vue

Fix the no resource hint

* Update zh_CN.js

Add 'Please select the waterdrop resources'

* Update en_US.js

Add  'Please select the waterdrop resources'

* Update waterdrop.vue

fix bug when click the waterdrop node again, it still show the `client` mode in deploy mode

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-30 23:20:59 +08:00
RedemptionC 0c5754c3f2
add feature : query top n success process instance order by running duration (#3315)
* add feature : query top n success process instance order by running duration

* add feature : add params check

* add feature : remove code smell

* add feature : remove code smell

* add feature : add param check

* add feature : add param check
improve handleEscape method in ParameterUtils.java

* top n : add  java doc and format code
2020-07-30 18:38:59 +08:00
zixi0825 5fe31546ba
[Fix][all] fix spell error (#3289)
* fix spell error

* test unit test

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
2020-07-30 18:33:16 +08:00
felix.wang b555d0a086
[Improvement][common] HttpClient singleton mode #3332 (#3350)
* Optimize the httpclient

* Optimize the httpclient singleton

* Merge branch 'github-local/dev' into feature/httpClientsingleton

# Conflicts:
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/HttpUtils.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java

* Merge branch 'github-local/dev' into feature/httpClientsingleton

# Conflicts:
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/HttpUtils.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java

* Merge branch 'github-local/dev' into feature/httpClientsingleton

# Conflicts:
#	dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/HttpUtils.java
#	dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java
2020-07-30 13:24:54 +08:00
felix.wang e026d303ea
Optimize the httpclient (#3337) 2020-07-29 19:03:44 +08:00
Zhou Zheng 3fceee9570 从强制成功的节点处启动-apiserver部分
1. 新添加executeType
2. 新添加对应的commandType
3. 修改controller对应的service部分
4. 补充processService中的verifyIsNeedCreateCommand的判断逻辑
2020-07-23 22:50:32 +08:00
Zhou Zheng d86ceeecbe [implement] change task state 2020-07-23 21:54:48 +08:00
t1mon 07c8193bb5
[Feature-2930][api] Displays path and host on the instance log panel. #2930 (#3154)
* Optimize PropertyUtils instantiation.

* Fix info error.

* [Feature-2930][api] Displays path and host on the instance log panel. #2930

* [update] Add log-head in download log.

* [update] clear code smell.
2020-07-23 10:57:56 +08:00
lgcareer f854c5f794
[Feature-2815]cherry pick commit from dev for one worker can belong different worker groups (#3277)
* [Feature-2815]cherry pick commit from dev for one worker can belong to different worker groups

* fix 2975:download resource need find the tenant code of resource owner

* change NetUtils to OSUtils because NetUtils didn't exist in the 1.3.2-release branch

Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-07-22 19:39:03 +08:00
CalvinKirs 38e7494f78
replace CollectionUtils deprecated method (#3027)
* replace deprecated method

* replace deprecated method

* replace deprecated method

* replace deprecated method

* replace deprecated method

* messy

* messy

* messy

* add missing license

Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 19:42:33 +08:00
CalvinKirs 6f9970b189
[Feature-2574][Server]Specify Network Interface (#3186)
* [Feature-2574][Server]Specify Network Interface
This closes #2574

* code smell

* code smell

* move DOLPHIN_SCHEDULER_PREFERRED_NETWORK_INTERFACE to Constants.java
2020-07-13 18:51:38 +08:00
Yichao Yang b4af3fd176
[Feature-2815][server] One worker can belong to different workergroups (#2934)
* [Feature-2815][server] One worker can belong to different workergroups

* Feature: Add test cases

* Update MonitorService.java

* Update MonitorService.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 17:25:13 +08:00
bao liang 657ea4752f
merge from 1.3.1-release (#3161)
* merge from 1.3.1-release

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* remove some standard notices in order to cut down the NOTICE file. (#3101)

* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version

* merge from 1.3.1-release

* fix 3112:download udf resource need find the tenant code of resource owner (#3114)

* fix 3112:download udf resource need find the tenant code of resource owner

* fix 2975:download resource need find the tenant code of resource owner

* remove useless code

* merge from 1.3.1-release

* remove some notes (#3127)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)

* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method

* add PropertyPlaceholderHelper on license (#3129)

* update license header location

* add PropertyPlaceholderHelper on license

* merge from 1.3.1-release

* change from 1.3.0  to 1.3.2

* add Set import

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
2020-07-13 16:35:42 +08:00
muzhongjiang c49626b877
Word spelling modification,Comment modification,Log level modification (#3180)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

* Word spelling modification
Comment modification
Word spelling modification,Comment modification,Log level modification

* Update EmailManager.java

* Update FlinkParameters.java

* Update SqlTask.java

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-12 22:20:14 +08:00
rockxsj dcdd7dedd0
Modify the AWS S3 request encryption method to V4. (#3182)
Co-authored-by: rockxsj <junjun@mgtv.com>
2020-07-11 08:56:38 +08:00
felix-thinkingdata 1e7582e910
#3176 optimize Gets the value of this property “resource.storage.type”, Comparison with enumerated types (#3178)
* fix bug #3165   get resource.storage.type value   toUpperCase

* fix bug #3176  optimize Gets the value of this property “resource.storage.type”

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-10 15:21:42 +08:00
felix-thinkingdata a23a3e2d12
fix bug #3165 get resource.storage.type value toUpperCase (#3166)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-09 18:28:04 +08:00
liukunyuan 0692dca4e6
update CommonUtils.java (#3137)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-05 21:50:42 +08:00
t1mon 83589c7b7c
Optimize kerberos tgt expire time in HadoopUtils. (#2700)
* Optimize PropertyUtils instantiation.

* Fix info error.

* Optimize kerberos tgt expire time.

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-04 18:22:30 +08:00
lenboo 572b142a53 [maven-release-plugin] prepare for next development iteration 2020-07-04 16:10:37 +08:00
lenboo a4272efa55 [maven-release-plugin] prepare release 1.3.1 2020-07-04 16:04:36 +08:00
lenboo fd3485ad0a revert version to 1.3.1 2020-07-04 15:54:37 +08:00
dailidong 4a4e940379
fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)
* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method
2020-07-04 15:49:46 +08:00
lenboo 3d4d816349 [maven-release-plugin] prepare for next development iteration 2020-07-03 15:56:45 +08:00
lenboo 2ff94a4362 [maven-release-plugin] prepare release 1.3.1 2020-07-03 15:55:56 +08:00
CalvinKirs d1282fdd96
In the case of multiple network cards, a valid host is returned (#2924)
* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* fix LOCAL_ADDRESS not assigned  value

* fix LOCAL_ADDRESS not assigned  value

* fix LOCAL_ADDRESS not assigned  value

* fix lock error

* fix lock error

* end

* end

* end

* end

* code smell

* code smell

* code smell

* code smell

* code smell

* code smell

* code smell

* lowering

* messy

* small change

* compile error

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
2020-07-02 19:16:37 +08:00
muzhongjiang a9eb7b18b0
[bugfix] fixed "jackson enum conversion : InvalidFormatException" (#3078)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-02 18:07:57 +08:00
dailidong 7445fa946b fix singe resource manager will NPE (#3013)
* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* update h2 database

* fix #2910 master server will show exception for some time when it restart

* fix oom when no master is active

* fix worker oom when master server restart

* fix oom

* fix

* add UT

* fix worker group config no effect

* fix singe resource manager will NPE

* Update HadoopUtilsTest.java
2020-06-30 14:11:07 +08:00
lenboo fd76f172c1 Merge remote-tracking branch 'upstream/dev' into dev-merge
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
2020-06-30 11:17:14 +08:00
gaojun2048 58897aa2c5 merge 1.3.0-release branch code 2020-06-29 18:42:26 +08:00
gaojun2048 3cf9afc195 merge 1.3.0-release branch code 2020-06-29 18:42:18 +08:00
bao liang 31b593d023 change max length of user name to 39 (#2856)
* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

* change max length of username to 39

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-06-29 17:39:41 +08:00
lgcareer a9cbca1e55 add source license full text into source license (#2840)
* add source license full text into source license

* exclude ScriptRunner.java on rat check
2020-06-29 17:39:38 +08:00
qiaozhanwei 5c9123f9ae blank in eamil and left font align #2648 (#2673)
* script variable has "processDefinitionId" is error #2664

* blank in eamil and left font align  #2648

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-06-29 17:39:17 +08:00
dailidong 88bdb20a91 Update HttpUtils.java 2020-06-29 17:38:48 +08:00
qiaozhanwei a085aa8fa3 modify UT 2020-06-29 17:38:23 +08:00
qiaozhanwei cc3ab0f191 timing online modify (#2296)
* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* 1, master persistent task
2. extract  master and worker communication model

* add license

* modify javadoc error

* TaskExecutionContext create modify

* buildAckCommand taskInstanceId not set modify

* java doc error modify

* add comment

* ExecutorManager interface add generic type

* add TaskInstanceCacheManager receive Worker report result

* TaskInstance setExecutePath

* add TaskInstanceCacheManager to receive Worker Task result report

* TaskInstanceCacheManager add remove method

* add license

* add dispatcht task method

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* AbstractCommandExecutor remove db access

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* taskInstanceCache is null ,need load from db

* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify

* worker remove db

* ShellTask modify

* master persistence processId and appIds

* master persistence processId and appIds

* master add kill task logic

* master add kill task logic

* master add kill task logic

* javadoc error modify

* remove chinese log

* executeDirectly method add Override

* remote module modify

* TaskKillResponseProcessor command type modify

* create buildKillCommand

* host add host:port format

* host add host:port format

* TaskAckProcessor modify

* TaskAckProcessor modify

* task prioriry refator

* remove ITaskQueue

* task prioriry refator

* remove ITaskQueue

* TaskPriority refactor

* remove logs

* WorkerServer refactor

* MasterSchedulerService modify

* WorkerConfig listen port modify

* modify master and worker listen port

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* cancelTaskInstance set TaskExecutionContext host,logPath,executePath

* Encapsulate the parameters required by sqltask

* 1,Encapsulate the parameters required by sqltask
2,SQLTask optimization

* AbstractTask modify

* ProcedureTask optimization

* MasterSchedulerService modify

* TaskUpdateQueueConsumer modify

* test

* DataxTask process run debug

* DataxTask process run debug

* add protobuf dependency,MR、Spark task etc need this

* TaskUpdateQueueConsumer modify

* TaskExecutionContextBuilder set TaskInstance workgroup

* WorkerGroupService queryAllGroup modify
query available work group

* 1,get workergroup from zk modify
2,SpringConnectionFactory repeat load modify

* master and worker register ip  use OSUtils.getHost()

* ProcessInstance host set ip:port format

* worker fault tolerance modify

* Constants and .env modify

* master fault tolerant bug modify

* UT add pom.xml

* timing online  modify

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-06-29 17:37:58 +08:00
songgg d8c7a44e82 some annotation optimizations (#2333)
Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:38 +08:00
lgcareer 82d6dc9e52 new feature for #404 add resource tree function (#2323)
* add create resource directory

* add create resource directory

* update the resource test

* add upgrade sql in version 1.2.2

* Adding request parameter id to update queryResourceListPaging

* set isDirectory value is false default

* add full name to update updateResource

* remove request parameter isDirectory to update createResource method

* update queryResourceListPaging with change get to post

* update updateResource method with remove fullName

* File management list modification (#1976)

* add resource component

* add resource tree visitor

* return json string

* update queryResourceList

* upload file need fullName

* add method rootNode

* Shell task resources and authorization resources (#1989)

* File management list modification

* Shell task resources and authorization resources

* download resource when execute task

* download resource when execute task

* update authorization type

* download resource when execute task

* Spark task resource changes (#1990)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* download resource when execute task

* update udf function service

* add resource type in ResourceComponent

* UDF resource tree and change DAG style (#2019)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* add deleteIds method in ResourceMapper and ResourceMapperTest

* Add comments on class and method

* add queryResourceByName method in controller

* update verify-name with change name to full name

* update queryResource with add parameter pid

* update queryResource with add parameter pid

* add resource ids in process definition and delete resource need judge whether it is used by any process definition

* Breadcrumb development (#2033)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix (#2040)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Fix github action rerun failed (#2067)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Change crumb position

* Change crumb position (#2068)

* build resource process definition map

* UDF changed to multiple choice

* UDF changed to multiple choice (#2077)

* Change crumb position

* UDF changed to multiple choice

* build resource process definition map (#2076)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* update resource name also need update all the children full name (#2096)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Limit customization file content to no more than 3000 lines(#2128) (#2140)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList (#2192)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* Modify the main jar package

* Modify the main jar package (#2200)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Modify the main jar package

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* choose main jar with resource tree (#2220)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* getResouDelete checksum and modify parameter namerceId

* revert .env

* remove parameter pid

* Delete request interface

* go back to the last page

* jar interface call

* Fix issue #2234 and #2228

* change resource name with full name

* Fix issue #2234 and #2228 (#2246)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* revert .env

* remove parameter pid

* Fix issue #2234 and #2228

* change resource name with full name

* Fix list query value error

* remove unauth-file with authorize-resource-tree

* Repair data cannot be echoed

* Repair data cannot be echoed

* execute mr and spark task need query resource name before

* Authorized resource interface replacement

* Authorized resource interface replacement

* Filter UDF resources

* Change parameters

* need query all authorized directory children when create task

* Change normalize.scss import method and animation.scss license modification

* Delete file list update processing

* It's fixed that resource not deleted in hdfs when delete it.

* add tooltips

* add tooltips (#2310)

* Echo workflow name and modify udf management name

* [new feature]add resource tree function

* revert front code in order to be same as dev branch

* revert front code in order to be same as dev branch

* revert common.properties and application.properties

* add super method

* update flink parameter test

* update flink parameter and unit test

* update resource service test

* If resource list is empty,need init it

* update flink parameter test

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-29 17:36:36 +08:00
gabry.wu f9c91b8cbe Adapting partial code(file name start with S #2) to the sonar cloud rule (#2270) 2020-06-29 17:36:36 +08:00
gabry.wu 30b98b89a2 Adapting partial code(file name start with T) to the sonar cloud rule (#2271) 2020-06-29 17:36:33 +08:00
songquanhe-gitstudy fc18d4a216 fix: No operations allowed after statement closed when running sql task (#2265)
* sqlTask failed to run

* get correct attribute value and logger content

Co-authored-by: songqh <songquanhe@foxmail.com>
2020-06-29 17:36:30 +08:00
gabry.wu e1771de45a Adapting partial code(file name start with R) to the sonar cloud rule (#2243) 2020-06-29 17:36:29 +08:00
Simon 5c30205574 support custom datax configuration (#2224)
* fix #1441

* support custom datax config

* support datax custom config

* support datax custom config

* support datax custom config

Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
2020-06-29 17:36:29 +08:00
ruson 2bb55e4993 add repace time when run history job and batch complement job ,not current time (#2196)
* add release notes in 1.2.1 (#1654)

* fix Monitor bug (#1656)

* fix Monitor bug

* fix zk monitor bug

* fix api url (#1673)

* get root path from zookeeper config

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679 (#1685)

* fix api url

* Fixed DAG zoom in and zoom out nodes separated from arrows #1679

* check license and update (#1725)

* Fixed tasks_queue and tasks_kill did not exist in zookeeper #1696 (#1734)

Co-authored-by: elonlo <gamelifedong@gmail.com>

* upgrade jackson from 2.9.8 to 2.10.1 (#1767)

* add out put log when master/worker server start (#1769)

* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test

* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh

* add out put log when master/worker server start...

* add start log when servers start

* add check download resource permission in order to fix issues 1770 (#1788)

* merge dev branch front-end code (#1786)

* fix #1775 bug,delete process definition when process instance is running (#1790)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* fix udfs assignment and task instance page jump #1789 (#1791)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* update method checkDownloadPermission in order to fix issues 1770 (#1794)

* add check download resource permission in order to fix issues 1770

* update method checkDownloadPermission in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770

* update method listUnauthorizedResource in order to fix issues 1770 (#1797)

* if login user is admin,it will has permission to query all udf function (#1799)

* Fixed space and icon display issues before and after the input box (#1798)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* donot submit task to queue if sub process (#1793)

* dockerfile modify (#1800)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* Fix data echo, style and popup cannot be closed (#1801)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value(#1810) (#1815)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* [Fix issue #1770]check udf and data source in order to fix issue 1770 (#1817)

* if login user is admin,it will has permission to query all udf function

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* check udf and data source in order to fix issue 1770

* revert common.properties

* update the test method name

* remove useless code and import in unit test

* refactor has permission and check permission

* #1813 remove "_001" after the master/server register path in zookeepe (#1824)

* donot submit task to queue if sub process

* [feature] #1813 remove "_001" after the master/server register path in zookeeper (#1820)

* change master/worker register path.

* remove "_" from register path.

* remove install.sh server.servlet.context-path replace (#1823)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface (#1827)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* rpm package add UI (#1846)

* [fix #1828] when the executor of process instance is not the owner of udf resouce, the path of the read resource file is incorrect (#1847)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* add LoggerServer RPC PORT modified. #1848 (#1849)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848 (#1857)

* fix #1775 bug,delete process definition when process instance is running

* revert CONTRIBUTING.md

* dockerfile modify

* dockerfile modify

* dockerfile modify

* remove install.sh server.servlet.context-path replace

* add LoggerServer RPC PORT modified. #1848

* LoggerService Logger RPC PORT get Error modify . #1848

* Fix UDF function list delete data without refresh and page data display widened #1851 (#1852)

* merge dev branch front-end code

* fix udfs assignment

* Fix task instance page jump

* fix udfs assignment and task instance page jump #1789

* Fixed space and icon display issues before and after the input box

* add license

* add license

* Fix data echo, style and popup cannot be closed

* Fix page number loading issue and dag not getting value

* Fix DAG add dependent project value exception # 1816 and keep requesting task interface

* Fix UDF function list delete data without refresh and page data display widened

* [Fix #1828]check whether has permission to download udf file or delete udf function (#1858)

* fix issue 1828:get the udf resource path error when create udf function

* update grantResources

* first verify whether udf resource is bound by udf function

* update grantResources

* update testListAuthorizedUdfFunc

* update getUserInfo in order to run success

* check whether has permission to download udf file or delete udf file

* update listAuthorizedResourceById in ResourceMapper.xml

* add log, don't swallow exception info (#1877)

* Added hints in Resource Center (#1891)

* update jackson version from 2.9.8 to 2.10.1 (#1892)

* update release notes and modify the plugin of rat (#1899)

* modify how to build (#1902)

* update release notes and modify the plugin of rat

* modify how to build

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* get authorized udf resources need exclude all created by self (#1905)

* get authorized udf resources need exclude all created by self

* reset pom version to 1.2.1-SNAPSHOT

* Update ReleaseNotes.md

add detail info

* Update ReleaseNotes.md

* update notice year form 2019 to 2019-2020 (#1907)

* [maven-release-plugin] prepare release 1.2.1

* [maven-release-plugin] prepare for next development iteration

* 当手动重跑历史活着补数的时候对shell和sql脚本中的[YYYYmmddd...]变量赋值与传递的日期,而不是当前日期

Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: elonlo <gamelifedong@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: zhangchunyang <18910529250@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: 谷雨 <lixun@guyudeMacBook-Pro.local>
2020-06-29 17:36:28 +08:00
t1mon 9c754378f3 Optimize PropertyUtils instantiation. (#2199)
* Optimize PropertyUtils instantiation.

* Fix info error.
2020-06-29 17:36:28 +08:00
tswstarplanet 585f833549 change static field to instance field (#2190) 2020-06-29 17:36:28 +08:00
tswstarplanet 77c4bcb96e Cache HadoopUtils instance with specific days expire time (#2181)
* Cache HadoopUtils instance with 7 days expire time

* solve sonar issue

* add kerberos expire time config

* move KERBEROS_EXPIRE_TIME to Constants.java
2020-06-29 17:36:28 +08:00
gabry.wu 2f4fda7cdb Adapting partial code(file name start with M #1) to the sonar cloud rule (#2173)
* Adapting partial code(file name start with M #1) to the sonar cloud rule

* remove unused import
2020-06-29 17:36:28 +08:00
gabry.wu 7f7f50a828 Adapting partial code(file name start with L) to the sonar cloud rule (#2171) 2020-06-29 17:36:28 +08:00
gabry.wu bdfb9d6a50 Adapting partial code(file name start with J) to the sonar cloud rule (#2165)
* Adapting partial code(file name start with J) to the sonar cloud rule

* fix unit failure
2020-06-29 17:36:27 +08:00
Rubik-W 8b61afc9be The master and worker server exit exception #2163 (#2176)
* fix: #2163

* fix: format
2020-06-29 17:36:27 +08:00
gabry.wu 610c95a470 use static method in static class JSON (#2156) 2020-06-29 17:36:27 +08:00
gabry.wu 33e86a97bf Adapting partial code(file name start with I) to the sonar cloud rule (#2155) 2020-06-29 17:36:27 +08:00
gabry.wu 156533ad5e Adapting partial code(file name start with H) to the sonar cloud rule (#2145) 2020-06-29 17:36:22 +08:00