Commit Graph

337 Commits

Author SHA1 Message Date
Kirs 14785f4e39
[Feature-#3828][server] Delete recipients and CCs in the process defi… (#4303)
* [Feature-#3828][server] Delete recipients and CCs in the process definition and select the associated alert group instead

* The specific alarm instance associated with the alert group
* The process definition only associates the alert group, and does not care about the specific alarm instance.

this closes #3828

* remove test cc

* remove test cc

* fix test error

* add sql upgrade script
delete alert column(show_type、alert_type、re、cc)

* code style

* update test
Note the statement to delete the column, and provide related instructions

* delete AlertGroup field(alert_type)

* fix ut error

* fix ut error

* fix ut error

* code style

* delete api-doc receiverCC

* delete UserAlertGroupMapper

* delete UserAlertGroup

* delete UserAlertGroupApi

* alert group add coulmn user_id

* alter user_id to create_user_id

* userinfo remove alert group info
2020-12-31 10:18:53 +08:00
Kirs 17d44216a4
[Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages (#4311)
* [Feature#4310][Alert-SPI] Plug-ins containing UI components provide display pages

*Some plugins (such as alert plugin) need to provide UI interfaces to users.
*We use from-creat to dynamically generate UI interfaces. Related parameters are mainly provided by pluginParams.
*From-create can generate dynamic ui based on this parameter.

this closes #4310

* add license head

* rename

* add ut

* add license

* add query plugin detail interface

* fix error
2020-12-28 11:25:00 +08:00
Yichao Yang cb25810668
[Feature-3311] Add alert plugin instance function (#4038) 2020-12-23 09:35:10 +08:00
hailin0 c4be3b5749
[Fix][API] Log dependency package conflict (#3833)
Co-authored-by: lisiying <lisiying@192.168.0.102>
2020-10-10 11:14:32 +08:00
AhahaGe 1a257b1077
[Feature][api]Export and import workflow add processDefinitionDescription #3808 (#3847)
* add .asf.yaml file

* export processDefinition add desc

* clear conentent in .asf.yaml

* delete .asf.yaml file

Co-authored-by: guirong.ggr <guirong.ggr@dtwave-inc.com>
2020-10-09 21:00:02 +08:00
Yarlung 1c96ae0944
[Fix-3840][server] Fix When the tenant code is a number, it should not be saved successfully (#3867)
* bug fixed #3840

* fix bug #3840

* bug fixed #3840

* bug fixed issue #3840

* bug fix #3840

* bug fix #3540

Co-authored-by: gechunfa <gechunfa@gechunfas-MacBook-Pro.local>
2020-10-08 17:08:26 +08:00
孙继峰 eb597e67e3
[Improvement][API] simplify with stream (#3764)
* simplify with stream

* add distinct

* compatible tasks is null

* add unit test
2020-09-26 15:45:24 +08:00
BoYiZhang 375ec125c5
[Feature-3773][processInstance, taskInstance] Change title name : Replacement System.currentTimeMillis () (#3774)
* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dolphinscheduler-daemon.sh

* Replacement System.currentTimeMillis () to DateUtils.getCurrentTimeStamp()

* update checkstyle

* update checkstyle

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
2020-09-22 17:38:17 +08:00
muzhongjiang c42e769ced
[Fix] fix bug #3487 Create folder duplicate name under multithreading (#3697)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

* Word spelling modification
Comment modification
Word spelling modification,Comment modification,Log level modification

* Update EmailManager.java

* Update FlinkParameters.java

* Update SqlTask.java

* fixed  "getNotifyGroupList cache"  bug

* fix bug "Creating folders with multiple threads will result in multiple identical folders #3487"

* fix "Creating folders with multiple threads will result in multiple identical folders" #3487

* fix "Creating folders with multiple threads will result in multiple identical folders" #3487

* fix bug  #3487   Create folder duplicate name under multithreading

* fix bug  #3487   Create folder duplicate name under multithreading

* fix bug  #3487   Create folder duplicate name under multithreading

* fix bug  #3487   Create folder duplicate name under multithreading

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-09-09 14:13:29 +08:00
zhuangchong 21a90f2161
[Feature][API] process definition delete api add check process instance is already running. #3581 (#3582)
* process definition delete api add check process instance is already running.

* add checkstyle.

* processinstancemapper replaces processinstanceservice.

* processinstancemapper replaces processinstanceservice.

Co-authored-by: unknown <zhuang.chong@zhaopin.com.cn>
2020-09-07 11:03:51 +08:00
赤月 69201ac346
[Improvement][ALL] Remove some un-used code and code review #3639 (#3640)
* [Improvement][ALL] Remove some un-used code and code review #3639

* [Improvement][ALL] Remove some un-used code and code review #3639

* [Improvement][ALL] Remove some un-used code and code review #3639

* [Improvement][ALL] Remove some un-used code and code review

* [Improvement][ALL] Remove some un-used code and code review

* [Improvement][ALL] Remove some un-used code and code review
2020-09-03 18:56:32 +08:00
sky 879feb79b3
[Feature-3567] batch activate user (#3578)
* feature batch activate user

* fix sonalcloud problem and code style problem

* fix code style problem

* fix sonar cloud problem

* fix sonar cloud problem

* fix sonar cloud problem

Co-authored-by: dev_sky <dev_sky@740051880@qq.com>
2020-08-25 15:47:21 +08:00
geosmart f7c4b0f351
[Improvement-3556] remove unnecessary try catch, and do the unittest … (#3563)
* [Improvement-3556] remove unnecessary try catch, and do the unittest to full coverage.

* [Improvement-3556] add REQUEST_PARAMS_NOT_VALID_ERROR res when param is not valid

* [fix-3548] Monitor gauge chart display blank with same pid

Co-authored-by: wanggang <wanggy01@servyou.com.cn>
2020-08-25 10:51:02 +08:00
Yichao Yang 93660f4d61
[Feature-2154][api] Workflow version control (#3053)
* [Feature][api] Workflow version control

* Update messages.properties

* Update messages_en_US.properties

* Update messages_zh_CN.properties
2020-08-25 10:46:58 +08:00
Yichao Yang c003cd133d
[Improvement-3369][api] Introduce tenant service interface for clear code (#3420)
* [Improvement][api] Introduce tenant service interface for clear code

* Checkstyle

* Checkstyle
2020-08-25 09:56:41 +08:00
easycode 78eb07b361
[Improvement-3558][API] Improve `DataSourceServiceTest.java` (#3579)
* Fixed this bug "multiple threads creating folders will result in multiple identical folders" #3487

* 1. Improvement `DataSourceServiceTest` #3558
2. Get rid of some useless arguments to functions in the `DataSourceService` class

* The rollback `ResourceService.java`

* The rollback `ResourceService.java`

* The rollback `ResourceService.java`

* 1.Modify code styles.
2.Make improvement according to Yichao Yang's advice. Thank you very much for guidance.@Yichao Yang

* Modify code styles.

* Circumventing the checkConnection problem

* Modify code styles.
2020-08-24 16:36:02 +08:00
Yichao Yang 836e86dca5
[Test][api] Add api module `service` layer test example (#3521)
* [Test][api] Add api service layer test example

* Remove unused exception

* Remove unused exception

* Format code according to the checkstyle

* Fix test case
2020-08-19 16:32:47 +08:00
vanilla111 a4ee351a3a
delay execution of tasks and improve some designs (#3427) 2020-08-19 11:36:30 +08:00
Yichao Yang fe9967b005
[Improvement-3369][api] Introduce ProjectService interface for more clear code (#3384)
* [Improvement][api] Introduce ProjectService interface for more clear code

* Fix ci e2e

* Update ProcessDefinitionServiceTest.java
2020-08-16 12:36:25 +08:00
rockxsj 2b46db0bf7
[Feature]add presto datasource support (#3308)
* * add presto datasource support
update .gitigonre to igonre some files

* * use another presto driver

* * add LICENSE files about presto-jdbc

* * just for test sonar

Co-authored-by: rockxsj <junjun@mgtv.com>
2020-08-14 08:52:31 +08:00
geosmart 6da7f1a2d1
resolve curator-client dependency confliction (#3488)
* dolphinscheduler-api.pom resolve curator-client conflict

the ds use curator-client:4.3.0 dependency by curator-frameworke,
but hadoop-common use  curator-client:2.7.1,
so make the ClassNotFoundException
```
Caused by: java.lang.ClassNotFoundException: org.apache.curator.connection.StandardConnectionHandlingPolicy
```
exclue curator-client in hadoop-common to resolve this

* resolve curator-client dependency confliction


## What is the purpose of the pull request

the ds use curator-client:4.3.0 dependency by curator-frameworke,
but hadoop-common use  curator-client:2.7.1,
so make the ClassNotFoundException
```
Caused by: java.lang.ClassNotFoundException: org.apache.curator.connection.StandardConnectionHandlingPolicy
```
exclue curator-client in hadoop-common to resolve this
2020-08-14 00:34:30 +08:00
zixi0825 46ab86d6df
batch copy or move process #2753 (#2884)
* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process sovle code smell

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* batch copy or move process

* Roll back the package.json file

* rerun test

* revert blank

* resolve conflicts

* resolve conflicts

* resolve ut error

* resolve sonar error

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-08-13 15:37:46 +08:00
Yichao Yang 133b4de129
[Improvement-3369][api] Introduce api service interface for ProcessDefinitionService (#3371)
* [Improvement-3369][api] Code clean and improvement

* Feature: import DATA_LIST static variable
2020-08-10 15:02:55 +08:00
Yichao Yang 379203b306
[Improvement-3369][api] Introduce session service interface for clear code (#3408)
* [Improvement][api] Introduce session service interface for clear code

* Update SessionService.java

* [Improvement][api] Add @Service

* Add licenses
2020-08-10 15:02:14 +08:00
Yichao Yang 3ed0afa6ac
[Improvement-3369][api] Introduce data analysis service interface for clear code (#3439)
* [Improvement][api] Introduce data analysis service interface for clear code

* Remove the bad smell

* Remove the bad smell
2020-08-10 14:47:45 +08:00
Yichao Yang dce403d062
[Improvement-3369][api] Introduce logger service interface for clear code (#3437)
* [Improvement][api] Introduce logger service interface for clear code

* Remove the code smell
2020-08-10 14:47:16 +08:00
Yichao Yang f7f07608ed
[Improvement][api] Introduce access token service interface for clear code (#3438) 2020-08-10 14:46:59 +08:00
Yichao Yang 4fb06a736d
[Test][api] Introduce api controller unit test example (#3447)
* [Test][api] Introduce api controller test example

* Add controller test
2020-08-10 14:36:31 +08:00
sky 6dc500915c
[Feature-3392][api-server] (#3403)
* feature user register

fix bug

fix security problem

fix security problem

* activate user

* fix confilct

* fix confilct and fix some bug

* fix cr problem

Co-authored-by: dev_sky <dev_sky@740051880@qq.com>
2020-08-05 17:27:55 +08:00
CalvinKirs 6ecc95a338
[FIX_BUG][*] fix some bugs from sonar analysis (#3322)
* [FIX_BUG][*] fix some bugs from sonar analysis

* remove invalid single test

* code smell

* remove exception catches that never happen

* delete invalid exception catch

* remove excess locks

* fix e2e test error

* recovery related exception removal
2020-08-04 02:23:50 +00:00
BoYiZhang d70d8a6054
[Feature-3222][datasource] Store password in ciphertext instead (#3330)
* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* dag add close buttion

dag add close buttion

* update  CLICK_SAVE_WORKFLOW_BUTTON  xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* updae CLICK_SAVE_WORKFLOW_BUTTON xpath

* Update CreateWorkflowLocator.java

modify submit workflow button

* Update CreateWorkflowLocator.java

* Update CreateWorkflowLocator.java

modify CLICK_ADD_BUTTON

* Update CreateWorkflowLocator.java

delete print

* Update CreateWorkflowLocator.java

1

* Update CreateWorkflowLocator.java

1

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Update dag.vue

* Update dag.vue

* Update dag.vue

* Update CreateWorkflowLocator.java

* Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

This reverts commit 3a2cba7a

* Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

in addition

update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

* Store password in ciphertext instead

* Store password in ciphertext instead

* Store password in ciphertext instead

* Store password in ciphertext instead

* Store password in ciphertext instead

* update test unit

* add test unit

* add test unit

* update e2e test

* update unit test

* update unit test

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
2020-08-01 14:28:12 +08:00
RedemptionC 0c5754c3f2
add feature : query top n success process instance order by running duration (#3315)
* add feature : query top n success process instance order by running duration

* add feature : add params check

* add feature : remove code smell

* add feature : remove code smell

* add feature : add param check

* add feature : add param check
improve handleEscape method in ParameterUtils.java

* top n : add  java doc and format code
2020-07-30 18:38:59 +08:00
zixi0825 5fe31546ba
[Fix][all] fix spell error (#3289)
* fix spell error

* test unit test

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
2020-07-30 18:33:16 +08:00
Yichao Yang d006bd3ba5
[Improvement][api] Refactor the transaction rollbackfor runtimeexception (#3313)
* [Improvement][api] Refactor the transaction rollbackfor runtimeexception

* Remove the unused import
2020-07-30 15:12:14 +08:00
LiemLin f27c62a5db
workflow lineage (#2421)
* workflow lineage

* workflow lineage unit test

* stash

* add apache license

* Update .env

* code optimize

* add unit test

* add apache license

* add apache license

* optimized code

* add postgresql support and optimized code

* Update pom.xml

* Update en_US.js

* Update zh_CN.js

* remove mock code

* inport i18n

* Update pom.xml

* Delete .env

* add env file

Co-authored-by: linhaiqiang <haiqiang.lin@five-star.cn>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: becarchal <yangbin_57@sina.com>
2020-07-30 13:32:29 +08:00
谌启成 e2a117032d
[Fix-3295][api] Add default value for request RequestParam (#3341) 2020-07-29 19:01:53 +08:00
tswstarplanet 5b7efd2d33
[Feature-3134]Refactor to use a shared singleton Curator Zookeeper Client (#3244)
* [Feature-3134]Refactor to use a shared singleton Curator Zookeeper Client

* autowire CuratorZookeeperClient to ZookeeperOperator

* [Improvement] Add the issue specifications reference (#3221)

* log zookeeper address when the connect state change

* resume the operation of add connect state listener in MasterRegistry

Co-authored-by: Yichao Yang <1048262223@qq.com>
2020-07-23 11:02:30 +08:00
t1mon 07c8193bb5
[Feature-2930][api] Displays path and host on the instance log panel. #2930 (#3154)
* Optimize PropertyUtils instantiation.

* Fix info error.

* [Feature-2930][api] Displays path and host on the instance log panel. #2930

* [update] Add log-head in download log.

* [update] clear code smell.
2020-07-23 10:57:56 +08:00
JinyLeeChina 0f6f138dc8
[BugFixed] issue #3258 (#3265)
* 'ExecutionStatus'

* '3258'

* Update WorkerGroupServiceTest.java

* Delete UserState.java

* Delete ResourceSyncService.java

* Delete core-site.xml

* Delete hdfs-site.xml

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-22 11:50:19 +08:00
Yichao Yang b4af3fd176
[Feature-2815][server] One worker can belong to different workergroups (#2934)
* [Feature-2815][server] One worker can belong to different workergroups

* Feature: Add test cases

* Update MonitorService.java

* Update MonitorService.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 17:25:13 +08:00
bao liang 657ea4752f
merge from 1.3.1-release (#3161)
* merge from 1.3.1-release

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* remove some standard notices in order to cut down the NOTICE file. (#3101)

* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version

* merge from 1.3.1-release

* fix 3112:download udf resource need find the tenant code of resource owner (#3114)

* fix 3112:download udf resource need find the tenant code of resource owner

* fix 2975:download resource need find the tenant code of resource owner

* remove useless code

* merge from 1.3.1-release

* remove some notes (#3127)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)

* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method

* add PropertyPlaceholderHelper on license (#3129)

* update license header location

* add PropertyPlaceholderHelper on license

* merge from 1.3.1-release

* change from 1.3.0  to 1.3.2

* add Set import

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
2020-07-13 16:35:42 +08:00
Yichao Yang 5868af8947
[Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)
* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-13 08:48:59 +08:00
samz406 6fa3701396
optimize SchedulerService.setScheduleState code (#3136)
* Optimize SchedulerService.setScheduleState code

* modify the test case to PowerMock

* modify code smell

* modify code smell

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-12 09:48:44 +08:00
sky d4d6aded11
feature user register (#3159)
fix bug

fix security problem

fix security problem

Co-authored-by: dev_sky <dev_sky@740051880@qq.com>
2020-07-07 16:34:50 +08:00
lenboo fd76f172c1 Merge remote-tracking branch 'upstream/dev' into dev-merge
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
2020-06-30 11:17:14 +08:00
zixi0825 9bf67d80d0
Revise annotation spelling errors & Enhanced code robustness (#3042)
* revise annotation spelling errors & enhanced code robustness

* revise annotation spelling errors & enhanced code robustness

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-28 10:48:14 +08:00
lenboo ea1c653e08 merge from dev 2020-06-22 10:21:15 +08:00
xsbai f7533e0bd0 modifiy the error comment 2020-06-22 09:15:13 +08:00
xsbai 0654757f15 modifiy the error comment 2020-06-22 09:10:24 +08:00
BoYiZhang 45785dbaef
Set default time zone (#3006)
Date type date problem in spring serializing Java
Solution: Spring serializes Java's date type date problem

Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-20 22:57:46 +08:00