Commit Graph

3096 Commits

Author SHA1 Message Date
bao liang 657ea4752f
merge from 1.3.1-release (#3161)
* merge from 1.3.1-release

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* remove some standard notices in order to cut down the NOTICE file. (#3101)

* #3084 remove some standard notices in order to cut down the NOTICE file.

* update exclude element sql/soft_version of rat plugin

* update soft_version

* merge from 1.3.1-release

* fix 3112:download udf resource need find the tenant code of resource owner (#3114)

* fix 3112:download udf resource need find the tenant code of resource owner

* fix 2975:download resource need find the tenant code of resource owner

* remove useless code

* merge from 1.3.1-release

* remove some notes (#3127)

Co-authored-by: lenboo <baoliang@analysys.com.cn>

* fix Permission denied if not found dolphinscheder-env.sh on 1.3.1 (#3128)

* fix Permission denied if not found dolphinscheder-env.sh

* update some comments on method

* add PropertyPlaceholderHelper on license (#3129)

* update license header location

* add PropertyPlaceholderHelper on license

* merge from 1.3.1-release

* change from 1.3.0  to 1.3.2

* add Set import

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: lenboo <baoliang@analysys.com.cn>
2020-07-13 16:35:42 +08:00
zhuangchong de41ac5083
timeoutAlarm model interval bug,You can now set negative integers (#3171)
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-13 16:22:20 +08:00
gabry.wu 75259dcc07
simply doExecute using RetryerUtils.retryCall (#2761) 2020-07-13 14:38:40 +08:00
tswstarplanet 88f9bed726
[Bug-3187]close Heartbeat thread pool when MasterRegistry unRegistry (#3188)
* [Bug-3187]close Heartbeat thread pool when MasterRegistry unRegistry

* use shutdownNow; delete redundant code
2020-07-13 14:27:38 +08:00
LEI SHENG 98fdba6740
[Bug 2923] Hive JDBC connection parameter ignored (#3194)
* fix the bug #2923 (Hive JDBC connection parameter ignored) , the hive jdbc connection url ;

* fix the bug index out of bound and add some test case

* add the Licensed

* fix the checkstyle and import StringBuilder

* put HiveDataSourceTest.java to root pom maven-surefire-plugin to avoiding the sonar 0.0% Coverage.

* Update HiveDataSource.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 13:58:01 +08:00
break60 6d43c21d80
[bug fix]Click the cancel button to prevent the input box from losing focus #3116 (#3115)
* [bugfix]Replace favicon icon with png

* Cancel the initial value of the node cache

* [bug fix]Click the cancel button to prevent the input box from losing focus

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-13 08:51:07 +08:00
Yichao Yang 5868af8947
[Hotfix-3131][api] Fix the new tenant already exists prompt (#3132)
* Bugfix: Fix the new tenant already exists prompt

* Feature: Add test cases

* Update TenantServiceTest.java

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-13 08:48:59 +08:00
CalvinKirs 933a67db8d
[Improvement][Server]Code Clean (#3196)
* [*][Server]Code Clean
redundant code removal

* redundant code removal

* redundant code removal
2020-07-12 22:25:25 +08:00
muzhongjiang c49626b877
Word spelling modification,Comment modification,Log level modification (#3180)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

* Word spelling modification
Comment modification
Word spelling modification,Comment modification,Log level modification

* Update EmailManager.java

* Update FlinkParameters.java

* Update SqlTask.java

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-12 22:20:14 +08:00
samz406 6fa3701396
optimize SchedulerService.setScheduleState code (#3136)
* Optimize SchedulerService.setScheduleState code

* modify the test case to PowerMock

* modify code smell

* modify code smell

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-12 09:48:44 +08:00
Yichao Yang aea702e3d4
[Feature][docs] Add the issue template according to the issue specification (#3185)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-12 08:42:55 +08:00
rockxsj dcdd7dedd0
Modify the AWS S3 request encryption method to V4. (#3182)
Co-authored-by: rockxsj <junjun@mgtv.com>
2020-07-11 08:56:38 +08:00
felix-thinkingdata 1e7582e910
#3176 optimize Gets the value of this property “resource.storage.type”, Comparison with enumerated types (#3178)
* fix bug #3165   get resource.storage.type value   toUpperCase

* fix bug #3176  optimize Gets the value of this property “resource.storage.type”

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-10 15:21:42 +08:00
qiaozhanwei 1eb8fb6db3
NettyDecoder class decode method switch case remove break (#3035) 2020-07-10 11:10:13 +08:00
tswstarplanet d87d2d80dd
add maxWaitTime config when build zookeeper client (#3133)
* add maxWaitTime config when build zookeeper client

* throw exception when connect zk timeout

* use dedicated exception instead of a generic one

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-07-09 20:27:41 +08:00
felix-thinkingdata a23a3e2d12
fix bug #3165 get resource.storage.type value toUpperCase (#3166)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-09 18:28:04 +08:00
BoYiZhang ae902e2308
fix flink web ui bug (#3079)
* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* fix bug

Delete invalid field: executorcores

Modify verification prompt

* dag  add close button

* reset last version

* reset last version

* Update dag.vue

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-09 17:59:20 +08:00
tswstarplanet bf3cc0d00e
[Bug][Refactor][issue-3157]use cas to avoid thread safe problem (#3158)
Co-authored-by: lgcareer <18610854716@163.com>
2020-07-07 18:45:20 +08:00
sky d4d6aded11
feature user register (#3159)
fix bug

fix security problem

fix security problem

Co-authored-by: dev_sky <dev_sky@740051880@qq.com>
2020-07-07 16:34:50 +08:00
iceblue510 1f8851b906
modify sqlTask (#3152) 2020-07-06 17:57:30 +08:00
liukunyuan 0692dca4e6
update CommonUtils.java (#3137)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-05 21:50:42 +08:00
t1mon 83589c7b7c
Optimize kerberos tgt expire time in HadoopUtils. (#2700)
* Optimize PropertyUtils instantiation.

* Fix info error.

* Optimize kerberos tgt expire time.

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-04 18:22:30 +08:00
CalvinKirs d1282fdd96
In the case of multiple network cards, a valid host is returned (#2924)
* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* In the case of multiple network cards, a valid host is returned

* fix LOCAL_ADDRESS not assigned  value

* fix LOCAL_ADDRESS not assigned  value

* fix LOCAL_ADDRESS not assigned  value

* fix lock error

* fix lock error

* end

* end

* end

* end

* code smell

* code smell

* code smell

* code smell

* code smell

* code smell

* code smell

* lowering

* messy

* small change

* compile error

Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
2020-07-02 19:16:37 +08:00
muzhongjiang a9eb7b18b0
[bugfix] fixed "jackson enum conversion : InvalidFormatException" (#3078)
* add state

* fixed bug "jackson enum conversion  : InvalidFormatException"

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-02 18:07:57 +08:00
satcblue 631cc6ea1f
fix dag zoom (#3103)
Co-authored-by: linhaojie <linhaojie@lizhi.fm>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
2020-07-02 14:19:04 +08:00
satcblue b0a7bb4d9c
support multi drag (#3068)
Co-authored-by: linhaojie <linhaojie@lizhi.fm>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-07-02 13:57:52 +08:00
break60 8da0f4a1f5
[bugfix]Replace favicon icon with png and Cancel the initial value of the node cache #3111 (#3107)
* [bugfix]Replace favicon icon with png

* Cancel the initial value of the node cache
2020-07-02 11:16:46 +08:00
bao liang 2d0fe8a8c6
Merge pull request #3095 from apache/dev-merge
branch dev merge from dev-1.3.0
2020-06-30 16:50:17 +08:00
bao liang 5bac559d22
Revert "branch dev merge from dev-1.3.0 (#3087)" (#3094)
This reverts commit ab6e4310b4.

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-30 16:36:56 +08:00
dailidong ab6e4310b4
branch dev merge from dev-1.3.0 (#3087)
* add DolphinScheduler slack channel

* Update README.md

Co-Authored-By: Jiajie Zhong <zhongjiajie955@hotmail.com>

* modify soft_version 1.2.2 to 1.3.0

* Repair worklist cannot be deleted

* update schema version (#2657)

* change schema version from 1.2.2 to 1.3.0

* change version value of table t_ds_version from 2.0.0 to 1.3.0

* fix bug: sub worker flow in complement data would run start date (#2581) (#2656)

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* Fix conditions node save exception

* add unique key for t_ds_process_definition

* add unique key for t_ds_process_definition

* script variable has "processDefinitionId" is error #2664 (#2666)

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* update filename

* Update dolphinscheduler_ddl.sql

* Delete dolphinscheduler_ddl.sql

* Delete dolphinscheduler_ddl.sql

* add unique key for t_ds_process_definition

add unique key for t_ds_process_definition

* add unique key for t_ds_process_definition

add unique key for t_ds_process_definition

* Modify the alert log file
 

* blank in eamil and left font align #2648 (#2673)

* script variable has "processDefinitionId" is error #2664

* blank in eamil and left font align  #2648

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* fix zookeeper health check and merge volumes into logs volume (#2674)

* fix zookeeper health check and merge volumes into logs volume

1. Fix zookeeper health check
2. Merge volumes into logs volume
3. Modify `dockerfile/hooks/build` script

* Modify ci_e2e and docker image registry

* Modify ci_e2e

* Modify ci_e2e

* Modify ci_e2e and move ./dockerfile/hooks/check to ./docker/docker-swarm/

* migrate docker composer and move dockerfile dir to docker/build

* ut docker

* alter dockerfile to docker/build dir

* delete docker-compose old

* delete dockerfile zk

* delete dockerfile zk

* delete dockerfile zk

* [bug fix] some bugs from 1.2.0 to 1.3.0 (#2676)

* fix bug: some bugs about worker group from 1.2.0 to 1.3.0

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix ut bug

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* add worker group config (#2681)

* update README about DolphinScheduler

* Update issue templates

* update

* regularize api pom xml
update rpc maven compile to 1.8

* regularize api pom xml

* change commons.lang3.StringUtils to common.utils.StringUtils

* update pom.xml

* update

* correct equals method

* jasper-runtime is needed when api server start

* jasper-runtime jar is needed when api server start

* combine logback config of master/worker/alert/api server to one logback.xml

* remove tomcat runtime jar

* add UT

* add license

* remove jasper-runtime jar, not need anymore

* sovle jar conflict, remove servlet-api 2.5 jar

* remove servlet 2.5

* add embedded database h2 and update embedded zookeeper

* delete logs

* add license

* change log level to INFO

* simplify and optimize config

* add development state

* simplify common config

* add worker group config

* add worker group config

Co-authored-by: lidongdai <escheduler@outlook.com>
Co-authored-by: dailidong <dolphinschedule@gmail.com>

* delete log files while deleting process instances #2463 (#2693)

* script variable has "processDefinitionId" is error #2664

* blank in eamil and left font align  #2648

* delete log files while deleting process instances #2463

* delete log files while deleting process instances #2463

* delete log files while deleting process instances #2463

* delete log files while deleting process instances #2463

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* Update dolphinscheduler_ddl.sql

* Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml (#2692)

* Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml

1. add initdb volume for pg service in docker-compose.yml
2. modify ci_ut.yml

* add sudo in ci_ut.yml

* Query the metadata of indexes is not correct in postgresql/dolphinscheduler_ddl.sql

1. replace information_schema.STATISTICS to pg_stat_all_indexes
2. modify ProcessDefinitionMapperTest.java

* Remove inner zk and pg

* add inner pg

* conditions The task node adds a background color to distinguish successful and failed connections

* add LoggerServerTest UT (#2713)

* add LoggerServerTest UT

* add LoggerServerTest UT

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* add UT (#2717)

* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* fix worker group config no effect (#2719)

* Conditional judgment adds text

* fix

* fix #2449:It's should successful to cancel authorize a resource of a user if all the online process definitions created by this user didn't use this resource. (#2725)

* Underline the list of workflow operations

* Modify HDFS/S3 path (#2736)

* Modify HDFS/S3 path
 

* Modify HDFS/S3 path
 

* Update common.properties

Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: dailidong <dailidong66@gmail.com>

* Modify ip configuration

* set worker group  (#2743)

* simple config

* fix worker group config no effect

* fix #2598::allow to update resource suffix,but if it is authorized to other users,it is not allowed (#2732)

* fix #2598:allow to update resource suffix,but if it is authorized to other users,it is not allowed.

* add RESOURCE_IS_AUTHORIZED status

* verify whether the suffix is empty

* remove extra variables

* fix code smell

* Update known-dependencies.txt

* update worker group config (#2746)

* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* Update datasource.properties

* remove codehaus janino jar (#2742)

the license about janino maybe not compatiable with Apache v2

* [bug fix] fix bug:2634:Process shows success,when the task of the process is killed in the terminal (#2731)

* fix bug: some bugs about worker group from 1.2.0 to 1.3.0

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix ut bug

* fix bug: Process shows success,when the task of the process is killed in the termina

* fix bug #2634: Process shows success,when the task of the process is killed in the terminal

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* Update known-dependencies.txt (#2748)

* fix getWorkers bug on RunConfig (#2747)

* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* master select worker filter high load worker #2704 (#2733)

* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* Fix rename check parameter

* fix

* fix 2598:change getId to getUserId (#2757)

* fix #2598:allow to update resource suffix,but if it is authorized to other users,it is not allowed.

* add RESOURCE_IS_AUTHORIZED status

* verify whether the suffix is empty

* remove extra variables

* fix code smell

* change getId to getUserId

* Fix MySQL autoDeserialize bug (#2728)

* Fix MySQL autoDeserialize bug

* add MySQLDataSourceTest

* Fix workflow definition history node data cannot be opened

* fix

* delete normalize

* update e2e chrome version (#2775)

Co-authored-by: chenxingchun <438044805@qq.com>

* Modify the background color of conditions task node connection

* [BUF FIX]fix: the master would be blocked when worker group not exists (#2764)

* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* [BUG FIX] fix bug: dependent task failed when conditions task exists (#2768)

* fix bug 2464: change dependent task for process.

* remove unused code

* add ut

* add ut

* update comments

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* Hourly dependency configuration, 'currentHour' and 'last24Hours' (#2696)

* [feature] Add hourly dependency scenarios

* [Feature] #2680

add DependentUtilsTest UT

Co-authored-by: changqun.xu <changqun.xu@17zuoye.com>
Co-authored-by: xuchangqun <changqun.x@gmail.com>

* fix sql task bug #2739 (#2741)

* Solve the failure to execute non-query sql

* fix sql task bug

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: lgcareer <18610854716@163.com>

* Worker group adds check

* fix

* Worker group adds check (#2787)

* Worker group adds check

* fix

* add not worker log and remove worker invalid property (#2784)

* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* add not worker log and remove worker invalid property

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory (#2786)

* fixed #2788  resolve the bug #2788  (#2789)

* Modify the ambari-plugin readme for packing RPM packages and add a description of the replication plug-in directory

* fixed #2788

* [BUG FIX] fix bug:cannot pause work flow when task state is "submit success" (#2783)

* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

* fix bug(2781): cannot pause work flow when task state is "submit success"

* fix code smell

* add mysql other param blank judge

* test

* update comments

* update comments

* add ut

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix bug:#2735 (#2770)

1. store process id in zk.
2. resolve host and port through zk path.
3. modify ui key.

* Modify startup parameter worker group value

* Fix [Bug] process definition json worker group convert #2794 (#2795)

* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* add not worker log and remove worker invalid property

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* [BUG FIX]fix bug: Restart the worker service again, the previously submitted successful tasks are not executed bug (#2800)

* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

* fix bug(2781): cannot pause work flow when task state is "submit success"

* fix code smell

* add mysql other param blank judge

* test

* update comments

* update comments

* add ut

* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed

* update comments

* add sleep

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* delete unnecessary configuration items (#2803)

* rename worker_group_id to worker_group and set the value of worker_group_name (#2801)

* process definition json worker group convert #2794 (#2809)

* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* add not worker log and remove worker invalid property

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* compatible old version host not port delete log file (#2811)

* add LoggerServerTest UT

* add LoggerServerTest UT

* add LoggerServerTest UT
add RemoveTaskLogRequestCommandTest UT
add RemoveTaskLogResponseCommandTest

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* master select worker filter high load worker #2704

* add not worker log and remove worker invalid property

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* process definition json worker group convert #2794

* compatible old version host not port delete log file

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>

* Added instructions for ambari to add and remove Dolphin Schdumer components on nodes (#2806)

* delete unnecessary configuration items

* Added instructions for ambari to add and remove Dolphin Schdumer components on nodes

* Added instructions for ambari to add and remove Dolphin Schdumer components on nodes

* fix oracle connect failure (#2812)

* use method getJdbcUrl of parent

* Remove the incorrect print message

* Modify process group workerGroup value display and filter child node duplicate values

* Repair process definition bulk delete function

* when init workflow, the tenantId linked by the tenantId of user

* dag download does not refresh, modify the task instance operation width value

* fix

* [bug fix]fix bug: After the master is fault-tolerant, it cannot resume operation  (#2813)

* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

* fix bug(2781): cannot pause work flow when task state is "submit success"

* fix code smell

* add mysql other param blank judge

* test

* update comments

* update comments

* add ut

* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed

* update comments

* add sleep

* add null point check

* fix bug:After the master is fault-tolerant, it cannot resume operation

* fix bug: do not failover the host is 'NULL' process

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* When update the data source,the host and port show error (#2821)

* When update the data source,the host and port show error

* update DataSourceServiceTest

* update DataSourceServiceTest

* update DataSourceServiceTest

* Fix ORACLE service name value display

* Add tips for download workflow

* fix

* [bug fix] fix: The workflow is fault-tolerant and 2 task instances are generated (#2833)

* feature: add number configuration for master dispatch tasks

* fix bug(#2762) the master would be blocked when worker group not exists

* fix bug(#2762) the master would be blocked when worker group not exists

* fix ut

* fix ut

* fix bug(2781): cannot pause work flow when task state is "submit success"

* fix code smell

* add mysql other param blank judge

* test

* update comments

* update comments

* add ut

* fix bug: Restart the worker service again, the previously submitted successful tasks are not executed

* update comments

* add sleep

* add null point check

* fix bug:After the master is fault-tolerant, it cannot resume operation

* fix bug: do not failover the host is 'NULL' process

* fix bug:worker failover error.

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* add maven Xmx to 5g

* fix bug: send task result to master until success. (#2839)

* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* Fix errors when running on win10 (#2842)

1. DS cant run bat script
2. DS cant run python script
3. DS cant run datax script

* Fix abnormal display of regular worker parameters

* add source license full text into source license (#2840)

* add source license full text into source license

* exclude ScriptRunner.java on rat check

* fix table not exist when init db metadata (#2838)

* fix table not exist when init db metadata

* fix db init error

* fix ds-1.3.0 init db

* init db modify process instance json

Co-authored-by: eights <yelli.hl@foxmail.com>

* change notice copyright date from 2019 to 2019-2020 (#2847)

* add source license full text into source license

* exclude ScriptRunner.java on rat check

* change notice copyright date from 2019 to 2019-2020

* fix [BUG] TaskExecutionContextCacheManagerImpl Do not execute removeByTaskInstanceId #2745 (#2754)

* mr resource display and create user username length cannot be less than 3

* fix

* fix

* fix

* fix table T_DS_ALERT (#2854)

* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* fix

* fix

* modify t_ds_task_instance app_link change sql (#2861)

* change max length of user name to 39 (#2856)

* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

* change max length of username to 39

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* copy resources need get top directory first (#2866)

* copy resources need get top directory first

* copy resources need get top directory first

* fix #2860:copy files and create empty directory (#2876)

* copy resources need get top directory first

* copy resources need get top directory first

* fix #2860:copy files and create empty directory

* Fix bug:If user didn't set the value of tenant,release process definition fail (#2885)

* copy resources need get top directory first

* copy resources need get top directory first

* fix #2860:copy files and create empty directory

* Fix bug:If user didn't set the value of tenant,release process definition online fail

* merge method updateuser if and fix bug#2892

fix [BUG] When editing user information, clear the phone number, the database table t_ds_user.phone is not cleared #2892

* fix bug: sqoop mysql import params error (#2896)

* fix bug: sqoop mysql import params error

* fix bug: sqoop mysql import params error

* Fixed selection of tenant value and defalut changed to lower case

* update how to build and remove docs link (#2906)

* Fix tasks not execute in sub-process when rerun scheduled process. #2903 (#2908)

* fix create table sql exception (#2912)

fix create table sql exception

* fix #2910 master server will show exception for some time when it restart (#2913)

* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* update h2 database

* fix #2910 master server will show exception for some time when it restart

* Fixbug datax task (#2909)

* fix:  local param bug

* fix: UT bug

Co-authored-by: Rubik-W <whm_777@163.com>

* [license] add license and notice (#2916)

* fix bug: send task result to master until success.

* fix bug: send task result to master until success.

* add sleep

* set null if send result failed

* set null if send result failed

* change max length of username to 39

* test for gc

* change version to 1.3.0

* check and add some license files

* check and add some license files

* revert

Co-authored-by: baoliang <baoliang@analysys.com.cn>

* fix oom when no master is active in dev-1.3.0 (#2918)

* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* update h2 database

* fix #2910 master server will show exception for some time when it restart

* fix oom when no master is active

* fix worker oom when master server restart

* fix oom

* fix

* add UT

* fix worker group config no effect

* Python, sql, sqoop,datax task node editor adds zoom function #2929 (#2933)

* Python, sql, sqoop task node editor adds zoom function

* fix

* change 1.3.0 to 1.3.0-snapshot

* add janino-3.0.12.jar

* delete duplicate codes

* add commons-compiler-3.0.12.jar

* change delete status according to delete action result

* Delete the pom files animation.scss, _normalize.scss and _animation.scss under the common folder

* Change email to default (#2981)

* add mybatis-3 notice in the top-level notice (#3003)

* fix download resource file that under the directory error (#3007)

* add mybatis-3 notice in the top-level notice

* fix download resource file that under the directory error

* Fix unfiltered jar files in resource directory and UDF upload resource parameters (#3008)

* Change email to default

* Fix unfiltered jar files in resource directory and UDF upload resource parameters

* fix singe resource manager will NPE (#3013)

* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT

* update h2 database

* fix #2910 master server will show exception for some time when it restart

* fix oom when no master is active

* fix worker oom when master server restart

* fix oom

* fix

* add UT

* fix worker group config no effect

* fix singe resource manager will NPE

* Update HadoopUtilsTest.java

* merge from dev-1.3.0

* update ut

* remove duplicate code

* remove exception ut

* remove exception ut

* remove exception ut

* remove exception ut

* remove exception ut

* [bugfix] remove Conflicting configuration sudo and user (#3038)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* [bugfix] 1. fix alert.json includes repeated tag 'DOLPHIN_ALERT' 2. metainfo.xml The version number of the RPM package is no longer specified in the metainfo.xml (#3032)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* [feature] readme.md adds instructions for changing environment variables (#3031)

Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>

* merge from dev

* Change favicon.ico icon introduction method and package compilation to increase version number (#3030)

Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
Co-authored-by: chenxingchun <438044805@qq.com>
Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: lgcareer <18610854716@163.com>
Co-authored-by: bao liang <29528966+lenboo@users.noreply.github.com>
Co-authored-by: baoliang <baoliang@analysys.com.cn>
Co-authored-by: Simon <3656562@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: 张世鸣 <zhangshiming@cvte.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
Co-authored-by: liwenhe1993 <liwenhe@apache.org>
Co-authored-by: xiaochun.liu <xiaochun.liu@centurygame.com>
Co-authored-by: lidongdai <escheduler@outlook.com>
Co-authored-by: dailidong <dolphinschedule@gmail.com>
Co-authored-by: Tboy <guo.jiwei@immomo.com>
Co-authored-by: xuchangqun <58328021+xuchangqun@users.noreply.github.com>
Co-authored-by: changqun.xu <changqun.xu@17zuoye.com>
Co-authored-by: xuchangqun <changqun.x@gmail.com>
Co-authored-by: zixi0825 <649790970@qq.com>
Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: zhangchunyang <zhangchunyang0123@gmail.com>
Co-authored-by: tlhhup <137273278@qq.com>
Co-authored-by: yelli.hl@foxmail.com <amarantine@my.com>
Co-authored-by: eights <yelli.hl@foxmail.com>
Co-authored-by: yaoyao <42244568+1445103710@users.noreply.github.com>
Co-authored-by: 小清 <62982788+597365581@users.noreply.github.com>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Co-authored-by: t1mon <178317391@qq.com>
Co-authored-by: Rubik-W <39549317+Rubik-W@users.noreply.github.com>
Co-authored-by: Rubik-W <whm_777@163.com>
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
2020-06-30 15:23:35 +08:00
lenboo 6613f823c0 Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-merge 2020-06-30 14:35:24 +08:00
break60 2a0e32ff8c
Change favicon.ico icon introduction method and package compilation to increase version number (#3030) 2020-06-30 14:34:11 +08:00
lenboo 2638b8147e Merge remote-tracking branch 'upstream/dev-merge' into dev-merge 2020-06-30 11:18:43 +08:00
lenboo 9c78e2888b merge from dev 2020-06-30 11:18:42 +08:00
lenboo fd76f172c1 Merge remote-tracking branch 'upstream/dev' into dev-merge
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
2020-06-30 11:17:14 +08:00
lenboo cb6c711833 merge from dev-1.3.0 2020-06-30 11:15:14 +08:00
zhangchunyang 8c29e2a1c0
[feature] readme.md adds instructions for changing environment variables (#3031)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
2020-06-30 11:02:25 +08:00
zhangchunyang 0fc49b2023
[bugfix] 1. fix alert.json includes repeated tag 'DOLPHIN_ALERT' 2. metainfo.xml The version number of the RPM package is no longer specified in the metainfo.xml (#3032)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
2020-06-30 10:44:52 +08:00
Yichao Yang 38e485373d
[Feature-2925][server] Init TaskLogger in TaskExecuteProcessor (#2925) (#2965)
* [Feature-2925][common] Add exitVal judge in OSUtils.exeCmd (#2925)

* optimize the logger utils
2020-06-28 17:43:13 +08:00
zhangchunyang 39b3a37abd
[bugfix] remove Conflicting configuration sudo and user (#3038)
Co-authored-by: zhangchunyang1024 <zhangchunyang0123@gamil.com>
2020-06-28 11:37:59 +08:00
zixi0825 9bf67d80d0
Revise annotation spelling errors & Enhanced code robustness (#3042)
* revise annotation spelling errors & enhanced code robustness

* revise annotation spelling errors & enhanced code robustness

Co-authored-by: sunchaohe <sunzhaohe@linklogis.com>
Co-authored-by: dailidong <dailidong66@gmail.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-06-28 10:48:14 +08:00
CalvinKirs 8213da50d3
[WIP] load balance #3054 (#3057)
* Load balancing abstract

* code
smell

* lower weight select
2020-06-28 10:20:59 +08:00
Jave-Chen f1632e22c5
Set up JDK 11 for SonarCloud in github action. (#3052)
* Set up JDK 11 for SonarCloud in github action.

* Fix javadoc error with JDK 11.

* Prevent Javadoc from stopping if it finds any html errors.
2020-06-25 23:41:43 +08:00
liuhuijuan 6dffa97fdd
fix release for nginx not package config files, eg: zookeeper.properties (#3041)
* fix release for nginx bug

* add state (#3040)

Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>

* [BUG_FIX]fix  multi-threaded tests error. (#3044)

* fix  test mock error

* fix  test mock error

Co-authored-by: muzhongjiang <mu_zhongjiang@163.com>
Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
Co-authored-by: CalvinKirs <acm_master@163.com>
2020-06-25 14:50:31 +08:00
xingchun-chen 0dca8d8868
Merge branch 'dev' into dev-merge 2020-06-24 13:51:38 +08:00
lenboo e343ee0ee0 remove exception ut 2020-06-24 13:02:37 +08:00
lenboo 6a52efae52 remove exception ut 2020-06-24 12:54:17 +08:00
lenboo 7da095d04d remove exception ut 2020-06-24 11:32:04 +08:00
lenboo 3d23a6c6e6 remove exception ut 2020-06-24 11:04:42 +08:00
lenboo 513950c036 remove exception ut 2020-06-24 10:53:00 +08:00