Commit Graph

82 Commits

Author SHA1 Message Date
lenboo fd76f172c1 Merge remote-tracking branch 'upstream/dev' into dev-merge
# Conflicts:
#	dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
2020-06-30 11:17:14 +08:00
muzhongjiang 40bbef5e3d
add state (#3040)
Co-authored-by: mzjnumber1@163.com <mzjnumber1@163.com>
2020-06-23 18:47:04 +08:00
lenboo 714ae33e40 merge from dev 2020-06-19 14:20:24 +08:00
Yichao Yang f7ea4f1978
[Feature-2127][sql-schema] Fix the mysql version limited error for 5.5x - 5.64 (#2911)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-06-15 23:21:33 +08:00
lenboo 1904a7022a change 1.3.0 to 1.3.0-snapshot 2020-06-10 11:42:15 +08:00
dailidong 0de8e58965
fix create table sql exception (#2912)
fix create table sql exception
2020-06-06 16:38:40 +08:00
Yelli 3cbe865776
modify t_ds_task_instance app_link change sql (#2861) 2020-06-01 17:21:23 +08:00
dailidong 6886ed51c5
fix table T_DS_ALERT (#2854)
* fix worker group config no effect

* remove codehaus janino jar
the license about janino maybe not compatiable with Apache v2

* Merge remote-tracking branch 'upstream/dev-1.3.0' into dev-1.3.0

# Conflicts:
#	dolphinscheduler-server/src/main/resources/config/install_config.conf

* datasource config

* Update datasource.properties

* fix RunConfig bug

* remove param monitor server state

* fix table T_DS_ALERT
2020-06-01 11:11:34 +08:00
lgcareer 470bd59db7
rename worker_group_id to worker_group and set the value of worker_group_name (#2801) 2020-05-25 10:18:01 +08:00
Simon 83006e971e
Merge pull request #17 from apache/dev-1.3.0
Dev 1.3.0
2020-05-14 20:00:52 +08:00
xiaochun.liu 4024465465 fix , 2020-05-14 16:23:07 +08:00
xiaochun.liu 91c544f24f sql 2020-05-14 00:04:52 +08:00
liwenhe1993 29a353e689
Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml (#2692)
* Using docker/docker-swarm/docker-compose.yml to run pg and zk in ci_ut.yml

1. add initdb volume for pg service in docker-compose.yml
2. modify ci_ut.yml

* add sudo in ci_ut.yml

* Query the metadata of indexes is not correct in postgresql/dolphinscheduler_ddl.sql

1. replace information_schema.STATISTICS to pg_stat_all_indexes
2. modify ProcessDefinitionMapperTest.java

* Remove inner zk and pg

* add inner pg
2020-05-12 20:24:17 +08:00
Simon da4863cbee
Merge pull request #16 from apache/dev-1.3.0
Dev 1.3.0
2020-05-12 18:55:39 +08:00
Simon 01045178b7
Update dolphinscheduler_ddl.sql 2020-05-12 18:35:14 +08:00
bao liang d99471806e
[bug fix] some bugs from 1.2.0 to 1.3.0 (#2676)
* fix bug: some bugs about worker group from 1.2.0 to 1.3.0

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix bug: scheduler from 1.2.0 to 1.3.0 do not work

* fix ut bug

Co-authored-by: baoliang <baoliang@analysys.com.cn>
2020-05-12 16:46:16 +08:00
Simon 028903b5d3
add unique key for t_ds_process_definition
add unique key for t_ds_process_definition
2020-05-11 12:58:12 +08:00
Simon 8121cee6a5
add unique key for t_ds_process_definition
add unique key for t_ds_process_definition
2020-05-11 12:57:26 +08:00
Simon 6b0d116345
Merge pull request #15 from apache/dev-1.3.0
Dev 1.3.0
2020-05-11 12:55:47 +08:00
Simon 7cb387aa5a
Delete dolphinscheduler_ddl.sql 2020-05-11 12:55:31 +08:00
Simon 4b98d4a0a3
Delete dolphinscheduler_ddl.sql 2020-05-11 12:55:09 +08:00
Simon d9efc6a8a9
Update dolphinscheduler_ddl.sql 2020-05-11 12:52:18 +08:00
张世鸣 1f3625f647 update filename 2020-05-11 12:36:08 +08:00
Rubik-W 68e3487e34
feature: Add user state #1923 (#2424)
* feature: #1923

* fix:docker sql script add column

* fix: update sql script

* fix: UT bug

* fix: UT bug

* fix: UT bug

* fix: e2e UT bug

* fix: e2e button location

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-05-11 11:45:29 +08:00
张世鸣 6f09935f4b add unique key for t_ds_process_definition 2020-05-09 15:23:02 +08:00
张世鸣 e988d021cc add unique key for t_ds_process_definition 2020-05-09 15:20:31 +08:00
lgcareer 050dae7ca4
update schema version (#2657)
* change schema version from 1.2.2 to 1.3.0

* change version value of table t_ds_version from 2.0.0 to 1.3.0
2020-05-09 15:02:06 +08:00
xingchun-chen a5b0536437
modify soft_version 1.2.2 to 1.3.0 (#2642)
Co-authored-by: chenxingchun <438044805@qq.com>
2020-05-08 18:49:43 +08:00
chenxingchun 80a4a445d9 modify soft_version 1.2.2 to 1.3.0 2020-05-08 18:30:36 +08:00
张世鸣 9395abb155 unique check before import ProcessDefinitions 2020-05-07 12:52:28 +08:00
张世鸣 97d09e1d6a init 2020-05-06 19:38:34 +08:00
张世鸣 82fee46d70 unique check before import ProcessDefinitions 2020-05-06 19:31:56 +08:00
qiaozhanwei d6798c6db7
1,task status statistics and process status statistics bug fix (#2357) 2,worker group bug fix (#2430)
* dispatch task fail will set task status failed

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,no worker condition , master will while ture wait for worker startup
2,worker response task status sync wait for result

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

* 1,task status statistics and process status statistics bug fix (#2357)
2,worker group bug fix

Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
2020-04-16 18:16:19 +08:00
lgcareer 8b4eb20b5c
fix NPE when grant resource or save process definition (#2434)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-04-15 22:28:49 +08:00
lgcareer 0bad7c2f53
add worker_group field and remove worker_group_id field of the table t_ds_command and t_ds_error_command (#2428) 2020-04-15 13:50:06 +08:00
qiaozhanwei 3b56ae2f82 init.sql modify 2020-04-09 22:43:04 +08:00
qiaozhanwei 46dbacef25 init.sql modify 2020-04-09 22:18:50 +08:00
qiaozhanwei eacc66d95d mysql and postgresql checkpoint full scale table modify 2020-04-08 15:10:47 +08:00
qiaozhanwei 94e8bb15c9 Merge remote-tracking branch 'remotes/upstream/dev' into dev 2020-04-01 16:20:30 +08:00
lgcareer db30721f5e
init full_name in dml of version 1.2.2 (#2341)
Co-authored-by: dailidong <dailidong66@gmail.com>
2020-04-01 11:56:28 +08:00
qiaozhanwei 152c814126 test 2020-03-31 16:51:19 +08:00
lgcareer 38c965ddc9
new feature for #404 add resource tree function (#2323)
* add create resource directory

* add create resource directory

* update the resource test

* add upgrade sql in version 1.2.2

* Adding request parameter id to update queryResourceListPaging

* set isDirectory value is false default

* add full name to update updateResource

* remove request parameter isDirectory to update createResource method

* update queryResourceListPaging with change get to post

* update updateResource method with remove fullName

* File management list modification (#1976)

* add resource component

* add resource tree visitor

* return json string

* update queryResourceList

* upload file need fullName

* add method rootNode

* Shell task resources and authorization resources (#1989)

* File management list modification

* Shell task resources and authorization resources

* download resource when execute task

* download resource when execute task

* update authorization type

* download resource when execute task

* Spark task resource changes (#1990)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* download resource when execute task

* update udf function service

* add resource type in ResourceComponent

* UDF resource tree and change DAG style (#2019)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* add deleteIds method in ResourceMapper and ResourceMapperTest

* Add comments on class and method

* add queryResourceByName method in controller

* update verify-name with change name to full name

* update queryResource with add parameter pid

* update queryResource with add parameter pid

* add resource ids in process definition and delete resource need judge whether it is used by any process definition

* Breadcrumb development (#2033)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix (#2040)

* File management list modification

* Shell task resources and authorization resources

* Spark task resource changes

* UDF resource tree and change DAG style

* Breadcrumb development

* Breadcrumb development

* Resource tree bug fix

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Fix github action rerun failed (#2067)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* Change crumb position

* Change crumb position (#2068)

* build resource process definition map

* UDF changed to multiple choice

* UDF changed to multiple choice (#2077)

* Change crumb position

* UDF changed to multiple choice

* build resource process definition map (#2076)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* update resource name also need update all the children full name (#2096)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Limit customization file content to no more than 3000 lines(#2128) (#2140)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList (#2192)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* Modify the main jar package

* Modify the main jar package (#2200)

* Change crumb position

* UDF changed to multiple choice

* Limit customization file content to no more than 3000 lines

* Limit customization file content to no more than 3000 lines(#2128)

* Modify the main jar package

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* choose main jar with resource tree (#2220)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* getResouDelete checksum and modify parameter namerceId

* revert .env

* remove parameter pid

* Delete request interface

* go back to the last page

* jar interface call

* Fix issue #2234 and #2228

* change resource name with full name

* Fix issue #2234 and #2228 (#2246)

* update resource service test

* Fix github action rerun failed

* add status of PARENT_RESOURCE_NOT_EXIST

* build resource process definition map

* update resource name also need update all the children full name

* need add queryResource

* add queryResourceJarList

* add queryResourceJarList

* add queryResourceJarList

* add resource filter in order to get filtered resource

* add comments of resource filter

* update list children by resource

* Return null if query resource list is empty

* update queryResource method change parameter pid to id

* revert .env

* remove parameter pid

* Fix issue #2234 and #2228

* change resource name with full name

* Fix list query value error

* remove unauth-file with authorize-resource-tree

* Repair data cannot be echoed

* Repair data cannot be echoed

* execute mr and spark task need query resource name before

* Authorized resource interface replacement

* Authorized resource interface replacement

* Filter UDF resources

* Change parameters

* need query all authorized directory children when create task

* Change normalize.scss import method and animation.scss license modification

* Delete file list update processing

* It's fixed that resource not deleted in hdfs when delete it.

* add tooltips

* add tooltips (#2310)

* Echo workflow name and modify udf management name

* [new feature]add resource tree function

* revert front code in order to be same as dev branch

* revert front code in order to be same as dev branch

* revert common.properties and application.properties

* add super method

* update flink parameter test

* update flink parameter and unit test

* update resource service test

* If resource list is empty,need init it

* update flink parameter test

Co-authored-by: break60 <790061044@qq.com>
Co-authored-by: xingchun-chen <55787491+xingchun-chen@users.noreply.github.com>
Co-authored-by: qiaozhanwei <qiaozhanwei@outlook.com>
2020-03-28 15:43:07 +08:00
lgcareer af8efe5184
add worker group and remove worker group id in dll (#2313) 2020-03-27 09:54:15 +08:00
Yelli 91ecc549a1
fix bug: data too long for column 'app_link' (#2146)
* fix bug: data to long for app_link field

* change app_link field length in 1.2.2 ddl

* change app_link field type to text
2020-03-12 21:52:48 +08:00
lgcareer bbe2cd4864
refactor worker group (#2132)
* replace worker group id with worker group

* add worker group field in ddl

* remove worker group id
2020-03-10 17:10:47 +08:00
dailidong 4230d5a737
add embedded database h2 and update embedded zookeeper (#2082)
add embedded database h2 and update embedded zookeeper
2020-03-05 17:32:40 +08:00
qiaozhanwei b8014d91cf
add workflow id, name for email content (#2071)
* add tenantService UT

* fix spell

* #464 email content

* modify mysql1.2.2 ddl
2020-03-05 11:04:00 +08:00
Yelli 619fb30c50
#1310 add executor user to the process instance & task instance pages (#1973)
* add executor name in processDefinitionInstance & taskInstance

* modify processInstanceTest

* modify processInstanceTest

* #1310 add executor user to the process instance & task instance pages

* add processInstanceUT & taskInstanceUT

* add processInstanceUT & taskInstanceUT

* modify processInstanceTest & taskInstanceTest

* Remove duplicate code

* add userServiceUT
2020-02-20 13:25:41 +08:00
Yelli d937a6ae94 Add modify user name for process definition (#1919)
* class overrides equals() and should therefore also override hashCode()

* #1862 add modify user in process difinition list

* #1862 add pg-1.2.2 ddl.sql

* modify ScriptRunnerTest

* add updateProessDifinition UT

* modify updateProcessDifinition UT

* modify updateProcessDifinition UT

* modify mysql 1.2.2 ddl.sql&dml.sql

* add scope test to mysql in pom

* modify pg-1.2.2 ddl.sql
2020-02-10 16:37:51 +08:00
liwenhe1993 30db5c1467 Api module relp server module/1756 (#1766)
* Remove server module on api module

1. Remove server module and add dao module.
2. Move quartz in server module to dao module.
3. Update config and class.


to resolve conflicts,I deleted CombinedApplicationServer.java

Co-authored-by: dailidong <dailidong66@gmail.com>
2020-01-20 16:26:23 +08:00