Commit Graph

6318 Commits

Author SHA1 Message Date
zhuangchong fe532d5242 [maven-release-plugin] prepare release 3.0.0-alpha 2022-04-14 09:21:18 +08:00
Jiajie Zhong 4ae7cbc003
[release] Change release version (#9483) 2022-04-14 09:07:31 +08:00
Kerwin a0c15ada3a
[3.0.0-alpha-prepare]3.0.0 alpha prepare 9481 9476 (#9482) 2022-04-13 20:56:56 +08:00
Kerwin 82756c3128
[FIX-9471][Script] fix run install.sh error (#9472) (#9477)
Co-authored-by: mazhong <316422240@qq.com>
2022-04-13 18:03:55 +08:00
Kerwin 2c4d44dcf9
Add python module dependency in the dist module (#9450) 2022-04-12 14:53:20 +08:00
caishunfeng 9953e86b8e
[Future-9396]Support output parameters transfer from parent workflow to child work flow (#9410) (#9442)
* [Future-9396]Support output parameters transfer from parent workflow to child work flow

* fix note
2022-04-11 23:30:05 +08:00
Jiajie Zhong a82f5026b7
[cherry-pick] some commit until apr 11 (#9446)
* [python] Add missing document

which including `configuration`, `run example`
`how to connect remote server`

close: #9286, #9284, #8917

* [python] Recover python release properties

This patch recovers the properties `python.sign.skip=false`
when the combined profile `release,python` is used.

also close: #9433

* [doc] Add some dev missing doc

Including general-setting, task-definition, audit-log
and they related img

Co-authored-by: Tq <tianqitobethefirst@gmail.com>

Co-authored-by: Tq <tianqitobethefirst@gmail.com>
2022-04-11 21:47:46 +08:00
caishunfeng 9fda8c5811
[cherry-pick-3.0.0-alpha] Cherry pick dev to 3.0.0 alpha (#9429)
* [Fix][UI Next][V1.0.0-Alpha]Add zh for dag execution policy (#9363)

* [Bug-9235][Alert]Fix wechat markdown message and change wechat form structure (#9367)

* fix wechat issues:
1. change table msg type to markdown.
2. change userId to not required and enrich hints
3. change 'app id' to 'app id and chat id'

* fix wechat issues:
1. revert table showtype and add markdown showtype.
2. enrich hints.
3. delete 'chatid', rename agentid to weChatAgentIdChatId.
4. modify code to send markdown message.

* fix wechat issues: Change the language pack of agentId to agentId/chatId.

* fix format

* fix param name

Co-authored-by: Amy <amywang0104@163.com>

* [FIX-9355] Fix scheduleTime of start-process-instance api in api-doc (#9359)

* fix #9355

* fix #9355

* fix ut error

* fix ut error

* [CI] try to fix ci (#9366)

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* [optimization] [Service] Optimization ProcessService and add ProcessService interface (#9370)

* [task-spark][docs] Corrected notice section (#9375)

* [python] Migrate pythonGatewayServer into api server (#9372)

Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

The distribute package size before and after this patch is:

```sh
# before
796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz

# after
647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
```

* [Fix][UI Next][V1.0.0-Alpha] Add light color theme to echarts. (#9381)

* [Bug][API-9364]fix ProcessInstance wrong alert group id (#9383)

* fix ProcessInstance wrong alert group id

* change  createComplementCommandList method to protected

* [BUG][WORKER-9349]fix param priority (#9379)

* fix param priority

* fix params priority code logic

* [Improvement] change method access (#9390)

* change method to protected

* change method access

* [Fix-9221] [alert-server] optimization and gracefully close (#9246)

* [Fix-9221] [alert-server] optimization and gracefully close

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unused mock data

This closes #9221

* [Fix-9221] [alert-server] remove unnecessary Mockito stubbings

* [Fix-9221] [alert-server] init AlertPluginManager in AlertServer

* [Fix-9221] [alert-server] AlertServerTest add AlertPluginManager installPlugin

* [Fix-9221] [alert-server] replace @Eventlistener with @PostConstruct

* [Fix-9221] [alert-server] sonar check solution

* [Improvement-9221] [alert] update constructor injection and replace IStoppable with Closeable

Co-authored-by: guoshupei <guoshupei@lixiang.com>

* [Fix][UI Next][V1.0.0-Alpha] Fix the task instance forced success button multi-language support error. (#9392)

* [doc] Change get help from dev mail list to slack (#9377)

* Change all get help from dev mailing list to slack, because
  we find out mailing list have many users ask for subscribe
  and they maybe subscribe by accident.
* remove join dev mailing list in faq.md because we already
  have it in https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html

* Add new code owner of docs module (#9388)

* [CI] Enable CI to remove unexpected files in /docs/img dir (#9393)

* [Bug][UI Next]Modify the display state logic of save buttons under workflow definition (#9403)

* Modifying site Configurations

* Modify the display state logic of save buttons under workflow definition

* [doc] Remove observability (#9402)

SkyWalking v9 is coming soon and there are without
DolphinScheduler menus anymore, So we should remove
the SW agent to avoid confusion.

close: #9242

* [DS-9387][refactor]Remove the lock in the start method of the MasterRegistryClient class (#9389)

* [Fix-9251] [WORKER] reslove the sql task about of add the udf resource failed (#9319)

* feat(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.extend the type of storage
3.add the s3utils
to manager resource
4.automatic inject the storage in addition to your
config

* fix(resource  manager): update the dependency

* fix(resource  manager): extend s3 to the storage of ds

fix the constant of hadooputils

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager):

merge  the unitTest:
1.TenantServiceImpl
2.ResourceServiceImpl
3.UserServiceImpl

* fix(resource  manager): extend s3 to the storage of ds

merge the resourceServiceTest

* fix(resource  manager): test  cancel the test method

createTenant verifyTenant

* fix(resource  manager): merge the code  follow the check-result of sonar

* fix(resource  manager): extend s3 to the storage of ds

fit the spell question

* fix(resource  manager): extend s3 to the storage of ds

revert the common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the storageConfig with None

* fix(resource  manager): extend s3 to the storage of ds

fix the judge of resourceType

* fix(resource  manager): extend s3 to the storage of ds

undo the compile-mysql

* fix(resource  manager): extend s3 to the storage of ds

delete hadoop aws

* fix(resource  manager): extend s3 to the storage of ds

update the know-dependencies to delete aws 1.7.4
update the e2e
file-manager common.properties

* fix(resource  manager): extend s3 to the storage of ds

update the aws-region

* fix(resource  manager): extend s3 to the storage of ds

fix the storageconfig init

* fix(resource  manager): update e2e docker-compose

update e2e docker-compose

* fix(resource  manager): extend s3 to the storage of ds

revent the e2e common.proprites

print the resource type in propertyUtil

* fix(resource  manager): extend s3 to the storage of ds
1.println the properties

* fix(resource  manager): println the s3 info

* fix(resource  manager): extend s3 to the storage of ds

delete the info  and upgrade the s3 info to e2e

* fix(resource  manager): extend s3 to the storage of ds

add the bucket init

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): extend s3 to the storage of ds

upgrade the s3 endpoint

* fix(resource  manager): withPathStyleAccessEnabled(true)

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* fix(resource  manager): upgrade the  s3client builder

* fix(resource  manager): correct  the s3 point to s3client

* fix(resource  manager): update the constant BUCKET_NAME

* fix(resource  manager): e2e  s3 endpoint -> s3:9000

* fix(resource  manager): extend s3 to the storage of ds

1.fix some spell question
2.delete the import *

* style(resource  manager): add info to createBucket

* style(resource  manager): debug the log

* ci(resource  manager): test

test s3

* ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql

* fix(resource  manager): update the h2 sql

* fix(resource  manager): solve to delete the tenant

* style(resource  manager): merge the style end delete the unuse s3 config

* fix(resource  manager): extend s3 to the storage of ds

UPDATE the rename resources when s3

* fix(resource  manager): extend s3 to the storage of ds

1.fix the code style of QuartzImpl

* fix(resource  manager): extend s3 to the storage of ds

1.impoort restore_type to CommonUtils

* fix(resource  manager): update the work thread

* fix(resource  manager): update  the baseTaskProcessor

* fix(resource  manager): upgrade dolphinscheduler-standalone-server.xml

* fix(resource  manager): add  user Info to dolphinscheduler_h2.sql

* fix(resource  manager): merge  the resourceType to NONE

* style(upgrade the log level to info):

* fix(resource  manager): sysnc the h2.sql

* fix(resource  manager): update the merge the user tenant

* fix(resource  manager): merge the resourcesServiceImpl

* fix(resource  manager):

when the storage is s3 ,that the directory can't be renamed

* fix(resource  manager): in s3 ,the directory cannot be renamed

* fix(resource  manager): delete the deleteRenameDirectory in E2E

* fix(resource  manager): check the style and  recoverd the test

* fix(resource  manager): delete the log.print(LoginUser)

* fix(server): fix the  udf serialize

* fix(master  task): update the udfTest to update the json string

* fix(test): update the udfFuncTest

* fix(common): syn the common.properties

* fix(udfTest): upgrade the udfTest

* fix(common): revent the common.properties

* [Fix-9316] [Task] Configure DB2 data source SQL script execution report ResultSet has been closed exception in SQL task  (#9317)

* fix db2 error in the sql task

* update limit in sql task

* [UI] Migrate NPM to PNPM in CI builds (#9431)

Co-authored-by: Devosend <devosend@gmail.com>
Co-authored-by: Tq <tianqitobethefirst@gmail.com>
Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: gaojun2048 <gaojun2048@gmail.com>
Co-authored-by: mans2singh <mans2singh@users.noreply.github.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
Co-authored-by: guoshupei <15764973965@163.com>
Co-authored-by: guoshupei <guoshupei@lixiang.com>
Co-authored-by: songjianet <1778651752@qq.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: labbomb <739955946@qq.com>
Co-authored-by: worry <7039986@qq.com>
Co-authored-by: nobolity <nobolity@users.noreply.github.com>
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-04-11 16:50:26 +08:00
Jiajie Zhong ff6a3bd6dd
[Bug][UI Next]Modify the display state logic of save buttons under workflow definition (#9403) (#9411)
* Modifying site Configurations

* Modify the display state logic of save buttons under workflow definition

Co-authored-by: labbomb <739955946@qq.com>
2022-04-09 17:48:41 +08:00
Devosend 83c745eb41
[cherry-pick][Fix][UI Next][V1.0.0-Alpha]Add zh for dag execution policy (#9397) 2022-04-08 13:36:02 +08:00
songjianet 6c9e07cce2
[cherry-pick][Fix][UI Next][V1.0.0-Alpha] Fix the task instance forced success button multi-language support error. (#9395) 2022-04-08 10:21:06 +08:00
Devosend aba257084b
[Fix][UI-Next] Rename process to workflow (#9350) 2022-04-06 10:23:30 +08:00
Devosend c21f8c650b
[Fix] [UI-Next][V1.0.0-Alpha] Rename node to task in the task creation modal (#9351) 2022-04-06 10:17:20 +08:00
Tq 4873ec8a45
fix task definition page info total count (#9354) 2022-04-06 09:57:20 +08:00
xiangzihao 40b73f7962
[Improve][CI] improve ci checking (#9325) 2022-04-06 09:33:41 +08:00
zchong 4a29c6a6c8
[Improvement-9338][API] show more create datasource exception message (#9336)
* Update DataSourceServiceImpl.java

fix  error message miss

* Update DataSourceServiceImpl.java

import optional jar
2022-04-06 09:17:29 +08:00
labbomb ee0e3beb9d
[docs] Modifying site Configurations (#9341) 2022-04-03 21:48:22 +08:00
xiangzihao 3c5568c16c
[Doc] There are some version errors in the 2.0.1-release (#9335) 2022-04-03 12:55:25 +08:00
songjianet a866465d01
[Fix][UI Next][V1.0.0-Alpha] Fix the time columns cannot be broken in the cron manage table. (#9334) 2022-04-02 18:12:31 +08:00
songjianet d7a4dc1dbc
[Fix][UI Next][V1.0.0-Alpha] Fix the two hover prompts appear. (#9333)
* [Fix][UI Next][V1.0.0-Alpha] Fix the two hover prompts appear.

* [Fix][UI Next][V1.0.0-Alpha] Fix the two hover prompts appear.
2022-04-02 17:01:07 +08:00
caishunfeng 40e44f6bc4
fix repeated submit task (#9323)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-04-02 14:06:35 +08:00
Jiajie Zhong 9f84dbbda0
[doc] Migrate dev doc from website repository (#9291)
* [doc] Migrate dev doc from website repository

* Correct release language

* Fix license issue

* Add all images from website

* Delete unused images

* Add CI

* Fix ci

* fix ci

* Remove no need file

* Add latest commit from website

* correct img_utils.py script

* Remove unused imgs
2022-04-02 10:43:34 +08:00
guoshupei 3e5781b6e1
[Fix-9065] [master] when task submit failed, remove from queue and add state event to thread (#9186)
* [Fix-9065] [master] when task submit failed, remove from queue and add state event to thread

This closes #9065

* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteThread.java

yes, your idea is better

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>

* [Bug-9065] [master] update logger info

Co-authored-by: guoshupei <guoshupei@lixiang.com>
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-04-01 22:23:35 +08:00
Amy0104 6b2a4e31ea
[Fix][UI-Next][V1.0.0-Alpha] Fix the notification strategy options display error in the set parameters before timing modal. (#9314) 2022-04-01 17:41:39 +08:00
songjianet c99b33a083
[Docs][UI Next][V1.0.0-Alpha] Delete E2E docs. (#9309) 2022-04-01 14:22:01 +08:00
mans2singh 4b3401a418
[dolphinscheduler-api][task-group-controller] Corrected notes for create, update and queryTaskGroupByStatus (#9176) 2022-04-01 10:31:41 +08:00
caishunfeng 1073fcae44
[Bug-9295][Master] fix repeated submit task (#9304)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-31 21:10:09 +08:00
songjianet 81f9b876f9
[Build] Update NaiveUI. (#9306) 2022-03-31 20:41:36 +08:00
Amy0104 c49ae015ec
[Feature][UI Next][V1.0.0-Alpha] Add a DISPATCH state to the task. (#9302) 2022-03-31 14:21:36 +08:00
caishunfeng 52ba2c6475
[Improvement-9288][Master] add task event thread pool (#9293)
* add task event thread

* license heander

* ci test

* delete unuse file

* fix CI test

Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-31 14:13:33 +08:00
songjianet 192ee99876
[Fix][UI Next][V1.0.0-Alpha] Fix the personal information did not return to edit the previous state. (#9301)
* [Fix][UI Next][V1.0.0-Alpha] Fix the personal information did not return to edit the previous state.
2022-03-31 13:59:46 +08:00
Amy0104 9d68b178d9
[Fix][UI Next][V1.0.0-Alpha] Fix the problem of not copying. (#9285)
* [Fix][UI Next][V1.0.0-Alpha] Fix the problem of not copying.

* [Fix][UI Next][V1.0.0-Alpha] Fix the warning.
2022-03-30 18:48:44 +08:00
caishunfeng ddf1ff98fa
[Improvement-7697][Master/Worker] Change the task ack to runnning callback (#8719)
* rebase dev

* change task state to dispatch if dispatch success

* update task host when dispatch

* add dispatch task event

* test

* check tenant after enable auto create

* handle dispatch state

Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-30 17:35:05 +08:00
worry b4b5241797
[DS-9266][feat][server]Implement show dolphinscheduler banner at launch (#9273)
- add banner.txt
- modify application.yaml

This closes #9266
2022-03-30 16:45:24 +08:00
wangxj3 fd76436d87
[Feature-#8318] [master-server] Sub process transfer parameters to father process (#8522)
* load param from subprocess

* unit test

* modify code style

Co-authored-by: wangxj <wangxj31>
2022-03-30 14:55:59 +08:00
caishunfeng df519f8044
remove common task log (#9274)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-03-30 11:42:47 +08:00
Devosend 11d9859c40
[Fix][UI Next][V1.0.0-Alpha] Fix the connection line center bug for workflow dag (#9272)
* fix the connection line center bug for workflow dag

* delete unnecessary code
2022-03-30 10:52:32 +08:00
songjianet 08af02151b
[Fix][UI Next][V1.0.0-Alpha] Fix the floating full-name display needed for menu bar. (#9264) 2022-03-29 23:57:09 +08:00
Amy0104 4ad1764932
[Fix] [UI-Next] [V1.0.0-Alpha] Fix the resource options in task node display incorrect after creating a new file. (#9269) 2022-03-29 23:11:41 +08:00
Amy0104 3516cd56b1
[Bug-FE][UI Next][V1.0.0-Alpha] Fix the datepicker panel up and down in start modal. (#9267) 2022-03-29 23:11:06 +08:00
Amy0104 1630147872
[Fix][UI Next][V1.0.0-Alpha] Fix the datasource display error in SQL. (#9261) 2022-03-29 23:10:07 +08:00
guoshupei e2c1cc0579
[Fix-9222] [master] Support for deploying multiple masters on one node (#9240)
This closes #9222

Co-authored-by: guoshupei <guoshupei@lixiang.com>
2022-03-29 22:46:02 +08:00
Amy0104 860771f922
[Fix] [UI-Next] [V1.0.0-Alpha]Fix the magic status display. (#9265) 2022-03-29 22:03:17 +08:00
Devosend c87af0eba8
[Fix][UI Next] Add workflow online status display (#9253) 2022-03-29 17:39:46 +08:00
Devosend fa56751a56
fix Alert instance form not init bug (#9258) 2022-03-29 17:39:04 +08:00
Eric Gao 4c1578a374
[Feature-8369][Document]Add docs building test instructions in pydolphinscheduler DEVELOP.md (#8369) (#8693) 2022-03-29 10:56:53 +08:00
Amy0104 a56e1d260d
[Fix][UI Next][V1.0.0-Alpha] Fix the timeout strategy display error. (#9245) 2022-03-28 22:59:20 +08:00
JinYong Li 85e56b5e2a
fix 9243 (#9244) 2022-03-28 20:45:01 +08:00
Tq 03a0c9f544
[Bug] [ALERT-9224] fix wechat alert functions: app and group chat (#9225)
* fix wechat alert with app and group chat

* add apache header and fix imports

* delete wechat alert 'userSendMsg' property code

* delete wechat alert 'userSendMsg' property code in Test

* delete wechat alert 'userSendMsg' property code in factory Test

* change `getSubWorkFlowParam` from private to protected
2022-03-28 20:33:14 +08:00
songjianet 0d0674df1f
[Fix][UI Next][V1.0.0-Alpha] Fix the cannot update tenant individually on user info site. (#9237) 2022-03-28 17:55:15 +08:00