Commit Graph

6997 Commits

Author SHA1 Message Date
jackfanwan b709cd9a12 [fix-10961][alert server]Change the content of alert to an array (#11033)
* Change the content of alert to an array

* Increase unit test coverage

* Modify the code format

* Modify the code format

* Modify judgment statement

* Deleting invalid References

* Modify the previous code implementation and improve the test data

* Modify the code format

* Move the location of the Content conversion

* Remove useless code

* Change the code format

* Modify unit tests

* Modify unit tests

* Modify unit tests

* Changing the location of the code

* Modify the code

Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
Co-authored-by: 范万龙 <fanwanlong@fanwanlongs-MBP.lan>
2022-09-19 08:55:15 +08:00
Kerwin 70ddb11636
[Fix][UI] Fix workflow name overlaps bug (#11862) (#12019)
Co-authored-by: Devosend <devosend@gmail.com>
2022-09-19 01:11:16 +08:00
Kerwin c4c943277c
[3.0.1-prepare]cherry-pick [Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11994)
* cherry-pick [Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
2022-09-18 14:23:35 +08:00
JinYong Li ab8f878631
fix heartBeatTaskCount bug (#12024) 2022-09-18 11:49:21 +08:00
Wenjun Ruan bc629f8f58
[3.0.1-prepare]cherry-pick Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469) (#12013)
* Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469)

(cherry picked from commit 4d13a5104b)
2022-09-18 09:37:57 +08:00
JiPeng Wang c10deeeda6
CP Workflow Definition Name Change Validation Optimization. (#11986) 2022-09-18 09:11:39 +08:00
Kerwin 2bdd0d9c6c
add unit test ci timeout (#11655) (#12021)
Co-authored-by: xiangzihao <460888207@qq.com>
2022-09-18 07:15:48 +08:00
Kerwin 1b2d8be489
[helm] fix worker template persistentVolumeClaim link (#11248) (#12018)
Co-authored-by: shiyu li <675590586@qq.com>
2022-09-18 07:15:11 +08:00
Wenjun Ruan 041e1fd3e6
Set master's task running status in `runTask` to avoid the task group acquire failed, but the task status is in running (#11451) (#12011)
(cherry picked from commit 05589606a2)
2022-09-17 22:55:40 +08:00
caishunfeng 5df86d0b7e
[Cherry-pick-3.0.1] #11697 #11400 #11696 (#12023)
* [Cherry-pick-3.0.1][fix-#11691] Add lz4-java in clickhouse datasource. #11697

* [Cherry-pick-3.0.1][fix-#11689] change replaceAll to replace in method of tryExecuteSqlResolveColumnNames #11696

* [fix-11378] fix API-Server release response message typo (#11400)

* [fix-11295] remove duplicate classes

* [fix-11378] fix response message error

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>

Co-authored-by: fuchanghai <33984497+fuchanghai@users.noreply.github.com>
Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
2022-09-17 22:41:15 +08:00
Jiajie Zhong 257187c521 [python] Refactor structure to avoid cycle import (#11167)
Currently, our core and side module dependent on each others. and will cause
cycle import in our codebase, especially in issue #10905, we try to refactor
to solve this problem.

This patch do the following change:

* Rename module `side` to `models`
* Move `core/base` and `core/sidebase` to dir `modules`
* Move `configuration` and `default_config.yaml` to the root of pydolphinscheduler

(cherry picked from commit 7a766cbcf2)
2022-09-17 22:13:03 +08:00
Jiajie Zhong 9f0b2ca550 [python] Add support interpreter version 310 and 311-dev (#11170)
* Change version of package
* Fix tox error
* Change dev status to beta
* py4j not work on py{10,11} OS Windows

(cherry picked from commit 4d427ee215)
2022-09-17 22:11:52 +08:00
Kerwin 8a12ae746d update PyDolphinScheduler documentation link. (#11474)
(cherry picked from commit c59a8503f4)
2022-09-17 22:08:45 +08:00
JieguangZhou 1f88c5a637 [bug][python] Fix task switch error when default branch not defined last (#11606)
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
(cherry picked from commit 635fde1981)
2022-09-17 22:08:28 +08:00
陈家名 59e745be0a [fix][python] Sql pattern add truncate. (#11666)
(cherry picked from commit 564a815d21)
2022-09-17 22:07:30 +08:00
fuchanghai 85b96ddc15 [improvement-#11630]fix document about common.properties (#11653)
(cherry picked from commit ce8db1c5fb)
2022-09-17 22:04:50 +08:00
旺阳 f41af79ef6 [doc] Correct E2E Doc, fix WorkerGroupPage typo (#11629)
(cherry picked from commit 6e4d974b91)
2022-09-17 22:04:32 +08:00
Eric Gao 11b78a871c Fix homepage email subscription link (#11622)
* Fix homepage email link bug

* Replace dev with latest

(cherry picked from commit 3e73829924)
2022-09-17 22:03:54 +08:00
Yann Ann 6797ae2d97 [DOC] should notice that need setting the full path when call the resource file #11620 (#11621)
(cherry picked from commit eda9f3d50c)
2022-09-17 22:02:53 +08:00
Eric Gao c8f0a66b37 [Doc][Bug] Fix resource center docs for standalone mode (#11600)
* Fix resource center docs for standalone mode

* Remove useless images

(cherry picked from commit ff4d2c9460)
2022-09-17 22:02:30 +08:00
labbomb 7723f5bcae feat: update slack (#11578)
(cherry picked from commit 5813a61f09)
2022-09-17 22:02:10 +08:00
monster 90cc1284a4 [Improvement-11550] [Doc]Document content update (#11577)
* [Improvement-11550] [Doc]Document content update

* Update pictures

* fix

* Update Process word to Workflow

Co-authored-by: David <davidzollo365@gmail.com>
(cherry picked from commit 3e360a8ecb)
2022-09-17 22:01:43 +08:00
Eric Gao 3d69e9edb1 [Doc][Security] Update instructions on worker groups (#11483)
* Update instructions on worker groups

(cherry picked from commit 9330d6cfcd)
2022-09-17 22:00:59 +08:00
Eric Gao 3447e6765c [Doc][DSIP] Move DSIP docs to the right place (#11442) (#11443)
(cherry picked from commit e3c6ace81d)
2022-09-17 22:00:27 +08:00
Eric Gao 059b66b64f [Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11435)
* [Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11427)

* Remove dead link in pyds README

* Add hyperlinks for docs

Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>

(cherry picked from commit 9d6fc92af9)
2022-09-17 22:00:05 +08:00
labbomb f7fb1acef9 feat: Modifying Slack Links (#11416)
* feat: Modifying Slack Links

* append

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>

(cherry picked from commit de4e4af844)
2022-09-17 21:57:34 +08:00
Jiajie Zhong 82deeab2fb [python] Add multiple version of document (#11391)
* Using sphinx-multiversion to support multiple version
of python api document
* change python api sidebar link
* fix whitelist_externals deprecated and add git

close: #8227

(cherry picked from commit 26afd655cb)
2022-09-17 21:54:29 +08:00
insist777 be37dd4886 [doc] Refine the deployment documentation (#11361)
(cherry picked from commit e9ccaffbe0)
2022-09-17 21:49:31 +08:00
阿福Chris b9cc2639d0 [Fix-11217] [Doc] add postgresql config in doc: datasource-setting (#11326)
* add postgresql config for datasoruce-setting

* Update docs/docs/en/guide/howto/datasource-setting.md

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit c76301ab51)
2022-09-17 21:47:55 +08:00
insist777 82a8956e7b [doc] Improve the document introduction of complement (#11275)
(cherry picked from commit 8ad6e10da5)
2022-09-17 21:47:00 +08:00
Wenjun Ruan 8fefb9ecfd
Add dependent task instance log (#11541) (#12014)
* Add dependent task instance log

* Optimize log

* Fix dependent task initialize failed will throw exception

(cherry picked from commit a41c6824fc)
2022-09-17 18:32:28 +08:00
Wenjun Ruan 84876b9f15
If the task processor is not found need to throw error rather than exception (#11461) (#12012)
(cherry picked from commit 4362fb8448)
2022-09-17 18:25:11 +08:00
Wenjun Ruan 41a6d09c4f
Fix workflow instance may failover many times due to doesn't check the restart time (#11445) (#12010)
(cherry picked from commit 0ca3086296)
2022-09-17 17:51:40 +08:00
Wenjun Ruan 0237654e96
Fix find last process instance in dependent task is not correct (#11424) (#12009)
(cherry picked from commit a2b5219315)
2022-09-17 17:50:56 +08:00
Wenjun Ruan a615e6ee3d
Fix quartz threadPriority config name error (#11596) (#12015)
* Fix quartz threadPriority config name error

* Add batchTriggerAcquisitionMaxCount config

(cherry picked from commit 8886d5a126)
2022-09-17 14:25:20 +08:00
Wenjun Ruan fc72f8bae1
[Fix-11413] Cannot set task status to kill if the task is not in running(#11414) (#12007)
(cherry picked from commit 496c2d4bfa)
2022-09-17 13:55:20 +08:00
Wenjun Ruan 36135d4151
Clear thread name in BaseTaskProcessor (#11422) (#12008)
(cherry picked from commit 47b50067ef)
2022-09-17 11:30:57 +08:00
Kerwin 901d01fb5f
cherry-pick [Bug](dependent) Dependent downstream trigger error when schedule cycle not day. (#11987)
Co-authored-by: Stalary <stalary@163.com>
2022-09-17 09:04:35 +08:00
Kerwin 08fc7daf49
[CI] Fix cluster test abnormal (#11688) (#11985)
* fix cluster test abnormal

Co-authored-by: xiangzihao <460888207@qq.com>
2022-09-16 19:27:34 +08:00
Kerwin f132b05ce3
[Improvement-11380][scp-host.sh] Set StrictHostKeyChecking=no option to ssh (#11382) (#11995)
* [Improvement][scp-host.sh] Set StrictHostKeyChecking=no option to ssh

* add StrictHostKeyChecking=no option to status-all.sh

Co-authored-by: rickchengx <38122586+rickchengx@users.noreply.github.com>
2022-09-16 19:25:46 +08:00
Kerwin c258cc0d3c
[DOC] improve zk digest doc (#11703) (#11992)
Co-authored-by: Evans Sun <abzymeinsjtu@gmail.com>
2022-09-16 19:23:43 +08:00
Kerwin 0680282d69
optimize the process instance query, change the date time range (#11719) (#11991)
Co-authored-by: juzimao <578961953@qq.com>
2022-09-16 19:22:49 +08:00
Kerwin 6824c72990
[Doc] Remove re-upload file function in the 3.0.0's doc (#11804) (#11984)
Co-authored-by: xiangzihao <460888207@qq.com>
2022-09-16 19:04:42 +08:00
Kerwin ea8b00fa89
[Fix-11051][Task]Fix the process exitValue exception when the process times out and th… (#11099) (#11983) 2022-09-16 19:03:01 +08:00
Kerwin 7c2d6b8e36
[fix-11404]: make the common.properties to configurable on values.yaml (#11441) (#11967)
and set the resource.storage.type default to hdfs

Co-authored-by: 爱穿格子裤 <30542109+yutianaiqingtian@users.noreply.github.com>
2022-09-16 18:57:57 +08:00
caishunfeng 3ea17820a7
[Cherry-pick-3.0.1][fix-11452][Bug] [Logger Server] Incorrect password regular expression (#11993)
Co-authored-by: jackfanwan <61672564+jackfanwan@users.noreply.github.com>
2022-09-16 15:26:16 +08:00
Kerwin bbafbc4749
[doc] Add how obtain IP address from network card in FAQ (#11311) (#11982)
This pull request update the FAQ doc (faq.md) by adding how to obtain the IP address
from the specified network card, which is part of relevant information created when 
the master service and worker service are registered with zookeeper.

Co-authored-by: Shuo Zhang <zhangshuocn@qq.com>
2022-09-16 14:47:50 +08:00
Kerwin d94797dfbc
Remove logger header in task log file (#11555) (#11968) 2022-09-16 13:03:22 +08:00
Kerwin 75f93b60ad
cherry-pick [Bug] [API] The task priority and process instance priority might be null and cause NPE (#11969) 2022-09-16 13:02:58 +08:00
fuchanghai 6fec9e2f08
[3.0.1 prepare] [cherry-pick ]fix doc about sub-process's child node describe (#11972) 2022-09-16 13:01:51 +08:00