Commit Graph

6990 Commits

Author SHA1 Message Date
rickchengx d9516e2236
add lombok config (#11212)
Co-authored-by: rick <rick@B-V6XRMD6M-1956.local>
2022-08-02 11:46:58 +08:00
Wenjun Ruan 04f3aa9713
Fix recovery from failed task will dead loop (#11239) 2022-08-02 09:30:18 +08:00
Wenjun Ruan caa6287d7f
Add alertPluginInstanceId in AlertInfo (#11231) 2022-08-01 17:32:26 +08:00
microeastcowboy 9a53c6ac34
fix work oom when task logs's size is very large (#11224) 2022-08-01 15:28:06 +08:00
Amy0104 0141aa511b
[Fix][UI] Fix datax task data instance replay error. (#11222) 2022-08-01 14:56:19 +08:00
Wenjun Ruan 8e21c38c00
Write alert result into db (#11221) 2022-08-01 13:04:27 +08:00
阿福Chris 01fc6c8a3a
[doc] Fix installation typo (#11220)
Co-authored-by: xiangzihao <460888207@qq.com>
2022-08-01 10:03:53 +08:00
fuchanghai cb001b581a
[Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle (#11213)
* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors

* [fix-10729]fix the code smell

* [fix-10729]decorate DataSourceControllerTest with public

* [fix-10729]replace import *

* [fix-10729]checkstyle

* [Improvement][ExecutorServiceImpl] remove some unnecessary

* [fix-11181]text check and word spell

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* Update ExecutorServiceImpl.java

remove the error logging and update the comment

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle

* [Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle

* [Improvement-10760][dolphinscheduler-api] add import

* [Improvement-10760][dolphinscheduler-api] remove not use

Co-authored-by: fuchanghai <changhai.fu@marketingforce.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-08-01 09:24:02 +08:00
songjianet 163df5f1a3
[Refactor][UI] Refactor task group option using NSpace component. (#11209) 2022-07-31 14:54:18 +08:00
JinYong Li e5cca0e79b
[Fix-11007] [Master] fix forced_success bug (#11088)
* fix forced_success bug

* add comments

* add transactional

* refactor code

Co-authored-by: JinyLeeChina <jiny.li@foxmail.com>
2022-07-30 23:28:31 +08:00
xuhhui bfff3a7c5d
fix error (#11206) 2022-07-30 18:20:20 +08:00
fuchanghai d73ad2e7d5
[BUG-11181] [Document] text check (#11183) 2022-07-29 23:49:46 +08:00
Chris Ho d85ea8f62f
[Improvement][UI] Unified local parameters UI (#11190) 2022-07-29 17:49:33 +08:00
Devosend 91a944f967
fix nodeSize is not a function bug (#11200) 2022-07-29 17:39:45 +08:00
Jiajie Zhong 93a73b1292
[ci] Make cluster test work on release branch (#11199)
When I try to release the 3.0.0 on branch `3.0.0-prepare`,
I find out cluster-test error due to version change of
tarball, I think we should make cluster test work on both
dev and prepare release branch, so I add this patch
2022-07-29 14:49:00 +08:00
Eric Gao 4b43d62356
[Improvement][Metrics] Upgrade plugin versions in grafana demo dashboards (#11192) (#11193) 2022-07-29 07:04:04 +08:00
Jiajie Zhong 9e692f919a
[python] Add directly install without clone source code (#11179)
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-07-28 21:40:20 +08:00
xuhhui 513f336015
[Feature] Add new task type chunjun (#10937) 2022-07-28 19:25:29 +08:00
Jiajie Zhong 7a766cbcf2
[python] Refactor structure to avoid cycle import (#11167)
Currently, our core and side module dependent on each others. and will cause
cycle import in our codebase, especially in issue #10905, we try to refactor
to solve this problem.

This patch do the following change:

* Rename module `side` to `models`
* Move `core/base` and `core/sidebase` to dir `modules`
* Move `configuration` and `default_config.yaml` to the root of pydolphinscheduler
2022-07-28 19:02:13 +08:00
Tq 9859b5da1f
add Feishu alert doc and configure email alert doc to the docsdev.js (#11173)
add Feishu alert doc and configure email alert doc to the docsdev.js
2022-07-28 17:21:32 +08:00
caishunfeng 818648df7d
[Bug-11071] fix local params (#11184)
* fix local params

* add CI case
2022-07-28 15:07:04 +08:00
pinkhello 2e81f30298
[Fix] Parameter values disappear when editing Zeppelin task (#11115) 2022-07-28 15:03:26 +08:00
JiPeng Wang cd0c50f3f2
[Fix-11140] Added resource parameters for function verification (#11141)
* Added resource parameters for function verification

* checkstyle fixed

* checkstyle fixed

* check style fixed for env test

* rerun

* rerun
2022-07-28 14:03:12 +08:00
Wenjun Ruan 052ceaacd0
Add is success in loop task status (#11169) 2022-07-28 12:12:01 +08:00
fuchanghai 81fa09678c
[Improvement][ExecutorServiceImpl] remove some unnecessary (#11122) 2022-07-28 09:57:19 +08:00
SongTao Zhuang a52d1b4acd
[11130] Refactor alert-api with lombok (#11133) 2022-07-28 08:31:44 +08:00
Sheldon efd68f4805
[Fix-10665] [S3] Fix s3 download method (#10675)
when overwrite param is true and dest file is exist, throw exception
make dirs for ancestor directories of the source file path if it is not exists
To describe the error more clearly, change "the file isn`t exists" to specific error message

Co-authored-by: sheldonliu <sheldonliu>
2022-07-27 21:25:13 +08:00
caishunfeng 6c7e00c4aa
[Bug-11101] fix task failover NPE (#11168) 2022-07-27 19:21:21 +08:00
Lyle Shaw 181cd2ff79
[Feature] Add CURD to the project/tenant/user section of the API and add its tests. (#10913) 2022-07-27 14:33:33 +08:00
zhuxt2015 cc2f17ddcc
[Enhancement][API] Copy workflow with scheduler info (#10677)
* support batch copy workflow with scheduler
2022-07-27 13:45:45 +08:00
Jiajie Zhong de5c22b276
[doc] Add new page incompatible change (#11153)
Add incompatibles and give some hint to user when then
want to upgrade to specific version.

ref: #10607
2022-07-27 11:15:09 +08:00
caishunfeng b03f2e22cd
fix k8s master and worker start error (#11156) 2022-07-27 09:23:17 +08:00
xuhhui 8b50578bdb
[doc] Fix bugs in doc installation, design and glossary (#11084) 2022-07-26 18:08:19 +08:00
Assert cdfe115247
[Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter (#10960)
* [Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter

* checkstyle
2022-07-26 17:41:13 +08:00
xuhhui 8004e97b4b
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController (#10832)
* [Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController
2022-07-26 15:24:48 +08:00
zhuxt2015 5ebd53893c
[Feature-10604] Task name in the copied and imported workflow without copy suffix (#10607)
* fix

* remove imported task suffix
2022-07-26 15:21:33 +08:00
Wenjun Ruan c7789bf049
Add base loop task execotor and http template parser (#11137)
* Add base loop task execotor and http template parser

* Add JsonPathUtils
2022-07-26 14:51:50 +08:00
qinchaofeng c28774e5dd
[Doc]Chinese translation updated (#11146)
* Update python.md Chinese translations

* Update zeppelin.md Chinese translations
2022-07-26 14:33:38 +08:00
Eric Gao 0e15ce3389
[Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#11128)
* [Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#10867)

* Update docs and grafana demo dashboards
2022-07-26 13:44:02 +08:00
Devosend d76d6da734
[Fix][UI] Add duplicate of file suffix (#11136) 2022-07-26 11:25:33 +08:00
zhuxt2015 4d9db3451e
[Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612) 2022-07-26 10:14:45 +08:00
songjianet fbfaca3dbd
[Refactor][UI] Refactor resource manage using NSpace component. (#11098)
* [Refactor][UI] Refactor resource manage using NSpace component.

* [Refactor][UI] Refactor resource manage using NSpace component.
2022-07-26 10:07:00 +08:00
liyangyang 6f0a609793
[Improvement 11072][common] Added HTTPS request support in kerberos environment (#11127)
* [Improvement-11072][dolphinscheduler-common] Added HTTPS request support in kerberos environment

* [Improvement-11072][dolphinscheduler-common] supplement

Co-authored-by: liyangyang <liyangyang@bizseer.com>
2022-07-25 23:21:42 +08:00
songjianet 7ed1e7cd74
[Refactor][UI] Refactor function manage using NSpace component. (#11124) 2022-07-25 22:22:04 +08:00
Devosend 011040f90e
[Fix][UI] fix bug where The Gantt chart does not show the execution status (#11135)
* fix bug where The Gantt chart does not show the execution status of all components

* modify round method
2022-07-25 15:53:57 +08:00
JieguangZhou 764a24384f
[python] add import SageMaker task (#11121) 2022-07-25 15:46:36 +08:00
liyangyang f13e7a95ae
[Improvement-11013][dolphinscheduler-common] YarnHAAdminUtils#getActiveRMName function Add HTTPS Hadoop environment support (#11017)
* [Improvement-11013][dolphinscheduler-common] YarnHAAdminUtils#getActiveRMName function Add HTTPS Hadoop environment support

* [Improvement-11013][dolphinscheduler-common] add test code

* [Improvement-11013][dolphinscheduler-common] code format

* [Improvement-11013][dolphinscheduler-common] add hadooputils test code

* [Improvement-11013][dolphinscheduler-common] remove extra blank lines

* [Improvement-11013][dolphinscheduler-common] adjusts test code

* [Improvement-11013][dolphinscheduler-common] remove useless import

* [Improvement-11013][dolphinscheduler-common] add test code

* [Improvement-11013][dolphinscheduler-common] shortened string

* [Improvement-11013][dolphinscheduler-common]  format test code style

Co-authored-by: liyangyang <liyangyang@bizseer.com>
2022-07-25 11:07:40 +08:00
jackfanwan ccb7641468
修改数据质量查询的权限判断 (#10977)
Co-authored-by: fanwanlong <fanwanlong@kezaihui.com>
2022-07-25 09:35:53 +08:00
Wenjun Ruan 71cf7e1baa
[Fix-10918] Close datasource when expire from guava cache (#11120)
* Close datasource when expire from guava cache
* Remove duplicate datasource in HiveDataSourceClient
2022-07-23 20:43:37 +08:00
JieguangZhou 952aee80e7
[fix] py sagemaker doc (#11109) 2022-07-22 22:57:50 +08:00