Commit Graph

571 Commits

Author SHA1 Message Date
dbac ef003c6211 Pre release change 2023-10-11 16:29:47 +08:00
Gallardot a8272b2606 [Doc]fix typo (#15005)
(cherry picked from commit 23b2c8a532)
2023-10-11 15:59:16 +08:00
Gallardot 7b43c66811 [Feature][Helm] Supports deploying only partial components (#14985)
* [Feature][Helm] Supports deploying only partial components

* chore: add doc

* chore: miss some doc

(cherry picked from commit 4fe41ac57f)
2023-10-11 15:58:46 +08:00
Gallardot a1a041082d [Improvement][K8S] K8S task support passing context parameters to downstream task (#14934)
* [Improvement][K8S] K8S task support passing context parameters to downstream task

Signed-off-by: Gallardot <gallardot@apache.org>

* [Improvement][K8S] K8S task support passing context parameters to downstream task

Signed-off-by: Gallardot <gallardot@apache.org>

* chore: update doc img

Signed-off-by: Gallardot <gallardot@apache.org>

* chore: update doc

Signed-off-by: Gallardot <gallardot@apache.org>

* chore: add UT

Signed-off-by: Gallardot <gallardot@apache.org>

* merge suggestion

Co-authored-by: 旺阳 <wang@lqwang.net>

* merge suggestion

Co-authored-by: 旺阳 <wang@lqwang.net>

* merge suggestion

Co-authored-by: 旺阳 <wang@lqwang.net>

---------

Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: David Zollo <dailidong66@gmail.com>
Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 049c1511fa)
2023-10-11 15:58:18 +08:00
Kerwin 3b3e2f1712 [Fix-14907]Fix click to view the history in the task node of the workflow instance (#14909)
(cherry picked from commit 4938470d20)
2023-10-11 15:48:15 +08:00
Aaron Wang fd00bb605a [Fix][Resource Center] Remove NONE storage type and set LOCAL type as default (#14661)
(cherry picked from commit b12df72a0c)
2023-10-11 15:42:29 +08:00
gaopeng fdc2421817 [Feature-14802][api] Use Casdoor SSO to log in, add admin user configuration (#14814)
* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration

    Use Casdoor SSO to log in, add admin user configuration

This closes #14802

* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration

    Use Casdoor SSO to log in, add admin user configuration

This closes #14802

* update doc

* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration

    Use Casdoor SSO to log in, add admin user configuration

This closes #14802

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: 旺阳 <qingwli@cisco.com>
(cherry picked from commit adf49fd23f)
2023-10-11 15:38:23 +08:00
yangyang zhong 99a43640b2 [Feature][Authentication] support oauth2 login (#14743)
* support oauth2 login

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit 70731a1617)
2023-08-30 17:10:12 +08:00
wangzheyuan cdc32b3dc3 Fix a typo in development-environment-setup.md (#14767)
(cherry picked from commit 34d2f67052)
2023-08-30 17:09:28 +08:00
Edison Catto 3e0569b4fa [Feature][Resource Center] Add support for Huawei Cloud OBS (#14643)
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center

* add license and add doc

* add 3-party dependency license

* Update LICENSE

* fix

* Update pom.xml

* fix

* fix

---------

Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
(cherry picked from commit 25480ae9e7)
2023-08-30 17:08:39 +08:00
Kerwin ad9d04c974 Add 3.1.8 release document (#14707)
(cherry picked from commit 420bcd69a6)
2023-08-30 17:05:42 +08:00
eye-gu 49a984b368 [Fix-14683][pom] upgrade mysql driver version (#14684)
(cherry picked from commit 4bae7b1ae2)
2023-08-30 17:05:30 +08:00
Gallardot 9af39ea893 [Doc] fix built-in typo (#14667)
Signed-off-by: Gallardot <gallardot@apache.org>
(cherry picked from commit 2f3ab18b29)
2023-08-30 17:02:34 +08:00
Wenjun Ruan 4398ab9136 Support filter restrict network interface (#14638)
(cherry picked from commit 2b99451ccd)
2023-08-30 17:00:19 +08:00
Wenjun Ruan 5f48b01591 Use AdHoc datasource client in sqlTask (#14631)
* Use AdHoc datasource client in sqlTask

* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient

(cherry picked from commit 4aab0b234f)
2023-08-30 16:59:38 +08:00
Aaron Wang f666fd34f9 [Improvement-14269][API] Bind task group with project (#14392)
* [Improvement-14269][API] Bind task group with project

* remove comment

* add project permission check for task group operation

* add doc

(cherry picked from commit 759756ae80)
2023-08-30 13:59:17 +08:00
Wenjun Ruan c047e2350c Add host/port in heartbeat (#14591)
(cherry picked from commit cbd354d134)
2023-08-30 13:56:10 +08:00
Xinxing b6313e1cc7 [Feature] doris add datasource (#14391)
* Added doris data source plug-in

* Update DbType.java

* XML configuration

* doc and ui

* Missing catch of NumberFormatException

* code format

---------

Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit 1d0d85b988)
2023-08-30 13:54:25 +08:00
Jay Chung e677a3816a impv: Correct datax execute and python execute script name (#14523)
* impv: Correct datax execute and python execute script name

we use PYTHON_LAUNCHER for python script execute and
DATAX_LAUNCHER for datax script name

* Add pr number

* fix ut

* style

---------

Co-authored-by: xiangzihao <460888207@qq.com>
(cherry picked from commit de2d215aee)
2023-07-20 11:30:46 +08:00
Orange Summer 8b922675c9 [Feature-14545][Master][UI]Cross workflow parameter passing (#14552)
* feat: cross-workflow parameter passing

* refactor: remove useless parameters

* refactor: remove useless code

* refactor: code format

* docs: update docs

* docs: docs format

* more effective information

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* more effective information

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* fix: wrong writing of log code

* fix: perfect error log

---------

Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
(cherry picked from commit af2986eca1)
2023-07-20 11:30:44 +08:00
旺阳 d414f2631b [Feature][Data Quality] Data Quality Support Choose Database (#14406)
* add dataquality database api

* change ui

* api change

* update

* fix spotless

* fix h2

* fix pg

* fix-dead-line

* update

* fix-spotless

* update pg sql

* add ut

* fix ut

(cherry picked from commit 175d976cd2)
2023-07-20 11:30:20 +08:00
Wenjun Ruan ded332c0e1 Polish config, expose rpc config in application.yml (#14501)
(cherry picked from commit 0246327083)
2023-07-20 11:30:07 +08:00
Wenjun Ruan e801e5dd57 Use jvm_args_env.sh to set JVM args (#14487)
(cherry picked from commit 406aeaf659)
2023-07-20 11:29:58 +08:00
Rick Cheng 0878578255 [Feature-14339][Parameter] Add parameter combining of project-level parameters (#14452)
(cherry picked from commit 3de0b5a38c)
2023-07-20 11:29:45 +08:00
Aaron Wang a99596a2df [Feature-14251][Task] Support yarn queue definition in yarn task (#14310)
(cherry picked from commit 0b69236b04)
2023-07-20 11:29:21 +08:00
Aaron Wang 09e0a11f21 [Feature-14421][K8S Task] Configurable image pull policy (#14426)
(cherry picked from commit 65b6a4b097)
2023-07-20 11:26:28 +08:00
Jay Chung da79a64316 chore: Add new step skip for ui build and spotless (#14427)
(cherry picked from commit 0f190ae3da)
2023-07-20 11:24:57 +08:00
QuakeWang 5d04238b13 fix issue#14375 (#14435)
add sqoop config in website

(cherry picked from commit 8f3d354c3c)
2023-07-20 11:24:54 +08:00
旺阳 28c2ec8e4b [Feature][Api] LDAP SSL supports&& Add LDAP K8s configuration (#14349)
(cherry picked from commit f186cd5dab)
2023-07-20 11:24:49 +08:00
Eric Gao 5d263b563f [Bug][Doc] Fix dead link in api test docs (#14419)
(cherry picked from commit 3c04d5c4fa)
2023-07-20 11:24:41 +08:00
Aaron Wang 63441d1222 [Improvement-14393][Resource Center] Remove deprecated resource description in api and UI page (#14394)
(cherry picked from commit bcd1df19b8)
2023-07-20 11:24:32 +08:00
IT-Kwj b0a7e15e3e [Feature-14404][datasource-plugin] add HANA datasource (#14404)
---------

Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
(cherry picked from commit 7a24be0672)
2023-07-20 11:24:09 +08:00
Rick Cheng 4be93325b0 [feature][Resource] Add support for Azure Blob Storage in the resource center (#14082)
(cherry picked from commit 4fa0ab4d7e)
2023-07-20 09:55:26 +08:00
Aaron Wang 1f8a77d79d fix: trim node selector requirement value (#14307)
(cherry picked from commit b7fb393eed)
2023-07-20 09:54:58 +08:00
JieguangZhou 1168fabf13 [Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters (#14127)
* add dynamic task plugin

* fix ui

* update code

* add DOC

* ADD UT

* fix doc

* fix sanity check

* add upgrade ddl
* fix sql

(cherry picked from commit e7cdc7c136)
2023-07-20 09:54:42 +08:00
Zinway cd486aa652 feat: Introduce new datasource vertica (#14232)
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
(cherry picked from commit 18d176c5f4)
2023-07-20 09:53:20 +08:00
Eric Gao 7a310f442a [Improvement][Metrics] Add more worker related metrics and fix some previous ones (#14254)
* Add more worker related metrics and fix some previous ones

* update metrics docs

(cherry picked from commit e86630bb7b)
2023-07-20 09:52:26 +08:00
Rick Cheng 7dc236730a [Improvement-14243][Metrics] Add user tag of response time of api server (#14248)
(cherry picked from commit 8aec3846b4)
2023-07-20 09:52:14 +08:00
Jay Chung 661522af56 chore: Release version 3.0.6 (#14234)
(cherry picked from commit 7226edd04e)
2023-07-20 09:51:48 +08:00
Aaron Wang 4e78248238
[Feature][K8S Task] support node selector (#14126)
* feat: support K8S node selector

* fix CI
2023-05-26 09:21:23 +08:00
Kerwin fa4a4f6938
Added 3.1.7 release document (#14206) 2023-05-25 18:08:04 +08:00
haibingtown af7da0630e
[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables (#14099)
* [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables
2023-05-25 11:02:39 +08:00
Rick Cheng 5c4ba4105d
[Feature-14138][Metrics] Add metrics for api server (#14177) 2023-05-23 12:22:49 +08:00
Rick Cheng 428f1559a3
[Improv-14083][Resource Center] Handle deleteSource in OSS / S3 / GCS (#14084) 2023-05-22 20:44:43 +08:00
Alex Ting 606b56403e
[Fix-13381][plugin] fix error while parsing hive load sql (#13378)
* [fix][task plugin][task sql] fix parsing error while using variables sometimes parsing exception happened if task sql contains a variable followed by \" or \' .

for example:
input sql: load inpath '/tmp/test_table/dt=${dt}'
into table test_table partition(dt=${dt})

after replace: preparing : load inpath '/tmp/test_table/dt=?
into table test_table partition(dt=?)
2023-05-22 11:19:30 +08:00
Kerwin 8315279620
[Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes (#14150)
* Added a dependent task failure policy in the dependent task node
2023-05-19 14:49:50 +08:00
Jay Chung 56b0f91f6e
fix: Save task seatunnel error (#14129) 2023-05-19 10:47:16 +08:00
Rick Cheng db2da822de
Add supported operations of workflow definition (#14122) 2023-05-18 10:27:36 +08:00
Aaron Wang c3809fbecf
[Improvement-13868][K8S Task] Support the custom labels for k8s task (#14067)
* [Improvement-13868] Support the custom labels for k8s task

* fix front-end type mismatched
2023-05-16 12:04:33 +08:00
Kerwin 8944fdc622
Added 3.1.6 release document (#14100) 2023-05-15 11:25:45 +08:00