Commit Graph

570 Commits

Author SHA1 Message Date
Kerwin 4938470d20
[Fix-14907]Fix click to view the history in the task node of the workflow instance (#14909) 2023-09-13 11:17:10 +08:00
sgw 76bbcbeb30
[Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent (#14824)
* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- delete useless output

* [Improvement][Master] Split the task dependency mode of the dependent node into workflow dependent and task dependent. (#11970)

- add log
- run spotless check

---------

Co-authored-by: 旺阳 <qingwli@cisco.com>
Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
2023-09-09 17:23:34 +08:00
Aaron Wang b12df72a0c
[Fix][Resource Center] Remove NONE storage type and set LOCAL type as default (#14661) 2023-09-07 14:09:15 +08:00
gaopeng adf49fd23f
[Feature-14802][api] Use Casdoor SSO to log in, add admin user configuration (#14814)
* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration

    Use Casdoor SSO to log in, add admin user configuration

This closes #14802

* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration

    Use Casdoor SSO to log in, add admin user configuration

This closes #14802

* update doc

* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration

    Use Casdoor SSO to log in, add admin user configuration

This closes #14802

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: 旺阳 <qingwli@cisco.com>
2023-09-04 13:25:40 +08:00
zhangbaipeng 7640c29495
[Improvement] [Task] Change Seatunnel task run-mode to deploy-mode (#14800) 2023-09-01 12:35:39 +08:00
Wenjun Ruan d223d654cc
Add rpc benchmark test (#14797) 2023-08-27 20:59:55 +08:00
yangyang zhong 70731a1617
[Feature][Authentication] support oauth2 login (#14743)
* support oauth2 login

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-08-21 15:57:30 +08:00
wangzheyuan 34d2f67052
Fix a typo in development-environment-setup.md (#14767) 2023-08-18 14:18:41 +08:00
Edison Catto 25480ae9e7
[Feature][Resource Center] Add support for Huawei Cloud OBS (#14643)
* [Feature][Resource Center] Add support for Huawei Cloud OBS as storage of resource center

* add license and add doc

* add 3-party dependency license

* Update LICENSE

* fix

* Update pom.xml

* fix

* fix

---------

Co-authored-by: sunkang <sunkang@hjfruit.com>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2023-08-14 13:25:48 +08:00
Kerwin 420bcd69a6
Add 3.1.8 release document (#14707) 2023-08-07 13:39:14 +08:00
eye-gu 4bae7b1ae2
[Fix-14683][pom] upgrade mysql driver version (#14684) 2023-08-04 10:51:15 +08:00
Gallardot 2f3ab18b29
[Doc] fix built-in typo (#14667)
Signed-off-by: Gallardot <gallardot@apache.org>
2023-08-01 08:56:08 +08:00
Wenjun Ruan 2b99451ccd
Support filter restrict network interface (#14638) 2023-07-26 23:03:58 +08:00
Wenjun Ruan 4aab0b234f
Use AdHoc datasource client in sqlTask (#14631)
* Use AdHoc datasource client in sqlTask

* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
2023-07-26 08:58:50 +08:00
Aaron Wang 759756ae80
[Improvement-14269][API] Bind task group with project (#14392)
* [Improvement-14269][API] Bind task group with project

* remove comment

* add project permission check for task group operation

* add doc
2023-07-23 12:11:37 +08:00
Wenjun Ruan cbd354d134
Add host/port in heartbeat (#14591) 2023-07-20 21:26:35 +08:00
Xinxing 1d0d85b988
[Feature] doris add datasource (#14391)
* Added doris data source plug-in

* Update DbType.java

* XML configuration

* doc and ui

* Missing catch of NumberFormatException

* code format

---------

Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-20 17:15:22 +08:00
Jay Chung de2d215aee
impv: Correct datax execute and python execute script name (#14523)
* impv: Correct datax execute and python execute script name

we use PYTHON_LAUNCHER for python script execute and
DATAX_LAUNCHER for datax script name

* Add pr number

* fix ut

* style

---------

Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-20 09:13:55 +08:00
Orange Summer af2986eca1
[Feature-14545][Master][UI]Cross workflow parameter passing (#14552)
* feat: cross-workflow parameter passing

* refactor: remove useless parameters

* refactor: remove useless code

* refactor: code format

* docs: update docs

* docs: docs format

* more effective information

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* more effective information

Co-authored-by: xiangzihao <zihaoxiang@apache.org>

* fix: wrong writing of log code

* fix: perfect error log

---------

Co-authored-by: xiangzihao <zihaoxiang@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2023-07-19 18:00:44 +08:00
旺阳 175d976cd2
[Feature][Data Quality] Data Quality Support Choose Database (#14406)
* add dataquality database api

* change ui

* api change

* update

* fix spotless

* fix h2

* fix pg

* fix-dead-line

* update

* fix-spotless

* update pg sql

* add ut

* fix ut
2023-07-12 09:00:28 +08:00
Wenjun Ruan 0246327083
Polish config, expose rpc config in application.yml (#14501) 2023-07-11 09:29:31 +08:00
Wenjun Ruan 406aeaf659
Use jvm_args_env.sh to set JVM args (#14487) 2023-07-10 17:39:57 +08:00
Rick Cheng 3de0b5a38c
[Feature-14339][Parameter] Add parameter combining of project-level parameters (#14452) 2023-07-10 10:17:46 +08:00
Aaron Wang 0b69236b04
[Feature-14251][Task] Support yarn queue definition in yarn task (#14310) 2023-07-07 14:24:47 +08:00
Aaron Wang 65b6a4b097
[Feature-14421][K8S Task] Configurable image pull policy (#14426) 2023-07-05 09:17:47 +08:00
Jay Chung 0f190ae3da
chore: Add new step skip for ui build and spotless (#14427) 2023-07-03 20:41:12 +08:00
QuakeWang 8f3d354c3c
fix issue#14375 (#14435)
add sqoop config in website
2023-07-03 20:36:45 +08:00
旺阳 f186cd5dab
[Feature][Api] LDAP SSL supports&& Add LDAP K8s configuration (#14349) 2023-06-29 16:14:37 +08:00
Eric Gao 3c04d5c4fa
[Bug][Doc] Fix dead link in api test docs (#14419) 2023-06-28 14:22:38 +08:00
Aaron Wang bcd1df19b8
[Improvement-14393][Resource Center] Remove deprecated resource description in api and UI page (#14394) 2023-06-28 14:13:20 +08:00
IT-Kwj 7a24be0672
[Feature-14404][datasource-plugin] add HANA datasource (#14404)
---------

Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
2023-06-28 10:56:41 +08:00
Rick Cheng 4fa0ab4d7e
[feature][Resource] Add support for Azure Blob Storage in the resource center (#14082) 2023-06-12 12:05:49 +08:00
Aaron Wang b7fb393eed
fix: trim node selector requirement value (#14307) 2023-06-09 18:06:52 +08:00
JieguangZhou e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters (#14127)
* add dynamic task plugin

* fix ui

* update code

* add DOC

* ADD UT

* fix doc

* fix sanity check

* add upgrade ddl
* fix sql
2023-06-07 16:01:06 +08:00
Zinway 18d176c5f4
feat: Introduce new datasource vertica (#14232)
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
2023-06-02 13:57:37 +08:00
Eric Gao e86630bb7b
[Improvement][Metrics] Add more worker related metrics and fix some previous ones (#14254)
* Add more worker related metrics and fix some previous ones

* update metrics docs
2023-06-01 16:54:48 +08:00
Rick Cheng 8aec3846b4
[Improvement-14243][Metrics] Add user tag of response time of api server (#14248) 2023-06-01 15:13:28 +08:00
Jay Chung 7226edd04e
chore: Release version 3.0.6 (#14234) 2023-05-30 10:36:58 +08:00
Aaron Wang 4e78248238
[Feature][K8S Task] support node selector (#14126)
* feat: support K8S node selector

* fix CI
2023-05-26 09:21:23 +08:00
Kerwin fa4a4f6938
Added 3.1.7 release document (#14206) 2023-05-25 18:08:04 +08:00
haibingtown af7da0630e
[Feature] Add project name/workflow definition name/instance id/task name to the built-in variables (#14099)
* [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables
2023-05-25 11:02:39 +08:00
Rick Cheng 5c4ba4105d
[Feature-14138][Metrics] Add metrics for api server (#14177) 2023-05-23 12:22:49 +08:00
Rick Cheng 428f1559a3
[Improv-14083][Resource Center] Handle deleteSource in OSS / S3 / GCS (#14084) 2023-05-22 20:44:43 +08:00
Alex Ting 606b56403e
[Fix-13381][plugin] fix error while parsing hive load sql (#13378)
* [fix][task plugin][task sql] fix parsing error while using variables sometimes parsing exception happened if task sql contains a variable followed by \" or \' .

for example:
input sql: load inpath '/tmp/test_table/dt=${dt}'
into table test_table partition(dt=${dt})

after replace: preparing : load inpath '/tmp/test_table/dt=?
into table test_table partition(dt=?)
2023-05-22 11:19:30 +08:00
Kerwin 8315279620
[Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes (#14150)
* Added a dependent task failure policy in the dependent task node
2023-05-19 14:49:50 +08:00
Jay Chung 56b0f91f6e
fix: Save task seatunnel error (#14129) 2023-05-19 10:47:16 +08:00
Rick Cheng db2da822de
Add supported operations of workflow definition (#14122) 2023-05-18 10:27:36 +08:00
Aaron Wang c3809fbecf
[Improvement-13868][K8S Task] Support the custom labels for k8s task (#14067)
* [Improvement-13868] Support the custom labels for k8s task

* fix front-end type mismatched
2023-05-16 12:04:33 +08:00
Kerwin 8944fdc622
Added 3.1.6 release document (#14100) 2023-05-15 11:25:45 +08:00
openopen2 fc97118bdc
[Doc] Fix upgrade-schema.sh cannot load driver class (#14072)
* fix upgrade-schema.sh cannot load driver class

When initializing and creating related tables for the first time, the support of related jdbc drivers is required. This article omits placing jdbc drivers in the tools directory.

* Update datasource-setting.md

update english doc
2023-05-10 13:44:23 +08:00