natas_hw
/
dolphinscheduler
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,221 Commits 27 Branches 54 Tags 416 MiB
Commit Graph

106 Commits

Author SHA1 Message Date
Jay Chung 8a4f111fd4 [maven-release-plugin] prepare release 3.2.1 2024-02-07 11:09:56 +08:00
Jay Chung d5c543f467
Revert some incompatible change for 3.2.1 (#15580) 
* Revert "Fix k8sTaskExecutionContext setting configYaml (#15116)"

This reverts commit ce11674668.

* Revert "[Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977)"

This reverts commit c532fea00e.
 2024-02-07 10:41:04 +08:00
Jay Chung 7a1be3f329 [maven-release-plugin] prepare for next development iteration 2024-02-07 00:15:26 +08:00
Jay Chung ba5ec7375c [maven-release-plugin] prepare release 3.2.1 2024-02-07 00:15:25 +08:00
Jay Chung e2d08fc27d fix: data quality may fail in docker mode (#15563) 
(cherry picked from commit 91d56f4860)
 2024-02-06 18:24:33 +08:00
Jay Chung acb1e0a09f fix: data quality can not use (#15551) 
* fix: data quality can not use

fix: #15468, #15249, #14858

---------

Co-authored-by: Rick Cheng <rickchengx@gmail.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
(cherry picked from commit 8efaa9fa1f)
 2024-02-06 18:22:51 +08:00
AliceXiaoLu cb5317f7ca
[bug] Exception when using host in ipv6 format (#14040) 
* DB2 edit again, no username.

* [bug] db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"

* Update Db2DataSourceProcessor.java

* Update Db2DataSourceProcessorTest.java

* 【Bug】OracleSource edit again, ConnectionType is null

* when user inputs ipv6,checkhost will return "!IPV4_PATTERN.matcher(host).matches()";
I think we should use isInetAddress to check the host is isValid

* Update AbstractDataSourceProcessor.java

* Update DatabendDataSourceProcessorTest.java

last week other modified this

* Update MySQLDataSourceProcessorTest.java

* Update DataSourceUtilsTest.java

* Update MySQLDataSourceProcessorTest.java

* Update KyuubiDataSourceProcessorTest.java

* Update AbstractDataSourceProcessor.java

if host is ipv6 or ipv4 will OK; if not check other

* Update AbstractDataSourceProcessor.java

* Update AbstractDataSourceProcessor.java

* Update DataSourceUtilsTest.java

增加ipv6数据源验证

* [Bug] [dolphinscheduler-api] queryDataSource'password should be hidden.

---------

Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
 2024-01-15 09:33:08 +08:00
lgcareer 309c8c97a9
[Enhancement][API]Enhance mysql connection properties (#15433) 2024-01-06 19:25:08 +08:00
Wenjun Ruan bf2cdc5300
Use Druid to split sql (#15367) 2024-01-02 12:00:53 +08:00
Wenjun Ruan 921253109a
[Bug] Close SSH session after remote shell finish (#15348) 
* Close SSH session after remote shell finish

---------

Co-authored-by: David Zollo <davidzollo365@gmail.com>
 2023-12-25 13:04:45 +08:00
Wenjun Ruan b73194bd35
Fix check value rather than key in AbstractDataSourceProcessor#checkOther (#15351) 2023-12-25 10:00:07 +08:00
xujiaqiang 0feb144180
[Improvement-15260][dolphinscheduler-datasource-hana] add hana related dependencies (#15260) 
* add hana related dependencies

* optimizing HANA data source bugs

* run mvn spotless:apply

* Fix hana datasource getValidationQuery()

* Fix hana datasource testGetJdbcUrl()

---------

Co-authored-by: xujiaqiang <“xujiaqiangwz@163.com”>
Co-authored-by: David Zollo <dailidong66@gmail.com>
 2023-12-12 17:11:27 +08:00
Xinxing 5e135ba041
fix: execute sql error: datasource plugin 'doris' is not found。 (#15123) 
Co-authored-by: xiangzihao <460888207@qq.com>
 2023-11-07 14:37:07 +08:00
izualzhy 8665951981
[fix-#11726] fix error when set connection proerty both in the URL and an argument (#15093) 
Co-authored-by: zhangying14 <zhangying14@zuoyebang.com>
Co-authored-by: xiangzihao <460888207@qq.com>
 2023-11-03 14:44:27 +08:00
chenrj c532fea00e
[Improvement] Refactoring K8S task plugin with connections managed in connection center (#14977) 2023-10-31 17:07:24 +08:00
chenrj 5a3827eef4
[DSIP-19] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks (#14976) 
* Refactoring `Sagemaker` task plugin with connections managed in connection center.

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2023-10-31 10:07:05 +08:00
chenrj e99c5950b9
[DSIP-19] Support zeppelin connections in the connection center, as well as external connections to the connection center in zeppelin tasks (#14434) 
* Refactoring zeppelin task plugin with connections managed in connection center

---------

Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
 2023-10-30 16:44:54 +08:00
Wenjun Ruan bc4326125d
Fix session/thread leak at ssh datasource (#14859) 2023-09-07 11:55:31 +08:00
Wenjun Ruan 9ce8871158
Support get property from env/jvm/common.properties (#14645) 2023-07-28 14:41:58 +08:00
Wenjun Ruan 4aab0b234f
Use AdHoc datasource client in sqlTask (#14631) 
* Use AdHoc datasource client in sqlTask

* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
 2023-07-26 08:58:50 +08:00
Wenjun Ruan 6617e3fb72
Revert "datasource test and sql task Remove connection pool issues is #14179 (#14193)" (#14626) 
This reverts commit e4fb5b30a4.
 2023-07-24 17:33:43 +08:00
xuhui1231 e4fb5b30a4
datasource test and sql task Remove connection pool issues is #14179 (#14193) 
* datasource test and sql task Remove connection pool issues is #14179

* datasource test and sql task Remove connection pool issues is #14179 uniform style

* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720

* datasource test and sql task Remove connection pool issues is #14179 uniform style by 20230720

---------

Co-authored-by: xuhui <xuhui@3h1china.com>
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2023-07-21 16:14:21 +08:00
Xinxing 1d0d85b988
[Feature] doris add datasource (#14391) 
* Added doris data source plug-in

* Update DbType.java

* XML configuration

* doc and ui

* Missing catch of NumberFormatException

* code format

---------

Co-authored-by: xiangzihao <460888207@qq.com>
 2023-07-20 17:15:22 +08:00
Rick Cheng 68a3a75211
[Improvement-14457][UT] Improve the expected and actual value of Assertions (#14459) 2023-07-06 13:12:48 +08:00
IT-Kwj 7a24be0672
[Feature-14404][datasource-plugin] add HANA datasource (#14404) 
---------

Co-authored-by: WeiJian Kong <>
Co-authored-by: xujiaqiang <Xjq634828243>
 2023-06-28 10:56:41 +08:00
Jay Chung 221df4a278
revert: Datasource need test binding to create prod (#14381) 
We should keep our step as less as possible, for now, we have to
test datasource and binding it to the prod datasource, I think it will
make our users do not thing to use datasource, so I do like to revert
it and keep step as less as possible

related to https://github.com/apache/dolphinscheduler/pull/11670
 2023-06-27 10:59:17 +08:00
Zinway 18d176c5f4
feat: Introduce new datasource vertica (#14232) 
Co-authored-by: Zinway Liu <zinway.liu@ucloud.cn>
 2023-06-02 13:57:37 +08:00
He Zhao f1e78627fc
[feature][plugin] Add snowflake datasource in datasource plugin (#13729) 
---------

Co-authored-by: weitong <weitong@cisco.com>
 2023-05-06 19:30:50 +08:00
Jeremy db6c061c8b
[Feature][plugin] Add databend datasource in datasource plugin (#13866) 
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2023-04-24 21:28:14 +08:00
AliceXiaoLu dcf0109db3
【Bug】 OracleSouce edited again, ConnectionType is null (#13881) 2023-04-19 15:57:29 +08:00
He Zhao c0126b7f30
[Feature][plugin] Support Kyuubi datasource (#13642) 
---------

Co-authored-by: weitong <weitong@cisco.com>
Co-authored-by: Rick Cheng <38122586+rickchengx@users.noreply.github.com>
 2023-04-18 11:12:39 +08:00
JieguangZhou 273a5e78eb
[FEATURE][Task Plugin]Add remote-shell task plugin (#13801) 2023-03-30 20:17:28 +08:00
AliceXiaoLu ad7ce2db56
fix: DB2 username can not be save (#13764) 
db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"
 2023-03-30 18:59:21 +08:00
Jrebel-i 3630b00c9f
Fix Bug [Athena Datasource] datasource other params illegal (#13032) 2023-03-30 16:15:29 +08:00
He Wang 35d8e8ac17
[Feature][DataSource] add oceanbase datasource (#13675) 
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
 2023-03-07 10:12:58 +08:00
seedscoder 8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger (#13509) 2023-02-07 20:32:53 +08:00
Wenjun Ruan b11431db95
Use single thread to refresh kerberos (#13456) 2023-02-01 20:41:05 +08:00
Tq 3a6e10397f
[feature][api-datasource]add redshift auth mode to get connection by access key (#13379) 
* submit redshift auth mode

* add front end

* add doc and fix frontend

Co-authored-by: devosend <devosend@gmail.com>
 2023-01-16 16:21:26 +08:00
lenian bcf03ad4d1
[Feature-9467] add DAMENG DataSource (#12860) 
* [Feature-9467][DataSource] add DAMENG （达梦） data source

* [Feature-9467] add DM datasource doc (Issue #9467)

* [Feature-9467] add DM IDataBaseOptionKeys

* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method

* [Feature-9467] modification package dm to dameng;

* [Feature-9467] add Dameng DmJdbcDriver18 License

* [Feature-9467] add DM DataSource: remove DruidDataSourceClient

Co-authored-by: lenian <yuenian@mananacare.com>
 2023-01-04 15:00:24 +08:00
Wenjun Ruan 14ec4a2398
Remove dao module in worker (#13242) 2022-12-22 12:25:29 +08:00
Wenjun Ruan b2336b0ce9
Remove datasource in procedure task (#13198) 2022-12-19 17:14:59 +08:00
Tq f0dd4f081f
[Feature][task-plugins-12820] azure azure-sql datasource plugins (#13209) 
* submit azure sql code

* change mssql-jdbc version

* merge changes to delete other

* merge changes to delete other

* add dependencies
 2022-12-19 10:27:03 +08:00
Wenjun Ruan 5c75d312fa
Add StarRocks datasource (#13204) 2022-12-17 08:44:39 +08:00
Wenjun Ruan ab96a3d0c0
Fix datasource jdbc url param doesn't work (#13098) 2022-12-12 11:46:40 +08:00
insist777 b40c63f02d
[Feature][DataSource]Add trino datasource plugin (#12911) 2022-11-23 10:09:02 +08:00
simsicon 43d69ae157
Upgrade clickhouse jdbc driver (#12639) 2022-11-11 13:38:58 +08:00
caishunfeng dedff70f90
[Improvement] Merge spi.utils into common.utils (#12502) 
* improve JSONUtil

* merge spi.utils into common.utils

* split constants && remove some copy utils method

* remove StringUtils
 2022-10-23 21:23:54 +08:00
Eric Gao d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports (#12412) 
* Fix remaining style errors and wildcard imports

* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link

* Fix spotless matching scope of markdown files
 2022-10-19 11:08:04 +08:00
Eric Gao 2f37da0dbc
[Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom (#12311) 
* move mockito dependencies from sub-modules to root pom

* Add check in CI to block the usage of powermock
 2022-10-14 12:06:51 +08:00
Eric Gao ac845046e7
[Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in datasource, registry and data-quality modules (#12352) 
* Migrate all UT cases from jUnit 4 to 5 in datasource, registry and data-quality modules

* Fix etcd registry test
 2022-10-14 08:08:20 +08:00