Commit Graph

55 Commits

Author SHA1 Message Date
Wenjun Ruan 3bc88e32c7
Rename dolphinscheduler-aop to dolphinscheduler-yarn-aop (#16095) 2024-05-31 17:12:22 +08:00
Wenjun Ruan cb0b4fa52f
Deprecated 2.0.9/3.0.6 upgrade test (#16012) 2024-05-17 19:17:54 +08:00
Wenjun Ruan e2c8b080f9
[DSIP-31] Reduce the thread pool size of hikari (#15890) 2024-04-22 17:05:24 +08:00
Wenjun Ruan 285c5a8eb5
[DSIP-28] Donnot scan whole bean under classpath (#15874) 2024-04-19 18:12:40 +08:00
Wenjun Ruan 163f8f01f0
Fix jdbc registry cannot work (#15861) 2024-04-18 15:43:28 +08:00
Wenjun Ruan 27d0563fe4
Bind processId to constructor CodeGenerator (#15848) 2024-04-16 10:58:43 +08:00
Wenjun Ruan dcf69cad20
Fix PostgresqlDatabaseContainerProvider get Image is incorrect (#15434) 2024-01-19 10:29:46 +08:00
Wenjun Ruan 5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) (#15355) 2023-12-23 22:47:59 +08:00
Wenjun Ruan 9be81be328
Remove API Result in Service (#15181) 2023-11-20 17:04:03 +08:00
Wenjun Ruan a5284e4024
Remove mapper usage in tools (#15073) 2023-10-25 22:58:47 +08:00
Wenjun Ruan 2350775272
Add IT for mysql5/postgresql16 initialize/upgrade (#15063) 2023-10-24 14:44:02 +08:00
Wenjun Ruan b750db3c37
Add IT for dolphinscheduler-tools module (#15043) 2023-10-20 15:39:42 +08:00
Wenjun Ruan 020092ed1e
Set charset in upgrading tools to solve read upgrade sql mojibake (#14955) 2023-09-26 09:46:25 +08:00
Jay Chung 25e841a228
chore: Correct db upgrade and add ci avoid regression (#14620)
add atlasgo to ci
https://atlasgo.io/declarative/diff
related: https://github.com/apache/dolphinscheduler/issues/11286

BTW, it seems some ddl change only exist in 3.1.3、3.1.5、3.1.6 but do not exist in dev which will also cause an error when updating from those version
2023-08-30 11:55:05 +08:00
sleo 139f23040a
[Fix-14668][dolphinscheduler-tools] Modify the imported jar package. Use org.apache.commons.lang3.StringUtils. (#14669) 2023-08-02 11:03:49 +08:00
Kerwin 79cb707b57
[Improvement-13932][API] Tenant parameters and Worker group parameters are put together in the task startup parameters (#13935)
* [Improvement][API] Tenant parameters and Worker group parameters are put together in the task startup parameters
2023-04-18 14:56:38 +08:00
Bingchang Chen 75544025df
[Improvement] Fix typos in tools for demo (#13538) 2023-02-09 12:13:36 +08:00
Aaron Wang f3134c7ce7
[Feature][Resource Center] One-time migration script for unmanaged resources (#13512) 2023-02-08 10:20:08 +08:00
seedscoder 8d12dc0702
[Improvement-13491] Use lombok @Slf4j annotation to generate logger (#13509) 2023-02-07 20:32:53 +08:00
Wenjun Ruan ef47e7efeb
Use parallelStream to improve the performance of upgrade (#13442) 2023-01-23 13:59:20 +08:00
Wenjun Ruan 713fdbc444
Split the upgrade code (#13436)
1. Split the upgrade code into different version
2. Log the dml/ddl sql in origin friendly format
3. Fix ddl of 3.0.0 and 3.1.0
4. Fix search resource regular expression injection
2023-01-20 20:48:07 +08:00
Wenjun Ruan 483330273d
Remove entity in QueryWrapper constructor (#13396) 2023-01-14 20:57:03 +08:00
Wenjun Ruan 8a479927f3
Add projectCode in t_ds_process_instance and t_ds_task_instance to remove join (#13284) 2023-01-03 09:52:28 +08:00
Yann Ann 6ef74073cc
[Improve-13001]migrate commons-collections -> commons-collections4 (#13002) 2022-12-10 23:50:19 +08:00
amao b71d4983dd
[Feature-6586][Server]add some ds process definition demo when init (#11759)
* [Feature-6586][Server]add some ds process definition demo when init
1.add some ds process definition demo when init, to display what task type can run and make user easy to
use ds.
2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service
3.need modify the tenant information in it
2022-10-31 22:35:11 +08:00
Aaron Wang 08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds (#12197)
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.

Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
caishunfeng dedff70f90
[Improvement] Merge spi.utils into common.utils (#12502)
* improve JSONUtil

* merge spi.utils into common.utils

* split constants && remove some copy utils method

* remove StringUtils
2022-10-23 21:23:54 +08:00
Eric Gao d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports (#12412)
* Fix remaining style errors and wildcard imports

* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link

* Fix spotless matching scope of markdown files
2022-10-19 11:08:04 +08:00
kezhenxu94 4ea7eec750
Use temurin Java docker images instead of deprecated ones (#12334) 2022-10-17 15:46:21 +08:00
Daniel Y 009cb68011
[Improvement-11834] Upgrade docker base image to support python3.9 (#11835)
* use openjdk:8-jre-slim-bullseye instead of buster to support python3.9

* use openjdk:8-jre-slim-bullseye instead of buster across all components
2022-09-14 09:29:45 +08:00
caishunfeng 2e61c76c22
[Improvement] Add remote task model (#11767) 2022-09-08 13:41:36 +08:00
kezhenxu94 efedc9a53c
Bump up dependencies to fix cves (#11765) 2022-09-08 09:05:05 +08:00
kezhenxu94 0b82755796
Fix some resource leak (#11576) 2022-08-21 01:19:16 +08:00
Wenjun Ruan 5e9c7dad23
Add dolphinscheduler-bom to manage the dependency version (#11025) 2022-07-20 10:37:31 +08:00
Eric Gao 42d4aba347
[Bug][Tools] Add datasource url for mysql profile in tools application.yaml (#10399) 2022-06-14 20:20:54 +08:00
kezhenxu94 d80cf21456
Clean up unused dependencies and packaging issues (#9944) 2022-05-31 15:22:41 +08:00
caishunfeng 4351148f36
[Bug] init docker compose DATABASE env (#10244)
skip source env if in docker
2022-05-26 13:50:59 +08:00
旺阳 aba5f8a40e
[improve] Change Mysql Driver (#10220) 2022-05-25 14:09:15 +08:00
旺阳 de5507fb19
[Fix-10103][k8s]Fix k8s Change DataSource Error (#10128) 2022-05-24 13:59:42 +08:00
caishunfeng b1885c7a26
[Bug] fix run on docker and k8s (#10026)
* fix docker-compose init schema

* recovery depend on zk

* update doc and dockerfile

* fix run on k8s

* udpate doc

* add DOCKER flag & update doc

* remove repeat DOCKER env
2022-05-16 15:41:45 +08:00
xiangzihao baf654cb47
[Feature-9474] [CI] Add cluster test script verify on shell script (#9997)
* cluster test

* fix init db failed

* fix init db failed

* fix init db failed

* fix init db failed

* fix init db failed

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add sudo

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* add github actions

* remove cluster-test workflows

* add github actions

* add github actions

* refactor test to docker mode

* refactor test to docker mode

* refactor test to docker mode

* refactor test to docker mode

* remove create schema logic

* remove create schema logic

* remove create schema logic

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* fix runnint cluster test error

* add github actions

* add github actions

* add github actions

* add cluster test timeout

* add cluster start test loop check

* add cluster start test loop check

* optimize cluster start test loop check
2022-05-14 12:30:57 +08:00
Eric Gao 515c363148
[Feature][Deploy] Add missing default values in dolphinscheduler_env.sh (#9733)
* [Feature][Deploy] Add missing default values in dolphinscheduler_env.sh (#9675)

* [Feature][Deploy] Restore dolphinscheduler_env.sh to avoid e2e failure (#9675)

* [Feature][Deploy] Change mysql driver class name to com.mysql.cj.jdbc.Driver (#9675)

* [Feature][Deploy] Remove redundant config file (#9675)
2022-04-27 13:44:37 +08:00
Jiajie Zhong de50f43de6
[common] Make dolphinscheduler_env.sh work when start server (#9726)
* [common] Make dolphinscheduler_env.sh work

* Change dist tarball `dolphinscheduler_env.sh` location
  from `bin/` to `conf/`, which users could finish their
  change configuration operation in one single directory.
  and we only need to add `$DOLPHINSCHEDULER_HOME/conf`
  when we start our sever instead of adding both
  `$DOLPHINSCHEDULER_HOME/conf` and `$DOLPHINSCHEDULER_HOME/bin`
* Change the `start.sh`'s path of `dolphinscheduler_env.sh`
* Change the setting order of `dolphinscheduler_env.sh`
* `bin/env/dolphinscheduler_env.sh` will overwrite the `<server>/conf/dolphinscheduler_env.sh`
when start the server using `bin/dolphinsceduler_daemon.sh` or `bin/install.sh`
* Change the related docs
2022-04-25 15:35:43 +08:00
Eric Gao 072ba731a2
[Bug][Doc]Update database init instruction docs (#9659) 2022-04-22 23:56:00 +08:00
Eric Gao a863c6f8f1
[Bug][DAO] Update db init script and soft_version (#9628) (#9637) 2022-04-21 11:01:19 +08:00
xiangzihao ec939fcc68
[Bug] [Dev] Fix start/stop/status/init script error (#9514)
* change /bin/sh to /bin/bash

* change /bin/sh to /bin/bash

* remove declear -A to adapt mac
2022-04-16 21:34:53 +08:00
Eric Gao fd6b43bc81
[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223) (#9299) 2022-04-07 18:15:19 +08:00
calvin 7c5bebea98
[Fix-8544][API] The folder's size can't be updated when creating, updating or deleting a resource file int the folder. (#9107) 2022-03-23 18:58:41 +08:00
Kerwin a1bef43981
[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584)
* Task plug-in optimization, task-related classes are migrated to the task plugin module

* remove tasktype enum class.

* fix code style.

* fix code style

* fix code style

* fix ProcessServiceTest.testUpdateResourceInfo test class

* fix code style

* fix test task plugin manager check parameters null

* fix test ut

* update aws license

* change TaskTimeoutStrategy class
2022-03-15 15:16:36 +08:00
Kerwin 103ccb01da
change version to 2.0.4-SNAPSHOT (#8188) 2022-01-25 15:17:17 +08:00