Commit Graph

21 Commits

Author SHA1 Message Date
Wenjun Ruan 32a8241ef2
Don't source env file in task, this will override tenant's environment (#13028) 2022-11-28 21:09:50 +08:00
Assert 2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis (#12693) 2022-11-17 10:14:36 +08:00
Aaron Wang 08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds (#12197)
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.

Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
kezhenxu94 4ea7eec750
Use temurin Java docker images instead of deprecated ones (#12334) 2022-10-17 15:46:21 +08:00
rickchengx 08a4c7981f
[improve] Remove the spark version of spark task (#11860) 2022-09-21 20:27:28 +08:00
kezhenxu94 b73a9511a7
Remove sensitive configs from dolphinscheduler_env.sh (#11917) 2022-09-15 19:03:47 +08:00
xuhhui 513f336015
[Feature] Add new task type chunjun (#10937) 2022-07-28 19:25:29 +08:00
Kerwin bd1be6cf36
[Task] Added Apache SeaTunnel 2.x task node (#10480) 2022-07-05 19:21:30 +08:00
旺阳 87d9ec1dee
[doc] Delete SPRING_DATASOURCE_DRIVER_CLASS_NAME in doc (#10228) 2022-05-25 11:45:16 +08:00
Jiajie Zhong de50f43de6
[common] Make dolphinscheduler_env.sh work when start server (#9726)
* [common] Make dolphinscheduler_env.sh work

* Change dist tarball `dolphinscheduler_env.sh` location
  from `bin/` to `conf/`, which users could finish their
  change configuration operation in one single directory.
  and we only need to add `$DOLPHINSCHEDULER_HOME/conf`
  when we start our sever instead of adding both
  `$DOLPHINSCHEDULER_HOME/conf` and `$DOLPHINSCHEDULER_HOME/bin`
* Change the `start.sh`'s path of `dolphinscheduler_env.sh`
* Change the setting order of `dolphinscheduler_env.sh`
* `bin/env/dolphinscheduler_env.sh` will overwrite the `<server>/conf/dolphinscheduler_env.sh`
when start the server using `bin/dolphinsceduler_daemon.sh` or `bin/install.sh`
* Change the related docs
2022-04-25 15:35:43 +08:00
caishunfeng 84f2cfec39
[Feature][Server] Custom timezone - Add timezone handler unified (#8477)
* date convert of timezone

* remove @JsonFormat

* add unit test

* fix time preview in scheduler

* optimization & add env config

Co-authored-by: caishunfeng <534328519@qq.com>
2022-02-24 18:20:34 +08:00
wind 1038e6bf24
[deploy] Add datasource driver name into env (#8100)
Co-authored-by: caishunfeng <534328519@qq.com>
2022-01-18 17:22:24 +08:00
kezhenxu94 2ff76c7e73
Fix some SQL issues and make docker-compose runnable (#7329) 2021-12-10 23:51:42 +08:00
kezhenxu94 67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
kezhenxu94 5400abc74f
Refactor registry plugin and simplify its usage (#6712) 2021-11-07 15:18:10 +08:00
Shiwen Cheng 1cd62b4a5e
[Improvement][Datax] Re-assign DATAX_HOME to /opt/soft/datax instead of /opt/soft/datax/bin/datax.py (#5019) 2021-03-17 14:38:04 +08:00
游侠 2d8c4ec86b
[Improvement][Worker-Task] Python Command (#5036)
* create the interpreter for python command

* Update README.md

Co-authored-by: zhouhong <zhouhong@zhangyue.com>
2021-03-15 10:22:13 +08:00
Shiwen Cheng cbada7c16d
[Improvement][Docker] Reduce docker image size, remove frontend/nginx, remove stdout log, improve compose/swarm deployment and fix random PSQLException (#4643)
* [Improvement][Docker] Reduce docker image size, remove frontend/nginx, remove stdout log and improve compose/swarm deployment

* [Improvement][Docker] Fix random PSQLException
2021-02-14 17:42:04 +08:00
xingchun-chen 22d4ee942c
modify dataX environment variable (#2577)
* add LoginTest license

* Delete useless packages

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* Update worker_group_id to worker_group in init.sql

* Update worker_group_id to worker_group in init.sql

* Update worker_group_id to worker_group

* Increase dataX environment variable, sslTrust default value

* modify dataX environment variable

Co-authored-by: chenxingchun <438044805@qq.com>
2020-04-29 18:29:50 +08:00
xingchun-chen 7a100086b7
Increase dataX environment variable, sslTrust default value (#2555)
* add LoginTest license

* Delete useless packages

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* e2e add project and workflow case

* Update worker_group_id to worker_group in init.sql

* Update worker_group_id to worker_group in init.sql

* Update worker_group_id to worker_group

* Increase dataX environment variable, sslTrust default value

Co-authored-by: chenxingchun <438044805@qq.com>
2020-04-28 17:57:30 +08:00
bao liang e0e0696b4f add out put log when master/worker server start (#1769)
* merge hadoop.properties into common.properties

* merge hadoop,zookeeper.properties into common.properties
remove combined.properties/master.properties/worker.properties

* change db user/pwd to test/test

* rename .dolphinscheduler_en.sh to dolphinscheduler_env.sh
remove some unused in install.sh

* add out put log when master/worker server start...

* add start log when servers start
2020-01-09 10:36:15 +08:00