22958682c4
add 3.1.3 release ( #13357 )
2023-01-10 17:17:51 +08:00
8f7b141fe7
[Fix-13287][doc]Fix Chinese workflow-definition.md duplication description in execution-type ( #13361 )
2023-01-10 11:33:42 +08:00
bcf03ad4d1
[Feature-9467] add DAMENG DataSource ( #12860 )
...
* [Feature-9467][DataSource] add DAMENG （达梦） data source
* [Feature-9467] add DM datasource doc (Issue #9467 )
* [Feature-9467] add DM IDataBaseOptionKeys
* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method
* [Feature-9467] modification package dm to dameng;
* [Feature-9467] add Dameng DmJdbcDriver18 License
* [Feature-9467] add DM DataSource: remove DruidDataSourceClient
Co-authored-by: lenian <yuenian@mananacare.com>
2023-01-04 15:00:24 +08:00
2c952139b4
feat: Release version 304 ( #13319 )
...
Release new version 304
2023-01-03 15:17:22 +08:00
8503ee0eed
[Improvement][Worker] Startup parameter should have the highest priority ( #13274 )
2022-12-29 11:05:00 +08:00
aec26f06db
[doc] Correct install_env.sh file name ( #13294 )
...
duplicate #12209
2022-12-28 17:09:57 +08:00
9199049180
[Improvement][docker] Add SPRING_JACKSON_TIME_ZONE to .env file ( #13271 )
...
* tz in docker
* update default tz and update doc
Co-authored-by: chenjiaming <chenjiaming@kezaihui.com>
2022-12-27 14:41:34 +08:00
1f7f900e78
[Fix][Doc] Replace the links. ( #13276 )
...
* [Fix][Doc] Replace the links.
* [Fix][Doc] Revised `.dlc.json`
* [Fix] Fix the dead link.
2022-12-26 14:32:40 +08:00
06b77b34c7
[doc][datasource-12820] add azure sql database doc ( #13254 )
2022-12-22 16:04:19 +08:00
14ec4a2398
Remove dao module in worker ( #13242 )
2022-12-22 12:25:29 +08:00
c213c6fe34
Fix a tiny bug in release doc ( #13239 )
2022-12-21 17:52:56 +08:00
66e20271ad
[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks ( #13194 )
...
* Supports task instance cache operation
* add task plugin cache
* use SHA-256 to generate key
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Optimizing database Scripts
* Optimize clear cache operation
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2022-12-18 18:17:09 +08:00
5c75d312fa
Add StarRocks datasource ( #13204 )
2022-12-17 08:44:39 +08:00
84496c5f04
[Fix][Doc]Update pseudo-cluster.md ( #13207 )
...
* Update pseudo-cluster.md
* Update pseudo-cluster.md
2022-12-16 19:58:43 +08:00
dcf02eab45
[Fix] Replaced dead links in the docs. ( #13170 )
...
* [Fix] Replaced dead links.
* [Improvement] Format markdowns.
2022-12-13 14:36:53 +08:00
560d232cbd
[Bug][Worker] Fix duplicated key bug when have out parameters of same name ( #12930 )
2022-12-12 11:07:23 +08:00
209af2ad29
Add release 3.1.2 documentation ( #13150 )
2022-12-11 14:26:52 +08:00
917df4f6ca
[fix-13041] fix example checkFormula ( #13110 )
...
Co-authored-by: zhangshunmin <zhangshunmin@kezaihui.com>
2022-12-10 12:23:44 +08:00
316496211a
[doc] refactor release ( #13135 )
...
* [doc] refactor release
* [doc] refactor release
2022-12-10 12:20:53 +08:00
e90cd9e13b
[release] version 3.0.3 ( #13131 )
2022-12-08 16:57:17 +08:00
e180e16981
[Fix] Fix Java path in Kubernetes Helm Chart ( #12987 )
2022-11-24 15:35:03 +08:00
38b876733c
[Feature-10498] Mask the password in the log of sqoop task ( #11589 )
2022-11-24 14:54:54 +08:00
b40c63f02d
[Feature][DataSource]Add trino datasource plugin ( #12911 )
2022-11-23 10:09:02 +08:00
6e224d08a2
[fix] Python task can not pass the parameters to downstream task. ( #12961 )
2022-11-22 16:11:45 +08:00
2f8f0952fb
[Bug-12954] [Schedule] Fix that workflow-level configuration information does not take effect when timing triggers execution ( #12955 )
2022-11-22 15:35:29 +08:00
27c00ed377
[Document] Fix grammar and spelling ( #12940 )
2022-11-21 20:58:03 +08:00
5672337156
[docs] Update new release for 3.0.2 ( #12949 )
2022-11-21 13:50:51 +08:00
27c37b8828
Fix grammar and spelling ( #12937 )
2022-11-18 23:03:34 +08:00
d803658abd
[Document] Fix spelling ( #12935 )
2022-11-18 16:01:20 +08:00
22532fe1d8
[Document] Update faq, fix spelling, remove repeated words ( #12934 )
2022-11-18 15:36:47 +08:00
7c90bf01bc
[Improvement][Task Plugin] Improvement Kubeflow task plugin ( #12928 )
...
add example
check phase in status:conditions
2022-11-17 15:59:44 +08:00
2dbc79693e
[feature][task plugin] Add new task plugin for apache linkis ( #12693 )
2022-11-17 10:14:36 +08:00
9e6f4d2bc1
[Improvement][Document]Update architecture and process-start-flow pic ( #12912 )
...
Co-authored-by: xuhaihui <xuhaihui@cmss.chinamobile.com>
2022-11-16 16:03:00 +08:00
0373e06615
[Feature-8030][docs] Add sqoop task doc ( #12855 )
...
* [Feature-8030][docs] Add sqoop task doc
* Update docs/docs/zh/guide/task/sqoop.md
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
* Update docs/docs/en/guide/task/sqoop.md
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
* [Feature-8030][docs] Add sqoop task doc
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-11-13 00:09:05 +08:00
229c554912
[feature][task] Add Kubeflow task plugin for MLOps scenario ( #12843 )
2022-11-11 16:08:38 +08:00
019e7475d1
The new Chinese document latest version is 3.1.1 ( #12869 )
2022-11-11 16:06:20 +08:00
a887303d48
add 3.1.1 release doc ( #12864 )
2022-11-11 15:39:47 +08:00
454f1303a0
Add minio as demo storage for helm charts ( #12861 )
2022-11-11 10:26:57 +08:00
6896fa9b47
[fix][doc] Dead link for python api ( #12848 )
...
* [fix][doc] Dead link for python api
ref: apache/dolphinscheduler#12779 and
apache/dolphinscheduler-sdk-python#1
* fix dead link
2022-11-10 14:49:11 +08:00
775b3f87b4
[doc] Add skip spotless check during ASF release. ( #12835 )
2022-11-10 14:26:24 +08:00
1347a8f94c
[chore] Separate Python API into another repository ( #12779 )
...
Currently, our Python API code is a module in apache/dolphinscheduler codebase,
each time users change Python API code, they need to run all requests CI check
for dolphinscheduler and Python API, But if the user does only change Python
code, it could be merged if Python API CI pass and do not dependent on others CI.
Besides, we release Python API as the same version of dolphinscheduler. It is
easy for user to match Python API version. But when Python API does not change
any code, but dolphinscheduler release a bugfix version, Python API has to
release the new version to match dolphinscheduler. This happened when we
released Python API 2.0.6 and 2.0.7. 2.0.6 and 2.0.7 is bugfix version, and
Python API does not change any code, so the PyPI package is the same.
Separate Python API also makes our code more sense, we will have more
distinguished code in dolphinscheduler and Python API new repository.
Have separate issue tracker and changelog for information to users.
ref PR in other repository: apache/dolphinscheduler-sdk-python#1
see more detail in mail thread: https://lists.apache.org/thread/4z7l5l54c4d81smjlk1n8nq380p9f0oo
2022-11-09 21:20:15 +08:00
9bba4b105c
[fix][doc] Fix sql-hive and hive-cli doc ( #12765 )
2022-11-07 17:39:02 +08:00
6d79414491
Add instructions on how to configure logging level for debugging purpose ( #12699 )
2022-11-07 10:30:56 +08:00
a47b4d0672
[Feature][Task] Transfer files between tasks ( #12552 )
...
* add data transfer between tasks
* add delete DATA_TRANSFER API
* convert Result to DeleteDataTransferResponse
* add api UT
* fix final
* fix doc
2022-11-03 16:33:39 +08:00
4e2701b57e
[doc][fix] Add new alert doc Slack ( #12567 )
2022-11-01 09:49:05 +08:00
b71d4983dd
[Feature-6586][Server]add some ds process definition demo when init ( #11759 )
...
* [Feature-6586][Server]add some ds process definition demo when init
1.add some ds process definition demo when init, to display what task type can run and make user easy to
use ds.
2.need configure the JVM parameters (-Ddemo=true) to turn on the StandaloneServer service
3.need modify the tenant information in it
2022-10-31 22:35:11 +08:00
3c31ddfd9f
[Improvement-12536][k8s] Support the command for the container in k8s task plugin ( #12538 )
2022-10-31 19:06:41 +08:00
08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2022-10-31 16:52:53 +08:00
e6da1ccf81
Add worker-group-refresh-interval in master config ( #12601 )
...
* Add worker-group-refresh-interval in master config
* Set interval cannot smaller than 10s
* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/config/MasterConfig.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-10-31 09:37:26 +08:00
064fec88b0
[Feature-12040][docs] Update Granted Permissions Guide for PR#12048 ( #12587 )
...
* Upload new ui screenshots.
* Update document of Granted Permissions.
* Delete unused images.
2022-10-30 17:28:30 +08:00
207b4e77d3
Fix document styles. ( #12600 )
2022-10-29 11:31:19 +08:00
4a13148bfc
[Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group. ( #12534 )
...
* [Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group.
Co-authored-by: houshitao <shitaohou@163.com>
2022-10-27 16:34:08 +08:00
21caec01db
fix doc CI ( #12560 )
2022-10-27 12:41:30 +08:00
63244c8065
Optimized complement documentation ( #12550 )
2022-10-27 09:31:12 +08:00
76a9e6d8a6
Add task executor threads full policy config in worker ( #12510 )
2022-10-26 10:29:22 +08:00
8add9744d4
[doc][feat] New release 2.0.7 ( #12507 )
2022-10-24 11:38:17 +08:00
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2022-10-19 11:08:04 +08:00
ba538067f2
[fix] Change the default unix shell executor from sh to bash ( #12180 )
...
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2022-10-18 12:57:37 +08:00
1d0d26a416
[fix#12170] If the worker's clock early than master node, will ignore dispatch time. ( #12219 )
...
* add doc to guide user configure synchronized clocks.
2022-10-17 10:42:51 +08:00
7aa8a77fb2
[Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @Ignore to @Disable
2022-10-16 12:49:11 +08:00
a725614429
[docs] change optional parameters --jar to --jars for Spark ( #12385 ) ( #12386 )
2022-10-16 12:40:17 +08:00
b953cdc559
update english oracle.md ( #12332 )
2022-10-14 11:39:06 +08:00
c50cc997a8
[Doc][Bug] Fix dead link caused by markdown cross-files anchor ( #12357 )
2022-10-14 10:57:11 +08:00
501119bbb6
[doc] Change release process base on new tool ( #12324 )
...
ref: #11478 we already have a new tool for releasing
2022-10-12 09:52:14 +08:00
a973054950
[ci] Milestone and Label are required before the PR merged ( #12222 )
...
To make our release process easier, I plan to mark milestone
and label is required before the PR merged, they use for
* label: In this path we required label with at least on with
'feature|bug|improvement|document|chore' to make our changelog
more convenient
* milestone: For easier to cherry-pick PR, currently some of
our milestone only exists in issue and not exists in related
PR, it may
2022-10-10 15:20:51 +08:00
ef328d9cea
Update ZooKeeper minimum version requirements ( #12284 ) ( #12288 )
2022-10-10 15:08:07 +08:00
c8bd106604
create oracle help.md ( #12218 )
...
* create oracle help.md
* Add files via upload
allow Oracle to choose schema
* move pic under img and add title for oracle.md
2022-10-10 13:53:20 +08:00
9b1ee1516f
[doc] Correct descriptions in glossary.md ( #12282 )
2022-10-10 09:37:15 +08:00
c7435a51ac
[Feature][task-plugins] aws datasync task plugins ( #11865 )
...
* submit datasync task
* add aws software package license
2022-10-09 10:47:01 +08:00
61457d1c2f
[Hotfix][Docs] Fix 404 dead links ( #12261 )
2022-10-08 19:43:35 +08:00
7bdbd62985
[doc] Update slack invitation link ( #12258 )
2022-10-08 11:59:53 +08:00
7beff83e9f
[Doc] Add chmod command after unzip DS tar.gz ( #12210 )
2022-10-07 13:07:36 +08:00
6fc209ab9d
[fix][doc] Correct release export key step ( #12228 )
...
We should only add KEY to release/KEYS instead of
dev/KEYS, according to
https://lists.apache.org/thread/2gppxc04n0r87vq4qy4j3n52zwhqd442
2022-10-01 21:58:59 +08:00
40090d459e
[Improvement] remove log-server and server module ( #12206 )
...
* remove log-server module
* remove dolphinscheduler-server module
2022-09-30 16:41:41 +08:00
aacabc6ec7
Remove workerGroup in registry ( #12217 )
2022-09-30 13:58:53 +08:00
a055cf5791
[Release][Doc] add 3.1.0 release ( #12227 )
...
* [Release] add 3.1.0 release to doc
2022-09-30 12:14:29 +08:00
308f632fcc
[Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) ( #11868 )
...
* add aws dms
2022-09-30 10:16:16 +08:00
d3f087e29b
[feat][ci] Add pre-commit for easy spotless config ( #12192 )
...
* Use pre-commit to handle spotless check
* Migrate exists python pre-commit-config into dir root one
* Change related document
2022-09-29 10:10:53 +08:00
995c1480f9
[Document][Improvement] Polish docs for standalone deployment ( #12181 )
...
* Polish docs for standalone deployment
2022-09-28 20:32:22 +08:00
4973b30efb
[Feature][doc] Add a guide to nominate new committer ( #12175 )
...
Co-authored-by: Kirs <ACM_China@yeah.net>
2022-09-28 14:23:09 +08:00
b95db44130
[fix][doc] Do not change docsdev.js during releasing ( #12151 )
...
ref: https://github.com/apache/dolphinscheduler/pull/12146#discussion_r979708629
and https://github.com/apache/dolphinscheduler-website/pull/829
2022-09-26 21:46:18 +08:00
4269fa7ea7
[release][doc] Add release 3.0.1 ( #12139 )
...
* add release 3.0.1
* add 30x version in the github bug report
2022-09-26 16:18:31 +08:00
779a0b9c75
[Improvement-#11608][task-plugin] New time variables are added to facilitate business development ( #11667 )
...
Add new time expressions for users' convenience
Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
Co-authored-by: David <dailidong66@gmail.com>
2022-09-26 16:01:32 +08:00
e6832220c3
[Improvement][SQL Task]use default sql segment separator ( #10869 )
2022-09-26 11:08:57 +08:00
064224696f
[Doc][Improvement] Add instructions for process execution type and data complement ( #12121 )
...
* Add instructions for process execution type and data complement
* Fix and polish the docs
2022-09-24 09:59:02 +08:00
08a4c7981f
[improve] Remove the spark version of spark task ( #11860 )
2022-09-21 20:27:28 +08:00
cf522e2fc0
[improve] Optimize MLFlow task plugin for easy of use ( #12071 )
...
optimize code
2022-09-21 10:45:05 +08:00
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center ( #11708 )
...
* Add support for Alibaba Cloud OSS as storage of resource center
* Add dependency licenses
* Refactor code, add unit tests and related docs
2022-09-20 22:44:06 +08:00
8cddb10773
[Feat][UI] Update package. ( #12046 )
...
* [Feat][UI] Update package.
* [Feat][UI] Update package.
2022-09-19 20:58:41 +08:00
89d55bc9b7
Fix link errors in release documentation. ( #12049 )
2022-09-19 20:01:23 +08:00
a44817fc46
[Doc][Style] Fix doc format once for all ( #12006 )
2022-09-17 11:33:32 +08:00
5b6bebf7d5
[fix][doc] Update the registry related configuration in values.yaml ( #11444 )
2022-09-16 10:38:27 +08:00
86291bb5e6
[Feature-10273][Datasource] Add dataSource for AWS Athena ( #11157 )
2022-09-16 09:59:20 +08:00
b73a9511a7
Remove sensitive configs from dolphinscheduler_env.sh ( #11917 )
2022-09-15 19:03:47 +08:00
a88cb02ee8
[improve][Doc] Put default task parameters in a new file ( #11776 )
...
* Put default task parameters in a new file
* Add links to task plugin default parameters appendix docs
* Update Chinese docs
2022-09-15 10:32:10 +08:00
0deafba12c
fix procedure task param pass ( #11919 )
2022-09-14 13:44:01 +08:00
892d867270
[Doc][Task Plugin] Fix MLflow task plugin doc ( #11905 )
...
* update mlflow doc
* fix punctuations
2022-09-13 16:20:03 +08:00
070b8c2546
[fix-#11830]fix doc about sub-process's child node describe ( #11837 )
...
* [fix-#11830]fix doc
* Update docs/docs/en/guide/task/sub-process.md
2022-09-13 10:05:59 +08:00
105773f54d
Remove htrace dependencies ( #11881 )
...
* Remove htrace dependencies
* Update pom.xml
2022-09-10 22:54:38 +08:00
5d9331f4ee
[Feature-10683][Task Plugin] Add Java Task Plugin. ( #10689 )
2022-09-09 10:37:24 +08:00
efedc9a53c
Bump up dependencies to fix cves ( #11765 )
2022-09-08 09:05:05 +08:00
f0db90f71d
[Doc] Remove re-upload file function in the 3.0.0's doc ( #11804 )
2022-09-07 09:45:51 +08:00
c76301ab51
[Fix-11217] [Doc] add postgresql config in doc: datasource-setting ( #11326 )
...
* add postgresql config for datasoruce-setting
* Update docs/docs/en/guide/howto/datasource-setting.md
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-09-05 10:54:01 +08:00
31ba49ac02
Update doc according to recent update of Spring / Swagger ( #11755 )
2022-09-02 19:06:15 +08:00
277f137358
Add Kubernetes configmap reload to all components ( #11730 )
2022-09-02 12:03:19 +08:00
5fabce783a
[DOC] improve zk digest doc ( #11703 )
2022-08-30 21:07:04 +08:00
4a37345436
[Feature][Task Plugin] Support hive cli task plugin ( #11651 )
...
* Add hive cli task plugin
* Add docs for hive-cli task plugin
* Fix hive-cli task related front-end nits and checkstyle
* Fix exception handling logic
2022-08-30 10:45:51 +08:00
6e4d974b91
[doc] Correct E2E Doc, fix WorkerGroupPage typo ( #11629 )
2022-08-28 17:44:06 +08:00
eda9f3d50c
[DOC] should notice that need setting the full path when call the resource file #11620 ( #11621 )
2022-08-26 10:58:56 +08:00
ce8db1c5fb
[improvement-#11630]fix document about common.properties ( #11653 )
2022-08-26 10:50:31 +08:00
b96d69701a
[Improvement-#11613] Add spi priority factory ( #11614 )
...
* Add spi priority factory
* Add doc
* Add override log
* Use lombok
* Add comment
2022-08-24 19:51:47 +08:00
3e73829924
Fix homepage email subscription link ( #11622 )
...
* Fix homepage email link bug
* Replace dev with latest
2022-08-24 09:25:23 +08:00
ff4d2c9460
[Doc][Bug] Fix resource center docs for standalone mode ( #11600 )
...
* Fix resource center docs for standalone mode
* Remove useless images
2022-08-23 09:46:01 +08:00
5813a61f09
feat: update slack ( #11578 )
2022-08-22 11:03:50 +08:00
3e360a8ecb
[Improvement-11550] [Doc]Document content update ( #11577 )
...
* [Improvement-11550] [Doc]Document content update
* Update pictures
* fix
* Update Process word to Workflow
Co-authored-by: David <davidzollo365@gmail.com>
2022-08-22 00:00:42 +08:00
277c78010f
Enable spotless to fix markdown checkstyle and refactor faq.md as an example ( #11384 )
2022-08-18 16:48:22 +08:00
a974ba74ab
[Feature] New task plugin Pytorch ( #11498 )
...
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-08-18 10:20:37 +08:00
8ec2f79248
[DS-11481][docs] Add log specification document to contribution guidelines. ( #11484 )
2022-08-17 21:57:02 +08:00
e5d06c40a5
[Feature-10182][task-datax] ( #11387 )
...
In the process of realizing datax synchronization, use custom json for better connection to enable kerberos-authenticated clusters, such as using plug-ins such as hdfs and hbase
2022-08-17 10:01:20 +08:00
9330d6cfcd
[Doc][Security] Update instructions on worker groups ( #11483 )
...
* Update instructions on worker groups
2022-08-15 17:44:00 +08:00
a4bdadbb84
[doc] Use Rainbond deployment DolphinScheduler Cluster ( #10951 )
...
* [doc] Use Rainbond deployment DolphinScheduler Cluster
Signed-off-by: Qi Zhang <smallqi1@163.com>
* Update docs/docs/en/guide/installation/rainbond.md
* Update docs/docs/en/guide/installation/rainbond.md
* Update docs/docs/zh/guide/installation/rainbond.md
* Update docs/docs/zh/guide/installation/rainbond.md
* [doc] modify rainbond doc and add image
Signed-off-by: Qi Zhang <smallqi1@163.com>
* fix: img link
Signed-off-by: Qi Zhang <smallqi1@163.com>
* fix: modify proposal
Signed-off-by: Qi Zhang <smallqi1@163.com>
* docs: move rainbond docs to integration
Signed-off-by: Qi Zhang <smallqi1@163.com>
Signed-off-by: Qi Zhang <smallqi1@163.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-08-15 14:33:37 +08:00
7ff34c3947
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost ( #11368 )
...
* Add waiting strategy to support master/worker can recover from registry lost
* throw exception when zookeeper registry start failed due to interrupted
2022-08-13 09:52:03 +08:00
e3c6ace81d
[Doc][DSIP] Move DSIP docs to the right place ( #11442 ) ( #11443 )
2022-08-12 11:20:12 +08:00
9d6fc92af9
[Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local ( #11435 )
...
* [Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11427 )
* Remove dead link in pyds README
* Add hyperlinks for docs
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
2022-08-12 00:18:43 +08:00
de4e4af844
feat: Modifying Slack Links ( #11416 )
...
* feat: Modifying Slack Links
* append
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-08-11 13:11:23 +08:00
26afd655cb
[python] Add multiple version of document ( #11391 )
...
* Using sphinx-multiversion to support multiple version
of python api document
* change python api sidebar link
* fix whitelist_externals deprecated and add git
close : #8227
2022-08-11 12:02:29 +08:00
50380e008f
[doc] Add how obtain IP address from network card in FAQ ( #11311 )
...
This pull request update the FAQ doc (faq.md) by adding how to obtain the IP address
from the specified network card, which is part of relevant information created when
the master service and worker service are registered with zookeeper.
2022-08-11 09:49:47 +08:00
278371bcb9
[doc] Recover traffic configuration ( #11288 )
...
some config about traffic is deleted by
accident in #11113 this patch recover it
2022-08-11 09:46:43 +08:00
0464123c2b
[Feature-11223] support stream task ( #11350 )
...
* add task execute type
* update task definition list paging
* update task instance list paging
* stream task start
* [Feature][UI] Some changes to execute task.
* Set the connection edge to dashed line.
* Add FLINK_STREAM task.
* add stream task
* flink savepoint and cancel
* fix query bug
* add stream task definition
* add task instance for stream task
* delete stream task definition state
* update api for stream task definition edit
* modify search for stream task instance
* add language
* delete task type search for stream task definition
* change task type search for stream task instance
* add jump button
* add savepoint
* add down log for stream task instance
* ui test
* stream task start
* run DAG
* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.
* [Feature][UI] Remove some fields for FLINK_STREAM.
* add start modal
* add dryRun column for stream task instance
* fix duration
* fix pon
* fix build error
* Add success tip
* add auto sync for stream task instance
* remove forgien key for task instance
* license header
* UT fix
* modify locales
* recover common config
* fix UT
* add doc
Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2022-08-10 21:44:43 +08:00
e9ccaffbe0
[doc] Refine the deployment documentation ( #11361 )
2022-08-09 19:35:49 +08:00
8ad6e10da5
[doc] Improve the document introduction of complement ( #11275 )
2022-08-09 19:35:25 +08:00
c4fdfc7a55
[doc] Fixed hadoop link to avoid dead link ( #11363 )
...
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CommandsManual.html#jar
point to the latest docs and will be changed more frequent than fixed link
https://hadoop.apache.org/docs/r3.2.4/hadoop-project-dist/hadoop-common/CommandsManual.html#jar .
So I change to the fixed link
2022-08-09 13:34:49 +08:00
2865b17ed4
[release] New release version 3.0.0 ( #11356 )
2022-08-09 09:47:15 +08:00
983f9b41bf
[Doc] Add tools/libs to the init database steps ( #11345 )
2022-08-08 21:53:23 +08:00
d16c63528f
[community] Remove stackoverflow ref link in website ( #11342 )
...
* Our community do not have official support for stack overflow any more
* Change the order side for Slack and mailing list, to avoid some users
just want to ask question but join mailing list, and then unsubscribe
found there are too many emails in mailing list
2022-08-08 11:24:09 +08:00
3aa9f2ea25
[Bug][Doc] Fix typos in docs which causes links to upgrade section to fail ( #11333 )
...
* [Bug][Doc] Fix typo in docs
* Remove dead links and update related docs
2022-08-06 20:50:48 +08:00
6a02870926
[Feature][style] Add spotless maven plugin for automatic style fix. ( #11272 )
...
* [Feature][style] Add spotless maven plugin for automatic style fix (#10963 )
* Fix spotless ratchet configuration
* Remove license-check and decrease line length threshold value
* Update related docs
* Remove checkstyle and add pre-commit hook
* Test updated pre-commit hook
* Replace checkstyle with spotless in CI
* Remove reviewdog
2022-08-06 19:29:23 +08:00
bf5f7a88c5
[doc] migrate community and development docs into main repo ( #11313 )
2022-08-05 17:58:39 +08:00
39186b1a6d
[Doc] update the configuration doc ( #11113 )
2022-08-04 11:55:05 +08:00
dc3ab6af11
[doc] Add notice when environment can not select ( #11276 )
...
ref: #11249
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2022-08-04 11:08:11 +08:00
9ca1eb96c4
[Improvement][Metrics] Add metrics for alert server ( #11240 )
...
* [Improvement][Metrics] Add metrics for alert server (#11131 )
* Update related docs of metrics
* Add grafana demo dashboards for alert server metrics
* Refactor metric classes with UtilityClass annotation
* Refactor meter names in camelCase for checkstyle
2022-08-03 15:42:06 +08:00
01fc6c8a3a
[doc] Fix installation typo ( #11220 )
...
Co-authored-by: xiangzihao <460888207@qq.com>
2022-08-01 10:03:53 +08:00
d73ad2e7d5
[BUG-11181] [Document] text check ( #11183 )
2022-07-29 23:49:46 +08:00
513f336015
[Feature] Add new task type chunjun ( #10937 )
2022-07-28 19:25:29 +08:00
9859b5da1f
add Feishu alert doc and configure email alert doc to the docsdev.js ( #11173 )
...
add Feishu alert doc and configure email alert doc to the docsdev.js
2022-07-28 17:21:32 +08:00
de5c22b276
[doc] Add new page incompatible change ( #11153 )
...
Add incompatibles and give some hint to user when then
want to upgrade to specific version.
ref: #10607
2022-07-27 11:15:09 +08:00
8b50578bdb
[doc] Fix bugs in doc installation, design and glossary ( #11084 )
2022-07-26 18:08:19 +08:00
c28774e5dd
[Doc]Chinese translation updated ( #11146 )
...
* Update python.md Chinese translations
* Update zeppelin.md Chinese translations
2022-07-26 14:33:38 +08:00
0e15ce3389
[Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics ( #11128 )
...
* [Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#10867 )
* Update docs and grafana demo dashboards
2022-07-26 13:44:02 +08:00
952aee80e7
[fix] py sagemaker doc ( #11109 )
2022-07-22 22:57:50 +08:00
5b429d29b8
add oracle dependency ( #11095 )
2022-07-22 09:49:20 +08:00
af7b359333
[Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario ( #10826 ) ( #10935 )
...
* add sagemaker pipeline task plugin
[DOC] add Sagemaker task plugin document
change license and fix dependencies
* [fix] Optimize the code
* Update dolphinscheduler-task-plugin/dolphinscheduler-task-sagemaker/src/main/java/org/apache/dolphinscheduler/plugin/task/sagemaker/SagemakerParameters.java
Optimization parameter judgment
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
* [fix] nips
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-07-21 18:16:58 +08:00
f689220290
[Feature][Task Plugin] Increase zeppelin task stability in production ( #11010 )
...
* [Feature][Task Plugin] Increase zeppelin task stability in production (#10584 )
* Add front-end and update docs for the production mode of zeppelin task
* Fix minor front-end bug of zeppelin task plugin
* Refactor ZeppelinParameters with lombok
* Fix formatting
* Replace @Data with @Getter, @Setter, @ToString to avoid decrease in test coverage
2022-07-20 14:16:47 +08:00
a38fa34d43
[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks ( #10925 )
...
* [Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#9814 )
2022-07-15 09:26:28 +08:00
08cfbc73e0
[doc] Add missing doc change by snehal ( #10892 )
2022-07-12 14:16:00 +08:00
2f7281c2d2
[Feature][Metrics] Add resource download related metrics for workers ( #10749 )
...
* [Feature][Metrics] Add resource download related metrics for workers (#9324 )
* [Feature][Metrics] Fix bugs and add grafana demos for worker resource download metrics (#9324 )
* [Feature][Metrics] Add docs to resource related metrics (#9324 )
* [Feature][Metrics] Use tags to indicate status in metrics (#9324 )
* [Feature][Metrics] Fix demos, docs and remove redundant code (#9324 )
* [Feature][Metrics] Remove .pnpm-debug.log (#9324 )
* [Feature][Metrics] Fix style check (#9324 )
* [Feature][Metrics] Replace KB with bytes for the unit of resource file size in metrics (#9324 )
* [Feature][Metrics] Make code neat (#9324 )
2022-07-12 11:44:34 +08:00
8eaf5a2309
[Feature-10219][EMR] EMR supports use <add-Steps> to add steps to an existing cluster ( #10657 )
...
* Add the ProgramType parameter to distinguish task types
* EmrAddStepsTask supports Add-Steps
* UI supports Add-Steps
* EmrTask modify the name of the class to EmrJobFlowTask
* add ERM Task abstract base class AbstractEmrTask
* add testcase for EmrAddStepsTask
* Modifying help Documents
2022-07-10 10:15:25 +08:00
057a9f6b24
[Fix-10830][Doc] Change the start or install script command sh in the document to bash ( #10831 )
...
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Change the script execution command sh to bash
* Change sh to bash
Co-authored-by: luke <luke.yan@china-entercom.net>
2022-07-09 15:41:56 +08:00
1387fb3ccf
[Doc][Upgrade] Fix tiny error in upgrade docs ( #10847 )
...
* [Doc][Upgrade] Fix tiny error in upgrade docs (#10846 )
* [Doc][Upgrade] Add useSSL=false and fix typo in upgrade docs (#10846 )
2022-07-09 14:53:05 +08:00
503cf179eb
[doc] 206 drop down doc not work ( #10855 )
2022-07-08 23:24:34 +08:00
d29592da1d
New version release 2.0.6 ( #10851 )
...
* New version release 2.0.6
* remove verison 1.3.8 and 1.3.9
2022-07-08 20:34:14 +08:00
1e48bbc5c6
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster ( #10650 )
...
* server code
* namespace ui
* fix dao test
* fix test
* fix
* docs and image update
Co-authored-by: qianl4 <qianl4@cicso.com>
2022-07-07 16:05:52 +08:00
71f0168510
[Feature] Enable users to create python env from requirements.txt ( #10658 )
2022-07-06 15:56:39 +08:00
2d8b4932b3
[doc] Correct custom jdbc driver for K8s Doc ( #10768 )
2022-07-06 10:57:14 +08:00
fa1458b3e0
[doc] Modify features, hw, intro and homepage docs. ( #10713 )
2022-07-05 21:58:05 +08:00
91b59dee54
[doc] Modified dq, monitor, security, resources ( #10715 )
2022-07-05 20:50:37 +08:00
bd1be6cf36
[Task] Added Apache SeaTunnel 2.x task node ( #10480 )
2022-07-05 19:21:30 +08:00
5b0347e8da
[Task] Add Dinky task to better support the development and execution of FlinkSQL ( #10640 )
2022-07-05 10:07:24 +08:00
7aad6a6515
[doc] Unify data source switching of the docs during installation ( #10688 )
2022-07-04 16:28:14 +08:00
6cf9d088ff
[doc] Change tasks docs ( #10751 )
2022-07-04 11:29:59 +08:00
423cc5b9be
[doc] Modified project doc. ( #10711 )
2022-07-04 10:23:25 +08:00
4f0a02eb43
[chore] update by release version ( #10742 )
...
* update 3.0.0-beta-2 doc
* update history-versions doc
2022-07-04 10:05:29 +08:00
fb8d0c858b
Merging general settings to faq ( #10716 )
...
* Merging general settings to faq
* also change the zh position
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-07-01 17:18:02 +08:00
fb90a0e558
[doc] Modify quick start. ( #10712 )
2022-07-01 16:51:07 +08:00
0cdae777ab
[doc] Modified alert doc files ( #10714 )
2022-07-01 16:29:00 +08:00
4a3c3e7797
[feature][task-flink] Support Flink application mode ( #9577 )
2022-07-01 12:20:38 +08:00
65ebdbef98
[doc] Change tasks doc ( #10639 )
2022-07-01 01:46:04 +08:00
7898f41dc0
[doc] Change datasource doc ( #10660 )
2022-06-30 23:33:59 +08:00
a45fc5b032
[doc] Add doc for pre-sql and post-sql ( #10685 )
2022-06-30 20:01:23 +08:00
c2f1886306
[doc] Update metadata and design doc ( #10600 )
2022-06-30 17:48:17 +08:00
1cdbbc1ba5
[doc] Modified parameter doc files. ( #10659 )
2022-06-29 14:42:37 +08:00
1f46078ab8
[doc] Change subprocess dependent ( #10637 )
2022-06-27 19:30:35 +08:00
a14b96b091
[doc] Modify Shell task file ( #10634 )
2022-06-27 17:22:09 +08:00
cd51a22940
[doc] Correct installPath when using related path ( #10597 )
...
correct the doc and add hint to setting
ref: #10511
2022-06-27 11:02:19 +08:00
16144a8858
[docs] Add links to metrics docs ( #10583 )
2022-06-24 17:27:40 +08:00
b2df357814
[doc] Add run the task alone doc. ( #10581 )
2022-06-24 16:59:22 +08:00
33cc68ed4a
[doc] Add doc about how to use healthcheck endpoints ( #10592 )
2022-06-24 16:56:26 +08:00
b21e211d2c
[doc] Add docs about complement dependent mode ( #10587 )
2022-06-24 16:33:10 +08:00
773ebc9e52
fix 10513 ( #10564 )
2022-06-23 13:44:22 +08:00
1b1a567fec
[doc] Correct the complement add docs ( #10521 )
2022-06-23 12:02:28 +08:00
3918140e7e
[fix][doc] Add DVC task plugin doc link ( #10529 )
2022-06-23 10:01:49 +08:00
0aba68f335
[Improvement][Metrics] Update some metrics names in grafana-demo dashboards ( #10432 ) ( #10552 )
2022-06-22 23:05:54 +08:00
cc06eaaf54
[Improvement][Metrics] Apply micrometer naming convention to metrics ( #10477 )
...
* Apply micrometer naming convention to worker metrics
* Apply micrometer naming convention all current metrics
* Fix remaining metrics names, update English docs and add Chinese docs
* Fix metrics names in grafana-demo dashboards
2022-06-21 14:27:06 +08:00
36e20cdfc8
[Feature][Task Plugin] Add DVC task plugin for MLops scenario ( #10372 ) ( #10407 )
2022-06-21 13:13:46 +08:00
6e4b2e6992
[Fix]Recovery Traffic Control Config ( #10505 )
...
* recovery traffic config
* run ci
* update
2022-06-20 18:43:48 +08:00
64cee03fed
[Feature] Enable users to switch and install conda env in jupyter task ( #10337 )
2022-06-20 14:47:44 +08:00
704e633659
[Doc] Add api test doc ( #10499 )
2022-06-19 13:55:23 +08:00
4be3b877c3
[feature][task] Enable zeppelin schedule a whole zeppelin note ( #10434 )
2022-06-18 13:34:24 +08:00
2c227ab384
[Feature-10363][server] S3 Resource center supports bucket customization ( #10433 )
...
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify the configuration. Sort out HDFS and S3 configurations to distinguish them
* Modify the configuration comment to clarify the configuration meaning
* Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage
* Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties
* Modified code to support bucket customization
* Check that the bucket on personal permission, Otherwise, the startup fails
* S3 increase support resource. Storage. Upload. Base. The base path path
* Change S3Utils implementation style to be consistent with HadoopUtils
* [Feature-10363][server] S3 Resource center supports bucket customization
* Modify code review comments
2022-06-18 10:20:01 +08:00
b8d5e78994
[Doc] Add how to set the alarm mailbox in faq ( #10401 )
2022-06-17 21:16:38 +08:00
Kerwin