natas_hw
/
dolphinscheduler
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[Doc][Bug] Fix dead link caused by markdown cross-files anchor (#12357)

This commit is contained in:
Eric Gao 2022-10-14 10:57:11 +08:00 committed by GitHub
parent ac845046e7
commit c50cc997a8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
62 changed files with 293 additions and 155 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/docs/en/contribute/join/pull-request.md
View File

@ -63,8 +63,12 @@ Please refer to the commit message section.
### Pull Request Code Style
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (DolphinScheduler uses `Spotless` to automatically fix code style and formatting errors,)
[//]: # (see [Development Environment Setup](../development-environment-setup.md#code-style) `Code Style` section for details.)
DolphinScheduler uses `Spotless` to automatically fix code style and formatting errors,
see [Code Style](../development-environment-setup.md#code-style) for details.
see [Development Environment Setup](../development-environment-setup.md) `Code Style` section for details.
### Question

5
docs/docs/en/guide/task/chunjun.md
View File

@ -11,7 +11,10 @@ ChunJun task type for executing ChunJun programs. For ChunJun nodes, the worker
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters
| **Parameter** | **Description** |
|-------------------|---------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/conditions.md
View File

@ -9,7 +9,10 @@ Condition is a conditional node, that determines which downstream task should ru
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

22
docs/docs/en/guide/task/datasync.md
View File

@ -30,23 +30,10 @@ The follow shows the DolphinScheduler DataSync task plugin features:
## Task Example
First, introduce some general parameters of DolphinScheduler:
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) `Default Task Parameters` section for default parameters.)
- **Node name**: The name of the task. Node names within the same workflow must be unique.
- **Run flag**: Indicating whether to schedule the task. If you do not need to execute the task, you can turn on the `Prohibition execution` switch.
- **Description**: Describing the function of this node.
- **Task priority**: When the number of the worker threads is insufficient, the worker executes task according to the priority. When two tasks have the same priority, the worker will execute them in `first come first served` fashion.
- **Worker group**: Machines which execute the tasks. If you choose `default`, scheduler will send the task to a random worker.
- **Task group name**: Resource group of tasks. It will not take effect if not configured.
- **Environment name**: Environment to execute the task.
- **Number of failed retries**: The number of task retries for failures. You could select it by drop-down menu or fill it manually.
- **Failure retry interval**: Interval of task retries for failures. You could select it by drop-down menu or fill it manually.
- **CPU quota**: Assign the specified CPU time quota to the task executed. Takes a percentage value. Default -1 means unlimited. For example, the full CPU load of one core is 100%, and that of 16 cores is 1600%. You could configure it by [task.resource.limit.state](../../architecture/configuration.md).
- **Max memory**: Assign the specified max memory to the task executed. Exceeding this limit will trigger oom to be killed and will not automatically retry. Takes an MB value. Default -1 means unlimited. You could configure it by [task.resource.limit.state](../../architecture/configuration.md).
- **Timeout alarm**: Alarm for task timeout. When the task exceeds the "timeout threshold", an alarm email will send.
- **Delayed execution time**: The time that a task delays for execution in minutes.
- **Resources**: Resources which your task node uses.
- **Predecessor task**: The upstream task of the current task node.
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
Here are some specific parameters for the DataSync plugin:
@ -74,4 +61,5 @@ resource.aws.access.key.id=<YOUR AWS ACCESS KEY>
resource.aws.secret.access.key=<YOUR AWS SECRET KEY>
# Defines AWS Region to use and is required
resource.aws.region=<AWS REGION>
```
```

5
docs/docs/en/guide/task/datax.md
View File

@ -11,7 +11,10 @@ DataX task type for executing DataX programs. For DataX nodes, the worker will e
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/dependent.md
View File

@ -11,7 +11,10 @@ Dependent nodes are **dependency check nodes**. For example, process A depends o
## Task Parameter
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------|---------------------------------------------|

5
docs/docs/en/guide/task/dinky.md
View File

@ -12,7 +12,10 @@ it will call `Dinky API` to trigger dinky task. Click [here](http://www.dlink.to
## Task Parameter
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

17
docs/docs/en/guide/task/dms.md
View File

@ -2,8 +2,8 @@
## Overview
[AWS Database Migration Service (AWS DMS)](https://aws.amazon.com/cn/dms) helps you migrate databases to AWS quickly and securely.
The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database.
[AWS Database Migration Service (AWS DMS)](https://aws.amazon.com/cn/dms) helps you migrate databases to AWS quickly and securely.
The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database.
The AWS Database Migration Service can migrate your data to and from the most widely used commercial and open-source databases.
DMS task plugin can help users to create and start DMS tasks in DolphinScheduler more conveniently.
@ -33,12 +33,10 @@ The task plugin picture is as follows
![dms](../../../../img/tasks/demo/dms_create_and_start.png)
**Restart DMS task by interface**
![dms](../../../../img/tasks/demo/dms_restart.png)
**Create and start DMS task by JSON data**
![dms](../../../../img/tasks/demo/dms_create_and_start_json.png)
@ -47,10 +45,12 @@ The task plugin picture is as follows
![dms](../../../../img/tasks/demo/dms_restart_json.png)
### First, introduce some general parameters of DolphinScheduler
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
### Here are some specific parameters for the DMS plugin
@ -71,10 +71,10 @@ Parameters of restarting the task by interface
- **replicationTaskArn**：The ARN of the task.
## Environment to prepare
Some AWS configuration is required, modify a field in file `common.properties`
```yaml
# The AWS access key. if resource.storage.type=S3 or use EMR-Task, This configuration is required
resource.aws.access.key.id=<YOUR AWS ACCESS KEY>
@ -82,4 +82,5 @@ resource.aws.access.key.id=<YOUR AWS ACCESS KEY>
resource.aws.secret.access.key=<YOUR AWS SECRET KEY>
# The AWS Region to use. if resource.storage.type=S3 or use EMR-Task, This configuration is required
resource.aws.region=<AWS REGION>
```
```

5
docs/docs/en/guide/task/dvc.md
View File

@ -20,7 +20,10 @@ The plugin provides the following three functions:
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/emr.md
View File

@ -15,7 +15,10 @@ Using [aws-java-sdk](https://aws.amazon.com/cn/sdk-for-java/) in the background
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/flink.md
View File

@ -15,7 +15,10 @@ Flink task type, used to execute Flink programs. For Flink nodes:
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---|

5
docs/docs/en/guide/task/hive-cli.md
View File

@ -24,7 +24,10 @@ You could choose between these two based on your needs.
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------------------|------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/http.md
View File

@ -11,7 +11,10 @@ This node is used to perform http type tasks such as the common POST and GET req
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|---|--------------------|---|---|-------------------------|-----------------------------------------------------------------------------------------------|

31
docs/docs/en/guide/task/java.md
View File

@ -10,25 +10,18 @@ This node is for executing java-type tasks and supports using files and jar pack
# Task Parameters
| **Parameter** | **Description** |
|--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Node Name | The name of the set task. The node name in a workflow definition is unique. |
| Run Flag | Indicates whether the node is scheduled properly and turns on the kill switch, if not needed. |
| Description | Describes the functionality of the node. |
| Task Priority | When the number of worker threads is insufficient, the worker executes tasks according to the priority. When the priority is the same, the worker executes tasks by order. |
| Worker Group | The group of machines who execute the tasks. If selecting `Default`, DolphinScheduler will randomly choose a worker machine to execute the task. |
| Environment Name | Configure the environment in which the task runs. |
| Number Of Failed Retries | Number of resubmitted tasks that failed. You can choose the number in the drop-down menu or fill it manually. |
| Failed Retry Interval | the interval between the failure and resubmission of a task. You can choose the number in the drop-down menu or fill it manually. |
| Delayed Execution Time | the amount of time a task is delayed, in units. |
| Timeout Alarm | Check timeout warning, timeout failure, when the task exceeds the“Timeout length”, send a warning message and the task execution fails. |
| Module Path | pick Java 9 + 's modularity feature, put all resources into-module-path, and require that the JDK version in your worker supports modularity. |
| Main Parameter | Java program main method entry parameter. |
| Java VM Parameters | JVM startup parameters. |
| Script | You need to write Java code if you use the Java run type. The public class must exist in the code without writing a package statement. |
| Resources | External JAR packages or other resource files that are added to the classpath or module path and can be easily retrieved in your JAVA script. |
| Custom parameter | A user-defined parameter that is part of HTTP and replaces `${ variable }` in the script . |
| Pre Tasks | Selects a pre-task for the current task and sets the pre-task as the upstream of the current task. |
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| Module Path | pick Java 9 + 's modularity feature, put all resources into-module-path, and require that the JDK version in your worker supports modularity. |
| Main Parameter | Java program main method entry parameter. |
| Java VM Parameters | JVM startup parameters. |
| Script | You need to write Java code if you use the Java run type. The public class must exist in the code without writing a package statement. |
| Resources | External JAR packages or other resource files that are added to the classpath or module path and can be easily retrieved in your JAVA script. |
## Example

5
docs/docs/en/guide/task/jupyter.md
View File

@ -102,7 +102,10 @@ papermill==2.3.4
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|---------------------------|-------------------------------------------------------------------|

5
docs/docs/en/guide/task/kubernetes.md
View File

@ -11,7 +11,10 @@ K8S task type used to execute a batch task. In this task, the worker submits the
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------|------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/map-reduce.md
View File

@ -11,7 +11,10 @@ MapReduce(MR) task type used for executing MapReduce programs. For MapReduce nod
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
### General

8
docs/docs/en/guide/task/mlflow.md
View File

@ -28,7 +28,10 @@ The MLflow plugin currently supports and will support the following:
## Task Parameters and Example
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
@ -100,6 +103,7 @@ You can now use this feature to run all MLFlow projects on Github (For example [
## Environment to Prepare
### Conda Environment
Please install [anaconda](https://docs.continuum.io/anaconda/install/) or [miniconda](https://docs.conda.io/en/latest/miniconda.html#installing) in advance.
**Method A:**
@ -113,7 +117,6 @@ Add the following content to the file:
export PATH=/opt/anaconda3/bin:$PATH
```
**Method B:**
You need to enter the admin account to configure a conda environment variable.
@ -153,3 +156,4 @@ ml.mlflow.preset_repository=https://github.com/apache/dolphinscheduler-mlflow
# mlflow task plugin preset repository version
ml.mlflow.preset_repository_version="main"
```

5
docs/docs/en/guide/task/openmldb.md
View File

@ -14,7 +14,10 @@ OpenMLDB task plugin used to execute tasks on OpenMLDB cluster.
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-------------------|--------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/pigeon.md
View File

@ -11,7 +11,10 @@ Pigeon is a task used to trigger remote tasks, acquire logs or status by calling
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------|---------------------------------------|

5
docs/docs/en/guide/task/python.md
View File

@ -12,7 +12,10 @@ it will generate a temporary python script, and executes the script by the Linux
## Task Parameter
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-------------------|--------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/pytorch.md
View File

@ -19,7 +19,10 @@ The task plugin picture is as follows
![pytorch](../../../../img/tasks/demo/pytorch_en.png)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
#### Run time parameters

5
docs/docs/en/guide/task/sagemaker.md
View File

@ -20,7 +20,10 @@ DolphinScheduler SageMaker task plugin features are as follows:
## Task Example
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
Here are some specific parameters for the SagaMaker plugin:

21
docs/docs/en/guide/task/seatunnel.md
View File

@ -12,16 +12,19 @@ Click [here](https://seatunnel.apache.org/) for more information about `Apache S
## Task Parameter
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
- Engine: Supports FLINK and SPARK
- FLINK
- Run model: supports `run` and `run-application` modes
- Option parameters: used to add the parameters of the Flink engine, such as `-m yarn-cluster -ynm seatunnel`
- SPARK
- Deployment mode: specify the deployment mode, `cluster` `client` `local`
- Master: Specify the `Master` model, `yarn` `local` `spark` `mesos`, where `spark` and `mesos` need to specify the `Master` service address, for example: 127.0.0.1:7077
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
> Click [here](https://seatunnel.apache.org/docs/2.1.2/command/usage) for more information on the usage of `Apache SeaTunnel command`
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
- Engine: Supports FLINK and SPARK
- FLINK
- Run model: supports `run` and `run-application` modes
- Option parameters: used to add the parameters of the Flink engine, such as `-m yarn-cluster -ynm seatunnel`
- SPARK
- Deployment mode: specify the deployment mode, `cluster` `client` `local`
- Master: Specify the `Master` model, `yarn` `local` `spark` `mesos`, where `spark` and `mesos` need to specify the `Master` service address, for example: 127.0.0.1:7077
> Click [here](https://seatunnel.apache.org/docs/2.1.2/command/usage) for more information on the usage of `Apache SeaTunnel command`
- Custom Configuration: Supports custom configuration or select configuration file from Resource Center

5
docs/docs/en/guide/task/shell.md
View File

@ -11,7 +11,10 @@ Shell task type, used to create a shell type task and execute a series of shell
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------------|-----------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/spark.md
View File

@ -15,7 +15,10 @@ Spark task type for executing Spark application. When executing the Spark task,
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/sql.md
View File

@ -15,7 +15,10 @@ Refer to [datasource-setting](../howto/datasource-setting.md) `DataSource Center
## Task Parameter
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/stored-procedure.md
View File

@ -10,7 +10,10 @@
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/sub-process.md
View File

@ -11,7 +11,10 @@ The sub-process node is to execute an external workflow definition as a task nod
## Task Parameter
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/switch.md
View File

@ -15,7 +15,10 @@ The switch is a conditional judgment node, decide the branch executes according
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/en/guide/task/zeppelin.md
View File

@ -12,7 +12,10 @@ it will call `Zeppelin Client API` to trigger zeppelin notebook paragraph. Click
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md#default-task-parameters) for default parameters.
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- Please refer to [DolphinScheduler Task Parameters Appendix]&#40;appendix.md#default-task-parameters&#41; `Default Task Parameters` section for default parameters.)
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameter** | **Description** |
|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|

6
docs/docs/zh/contribute/join/pull-request.md
View File

@ -65,8 +65,12 @@ waste time。
### Pull Request Code Style
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (DolphinScheduler使用`Spotless`为您自动修复代码风格和格式问题，)
[//]: # (详情见[开发手册]&#40;../development-environment-setup.md#代码风格&#41;`代码风格`一栏。)
DolphinScheduler使用`Spotless`为您自动修复代码风格和格式问题，
详情见[代码风格](../development-environment-setup.md#代码风格)。
详情见[开发手册](../development-environment-setup.md)`代码风格`一栏。
### 相关问题

5
docs/docs/zh/guide/task/chunjun.md
View File

@ -11,7 +11,10 @@ ChunJun 任务类型，用于执行 ChunJun 程序。对于 ChunJun 节点，wor
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|------------------------------------------------------------|

5
docs/docs/zh/guide/task/conditions.md
View File

@ -9,7 +9,10 @@ Conditions 是一个条件节点，根据上游任务运行状态，判断应该
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|---------------------------------------------------------------------------------------------------------------------------------------|

34
docs/docs/zh/guide/task/datasync.md
View File

@ -18,39 +18,23 @@ DataSync 支持的组件:
- Amazon FSx for NetApp ONTAP file systems
- AWS Snowcone devices
DolphinScheduler DataSync 组件的功能:
- 创建 AWS DataSync 任务并启动，持续获取状态，直至任务执行完成。
DolphinScheduler DataSync 组件的功能:
- 创建 AWS DataSync 任务并启动，持续获取状态，直至任务执行完成。
## 创建任务
- 点击项目管理-项目名称-工作流定义，点击“创建工作流”按钮，进入 DAG 编辑页面；
- 拖动工具栏的 <img src="../../../../img/tasks/icons/datasync.png" width="15"/> 任务节点到画板中。
## 任务样例
首先介绍一些DS通用参数
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- **任务名称** ：任务的名称，同一个工作流定义中的节点名称不能重复。
- **运行标志** ：标识这个节点是否需要调度执行，如果不需要执行，可以打开禁止执行开关。
- **描述** ：当前节点的功能描述。
- **任务优先级** ：worker线程数不足时，根据优先级从高到低依次执行任务，优先级一样时根据先到先得原则执行。
- **Worker 分组** ：设置分组后，任务会被分配给worker组的机器机执行。若选择Default，则会随机选择一个worker执行。
- **任务组名称** ：任务资源组，未配置则不生效。
- **组内优先级** ：一个任务组内此任务的优先级。
- **环境名称** ：配置任务执行的环境。
- **失败重试次数** ：任务失败重新提交的次数，可以在下拉菜单中选择或者手动填充。
- **失败重试间隔** ：任务失败重新提交任务的时间间隔，可以在下拉菜单中选择或者手动填充。
- **CPU 配额** ：为执行的任务分配指定的CPU时间配额，单位百分比，默认-1代表不限制，例如1个核心的CPU满载是100%，16个核心的是1600%。 [task.resource.limit.state](../../architecture/configuration.md)
- **最大内存** ：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 [task.resource.limit.state](../../architecture/configuration.md) 控制。
- **超时告警** ：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.这个功能由 [task.resource.limit.state](../../architecture/configuration.md) 控制。
- **资源** ：任务执行时所需资源文件。
- **前置任务** ：设置当前任务的前置（上游）任务。
- **延时执行时间** ：任务延迟执行的时间，以分为单位。
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
以上参数如无特殊需求，可以默认即可
### 独有参数
- **name**: 任务名称
- **destinationLocationArn**: 目标 AWS 存储资源位置的 Amazon Resource Name (ARN) ，可见 [AWS API](https://docs.aws.amazon.com/datasync/latest/userguide/API_CreateTask.html#DataSync-CreateTask-request-DestinationLocationArn)
@ -61,13 +45,10 @@ DolphinScheduler DataSync 组件的功能:
- **json**: 创建 datasync 任务的JSON结构任务参数，可以支持options等参数，可见 [AWS CreateTask API] 的 Request Syntax (https://docs.aws.amazon.com/datasync/latest/userguide/API_CreateTask.html)
组件图示如下：
![datasync](../../../../img/tasks/demo/datasync_task01.png)
## 环境配置
需要进行AWS的一些配置，修改`common.properties`中的`xxxxx`为你的配置信息
@ -79,4 +60,5 @@ resource.aws.access.key.id=<YOUR AWS ACCESS KEY>
resource.aws.secret.access.key=<YOUR AWS SECRET KEY>
# Defines AWS Region to use and is required
resource.aws.region=<AWS REGION>
```
```

5
docs/docs/zh/guide/task/datax.md
View File

@ -11,7 +11,10 @@ DataX 任务类型，用于执行 DataX 程序。对于 DataX 节点，worker
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-------------------------------------------------------------------------------------------------------|

5
docs/docs/zh/guide/task/dependent.md
View File

@ -11,7 +11,10 @@ Dependent 节点，就是**依赖检查节点**。比如 A 流程依赖昨天的
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
- 此任务除上述链接中的默认参数外无其他参数。
## 任务样例

5
docs/docs/zh/guide/task/dinky.md
View File

@ -12,7 +12,10 @@
## Task Parameter
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|-------------|---------------------------------------------------------------------|

13
docs/docs/zh/guide/task/dms.md
View File

@ -16,12 +16,10 @@ DMS任务组件帮助用户在DolphinScheduler中创建和启动DMS任务。
- 通过界面创建
- 通过Json数据创建
DolphinScheduler 在 启动DMS 任务后，会跟中DMS任务状态，直至DMS任务完成后才将任务设为成功状态。除了以下情况：
不跟踪无结束时间的CDC任务，即 当迁移类型为 `full-load-and-cdc` 或者 `cdc` 时，且没有配置 `cdcStopPosition` 参数时，DolphinScheduler 在成功启动任务后，则会将任务状态设为 成功。
## 创建任务
- 点击项目管理-项目名称-工作流定义，点击“创建工作流”按钮，进入 DAG 编辑页面；
@ -47,11 +45,12 @@ DolphinScheduler 在 启动DMS 任务后，会跟中DMS任务状态，直至DMS
![dms](../../../../img/tasks/demo/dms_restart_json.png)
### 首先介绍一些DS通用参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
### DMS组件独有的参数
@ -61,7 +60,7 @@ DolphinScheduler 在 启动DMS 任务后，会跟中DMS任务状态，直至DMS
创建并启动迁移任务时参数
- **migrationType**：迁移类型, 可选值为：[ `full-load`, `full-load-and-cdc`, `cdc` ]
- **migrationType**：迁移类型, 可选值为：[ `full-load`, `full-load-and-cdc`, `cdc`]
- **replicationTaskIdentifier**：迁移任务标识符, 任务名称
- **replicationInstanceArn**：迁移实例的ARN
- **sourceEndpointArn**：源端点的ARN
@ -75,6 +74,7 @@ DolphinScheduler 在 启动DMS 任务后，会跟中DMS任务状态，直至DMS
## 环境配置
需要进行AWS的一些配置，修改`common.properties`中的以下配置信息
```yaml
# The AWS access key. if resource.storage.type=S3 or use EMR-Task, This configuration is required
resource.aws.access.key.id=<YOUR AWS ACCESS KEY>
@ -83,3 +83,4 @@ resource.aws.secret.access.key=<YOUR AWS SECRET KEY>
# The AWS Region to use. if resource.storage.type=S3 or use EMR-Task, This configuration is required
resource.aws.region=<AWS REGION>
```

5
docs/docs/zh/guide/task/dvc.md
View File

@ -17,7 +17,10 @@ DVC 组件用于在DS上使用DVC的数据版本管理功能，帮助用户简
## 任务样例
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-------------------------------|

5
docs/docs/zh/guide/task/emr.md
View File

@ -10,7 +10,10 @@ Amazon EMR 任务类型，用于在AWS上操作EMR集群并执行计算任务。
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/zh/guide/task/flink.md
View File

@ -15,7 +15,10 @@ Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点：
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

5
docs/docs/zh/guide/task/hive-cli.md
View File

@ -22,7 +22,10 @@
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|---------------|-----------------------------------------------------|

5
docs/docs/zh/guide/task/http.md
View File

@ -12,7 +12,10 @@
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-------------------------------------|

5
docs/docs/zh/guide/task/java.md
View File

@ -12,7 +12,10 @@
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|---------------------------------------------------------------|

5
docs/docs/zh/guide/task/jupyter.md
View File

@ -101,7 +101,10 @@ papermill==2.3.4
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|---------------------------|------------------------------------|

5
docs/docs/zh/guide/task/kubernetes.md
View File

@ -11,7 +11,10 @@ kubernetes任务类型，用于在kubernetes上执行一个短时和批处理的
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-----------------------------------------------------------------|

5
docs/docs/zh/guide/task/map-reduce.md
View File

@ -11,7 +11,10 @@ MapReduce(MR) 任务类型，用于执行 MapReduce 程序。对于 MapReduce
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
### JAVA/SCALA 程序

8
docs/docs/zh/guide/task/mlflow.md
View File

@ -27,7 +27,10 @@ MLflow 组件用于执行 MLflow 任务，目前包含Mlflow Projects，和MLflo
## 任务样例
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
以下是一些MLflow 组件的常用参数
@ -100,6 +103,7 @@ MLflow 组件用于执行 MLflow 任务，目前包含Mlflow Projects，和MLflo
配置文件：/dolphinscheduler/conf/env/dolphinscheduler_env.sh。
在文件最后添加内容
```
# 配置你的conda环境路径
export PATH=/opt/anaconda3/bin:$PATH
@ -115,7 +119,6 @@ export PATH=/opt/anaconda3/bin:$PATH
![mlflow-set-conda-env](../../../../img/tasks/demo/mlflow-set-conda-env.png)
### MLflow service 启动
确保你已经安装MLflow，可以使用`pip install mlflow`进行安装。
@ -134,7 +137,6 @@ mlflow server -h 0.0.0.0 -p 5000 --serve-artifacts --backend-store-uri sqlite://
![mlflow-server](../../../../img/tasks/demo/mlflow-server.png)
### 内置算法仓库配置
如果遇到github无法访问的情况，可以修改`commom.properties`配置文件的以下字段，将github地址替换能访问的地址。

5
docs/docs/zh/guide/task/openmldb.md
View File

@ -13,7 +13,10 @@ OpenMLDB任务组件可以连接OpenMLDB集群执行任务。
## 任务样例
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|-------------|--------------------------------------------------|

5
docs/docs/zh/guide/task/pigeon.md
View File

@ -8,7 +8,10 @@ Pigeon任务类型是通过调用远程websocket服务，实现远程任务的
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-------------------|

5
docs/docs/zh/guide/task/python.md
View File

@ -12,7 +12,10 @@ Python 任务类型，用于创建 Python 类型的任务并执行一系列的 P
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-----------------------------------|

5
docs/docs/zh/guide/task/pytorch.md
View File

@ -21,7 +21,10 @@
### Pytorch参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
#### 运行参数

5
docs/docs/zh/guide/task/sagemaker.md
View File

@ -18,7 +18,10 @@ DolphinScheduler SageMaker 组件的功能:
## 任务样例
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------|

21
docs/docs/zh/guide/task/seatunnel.md
View File

@ -12,16 +12,19 @@
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
- 引擎：支持 FLINK 和 SPARK
- FLINK
- 运行模型：支持 `run` 和 `run-application` 两种模式
- 选项参数：用于添加 Flink 引擎本身参数，例如 `-m yarn-cluster -ynm seatunnel`
- SPARK
- 部署方式：指定部署模式，`cluster` `client` `local`
- Master：指定 `Master` 模型，`yarn` `local` `spark` `mesos`，其中 `spark` 和 `mesos` 需要指定 `Master` 服务地址，例如：127.0.0.1:7077
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
> 点击 [这里](https://seatunnel.apache.org/docs/2.1.2/command/usage) 获取更多关于`Apache SeaTunnel command` 使用的信息
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
- 引擎：支持 FLINK 和 SPARK
- FLINK
- 运行模型：支持 `run` 和 `run-application` 两种模式
- 选项参数：用于添加 Flink 引擎本身参数，例如 `-m yarn-cluster -ynm seatunnel`
- SPARK
- 部署方式：指定部署模式，`cluster` `client` `local`
- Master：指定 `Master` 模型，`yarn` `local` `spark` `mesos`，其中 `spark` 和 `mesos` 需要指定 `Master` 服务地址，例如：127.0.0.1:7077
> 点击 [这里](https://seatunnel.apache.org/docs/2.1.2/command/usage) 获取更多关于`Apache SeaTunnel command` 使用的信息
- 自定义配置：支持自定义配置或从资源中心选择配置文件

5
docs/docs/zh/guide/task/shell.md
View File

@ -11,7 +11,10 @@ Shell 任务类型，用于创建 Shell 类型的任务并执行一系列的 She
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
- 除上述默认参数，此任务没有其他参数
## 任务样例

5
docs/docs/zh/guide/task/spark.md
View File

@ -16,7 +16,10 @@ Spark 任务类型用于执行 Spark 应用。对于 Spark 节点，worker 支
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
- 程序类型：支持 Java、Scala、Python 和 SQL 四种语言。
- 主函数的 Class：Spark 程序的入口 Main class 的全路径。
- 主程序包：执行 Spark 程序的 jar 包（通过资源中心上传）。

11
docs/docs/zh/guide/task/sql.md
View File

@ -15,12 +15,15 @@ SQL任务类型，用于连接数据库并执行相应SQL。
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
- 数据源：选择对应的数据源
- sql类型：支持查询和非查询两种。
- 查询：支持 `DML select` 类型的命令，是有结果集返回的，可以指定邮件通知为表格、附件或表格附件三种模板；
- 非查询：支持 `DDL`全部命令 和 `DML update、delete、insert` 三种类型的命令；
- 默认采用`;\n`作为SQL分隔符,拆分成多段SQL语句执行。Hive支持一次执行多段SQL语句,故不会拆分。
- 查询：支持 `DML select` 类型的命令，是有结果集返回的，可以指定邮件通知为表格、附件或表格附件三种模板；
- 非查询：支持 `DDL`全部命令 和 `DML update、delete、insert` 三种类型的命令；
- 默认采用`;\n`作为SQL分隔符,拆分成多段SQL语句执行。Hive支持一次执行多段SQL语句,故不会拆分。
- sql参数：输入参数格式为key1=value1;key2=value2…
- sql语句：SQL语句
- UDF函数：对于HIVE类型的数据源，可以引用资源中心中创建的UDF函数，其他类型的数据源暂不支持UDF函数。

5
docs/docs/zh/guide/task/stored-procedure.md
View File

@ -19,7 +19,10 @@ END
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|---------------|--------------------------------------------------------------------------------------------------|

5
docs/docs/zh/guide/task/sub-process.md
View File

@ -12,7 +12,10 @@
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|----------------------------------------|

5
docs/docs/zh/guide/task/switch.md
View File

@ -11,7 +11,10 @@ Switch 是一个条件判断节点，依据[全局变量](../parameter/global.md
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|----------|-------------------------------------------------------------------------|

5
docs/docs/zh/guide/task/zeppelin.md
View File

@ -12,7 +12,10 @@
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md#默认任务参数)。
[//]: # (TODO: use the commented anchor below once our website template supports this syntax)
[//]: # (- 默认参数说明请参考[DolphinScheduler任务参数附录]&#40;appendix.md#默认任务参数&#41;`默认任务参数`一栏。)
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|------------------------------------|---------------------------------------------------|