This commit is contained in:
EricGao888 2024-06-07 16:13:05 +08:00
parent 7370c1e46b
commit e1f09d2c1f
6 changed files with 90 additions and 0 deletions

View File

@ -0,0 +1,45 @@
# Aliyun EMR Serverless Spark
## Introduction
`Aliyun EMR Serverless Spark` task plugin submits spark job to
[`Aliyun EMR Serverless Spark`](https://help.aliyun.com/zh/emr/emr-serverless-spark/product-overview/what-is-emr-serverless-spark) service.
## Create Connections
- Click `Datasource -> Create Datasource -> ALIYUN_SERVERLESS_SPARK` to create a connection.
![demo-aliyun-serverless-spark-create-datasource-1](../../../../img/tasks/demo/aliyun_serverless_spark_1.png)
- Fill in `Datasource Name`, `Access Key Id`, `Access Key Secret`, `Region Id` and click `Confirm`.
![demo-aliyun-serverless-spark-create-datasource-2](../../../../img/tasks/demo/aliyun_serverless_spark_2.png)
## Create Tasks
- Click `Porject -> Workflow Definition -> Create Workflow` and drag the `ALIYUN_SERVERLESS_SPARK` task to the canvas.
![demo-aliyun-serverless-spark-create-task-1](../../../../img/tasks/demo/aliyun_serverless_spark_3.png)
- Fill in the task parameters and click `Confirm` to create the task node.
![demo-aliyun-serverless-spark-create-task-2](../../../../img/tasks/demo/aliyun_serverless_spark_4.png)
## Task Parameters
- Please refer to [DolphinScheduler Task Parameters Appendix](appendix.md) `Default Task Parameters` section for default parameters.
| **Parameters** | **Description** |
|-------------------------|-----------------------------------------------------------------------------------------------------|
| Datasource types | The type of datasource the task uses, should be `ALIYUN_SERVERLESS_SPARK`. |
| Datasource instances | The instance of `ALIYUN_SERVERLESS_SPARK` datasource. |
| workspace id | `Aliyun Serverless Spark` workspace id. |
| resource queue id | `Aliyun Serverless Spark` resource queue the task uses to submit spark job. |
| code type | `Aliyun Serverless Spark` code type, could be `JAR`, `PYTHON` or `SQL`. |
| job name | `Aliyun Serverless Spark` job name. |
| entry point | The location of the job code such as jar package, python file, or sql file. OSS location supported. |
| entry point arguments | Arguments of the job main program. |
| spark submit parameters | Spark-submit related parameters. |
| engine release version | Spark engine release version. |
| is production | Whether the spark job runs in production or development environment. |

View File

@ -0,0 +1,45 @@
# Aliyun EMR Serverless Spark
## 简介
`Aliyun EMR Serverless Spark` 任务插件用于向
[`阿里云EMR Serverless Spark`](https://help.aliyun.com/zh/emr/emr-serverless-spark/product-overview/what-is-emr-serverless-spark) 服务提交作业。
## 创建链接
- 点击 `数据源 -> 创建数据源 -> ALIYUN_SERVERLESS_SPARK` 创建链接。
![demo-aliyun-serverless-spark-create-datasource-1](../../../../img/tasks/demo/aliyun_serverless_spark_1.png)
- 填入 `Datasource Name`, `Access Key Id`, `Access Key Secret`, `Region Id` 参数并且点击 `确认`.
![demo-aliyun-serverless-spark-create-datasource-2](../../../../img/tasks/demo/aliyun_serverless_spark_2.png)
## 创建任务节点
- 点击 `项目 -> 工作流定义 -> 创建工作流` 并且将 `ALIYUN_SERVERLESS_SPARK` 任务拖到画板中。
![demo-aliyun-serverless-spark-create-task-1](../../../../img/tasks/demo/aliyun_serverless_spark_3.png)
- 填入相关任务参数并且点击 `确认` 创建任务节点。
![demo-aliyun-serverless-spark-create-task-2](../../../../img/tasks/demo/aliyun_serverless_spark_4.png)
## 任务参数
- 默认参数说明请参考[DolphinScheduler任务参数附录](appendix.md)`默认任务参数`一栏。
| **任务参数** | **描述** |
|-------------------------|----------------------------------------------------------|
| Datasource types | 链接类型,应该选择 `ALIYUN_SERVERLESS_SPARK`。 |
| Datasource instances | `ALIYUN_SERVERLESS_SPARK` 链接实例。 |
| workspace id | `Aliyun Serverless Spark` 工作空间id。 |
| resource queue id | `Aliyun Serverless Spark` 任务队列id。 |
| code type | `Aliyun Serverless Spark` 任务类型,可以是`JAR`、`PYTHON`或者`SQL`。 |
| job name | `Aliyun Serverless Spark` 任务名。 |
| entry point | 任务代码JAR包、PYTHON / SQL脚本的位置支持OSS中的文件。 |
| entry point arguments | 主程序入口参数。 |
| spark submit parameters | Spark-submit相关参数。 |
| engine release version | Spark引擎版本。 |
| is production | Spark任务是否运行在生产环境中。 |

Binary file not shown.

After

Width:  |  Height:  |  Size: 762 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 945 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 919 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 992 KiB