natas_hw
/
dolphinscheduler
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Bump up dependencies to fix cves (#11765)

This commit is contained in:
kezhenxu94 2022-09-08 09:05:05 +08:00 committed by caishunfeng
parent 1aba077fb8
commit fcc75ef1c6
134 changed files with 2293 additions and 6856 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
docs/docs/en/contribute/development-environment-setup.md
View File

@ -1,6 +1,7 @@
# DolphinScheduler development
## Software Requirements
Before setting up the DolphinScheduler development environment, please make sure you have installed the software as below:
* [Git](https://git-scm.com/downloads)
@ -46,6 +47,7 @@ fix things for you.
DolphinScheduler will release new Docker images after it released, you could find them in [Docker Hub](https://hub.docker.com/search?q=DolphinScheduler).
* If you want to modify DolphinScheduler source code, and build Docker images locally, you can run when finished the modification
```shell
cd dolphinscheduler
./mvnw -B clean package \
@ -59,6 +61,7 @@ cd dolphinscheduler
When the command is finished you could find them by command `docker imaegs`.
* If you want to modify DolphinScheduler source code, build and push Docker images to your registry <HUB_URL>，you can run when finished the modification
```shell
cd dolphinscheduler
./mvnw -B clean deploy \
@ -92,7 +95,6 @@ RUN apt update ; \
>
> Have to use version after Docker 19.03, because after 19.03 docker contains buildx
## Notice
There are two ways to configure the DolphinScheduler development environment, standalone mode and normal mode
@ -122,6 +124,7 @@ Find the class `org.apache.dolphinscheduler.StandaloneServer` in Intellij IDEA a
### Start frontend server
Install frontend dependencies and run it.
> Note: You can see more detail about the frontend setting in [frontend development](./frontend-development.md).
```shell
@ -143,12 +146,11 @@ Download [ZooKeeper](https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3
* Create directory `zkData` and `zkLog`
* Go to the zookeeper installation directory, copy configure file `zoo_sample.cfg` to `conf/zoo.cfg`, and change value of dataDir in conf/zoo.cfg to dataDir=./tmp/zookeeper
```shell
# We use path /data/zookeeper/data and /data/zookeeper/datalog here as example
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/datalog
```
```shell
# We use path /data/zookeeper/data and /data/zookeeper/datalog here as example
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/datalog
```
* Run `./bin/zkServer.sh` in terminal by command `./bin/zkServer.sh start`.
#### Database
@ -166,21 +168,22 @@ Following steps will guide how to start the DolphinScheduler backend service
* Open project: Use IDE open the project, here we use Intellij IDEA as an example, after opening it will take a while for Intellij IDEA to complete the dependent download
* File change
* If you use MySQL as your metadata database, you need to modify `dolphinscheduler/pom.xml` and change the `scope` of the `mysql-connector-java` dependency to `compile`. This step is not necessary to use PostgreSQL
* Modify database configuration, modify the database configuration in the `dolphinscheduler-master/src/main/resources/application.yaml`
* Modify database configuration, modify the database configuration in the `dolphinscheduler-worker/src/main/resources/application.yaml`
* Modify database configuration, modify the database configuration in the `dolphinscheduler-api/src/main/resources/application.yaml`
We here use MySQL with database, username, password named dolphinscheduler as an example
```application.yaml
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
username: dolphinscheduler
password: dolphinscheduler
```
```application.yaml
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
username: dolphinscheduler
password: dolphinscheduler
```
* Log level: add a line `<appender-ref ref="STDOUT"/>` to the following configuration to enable the log to be displayed on the command line

2
docs/docs/en/faq.md
View File

@ -280,7 +280,7 @@ A ： Will hive pom
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>2.1.0</version>
<version>2.3.3</version>
</dependency>
```

8
docs/docs/en/guide/open-api.md
View File

@ -28,13 +28,13 @@ Generally, projects and processes are created through pages, but considering the
2. select a test API, the API selected for this test is `queryAllProjectList`
> projects/list
>
> projects/list
3. Open `Postman`, fill in the API address, enter the `Token` in `Headers`, and then send the request to view the result:
```
token: The Token just generated
```
token: The Token just generated
```
![api-test](../../../img/new_ui/dev/open-api/api_test.png)

32
docs/docs/zh/contribute/development-environment-setup.md
View File

@ -1,6 +1,7 @@
# DolphinScheduler 开发手册
## 软件要求
在搭建 DolphinScheduler 开发环境之前请确保你已经安装以下软件:
* [Git](https://git-scm.com/downloads)
@ -27,7 +28,6 @@ git clone git@github.com:apache/dolphinscheduler.git
运行 `mvn clean install -Prelease -Dmaven.test.skip=true`
### 代码风格
DolphinScheduler使用`Spotless`检查并修复代码风格和格式问题。
@ -45,6 +45,7 @@ DolphinScheduler使用`Spotless`检查并修复代码风格和格式问题。
DolphinScheduler 每次发版都会同时发布 Docker 镜像，你可以在 [Docker Hub](https://hub.docker.com/search?q=DolphinScheduler) 中找到这些镜像
* 如果你想基于源码进行改造，然后在本地构建Docker镜像，可以在代码改造完成后运行
```shell
cd dolphinscheduler
./mvnw -B clean package \
@ -54,9 +55,11 @@ cd dolphinscheduler
-Ddocker.tag=<TAG> \
-Pdocker,release
```
当命令运行完了后你可以通过 `docker images` 命令查看刚刚创建的镜像
* 如果你想基于源码进行改造，然后构建Docker镜像并推送到 <HUB_URL>，可以在代码改造完成后运行
```shell
cd dolphinscheduler
./mvnw -B clean deploy \
@ -117,6 +120,7 @@ DolphinScheduler 开发环境配置有两个方式，分别是standalone模式
### 启动前端
安装前端依赖并运行前端组件
> 注意：你可以在[frontend development](./frontend-development.md)里查看更多前端的相关配置
```shell
@ -138,11 +142,10 @@ pnpm run dev
* 在 ZooKeeper 的目录下新建 zkData、zkLog文件夹
* 将 conf 目录下的 `zoo_sample.cfg` 文件，复制一份，重命名为 `zoo.cfg`，修改其中数据和日志的配置，如：
```shell
dataDir=/data/zookeeper/data ## 此处使用绝对路径
dataLogDir=/data/zookeeper/datalog
```
```shell
dataDir=/data/zookeeper/data ## 此处使用绝对路径
dataLogDir=/data/zookeeper/datalog
```
* 运行 `./bin/zkServer.sh`
#### 数据库
@ -160,21 +163,22 @@ DolphinScheduler 的元数据存储在关系型数据库中，目前支持的关
* 打开项目：使用开发工具打开项目，这里以 Intellij IDEA 为例，打开后需要一段时间，让 Intellij IDEA 完成以依赖的下载
* 必要的修改
* 如果使用 MySQL 作为元数据库，需要先修改 `dolphinscheduler/pom.xml`，将 `mysql-connector-java` 依赖的 `scope` 改为 `compile`，使用 PostgreSQL 则不需要
* 修改 Master 数据库配置，修改 `dolphinscheduler-master/src/main/resources/application.yaml` 文件中的数据库配置
* 修改 Worker 数据库配置，修改 `dolphinscheduler-worker/src/main/resources/application.yaml` 文件中的数据库配置
* 修改 Api 数据库配置，修改 `dolphinscheduler-api/src/main/resources/application.yaml` 文件中的数据库配置
本样例以 MySQL 为例，其中数据库名为 dolphinscheduler，账户名密码均为 dolphinscheduler
```application.yaml
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
username: dolphinscheduler
password: dolphinscheduler
```
```application.yaml
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
username: dolphinscheduler
password: dolphinscheduler
```
* 修改日志级别：为以下配置增加一行内容 `<appender-ref ref="STDOUT"/>` 使日志能在命令行中显示
`dolphinscheduler-master/src/main/resources/logback-spring.xml`

153
docs/docs/zh/faq.md
View File

@ -1,4 +1,5 @@
<!-- markdown-link-check-disable -->
## Q：项目的名称是？
A：DolphinScheduler
@ -9,19 +10,18 @@ A：DolphinScheduler
A：DolphinScheduler 由 5 个服务组成，MasterServer、WorkerServer、ApiServer、AlertServer、LoggerServer 和 UI。
| 服务 | 说明 |
| ------------------------- | ------------------------------------------------------------ |
| MasterServer | 主要负责 **DAG** 的切分和任务状态的监控 |
| 服务 | 说明 |
|---------------------------|---------------------------------------------------------------|
| MasterServer | 主要负责 **DAG** 的切分和任务状态的监控 |
| WorkerServer/LoggerServer | 主要负责任务的提交、执行和任务状态的更新。LoggerServer 用于 Rest Api 通过 **RPC** 查看日志 |
| ApiServer | 提供 Rest Api 服务，供 UI 进行调用 |
| AlertServer | 提供告警服务 |
| UI | 前端页面展示 |
| ApiServer | 提供 Rest Api 服务，供 UI 进行调用 |
| AlertServer | 提供告警服务 |
| UI | 前端页面展示 |
注意：**由于服务比较多，建议单机部署最好是 4 核 16G 以上**
---
## Q：系统支持哪些邮箱？
A：支持绝大多数邮箱，qq、163、126、139、outlook、aliyun 等皆支持。支持 **TLS 和 SSL** 协议，可以在dolphinscheduler的ui中进行配置：
@ -177,21 +177,16 @@ A ： 1.0.3 版本只实现了 Master 启动流程容错，不走 Worker 容错
A ： 设置定时的时候需要注意，如果第一位（* * * * * ? *）设置成 \* ，则表示每秒执行。**我们将会在 1.1.0 版本中加入显示最近调度的时间列表** ，使用 http://cron.qqe2.com/ 可以在线看近 5 次运行时间
## Q：定时有有效时间范围吗
A：有的，**如果定时的起止时间是同一个时间，那么此定时将是无效的定时**。**如果起止时间的结束时间比当前的时间小，很有可能定时会被自动删除**
## Q：任务依赖有几种实现
A： 1，**DAG** 之间的任务依赖关系，是从 **入度为零** 进行 DAG 切分的
​ 2，有 **任务依赖节点** ，可以实现跨流程的任务或者流程依赖，具体请参考 依赖(DEPENDENT)节点：https://analysys.github.io/easyscheduler_docs_cn/%E7%B3%BB%E7%BB%9F%E4%BD%BF%E7%94%A8%E6%89%8B%E5%86%8C.html#%E4%BB%BB%E5%8A%A1%E8%8A%82%E7%82%B9%E7%B1%BB%E5%9E%8B%E5%92%8C%E5%8F%82%E6%95%B0%E8%AE%BE%E7%BD%AE
## Q：流程定义有几种启动方式
A： 1，在 **流程定义列表**，点击 **启动** 按钮
@ -216,13 +211,10 @@ export PYTHON_HOME=/bin/python
export PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH
```
## Q：Worker Task 通过 sudo -u 租户 sh xxx.command 会产生子进程，在 kill 的时候，是否会杀掉
A： 我们会在 1.0.4 中增加 kill 任务同时，kill 掉任务产生的各种所有子进程
## Q：DolphinScheduler 中的队列怎么用，用户队列和租户队列是什么意思
A ： DolphinScheduler 中的队列可以在用户或者租户上指定队列，**用户指定的队列优先级是高于租户队列的优先级的。**，例如：对 MR 任务指定队列，是通过 mapreduce.job.queuename 来指定队列的。
@ -230,44 +222,36 @@ A ： DolphinScheduler 中的队列可以在用户或者租户上指定队列，
注意：MR 在用以上方法指定队列的时候，传递参数请使用如下方式：
```
Configuration conf = new Configuration();
GenericOptionsParser optionParser = new GenericOptionsParser(conf, args);
String[] remainingArgs = optionParser.getRemainingArgs();
Configuration conf = new Configuration();
GenericOptionsParser optionParser = new GenericOptionsParser(conf, args);
String[] remainingArgs = optionParser.getRemainingArgs();
```
如果是 Spark 任务 --queue 方式指定队列
## Q：Master 或者 Worker 报如下告警
<p align="center">
<img src="https://analysys.github.io/easyscheduler_docs_cn/images/master_worker_lack_res.png" width="60%" />
</p>
A ： 修改 conf 下的 master.properties **master.reserved.memory** 的值为更小的值，比如说 0.1 或者
worker.properties **worker.reserved.memory** 的值为更小的值，比如说 0.1
## Q：hive 版本是 1.1.0+cdh5.15.0，SQL hive 任务连接报错
<p align="center">
<img src="https://analysys.github.io/easyscheduler_docs_cn/images/cdh_hive_error.png" width="60%" />
</p>
A： 将 hive pom
```
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>2.1.0</version>
<version>2.3.3</version>
</dependency>
```
@ -284,6 +268,7 @@ A： 将 hive pom
---
## Q：如何增加一台工作服务器
A： 1，参考官网[部署文档](https://dolphinscheduler.apache.org/zh-cn/docs/laster/user_doc/installation/cluster.html) 1.3 小节，创建部署用户和 hosts 映射
​ 2，参考官网[部署文档](https://dolphinscheduler.apache.org/zh-cn/docs/laster/user_doc/installation/cluster.html) 1.4 小节，配置 hosts 映射和 ssh 打通及修改目录权限.
@ -292,31 +277,36 @@ A： 1，参考官网[部署文档](https://dolphinscheduler.apache.org/zh-cn/do
​ 3，复制正在运行的服务器上的部署目录到新机器的同样的部署目录下
​ 4，到 bin 下，启动 worker server
```
./dolphinscheduler-daemon.sh start worker-server
./dolphinscheduler-daemon.sh start worker-server
```
---
## Q：DolphinScheduler 什么时候发布新版本，同时新旧版本区别，以及如何升级，版本号规范
A：1，Apache 项目的发版流程是通过邮件列表完成的。 你可以订阅 DolphinScheduler 的邮件列表，订阅之后如果有发版，你就可以收到邮件。请参照这篇[指引](https://github.com/apache/dolphinscheduler#get-help)来订阅 DolphinScheduler 的邮件列表。
2，当项目发版的时候，会有发版说明告知具体的变更内容，同时也会有从旧版本升级到新版本的升级文档。
2，当项目发版的时候，会有发版说明告知具体的变更内容，同时也会有从旧版本升级到新版本的升级文档。
3，版本号为 x.y.z, 当 x 增加时代表全新架构的版本。当 y 增加时代表与 y 版本之前的不兼容需要升级脚本或其他人工处理才能升级。当 z 增加代表是 bug 修复，升级完全兼容。无需额外处理。之前有个问题 1.0.2 的升级不兼容 1.0.1 需要升级脚本。
3，版本号为 x.y.z, 当 x 增加时代表全新架构的版本。当 y 增加时代表与 y 版本之前的不兼容需要升级脚本或其他人工处理才能升级。当 z 增加代表是 bug 修复，升级完全兼容。无需额外处理。之前有个问题 1.0.2 的升级不兼容 1.0.1 需要升级脚本。
---
## Q：后续任务在前置任务失败情况下仍旧可以执行
A：在启动工作流的时候，你可以设置失败策略：继续还是失败。
![设置任务失败策略](https://user-images.githubusercontent.com/15833811/80368215-ee378080-88be-11ea-9074-01a33d012b23.png)
---
## Q：工作流模板 DAG、工作流实例、工作任务及实例之间是什么关系 工作流模板 DAG、工作流实例、工作任务及实例之间是什么关系，一个 dag 支持最大并发 100，是指产生 100 个工作流实例并发运行吗？一个 dag 中的任务节点，也有并发数的配置，是指任务也可以并发多个线程运行吗？最大数 100 吗？
A：
1.2.1 version
```
master.properties
设置 master 节点并发执行的最大工作流数
@ -334,6 +324,7 @@ A：
---
## Q：工作组管理页面没有展示按钮
<p align="center">
<img src="https://user-images.githubusercontent.com/39816903/81903776-d8cb9180-95f4-11ea-98cb-94ca1e6a1db5.png" width="60%" />
</p>
@ -342,11 +333,13 @@ A：1.3.0 版本，为了支持 k8s，worker ip 一直变动，因此我们不
---
## Q：为什么不把 mysql 的 jdbc 连接包添加到 docker 镜像里面
A：Mysql jdbc 连接包的许可证和 apache v2 的许可证不兼容，因此它不能被加入到 docker 镜像里面。
---
## Q：当一个任务提交多个 yarn 程序的时候经常失败
<p align="center">
<img src="https://user-images.githubusercontent.com/16174111/81312485-476e9380-90b9-11ea-9aad-ed009db899b1.png" width="60%" />
</p>
@ -355,32 +348,35 @@ A：这个 Bug 在 dev 分支已修复，并加入到需求/待做列表。
---
## Q：Master 服务和 Worker 服务在运行几天之后停止了
<p align="center">
<img src="https://user-images.githubusercontent.com/18378986/81293969-c3101680-90a0-11ea-87e5-ac9f0dd53f5e.png" width="60%" />
</p>
A：会话超时时间太短了，只有 0.3 秒，修改 zookeeper.properties 的配置项：
```
zookeeper.session.timeout=60000
zookeeper.connection.timeout=30000
zookeeper.session.timeout=60000
zookeeper.connection.timeout=30000
```
---
## Q：使用 docker-compose 默认配置启动，显示 zookeeper 错误
<p align="center">
<img src="https://user-images.githubusercontent.com/42579056/80374318-13c98780-88c9-11ea-8d5f-53448b957f02.png" width="60%" />
</p>
A：这个问题在 dev-1.3.0 版本解决了。这个 [pr](https://github.com/apache/dolphinscheduler/pull/2595) 已经解决了这个 bug，主要的改动点：
```
在docker-compose.yml文件中增加zookeeper的环境变量ZOO_4LW_COMMANDS_WHITELIST。
把minLatency,avgLatency and maxLatency的类型从int改成float。
在docker-compose.yml文件中增加zookeeper的环境变量ZOO_4LW_COMMANDS_WHITELIST。
把minLatency,avgLatency and maxLatency的类型从int改成float。
```
---
## Q：界面上显示任务一直运行，结束不了，从日志上看任务实例为空
<p align="center">
<img src="https://user-images.githubusercontent.com/51871547/80302626-b1478d00-87dd-11ea-97d4-08aa2244a6d0.jpg" width="60%" />
</p>
@ -399,49 +395,62 @@ A：这个 [bug](https://github.com/apache/dolphinscheduler/issues/1477) 描述
---
## Q：zk 中注册的 master 信息 ip 地址是 127.0.0.1，而不是配置的域名所对应或者解析的 ip 地址，可能导致不能查看任务日志
A：修复 bug：
```
1、confirm hostname
$hostname
hadoop1
2、hostname -i
127.0.0.1 10.3.57.15
3、edit /etc/hosts,delete hadoop1 from 127.0.0.1 record
$cat /etc/hosts
127.0.0.1 localhost
10.3.57.15 ds1 hadoop1
4、hostname -i
10.3.57.15
1、confirm hostname
$hostname
hadoop1
2、hostname -i
127.0.0.1 10.3.57.15
3、edit /etc/hosts,delete hadoop1 from 127.0.0.1 record
$cat /etc/hosts
127.0.0.1 localhost
10.3.57.15 ds1 hadoop1
4、hostname -i
10.3.57.15
```
hostname 命令返回服务器主机名，hostname -i 返回的是服务器主机名在 /etc/hosts 中所有匹配的ip地址。所以我把 /etc/hosts 中 127.0.0.1 中的主机名删掉，只保留内网 ip 的解析就可以了，没必要把 127.0.0.1 整条注释掉, 只要 hostname 命令返回值在 /etc/hosts 中对应的内网 ip 正确就可以，ds 程序取了第一个值，我理解上 ds 程序不应该用 hostname -i 取值这样有点问题，因为好多公司服务器的主机名都是运维配置的，感觉还是直接取配置文件的域名解析的返回 ip 更准确，或者 znode 中存域名信息而不是 /etc/hosts。
hostname 命令返回服务器主机名，hostname -i 返回的是服务器主机名在 /etc/hosts 中所有匹配的ip地址。所以我把 /etc/hosts 中 127.0.0.1 中的主机名删掉，只保留内网 ip 的解析就可以了，没必要把 127.0.0.1 整条注释掉, 只要 hostname 命令返回值在 /etc/hosts 中对应的内网 ip 正确就可以，ds 程序取了第一个值，我理解上 ds 程序不应该用 hostname -i 取值这样有点问题，因为好多公司服务器的主机名都是运维配置的，感觉还是直接取配置文件的域名解析的返回 ip 更准确，或者 znode 中存域名信息而不是 /etc/hosts。
---
## Q：调度系统设置了一个秒级的任务，导致系统挂掉
A：调度系统不支持秒级任务。
---
## Q：编译前后端代码 (dolphinscheduler-ui) 报错不能下载"https://github.com/sass/node-sass/releases/download/v4.13.1/darwin-x64-72_binding.node"
A：1，cd dolphinscheduler-ui 然后删除 node_modules 目录
```
sudo rm -rf node_modules
```
​ 2，通过 npm.taobao.org 下载 node-sass
```
sudo npm uninstall node-sass
sudo npm i node-sass --sass_binary_site=https://npm.taobao.org/mirrors/node-sass/
```
3，如果步骤 2 报错，请重新构建 node-saas [参考链接](https://dolphinscheduler.apache.org/en-us/development/frontend-development.html)
​ 2，通过 npm.taobao.org 下载 node-sass
```
sudo npm rebuild node-sass
sudo npm uninstall node-sass
sudo npm i node-sass --sass_binary_site=https://npm.taobao.org/mirrors/node-sass/
```
3，如果步骤 2 报错，请重新构建 node-saas [参考链接](https://dolphinscheduler.apache.org/en-us/development/frontend-development.html)
```
sudo npm rebuild node-sass
```
当问题解决之后，如果你不想每次编译都下载这个 node，你可以设置系统环境变量：SASS_BINARY_PATH= /xxx/xxx/xxx/xxx.node。
---
## Q：当使用 mysql 作为 ds 数据库需要如何配置
A：1，修改项目根目录 maven 配置文件，移除 scope 的 test 属性，这样 mysql 的包就可以在其它阶段被加载
```
<dependency>
<groupId>mysql</groupId>
@ -450,31 +459,36 @@ A：1，修改项目根目录 maven 配置文件，移除 scope 的 test 属性
<scope>test<scope>
</dependency>
```
​ 2，修改 application-dao.properties 和 quzrtz.properties 来使用 mysql 驱动
默认驱动是 postgres 主要由于许可证原因。
​ 2，修改 application-dao.properties 和 quzrtz.properties 来使用 mysql 驱动
默认驱动是 postgres 主要由于许可证原因。
---
## Q：shell 任务是如何运行的
A：1，被执行的服务器在哪里配置，以及实际执行的服务器是哪台? 要指定在某个 worker 上去执行，可以在 worker 分组中配置，固定 IP，这样就可以把路径写死。如果配置的 worker 分组有多个 worker，实际执行的服务器由调度决定的，具有随机性。
​ 2，如果是服务器上某个路径的一个 shell 文件，怎么指向这个路径？服务器上某个路径下的 shell 文件，涉及到权限问题，不建议这么做。建议你可以使用资源中心的存储功能，然后在 shell 编辑器里面使用资源引用就可以，系统会帮助你把脚本下载到执行目录下。如果以 hdfs 作为资源中心，在执行的时候，调度器会把依赖的 jar 包，文件等资源拉到 worker 的执行目录上，我这边是 /tmp/escheduler/exec/process，该配置可以在 install.sh 中进行指定。
​ 2，如果是服务器上某个路径的一个 shell 文件，怎么指向这个路径？服务器上某个路径下的 shell 文件，涉及到权限问题，不建议这么做。建议你可以使用资源中心的存储功能，然后在 shell 编辑器里面使用资源引用就可以，系统会帮助你把脚本下载到执行目录下。如果以 hdfs 作为资源中心，在执行的时候，调度器会把依赖的 jar 包，文件等资源拉到 worker 的执行目录上，我这边是 /tmp/escheduler/exec/process，该配置可以在 install.sh 中进行指定。
3，以哪个用户来执行任务？执行任务的时候，调度器会采用 sudo -u 租户的方式去执行，租户是一个 linux 用户。
3，以哪个用户来执行任务？执行任务的时候，调度器会采用 sudo -u 租户的方式去执行，租户是一个 linux 用户。
---
## Q：生产环境部署方式有推荐的最佳实践吗
A：1，如果没有很多任务要运行，出于稳定性考虑我们建议使用 3 个节点，并且最好把 Master/Worder 服务部署在不同的节点。如果你只有一个节点，当然只能把所有的服务部署在同一个节点！通常来说，需要多少节点取决于你的业务，海豚调度系统本身不需要很多的资源。充分测试之后，你们将找到使用较少节点的合适的部署方式。
---
## Q：DEPENDENT 节点
A：1，DEPENDENT 节点实际是没有执行体的，是专门用来配置数据周期依赖逻辑，然后再把执行节点挂载后面，来实现任务间的周期依赖。
---
## Q：如何改变 Master 服务的启动端口
<p align="center">
<img src="https://user-images.githubusercontent.com/8263441/62352160-0f3e9100-b53a-11e9-95ba-3ae3dde49c72.png" width="60%" />
</p>
@ -483,16 +497,19 @@ A：1，修改 application_master.properties 配置文件，例如：server.port
---
## Q：调度任务不能上线
A：1，我们可以成功创建调度任务，并且表 t_scheduler_schedules 中也成功加入了一条记录，但当我点击上线后，前端页面无反应且会把 t_scheduler_schedules 这张表锁定，我测试过将 t_scheduler_schedules 中的 RELEASE_state 字段手动更新为 1 这样前端会显示为上线状态。DS 版本 1.2+ 表名是 t_ds_schedules，其它版本表名是 t_scheduler_schedules。
---
## Q：请问 swagger ui 的地址是什么
A：1， 3.1.0+ 版本地址是 http://apiServerIp:apiServerPort/dolphinscheduler/swagger-ui/index.html, 1.2+ 版本地址是：http://apiServerIp:apiServerPort/dolphinscheduler/swagger-ui/index.html?language=zh_CN&lang=cn，其它版本是 http://apiServerIp:apiServerPort/escheduler/swagger-ui/index.html?language=zh_CN&lang=cn。
---
## Q：前端安装包缺少文件
<p align="center">
<img src="https://user-images.githubusercontent.com/41460919/61437083-d960b080-a96e-11e9-87f1-297ba3aca5e3.png" width="60%" />
</p>
@ -504,33 +521,37 @@ A： 1，用户修改了 api server 配置文件中的![apiServerContextPath](ht
---
## Q：上传比较大的文件卡住
<p align="center">
<img src="https://user-images.githubusercontent.com/21357069/58231400-805b0e80-7d69-11e9-8107-7f37b06a95df.png" width="60%" />
</p>
A：1，编辑 ngnix 配置文件 vi /etc/nginx/nginx.conf，更改上传大小 client_max_body_size 1024m。
​ 2，更新 google chrome 版本到最新版本。
​ 2，更新 google chrome 版本到最新版本。
---
## Q：创建 spark 数据源，点击“测试连接”，系统回退回到登入页面
A：1，edit /etc/nginx/conf.d/escheduler.conf
```
proxy_connect_timeout 300s;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
proxy_connect_timeout 300s;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
```
---
## Q：工作流依赖
A：1，目前是按照自然天来判断，上月末：判断时间是工作流 A start_time/scheduler_time between '2019-05-31 00:00:00' and '2019-05-31 23:59:59'。上月：是判断上个月从 1 号到月末每天都要有完成的A实例。上周： 上周 7 天都要有完成的 A 实例。前两天： 判断昨天和前天，两天都要有完成的 A 实例。
---
## Q：DS 后端接口文档
A：1，http://106.75.43.194:8888/dolphinscheduler/swagger-ui/index.html?language=zh_CN&lang=zh。
A：1，http://106.75.43.194:8888/dolphinscheduler/swagger-ui/index.html?language=zh_CN&lang=zh。
## dolphinscheduler 在运行过程中，ip 地址获取错误的问题
@ -582,13 +603,14 @@ sed -i 's/Defaults requirett/#Defaults requirett/g' /etc/sudoers
---
## Q：Yarn多集群支持
A：将Worker节点分别部署至多个Yarn集群，步骤如下（例如AWS EMR）：
1. 将 Worker 节点部署至 EMR 集群的 Master 节点
1. 将 Worker 节点部署至 EMR 集群的 Master 节点
2. 将 `conf/common.properties` 中的 `yarn.application.status.address` 修改为当前集群的 Yarn 的信息
2. 将 `conf/common.properties` 中的 `yarn.application.status.address` 修改为当前集群的 Yarn 的信息
3. 通过 `bin/dolphinscheduler-daemon.sh start worker-server` 启动 worker-server
3. 通过 `bin/dolphinscheduler-daemon.sh start worker-server` 启动 worker-server
---
@ -683,6 +705,7 @@ DELETE FROM t_ds_task_definition_log WHERE id IN
## Q：使用Postgresql数据库从2.0.1升级至2.0.5更新失败
A：在数据库中执行以下SQL即可完成修复:
```SQL
update t_ds_version set version='2.0.1';
```

6
docs/docs/zh/guide/open-api.md
View File

@ -32,9 +32,9 @@
3. 打开 Postman，填写接口地址，并在 Headers 中填写 Token，发送请求后即可查看结果
```
token: 刚刚生成的 Token
```
```
token: 刚刚生成的 Token
```
![api-test](../../../img/new_ui/dev/open-api/api_test.png)

1
dolphinscheduler-alert/dolphinscheduler-alert-server/src/test/java/org/apache/dolphinscheduler/alert/runner/AlertSenderServiceTest.java
View File

@ -92,6 +92,7 @@ public class AlertSenderServiceTest {
List<AlertPluginInstance> alertInstanceList = new ArrayList<>();
AlertPluginInstance alertPluginInstance = new AlertPluginInstance(
pluginDefineId, pluginInstanceParams, pluginInstanceName);
alertPluginInstance.setId(1);
alertInstanceList.add(alertPluginInstance);
when(alertDao.listInstanceByAlertGroupId(1)).thenReturn(alertInstanceList);

2
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/LoginController.java
View File

@ -32,8 +32,8 @@ import org.apache.dolphinscheduler.dao.entity.User;
import springfox.documentation.annotations.ApiIgnore;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpStatus;
import java.util.Map;

75
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/ClusterDto.java
View File

@ -20,9 +20,12 @@ package org.apache.dolphinscheduler.api.dto;
import java.util.Date;
import java.util.List;
import lombok.Data;
/**
* ClusterDto
*/
@Data
public class ClusterDto {
private int id;
@ -54,76 +57,4 @@ public class ClusterDto {
private Date createTime;
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Long getCode() {
return this.code;
}
public void setCode(Long code) {
this.code = code;
}
public String getConfig() {
return this.config;
}
public void setConfig(String config) {
this.config = config;
}
public String getDescription() {
return this.description;
}
public void setDescription(String description) {
this.description = description;
}
public Integer getOperator() {
return this.operator;
}
public void setOperator(Integer operator) {
this.operator = operator;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public List<String> getProcessDefinitions() {
return processDefinitions;
}
public void setProcessDefinitions(List<String> processDefinitions) {
this.processDefinitions = processDefinitions;
}
}

80
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/EnvironmentDto.java
View File

@ -20,12 +20,12 @@ package org.apache.dolphinscheduler.api.dto;
import java.util.Date;
import java.util.List;
/**
* EnvironmentDto
*/
import lombok.Data;
@Data
public class EnvironmentDto {
private int id;
private Integer id;
/**
* environment code
@ -54,76 +54,4 @@ public class EnvironmentDto {
private Date createTime;
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Long getCode() {
return this.code;
}
public void setCode(Long code) {
this.code = code;
}
public String getConfig() {
return this.config;
}
public void setConfig(String config) {
this.config = config;
}
public String getDescription() {
return this.description;
}
public void setDescription(String description) {
this.description = description;
}
public Integer getOperator() {
return this.operator;
}
public void setOperator(Integer operator) {
this.operator = operator;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public List<String> getWorkerGroups() {
return workerGroups;
}
public void setWorkerGroups(List<String> workerGroups) {
this.workerGroups = workerGroups;
}
}

107
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/resources/ResourceComponent.java
View File

@ -22,15 +22,18 @@ import org.apache.dolphinscheduler.spi.enums.ResourceType;
import java.util.ArrayList;
import java.util.List;
import lombok.Data;
import lombok.NoArgsConstructor;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
/**
* resource component
*/
@JsonPropertyOrder({"id","pid","name","fullName","description","isDirctory","children","type"})
@Data
@NoArgsConstructor
@JsonPropertyOrder({"id", "pid", "name", "fullName", "description", "isDirctory", "children", "type"})
public abstract class ResourceComponent {
public ResourceComponent() {
}
public ResourceComponent(int id, int pid, String name, String fullName, String description, boolean isDirctory) {
this.id = id;
@ -39,11 +42,10 @@ public abstract class ResourceComponent {
this.fullName = fullName;
this.description = description;
this.isDirctory = isDirctory;
int directoryFlag = isDirctory ? 1:0;
this.idValue = String.format("%s_%s",id,directoryFlag);
int directoryFlag = isDirctory ? 1 : 0;
this.idValue = String.format("%s_%s", id, directoryFlag);
}
/**
* id
*/
@ -89,97 +91,12 @@ public abstract class ResourceComponent {
* add resource component
* @param resourceComponent resource component
*/
public void add(ResourceComponent resourceComponent){
public void add(ResourceComponent resourceComponent) {
children.add(resourceComponent);
}
public String getName(){
return this.name;
public void setIdValue(int id, boolean isDirctory) {
int directoryFlag = isDirctory ? 1 : 0;
this.idValue = String.format("%s_%s", id, directoryFlag);
}
public String getDescription(){
return this.description;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getPid() {
return pid;
}
public void setPid(int pid) {
this.pid = pid;
}
public void setName(String name) {
this.name = name;
}
public String getFullName() {
return fullName;
}
public void setFullName(String fullName) {
this.fullName = fullName;
}
public void setDescription(String description) {
this.description = description;
}
public boolean isDirctory() {
return isDirctory;
}
public void setDirctory(boolean dirctory) {
isDirctory = dirctory;
}
public String getIdValue() {
return idValue;
}
public void setIdValue(int id,boolean isDirctory) {
int directoryFlag = isDirctory ? 1:0;
this.idValue = String.format("%s_%s",id,directoryFlag);
}
public ResourceType getType() {
return type;
}
public void setType(ResourceType type) {
this.type = type;
}
public List<ResourceComponent> getChildren() {
return children;
}
public void setChildren(List<ResourceComponent> children) {
this.children = children;
}
@Override
public String toString() {
return "ResourceComponent{" +
"id=" + id +
", pid=" + pid +
", name='" + name + '\'' +
", currentDir='" + currentDir + '\'' +
", fullName='" + fullName + '\'' +
", description='" + description + '\'' +
", isDirctory=" + isDirctory +
", idValue='" + idValue + '\'' +
", type=" + type +
", children=" + children +
'}';
}
}

95
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/dto/treeview/Instance.java
View File

@ -19,12 +19,12 @@ package org.apache.dolphinscheduler.api.dto.treeview;
import java.util.Date;
/**
* Instance
*/
import lombok.Data;
@Data
public class Instance {
private int id;
private Integer id;
/**
* node name
@ -56,7 +56,6 @@ public class Instance {
*/
private Date endTime;
/**
* node running on which host
*/
@ -79,7 +78,8 @@ public class Instance {
this.type = type;
}
public Instance(int id, String name, long code, String type, String state, Date startTime, Date endTime, String host, String duration, long subflowCode) {
public Instance(int id, String name, long code, String type, String state, Date startTime, Date endTime,
String host, String duration, long subflowCode) {
this.id = id;
this.name = name;
this.code = code;
@ -92,87 +92,8 @@ public class Instance {
this.subflowCode = subflowCode;
}
public Instance(int id, String name, long code, String type, String state, Date startTime, Date endTime, String host, String duration) {
public Instance(int id, String name, long code, String type, String state, Date startTime, Date endTime,
String host, String duration) {
this(id, name, code, type, state, startTime, endTime, host, duration, 0);
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public long getCode() {
return code;
}
public void setCode(long code) {
this.code = code;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getState() {
return state;
}
public void setState(String state) {
this.state = state;
}
public Date getStartTime() {
return startTime;
}
public void setStartTime(Date startTime) {
this.startTime = startTime;
}
public Date getEndTime() {
return endTime;
}
public void setEndTime(Date endTime) {
this.endTime = endTime;
}
public String getHost() {
return host;
}
public void setHost(String host) {
this.host = host;
}
public String getDuration() {
return duration;
}
public void setDuration(String duration) {
this.duration = duration;
}
public long getSubflowCode() {
return subflowCode;
}
public void setSubflowCode(long subflowCode) {
this.subflowCode = subflowCode;
}
}

10
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/interceptor/LoginHandlerInterceptor.java
View File

@ -25,8 +25,10 @@ import org.apache.dolphinscheduler.common.thread.ThreadLocalContext;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.UserMapper;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpStatus;
import java.util.Date;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@ -37,12 +39,11 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;
import java.util.Date;
/**
* login interceptor, must log in first
*/
public class LoginHandlerInterceptor implements HandlerInterceptor {
private static final Logger logger = LoggerFactory.getLogger(LoginHandlerInterceptor.class);
@Autowired
@ -93,7 +94,8 @@ public class LoginHandlerInterceptor implements HandlerInterceptor {
}
@Override
public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler,
ModelAndView modelAndView) throws Exception {
ThreadLocalContext.getTimezoneThreadLocal().remove();
}
}

14
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ClusterServiceImpl.java
View File

@ -248,8 +248,8 @@ public class ClusterServiceImpl extends BaseServiceImpl implements ClusterServic
return result;
}
Integer relatedNamespaceNumber = k8sNamespaceMapper
.selectCount(new QueryWrapper<K8sNamespace>().lambda().eq(K8sNamespace::getClusterCode, code));
Long relatedNamespaceNumber = k8sNamespaceMapper
.selectCount(new QueryWrapper<K8sNamespace>().lambda().eq(K8sNamespace::getClusterCode, code));
if (relatedNamespaceNumber > 0) {
putMsg(result, Status.DELETE_CLUSTER_RELATED_NAMESPACE_EXISTS);
@ -265,7 +265,6 @@ public class ClusterServiceImpl extends BaseServiceImpl implements ClusterServic
return result;
}
/**
* update cluster
*
@ -283,7 +282,7 @@ public class ClusterServiceImpl extends BaseServiceImpl implements ClusterServic
return result;
}
if(checkDescriptionLength(desc)){
if (checkDescriptionLength(desc)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -306,7 +305,7 @@ public class ClusterServiceImpl extends BaseServiceImpl implements ClusterServic
}
if (!Constants.K8S_LOCAL_TEST_CLUSTER_CODE.equals(clusterExist.getCode())
&& !config.equals(ClusterConfUtils.getK8sConfig(clusterExist.getConfig()))) {
&& !config.equals(ClusterConfUtils.getK8sConfig(clusterExist.getConfig()))) {
try {
k8sManager.getAndUpdateK8sClient(code, true);
} catch (RemotingException e) {
@ -315,12 +314,12 @@ public class ClusterServiceImpl extends BaseServiceImpl implements ClusterServic
}
}
//update cluster
// update cluster
clusterExist.setConfig(config);
clusterExist.setName(name);
clusterExist.setDescription(desc);
clusterMapper.updateById(clusterExist);
//need not update relation
// need not update relation
putMsg(result, Status.SUCCESS);
return result;
@ -366,4 +365,3 @@ public class ClusterServiceImpl extends BaseServiceImpl implements ClusterServic
}
}

80
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/DqRuleServiceImpl.java
View File

@ -17,19 +17,15 @@
package org.apache.dolphinscheduler.api.service.impl;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.commons.collections4.CollectionUtils;
import static org.apache.dolphinscheduler.common.Constants.DATA_LIST;
import static org.apache.dolphinscheduler.spi.utils.Constants.CHANGE;
import static org.apache.dolphinscheduler.spi.utils.Constants.SMALL;
import org.apache.dolphinscheduler.api.dto.RuleDefinition;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.service.DqRuleService;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.enums.AuthorizationType;
import org.apache.dolphinscheduler.common.utils.DateUtils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.dao.entity.DataSource;
@ -57,10 +53,8 @@ import org.apache.dolphinscheduler.spi.params.input.InputParam;
import org.apache.dolphinscheduler.spi.params.input.InputParamProps;
import org.apache.dolphinscheduler.spi.params.select.SelectParam;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.apache.commons.collections4.CollectionUtils;
import java.util.ArrayList;
import java.util.Collections;
@ -70,9 +64,17 @@ import java.util.List;
import java.util.Map;
import java.util.Objects;
import static org.apache.dolphinscheduler.common.Constants.DATA_LIST;
import static org.apache.dolphinscheduler.spi.utils.Constants.CHANGE;
import static org.apache.dolphinscheduler.spi.utils.Constants.SMALL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
/**
* DqRuleServiceImpl
@ -136,9 +138,9 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
if (CollectionUtils.isNotEmpty(dataSourceList)) {
options = new ArrayList<>();
for (DataSource dataSource: dataSourceList) {
for (DataSource dataSource : dataSourceList) {
ParamsOptions childrenOption =
new ParamsOptions(dataSource.getName(),dataSource.getId(),false);
new ParamsOptions(dataSource.getName(), dataSource.getId(), false);
options.add(childrenOption);
}
}
@ -158,7 +160,7 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
Integer pageNo,
Integer pageSize) {
Result result = new Result();
Date start = null;
Date end = null;
try {
@ -194,7 +196,7 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
DqRuleUtils.transformInputEntry(dqRuleInputEntryMapper.getRuleInputEntryList(dqRule.getId()));
List<DqRuleExecuteSql> ruleExecuteSqlList = dqRuleExecuteSqlMapper.getExecuteSqlList(dqRule.getId());
RuleDefinition ruleDefinition = new RuleDefinition(ruleInputEntryList,ruleExecuteSqlList);
RuleDefinition ruleDefinition = new RuleDefinition(ruleInputEntryList, ruleExecuteSqlList);
dqRule.setRuleJson(JSONUtils.toJsonString(ruleDefinition));
});
@ -211,7 +213,7 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
List<PluginParams> params = new ArrayList<>();
for (DqRuleInputEntry inputEntry : ruleInputEntryList) {
if (Boolean.TRUE.equals(inputEntry.getShow())) {
if (Boolean.TRUE.equals(inputEntry.getIsShow())) {
switch (Objects.requireNonNull(FormType.of(inputEntry.getType()))) {
case INPUT:
params.add(getInputParam(inputEntry));
@ -254,14 +256,14 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
paramProps.setRows(1);
return InputParam
.newBuilder(inputEntry.getField(),inputEntry.getTitle())
.newBuilder(inputEntry.getField(), inputEntry.getTitle())
.addValidate(Validate.newBuilder()
.setRequired(inputEntry.getValidate())
.setRequired(inputEntry.getIsValidate())
.build())
.setProps(paramProps)
.setValue(inputEntry.getValue())
.setPlaceholder(inputEntry.getPlaceholder())
.setEmit(Boolean.TRUE.equals(inputEntry.getEmit()) ? Collections.singletonList(CHANGE) : null)
.setEmit(Boolean.TRUE.equals(inputEntry.getIsEmit()) ? Collections.singletonList(CHANGE) : null)
.build();
}
@ -278,18 +280,19 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
case DATASOURCE_TYPE:
options = new ArrayList<>();
ParamsOptions paramsOptions = null;
for (DbType dbtype: DbType.values()) {
paramsOptions = new ParamsOptions(dbtype.name(),dbtype.getCode(),false);
for (DbType dbtype : DbType.values()) {
paramsOptions = new ParamsOptions(dbtype.name(), dbtype.getCode(), false);
options.add(paramsOptions);
}
break;
case COMPARISON_TYPE:
options = new ArrayList<>();
ParamsOptions comparisonOptions = null;
List<DqComparisonType> list = dqComparisonTypeMapper.selectList(new QueryWrapper<DqComparisonType>().orderByAsc("id"));
List<DqComparisonType> list =
dqComparisonTypeMapper.selectList(new QueryWrapper<DqComparisonType>().orderByAsc("id"));
for (DqComparisonType type: list) {
comparisonOptions = new ParamsOptions(type.getType(), type.getId(),false);
for (DqComparisonType type : list) {
comparisonOptions = new ParamsOptions(type.getType(), type.getId(), false);
options.add(comparisonOptions);
}
break;
@ -298,12 +301,12 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
}
return SelectParam
.newBuilder(inputEntry.getField(),inputEntry.getTitle())
.newBuilder(inputEntry.getField(), inputEntry.getTitle())
.setOptions(options)
.setValue(inputEntry.getValue())
.setSize(SMALL)
.setPlaceHolder(inputEntry.getPlaceholder())
.setEmit(Boolean.TRUE.equals(inputEntry.getEmit()) ? Collections.singletonList(CHANGE) : null)
.setEmit(Boolean.TRUE.equals(inputEntry.getIsEmit()) ? Collections.singletonList(CHANGE) : null)
.build();
}
@ -315,25 +318,26 @@ public class DqRuleServiceImpl extends BaseServiceImpl implements DqRuleService
paramProps.setRows(2);
return InputParam
.newBuilder(inputEntry.getField(),inputEntry.getTitle())
.newBuilder(inputEntry.getField(), inputEntry.getTitle())
.addValidate(Validate.newBuilder()
.setRequired(inputEntry.getValidate())
.build())
.setRequired(inputEntry.getIsValidate())
.build())
.setProps(paramProps)
.setValue(inputEntry.getValue())
.setPlaceholder(inputEntry.getPlaceholder())
.setEmit(Boolean.TRUE.equals(inputEntry.getEmit()) ? Collections.singletonList(CHANGE) : null)
.setEmit(Boolean.TRUE.equals(inputEntry.getIsEmit()) ? Collections.singletonList(CHANGE) : null)
.build();
}
private GroupParam getGroupParam(DqRuleInputEntry inputEntry) {
return GroupParam
.newBuilder(inputEntry.getField(),inputEntry.getTitle())
.newBuilder(inputEntry.getField(), inputEntry.getTitle())
.addValidate(Validate.newBuilder()
.setRequired(inputEntry.getValidate())
.setRequired(inputEntry.getIsValidate())
.build())
.setProps(new GroupParamsProps().setRules(JSONUtils.toList(inputEntry.getOptions(),PluginParams.class)).setFontSize(20))
.setEmit(Boolean.TRUE.equals(inputEntry.getEmit()) ? Collections.singletonList(CHANGE) : null)
.setProps(new GroupParamsProps().setRules(JSONUtils.toList(inputEntry.getOptions(), PluginParams.class))
.setFontSize(20))
.setEmit(Boolean.TRUE.equals(inputEntry.getIsEmit()) ? Collections.singletonList(CHANGE) : null)
.build();
}
}

94
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/EnvironmentServiceImpl.java
View File

@ -17,6 +17,8 @@
package org.apache.dolphinscheduler.api.service.impl;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.*;
import org.apache.dolphinscheduler.api.dto.EnvironmentDto;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.service.EnvironmentService;
@ -64,8 +66,6 @@ import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.fasterxml.jackson.core.type.TypeReference;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.*;
/**
* task definition service impl
*/
@ -94,17 +94,18 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
*/
@Override
@Transactional
public Map<String, Object> createEnvironment(User loginUser, String name, String config, String desc, String workerGroups) {
public Map<String, Object> createEnvironment(User loginUser, String name, String config, String desc,
String workerGroups) {
Map<String, Object> result = new HashMap<>();
if (!canOperatorPermissions(loginUser, null, AuthorizationType.ENVIRONMENT, ENVIRONMENT_CREATE)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
if(checkDescriptionLength(desc)){
if (checkDescriptionLength(desc)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
Map<String, Object> checkResult = checkParams(name,config,workerGroups);
Map<String, Object> checkResult = checkParams(name, config, workerGroups);
if (checkResult.get(Constants.STATUS) != Status.SUCCESS) {
return checkResult;
}
@ -136,7 +137,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
if (environmentMapper.insert(env) > 0) {
if (!StringUtils.isEmpty(workerGroups)) {
List<String> workerGroupList = JSONUtils.parseObject(workerGroups, new TypeReference<List<String>>(){});
List<String> workerGroupList = JSONUtils.parseObject(workerGroups, new TypeReference<List<String>>() {
});
if (CollectionUtils.isNotEmpty(workerGroupList)) {
workerGroupList.stream().forEach(workerGroup -> {
if (!StringUtils.isEmpty(workerGroup)) {
@ -153,7 +155,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
}
result.put(Constants.DATA_LIST, env.getCode());
putMsg(result, Status.SUCCESS);
permissionPostHandle(AuthorizationType.ENVIRONMENT, loginUser.getId(), Collections.singletonList(env.getId()), logger);
permissionPostHandle(AuthorizationType.ENVIRONMENT, loginUser.getId(),
Collections.singletonList(env.getId()), logger);
} else {
putMsg(result, Status.CREATE_ENVIRONMENT_ERROR);
}
@ -178,7 +181,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
if (loginUser.getUserType().equals(UserType.ADMIN_USER)) {
environmentIPage = environmentMapper.queryEnvironmentListPaging(page, searchVal);
} else {
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.ENVIRONMENT, loginUser.getId(), logger);
Set<Integer> ids = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(AuthorizationType.ENVIRONMENT, loginUser.getId(), logger);
if (ids.isEmpty()) {
result.setData(pageInfo);
putMsg(result, Status.SUCCESS);
@ -191,12 +195,13 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
if (CollectionUtils.isNotEmpty(environmentIPage.getRecords())) {
Map<Long, List<String>> relationMap = relationMapper.selectList(null).stream()
.collect(Collectors.groupingBy(EnvironmentWorkerGroupRelation::getEnvironmentCode,Collectors.mapping(EnvironmentWorkerGroupRelation::getWorkerGroup,Collectors.toList())));
.collect(Collectors.groupingBy(EnvironmentWorkerGroupRelation::getEnvironmentCode,
Collectors.mapping(EnvironmentWorkerGroupRelation::getWorkerGroup, Collectors.toList())));
List<EnvironmentDto> dtoList = environmentIPage.getRecords().stream().map(environment -> {
EnvironmentDto dto = new EnvironmentDto();
BeanUtils.copyProperties(environment,dto);
List<String> workerGroups = relationMap.getOrDefault(environment.getCode(),new ArrayList<String>());
BeanUtils.copyProperties(environment, dto);
List<String> workerGroups = relationMap.getOrDefault(environment.getCode(), new ArrayList<String>());
dto.setWorkerGroups(workerGroups);
return dto;
}).collect(Collectors.toList());
@ -219,31 +224,33 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
*/
@Override
public Map<String, Object> queryAllEnvironmentList(User loginUser) {
Map<String,Object> result = new HashMap<>();
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.ENVIRONMENT, loginUser.getId(), logger);
Map<String, Object> result = new HashMap<>();
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.ENVIRONMENT,
loginUser.getId(), logger);
if (ids.isEmpty()) {
result.put(Constants.DATA_LIST, Collections.emptyList());
putMsg(result,Status.SUCCESS);
putMsg(result, Status.SUCCESS);
return result;
}
List<Environment> environmentList = environmentMapper.selectBatchIds(ids);
if (CollectionUtils.isNotEmpty(environmentList)) {
Map<Long, List<String>> relationMap = relationMapper.selectList(null).stream()
.collect(Collectors.groupingBy(EnvironmentWorkerGroupRelation::getEnvironmentCode,Collectors.mapping(EnvironmentWorkerGroupRelation::getWorkerGroup,Collectors.toList())));
.collect(Collectors.groupingBy(EnvironmentWorkerGroupRelation::getEnvironmentCode,
Collectors.mapping(EnvironmentWorkerGroupRelation::getWorkerGroup, Collectors.toList())));
List<EnvironmentDto> dtoList = environmentList.stream().map(environment -> {
EnvironmentDto dto = new EnvironmentDto();
BeanUtils.copyProperties(environment,dto);
List<String> workerGroups = relationMap.getOrDefault(environment.getCode(),new ArrayList<String>());
BeanUtils.copyProperties(environment, dto);
List<String> workerGroups = relationMap.getOrDefault(environment.getCode(), new ArrayList<String>());
dto.setWorkerGroups(workerGroups);
return dto;
}).collect(Collectors.toList());
result.put(Constants.DATA_LIST,dtoList);
result.put(Constants.DATA_LIST, dtoList);
} else {
result.put(Constants.DATA_LIST, new ArrayList<>());
}
putMsg(result,Status.SUCCESS);
putMsg(result, Status.SUCCESS);
return result;
}
@ -266,7 +273,7 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
.collect(Collectors.toList());
EnvironmentDto dto = new EnvironmentDto();
BeanUtils.copyProperties(env,dto);
BeanUtils.copyProperties(env, dto);
dto.setWorkerGroups(workerGroups);
result.put(Constants.DATA_LIST, dto);
putMsg(result, Status.SUCCESS);
@ -292,7 +299,7 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
.collect(Collectors.toList());
EnvironmentDto dto = new EnvironmentDto();
BeanUtils.copyProperties(env,dto);
BeanUtils.copyProperties(env, dto);
dto.setWorkerGroups(workerGroups);
result.put(Constants.DATA_LIST, dto);
putMsg(result, Status.SUCCESS);
@ -310,13 +317,13 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
@Override
public Map<String, Object> deleteEnvironmentByCode(User loginUser, Long code) {
Map<String, Object> result = new HashMap<>();
if (!canOperatorPermissions(loginUser,null, AuthorizationType.ENVIRONMENT,ENVIRONMENT_DELETE)) {
if (!canOperatorPermissions(loginUser, null, AuthorizationType.ENVIRONMENT, ENVIRONMENT_DELETE)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
Integer relatedTaskNumber = taskDefinitionMapper
.selectCount(new QueryWrapper<TaskDefinition>().lambda().eq(TaskDefinition::getEnvironmentCode,code));
Long relatedTaskNumber = taskDefinitionMapper
.selectCount(new QueryWrapper<TaskDefinition>().lambda().eq(TaskDefinition::getEnvironmentCode, code));
if (relatedTaskNumber > 0) {
putMsg(result, Status.DELETE_ENVIRONMENT_RELATED_TASK_EXISTS);
@ -327,7 +334,7 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
if (delete > 0) {
relationMapper.delete(new QueryWrapper<EnvironmentWorkerGroupRelation>()
.lambda()
.eq(EnvironmentWorkerGroupRelation::getEnvironmentCode,code));
.eq(EnvironmentWorkerGroupRelation::getEnvironmentCode, code));
putMsg(result, Status.SUCCESS);
} else {
putMsg(result, Status.DELETE_ENVIRONMENT_ERROR);
@ -347,18 +354,19 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
*/
@Transactional
@Override
public Map<String, Object> updateEnvironmentByCode(User loginUser, Long code, String name, String config, String desc, String workerGroups) {
public Map<String, Object> updateEnvironmentByCode(User loginUser, Long code, String name, String config,
String desc, String workerGroups) {
Map<String, Object> result = new HashMap<>();
if (!canOperatorPermissions(loginUser,null, AuthorizationType.ENVIRONMENT,ENVIRONMENT_UPDATE)) {
if (!canOperatorPermissions(loginUser, null, AuthorizationType.ENVIRONMENT, ENVIRONMENT_UPDATE)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
Map<String, Object> checkResult = checkParams(name,config,workerGroups);
Map<String, Object> checkResult = checkParams(name, config, workerGroups);
if (checkResult.get(Constants.STATUS) != Status.SUCCESS) {
return checkResult;
}
if(checkDescriptionLength(desc)){
if (checkDescriptionLength(desc)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -371,7 +379,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
Set<String> workerGroupSet;
if (!StringUtils.isEmpty(workerGroups)) {
workerGroupSet = JSONUtils.parseObject(workerGroups, new TypeReference<Set<String>>() {});
workerGroupSet = JSONUtils.parseObject(workerGroups, new TypeReference<Set<String>>() {
});
} else {
workerGroupSet = new TreeSet<>();
}
@ -382,8 +391,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
.map(item -> item.getWorkerGroup())
.collect(Collectors.toSet());
Set<String> deleteWorkerGroupSet = SetUtils.difference(existWorkerGroupSet,workerGroupSet).toSet();
Set<String> addWorkerGroupSet = SetUtils.difference(workerGroupSet,existWorkerGroupSet).toSet();
Set<String> deleteWorkerGroupSet = SetUtils.difference(existWorkerGroupSet, workerGroupSet).toSet();
Set<String> addWorkerGroupSet = SetUtils.difference(workerGroupSet, existWorkerGroupSet).toSet();
// verify whether the relation of this environment and worker groups can be adjusted
checkResult = checkUsedEnvironmentWorkerGroupRelation(deleteWorkerGroupSet, name, code);
@ -399,7 +408,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
env.setOperator(loginUser.getId());
env.setUpdateTime(new Date());
int update = environmentMapper.update(env, new UpdateWrapper<Environment>().lambda().eq(Environment::getCode, code));
int update =
environmentMapper.update(env, new UpdateWrapper<Environment>().lambda().eq(Environment::getCode, code));
if (update > 0) {
deleteWorkerGroupSet.stream().forEach(key -> {
if (StringUtils.isNotEmpty(key)) {
@ -427,8 +437,6 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
return result;
}
/**
* verify environment name
*
@ -454,17 +462,20 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
return result;
}
private Map<String, Object> checkUsedEnvironmentWorkerGroupRelation(Set<String> deleteKeySet,String environmentName, Long environmentCode) {
private Map<String, Object> checkUsedEnvironmentWorkerGroupRelation(Set<String> deleteKeySet,
String environmentName, Long environmentCode) {
Map<String, Object> result = new HashMap<>();
for (String workerGroup : deleteKeySet) {
List<TaskDefinition> taskDefinitionList = taskDefinitionMapper
.selectList(new QueryWrapper<TaskDefinition>().lambda()
.eq(TaskDefinition::getEnvironmentCode,environmentCode)
.eq(TaskDefinition::getWorkerGroup,workerGroup));
.eq(TaskDefinition::getEnvironmentCode, environmentCode)
.eq(TaskDefinition::getWorkerGroup, workerGroup));
if (Objects.nonNull(taskDefinitionList) && taskDefinitionList.size() != 0) {
Set<String> collect = taskDefinitionList.stream().map(TaskDefinition::getName).collect(Collectors.toSet());
putMsg(result, Status.UPDATE_ENVIRONMENT_WORKER_GROUP_RELATION_ERROR,workerGroup,environmentName, collect);
Set<String> collect =
taskDefinitionList.stream().map(TaskDefinition::getName).collect(Collectors.toSet());
putMsg(result, Status.UPDATE_ENVIRONMENT_WORKER_GROUP_RELATION_ERROR, workerGroup, environmentName,
collect);
return result;
}
}
@ -483,7 +494,8 @@ public class EnvironmentServiceImpl extends BaseServiceImpl implements Environme
return result;
}
if (!StringUtils.isEmpty(workerGroups)) {
List<String> workerGroupList = JSONUtils.parseObject(workerGroups, new TypeReference<List<String>>(){});
List<String> workerGroupList = JSONUtils.parseObject(workerGroups, new TypeReference<List<String>>() {
});
if (Objects.isNull(workerGroupList)) {
putMsg(result, Status.ENVIRONMENT_WORKER_GROUPS_IS_INVALID);
return result;

32
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java
View File

@ -35,7 +35,18 @@ import org.apache.dolphinscheduler.api.service.ExecutorService;
import org.apache.dolphinscheduler.api.service.MonitorService;
import org.apache.dolphinscheduler.api.service.ProjectService;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.*;
import org.apache.dolphinscheduler.common.enums.CommandType;
import org.apache.dolphinscheduler.common.enums.ComplementDependentMode;
import org.apache.dolphinscheduler.common.enums.CycleEnum;
import org.apache.dolphinscheduler.common.enums.FailureStrategy;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.common.enums.Priority;
import org.apache.dolphinscheduler.common.enums.ReleaseState;
import org.apache.dolphinscheduler.common.enums.RunMode;
import org.apache.dolphinscheduler.common.enums.TaskDependType;
import org.apache.dolphinscheduler.common.enums.TaskGroupQueueStatus;
import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.common.enums.WorkflowExecutionStatus;
import org.apache.dolphinscheduler.common.model.Server;
import org.apache.dolphinscheduler.common.utils.DateUtils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
@ -59,9 +70,9 @@ import org.apache.dolphinscheduler.dao.mapper.TaskGroupQueueMapper;
import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
import org.apache.dolphinscheduler.plugin.task.api.TaskConstants;
import org.apache.dolphinscheduler.remote.command.TaskExecuteStartCommand;
import org.apache.dolphinscheduler.remote.command.WorkflowStateEventChangeCommand;
import org.apache.dolphinscheduler.remote.command.WorkflowExecutingDataRequestCommand;
import org.apache.dolphinscheduler.remote.command.WorkflowExecutingDataResponseCommand;
import org.apache.dolphinscheduler.remote.command.WorkflowStateEventChangeCommand;
import org.apache.dolphinscheduler.remote.dto.WorkflowExecuteDto;
import org.apache.dolphinscheduler.remote.processor.StateEventCallbackService;
import org.apache.dolphinscheduler.remote.utils.Host;
@ -329,8 +340,8 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
/**
* do action to process instance：pause, stop, repeat, recover from pause, recover from stop，rerun failed task
*
* @param loginUser login user
* @param projectCode project code
@ -1027,12 +1038,14 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
}
@Override
public Map<String, Object> execStreamTaskInstance(User loginUser, long projectCode, long taskDefinitionCode, int taskDefinitionVersion,
int warningGroupId, String workerGroup, Long environmentCode, Map<String, String> startParams, int dryRun) {
public Map<String, Object> execStreamTaskInstance(User loginUser, long projectCode, long taskDefinitionCode,
int taskDefinitionVersion,
int warningGroupId, String workerGroup, Long environmentCode,
Map<String, String> startParams, int dryRun) {
Project project = projectMapper.queryByCode(projectCode);
//check user access for project
// check user access for project
Map<String, Object> result =
projectService.checkProjectAndAuth(loginUser, project, projectCode, WORKFLOW_START);
projectService.checkProjectAndAuth(loginUser, project, projectCode, WORKFLOW_START);
if (result.get(Constants.STATUS) != Status.SUCCESS) {
return result;
}
@ -1058,7 +1071,8 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
taskExecuteStartCommand.setStartParams(startParams);
taskExecuteStartCommand.setDryRun(dryRun);
org.apache.dolphinscheduler.remote.command.Command response = stateEventCallbackService.sendSync(host, taskExecuteStartCommand.convert2Command());
org.apache.dolphinscheduler.remote.command.Command response =
stateEventCallbackService.sendSync(host, taskExecuteStartCommand.convert2Command());
if (response != null) {
putMsg(result, Status.SUCCESS);
} else {

58
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessInstanceServiceImpl.java
View File

@ -17,15 +17,23 @@
package org.apache.dolphinscheduler.api.service.impl;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.*;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.INSTANCE_DELETE;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.INSTANCE_UPDATE;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.WORKFLOW_INSTANCE;
import static org.apache.dolphinscheduler.common.Constants.*;
import static org.apache.dolphinscheduler.common.Constants.DATA_LIST;
import static org.apache.dolphinscheduler.common.Constants.DEPENDENT_SPLIT;
import static org.apache.dolphinscheduler.common.Constants.GLOBAL_PARAMS;
import static org.apache.dolphinscheduler.common.Constants.LOCAL_PARAMS;
import static org.apache.dolphinscheduler.common.Constants.PROCESS_INSTANCE_STATE;
import static org.apache.dolphinscheduler.common.Constants.TASK_LIST;
import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.TASK_TYPE_DEPENDENT;
import org.apache.dolphinscheduler.api.dto.gantt.GanttDto;
import org.apache.dolphinscheduler.api.dto.gantt.Task;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.exceptions.ServiceException;
import org.apache.dolphinscheduler.api.service.*;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.Constants;
@ -38,8 +46,25 @@ import org.apache.dolphinscheduler.common.utils.DateUtils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.ParameterUtils;
import org.apache.dolphinscheduler.common.utils.placeholder.BusinessTimeUtils;
import org.apache.dolphinscheduler.dao.entity.*;
import org.apache.dolphinscheduler.dao.mapper.*;
import org.apache.dolphinscheduler.dao.entity.ProcessDefinition;
import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
import org.apache.dolphinscheduler.dao.entity.ProcessTaskRelationLog;
import org.apache.dolphinscheduler.dao.entity.Project;
import org.apache.dolphinscheduler.dao.entity.ResponseTaskLog;
import org.apache.dolphinscheduler.dao.entity.TaskDefinition;
import org.apache.dolphinscheduler.dao.entity.TaskDefinitionLog;
import org.apache.dolphinscheduler.dao.entity.TaskInstance;
import org.apache.dolphinscheduler.dao.entity.Tenant;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionLogMapper;
import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper;
import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapper;
import org.apache.dolphinscheduler.dao.mapper.ProjectMapper;
import org.apache.dolphinscheduler.dao.mapper.ScheduleMapper;
import org.apache.dolphinscheduler.dao.mapper.TaskDefinitionLogMapper;
import org.apache.dolphinscheduler.dao.mapper.TaskDefinitionMapper;
import org.apache.dolphinscheduler.dao.mapper.TaskInstanceMapper;
import org.apache.dolphinscheduler.dao.mapper.TenantMapper;
import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
import org.apache.dolphinscheduler.plugin.task.api.enums.DependResult;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
@ -47,22 +72,24 @@ import org.apache.dolphinscheduler.plugin.task.api.parameters.ParametersNode;
import org.apache.dolphinscheduler.service.expand.CuringParamsService;
import org.apache.dolphinscheduler.service.process.ProcessService;
import org.apache.dolphinscheduler.service.task.TaskPluginManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.function.Function;
import java.util.stream.Collectors;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.*;
import static org.apache.dolphinscheduler.common.Constants.*;
import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.TASK_TYPE_DEPENDENT;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
/**
* process instance service impl
@ -787,8 +814,7 @@ public class ProcessInstanceServiceImpl extends BaseServiceImpl implements Proce
if (!nodeList.isEmpty()) {
List<Long> taskCodes = nodeList.stream().map(Long::parseLong).collect(Collectors.toList());
List<TaskInstance> taskInstances = taskInstanceMapper.queryByProcessInstanceIdsAndTaskCodes(
Collections.singletonList(processInstanceId), taskCodes
);
Collections.singletonList(processInstanceId), taskCodes);
for (String node : nodeList) {
TaskInstance taskInstance = null;
for (TaskInstance instance : taskInstances) {

73
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectServiceImpl.java
View File

@ -50,6 +50,7 @@ import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import org.slf4j.Logger;
@ -81,7 +82,6 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
@Autowired
private UserMapper userMapper;
/**
* create project
*
@ -99,7 +99,7 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
if (result.getCode() != Status.SUCCESS.getCode()) {
return result;
}
if (!canOperatorPermissions(loginUser, null,AuthorizationType.PROJECTS, PROJECT_CREATE)) {
if (!canOperatorPermissions(loginUser, null, AuthorizationType.PROJECTS, PROJECT_CREATE)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
@ -114,7 +114,7 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
try {
project = Project
.newBuilder()
.builder()
.name(name)
.code(CodeGenerateUtils.getInstance().genCode())
.description(desc)
@ -131,7 +131,8 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
if (projectMapper.insert(project) > 0) {
result.setData(project);
putMsg(result, Status.SUCCESS);
permissionPostHandle(AuthorizationType.PROJECTS, loginUser.getId(), Collections.singletonList(project.getId()), logger);
permissionPostHandle(AuthorizationType.PROJECTS, loginUser.getId(),
Collections.singletonList(project.getId()), logger);
} else {
putMsg(result, Status.CREATE_PROJECT_ERROR);
}
@ -179,7 +180,7 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
public Map<String, Object> queryByName(User loginUser, String projectName) {
Map<String, Object> result = new HashMap<>();
Project project = projectMapper.queryByName(projectName);
boolean hasProjectAndPerm = hasProjectAndPerm(loginUser, project, result,PROJECT);
boolean hasProjectAndPerm = hasProjectAndPerm(loginUser, project, result, PROJECT);
if (!hasProjectAndPerm) {
return result;
}
@ -199,11 +200,13 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
* @return true if the login user have permission to see the project
*/
@Override
public Map<String, Object> checkProjectAndAuth(User loginUser, Project project, long projectCode, String permission) {
public Map<String, Object> checkProjectAndAuth(User loginUser, Project project, long projectCode,
String permission) {
Map<String, Object> result = new HashMap<>();
if (project == null) {
putMsg(result, Status.PROJECT_NOT_EXIST);
} else if (!canOperatorPermissions(loginUser, new Object[] {project.getId()}, AuthorizationType.PROJECTS, permission)) {
} else if (!canOperatorPermissions(loginUser, new Object[]{project.getId()}, AuthorizationType.PROJECTS,
permission)) {
// check read permission
putMsg(result, Status.USER_NO_OPERATION_PROJECT_PERM, loginUser.getUserName(), projectCode);
} else {
@ -217,7 +220,8 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
boolean checkResult = false;
if (project == null) {
putMsg(result, Status.PROJECT_NOT_FOUND, "");
} else if (!canOperatorPermissions(loginUser, new Object[] {project.getId()}, AuthorizationType.PROJECTS, permission)) {
} else if (!canOperatorPermissions(loginUser, new Object[]{project.getId()}, AuthorizationType.PROJECTS,
permission)) {
putMsg(result, Status.USER_NO_OPERATION_PROJECT_PERM, loginUser.getUserName(), project.getCode());
} else {
checkResult = true;
@ -230,7 +234,8 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
boolean checkResult = false;
if (project == null) {
putMsg(result, Status.PROJECT_NOT_FOUND, "");
} else if (!canOperatorPermissions(loginUser, new Object[] {project.getId()}, AuthorizationType.PROJECTS, permission)) {
} else if (!canOperatorPermissions(loginUser, new Object[]{project.getId()}, AuthorizationType.PROJECTS,
permission)) {
putMsg(result, Status.USER_NO_OPERATION_PROJECT_PERM, loginUser.getUserName(), project.getName());
} else {
checkResult = true;
@ -252,13 +257,15 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
Result result = new Result();
PageInfo<Project> pageInfo = new PageInfo<>(pageNo, pageSize);
Page<Project> page = new Page<>(pageNo, pageSize);
Set<Integer> projectIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), logger);
Set<Integer> projectIds = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), logger);
if (projectIds.isEmpty()) {
result.setData(pageInfo);
putMsg(result, Status.SUCCESS);
return result;
}
IPage<Project> projectIPage = projectMapper.queryProjectListPaging(page, new ArrayList<>(projectIds), searchVal);
IPage<Project> projectIPage =
projectMapper.queryProjectListPaging(page, new ArrayList<>(projectIds), searchVal);
List<Project> projectList = projectIPage.getRecords();
if (loginUser.getUserType() != UserType.ADMIN_USER) {
@ -290,7 +297,10 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
return result;
}
List<ProcessDefinition> processDefinitionList = processDefinitionMapper.queryAllDefinitionList(project.getCode());
assert project != null;
List<ProcessDefinition> processDefinitionList =
processDefinitionMapper.queryAllDefinitionList(project.getCode());
if (!processDefinitionList.isEmpty()) {
putMsg(result, Status.DELETE_PROJECT_ERROR_DEFINES_NOT_NULL);
@ -313,8 +323,9 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
* @param project project
* @return check result
*/
private Map<String, Object> getCheckResult(User loginUser, Project project,String perm) {
Map<String, Object> checkResult = checkProjectAndAuth(loginUser, project, project == null ? 0L : project.getCode(),perm);
private Map<String, Object> getCheckResult(User loginUser, Project project, String perm) {
Map<String, Object> checkResult =
checkProjectAndAuth(loginUser, project, project == null ? 0L : project.getCode(), perm);
Status status = (Status) checkResult.get(Constants.STATUS);
if (status != Status.SUCCESS) {
return checkResult;
@ -370,7 +381,6 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
return result;
}
/**
* query unauthorized project
*
@ -382,13 +392,16 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
public Result queryUnauthorizedProject(User loginUser, Integer userId) {
Result result = new Result();
Set<Integer> projectIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), logger);
Set<Integer> projectIds = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), logger);
if (projectIds.isEmpty()) {
result.setData(Collections.emptyList());
putMsg(result, Status.SUCCESS);
return result;
}
List<Project> projectList = projectMapper.listAuthorizedProjects(loginUser.getUserType().equals(UserType.ADMIN_USER) ? 0 : loginUser.getId(), new ArrayList<>(projectIds));
List<Project> projectList = projectMapper.listAuthorizedProjects(
loginUser.getUserType().equals(UserType.ADMIN_USER) ? 0 : loginUser.getId(),
new ArrayList<>(projectIds));
List<Project> resultList = new ArrayList<>();
Set<Project> projectSet;
@ -493,7 +506,8 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
public Result queryProjectCreatedAndAuthorizedByUser(User loginUser) {
Result result = new Result();
Set<Integer> projectIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), logger);
Set<Integer> projectIds = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), logger);
if (projectIds.isEmpty()) {
result.setData(Collections.emptyList());
putMsg(result, Status.SUCCESS);
@ -507,18 +521,6 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
return result;
}
/**
* check whether have read permission
*
* @param user user
* @param project project
* @return true if the user have permission to see the project, otherwise return false
*/
private boolean checkReadPermission(User user, Project project) {
int permissionId = queryPermission(user, project);
return (permissionId & Constants.READ_PERMISSION) != 0;
}
/**
* query permission id
*
@ -531,7 +533,7 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
return Constants.READ_PERMISSION;
}
if (project.getUserId() == user.getId()) {
if (Objects.equals(project.getUserId(), user.getId())) {
return Constants.ALL_PERMISSIONS;
}
@ -553,7 +555,8 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
@Override
public Result queryAllProjectList(User user) {
Result result = new Result();
List<Project> projects = projectMapper.queryAllProject(user.getUserType() == UserType.ADMIN_USER ? 0 : user.getId());
List<Project> projects =
projectMapper.queryAllProject(user.getUserType() == UserType.ADMIN_USER ? 0 : user.getId());
result.setData(projects);
putMsg(result, Status.SUCCESS);
@ -570,10 +573,12 @@ public class ProjectServiceImpl extends BaseServiceImpl implements ProjectServic
* @return true if the login user have permission to see the project
*/
@Override
public void checkProjectAndAuth(Result result, User loginUser, Project project, long projectCode, String permission) {
public void checkProjectAndAuth(Result result, User loginUser, Project project, long projectCode,
String permission) {
if (project == null) {
putMsg(result, Status.PROJECT_NOT_EXIST);
} else if (!canOperatorPermissions(loginUser, new Object[] {project.getId()}, AuthorizationType.PROJECTS, permission)) {
} else if (!canOperatorPermissions(loginUser, new Object[]{project.getId()}, AuthorizationType.PROJECTS,
permission)) {
// check read permission
putMsg(result, Status.USER_NO_OPERATION_PROJECT_PERM, loginUser.getUserName(), projectCode);
} else {

213
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ResourcesServiceImpl.java
View File

@ -123,7 +123,6 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
@Autowired(required = false)
private StorageOperate storageOperate;
/**
* create directory
*
@ -144,8 +143,10 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
int pid,
String currentDir) {
Result<Object> result = new Result<>();
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FOLDER_ONLINE_CREATE : ApiFuncIdentificationConstant.UDF_FOLDER_ONLINE_CREATE;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, funcPermissionKey);
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FOLDER_ONLINE_CREATE
: ApiFuncIdentificationConstant.UDF_FOLDER_ONLINE_CREATE;
boolean canOperatorPermissions =
canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
@ -160,7 +161,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return result;
}
if(checkDescriptionLength(description)){
if (checkDescriptionLength(description)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -179,7 +180,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
Date now = new Date();
Resource resource = new Resource(pid, name, fullName, true, description, name, loginUser.getId(), type, 0, now, now);
Resource resource =
new Resource(pid, name, fullName, true, description, name, loginUser.getId(), type, 0, now, now);
try {
resourcesMapper.insert(resource);
@ -200,13 +202,14 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
logger.error("resource already exists, can't recreate ", e);
throw new ServiceException("resource already exists, can't recreate");
}
//create directory in storage
// create directory in storage
createDirectory(loginUser, fullName, type, result);
return result;
}
private String getFullName(String currentDir, String name) {
return currentDir.equals(FOLDER_SEPARATOR) ? String.format(FORMAT_SS, currentDir, name) : String.format(FORMAT_S_S, currentDir, name);
return currentDir.equals(FOLDER_SEPARATOR) ? String.format(FORMAT_SS, currentDir, name)
: String.format(FORMAT_S_S, currentDir, name);
}
/**
@ -231,8 +234,10 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
int pid,
String currentDir) {
Result<Object> result = new Result<>();
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_UPLOAD : ApiFuncIdentificationConstant.UDF_UPLOAD;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, funcPermissionKey);
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_UPLOAD
: ApiFuncIdentificationConstant.UDF_UPLOAD;
boolean canOperatorPermissions =
canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
@ -246,7 +251,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
if (!result.getCode().equals(Status.SUCCESS.getCode())) {
return result;
}
if(checkDescriptionLength(desc)){
if (checkDescriptionLength(desc)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -270,13 +275,15 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return result;
}
if (fullName.length() > Constants.RESOURCE_FULL_NAME_MAX_LENGTH) {
logger.error("resource {}'s full name {}' is longer than the max length {}", RegexUtils.escapeNRT(name), fullName, Constants.RESOURCE_FULL_NAME_MAX_LENGTH);
logger.error("resource {}'s full name {}' is longer than the max length {}", RegexUtils.escapeNRT(name),
fullName, Constants.RESOURCE_FULL_NAME_MAX_LENGTH);
putMsg(result, Status.RESOURCE_FULL_NAME_TOO_LONG_ERROR);
return result;
}
Date now = new Date();
Resource resource = new Resource(pid, name, fullName, false, desc, file.getOriginalFilename(), loginUser.getId(), type, file.getSize(), now, now);
Resource resource = new Resource(pid, name, fullName, false, desc, file.getOriginalFilename(),
loginUser.getId(), type, file.getSize(), now, now);
try {
resourcesMapper.insert(resource);
@ -297,9 +304,11 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
// fail upload
if (!upload(loginUser, fullName, file, type)) {
logger.error("upload resource: {} file: {} failed.", RegexUtils.escapeNRT(name), RegexUtils.escapeNRT(file.getOriginalFilename()));
logger.error("upload resource: {} file: {} failed.", RegexUtils.escapeNRT(name),
RegexUtils.escapeNRT(file.getOriginalFilename()));
putMsg(result, Status.STORE_OPERATE_CREATE_ERROR);
throw new ServiceException(String.format("upload resource: %s file: %s failed.", name, file.getOriginalFilename()));
throw new ServiceException(
String.format("upload resource: %s file: %s failed.", name, file.getOriginalFilename()));
}
return result;
}
@ -316,7 +325,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
for (int i = 1; i < splits.length; i++) {
String parentFullName = Joiner.on("/").join(Arrays.copyOfRange(splits, 0, i));
if (StringUtils.isNotBlank(parentFullName)) {
List<Resource> resources = resourcesMapper.queryResource(parentFullName, resource.getType().ordinal());
List<Resource> resources =
resourcesMapper.queryResource(parentFullName, resource.getType().ordinal());
if (CollectionUtils.isNotEmpty(resources)) {
Resource parentResource = resources.get(0);
if (parentResource.getSize() + size >= 0) {
@ -363,8 +373,10 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
ResourceType type,
MultipartFile file) {
Result<Object> result = new Result<>();
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_UPDATE : ApiFuncIdentificationConstant.UDF_UPDATE;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId}, checkResourceType(type), funcPermissionKey);
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_UPDATE
: ApiFuncIdentificationConstant.UDF_UPDATE;
boolean canOperatorPermissions =
canOperatorPermissions(loginUser, new Object[]{resourceId}, checkResourceType(type), funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
@ -379,7 +391,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
putMsg(result, Status.RESOURCE_NOT_EXIST);
return result;
}
if(checkDescriptionLength(desc)){
if (checkDescriptionLength(desc)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -389,7 +401,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return result;
}
if (resource.isDirectory() && storageOperate.returnStorageType().equals(ResUploadType.S3) && !resource.getFileName().equals(name)) {
if (resource.isDirectory() && storageOperate.returnStorageType().equals(ResUploadType.S3)
&& !resource.getFileName().equals(name)) {
putMsg(result, Status.S3_CANNOT_RENAME);
return result;
}
@ -399,11 +412,12 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return result;
}
//check resource already exists
// check resource already exists
String originFullName = resource.getFullName();
String originResourceName = resource.getAlias();
String fullName = String.format(FORMAT_SS, originFullName.substring(0, originFullName.lastIndexOf(FOLDER_SEPARATOR) + 1), name);
String fullName = String.format(FORMAT_SS,
originFullName.substring(0, originFullName.lastIndexOf(FOLDER_SEPARATOR) + 1), name);
if (!originResourceName.equals(name) && checkResourceExists(fullName, type.ordinal())) {
logger.error("resource {} already exists, can't recreate", name);
putMsg(result, Status.RESOURCE_EXIST);
@ -435,7 +449,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
}
if (!resource.isDirectory()) {
//get the origin file suffix
// get the origin file suffix
String originSuffix = Files.getFileExtension(originFullName);
String suffix = Files.getFileExtension(fullName);
boolean suffixIsChanged = false;
@ -445,15 +459,16 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
if (StringUtils.isNotBlank(suffix) && !suffix.equals(originSuffix)) {
suffixIsChanged = true;
}
//verify whether suffix is changed
// verify whether suffix is changed
if (suffixIsChanged) {
//need verify whether this resource is authorized to other users
// need verify whether this resource is authorized to other users
Map<String, Object> columnMap = new HashMap<>();
columnMap.put("resources_id", resourceId);
List<ResourcesUser> resourcesUsers = resourceUserMapper.selectByMap(columnMap);
if (CollectionUtils.isNotEmpty(resourcesUsers)) {
List<Integer> userIds = resourcesUsers.stream().map(ResourcesUser::getUserId).collect(Collectors.toList());
List<Integer> userIds =
resourcesUsers.stream().map(ResourcesUser::getUserId).collect(Collectors.toList());
List<User> users = userMapper.selectBatchIds(userIds);
String userNames = users.stream().map(User::getUserName).collect(Collectors.toList()).toString();
logger.error("resource is authorized to user {},suffix not allowed to be modified", userNames);
@ -538,9 +553,11 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
if (file != null) {
// fail upload
if (!upload(loginUser, fullName, file, type)) {
logger.error("upload resource: {} file: {} failed.", name, RegexUtils.escapeNRT(file.getOriginalFilename()));
logger.error("upload resource: {} file: {} failed.", name,
RegexUtils.escapeNRT(file.getOriginalFilename()));
putMsg(result, Status.HDFS_OPERATION_ERROR);
throw new ServiceException(String.format("upload resource: %s file: %s failed.", name, file.getOriginalFilename()));
throw new ServiceException(
String.format("upload resource: %s file: %s failed.", name, file.getOriginalFilename()));
}
if (!fullName.equals(originFullName)) {
try {
@ -601,12 +618,13 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
// determine file suffix
if (!fileSuffix.equalsIgnoreCase(nameSuffix)) {
// rename file suffix and original suffix must be consistent
logger.error("rename file suffix and original suffix must be consistent: {}", RegexUtils.escapeNRT(file.getOriginalFilename()));
logger.error("rename file suffix and original suffix must be consistent: {}",
RegexUtils.escapeNRT(file.getOriginalFilename()));
putMsg(result, Status.RESOURCE_SUFFIX_FORBID_CHANGE);
return result;
}
//If resource type is UDF, only jar packages are allowed to be uploaded, and the suffix must be .jar
// If resource type is UDF, only jar packages are allowed to be uploaded, and the suffix must be .jar
if (Constants.UDF.equals(type.name()) && !JAR.equalsIgnoreCase(fileSuffix)) {
logger.error(Status.UDF_RESOURCE_SUFFIX_NOT_JAR.getMsg());
putMsg(result, Status.UDF_RESOURCE_SUFFIX_NOT_JAR);
@ -632,7 +650,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
* @return resource list page
*/
@Override
public Result queryResourceListPaging(User loginUser, int directoryId, ResourceType type, String searchVal, Integer pageNo, Integer pageSize) {
public Result queryResourceListPaging(User loginUser, int directoryId, ResourceType type, String searchVal,
Integer pageNo, Integer pageSize) {
Result<Object> result = new Result<>();
Page<Resource> page = new Page<>(pageNo, pageSize);
if (directoryId != -1) {
@ -643,13 +662,15 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
}
}
PageInfo<Resource> pageInfo = new PageInfo<>(pageNo, pageSize);
Set<Integer> resourcesIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(checkResourceType(type), loginUser.getId(), logger);
Set<Integer> resourcesIds = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(checkResourceType(type), loginUser.getId(), logger);
if (resourcesIds.isEmpty()) {
result.setData(pageInfo);
putMsg(result, Status.SUCCESS);
return result;
}
IPage<Resource> resourceIPage = resourcesMapper.queryResourcePaging(page, directoryId, type.ordinal(), searchVal, new ArrayList<>(resourcesIds));
IPage<Resource> resourceIPage = resourcesMapper.queryResourcePaging(page, directoryId, type.ordinal(),
searchVal, new ArrayList<>(resourcesIds));
pageInfo.setTotal((int) resourceIPage.getTotal());
pageInfo.setTotalList(resourceIPage.getRecords());
result.setData(pageInfo);
@ -756,7 +777,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
public Result<Object> queryResourceByProgramType(User loginUser, ResourceType type, ProgramType programType) {
Result<Object> result = new Result<>();
Set<Integer> resourceIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(checkResourceType(type), loginUser.getId(), logger);
Set<Integer> resourceIds = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(checkResourceType(type), loginUser.getId(), logger);
if (resourceIds.isEmpty()) {
result.setData(Collections.emptyList());
putMsg(result, Status.SUCCESS);
@ -801,8 +823,11 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
putMsg(resultCheck, Status.RESOURCE_NOT_EXIST);
return resultCheck;
}
String funcPermissionKey = resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_DELETE : ApiFuncIdentificationConstant.UDF_DELETE;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId}, checkResourceType(resource.getType()), funcPermissionKey);
String funcPermissionKey =
resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_DELETE
: ApiFuncIdentificationConstant.UDF_DELETE;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId},
checkResourceType(resource.getType()), funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(resultCheck, Status.NO_CURRENT_OPERATING_PERMISSION);
return resultCheck;
@ -824,21 +849,21 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
// get all resource id of process definitions those is released
List<Map<String, Object>> list = processDefinitionMapper.listResources();
Map<Integer, Set<Long>> resourceProcessMap = ResourceProcessDefinitionUtils.getResourceProcessDefinitionMap(list);
Map<Integer, Set<Long>> resourceProcessMap =
ResourceProcessDefinitionUtils.getResourceProcessDefinitionMap(list);
Set<Integer> resourceIdSet = resourceProcessMap.keySet();
// get all children of the resource
List<Integer> allChildren = listAllChildren(resource, true);
Integer[] needDeleteResourceIdArray = allChildren.toArray(new Integer[allChildren.size()]);
if (needDeleteResourceIdArray.length >= 2){
if (needDeleteResourceIdArray.length >= 2) {
logger.error("can't be deleted,because There are files or folders in the current directory:{}", resource);
putMsg(result, Status.RESOURCE_HAS_FOLDER, resource.getFileName());
return result;
}
//if resource type is UDF,need check whether it is bound by UDF function
// if resource type is UDF,need check whether it is bound by UDF function
if (resource.getType() == (ResourceType.UDF)) {
List<UdfFunc> udfFuncs = udfFunctionMapper.listUdfByResourceId(needDeleteResourceIdArray);
if (CollectionUtils.isNotEmpty(udfFuncs)) {
@ -848,8 +873,6 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
}
}
if (resourceIdSet.contains(resource.getPid())) {
logger.error("can't be deleted,because it is used of process definition");
putMsg(result, Status.RESOURCE_IS_USED);
@ -859,7 +882,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
if (CollectionUtils.isNotEmpty(resourceIdSet)) {
logger.error("can't be deleted,because it is used of process definition");
for (Integer resId : resourceIdSet) {
logger.error("resource id:{} is used of process definition {}",resId,resourceProcessMap.get(resId));
logger.error("resource id:{} is used of process definition {}", resId, resourceProcessMap.get(resId));
}
putMsg(result, Status.RESOURCE_IS_USED);
return result;
@ -867,16 +890,16 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
// get hdfs file by type
String storageFilename = storageOperate.getFileName(resource.getType(), tenantCode, resource.getFullName());
//delete data in database
// delete data in database
resourcesMapper.selectBatchIds(Arrays.asList(needDeleteResourceIdArray)).forEach(item -> {
updateParentResourceSize(item, item.getSize() * -1);
});
resourcesMapper.deleteIds(needDeleteResourceIdArray);
resourceUserMapper.deleteResourceUserArray(0, needDeleteResourceIdArray);
//delete file on hdfs
// delete file on hdfs
//delete file on storage
// delete file on storage
storageOperate.delete(tenantCode, storageFilename, true);
putMsg(result, Status.SUCCESS);
@ -894,15 +917,18 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
@Override
public Result<Object> verifyResourceName(String fullName, ResourceType type, User loginUser) {
Result<Object> result = new Result<>();
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_RENAME : ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, funcPermissionKey);
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_RENAME
: ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions =
canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
putMsg(result, Status.SUCCESS);
if (checkResourceExists(fullName, type.ordinal())) {
logger.error("resource type:{} name:{} has exist, can't create again.", type, RegexUtils.escapeNRT(fullName));
logger.error("resource type:{} name:{} has exist, can't create again.", type,
RegexUtils.escapeNRT(fullName));
putMsg(result, Status.RESOURCE_EXIST);
} else {
// query tenant
@ -962,8 +988,10 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return result;
}
}
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_VIEW : ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resource.getId()}, checkResourceType(type), funcPermissionKey);
String funcPermissionKey = type.equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_VIEW
: ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resource.getId()},
checkResourceType(type), funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
@ -986,8 +1014,11 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
putMsg(result, Status.RESOURCE_NOT_EXIST);
return result;
}
String funcPermissionKey = resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_VIEW : ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{id}, checkResourceType(resource.getType()), funcPermissionKey);
String funcPermissionKey =
resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_VIEW
: ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{id},
checkResourceType(resource.getType()), funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
@ -1017,13 +1048,16 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
putMsg(result, Status.RESOURCE_NOT_EXIST);
return result;
}
String funcPermissionKey = resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_VIEW : ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId}, checkResourceType(resource.getType()), funcPermissionKey);
String funcPermissionKey =
resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_VIEW
: ApiFuncIdentificationConstant.UDF_FILE_VIEW;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId},
checkResourceType(resource.getType()), funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
//check preview or not by file suffix
// check preview or not by file suffix
String nameSuffix = Files.getFileExtension(resource.getAlias());
String resourceViewSuffixes = FileUtils.getResourceViewSuffixes();
if (StringUtils.isNotEmpty(resourceViewSuffixes)) {
@ -1080,9 +1114,11 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
*/
@Override
@Transactional
public Result<Object> onlineCreateResource(User loginUser, ResourceType type, String fileName, String fileSuffix, String desc, String content, int pid, String currentDir) {
public Result<Object> onlineCreateResource(User loginUser, ResourceType type, String fileName, String fileSuffix,
String desc, String content, int pid, String currentDir) {
Result<Object> result = new Result<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID, ApiFuncIdentificationConstant.FILE_ONLINE_CREATE);
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.RESOURCE_FILE_ID,
ApiFuncIdentificationConstant.FILE_ONLINE_CREATE);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
@ -1096,12 +1132,12 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
putMsg(result, Status.VERIFY_PARAMETER_NAME_FAILED);
return result;
}
if(checkDescriptionLength(desc)){
if (checkDescriptionLength(desc)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
//check file suffix
// check file suffix
String nameSuffix = fileSuffix.trim();
String resourceViewSuffixes = FileUtils.getResourceViewSuffixes();
if (StringUtils.isNotEmpty(resourceViewSuffixes)) {
@ -1122,7 +1158,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
// save data
Date now = new Date();
Resource resource = new Resource(pid, name, fullName, false, desc, name, loginUser.getId(), type, content.getBytes().length, now, now);
Resource resource = new Resource(pid, name, fullName, false, desc, name, loginUser.getId(), type,
content.getBytes().length, now, now);
resourcesMapper.insert(resource);
updateParentResourceSize(resource, resource.getSize());
@ -1158,7 +1195,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
*/
@Override
@Transactional
public Result<Object> onlineCreateOrUpdateResourceWithDir(User loginUser, String fileFullName, String desc, String content) {
public Result<Object> onlineCreateOrUpdateResourceWithDir(User loginUser, String fileFullName, String desc,
String content) {
if (checkResourceExists(fileFullName, ResourceType.FILE.ordinal())) {
Resource resource = resourcesMapper.queryResource(fileFullName, ResourceType.FILE.ordinal()).get(0);
Result<Object> result = this.updateResourceContent(loginUser, resource.getId(), content);
@ -1188,13 +1226,15 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
}
}
return this.onlineCreateResource(
loginUser, ResourceType.FILE, resourceName, resourceSuffix, desc, content, pid, currDirPath.toString());
loginUser, ResourceType.FILE, resourceName, resourceSuffix, desc, content, pid,
currDirPath.toString());
}
}
@Override
@Transactional
public Integer createOrUpdateResource(String userName, String fullName, String description, String resourceContent) {
public Integer createOrUpdateResource(String userName, String fullName, String description,
String resourceContent) {
User user = userMapper.queryByUserNameAccurately(userName);
int suffixLabelIndex = fullName.indexOf(PERIOD);
if (suffixLabelIndex == -1) {
@ -1227,7 +1267,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
user, dirName, EMPTY_STRING, ResourceType.FILE, pid, currentDir);
if (createDirResult.getCode() == Status.SUCCESS.getCode()) {
Map<String, Object> resultMap = (Map<String, Object>) createDirResult.getData();
return (int) resultMap.get("id");
return resultMap.get("id") == null ? -1 : (Integer) resultMap.get("id");
} else {
String msg = String.format("Can not create dir %s", dirFullName);
logger.error(msg);
@ -1237,7 +1277,9 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
}
private void permissionPostHandle(ResourceType resourceType, User loginUser, Integer resourceId) {
AuthorizationType authorizationType = resourceType.equals(ResourceType.FILE) ? AuthorizationType.RESOURCE_FILE_ID : AuthorizationType.UDF_FILE;
AuthorizationType authorizationType =
resourceType.equals(ResourceType.FILE) ? AuthorizationType.RESOURCE_FILE_ID
: AuthorizationType.UDF_FILE;
permissionPostHandle(authorizationType, loginUser.getId(), Collections.singletonList(resourceId), logger);
}
@ -1299,19 +1341,23 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
putMsg(result, Status.RESOURCE_NOT_EXIST);
return result;
}
String funcPermissionKey = resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_UPDATE : ApiFuncIdentificationConstant.UDF_UPDATE;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId}, checkResourceType(resource.getType()), funcPermissionKey);
String funcPermissionKey =
resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_UPDATE
: ApiFuncIdentificationConstant.UDF_UPDATE;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId},
checkResourceType(resource.getType()), funcPermissionKey);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
//check can edit by file suffix
// check can edit by file suffix
String nameSuffix = Files.getFileExtension(resource.getAlias());
String resourceViewSuffixes = FileUtils.getResourceViewSuffixes();
if (StringUtils.isNotEmpty(resourceViewSuffixes)) {
List<String> strList = Arrays.asList(resourceViewSuffixes.split(","));
if (!strList.contains(nameSuffix)) {
logger.error("resource suffix {} not support updateProcessInstance, resource id {}", nameSuffix, resourceId);
logger.error("resource suffix {} not support updateProcessInstance, resource id {}", nameSuffix,
resourceId);
putMsg(result, Status.RESOURCE_SUFFIX_NOT_SUPPORT_VIEW);
return result;
}
@ -1341,7 +1387,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
* @param content content
* @return result
*/
private Result<Object> uploadContentToStorage(User loginUser,String resourceName, String tenantCode, String content) {
private Result<Object> uploadContentToStorage(User loginUser, String resourceName, String tenantCode,
String content) {
Result<Object> result = new Result<>();
String localFilename = "";
String storageFileName = "";
@ -1355,7 +1402,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return result;
}
// get resource file path
// get resource file path
storageFileName = storageOperate.getResourceFileName(tenantCode, resourceName);
String resourcePath = storageOperate.getResDir(tenantCode);
logger.info("resource path is {}, resource dir is {}", storageFileName, resourcePath);
@ -1400,10 +1447,14 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
return null;
}
String funcPermissionKey = resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_DOWNLOAD : ApiFuncIdentificationConstant.UDF_DOWNLOAD;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId}, checkResourceType(resource.getType()), funcPermissionKey);
String funcPermissionKey =
resource.getType().equals(ResourceType.FILE) ? ApiFuncIdentificationConstant.FILE_DOWNLOAD
: ApiFuncIdentificationConstant.UDF_DOWNLOAD;
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{resourceId},
checkResourceType(resource.getType()), funcPermissionKey);
if (!canOperatorPermissions) {
logger.error("{}: {}", Status.NO_CURRENT_OPERATING_PERMISSION.getMsg(), PropertyUtils.getResUploadStartupState());
logger.error("{}: {}", Status.NO_CURRENT_OPERATING_PERMISSION.getMsg(),
PropertyUtils.getResUploadStartupState());
throw new ServiceException(Status.NO_CURRENT_OPERATING_PERMISSION.getMsg());
}
if (resource.isDirectory()) {
@ -1421,7 +1472,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
Tenant tenant = tenantMapper.queryById(user.getTenantId());
if (tenant == null) {
logger.error("tenant id {} not exists", user.getTenantId());
throw new ServiceException(String.format("The tenant id %d of resource owner not exist", user.getTenantId()));
throw new ServiceException(
String.format("The tenant id %d of resource owner not exist", user.getTenantId()));
}
String tenantCode = tenant.getTenantCode();
@ -1435,7 +1487,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
storageOperate.download(tenantCode, fileName, localFileName, false, true);
return org.apache.dolphinscheduler.api.utils.FileUtils.file2Resource(localFileName);
} catch (IOException e) {
logger.error("download resource error, the path is {}, and local filename is {}, the error message is {}", fileName, localFileName, e.getMessage());
logger.error("download resource error, the path is {}, and local filename is {}, the error message is {}",
fileName, localFileName, e.getMessage());
throw new ServerException("download the resource file failed ,it may be related to your storage");
}
}
@ -1598,7 +1651,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
Visitor visitor = new ResourceTreeVisitor(authedResources);
String visit = JSONUtils.toJsonString(visitor.visit(), SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
logger.info(visit);
String jsonTreeStr = JSONUtils.toJsonString(visitor.visit().getChildren(), SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
String jsonTreeStr =
JSONUtils.toJsonString(visitor.visit().getChildren(), SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
logger.info(jsonTreeStr);
result.put(Constants.DATA_LIST, visitor.visit().getChildren());
putMsg(result, Status.SUCCESS);
@ -1652,7 +1706,7 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
*/
List<Integer> listAllChildren(Resource resource, boolean containSelf) {
List<Integer> childList = new ArrayList<>();
if (resource.getId() != -1 && containSelf) {
if (resource.getId() != null && containSelf) {
childList.add(resource.getId());
}
@ -1684,7 +1738,8 @@ public class ResourcesServiceImpl extends BaseServiceImpl implements ResourcesSe
* @return all authored resource list
*/
private List<Resource> queryAuthoredResourceList(User loginUser, ResourceType type) {
Set<Integer> resourceIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(checkResourceType(type), loginUser.getId(), logger);
Set<Integer> resourceIds = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(checkResourceType(type), loginUser.getId(), logger);
if (resourceIds.isEmpty()) {
return Collections.emptyList();
}

64
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskGroupServiceImpl.java
View File

@ -26,7 +26,6 @@ import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.AuthorizationType;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.dao.entity.TaskGroup;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.TaskGroupMapper;
@ -82,14 +81,16 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
*/
@Override
@Transactional
public Map<String, Object> createTaskGroup(User loginUser, Long projectCode, String name, String description, int groupSize) {
public Map<String, Object> createTaskGroup(User loginUser, Long projectCode, String name, String description,
int groupSize) {
Map<String, Object> result = new HashMap<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP, ApiFuncIdentificationConstant.TASK_GROUP_CREATE);
if (!canOperatorPermissions){
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP,
ApiFuncIdentificationConstant.TASK_GROUP_CREATE);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
if(checkDescriptionLength(description)){
if (checkDescriptionLength(description)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -112,7 +113,8 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
taskGroup.setCreateTime(new Date());
taskGroup.setUpdateTime(new Date());
if (taskGroupMapper.insert(taskGroup) > 0) {
permissionPostHandle(AuthorizationType.TASK_GROUP, loginUser.getId(), Collections.singletonList(taskGroup.getId()),logger);
permissionPostHandle(AuthorizationType.TASK_GROUP, loginUser.getId(),
Collections.singletonList(taskGroup.getId()), logger);
putMsg(result, Status.SUCCESS);
} else {
putMsg(result, Status.CREATE_TASK_GROUP_ERROR);
@ -134,12 +136,13 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
@Override
public Map<String, Object> updateTaskGroup(User loginUser, int id, String name, String description, int groupSize) {
Map<String, Object> result = new HashMap<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP, ApiFuncIdentificationConstant.TASK_GROUP_EDIT);
if (!canOperatorPermissions){
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP,
ApiFuncIdentificationConstant.TASK_GROUP_EDIT);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
if(checkDescriptionLength(description)){
if (checkDescriptionLength(description)) {
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR);
return result;
}
@ -151,7 +154,7 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
putMsg(result, Status.TASK_GROUP_SIZE_ERROR);
return result;
}
Integer exists = taskGroupMapper.selectCount(new QueryWrapper<TaskGroup>().lambda()
Long exists = taskGroupMapper.selectCount(new QueryWrapper<TaskGroup>().lambda()
.eq(TaskGroup::getName, name)
.eq(TaskGroup::getUserId, loginUser.getId())
.ne(TaskGroup::getId, id));
@ -197,7 +200,8 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
* @return the result code and msg
*/
@Override
public Map<String, Object> queryAllTaskGroup(User loginUser, String name, Integer status, int pageNo, int pageSize) {
public Map<String, Object> queryAllTaskGroup(User loginUser, String name, Integer status, int pageNo,
int pageSize) {
return this.doQuery(loginUser, pageNo, pageSize, loginUser.getId(), name, status);
}
@ -229,18 +233,21 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
Map<String, Object> result = new HashMap<>();
Page<TaskGroup> page = new Page<>(pageNo, pageSize);
PageInfo<TaskGroup> emptyPageInfo = new PageInfo<>(pageNo, pageSize);
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.TASK_GROUP, loginUser.getId(), logger);
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.TASK_GROUP,
loginUser.getId(), logger);
if (ids.isEmpty()) {
result.put(Constants.DATA_LIST, emptyPageInfo);
putMsg(result, Status.SUCCESS);
return result;
}
IPage<TaskGroup> taskGroupPaging = taskGroupMapper.queryTaskGroupPagingByProjectCode(page, new ArrayList<>(ids), projectCode);
IPage<TaskGroup> taskGroupPaging =
taskGroupMapper.queryTaskGroupPagingByProjectCode(page, new ArrayList<>(ids), projectCode);
return getStringObjectMap(pageNo, pageSize, result, taskGroupPaging);
}
private Map<String, Object> getStringObjectMap(int pageNo, int pageSize, Map<String, Object> result, IPage<TaskGroup> taskGroupPaging) {
private Map<String, Object> getStringObjectMap(int pageNo, int pageSize, Map<String, Object> result,
IPage<TaskGroup> taskGroupPaging) {
PageInfo<TaskGroup> pageInfo = new PageInfo<>(pageNo, pageSize);
int total = taskGroupPaging == null ? 0 : (int) taskGroupPaging.getTotal();
List<TaskGroup> list = taskGroupPaging == null ? new ArrayList<TaskGroup>() : taskGroupPaging.getRecords();
@ -279,17 +286,20 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
* @return the result code and msg
*/
@Override
public Map<String, Object> doQuery(User loginUser, int pageNo, int pageSize, int userId, String name, Integer status) {
public Map<String, Object> doQuery(User loginUser, int pageNo, int pageSize, int userId, String name,
Integer status) {
Map<String, Object> result = new HashMap<>();
Page<TaskGroup> page = new Page<>(pageNo, pageSize);
PageInfo<TaskGroup> pageInfo = new PageInfo<>(pageNo, pageSize);
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.TASK_GROUP, userId, logger);
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.TASK_GROUP,
userId, logger);
if (ids.isEmpty()) {
result.put(Constants.DATA_LIST, pageInfo);
putMsg(result, Status.SUCCESS);
return result;
}
IPage<TaskGroup> taskGroupPaging = taskGroupMapper.queryTaskGroupPaging(page, new ArrayList<>(ids), name, status);
IPage<TaskGroup> taskGroupPaging =
taskGroupMapper.queryTaskGroupPaging(page, new ArrayList<>(ids), name, status);
return getStringObjectMap(pageNo, pageSize, result, taskGroupPaging);
}
@ -305,8 +315,9 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
public Map<String, Object> closeTaskGroup(User loginUser, int id) {
Map<String, Object> result = new HashMap<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP, ApiFuncIdentificationConstant.TASK_GROUP_CLOSE);
if (!canOperatorPermissions){
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP,
ApiFuncIdentificationConstant.TASK_GROUP_CLOSE);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
@ -332,8 +343,9 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
public Map<String, Object> startTaskGroup(User loginUser, int id) {
Map<String, Object> result = new HashMap<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP, ApiFuncIdentificationConstant.TASK_GROUP_CLOSE);
if (!canOperatorPermissions){
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP,
ApiFuncIdentificationConstant.TASK_GROUP_CLOSE);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
@ -359,8 +371,9 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
@Override
public Map<String, Object> forceStartTask(User loginUser, int queueId) {
Map<String, Object> result = new HashMap<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP, ApiFuncIdentificationConstant.TASK_GROUP_QUEUE_START);
if (!canOperatorPermissions){
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP,
ApiFuncIdentificationConstant.TASK_GROUP_QUEUE_START);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}
@ -371,8 +384,9 @@ public class TaskGroupServiceImpl extends BaseServiceImpl implements TaskGroupSe
public Map<String, Object> modifyPriority(User loginUser, Integer queueId, Integer priority) {
Map<String, Object> result = new HashMap<>();
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP, ApiFuncIdentificationConstant.TASK_GROUP_QUEUE_PRIORITY);
if (!canOperatorPermissions){
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.TASK_GROUP,
ApiFuncIdentificationConstant.TASK_GROUP_QUEUE_PRIORITY);
if (!canOperatorPermissions) {
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION);
return result;
}

72
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UsersServiceImpl.java
View File

@ -83,7 +83,6 @@ import org.springframework.transaction.annotation.Transactional;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
/**
* users service impl
*/
@ -131,7 +130,6 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
@Autowired
private K8sNamespaceUserMapper k8sNamespaceUserMapper;
/**
* create user, only system admin have permission
*
@ -157,7 +155,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
int state) throws Exception {
Map<String, Object> result = new HashMap<>();
//check all user params
// check all user params
String msg = this.checkUserParams(userName, userPassword, email, phone);
if (resourcePermissionCheckService.functionDisabled()) {
putMsg(result, Status.FUNCTION_DISABLED, msg);
@ -310,7 +308,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
*/
@Override
public int getUserIdByName(String name) {
//executor name query
// executor name query
int executorId = 0;
if (StringUtils.isNotEmpty(name)) {
User executor = queryUser(name);
@ -434,7 +432,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
return result;
}
if (state == 0 && user.getState() != state && loginUser.getId() == user.getId()) {
if (state == 0 && user.getState() != state && Objects.equals(loginUser.getId(), user.getId())) {
putMsg(result, Status.NOT_ALLOW_TO_DISABLE_OWN_ACCOUNT);
return result;
}
@ -475,12 +473,12 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
putMsg(result, Status.FUNCTION_DISABLED);
return result;
}
//only admin can operate
// only admin can operate
if (!isAdmin(loginUser)) {
putMsg(result, Status.USER_NO_OPERATION_PERM, id);
return result;
}
//check exist
// check exist
User tempUser = userMapper.selectById(id);
if (tempUser == null) {
putMsg(result, Status.USER_NOT_EXIST, id);
@ -495,7 +493,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
}
// delete user
userMapper.queryTenantCodeByUserId(id);
accessTokenMapper.deleteAccessTokenByUserId(id);
userMapper.deleteById(id);
@ -523,7 +521,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
putMsg(result, Status.FUNCTION_DISABLED);
return result;
}
//check exist
// check exist
User tempUser = userMapper.selectById(userId);
if (tempUser == null) {
putMsg(result, Status.USER_NOT_EXIST, userId);
@ -683,27 +681,31 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
}
}
//get the authorized resource id list by user id
List<Integer> resIds = resourceUserMapper.queryResourcesIdListByUserIdAndPerm(userId, Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> oldAuthorizedRes = CollectionUtils.isEmpty(resIds) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds);
//if resource type is UDF,need check whether it is bound by UDF function
// get the authorized resource id list by user id
List<Integer> resIds =
resourceUserMapper.queryResourcesIdListByUserIdAndPerm(userId, Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> oldAuthorizedRes =
CollectionUtils.isEmpty(resIds) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds);
// if resource type is UDF,need check whether it is bound by UDF function
Set<Integer> oldAuthorizedResIds = oldAuthorizedRes.stream().map(Resource::getId).collect(Collectors.toSet());
//get the unauthorized resource id list
// get the unauthorized resource id list
oldAuthorizedResIds.removeAll(needAuthorizeResIds);
if (CollectionUtils.isNotEmpty(oldAuthorizedResIds)) {
// get all resource id of process definitions those is released
List<Map<String, Object>> list = processDefinitionMapper.listResourcesByUser(userId);
Map<Integer, Set<Long>> resourceProcessMap = ResourceProcessDefinitionUtils.getResourceProcessDefinitionMap(list);
Map<Integer, Set<Long>> resourceProcessMap =
ResourceProcessDefinitionUtils.getResourceProcessDefinitionMap(list);
Set<Integer> resourceIdSet = resourceProcessMap.keySet();
resourceIdSet.retainAll(oldAuthorizedResIds);
if (CollectionUtils.isNotEmpty(resourceIdSet)) {
logger.error("can't be deleted,because it is used of process definition");
for (Integer resId : resourceIdSet) {
logger.error("resource id:{} is used of process definition {}", resId, resourceProcessMap.get(resId));
logger.error("resource id:{} is used of process definition {}", resId,
resourceProcessMap.get(resId));
}
putMsg(result, Status.RESOURCE_IS_USED);
return result;
@ -792,7 +794,6 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
return result;
}
/**
* grant namespace
*
@ -810,12 +811,12 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
putMsg(result, Status.FUNCTION_DISABLED);
return result;
}
//only admin can operate
// only admin can operate
if (this.check(result, !this.isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) {
return result;
}
//check exist
// check exist
User tempUser = userMapper.selectById(userId);
if (tempUser == null) {
putMsg(result, Status.USER_NOT_EXIST, userId);
@ -842,7 +843,6 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
return result;
}
/**
* grant datasource
*
@ -950,7 +950,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
putMsg(result, Status.FUNCTION_DISABLED);
return result;
}
//only admin can operate
// only admin can operate
if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) {
return result;
}
@ -971,8 +971,8 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
@Override
public Map<String, Object> queryUserList(User loginUser) {
Map<String, Object> result = new HashMap<>();
//only admin can operate
if (!canOperatorPermissions(loginUser,null, AuthorizationType.ACCESS_TOKEN, USER_MANAGER)) {
// only admin can operate
if (!canOperatorPermissions(loginUser, null, AuthorizationType.ACCESS_TOKEN, USER_MANAGER)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
@ -1018,7 +1018,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
putMsg(result, Status.FUNCTION_DISABLED);
return result;
}
//only admin can operate
// only admin can operate
if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) {
return result;
}
@ -1058,7 +1058,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
putMsg(result, Status.FUNCTION_DISABLED);
return result;
}
//only admin can operate
// only admin can operate
if (check(result, !isAdmin(loginUser), Status.USER_NO_OPERATION_PERM)) {
return result;
}
@ -1076,7 +1076,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
private boolean checkTenantExists(int tenantId) {
return tenantMapper.queryById(tenantId) != null;
}
/**
* @return if check failed return the field, otherwise return null
*/
@ -1109,28 +1109,33 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
* @param dstBasePath dst base path
* @throws IOException io exception
*/
private void copyResourceFiles(String oldTenantCode, String newTenantCode, ResourceComponent resourceComponent, String srcBasePath, String dstBasePath) {
private void copyResourceFiles(String oldTenantCode, String newTenantCode, ResourceComponent resourceComponent,
String srcBasePath, String dstBasePath) {
List<ResourceComponent> components = resourceComponent.getChildren();
try {
if (CollectionUtils.isNotEmpty(components)) {
for (ResourceComponent component : components) {
// verify whether exist
if (!storageOperate.exists(oldTenantCode, String.format(Constants.FORMAT_S_S, srcBasePath, component.getFullName()))) {
if (!storageOperate.exists(oldTenantCode,
String.format(Constants.FORMAT_S_S, srcBasePath, component.getFullName()))) {
logger.error("resource file: {} not exist,copy error", component.getFullName());
throw new ServiceException(Status.RESOURCE_NOT_EXIST);
}
if (!component.isDirctory()) {
// copy it to dst
storageOperate.copy(String.format(Constants.FORMAT_S_S, srcBasePath, component.getFullName()), String.format(Constants.FORMAT_S_S, dstBasePath, component.getFullName()), false, true);
storageOperate.copy(String.format(Constants.FORMAT_S_S, srcBasePath, component.getFullName()),
String.format(Constants.FORMAT_S_S, dstBasePath, component.getFullName()), false, true);
continue;
}
if (CollectionUtils.isEmpty(component.getChildren())) {
// if not exist,need create it
if (!storageOperate.exists(oldTenantCode, String.format(Constants.FORMAT_S_S, dstBasePath, component.getFullName()))) {
storageOperate.mkdir(newTenantCode, String.format(Constants.FORMAT_S_S, dstBasePath, component.getFullName()));
if (!storageOperate.exists(oldTenantCode,
String.format(Constants.FORMAT_S_S, dstBasePath, component.getFullName()))) {
storageOperate.mkdir(newTenantCode,
String.format(Constants.FORMAT_S_S, dstBasePath, component.getFullName()));
}
} else {
copyResourceFiles(oldTenantCode, newTenantCode, component, srcBasePath, dstBasePath);
@ -1158,7 +1163,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
public Map<String, Object> registerUser(String userName, String userPassword, String repeatPassword, String email) {
Map<String, Object> result = new HashMap<>();
//check user params
// check user params
String msg = this.checkUserParams(userName, userPassword, email, "");
if (resourcePermissionCheckService.functionDisabled()) {
putMsg(result, Status.FUNCTION_DISABLED);
@ -1296,7 +1301,8 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
*/
@Override
@Transactional
public User createUserIfNotExists(String userName, String userPassword, String email, String phone, String tenantCode,
public User createUserIfNotExists(String userName, String userPassword, String email, String phone,
String tenantCode,
String queue,
int state) throws IOException {
User user = userMapper.queryByUserNameAccurately(userName);

41
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/WorkerGroupServiceImpl.java
View File

@ -28,7 +28,6 @@ import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.AuthorizationType;
import org.apache.dolphinscheduler.common.enums.NodeType;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.common.model.HeartBeat;
import org.apache.dolphinscheduler.common.model.WorkerHeartBeat;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
@ -48,6 +47,7 @@ import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
@ -87,9 +87,10 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
*/
@Override
@Transactional
public Map<String, Object> saveWorkerGroup(User loginUser, int id, String name, String addrList, String description, String otherParamsJson) {
public Map<String, Object> saveWorkerGroup(User loginUser, int id, String name, String addrList, String description,
String otherParamsJson) {
Map<String, Object> result = new HashMap<>();
if (!canOperatorPermissions(loginUser,null, AuthorizationType.WORKER_GROUP, WORKER_GROUP_CREATE)) {
if (!canOperatorPermissions(loginUser, null, AuthorizationType.WORKER_GROUP, WORKER_GROUP_CREATE)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
@ -129,12 +130,14 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
return result;
}
protected void handleDefaultWorkGroup(WorkerGroupMapper workerGroupMapper, WorkerGroup workerGroup, User loginUser, String otherParamsJson) {
if (workerGroup.getId() != 0) {
protected void handleDefaultWorkGroup(WorkerGroupMapper workerGroupMapper, WorkerGroup workerGroup, User loginUser,
String otherParamsJson) {
if (workerGroup.getId() != null) {
workerGroupMapper.updateById(workerGroup);
} else {
workerGroupMapper.insert(workerGroup);
permissionPostHandle(AuthorizationType.WORKER_GROUP, loginUser.getId(), Collections.singletonList(workerGroup.getId()),logger);
permissionPostHandle(AuthorizationType.WORKER_GROUP, loginUser.getId(),
Collections.singletonList(workerGroup.getId()), logger);
}
}
@ -148,18 +151,19 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
List<WorkerGroup> workerGroupList = workerGroupMapper.queryWorkerGroupByName(workerGroup.getName());
if (CollectionUtils.isNotEmpty(workerGroupList)) {
// new group has same name
if (workerGroup.getId() == 0) {
if (workerGroup.getId() == null) {
return true;
}
// check group id
for (WorkerGroup group : workerGroupList) {
if (group.getId() != workerGroup.getId()) {
if (Objects.equals(group.getId(), workerGroup.getId())) {
return true;
}
}
}
// check zookeeper
String workerGroupPath = Constants.REGISTRY_DOLPHINSCHEDULER_WORKERS + Constants.SINGLE_SLASH + workerGroup.getName();
String workerGroupPath =
Constants.REGISTRY_DOLPHINSCHEDULER_WORKERS + Constants.SINGLE_SLASH + workerGroup.getName();
return registryClient.exists(workerGroupPath);
}
@ -203,7 +207,8 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
if (loginUser.getUserType().equals(UserType.ADMIN_USER)) {
workerGroups = getWorkerGroups(true, null);
} else {
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.WORKER_GROUP, loginUser.getId(), logger);
Set<Integer> ids = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(AuthorizationType.WORKER_GROUP, loginUser.getId(), logger);
workerGroups = getWorkerGroups(true, ids.isEmpty() ? Collections.emptyList() : new ArrayList<>(ids));
}
List<WorkerGroup> resultDataList = new ArrayList<>();
@ -252,7 +257,8 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
if (loginUser.getUserType().equals(UserType.ADMIN_USER)) {
workerGroups = getWorkerGroups(false, null);
} else {
Set<Integer> ids = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.WORKER_GROUP, loginUser.getId(), logger);
Set<Integer> ids = resourcePermissionCheckService
.userOwnedResourceIdsAcquisition(AuthorizationType.WORKER_GROUP, loginUser.getId(), logger);
workerGroups = getWorkerGroups(false, ids.isEmpty() ? Collections.emptyList() : new ArrayList<>(ids));
}
List<String> availableWorkerGroupList = workerGroups.stream()
@ -302,7 +308,8 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
}
Map<String, WorkerGroup> workerGroupsMap = null;
if (workerGroups.size() != 0) {
workerGroupsMap = workerGroups.stream().collect(Collectors.toMap(WorkerGroup::getName, workerGroupItem -> workerGroupItem, (oldWorkerGroup, newWorkerGroup) -> oldWorkerGroup));
workerGroupsMap = workerGroups.stream().collect(Collectors.toMap(WorkerGroup::getName,
workerGroupItem -> workerGroupItem, (oldWorkerGroup, newWorkerGroup) -> oldWorkerGroup));
}
for (String workerGroup : workerGroupList) {
String workerGroupPath = workerPath + Constants.SINGLE_SLASH + workerGroup;
@ -319,7 +326,8 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
handleAddrList(wg, workerGroup, childrenNodes);
wg.setName(workerGroup);
if (isPaging) {
String registeredValue = registryClient.get(workerGroupPath + Constants.SINGLE_SLASH + childrenNodes.iterator().next());
String registeredValue =
registryClient.get(workerGroupPath + Constants.SINGLE_SLASH + childrenNodes.iterator().next());
WorkerHeartBeat workerHeartBeat = JSONUtils.parseObject(registeredValue, WorkerHeartBeat.class);
wg.setCreateTime(new Date(workerHeartBeat.getStartupTime()));
wg.setUpdateTime(new Date(workerHeartBeat.getReportTime()));
@ -333,7 +341,7 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
}
return workerGroups;
}
protected void handleAddrList(WorkerGroup wg, String workerGroup, Collection<String> childrenNodes) {
wg.setAddrList(String.join(Constants.COMMA, childrenNodes));
}
@ -348,7 +356,7 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
@Transactional
public Map<String, Object> deleteWorkerGroupById(User loginUser, Integer id) {
Map<String, Object> result = new HashMap<>();
if (!canOperatorPermissions(loginUser,null, AuthorizationType.WORKER_GROUP,WORKER_GROUP_DELETE)) {
if (!canOperatorPermissions(loginUser, null, AuthorizationType.WORKER_GROUP, WORKER_GROUP_DELETE)) {
putMsg(result, Status.USER_NO_OPERATION_PERM);
return result;
}
@ -357,7 +365,8 @@ public class WorkerGroupServiceImpl extends BaseServiceImpl implements WorkerGro
putMsg(result, Status.DELETE_WORKER_GROUP_NOT_EXIST);
return result;
}
List<ProcessInstance> processInstances = processInstanceMapper.queryByWorkerGroupNameAndStatus(workerGroup.getName(), Constants.NOT_TERMINATED_STATES);
List<ProcessInstance> processInstances = processInstanceMapper
.queryByWorkerGroupNameAndStatus(workerGroup.getName(), Constants.NOT_TERMINATED_STATES);
if (CollectionUtils.isNotEmpty(processInstances)) {
putMsg(result, Status.DELETE_WORKER_GROUP_BY_ID_FAIL, processInstances.size());
return result;

62
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo/AlertPluginInstanceVO.java
View File

@ -19,9 +19,9 @@ package org.apache.dolphinscheduler.api.vo;
import java.util.Date;
/**
* AlertPluginInstanceVO
*/
import lombok.Data;
@Data
public class AlertPluginInstanceVO {
/**
@ -58,60 +58,4 @@ public class AlertPluginInstanceVO {
* alert plugin name
*/
private String alertPluginName;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getPluginDefineId() {
return pluginDefineId;
}
public void setPluginDefineId(int pluginDefineId) {
this.pluginDefineId = pluginDefineId;
}
public String getInstanceName() {
return instanceName;
}
public void setInstanceName(String instanceName) {
this.instanceName = instanceName;
}
public String getPluginInstanceParams() {
return pluginInstanceParams;
}
public void setPluginInstanceParams(String pluginInstanceParams) {
this.pluginInstanceParams = pluginInstanceParams;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getAlertPluginName() {
return alertPluginName;
}
public void setAlertPluginName(String alertPluginName) {
this.alertPluginName = alertPluginName;
}
}

190
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo/ScheduleVo.java
View File

@ -27,6 +27,9 @@ import org.apache.dolphinscheduler.dao.entity.Schedule;
import java.time.ZoneId;
import java.util.Date;
import lombok.Data;
@Data
public class ScheduleVo {
private int id;
@ -112,7 +115,6 @@ public class ScheduleVo {
*/
private int warningGroupId;
/**
* process instance priority
*/
@ -149,190 +151,4 @@ public class ScheduleVo {
this.setStartTime(DateUtils.dateToString(schedule.getStartTime(), ZoneId.systemDefault().getId()));
this.setEndTime(DateUtils.dateToString(schedule.getEndTime(), ZoneId.systemDefault().getId()));
}
public int getWarningGroupId() {
return warningGroupId;
}
public void setWarningGroupId(int warningGroupId) {
this.warningGroupId = warningGroupId;
}
public String getProjectName() {
return projectName;
}
public void setProjectName(String projectName) {
this.projectName = projectName;
}
public String getStartTime() {
return startTime;
}
public void setStartTime(String startTime) {
this.startTime = startTime;
}
public String getEndTime() {
return endTime;
}
public void setEndTime(String endTime) {
this.endTime = endTime;
}
public String getTimezoneId() {
return timezoneId;
}
public void setTimezoneId(String timezoneId) {
this.timezoneId = timezoneId;
}
public String getCrontab() {
return crontab;
}
public void setCrontab(String crontab) {
this.crontab = crontab;
}
public FailureStrategy getFailureStrategy() {
return failureStrategy;
}
public void setFailureStrategy(FailureStrategy failureStrategy) {
this.failureStrategy = failureStrategy;
}
public WarningType getWarningType() {
return warningType;
}
public void setWarningType(WarningType warningType) {
this.warningType = warningType;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public ReleaseState getReleaseState() {
return releaseState;
}
public void setReleaseState(ReleaseState releaseState) {
this.releaseState = releaseState;
}
public long getProcessDefinitionCode() {
return processDefinitionCode;
}
public void setProcessDefinitionCode(long processDefinitionCode) {
this.processDefinitionCode = processDefinitionCode;
}
public String getProcessDefinitionName() {
return processDefinitionName;
}
public void setProcessDefinitionName(String processDefinitionName) {
this.processDefinitionName = processDefinitionName;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public Priority getProcessInstancePriority() {
return processInstancePriority;
}
public void setProcessInstancePriority(Priority processInstancePriority) {
this.processInstancePriority = processInstancePriority;
}
public String getWorkerGroup() {
return workerGroup;
}
public void setWorkerGroup(String workerGroup) {
this.workerGroup = workerGroup;
}
public Long getEnvironmentCode() {
return this.environmentCode;
}
public void setEnvironmentCode(Long environmentCode) {
this.environmentCode = environmentCode;
}
@Override
public String toString() {
return "Schedule{"
+ "id=" + id
+ ", processDefinitionCode=" + processDefinitionCode
+ ", processDefinitionName='" + processDefinitionName + '\''
+ ", projectName='" + projectName + '\''
+ ", description='" + definitionDescription + '\''
+ ", startTime=" + startTime
+ ", endTime=" + endTime
+ ", timezoneId='" + timezoneId + +'\''
+ ", crontab='" + crontab + '\''
+ ", failureStrategy=" + failureStrategy
+ ", warningType=" + warningType
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", releaseState=" + releaseState
+ ", warningGroupId=" + warningGroupId
+ ", processInstancePriority=" + processInstancePriority
+ ", workerGroup='" + workerGroup + '\''
+ ", environmentCode='" + environmentCode + '\''
+ '}';
}
public String getDefinitionDescription() {
return definitionDescription;
}
public void setDefinitionDescription(String definitionDescription) {
this.definitionDescription = definitionDescription;
}
}

77
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/AlertGroupServiceTest.java
View File

@ -21,8 +21,8 @@ import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationCon
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import org.apache.commons.compress.utils.Sets;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.permission.ResourcePermissionCheckService;
import org.apache.dolphinscheduler.api.service.impl.AlertGroupServiceImpl;
import org.apache.dolphinscheduler.api.service.impl.BaseServiceImpl;
import org.apache.dolphinscheduler.api.utils.PageInfo;
@ -42,7 +42,6 @@ import java.util.List;
import java.util.Map;
import java.util.Set;
import org.apache.dolphinscheduler.api.permission.ResourcePermissionCheckService;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
@ -63,6 +62,7 @@ import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
*/
@RunWith(MockitoJUnitRunner.class)
public class AlertGroupServiceTest {
private static final Logger baseServiceLogger = LoggerFactory.getLogger(BaseServiceImpl.class);
private static final Logger logger = LoggerFactory.getLogger(AlertGroupServiceTest.class);
private static final Logger alertGroupServiceLogger = LoggerFactory.getLogger(AlertGroupServiceImpl.class);
@ -104,9 +104,9 @@ public class AlertGroupServiceTest {
Result result = alertGroupService.listPaging(user, groupName, 1, 10);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS.getCode(), (int) result.getCode());
//success
// success
user.setUserType(UserType.ADMIN_USER);
user.setId(1);
user.setId(0);
result = alertGroupService.listPaging(user, groupName, 1, 10);
logger.info(result.toString());
PageInfo<AlertGroup> pageInfo = (PageInfo<AlertGroup>) result.getData();
@ -119,16 +119,19 @@ public class AlertGroupServiceTest {
Mockito.when(alertGroupMapper.insert(any(AlertGroup.class))).thenReturn(2);
User user = new User();
//no operate
user.setId(0);
// no operate
user.setUserType(UserType.GENERAL_USER);
Map<String, Object> result = alertGroupService.createAlertgroup(user, groupName, groupName, null);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
user.setUserType(UserType.ADMIN_USER);
user.setId(1);
//success
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null, 1, ALERT_GROUP_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, null, 0, baseServiceLogger)).thenReturn(true);
user.setId(0);
// success
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), ALERT_GROUP_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), baseServiceLogger)).thenReturn(true);
result = alertGroupService.createAlertgroup(user, groupName, groupName, null);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -138,12 +141,15 @@ public class AlertGroupServiceTest {
@Test
public void testCreateAlertgroupDuplicate() {
Mockito.when(alertGroupMapper.insert(any(AlertGroup.class))).thenThrow(new DuplicateKeyException("group name exist"));
Mockito.when(alertGroupMapper.insert(any(AlertGroup.class)))
.thenThrow(new DuplicateKeyException("group name exist"));
User user = new User();
user.setUserType(UserType.ADMIN_USER);
user.setId(1);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null, 1, ALERT_GROUP_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, null, 0, baseServiceLogger)).thenReturn(true);
user.setId(0);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), ALERT_GROUP_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), baseServiceLogger)).thenReturn(true);
Map<String, Object> result = alertGroupService.createAlertgroup(user, groupName, groupName, null);
logger.info(result.toString());
Assert.assertEquals(Status.ALERT_GROUP_EXIST, result.get(Constants.STATUS));
@ -153,6 +159,7 @@ public class AlertGroupServiceTest {
public void testUpdateAlertgroup() {
User user = new User();
user.setId(0);
// no operate
user.setUserType(UserType.GENERAL_USER);
Map<String, Object> result = alertGroupService.updateAlertgroup(user, 1, groupName, groupName, null);
@ -161,13 +168,16 @@ public class AlertGroupServiceTest {
user.setUserType(UserType.ADMIN_USER);
// not exist
user.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null, user.getId(), ALERT_GROUP_UPDATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, new Object[]{1}, 0, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), ALERT_GROUP_UPDATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP,
new Object[]{1}, 0, baseServiceLogger)).thenReturn(true);
result = alertGroupService.updateAlertgroup(user, 1, groupName, groupName, null);
logger.info(result.toString());
Assert.assertEquals(Status.ALERT_GROUP_NOT_EXIST, result.get(Constants.STATUS));
//success
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, new Object[]{2}, user.getId(), baseServiceLogger)).thenReturn(true);
// success
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP,
new Object[]{2}, user.getId(), baseServiceLogger)).thenReturn(true);
Mockito.when(alertGroupMapper.selectById(2)).thenReturn(getEntity());
result = alertGroupService.updateAlertgroup(user, 2, groupName, groupName, null);
logger.info(result.toString());
@ -178,11 +188,15 @@ public class AlertGroupServiceTest {
@Test
public void testUpdateAlertgroupDuplicate() {
User user = new User();
user.setId(0);
user.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null, user.getId(), ALERT_GROUP_UPDATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, new Object[]{2}, user.getId(), baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), ALERT_GROUP_UPDATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP,
new Object[]{2}, user.getId(), baseServiceLogger)).thenReturn(true);
Mockito.when(alertGroupMapper.selectById(2)).thenReturn(getEntity());
Mockito.when(alertGroupMapper.updateById(Mockito.any())).thenThrow(new DuplicateKeyException("group name exist"));
Mockito.when(alertGroupMapper.updateById(Mockito.any()))
.thenThrow(new DuplicateKeyException("group name exist"));
Map<String, Object> result = alertGroupService.updateAlertgroup(user, 2, groupName, groupName, null);
Assert.assertEquals(Status.ALERT_GROUP_EXIST, result.get(Constants.STATUS));
}
@ -191,23 +205,28 @@ public class AlertGroupServiceTest {
public void testDelAlertgroupById() {
User user = new User();
user.setId(0);
// no operate
user.setUserType(UserType.GENERAL_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null, user.getId(),ALERT_GROUP_DELETE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), ALERT_GROUP_DELETE, baseServiceLogger)).thenReturn(true);
Map<String, Object> result = alertGroupService.delAlertgroupById(user, 1);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
// not exist
user.setUserType(UserType.ADMIN_USER);
user.setId(1);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP,null, user.getId(), ALERT_GROUP_DELETE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, new Object[]{2}, 0, baseServiceLogger)).thenReturn(true);
user.setId(0);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ALERT_GROUP, null,
user.getId(), ALERT_GROUP_DELETE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP,
new Object[]{2}, 0, baseServiceLogger)).thenReturn(true);
result = alertGroupService.delAlertgroupById(user, 2);
logger.info(result.toString());
Assert.assertEquals(Status.ALERT_GROUP_NOT_EXIST, result.get(Constants.STATUS));
//success
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP, new Object[]{2}, 0, baseServiceLogger)).thenReturn(true);
// success
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ALERT_GROUP,
new Object[]{2}, 0, baseServiceLogger)).thenReturn(true);
Mockito.when(alertGroupMapper.selectById(2)).thenReturn(getEntity());
result = alertGroupService.delAlertgroupById(user, 2);
logger.info(result.toString());
@ -217,12 +236,12 @@ public class AlertGroupServiceTest {
@Test
public void testVerifyGroupName() {
//group name not exist
// group name not exist
boolean result = alertGroupService.existGroupName(groupName);
Assert.assertFalse(result);
Mockito.when(alertGroupMapper.existGroupName(groupName)).thenReturn(true);
//group name exist
// group name exist
result = alertGroupService.existGroupName(groupName);
Assert.assertTrue(result);
}
@ -252,7 +271,7 @@ public class AlertGroupServiceTest {
*/
private AlertGroup getEntity() {
AlertGroup alertGroup = new AlertGroup();
alertGroup.setId(1);
alertGroup.setId(0);
alertGroup.setGroupName(groupName);
return alertGroup;
}

64
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ClusterServiceTest.java
View File

@ -48,10 +48,7 @@ import org.mockito.Mockito;
import org.mockito.junit.MockitoJUnitRunner;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.configurationprocessor.json.JSONException;
import org.springframework.boot.configurationprocessor.json.JSONObject;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
@ -75,85 +72,85 @@ public class ClusterServiceTest {
@Mock
private K8sManager k8sManager;
public static final String testUserName = "clusterServerTest";
public static final String clusterName = "Env1";
@Before
public void setUp(){
public void setUp() {
}
@After
public void after(){
public void after() {
}
@Test
public void testCreateCluster() {
User loginUser = getGeneralUser();
Map<String, Object> result = clusterService.createCluster(loginUser,clusterName,getConfig(),getDesc());
Map<String, Object> result = clusterService.createCluster(loginUser, clusterName, getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
loginUser = getAdminUser();
result = clusterService.createCluster(loginUser,clusterName,"",getDesc());
result = clusterService.createCluster(loginUser, clusterName, "", getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_CONFIG_IS_NULL, result.get(Constants.STATUS));
result = clusterService.createCluster(loginUser,"",getConfig(),getDesc());
result = clusterService.createCluster(loginUser, "", getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_NAME_IS_NULL, result.get(Constants.STATUS));
Mockito.when(clusterMapper.queryByClusterName(clusterName)).thenReturn(getCluster());
result = clusterService.createCluster(loginUser,clusterName,getConfig(),getDesc());
result = clusterService.createCluster(loginUser, clusterName, getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_NAME_EXISTS, result.get(Constants.STATUS));
Mockito.when(clusterMapper.insert(Mockito.any(Cluster.class))).thenReturn(1);
result = clusterService.createCluster(loginUser,"testName","testConfig","testDesc");
result = clusterService.createCluster(loginUser, "testName", "testConfig", "testDesc");
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
public void testCheckParams() {
Map<String, Object> result = clusterService.checkParams(clusterName,getConfig());
Map<String, Object> result = clusterService.checkParams(clusterName, getConfig());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
result = clusterService.checkParams("",getConfig());
result = clusterService.checkParams("", getConfig());
Assert.assertEquals(Status.CLUSTER_NAME_IS_NULL, result.get(Constants.STATUS));
result = clusterService.checkParams(clusterName,"");
result = clusterService.checkParams(clusterName, "");
Assert.assertEquals(Status.CLUSTER_CONFIG_IS_NULL, result.get(Constants.STATUS));
}
@Test
public void testUpdateClusterByCode() throws RemotingException {
User loginUser = getGeneralUser();
Map<String, Object> result = clusterService.updateClusterByCode(loginUser,1L,clusterName,getConfig(),getDesc());
Map<String, Object> result =
clusterService.updateClusterByCode(loginUser, 1L, clusterName, getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
loginUser = getAdminUser();
result = clusterService.updateClusterByCode(loginUser,1L,clusterName,"",getDesc());
result = clusterService.updateClusterByCode(loginUser, 1L, clusterName, "", getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_CONFIG_IS_NULL, result.get(Constants.STATUS));
result = clusterService.updateClusterByCode(loginUser,1L,"",getConfig(),getDesc());
result = clusterService.updateClusterByCode(loginUser, 1L, "", getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_NAME_IS_NULL, result.get(Constants.STATUS));
result = clusterService.updateClusterByCode(loginUser,2L,clusterName,getConfig(),getDesc());
result = clusterService.updateClusterByCode(loginUser, 2L, clusterName, getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_NOT_EXISTS, result.get(Constants.STATUS));
Mockito.when(clusterMapper.queryByClusterName(clusterName)).thenReturn(getCluster());
result = clusterService.updateClusterByCode(loginUser,2L,clusterName,getConfig(),getDesc());
result = clusterService.updateClusterByCode(loginUser, 2L, clusterName, getConfig(), getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.CLUSTER_NAME_EXISTS, result.get(Constants.STATUS));
Mockito.when(clusterMapper.updateById(Mockito.any(Cluster.class))).thenReturn(1);
Mockito.when(clusterMapper.queryByClusterCode(1L)).thenReturn(getCluster());
result = clusterService.updateClusterByCode(loginUser,1L,"testName",getConfig(),"test");
result = clusterService.updateClusterByCode(loginUser, 1L, "testName", getConfig(), "test");
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -162,12 +159,12 @@ public class ClusterServiceTest {
@Test
public void testQueryAllClusterList() {
Mockito.when(clusterMapper.queryAllClusterList()).thenReturn(Lists.newArrayList(getCluster()));
Map<String, Object> result = clusterService.queryAllClusterList();
Map<String, Object> result = clusterService.queryAllClusterList();
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
List<Cluster> list = (List<Cluster>)(result.get(Constants.DATA_LIST));
Assert.assertEquals(1,list.size());
List<Cluster> list = (List<Cluster>) (result.get(Constants.DATA_LIST));
Assert.assertEquals(1, list.size());
}
@Test
@ -175,7 +172,8 @@ public class ClusterServiceTest {
IPage<Cluster> page = new Page<>(1, 10);
page.setRecords(getList());
page.setTotal(1L);
Mockito.when(clusterMapper.queryClusterListPaging(Mockito.any(Page.class), Mockito.eq(clusterName))).thenReturn(page);
Mockito.when(clusterMapper.queryClusterListPaging(Mockito.any(Page.class), Mockito.eq(clusterName)))
.thenReturn(page);
Result result = clusterService.queryClusterListPaging(1, 10, clusterName);
logger.info(result.toString());
@ -188,12 +186,12 @@ public class ClusterServiceTest {
Mockito.when(clusterMapper.queryByClusterName(clusterName)).thenReturn(null);
Map<String, Object> result = clusterService.queryClusterByName(clusterName);
logger.info(result.toString());
Assert.assertEquals(Status.QUERY_CLUSTER_BY_NAME_ERROR,result.get(Constants.STATUS));
Assert.assertEquals(Status.QUERY_CLUSTER_BY_NAME_ERROR, result.get(Constants.STATUS));
Mockito.when(clusterMapper.queryByClusterName(clusterName)).thenReturn(getCluster());
result = clusterService.queryClusterByName(clusterName);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
@ -201,29 +199,29 @@ public class ClusterServiceTest {
Mockito.when(clusterMapper.queryByClusterCode(1L)).thenReturn(null);
Map<String, Object> result = clusterService.queryClusterByCode(1L);
logger.info(result.toString());
Assert.assertEquals(Status.QUERY_CLUSTER_BY_CODE_ERROR,result.get(Constants.STATUS));
Assert.assertEquals(Status.QUERY_CLUSTER_BY_CODE_ERROR, result.get(Constants.STATUS));
Mockito.when(clusterMapper.queryByClusterCode(1L)).thenReturn(getCluster());
result = clusterService.queryClusterByCode(1L);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
public void testDeleteClusterByCode() {
User loginUser = getGeneralUser();
Map<String, Object> result = clusterService.deleteClusterByCode(loginUser,1L);
Map<String, Object> result = clusterService.deleteClusterByCode(loginUser, 1L);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
loginUser = getAdminUser();
Mockito.when(clusterMapper.deleteByCode(1L)).thenReturn(1);
result = clusterService.deleteClusterByCode(loginUser,1L);
result = clusterService.deleteClusterByCode(loginUser, 1L);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
Mockito.when(k8sNamespaceMapper.selectCount(Mockito.any())).thenReturn(1);
result = clusterService.deleteClusterByCode(loginUser,1L);
Mockito.when(k8sNamespaceMapper.selectCount(Mockito.any())).thenReturn(1L);
result = clusterService.deleteClusterByCode(loginUser, 1L);
logger.info(result.toString());
Assert.assertEquals(Status.DELETE_CLUSTER_RELATED_NAMESPACE_EXISTS, result.get(Constants.STATUS));
}

48
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/DqRuleServiceTest.java
View File

@ -23,6 +23,7 @@ import static org.mockito.Mockito.when;
import org.apache.dolphinscheduler.api.ApiApplicationServer;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.permission.ResourcePermissionCheckService;
import org.apache.dolphinscheduler.api.service.impl.BaseServiceImpl;
import org.apache.dolphinscheduler.api.service.impl.DqRuleServiceImpl;
import org.apache.dolphinscheduler.api.utils.Result;
@ -44,7 +45,6 @@ import org.apache.dolphinscheduler.plugin.task.api.enums.dp.InputType;
import org.apache.dolphinscheduler.plugin.task.api.enums.dp.OptionSourceType;
import org.apache.dolphinscheduler.plugin.task.api.enums.dp.RuleType;
import org.apache.dolphinscheduler.plugin.task.api.enums.dp.ValueType;
import org.apache.dolphinscheduler.api.permission.ResourcePermissionCheckService;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.apache.dolphinscheduler.spi.params.base.FormType;
@ -71,6 +71,7 @@ import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
@RunWith(MockitoJUnitRunner.Silent.class)
@SpringBootTest(classes = ApiApplicationServer.class)
public class DqRuleServiceTest {
private static final Logger baseServiceLogger = LoggerFactory.getLogger(BaseServiceImpl.class);
@InjectMocks
@ -105,22 +106,22 @@ public class DqRuleServiceTest {
+ "\"statistics_execute_sql\",\"name\":\"统计值计算SQL\",\"type\":\"input\",\"title\":"
+ "\"统计值计算SQL\",\"validate\":[{\"required\":true,\"type\":\"string\",\"trigger\":\"blur\"}]}]";
when(dqRuleInputEntryMapper.getRuleInputEntryList(1)).thenReturn(getRuleInputEntryList());
Map<String,Object> result = dqRuleService.getRuleFormCreateJsonById(1);
Assert.assertEquals(json,result.get(Constants.DATA_LIST));
Map<String, Object> result = dqRuleService.getRuleFormCreateJsonById(1);
Assert.assertEquals(json, result.get(Constants.DATA_LIST));
}
@Test
public void testQueryAllRuleList() {
when(dqRuleMapper.selectList(new QueryWrapper<>())).thenReturn(getRuleList());
Map<String,Object> result = dqRuleService.queryAllRuleList();
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Map<String, Object> result = dqRuleService.queryAllRuleList();
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
public void testGetDatasourceOptionsById() {
when(dataSourceMapper.listAllDataSourceByType(DbType.MYSQL.getCode())).thenReturn(dataSourceList());
Map<String,Object> result = dqRuleService.queryAllRuleList();
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Map<String, Object> result = dqRuleService.queryAllRuleList();
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
@ -134,8 +135,10 @@ public class DqRuleServiceTest {
User loginUser = new User();
loginUser.setId(1);
loginUser.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.DATA_QUALITY, null, loginUser.getId(), null, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.DATA_QUALITY, null, 0, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.DATA_QUALITY, null,
loginUser.getId(), null, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.DATA_QUALITY, null, 0,
baseServiceLogger)).thenReturn(true);
Page<DqRule> page = new Page<>(1, 10);
page.setTotal(1);
page.setRecords(getRuleList());
@ -147,11 +150,11 @@ public class DqRuleServiceTest {
when(dqRuleExecuteSqlMapper.getExecuteSqlList(1)).thenReturn(getRuleExecuteSqlList());
Result result = dqRuleService.queryRuleListPaging(
loginUser,searchVal,0,"2020-01-01 00:00:00","2020-01-02 00:00:00",1,10);
Assert.assertEquals(Integer.valueOf(Status.SUCCESS.getCode()),result.getCode());
loginUser, searchVal, 0, "2020-01-01 00:00:00", "2020-01-02 00:00:00", 1, 10);
Assert.assertEquals(Integer.valueOf(Status.SUCCESS.getCode()), result.getCode());
}
private List<DataSource> dataSourceList() {
private List<DataSource> dataSourceList() {
List<DataSource> dataSourceList = new ArrayList<>();
DataSource dataSource = new DataSource();
dataSource.setId(1);
@ -191,40 +194,41 @@ public class DqRuleServiceTest {
srcConnectorType.setField("src_connector_type");
srcConnectorType.setType(FormType.SELECT.getFormType());
srcConnectorType.setCanEdit(true);
srcConnectorType.setShow(true);
srcConnectorType.setIsShow(true);
srcConnectorType.setValue("JDBC");
srcConnectorType.setPlaceholder("Please select the source connector type");
srcConnectorType.setOptionSourceType(OptionSourceType.DEFAULT.getCode());
srcConnectorType.setOptions("[{\"label\":\"HIVE\",\"value\":\"HIVE\"},{\"label\":\"JDBC\",\"value\":\"JDBC\"}]");
srcConnectorType
.setOptions("[{\"label\":\"HIVE\",\"value\":\"HIVE\"},{\"label\":\"JDBC\",\"value\":\"JDBC\"}]");
srcConnectorType.setInputType(InputType.DEFAULT.getCode());
srcConnectorType.setValueType(ValueType.NUMBER.getCode());
srcConnectorType.setEmit(true);
srcConnectorType.setValidate(true);
srcConnectorType.setIsEmit(true);
srcConnectorType.setIsValidate(true);
DqRuleInputEntry statisticsName = new DqRuleInputEntry();
statisticsName.setTitle("统计值名");
statisticsName.setField("statistics_name");
statisticsName.setType(FormType.INPUT.getFormType());
statisticsName.setCanEdit(true);
statisticsName.setShow(true);
statisticsName.setIsShow(true);
statisticsName.setPlaceholder("Please enter statistics name, the alias in statistics execute sql");
statisticsName.setOptionSourceType(OptionSourceType.DEFAULT.getCode());
statisticsName.setInputType(InputType.DEFAULT.getCode());
statisticsName.setValueType(ValueType.STRING.getCode());
statisticsName.setEmit(false);
statisticsName.setValidate(true);
statisticsName.setIsEmit(false);
statisticsName.setIsValidate(true);
DqRuleInputEntry statisticsExecuteSql = new DqRuleInputEntry();
statisticsExecuteSql.setTitle("统计值计算SQL");
statisticsExecuteSql.setField("statistics_execute_sql");
statisticsExecuteSql.setType(FormType.TEXTAREA.getFormType());
statisticsExecuteSql.setCanEdit(true);
statisticsExecuteSql.setShow(true);
statisticsExecuteSql.setIsShow(true);
statisticsExecuteSql.setPlaceholder("Please enter the statistics execute sql");
statisticsExecuteSql.setOptionSourceType(OptionSourceType.DEFAULT.getCode());
statisticsExecuteSql.setValueType(ValueType.LIKE_SQL.getCode());
statisticsExecuteSql.setEmit(false);
statisticsExecuteSql.setValidate(true);
statisticsExecuteSql.setIsEmit(false);
statisticsExecuteSql.setIsValidate(true);
list.add(srcConnectorType);
list.add(statisticsName);

67
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/EnvironmentServiceTest.java
View File

@ -96,34 +96,35 @@ public class EnvironmentServiceTest {
public void testCreateEnvironment() {
User loginUser = getGeneralUser();
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ENVIRONMENT, null,
loginUser.getId(),ENVIRONMENT_CREATE, baseServiceLogger)).thenReturn(true);
loginUser.getId(), ENVIRONMENT_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ENVIRONMENT, null,
0, baseServiceLogger)).thenReturn(true);
Map<String, Object> result = environmentService.createEnvironment(loginUser,environmentName,getConfig(),getDesc(),workerGroups);
Map<String, Object> result =
environmentService.createEnvironment(loginUser, environmentName, getConfig(), getDesc(), workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
loginUser = getAdminUser();
result = environmentService.createEnvironment(loginUser,environmentName,"",getDesc(),workerGroups);
result = environmentService.createEnvironment(loginUser, environmentName, "", getDesc(), workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_CONFIG_IS_NULL, result.get(Constants.STATUS));
result = environmentService.createEnvironment(loginUser,"",getConfig(),getDesc(),workerGroups);
result = environmentService.createEnvironment(loginUser, "", getConfig(), getDesc(), workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_NAME_IS_NULL, result.get(Constants.STATUS));
result = environmentService.createEnvironment(loginUser,environmentName,getConfig(),getDesc(),"test");
result = environmentService.createEnvironment(loginUser, environmentName, getConfig(), getDesc(), "test");
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_WORKER_GROUPS_IS_INVALID, result.get(Constants.STATUS));
Mockito.when(environmentMapper.queryByEnvironmentName(environmentName)).thenReturn(getEnvironment());
result = environmentService.createEnvironment(loginUser,environmentName,getConfig(),getDesc(),workerGroups);
result = environmentService.createEnvironment(loginUser, environmentName, getConfig(), getDesc(), workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_NAME_EXISTS, result.get(Constants.STATUS));
Mockito.when(environmentMapper.insert(Mockito.any(Environment.class))).thenReturn(1);
Mockito.when(relationMapper.insert(Mockito.any(EnvironmentWorkerGroupRelation.class))).thenReturn(1);
result = environmentService.createEnvironment(loginUser,"testName","test","test",workerGroups);
result = environmentService.createEnvironment(loginUser, "testName", "test", "test", workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -131,7 +132,7 @@ public class EnvironmentServiceTest {
@Test
public void testCheckParams() {
Map<String, Object> result = environmentService.checkParams(environmentName,getConfig(),"test");
Map<String, Object> result = environmentService.checkParams(environmentName, getConfig(), "test");
Assert.assertEquals(Status.ENVIRONMENT_WORKER_GROUPS_IS_INVALID, result.get(Constants.STATUS));
}
@ -139,33 +140,38 @@ public class EnvironmentServiceTest {
public void testUpdateEnvironmentByCode() {
User loginUser = getGeneralUser();
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ENVIRONMENT, null,
loginUser.getId(),ENVIRONMENT_UPDATE, baseServiceLogger)).thenReturn(true);
loginUser.getId(), ENVIRONMENT_UPDATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ENVIRONMENT, null,
0, baseServiceLogger)).thenReturn(true);
Map<String, Object> result = environmentService.updateEnvironmentByCode(loginUser,1L,environmentName,getConfig(),getDesc(),workerGroups);
Map<String, Object> result = environmentService.updateEnvironmentByCode(loginUser, 1L, environmentName,
getConfig(), getDesc(), workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
loginUser = getAdminUser();
result = environmentService.updateEnvironmentByCode(loginUser,1L,environmentName,"",getDesc(),workerGroups);
result = environmentService.updateEnvironmentByCode(loginUser, 1L, environmentName, "", getDesc(),
workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_CONFIG_IS_NULL, result.get(Constants.STATUS));
result = environmentService.updateEnvironmentByCode(loginUser,1L,"",getConfig(),getDesc(),workerGroups);
result = environmentService.updateEnvironmentByCode(loginUser, 1L, "", getConfig(), getDesc(), workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_NAME_IS_NULL, result.get(Constants.STATUS));
result = environmentService.updateEnvironmentByCode(loginUser,1L,environmentName,getConfig(),getDesc(),"test");
result = environmentService.updateEnvironmentByCode(loginUser, 1L, environmentName, getConfig(), getDesc(),
"test");
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_WORKER_GROUPS_IS_INVALID, result.get(Constants.STATUS));
Mockito.when(environmentMapper.queryByEnvironmentName(environmentName)).thenReturn(getEnvironment());
result = environmentService.updateEnvironmentByCode(loginUser,2L,environmentName,getConfig(),getDesc(),workerGroups);
result = environmentService.updateEnvironmentByCode(loginUser, 2L, environmentName, getConfig(), getDesc(),
workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.ENVIRONMENT_NAME_EXISTS, result.get(Constants.STATUS));
Mockito.when(environmentMapper.update(Mockito.any(Environment.class),Mockito.any(Wrapper.class))).thenReturn(1);
result = environmentService.updateEnvironmentByCode(loginUser,1L,"testName","test","test",workerGroups);
Mockito.when(environmentMapper.update(Mockito.any(Environment.class), Mockito.any(Wrapper.class)))
.thenReturn(1);
result = environmentService.updateEnvironmentByCode(loginUser, 1L, "testName", "test", "test", workerGroups);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@ -178,12 +184,12 @@ public class EnvironmentServiceTest {
1, environmentServiceLogger)).thenReturn(ids);
Mockito.when(environmentMapper.selectBatchIds(ids)).thenReturn(Lists.newArrayList(getEnvironment()));
Map<String, Object> result = environmentService.queryAllEnvironmentList(getAdminUser());
Map<String, Object> result = environmentService.queryAllEnvironmentList(getAdminUser());
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
List<Environment> list = (List<Environment>)(result.get(Constants.DATA_LIST));
Assert.assertEquals(1,list.size());
List<Environment> list = (List<Environment>) (result.get(Constants.DATA_LIST));
Assert.assertEquals(1, list.size());
}
@Test
@ -191,7 +197,8 @@ public class EnvironmentServiceTest {
IPage<Environment> page = new Page<>(1, 10);
page.setRecords(getList());
page.setTotal(1L);
Mockito.when(environmentMapper.queryEnvironmentListPaging(Mockito.any(Page.class), Mockito.eq(environmentName))).thenReturn(page);
Mockito.when(environmentMapper.queryEnvironmentListPaging(Mockito.any(Page.class), Mockito.eq(environmentName)))
.thenReturn(page);
Result result = environmentService.queryEnvironmentListPaging(getAdminUser(), 1, 10, environmentName);
logger.info(result.toString());
@ -204,12 +211,12 @@ public class EnvironmentServiceTest {
Mockito.when(environmentMapper.queryByEnvironmentName(environmentName)).thenReturn(null);
Map<String, Object> result = environmentService.queryEnvironmentByName(environmentName);
logger.info(result.toString());
Assert.assertEquals(Status.QUERY_ENVIRONMENT_BY_NAME_ERROR,result.get(Constants.STATUS));
Assert.assertEquals(Status.QUERY_ENVIRONMENT_BY_NAME_ERROR, result.get(Constants.STATUS));
Mockito.when(environmentMapper.queryByEnvironmentName(environmentName)).thenReturn(getEnvironment());
result = environmentService.queryEnvironmentByName(environmentName);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
@ -217,12 +224,12 @@ public class EnvironmentServiceTest {
Mockito.when(environmentMapper.queryByEnvironmentCode(1L)).thenReturn(null);
Map<String, Object> result = environmentService.queryEnvironmentByCode(1L);
logger.info(result.toString());
Assert.assertEquals(Status.QUERY_ENVIRONMENT_BY_CODE_ERROR,result.get(Constants.STATUS));
Assert.assertEquals(Status.QUERY_ENVIRONMENT_BY_CODE_ERROR, result.get(Constants.STATUS));
Mockito.when(environmentMapper.queryByEnvironmentCode(1L)).thenReturn(getEnvironment());
result = environmentService.queryEnvironmentByCode(1L);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS,result.get(Constants.STATUS));
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}
@Test
@ -232,19 +239,19 @@ public class EnvironmentServiceTest {
loginUser.getId(), ENVIRONMENT_DELETE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ENVIRONMENT, null,
0, baseServiceLogger)).thenReturn(true);
Map<String, Object> result = environmentService.deleteEnvironmentByCode(loginUser,1L);
Map<String, Object> result = environmentService.deleteEnvironmentByCode(loginUser, 1L);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
loginUser = getAdminUser();
Mockito.when(taskDefinitionMapper.selectCount(Mockito.any(LambdaQueryWrapper.class))).thenReturn(1);
result = environmentService.deleteEnvironmentByCode(loginUser,1L);
Mockito.when(taskDefinitionMapper.selectCount(Mockito.any(LambdaQueryWrapper.class))).thenReturn(1L);
result = environmentService.deleteEnvironmentByCode(loginUser, 1L);
logger.info(result.toString());
Assert.assertEquals(Status.DELETE_ENVIRONMENT_RELATED_TASK_EXISTS, result.get(Constants.STATUS));
Mockito.when(taskDefinitionMapper.selectCount(Mockito.any(LambdaQueryWrapper.class))).thenReturn(0);
Mockito.when(taskDefinitionMapper.selectCount(Mockito.any(LambdaQueryWrapper.class))).thenReturn(0L);
Mockito.when(environmentMapper.deleteByCode(1L)).thenReturn(1);
result = environmentService.deleteEnvironmentByCode(loginUser,1L);
result = environmentService.deleteEnvironmentByCode(loginUser, 1L);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
}

28
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/K8SNamespaceServiceTest.java
View File

@ -18,6 +18,7 @@
package org.apache.dolphinscheduler.api.service;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.k8s.K8sClientService;
import org.apache.dolphinscheduler.api.service.impl.K8SNamespaceServiceImpl;
import org.apache.dolphinscheduler.api.utils.PageInfo;
import org.apache.dolphinscheduler.api.utils.Result;
@ -29,10 +30,10 @@ import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.ClusterMapper;
import org.apache.dolphinscheduler.dao.mapper.K8sNamespaceMapper;
import org.apache.dolphinscheduler.dao.mapper.UserMapper;
import org.apache.dolphinscheduler.api.k8s.K8sClientService;
import org.apache.commons.collections.CollectionUtils;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@ -77,7 +78,9 @@ public class K8SNamespaceServiceTest {
@Before
public void setUp() throws Exception {
Mockito.when(k8sClientService.upsertNamespaceAndResourceToK8s(Mockito.any(K8sNamespace.class), Mockito.anyString())).thenReturn(null);
Mockito.when(
k8sClientService.upsertNamespaceAndResourceToK8s(Mockito.any(K8sNamespace.class), Mockito.anyString()))
.thenReturn(null);
Mockito.when(k8sClientService.deleteNamespaceToK8s(Mockito.anyString(), Mockito.anyLong())).thenReturn(null);
}
@ -90,7 +93,8 @@ public class K8SNamespaceServiceTest {
IPage<K8sNamespace> page = new Page<>(1, 10);
page.setTotal(1L);
page.setRecords(getNamespaceList());
Mockito.when(k8sNamespaceMapper.queryK8sNamespacePaging(Mockito.any(Page.class), Mockito.eq(namespace))).thenReturn(page);
Mockito.when(k8sNamespaceMapper.queryK8sNamespacePaging(Mockito.any(Page.class), Mockito.eq(namespace)))
.thenReturn(page);
Result result = k8sNamespaceService.queryListPaging(getLoginUser(), namespace, 1, 10);
logger.info(result.toString());
PageInfo<K8sNamespace> pageInfo = (PageInfo<K8sNamespace>) result.getData();
@ -100,7 +104,8 @@ public class K8SNamespaceServiceTest {
@Test
public void createK8sNamespace() {
// namespace is null
Map<String, Object> result = k8sNamespaceService.createK8sNamespace(getLoginUser(), null, clusterCode, 10.0, 100);
Map<String, Object> result =
k8sNamespaceService.createK8sNamespace(getLoginUser(), null, clusterCode, 10.0, 100);
logger.info(result.toString());
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
// k8s is null
@ -112,7 +117,7 @@ public class K8SNamespaceServiceTest {
result = k8sNamespaceService.createK8sNamespace(getLoginUser(), namespace, clusterCode, 10.0, 100);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
//null limit cpu and mem
// null limit cpu and mem
result = k8sNamespaceService.createK8sNamespace(getLoginUser(), namespace, clusterCode, null, null);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -140,22 +145,22 @@ public class K8SNamespaceServiceTest {
Mockito.when(k8sNamespaceMapper.existNamespace(namespace, clusterCode)).thenReturn(true);
//namespace null
// namespace null
Result result = k8sNamespaceService.verifyNamespaceK8s(null, clusterCode);
logger.info(result.toString());
Assert.assertEquals(result.getCode().intValue(), Status.REQUEST_PARAMS_NOT_VALID_ERROR.getCode());
//k8s null
// k8s null
result = k8sNamespaceService.verifyNamespaceK8s(namespace, null);
logger.info(result.toString());
Assert.assertEquals(result.getCode().intValue(), Status.REQUEST_PARAMS_NOT_VALID_ERROR.getCode());
//exist
// exist
result = k8sNamespaceService.verifyNamespaceK8s(namespace, clusterCode);
logger.info(result.toString());
Assert.assertEquals(result.getCode().intValue(), Status.K8S_NAMESPACE_EXIST.getCode());
//not exist
// not exist
result = k8sNamespaceService.verifyNamespaceK8s(namespace, 9999L);
logger.info(result.toString());
Assert.assertEquals(result.getCode().intValue(), Status.SUCCESS.getCode());
@ -163,7 +168,7 @@ public class K8SNamespaceServiceTest {
@Test
public void deleteNamespaceById() {
Mockito.when(k8sNamespaceMapper.deleteById(Mockito.any())).thenReturn(1);
Mockito.when(k8sNamespaceMapper.deleteById(Mockito.<Serializable>any())).thenReturn(1);
Mockito.when(k8sNamespaceMapper.selectById(1)).thenReturn(getNamespace());
Map<String, Object> result = k8sNamespaceService.deleteNamespaceById(getLoginUser(), 1);
@ -216,7 +221,6 @@ public class K8SNamespaceServiceTest {
Assert.assertTrue(CollectionUtils.isEmpty(namespaces));
}
private User getLoginUser() {
User loginUser = new User();
@ -248,4 +252,4 @@ public class K8SNamespaceServiceTest {
cluster.setOperator(1);
return cluster;
}
}
}

36
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProcessInstanceServiceTest.java
View File

@ -17,15 +17,11 @@
package org.apache.dolphinscheduler.api.service;
import java.io.IOException;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.INSTANCE_DELETE;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.INSTANCE_UPDATE;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.WORKFLOW_INSTANCE;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.when;
import org.apache.dolphinscheduler.api.enums.Status;
import org.apache.dolphinscheduler.api.service.impl.LoggerServiceImpl;
@ -37,9 +33,6 @@ import org.apache.dolphinscheduler.common.enums.CommandType;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.common.enums.WorkflowExecutionStatus;
import org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus;
import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
import org.apache.dolphinscheduler.service.expand.CuringParamsService;
import org.apache.dolphinscheduler.common.graph.DAG;
import org.apache.dolphinscheduler.common.model.TaskNode;
import org.apache.dolphinscheduler.common.model.TaskNodeRelation;
@ -63,9 +56,21 @@ import org.apache.dolphinscheduler.dao.mapper.ScheduleMapper;
import org.apache.dolphinscheduler.dao.mapper.TaskDefinitionMapper;
import org.apache.dolphinscheduler.dao.mapper.TaskInstanceMapper;
import org.apache.dolphinscheduler.dao.mapper.TenantMapper;
import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
import org.apache.dolphinscheduler.plugin.task.api.enums.DependResult;
import org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus;
import org.apache.dolphinscheduler.service.expand.CuringParamsService;
import org.apache.dolphinscheduler.service.process.ProcessService;
import org.apache.dolphinscheduler.service.task.TaskPluginManager;
import java.io.IOException;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
@ -74,11 +79,7 @@ import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.MockitoJUnitRunner;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.INSTANCE_DELETE;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.INSTANCE_UPDATE;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.WORKFLOW_INSTANCE;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.when;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
/**
* process instance service test
@ -343,6 +344,7 @@ public class ProcessInstanceServiceTest {
ProcessInstance processInstance = getProcessInstance();
processInstance.setState(WorkflowExecutionStatus.SUCCESS);
TaskInstance taskInstance = new TaskInstance();
taskInstance.setId(0);
taskInstance.setTaskType("SHELL");
List<TaskInstance> taskInstanceList = new ArrayList<>();
taskInstanceList.add(taskInstance);

109
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProjectServiceTest.java
View File

@ -96,19 +96,21 @@ public class ProjectServiceTest {
User loginUser = getLoginUser();
loginUser.setId(1);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, null, 1, PROJECT_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, null, 1, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, null, 1,
PROJECT_CREATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, null, 1,
baseServiceLogger)).thenReturn(true);
Result result = projectService.createProject(loginUser, projectName, getDesc());
logger.info(result.toString());
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR.getCode(), 10001);
//project name exist
// project name exist
Mockito.when(projectMapper.queryByName(projectName)).thenReturn(getProject());
result = projectService.createProject(loginUser, projectName, projectName);
logger.info(result.toString());
Assert.assertEquals(Status.PROJECT_ALREADY_EXISTS.getCode(), result.getCode().intValue());
//success
// success
Mockito.when(projectMapper.insert(Mockito.any(Project.class))).thenReturn(1);
result = projectService.createProject(loginUser, "test", "test");
logger.info(result.toString());
@ -128,20 +130,22 @@ public class ProjectServiceTest {
Assert.assertEquals(Status.PROJECT_NOT_EXIST, result.get(Constants.STATUS));
Project project = getProject();
//USER_NO_OPERATION_PROJECT_PERM
// USER_NO_OPERATION_PROJECT_PERM
project.setUserId(2);
result = projectService.checkProjectAndAuth(loginUser, project, projectCode, PROJECT);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PROJECT_PERM, result.get(Constants.STATUS));
//success
// success
project.setUserId(1);
loginUser.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{project.getId()},
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS,
new Object[]{project.getId()},
project.getUserId(), PROJECT, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{project.getId()},
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS,
new Object[]{project.getId()},
0, baseServiceLogger)).thenReturn(true);
result = projectService.checkProjectAndAuth(loginUser, project, projectCode,PROJECT);
result = projectService.checkProjectAndAuth(loginUser, project, projectCode, PROJECT);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -154,12 +158,13 @@ public class ProjectServiceTest {
// USER_NO_OPERATION_PROJECT_PERM
project1.setUserId(2);
loginUser.setUserType(UserType.GENERAL_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{project.getId()},
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS,
new Object[]{project.getId()},
loginUser.getId(), PROJECT, baseServiceLogger)).thenReturn(true);
result2 = projectService.checkProjectAndAuth(loginUser, project1, projectCode,PROJECT);
result2 = projectService.checkProjectAndAuth(loginUser, project1, projectCode, PROJECT);
Assert.assertEquals(Status.USER_NO_OPERATION_PROJECT_PERM, result2.get(Constants.STATUS));
//success
// success
project1.setUserId(1);
projectService.checkProjectAndAuth(loginUser, project1, projectCode, PROJECT);
@ -176,21 +181,24 @@ public class ProjectServiceTest {
User tempUser = new User();
tempUser.setId(Integer.MAX_VALUE);
tempUser.setUserType(UserType.GENERAL_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{project.getId()},
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS,
new Object[]{project.getId()},
tempUser.getId(), null, baseServiceLogger)).thenReturn(true);
boolean checkResult = projectService.hasProjectAndPerm(tempUser, project, result,null);
boolean checkResult = projectService.hasProjectAndPerm(tempUser, project, result, null);
logger.info(result.toString());
Assert.assertFalse(checkResult);
//success
// success
result = new HashMap<>();
project.setUserId(1);
loginUser.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{project.getId()},
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS,
new Object[]{project.getId()},
loginUser.getId(), null, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{project.getId()},
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS,
new Object[]{project.getId()},
0, baseServiceLogger)).thenReturn(true);
checkResult = projectService.hasProjectAndPerm(loginUser, project, result,null);
checkResult = projectService.hasProjectAndPerm(loginUser, project, result, null);
logger.info(result.toString());
Assert.assertTrue(checkResult);
}
@ -199,31 +207,35 @@ public class ProjectServiceTest {
public void testDeleteProject() {
User loginUser = getLoginUser();
Mockito.when(projectMapper.queryByCode(1L)).thenReturn(getProject());
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{1}, loginUser.getId(),
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS,
new Object[]{1}, loginUser.getId(),
PROJECT_DELETE, baseServiceLogger)).thenReturn(true);
//PROJECT_NOT_FOUNT
// PROJECT_NOT_FOUNT
Result result = projectService.deleteProject(loginUser, 11L);
logger.info(result.toString());
Assert.assertTrue(Status.PROJECT_NOT_EXIST.getCode() == result.getCode());
loginUser.setId(2);
//USER_NO_OPERATION_PROJECT_PERM
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},loginUser.getId(),
// USER_NO_OPERATION_PROJECT_PERM
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
loginUser.getId(),
baseServiceLogger)).thenReturn(true);
result = projectService.deleteProject(loginUser, 1L);
logger.info(result.toString());
Assert.assertTrue(Status.USER_NO_OPERATION_PROJECT_PERM.getCode() == result.getCode());
//DELETE_PROJECT_ERROR_DEFINES_NOT_NULL
// DELETE_PROJECT_ERROR_DEFINES_NOT_NULL
Mockito.when(processDefinitionMapper.queryAllDefinitionList(1L)).thenReturn(getProcessDefinitions());
loginUser.setUserType(UserType.ADMIN_USER);
loginUser.setId(1);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},0,
baseServiceLogger)).thenReturn(true);
Mockito.when(
resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1}, 0,
baseServiceLogger))
.thenReturn(true);
result = projectService.deleteProject(loginUser, 1L);
logger.info(result.toString());
Assert.assertTrue(Status.DELETE_PROJECT_ERROR_DEFINES_NOT_NULL.getCode() == result.getCode());
//success
// success
Mockito.when(projectMapper.deleteById(1)).thenReturn(1);
Mockito.when(processDefinitionMapper.queryAllDefinitionList(1L)).thenReturn(new ArrayList<>());
result = projectService.deleteProject(loginUser, 1L);
@ -235,11 +247,14 @@ public class ProjectServiceTest {
public void testUpdate() {
User loginUser = getLoginUser();
loginUser.setId(1);
Project project = getProject();
project.setCode(2L);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{1}, loginUser.getId(),
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS,
new Object[]{1}, loginUser.getId(),
PROJECT_UPDATE, baseServiceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},loginUser.getId(),
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
loginUser.getId(),
baseServiceLogger)).thenReturn(true);
Mockito.when(projectMapper.queryByName(projectName)).thenReturn(project);
Mockito.when(projectMapper.queryByCode(2L)).thenReturn(getProject());
@ -248,7 +263,7 @@ public class ProjectServiceTest {
logger.info(result.toString());
Assert.assertTrue(Status.PROJECT_NOT_FOUND.getCode() == result.getCode());
//PROJECT_ALREADY_EXISTS
// PROJECT_ALREADY_EXISTS
result = projectService.update(loginUser, 2L, projectName, "desc", userName);
logger.info(result.toString());
Assert.assertTrue(Status.PROJECT_ALREADY_EXISTS.getCode() == result.getCode());
@ -257,7 +272,7 @@ public class ProjectServiceTest {
result = projectService.update(loginUser, 2L, "test", "desc", "testuser");
Assert.assertTrue(Status.USER_NOT_EXIST.getCode() == result.getCode());
//success
// success
Mockito.when(userMapper.queryByUserNameAccurately(Mockito.any())).thenReturn(new User());
project.setUserId(1);
Mockito.when(projectMapper.updateById(Mockito.any(Project.class))).thenReturn(1);
@ -306,8 +321,10 @@ public class ProjectServiceTest {
// SUCCESS
loginUser.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
loginUser.getId(), PROJECT, baseServiceLogger)).thenReturn(true);
Mockito.when(
resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
loginUser.getId(), PROJECT, baseServiceLogger))
.thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
0, baseServiceLogger)).thenReturn(true);
Mockito.when(this.userMapper.queryAuthedUserListByProjectId(1)).thenReturn(this.getUserList());
@ -318,8 +335,10 @@ public class ProjectServiceTest {
loginUser.setId(1);
loginUser.setUserType(UserType.GENERAL_USER);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
loginUser.getId(), PROJECT, baseServiceLogger)).thenReturn(true);
Mockito.when(
resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
loginUser.getId(), PROJECT, baseServiceLogger))
.thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.PROJECTS, new Object[]{1},
1, baseServiceLogger)).thenReturn(true);
result = this.projectService.queryAuthorizedUser(loginUser, 3682329499136L);
@ -335,7 +354,7 @@ public class ProjectServiceTest {
Mockito.when(projectMapper.queryProjectCreatedByUser(1)).thenReturn(getList());
//success
// success
loginUser.setUserType(UserType.ADMIN_USER);
Map<String, Object> result = projectService.queryProjectCreatedByUser(loginUser);
logger.info(result.toString());
@ -361,7 +380,7 @@ public class ProjectServiceTest {
List<Project> notAdminUserResult = (List<Project>) result.getData();
Assert.assertTrue(CollectionUtils.isNotEmpty(notAdminUserResult));
//admin user
// admin user
loginUser.setUserType(UserType.ADMIN_USER);
Mockito.when(resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS,
loginUser.getId(), projectLogger)).thenReturn(set);
@ -396,8 +415,11 @@ public class ProjectServiceTest {
loginUser.setId(1);
List<Integer> list = new ArrayList<>(1);
list.add(1);
Mockito.when(resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS, loginUser.getId(), projectLogger)).thenReturn(set);
Mockito.when(projectMapper.listAuthorizedProjects(loginUser.getUserType().equals(UserType.ADMIN_USER) ? 0 : loginUser.getId(), list)).thenReturn(getList());
Mockito.when(resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS,
loginUser.getId(), projectLogger)).thenReturn(set);
Mockito.when(projectMapper.listAuthorizedProjects(
loginUser.getUserType().equals(UserType.ADMIN_USER) ? 0 : loginUser.getId(), list))
.thenReturn(getList());
Result result = projectService.queryUnauthorizedProject(loginUser, 2);
logger.info(result.toString());
List<Project> projects = (List<Project>) result.getData();
@ -408,7 +430,8 @@ public class ProjectServiceTest {
loginUser.setUserType(UserType.GENERAL_USER);
Mockito.when(resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.PROJECTS,
loginUser.getId(), projectLogger)).thenReturn(set);
Mockito.when(projectMapper.listAuthorizedProjects(loginUser.getUserType().equals(UserType.ADMIN_USER) ? 0 : loginUser.getId(),list))
Mockito.when(projectMapper.listAuthorizedProjects(
loginUser.getUserType().equals(UserType.ADMIN_USER) ? 0 : loginUser.getId(), list))
.thenReturn(getList());
result = projectService.queryUnauthorizedProject(loginUser, 3);
logger.info(result.toString());
@ -505,9 +528,9 @@ public class ProjectServiceTest {
private String getDesc() {
return "projectUserMapper.deleteProjectRelation(projectId,userId)projectUserMappe"
+ ".deleteProjectRelation(projectId,userId)projectUserMappe"
+ "r.deleteProjectRelation(projectId,userId)projectUserMapper"
+ ".deleteProjectRelation(projectId,userId)projectUserMapper.deleteProjectRelation(projectId,userId)";
+ ".deleteProjectRelation(projectId,userId)projectUserMappe"
+ "r.deleteProjectRelation(projectId,userId)projectUserMapper"
+ ".deleteProjectRelation(projectId,userId)projectUserMapper.deleteProjectRelation(projectId,userId)";
}
}
}

129
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/UsersServiceTest.java
View File

@ -18,7 +18,6 @@
package org.apache.dolphinscheduler.api.service;
import static org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant.USER_MANAGER;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.when;
@ -158,38 +157,39 @@ public class UsersServiceTest {
String phone = "13456432345";
int state = 1;
try {
//userName error
Map<String, Object> result = usersService.createUser(user, userName, userPassword, email, tenantId, phone, queueName, state);
// userName error
Map<String, Object> result =
usersService.createUser(user, userName, userPassword, email, tenantId, phone, queueName, state);
logger.info(result.toString());
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
userName = "userTest0001";
userPassword = "userTest000111111111111111";
//password error
// password error
result = usersService.createUser(user, userName, userPassword, email, tenantId, phone, queueName, state);
logger.info(result.toString());
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
userPassword = "userTest0001";
email = "1q.com";
//email error
// email error
result = usersService.createUser(user, userName, userPassword, email, tenantId, phone, queueName, state);
logger.info(result.toString());
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
email = "122222@qq.com";
phone = "2233";
//phone error
// phone error
result = usersService.createUser(user, userName, userPassword, email, tenantId, phone, queueName, state);
logger.info(result.toString());
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
phone = "13456432345";
//tenantId not exists
// tenantId not exists
result = usersService.createUser(user, userName, userPassword, email, tenantId, phone, queueName, state);
logger.info(result.toString());
Assert.assertEquals(Status.TENANT_NOT_EXIST, result.get(Constants.STATUS));
//success
// success
Mockito.when(tenantMapper.queryById(1)).thenReturn(getTenant());
result = usersService.createUser(user, userName, userPassword, email, 1, phone, queueName, state);
logger.info(result.toString());
@ -205,7 +205,8 @@ public class UsersServiceTest {
public void testQueryUser() {
String userName = "userTest0001";
String userPassword = "userTest0001";
when(userMapper.queryUserByNamePassword(userName, EncryptionUtils.getMd5(userPassword))).thenReturn(getGeneralUser());
when(userMapper.queryUserByNamePassword(userName, EncryptionUtils.getMd5(userPassword)))
.thenReturn(getGeneralUser());
User queryUser = usersService.queryUser(userName, userPassword);
logger.info(queryUser.toString());
Assert.assertTrue(queryUser != null);
@ -231,18 +232,18 @@ public class UsersServiceTest {
user.setUserType(UserType.ADMIN_USER);
user.setUserName("test_user");
//user name null
// user name null
int userId = usersService.getUserIdByName("");
Assert.assertEquals(0, userId);
//user not exist
// user not exist
when(usersService.queryUser(user.getUserName())).thenReturn(null);
int userNotExistId = usersService.getUserIdByName(user.getUserName());
Assert.assertEquals(-1, userNotExistId);
//user exist
// user exist
when(usersService.queryUser(user.getUserName())).thenReturn(user);
int userExistId = usersService.getUserIdByName(user.getUserName());
Integer userExistId = usersService.getUserIdByName(user.getUserName());
Assert.assertEquals(user.getId(), userExistId);
}
@ -252,15 +253,19 @@ public class UsersServiceTest {
user.setUserType(UserType.ADMIN_USER);
user.setId(1);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ACCESS_TOKEN,null, 1, USER_MANAGER, serviceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ACCESS_TOKEN, null, 0, serviceLogger)).thenReturn(false);
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ACCESS_TOKEN, null, 1,
USER_MANAGER, serviceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ACCESS_TOKEN, null, 0,
serviceLogger)).thenReturn(false);
Map<String, Object> result = usersService.queryUserList(user);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
//success
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ACCESS_TOKEN,null, 1, USER_MANAGER, serviceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ACCESS_TOKEN, null, 0, serviceLogger)).thenReturn(true);
// success
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.ACCESS_TOKEN, null, 1,
USER_MANAGER, serviceLogger)).thenReturn(true);
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.ACCESS_TOKEN, null, 0,
serviceLogger)).thenReturn(true);
user.setUserType(UserType.ADMIN_USER);
when(userMapper.queryEnabledUsers()).thenReturn(getUserList());
result = usersService.queryUserList(user);
@ -275,12 +280,12 @@ public class UsersServiceTest {
page.setRecords(getUserList());
when(userMapper.queryUserPaging(any(Page.class), eq("userTest"))).thenReturn(page);
//no operate
// no operate
Result result = usersService.queryUserList(user, "userTest", 1, 10);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM.getCode(), (int) result.getCode());
//success
// success
user.setUserType(UserType.ADMIN_USER);
result = usersService.queryUserList(user, "userTest", 1, 10);
Assert.assertEquals(Status.SUCCESS.getCode(), (int) result.getCode());
@ -293,14 +298,16 @@ public class UsersServiceTest {
String userName = "userTest0001";
String userPassword = "userTest0001";
try {
//user not exist
Map<String, Object> result = usersService.updateUser(getLoginUser(), 0, userName, userPassword, "3443@qq.com", 1, "13457864543", "queue", 1, "Asia/Shanghai");
// user not exist
Map<String, Object> result = usersService.updateUser(getLoginUser(), 0, userName, userPassword,
"3443@qq.com", 1, "13457864543", "queue", 1, "Asia/Shanghai");
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
logger.info(result.toString());
//success
// success
when(userMapper.selectById(1)).thenReturn(getUser());
result = usersService.updateUser(getLoginUser(), 1, userName, userPassword, "32222s@qq.com", 1, "13457864543", "queue", 1, "Asia/Shanghai");
result = usersService.updateUser(getLoginUser(), 1, userName, userPassword, "32222s@qq.com", 1,
"13457864543", "queue", 1, "Asia/Shanghai");
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
} catch (Exception e) {
@ -316,7 +323,7 @@ public class UsersServiceTest {
when(userMapper.queryTenantCodeByUserId(1)).thenReturn(getUser());
when(userMapper.selectById(1)).thenReturn(getUser());
when(accessTokenMapper.deleteAccessTokenByUserId(1)).thenReturn(0);
//no operate
// no operate
Map<String, Object> result = usersService.deleteUserById(loginUser, 3);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
@ -332,7 +339,7 @@ public class UsersServiceTest {
result = usersService.deleteUserById(loginUser, 1);
Assert.assertEquals(Status.TRANSFORM_PROJECT_OWNERSHIP, result.get(Constants.STATUS));
//success
// success
Mockito.when(projectMapper.queryProjectCreatedByUser(1)).thenReturn(null);
result = usersService.deleteUserById(loginUser, 1);
logger.info(result.toString());
@ -350,7 +357,7 @@ public class UsersServiceTest {
User loginUser = new User();
int userId = 3;
//user not exist
// user not exist
loginUser.setId(1);
loginUser.setUserType(UserType.ADMIN_USER);
when(userMapper.selectById(userId)).thenReturn(null);
@ -358,7 +365,7 @@ public class UsersServiceTest {
logger.info(result.toString());
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
//SUCCESS
// SUCCESS
when(userMapper.selectById(userId)).thenReturn(getUser());
result = usersService.grantProject(loginUser, userId, projectIds);
logger.info(result.toString());
@ -416,6 +423,7 @@ public class UsersServiceTest {
// user no permission
User loginUser = new User();
loginUser.setId(0);
Map<String, Object> result = this.usersService.revokeProject(loginUser, 1, projectCode);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
@ -427,7 +435,9 @@ public class UsersServiceTest {
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
// success
Mockito.when(this.projectMapper.queryByCode(Mockito.anyLong())).thenReturn(new Project());
Project project = new Project();
project.setId(0);
Mockito.when(this.projectMapper.queryByCode(Mockito.anyLong())).thenReturn(project);
result = this.usersService.revokeProject(loginUser, 1, projectCode);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -439,12 +449,12 @@ public class UsersServiceTest {
when(userMapper.selectById(1)).thenReturn(getUser());
User loginUser = new User();
//user not exist
// user not exist
loginUser.setUserType(UserType.ADMIN_USER);
Map<String, Object> result = usersService.grantResources(loginUser, 2, resourceIds);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
//success
// success
when(resourceMapper.selectById(Mockito.anyInt())).thenReturn(getResource());
when(resourceUserMapper.deleteResourceUser(1, 0)).thenReturn(1);
result = usersService.grantResources(loginUser, 1, resourceIds);
@ -459,12 +469,12 @@ public class UsersServiceTest {
when(userMapper.selectById(1)).thenReturn(getUser());
User loginUser = new User();
//user not exist
// user not exist
loginUser.setUserType(UserType.ADMIN_USER);
Map<String, Object> result = usersService.grantUDFFunction(loginUser, 2, udfIds);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
//success
// success
when(udfUserMapper.deleteByUserId(1)).thenReturn(1);
result = usersService.grantUDFFunction(loginUser, 1, udfIds);
logger.info(result.toString());
@ -477,12 +487,12 @@ public class UsersServiceTest {
when(userMapper.selectById(1)).thenReturn(getUser());
User loginUser = new User();
//user not exist
// user not exist
loginUser.setUserType(UserType.ADMIN_USER);
Map<String, Object> result = usersService.grantNamespaces(loginUser, 2, namespaceIds);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
//success
// success
when(k8sNamespaceUserMapper.deleteNamespaceRelation(0, 1)).thenReturn(1);
result = usersService.grantNamespaces(loginUser, 1, namespaceIds);
logger.info(result.toString());
@ -495,7 +505,7 @@ public class UsersServiceTest {
User loginUser = new User();
int userId = 3;
//user not exist
// user not exist
loginUser.setId(1);
loginUser.setUserType(UserType.ADMIN_USER);
when(userMapper.selectById(userId)).thenReturn(null);
@ -536,10 +546,10 @@ public class UsersServiceTest {
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
User tempUser = (User) result.get(Constants.DATA_LIST);
//check userName
// check userName
Assert.assertEquals("admin", tempUser.getUserName());
//get general user
// get general user
loginUser.setUserType(null);
loginUser.setId(1);
when(userMapper.queryDetailsById(1)).thenReturn(getGeneralUser());
@ -548,18 +558,18 @@ public class UsersServiceTest {
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
tempUser = (User) result.get(Constants.DATA_LIST);
//check userName
// check userName
Assert.assertEquals("userTest0001", tempUser.getUserName());
}
@Test
public void testQueryAllGeneralUsers() {
User loginUser = new User();
//no operate
// no operate
Map<String, Object> result = usersService.queryAllGeneralUsers(loginUser);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
//success
// success
loginUser.setUserType(UserType.ADMIN_USER);
when(userMapper.queryAllGeneralUser()).thenReturn(getUserList());
result = usersService.queryAllGeneralUsers(loginUser);
@ -571,11 +581,11 @@ public class UsersServiceTest {
@Test
public void testVerifyUserName() {
//not exist user
// not exist user
Result result = usersService.verifyUserName("admin89899");
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS.getMsg(), result.getMsg());
//exist user
// exist user
when(userMapper.queryByUserNameAccurately("userTest0001")).thenReturn(getUser());
result = usersService.verifyUserName("userTest0001");
logger.info(result.toString());
@ -587,12 +597,12 @@ public class UsersServiceTest {
User loginUser = new User();
when(userMapper.selectList(null)).thenReturn(getUserList());
when(userMapper.queryUserListByAlertGroupId(2)).thenReturn(getUserList());
//no operate
// no operate
Map<String, Object> result = usersService.unauthorizedUser(loginUser, 2);
logger.info(result.toString());
loginUser.setUserType(UserType.ADMIN_USER);
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
//success
// success
result = usersService.unauthorizedUser(loginUser, 2);
logger.info(result.toString());
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -602,11 +612,11 @@ public class UsersServiceTest {
public void testAuthorizedUser() {
User loginUser = new User();
when(userMapper.queryUserListByAlertGroupId(2)).thenReturn(getUserList());
//no operate
// no operate
Map<String, Object> result = usersService.authorizedUser(loginUser, 2);
logger.info(result.toString());
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
//success
// success
loginUser.setUserType(UserType.ADMIN_USER);
result = usersService.authorizedUser(loginUser, 2);
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -622,29 +632,29 @@ public class UsersServiceTest {
String repeatPassword = "userTest";
String email = "123@qq.com";
try {
//userName error
// userName error
Map<String, Object> result = usersService.registerUser(userName, userPassword, repeatPassword, email);
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
userName = "userTest0002";
userPassword = "userTest000111111111111111";
//password error
// password error
result = usersService.registerUser(userName, userPassword, repeatPassword, email);
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
userPassword = "userTest0002";
email = "1q.com";
//email error
// email error
result = usersService.registerUser(userName, userPassword, repeatPassword, email);
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
//repeatPassword error
// repeatPassword error
email = "7400@qq.com";
repeatPassword = "userPassword";
result = usersService.registerUser(userName, userPassword, repeatPassword, email);
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
//success
// success
repeatPassword = "userTest0002";
result = usersService.registerUser(userName, userPassword, repeatPassword, email);
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -660,27 +670,27 @@ public class UsersServiceTest {
user.setUserType(UserType.GENERAL_USER);
String userName = "userTest0002~";
try {
//not admin
// not admin
Map<String, Object> result = usersService.activateUser(user, userName);
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
//userName error
// userName error
user.setUserType(UserType.ADMIN_USER);
result = usersService.activateUser(user, userName);
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
//user not exist
// user not exist
userName = "userTest10013";
result = usersService.activateUser(user, userName);
Assert.assertEquals(Status.USER_NOT_EXIST, result.get(Constants.STATUS));
//user state error
// user state error
userName = "userTest0001";
when(userMapper.queryByUserNameAccurately(userName)).thenReturn(getUser());
result = usersService.activateUser(user, userName);
Assert.assertEquals(Status.REQUEST_PARAMS_NOT_VALID_ERROR, result.get(Constants.STATUS));
//success
// success
when(userMapper.queryByUserNameAccurately(userName)).thenReturn(getDisabledUser());
result = usersService.activateUser(user, userName);
Assert.assertEquals(Status.SUCCESS, result.get(Constants.STATUS));
@ -700,11 +710,11 @@ public class UsersServiceTest {
userNames.add("userTest0004");
try {
//not admin
// not admin
Map<String, Object> result = usersService.batchActivateUser(user, userNames);
Assert.assertEquals(Status.USER_NO_OPERATION_PERM, result.get(Constants.STATUS));
//batch activate user names
// batch activate user names
user.setUserType(UserType.ADMIN_USER);
when(userMapper.queryByUserNameAccurately("userTest0001")).thenReturn(getUser());
when(userMapper.queryByUserNameAccurately("userTest0002")).thenReturn(getDisabledUser());
@ -799,6 +809,7 @@ public class UsersServiceTest {
*/
private User getUser() {
User user = new User();
user.setId(0);
user.setUserType(UserType.ADMIN_USER);
user.setUserName("userTest0001");
user.setUserPassword("userTest0001");

39
dolphinscheduler-bom/pom.xml
View File

@ -32,11 +32,11 @@
<spring-boot.version>2.7.3</spring-boot.version>
<spring.version>5.3.19</spring.version>
<java-websocket.version>1.5.1</java-websocket.version>
<mybatis-plus.version>3.2.0</mybatis-plus.version>
<mybatis-plus.version>3.5.2</mybatis-plus.version>
<quartz.version>2.3.2</quartz.version>
<druid.version>1.2.4</druid.version>
<zookeeper.version>3.4.14</zookeeper.version>
<curator.version>4.3.0</curator.version>
<zookeeper.version>3.8.0</zookeeper.version>
<curator.version>5.3.0</curator.version>
<curator-test.version>2.12.0</curator-test.version>
<jetcd.version>0.5.11</jetcd.version>
<jetcd.test.version>0.7.1</jetcd.test.version>
@ -55,9 +55,9 @@
<protostuff.version>1.7.2</protostuff.version>
<byte-buddy.version>1.9.16</byte-buddy.version>
<logback.version>1.2.11</logback.version>
<hadoop.version>2.7.3</hadoop.version>
<cron-utils.version>9.1.3</cron-utils.version>
<h2.version>1.4.200</h2.version>
<hadoop.version>2.7.7</hadoop.version>
<cron-utils.version>9.1.6</cron-utils.version>
<h2.version>2.1.210</h2.version>
<mysql-connector.version>8.0.16</mysql-connector.version>
<oracle-jdbc.version>21.5.0.0</oracle-jdbc.version>
<slf4j.version>1.7.36</slf4j.version>
@ -66,8 +66,8 @@
<activation.version>1.1</activation.version>
<javax-mail>1.6.2</javax-mail>
<guava.version>24.1-jre</guava.version>
<postgresql.version>42.3.4</postgresql.version>
<hive-jdbc.version>2.1.0</hive-jdbc.version>
<postgresql.version>42.4.1</postgresql.version>
<hive-jdbc.version>2.3.3</hive-jdbc.version>
<commons-io.version>2.11.0</commons-io.version>
<oshi-core.version>6.1.1</oshi-core.version>
<clickhouse-jdbc.version>0.1.52</clickhouse-jdbc.version>
@ -89,6 +89,9 @@
<okhttp.version>3.14.9</okhttp.version>
<json-path.version>2.7.0</json-path.version>
<spring-cloud-dependencies.version>2021.0.3</spring-cloud-dependencies.version>
<gson.version>2.9.1</gson.version>
<dropwizard.metrics-version>4.2.11</dropwizard.metrics-version>
<snappy.version>1.1.8.4</snappy.version>
</properties>
<dependencyManagement>
@ -207,6 +210,16 @@
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>${dropwizard.metrics-version}</version>
</dependency>
<dependency>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
<version>${snappy.version}</version>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
@ -243,7 +256,7 @@
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-test</artifactId>
<version>${curator-test.version}</version>
<version>${curator.version}</version>
</dependency>
<!-- Etcd -->
@ -262,8 +275,8 @@
<groupId>io.grpc</groupId>
<artifactId>grpc-bom</artifactId>
<version>${io.grpc.version}</version>
<scope>import</scope>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
@ -645,6 +658,12 @@
<artifactId>spring-ldap</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>${gson.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

62
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/model/WorkerServerModel.java
View File

@ -20,9 +20,9 @@ package org.apache.dolphinscheduler.common.model;
import java.util.Date;
import java.util.Set;
/**
* server
*/
import lombok.Data;
@Data
public class WorkerServerModel {
/**
@ -59,60 +59,4 @@ public class WorkerServerModel {
* last heart beat time
*/
private Date lastHeartbeatTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Set<String> getZkDirectories() {
return zkDirectories;
}
public void setZkDirectories(Set<String> zkDirectories) {
this.zkDirectories = zkDirectories;
}
public Date getLastHeartbeatTime() {
return lastHeartbeatTime;
}
public void setLastHeartbeatTime(Date lastHeartbeatTime) {
this.lastHeartbeatTime = lastHeartbeatTime;
}
public String getResInfo() {
return resInfo;
}
public void setResInfo(String resInfo) {
this.resInfo = resInfo;
}
public String getHost() {
return host;
}
public void setHost(String host) {
this.host = host;
}
public int getPort() {
return port;
}
public void setPort(int port) {
this.port = port;
}
}

52
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/datasource/SpringConnectionFactory.java
View File

@ -24,21 +24,24 @@ import org.apache.ibatis.type.JdbcType;
import java.util.Properties;
import javax.annotation.Resource;
import javax.sql.DataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.jdbc.init.DataSourceScriptDatabaseInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import com.baomidou.mybatisplus.annotation.DbType;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.core.MybatisConfiguration;
import com.baomidou.mybatisplus.core.config.GlobalConfig;
import com.baomidou.mybatisplus.extension.plugins.PaginationInterceptor;
import com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor;
import com.baomidou.mybatisplus.extension.plugins.inner.PaginationInnerInterceptor;
import com.baomidou.mybatisplus.extension.spring.MybatisSqlSessionFactoryBean;
@Configuration
@ -51,8 +54,10 @@ public class SpringConnectionFactory {
public DataSourceScriptDatabaseInitializer dataSourceScriptDatabaseInitializer;
@Bean
public PaginationInterceptor paginationInterceptor() {
return new PaginationInterceptor();
public MybatisPlusInterceptor paginationInterceptor(DbType dbType) {
MybatisPlusInterceptor interceptor = new MybatisPlusInterceptor();
interceptor.addInnerInterceptor(new PaginationInnerInterceptor(dbType));
return interceptor;
}
@Bean
@ -61,32 +66,34 @@ public class SpringConnectionFactory {
}
@Bean
public SqlSessionFactory sqlSessionFactory(DataSource dataSource) throws Exception {
public SqlSessionFactory sqlSessionFactory(DataSource dataSource, GlobalConfig globalConfig,
DbType dbType) throws Exception {
MybatisConfiguration configuration = new MybatisConfiguration();
configuration.setMapUnderscoreToCamelCase(true);
configuration.setCacheEnabled(false);
configuration.setCallSettersOnNulls(true);
configuration.setJdbcTypeForNull(JdbcType.NULL);
configuration.addInterceptor(paginationInterceptor());
configuration.addInterceptor(paginationInterceptor(dbType));
configuration.setGlobalConfig(new GlobalConfig().setBanner(false));
MybatisSqlSessionFactoryBean sqlSessionFactoryBean = new MybatisSqlSessionFactoryBean();
sqlSessionFactoryBean.setConfiguration(configuration);
sqlSessionFactoryBean.setDataSource(dataSource);
GlobalConfig.DbConfig dbConfig = new GlobalConfig.DbConfig();
dbConfig.setIdType(IdType.AUTO);
GlobalConfig globalConfig = new GlobalConfig().setBanner(false);
globalConfig.setDbConfig(dbConfig);
sqlSessionFactoryBean.setGlobalConfig(globalConfig);
sqlSessionFactoryBean.setTypeAliasesPackage("org.apache.dolphinscheduler.dao.entity");
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
sqlSessionFactoryBean.setMapperLocations(resolver.getResources("org/apache/dolphinscheduler/dao/mapper/*Mapper.xml"));
sqlSessionFactoryBean.setTypeEnumsPackage("org.apache.dolphinscheduler.**.enums");
sqlSessionFactoryBean
.setMapperLocations(resolver.getResources("org/apache/dolphinscheduler/dao/mapper/*Mapper.xml"));
sqlSessionFactoryBean.setDatabaseIdProvider(databaseIdProvider());
return sqlSessionFactoryBean.getObject();
}
@Bean
public GlobalConfig globalConfig() {
return new GlobalConfig().setDbConfig(new GlobalConfig.DbConfig()
.setIdType(IdType.AUTO)).setBanner(false);
}
@Bean
public DatabaseIdProvider databaseIdProvider() {
DatabaseIdProvider databaseIdProvider = new VendorDatabaseIdProvider();
@ -97,4 +104,23 @@ public class SpringConnectionFactory {
databaseIdProvider.setProperties(properties);
return databaseIdProvider;
}
@Bean
@Primary
@Profile("mysql")
public DbType mysql() {
return DbType.MYSQL;
}
@Bean
public DbType h2() {
return DbType.H2;
}
@Bean
@Primary
@Profile("postgresql")
public DbType postgresql() {
return DbType.POSTGRE_SQL;
}
}

75
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AccessToken.java
View File

@ -19,18 +19,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_access_token")
public class AccessToken {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user_id
*/
@ -59,62 +63,6 @@ public class AccessToken {
@TableField(exist = false)
private String userName;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getToken() {
return token;
}
public void setToken(String token) {
this.token = token;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public Date getExpireTime() {
return expireTime;
}
public void setExpireTime(Date expireTime) {
this.expireTime = expireTime;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -160,17 +108,4 @@ public class AccessToken {
result = 31 * result + (updateTime != null ? updateTime.hashCode() : 0);
return result;
}
@Override
public String toString() {
return "AccessToken{"
+ "id=" + id
+ ", userId=" + userId
+ ", token='" + token + '\''
+ ", userName='" + userName + '\''
+ ", expireTime=" + expireTime
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

80
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertGroup.java
View File

@ -19,18 +19,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_alertgroup")
public class AlertGroup {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* group_name
*/
@ -62,62 +66,6 @@ public class AlertGroup {
@TableField(value = "create_user_id")
private int createUserId;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getGroupName() {
return groupName;
}
public void setGroupName(String groupName) {
this.groupName = groupName;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public int getCreateUserId() {
return createUserId;
}
public void setCreateUserId(int createUserId) {
this.createUserId = createUserId;
}
public String getAlertInstanceIds() {
return alertInstanceIds;
}
public void setAlertInstanceIds(String alertInstanceIds) {
this.alertInstanceIds = alertInstanceIds;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -138,13 +86,15 @@ public class AlertGroup {
if (groupName != null ? !groupName.equals(that.groupName) : that.groupName != null) {
return false;
}
if (alertInstanceIds != null ? !alertInstanceIds.equals(that.alertInstanceIds) : that.alertInstanceIds != null) {
if (alertInstanceIds != null ? !alertInstanceIds.equals(that.alertInstanceIds)
: that.alertInstanceIds != null) {
return false;
}
if (description != null ? !description.equals(that.description) : that.description != null) {
return false;
}
return !(createTime != null ? !createTime.equals(that.createTime) : that.createTime != null) && !(updateTime != null ? !updateTime.equals(that.updateTime) : that.updateTime != null);
return !(createTime != null ? !createTime.equals(that.createTime) : that.createTime != null)
&& !(updateTime != null ? !updateTime.equals(that.updateTime) : that.updateTime != null);
}
@ -159,16 +109,4 @@ public class AlertGroup {
result = 31 * result + (updateTime != null ? updateTime.hashCode() : 0);
return result;
}
@Override
public String toString() {
return "AlertGroup{"
+ "id=" + id
+ "createUserId=" + createUserId
+ ", groupName='" + groupName + '\''
+ ", description='" + description + '\''
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

57
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertPluginInstance.java
View File

@ -19,15 +19,15 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.FieldStrategy;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* t_ds_alert_plugin_instance
*/
@Data
@TableName("t_ds_alert_plugin_instance")
public class AlertPluginInstance {
@ -35,7 +35,7 @@ public class AlertPluginInstance {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* plugin_define_id
@ -86,53 +86,4 @@ public class AlertPluginInstance {
this.updateTime = updateDate;
this.instanceName = instanceName;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getPluginDefineId() {
return pluginDefineId;
}
public void setPluginDefineId(int pluginDefineId) {
this.pluginDefineId = pluginDefineId;
}
public String getPluginInstanceParams() {
return pluginInstanceParams;
}
public void setPluginInstanceParams(String pluginInstanceParams) {
this.pluginInstanceParams = pluginInstanceParams;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getInstanceName() {
return instanceName;
}
public void setInstanceName(String instanceName) {
this.instanceName = instanceName;
}
}

67
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AlertSendStatus.java
View File

@ -20,7 +20,8 @@ package org.apache.dolphinscheduler.dao.entity;
import org.apache.dolphinscheduler.common.enums.AlertStatus;
import java.util.Date;
import java.util.StringJoiner;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
@ -28,13 +29,15 @@ import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.google.common.base.Objects;
@Data
@TableName("t_ds_alert_send_status")
public class AlertSendStatus {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* alert id
@ -66,54 +69,6 @@ public class AlertSendStatus {
@TableField("create_time")
private Date createTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getAlertId() {
return alertId;
}
public void setAlertId(int alertId) {
this.alertId = alertId;
}
public int getAlertPluginInstanceId() {
return alertPluginInstanceId;
}
public void setAlertPluginInstanceId(int alertPluginInstanceId) {
this.alertPluginInstanceId = alertPluginInstanceId;
}
public AlertStatus getSendStatus() {
return sendStatus;
}
public void setSendStatus(AlertStatus sendStatus) {
this.sendStatus = sendStatus;
}
public String getLog() {
return log;
}
public void setLog(String log) {
this.log = log;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -130,16 +85,4 @@ public class AlertSendStatus {
public int hashCode() {
return Objects.hashCode(alertId, alertPluginInstanceId);
}
@Override
public String toString() {
return new StringJoiner(", ", AlertSendStatus.class.getSimpleName() + "[", "]")
.add("id=" + id)
.add("alertId=" + alertId)
.add("alertPluginInstanceId=" + alertPluginInstanceId)
.add("sendStatus=" + sendStatus)
.add("log='" + log + "'")
.add("createTime=" + createTime)
.toString();
}
}

2
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/AuditLog.java
View File

@ -31,7 +31,7 @@ public class AuditLog {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user id

87
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Cluster.java
View File

@ -19,18 +19,18 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* Cluster
*/
@Data
@TableName("t_ds_cluster")
public class Cluster {
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* cluster code
@ -57,83 +57,4 @@ public class Cluster {
private Date createTime;
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Long getCode() {
return this.code;
}
public void setCode(Long code) {
this.code = code;
}
public String getConfig() {
return this.config;
}
public void setConfig(String config) {
this.config = config;
}
public String getDescription() {
return this.description;
}
public void setDescription(String description) {
this.description = description;
}
public Integer getOperator() {
return this.operator;
}
public void setOperator(Integer operator) {
this.operator = operator;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "Cluster{"
+ "id= " + id
+ ", code= " + code
+ ", name= " + name
+ ", config= " + config
+ ", description= " + description
+ ", operator= " + operator
+ ", createTime= " + createTime
+ ", updateTime= " + updateTime
+ "}";
}
}

212
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Command.java
View File

@ -25,14 +25,14 @@ import org.apache.dolphinscheduler.common.enums.WarningType;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* command
*/
@Data
@TableName("t_ds_command")
public class Command {
@ -40,7 +40,7 @@ public class Command {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* command type
@ -146,22 +146,21 @@ public class Command {
}
public Command(
CommandType commandType,
TaskDependType taskDependType,
FailureStrategy failureStrategy,
int executorId,
long processDefinitionCode,
String commandParam,
WarningType warningType,
int warningGroupId,
Date scheduleTime,
String workerGroup,
Long environmentCode,
Priority processInstancePriority,
int dryRun,
int processInstanceId,
int processDefinitionVersion
) {
CommandType commandType,
TaskDependType taskDependType,
FailureStrategy failureStrategy,
int executorId,
long processDefinitionCode,
String commandParam,
WarningType warningType,
int warningGroupId,
Date scheduleTime,
String workerGroup,
Long environmentCode,
Priority processInstancePriority,
int dryRun,
int processInstanceId,
int processDefinitionVersion) {
this.commandType = commandType;
this.executorId = executorId;
this.processDefinitionCode = processDefinitionCode;
@ -181,150 +180,6 @@ public class Command {
this.processDefinitionVersion = processDefinitionVersion;
}
public TaskDependType getTaskDependType() {
return taskDependType;
}
public void setTaskDependType(TaskDependType taskDependType) {
this.taskDependType = taskDependType;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public CommandType getCommandType() {
return commandType;
}
public void setCommandType(CommandType commandType) {
this.commandType = commandType;
}
public long getProcessDefinitionCode() {
return processDefinitionCode;
}
public void setProcessDefinitionCode(long processDefinitionCode) {
this.processDefinitionCode = processDefinitionCode;
}
public FailureStrategy getFailureStrategy() {
return failureStrategy;
}
public void setFailureStrategy(FailureStrategy failureStrategy) {
this.failureStrategy = failureStrategy;
}
public void setCommandParam(String commandParam) {
this.commandParam = commandParam;
}
public String getCommandParam() {
return commandParam;
}
public WarningType getWarningType() {
return warningType;
}
public void setWarningType(WarningType warningType) {
this.warningType = warningType;
}
public Integer getWarningGroupId() {
return warningGroupId;
}
public void setWarningGroupId(Integer warningGroupId) {
this.warningGroupId = warningGroupId;
}
public Date getScheduleTime() {
return scheduleTime;
}
public void setScheduleTime(Date scheduleTime) {
this.scheduleTime = scheduleTime;
}
public Date getStartTime() {
return startTime;
}
public void setStartTime(Date startTime) {
this.startTime = startTime;
}
public int getExecutorId() {
return executorId;
}
public void setExecutorId(int executorId) {
this.executorId = executorId;
}
public Priority getProcessInstancePriority() {
return processInstancePriority;
}
public void setProcessInstancePriority(Priority processInstancePriority) {
this.processInstancePriority = processInstancePriority;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getWorkerGroup() {
return workerGroup;
}
public void setWorkerGroup(String workerGroup) {
this.workerGroup = workerGroup;
}
public Long getEnvironmentCode() {
return this.environmentCode;
}
public void setEnvironmentCode(Long environmentCode) {
this.environmentCode = environmentCode;
}
public int getDryRun() {
return dryRun;
}
public void setDryRun(int dryRun) {
this.dryRun = dryRun;
}
public int getProcessInstanceId() {
return processInstanceId;
}
public void setProcessInstanceId(int processInstanceId) {
this.processInstanceId = processInstanceId;
}
public int getProcessDefinitionVersion() {
return processDefinitionVersion;
}
public void setProcessDefinitionVersion(int processDefinitionVersion) {
this.processDefinitionVersion = processDefinitionVersion;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -349,7 +204,8 @@ public class Command {
return false;
}
if (environmentCode != null ? environmentCode.equals(command.environmentCode) : command.environmentCode == null) {
if (environmentCode != null ? environmentCode.equals(command.environmentCode)
: command.environmentCode == null) {
return false;
}
@ -411,30 +267,4 @@ public class Command {
result = 31 * result + processDefinitionVersion;
return result;
}
@Override
public String toString() {
return "Command{"
+ "id=" + id
+ ", commandType=" + commandType
+ ", processDefinitionCode=" + processDefinitionCode
+ ", executorId=" + executorId
+ ", commandParam='" + commandParam + '\''
+ ", taskDependType=" + taskDependType
+ ", failureStrategy=" + failureStrategy
+ ", warningType=" + warningType
+ ", warningGroupId=" + warningGroupId
+ ", scheduleTime=" + scheduleTime
+ ", startTime=" + startTime
+ ", processInstancePriority=" + processInstancePriority
+ ", updateTime=" + updateTime
+ ", workerGroup='" + workerGroup + '\''
+ ", environmentCode='" + environmentCode + '\''
+ ", dryRun='" + dryRun + '\''
+ ", processInstanceId='" + processInstanceId + '\''
+ ", processDefinitionVersion='" + processDefinitionVersion + '\''
+ '}';
}
}

96
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DataSource.java
View File

@ -21,18 +21,22 @@ import org.apache.dolphinscheduler.spi.enums.DbType;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_datasource")
public class DataSource {
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user id
@ -75,96 +79,6 @@ public class DataSource {
*/
private Date updateTime;
public DataSource() {
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getNote() {
return note;
}
public void setNote(String note) {
this.note = note;
}
public DbType getType() {
return type;
}
public void setType(DbType type) {
this.type = type;
}
public String getConnectionParams() {
return connectionParams;
}
public void setConnectionParams(String connectionParams) {
this.connectionParams = connectionParams;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "DataSource{"
+ "id=" + id
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", name='" + name + '\''
+ ", note='" + note + '\''
+ ", type=" + type
+ ", connectionParams='" + connectionParams + '\''
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
@Override
public boolean equals(Object o) {
if (this == o) {

68
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DatasourceUser.java
View File

@ -19,13 +19,13 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* data source user relation
*/
@Data
@TableName("t_ds_relation_datasource_user")
public class DatasourceUser {
@ -33,7 +33,7 @@ public class DatasourceUser {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user id
@ -58,64 +58,4 @@ public class DatasourceUser {
* update time
*/
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public int getDatasourceId() {
return datasourceId;
}
public void setDatasourceId(int datasourceId) {
this.datasourceId = datasourceId;
}
public int getPerm() {
return perm;
}
public void setPerm(int perm) {
this.perm = perm;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "DatasourceUser{"
+ "id=" + id
+ ", userId=" + userId
+ ", datasourceId=" + datasourceId
+ ", perm=" + perm
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

84
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DqComparisonType.java
View File

@ -20,18 +20,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_dq_comparison_type")
public class DqComparisonType implements Serializable {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* type
*/
@ -67,82 +71,4 @@ public class DqComparisonType implements Serializable {
*/
@TableField(value = "update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getExecuteSql() {
return executeSql;
}
public void setExecuteSql(String executeSql) {
this.executeSql = executeSql;
}
public String getOutputTable() {
return outputTable;
}
public void setOutputTable(String outputTable) {
this.outputTable = outputTable;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Boolean getInnerSource() {
return isInnerSource;
}
public void setInnerSource(Boolean innerSource) {
isInnerSource = innerSource;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "DqComparisonType{"
+ "id=" + id
+ ", type='" + type + '\''
+ ", executeSql='" + executeSql + '\''
+ ", outputTable='" + outputTable + '\''
+ ", name='" + name + '\''
+ ", isInnerSource='" + isInnerSource + '\''
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

239
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DqExecuteResult.java
View File

@ -20,18 +20,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_dq_execute_result")
public class DqExecuteResult implements Serializable {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* process defined id
*/
@ -41,7 +45,7 @@ public class DqExecuteResult implements Serializable {
* process definition name
*/
@TableField(exist = false)
private String processDefinitionName;
private String processDefinitionName;
/**
* process definition code
*/
@ -152,235 +156,4 @@ public class DqExecuteResult implements Serializable {
*/
@TableField(value = "update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public long getProcessDefinitionId() {
return processDefinitionId;
}
public void setProcessDefinitionId(long processDefinitionId) {
this.processDefinitionId = processDefinitionId;
}
public long getTaskInstanceId() {
return taskInstanceId;
}
public void setTaskInstanceId(long taskInstanceId) {
this.taskInstanceId = taskInstanceId;
}
public long getProcessInstanceId() {
return processInstanceId;
}
public void setProcessInstanceId(long processInstanceId) {
this.processInstanceId = processInstanceId;
}
public String getProcessInstanceName() {
return processInstanceName;
}
public void setProcessInstanceName(String processInstanceName) {
this.processInstanceName = processInstanceName;
}
public long getProjectCode() {
return projectCode;
}
public void setProjectCode(long projectCode) {
this.projectCode = projectCode;
}
public String getRuleName() {
return ruleName;
}
public void setRuleName(String ruleName) {
this.ruleName = ruleName;
}
public double getStatisticsValue() {
return statisticsValue;
}
public void setStatisticsValue(double statisticsValue) {
this.statisticsValue = statisticsValue;
}
public double getComparisonValue() {
return comparisonValue;
}
public void setComparisonValue(double comparisonValue) {
this.comparisonValue = comparisonValue;
}
public double getThreshold() {
return threshold;
}
public void setThreshold(double threshold) {
this.threshold = threshold;
}
public int getOperator() {
return operator;
}
public void setOperator(int operator) {
this.operator = operator;
}
public int getFailureStrategy() {
return failureStrategy;
}
public void setFailureStrategy(int failureStrategy) {
this.failureStrategy = failureStrategy;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public int getRuleType() {
return ruleType;
}
public void setRuleType(int ruleType) {
this.ruleType = ruleType;
}
public int getCheckType() {
return checkType;
}
public void setCheckType(int checkType) {
this.checkType = checkType;
}
public int getState() {
return state;
}
public void setState(int state) {
this.state = state;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getProcessDefinitionName() {
return processDefinitionName;
}
public void setProcessDefinitionName(String processDefinitionName) {
this.processDefinitionName = processDefinitionName;
}
public long getProcessDefinitionCode() {
return processDefinitionCode;
}
public void setProcessDefinitionCode(long processDefinitionCode) {
this.processDefinitionCode = processDefinitionCode;
}
public String getTaskName() {
return taskName;
}
public void setTaskName(String taskName) {
this.taskName = taskName;
}
public int getComparisonType() {
return comparisonType;
}
public void setComparisonType(int comparisonType) {
this.comparisonType = comparisonType;
}
public String getComparisonTypeName() {
return comparisonTypeName;
}
public void setComparisonTypeName(String comparisonTypeName) {
this.comparisonTypeName = comparisonTypeName;
}
public String getErrorOutputPath() {
return errorOutputPath;
}
public void setErrorOutputPath(String errorOutputPath) {
this.errorOutputPath = errorOutputPath;
}
@Override
public String toString() {
return "DqExecuteResult{"
+ "id=" + id
+ ", processDefinitionId=" + processDefinitionId
+ ", processDefinitionName='" + processDefinitionName + '\''
+ ", processDefinitionCode='" + processDefinitionCode + '\''
+ ", processInstanceId=" + processInstanceId
+ ", processInstanceName='" + processInstanceName + '\''
+ ", projectCode='" + projectCode + '\''
+ ", taskInstanceId=" + taskInstanceId
+ ", taskName='" + taskName + '\''
+ ", ruleType=" + ruleType
+ ", ruleName='" + ruleName + '\''
+ ", statisticsValue=" + statisticsValue
+ ", comparisonValue=" + comparisonValue
+ ", comparisonType=" + comparisonType
+ ", comparisonTypeName=" + comparisonTypeName
+ ", checkType=" + checkType
+ ", threshold=" + threshold
+ ", operator=" + operator
+ ", failureStrategy=" + failureStrategy
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", state=" + state
+ ", errorOutputPath=" + errorOutputPath
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

83
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DqRule.java
View File

@ -20,18 +20,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_dq_rule")
public class DqRule implements Serializable {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* name
*/
@ -67,81 +71,4 @@ public class DqRule implements Serializable {
*/
@TableField(value = "update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getType() {
return type;
}
public void setType(int type) {
this.type = type;
}
public String getRuleJson() {
return ruleJson;
}
public void setRuleJson(String ruleJson) {
this.ruleJson = ruleJson;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
@Override
public String toString() {
return "DqRule{"
+ "id=" + id
+ ", name='" + name + '\''
+ ", type=" + type
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

89
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DqRuleExecuteSql.java
View File

@ -22,21 +22,22 @@ import org.apache.dolphinscheduler.plugin.task.api.enums.dp.ExecuteSqlType;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* RuleExecuteSql
*/
@Data
@TableName("t_ds_dq_rule_execute_sql")
public class DqRuleExecuteSql implements Serializable {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* index，ensure the execution order of sql
*/
@ -72,82 +73,4 @@ public class DqRuleExecuteSql implements Serializable {
*/
@TableField(value = "update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getIndex() {
return index;
}
public void setIndex(int index) {
this.index = index;
}
public String getSql() {
return sql;
}
public void setSql(String sql) {
this.sql = sql;
}
public String getTableAlias() {
return tableAlias;
}
public void setTableAlias(String tableAlias) {
this.tableAlias = tableAlias;
}
public int getType() {
return type;
}
public void setType(int type) {
this.type = type;
}
public boolean isErrorOutputSql() {
return isErrorOutputSql;
}
public void setErrorOutputSql(boolean errorOutputSql) {
isErrorOutputSql = errorOutputSql;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "DqRuleExecuteSql{"
+ "id=" + id
+ ", index=" + index
+ ", sql='" + sql + '\''
+ ", tableAlias='" + tableAlias + '\''
+ ", type=" + type
+ ", isErrorOutputSql=" + isErrorOutputSql
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}
}

179
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DqRuleInputEntry.java
View File

@ -24,21 +24,22 @@ import org.apache.dolphinscheduler.plugin.task.api.enums.dp.ValueType;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* RuleInputEntry
*/
@Data
@TableName("t_ds_dq_rule_input_entry")
public class DqRuleInputEntry implements Serializable {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* form field name
*/
@ -126,172 +127,4 @@ public class DqRuleInputEntry implements Serializable {
*/
@TableField(value = "update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getField() {
return field;
}
public void setField(String field) {
this.field = field;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
public String getOptions() {
return options;
}
public void setOptions(String options) {
this.options = options;
}
public String getPlaceholder() {
return placeholder;
}
public void setPlaceholder(String placeholder) {
this.placeholder = placeholder;
}
public int getOptionSourceType() {
return optionSourceType;
}
public void setOptionSourceType(int optionSourceType) {
this.optionSourceType = optionSourceType;
}
public int getValueType() {
return valueType;
}
public void setValueType(int valueType) {
this.valueType = valueType;
}
public int getInputType() {
return inputType;
}
public void setInputType(int inputType) {
this.inputType = inputType;
}
public Boolean getShow() {
return isShow;
}
public void setShow(Boolean show) {
isShow = show;
}
public Boolean getCanEdit() {
return canEdit;
}
public void setCanEdit(Boolean canEdit) {
this.canEdit = canEdit;
}
public Boolean getEmit() {
return isEmit;
}
public void setEmit(Boolean emit) {
isEmit = emit;
}
public Boolean getValidate() {
return isValidate;
}
public void setValidate(Boolean validate) {
isValidate = validate;
}
public String getValuesMap() {
return valuesMap;
}
public void setValuesMap(String valuesMap) {
this.valuesMap = valuesMap;
}
public Integer getIndex() {
return index;
}
public void setIndex(Integer index) {
this.index = index;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "DqRuleInputEntry{"
+ "id=" + id
+ ", field='" + field + '\''
+ ", type=" + type
+ ", title='" + title + '\''
+ ", value='" + value + '\''
+ ", options='" + options + '\''
+ ", placeholder='" + placeholder + '\''
+ ", optionSourceType=" + optionSourceType
+ ", valueType=" + valueType
+ ", inputType=" + inputType
+ ", isShow=" + isShow
+ ", canEdit=" + canEdit
+ ", isEmit=" + isEmit
+ ", isValidate=" + isValidate
+ ", valuesMap='" + valuesMap + '\''
+ ", index=" + index
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}
}

131
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/DqTaskStatisticsValue.java
View File

@ -20,18 +20,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_dq_task_statistics_value")
public class DqTaskStatisticsValue implements Serializable {
/**
* primary key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* process defined id
*/
@ -41,7 +45,7 @@ public class DqTaskStatisticsValue implements Serializable {
* process definition name
*/
@TableField(exist = false)
private String processDefinitionName;
private String processDefinitionName;
/**
* task instance id
*/
@ -92,127 +96,4 @@ public class DqTaskStatisticsValue implements Serializable {
*/
@TableField(value = "update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public long getProcessDefinitionId() {
return processDefinitionId;
}
public void setProcessDefinitionId(long processDefinitionId) {
this.processDefinitionId = processDefinitionId;
}
public String getProcessDefinitionName() {
return processDefinitionName;
}
public void setProcessDefinitionName(String processDefinitionName) {
this.processDefinitionName = processDefinitionName;
}
public long getTaskInstanceId() {
return taskInstanceId;
}
public void setTaskInstanceId(long taskInstanceId) {
this.taskInstanceId = taskInstanceId;
}
public String getTaskName() {
return taskName;
}
public void setTaskName(String taskName) {
this.taskName = taskName;
}
public long getRuleId() {
return ruleId;
}
public void setRuleId(long ruleId) {
this.ruleId = ruleId;
}
public int getRuleType() {
return ruleType;
}
public void setRuleType(int ruleType) {
this.ruleType = ruleType;
}
public String getRuleName() {
return ruleName;
}
public void setRuleName(String ruleName) {
this.ruleName = ruleName;
}
public double getStatisticsValue() {
return statisticsValue;
}
public void setStatisticsValue(double statisticsValue) {
this.statisticsValue = statisticsValue;
}
public String getStatisticsName() {
return statisticsName;
}
public void setStatisticsName(String statisticsName) {
this.statisticsName = statisticsName;
}
public Date getDataTime() {
return dataTime;
}
public void setDataTime(Date dataTime) {
this.dataTime = dataTime;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "DqTaskStatisticsValue{"
+ "id=" + id
+ ", processDefinitionId=" + processDefinitionId
+ ", processDefinitionName='" + processDefinitionName + '\''
+ ", taskInstanceId=" + taskInstanceId
+ ", taskName='" + taskName + '\''
+ ", ruleId=" + ruleId
+ ", ruleType=" + ruleType
+ ", ruleName='" + ruleName + '\''
+ ", statisticsValue=" + statisticsValue
+ ", statisticsName='" + statisticsName + '\''
+ ", dataTime=" + dataTime
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

87
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Environment.java
View File

@ -19,18 +19,18 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* Environment
*/
@Data
@TableName("t_ds_environment")
public class Environment {
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* environment code
@ -57,83 +57,4 @@ public class Environment {
private Date createTime;
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Long getCode() {
return this.code;
}
public void setCode(Long code) {
this.code = code;
}
public String getConfig() {
return this.config;
}
public void setConfig(String config) {
this.config = config;
}
public String getDescription() {
return this.description;
}
public void setDescription(String description) {
this.description = description;
}
public Integer getOperator() {
return this.operator;
}
public void setOperator(Integer operator) {
this.operator = operator;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "Environment{"
+ "id= " + id
+ ", code= " + code
+ ", name= " + name
+ ", config= " + config
+ ", description= " + description
+ ", operator= " + operator
+ ", createTime= " + createTime
+ ", updateTime= " + updateTime
+ "}";
}
}

69
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java
View File

@ -19,18 +19,18 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* EnvironmentWorkerGroupRelation
*/
@Data
@TableName("t_ds_environment_worker_group_relation")
public class EnvironmentWorkerGroupRelation {
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* environment code
@ -50,65 +50,4 @@ public class EnvironmentWorkerGroupRelation {
private Date createTime;
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getWorkerGroup() {
return workerGroup;
}
public void setWorkerGroup(String workerGroup) {
this.workerGroup = workerGroup;
}
public Long getEnvironmentCode() {
return this.environmentCode;
}
public void setEnvironmentCode(Long environmentCode) {
this.environmentCode = environmentCode;
}
public Integer getOperator() {
return this.operator;
}
public void setOperator(Integer operator) {
this.operator = operator;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "EnvironmentWorkerGroupRelation{"
+ "id= " + id
+ ", environmentCode= " + environmentCode
+ ", workerGroup= " + workerGroup
+ ", operator= " + operator
+ ", createTime= " + createTime
+ ", updateTime= " + updateTime
+ "}";
}
}

170
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ErrorCommand.java
View File

@ -25,13 +25,13 @@ import org.apache.dolphinscheduler.common.enums.WarningType;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* command
*/
@Data
@TableName("t_ds_error_command")
public class ErrorCommand {
@ -39,7 +39,7 @@ public class ErrorCommand {
* id
*/
@TableId(value = "id", type = IdType.INPUT)
private int id;
private Integer id;
/**
* command type
@ -121,7 +121,8 @@ public class ErrorCommand {
*/
private int dryRun;
public ErrorCommand() {}
public ErrorCommand() {
}
public ErrorCommand(Command command, String message) {
this.id = command.getId();
@ -141,163 +142,4 @@ public class ErrorCommand {
this.message = message;
this.dryRun = command.getDryRun();
}
public TaskDependType getTaskDependType() {
return taskDependType;
}
public void setTaskDependType(TaskDependType taskDependType) {
this.taskDependType = taskDependType;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public CommandType getCommandType() {
return commandType;
}
public void setCommandType(CommandType commandType) {
this.commandType = commandType;
}
public long getProcessDefinitionCode() {
return processDefinitionCode;
}
public void setProcessDefinitionCode(long processDefinitionCode) {
this.processDefinitionCode = processDefinitionCode;
}
public FailureStrategy getFailureStrategy() {
return failureStrategy;
}
public void setFailureStrategy(FailureStrategy failureStrategy) {
this.failureStrategy = failureStrategy;
}
public void setCommandParam(String commandParam) {
this.commandParam = commandParam;
}
public String getCommandParam() {
return commandParam;
}
public WarningType getWarningType() {
return warningType;
}
public void setWarningType(WarningType warningType) {
this.warningType = warningType;
}
public Integer getWarningGroupId() {
return warningGroupId;
}
public void setWarningGroupId(Integer warningGroupId) {
this.warningGroupId = warningGroupId;
}
public Date getScheduleTime() {
return scheduleTime;
}
public void setScheduleTime(Date scheduleTime) {
this.scheduleTime = scheduleTime;
}
public Date getStartTime() {
return startTime;
}
public void setStartTime(Date startTime) {
this.startTime = startTime;
}
public int getExecutorId() {
return executorId;
}
public void setExecutorId(int executorId) {
this.executorId = executorId;
}
public Priority getProcessInstancePriority() {
return processInstancePriority;
}
public void setProcessInstancePriority(Priority processInstancePriority) {
this.processInstancePriority = processInstancePriority;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getWorkerGroup() {
return workerGroup;
}
public void setWorkerGroup(String workerGroup) {
this.workerGroup = workerGroup;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
public Long getEnvironmentCode() {
return this.environmentCode;
}
public void setEnvironmentCode(Long environmentCode) {
this.environmentCode = environmentCode;
}
public int getDryRun() {
return dryRun;
}
public void setDryRun(int dryRun) {
this.dryRun = dryRun;
}
@Override
public String toString() {
return "ErrorCommand{"
+ "id=" + id
+ ", commandType=" + commandType
+ ", processDefinitionCode=" + processDefinitionCode
+ ", executorId=" + executorId
+ ", commandParam='" + commandParam + '\''
+ ", taskDependType=" + taskDependType
+ ", failureStrategy=" + failureStrategy
+ ", warningType=" + warningType
+ ", warningGroupId=" + warningGroupId
+ ", scheduleTime=" + scheduleTime
+ ", startTime=" + startTime
+ ", processInstancePriority=" + processInstancePriority
+ ", updateTime=" + updateTime
+ ", message='" + message + '\''
+ ", workerGroup='" + workerGroup + '\''
+ ", environmentCode='" + environmentCode + '\''
+ ", dryRun='" + dryRun + '\''
+ '}';
}
}

53
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/K8s.java
View File

@ -19,21 +19,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* multi-data centre k8s temporary structure, waiting for new feature to complete will switch
*/
@Data
@TableName("t_ds_k8s")
public class K8s {
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* k8s name
*/
@ -55,48 +56,4 @@ public class K8s {
*/
@TableField("update_time")
private Date updateTime;
public K8s() {
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getK8sName() {
return k8sName;
}
public void setK8sName(String k8sName) {
this.k8sName = k8sName;
}
public String getK8sConfig() {
return k8sConfig;
}
public void setK8sConfig(String k8sConfig) {
this.k8sConfig = k8sConfig;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
}

138
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/K8sNamespace.java
View File

@ -19,16 +19,17 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* k8s namespace
*/
@Data
@TableName("t_ds_k8s_namespace")
public class K8sNamespace {
@TableId(value = "id", type = IdType.AUTO)
private Integer id;
@ -108,127 +109,6 @@ public class K8sNamespace {
@TableField(exist = false)
private String clusterName;
public Integer getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getNamespace() {
return namespace;
}
public void setNamespace(String namespace) {
this.namespace = namespace;
}
public Double getLimitsCpu() {
return limitsCpu;
}
public void setLimitsCpu(Double limitsCpu) {
this.limitsCpu = limitsCpu;
}
public Integer getLimitsMemory() {
return limitsMemory;
}
public void setLimitsMemory(Integer limitsMemory) {
this.limitsMemory = limitsMemory;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public Integer getPodRequestMemory() {
return podRequestMemory;
}
public void setPodRequestMemory(Integer podRequestMemory) {
this.podRequestMemory = podRequestMemory;
}
public Integer getPodReplicas() {
return podReplicas;
}
public void setPodReplicas(Integer podReplicas) {
this.podReplicas = podReplicas;
}
public Long getClusterCode() {
return clusterCode;
}
public void setClusterCode(Long clusterCode) {
this.clusterCode = clusterCode;
}
public String getClusterName() {
return clusterName;
}
public void setClusterName(String clusterName) {
this.clusterName = clusterName;
}
public Double getPodRequestCpu() {
return podRequestCpu;
}
public void setPodRequestCpu(Double podRequestCpu) {
this.podRequestCpu = podRequestCpu;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
@Override
public String toString() {
return "K8sNamespace{" +
"id=" + id +
"code=" + code +
", namespace=" + namespace +
", limitsCpu=" + limitsCpu +
", limitsMemory=" + limitsMemory +
", podRequestCpu=" + podRequestCpu +
", podRequestMemory=" + podRequestMemory +
", podReplicas=" + podReplicas +
", clusterCode=" + clusterCode +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -253,12 +133,4 @@ public class K8sNamespace {
result = 31 * result + (clusterName + namespace).hashCode();
return result;
}
public Long getCode() {
return code;
}
public void setCode(Long code) {
this.code = code;
}
}
}

95
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/K8sNamespaceUser.java
View File

@ -19,21 +19,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* k8s namespace and user relation
*/
@Data
@TableName("t_ds_relation_namespace_user")
public class K8sNamespaceUser {
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user id
@ -75,90 +76,4 @@ public class K8sNamespaceUser {
@TableField("update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public int getNamespaceId() {
return namespaceId;
}
public void setNamespaceId(int namespaceId) {
this.namespaceId = namespaceId;
}
public String getK8s() {
return k8s;
}
public void setK8s(String k8s) {
this.k8s = k8s;
}
public String getNamespaceName() {
return namespaceName;
}
public void setNamespaceName(String namespaceName) {
this.namespaceName = namespaceName;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public int getPerm() {
return perm;
}
public void setPerm(int perm) {
this.perm = perm;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "K8sNamespaceUser{" +
"id=" + id +
", userId=" + userId +
", namespaceId=" + namespaceId +
", k8s=" + k8s +
", namespaceName=" + namespaceName +
", perm=" + perm +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
}

57
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/PluginDefine.java
View File

@ -19,14 +19,14 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* t_ds_plugin_define
*/
@Data
@TableName("t_ds_plugin_define")
public class PluginDefine {
@ -34,7 +34,7 @@ public class PluginDefine {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* plugin name
@ -73,53 +73,4 @@ public class PluginDefine {
this.createTime = new Date();
this.updateTime = new Date();
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getPluginName() {
return pluginName;
}
public void setPluginName(String pluginName) {
this.pluginName = pluginName;
}
public String getPluginType() {
return pluginType;
}
public void setPluginType(String pluginType) {
this.pluginType = pluginType;
}
public String getPluginParams() {
return pluginParams;
}
public void setPluginParams(String pluginParams) {
this.pluginParams = pluginParams;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
}

77
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessDefinition.java
View File

@ -46,7 +46,7 @@ public class ProcessDefinition {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* code
@ -171,7 +171,8 @@ public class ProcessDefinition {
*/
private ProcessExecutionTypeEnum executionType;
public ProcessDefinition() { }
public ProcessDefinition() {
}
public ProcessDefinition(long projectCode,
String name,
@ -223,11 +224,11 @@ public class ProcessDefinition {
this.version = version;
}
public int getId() {
public Integer getId() {
return id;
}
public void setId(int id) {
public void setId(Integer id) {
this.id = id;
}
@ -418,44 +419,44 @@ public class ProcessDefinition {
}
ProcessDefinition that = (ProcessDefinition) o;
return projectCode == that.projectCode
&& userId == that.userId
&& timeout == that.timeout
&& tenantId == that.tenantId
&& Objects.equals(name, that.name)
&& releaseState == that.releaseState
&& Objects.equals(description, that.description)
&& Objects.equals(globalParams, that.globalParams)
&& flag == that.flag
&& executionType == that.executionType
&& Objects.equals(locations, that.locations);
&& userId == that.userId
&& timeout == that.timeout
&& tenantId == that.tenantId
&& Objects.equals(name, that.name)
&& releaseState == that.releaseState
&& Objects.equals(description, that.description)
&& Objects.equals(globalParams, that.globalParams)
&& flag == that.flag
&& executionType == that.executionType
&& Objects.equals(locations, that.locations);
}
@Override
public String toString() {
return "ProcessDefinition{"
+ "id=" + id
+ ", code=" + code
+ ", name='" + name + '\''
+ ", version=" + version
+ ", releaseState=" + releaseState
+ ", projectCode=" + projectCode
+ ", description='" + description + '\''
+ ", globalParams='" + globalParams + '\''
+ ", globalParamList=" + globalParamList
+ ", globalParamMap=" + globalParamMap
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ ", flag=" + flag
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", projectName='" + projectName + '\''
+ ", locations='" + locations + '\''
+ ", scheduleReleaseState=" + scheduleReleaseState
+ ", timeout=" + timeout
+ ", tenantId=" + tenantId
+ ", tenantCode='" + tenantCode + '\''
+ ", modifyBy='" + modifyBy + '\''
+ ", warningGroupId=" + warningGroupId
+ '}';
+ "id=" + id
+ ", code=" + code
+ ", name='" + name + '\''
+ ", version=" + version
+ ", releaseState=" + releaseState
+ ", projectCode=" + projectCode
+ ", description='" + description + '\''
+ ", globalParams='" + globalParams + '\''
+ ", globalParamList=" + globalParamList
+ ", globalParamMap=" + globalParamMap
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ ", flag=" + flag
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", projectName='" + projectName + '\''
+ ", locations='" + locations + '\''
+ ", scheduleReleaseState=" + scheduleReleaseState
+ ", timeout=" + timeout
+ ", tenantId=" + tenantId
+ ", tenantCode='" + tenantCode + '\''
+ ", modifyBy='" + modifyBy + '\''
+ ", warningGroupId=" + warningGroupId
+ '}';
}
}

12
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstance.java
View File

@ -32,7 +32,10 @@ import org.apache.commons.lang3.StringUtils;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Objects;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
@ -40,10 +43,6 @@ import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.google.common.base.Strings;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* process instance
*/
@ -56,7 +55,7 @@ public class ProcessInstance {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* process definition code
@ -349,6 +348,7 @@ public class ProcessInstance {
@NoArgsConstructor
@AllArgsConstructor
public static class StateDesc {
Date time;
WorkflowExecutionStatus state;
String desc;

50
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessInstanceMap.java
View File

@ -17,13 +17,13 @@
package org.apache.dolphinscheduler.dao.entity;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* process instance map
*/
@Data
@TableName("t_ds_relation_process_instance")
public class ProcessInstanceMap {
@ -31,7 +31,7 @@ public class ProcessInstanceMap {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* parent process instance id
@ -48,48 +48,6 @@ public class ProcessInstanceMap {
*/
private int processInstanceId;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getParentProcessInstanceId() {
return parentProcessInstanceId;
}
public void setParentProcessInstanceId(int parentProcessInstanceId) {
this.parentProcessInstanceId = parentProcessInstanceId;
}
public int getParentTaskInstanceId() {
return parentTaskInstanceId;
}
public void setParentTaskInstanceId(int parentTaskInstanceId) {
this.parentTaskInstanceId = parentTaskInstanceId;
}
public int getProcessInstanceId() {
return processInstanceId;
}
public void setProcessInstanceId(int processInstanceId) {
this.processInstanceId = processInstanceId;
}
@Override
public String toString() {
return "ProcessInstanceMap{"
+ "id=" + id
+ ", parentProcessInstanceId=" + parentProcessInstanceId
+ ", parentTaskInstanceId=" + parentTaskInstanceId
+ ", processInstanceId=" + processInstanceId
+ '}';
}
@Override
public boolean equals(Object o) {
if (this == o) {

148
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProcessTaskRelation.java
View File

@ -23,15 +23,15 @@ import org.apache.dolphinscheduler.common.utils.JSONUtils;
import java.util.Date;
import java.util.Objects;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
/**
* process task relation
*/
@Data
@TableName("t_ds_process_task_relation")
public class ProcessTaskRelation {
@ -39,7 +39,7 @@ public class ProcessTaskRelation {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* name
@ -132,110 +132,6 @@ public class ProcessTaskRelation {
this.updateTime = updateTime;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getConditionParams() {
return conditionParams;
}
public void setConditionParams(String conditionParams) {
this.conditionParams = conditionParams;
}
public int getProcessDefinitionVersion() {
return processDefinitionVersion;
}
public void setProcessDefinitionVersion(int processDefinitionVersion) {
this.processDefinitionVersion = processDefinitionVersion;
}
public long getProjectCode() {
return projectCode;
}
public void setProjectCode(long projectCode) {
this.projectCode = projectCode;
}
public long getProcessDefinitionCode() {
return processDefinitionCode;
}
public void setProcessDefinitionCode(long processDefinitionCode) {
this.processDefinitionCode = processDefinitionCode;
}
public long getPreTaskCode() {
return preTaskCode;
}
public void setPreTaskCode(long preTaskCode) {
this.preTaskCode = preTaskCode;
}
public long getPostTaskCode() {
return postTaskCode;
}
public void setPostTaskCode(long postTaskCode) {
this.postTaskCode = postTaskCode;
}
public ConditionType getConditionType() {
return conditionType;
}
public void setConditionType(ConditionType conditionType) {
this.conditionType = conditionType;
}
public int getPreTaskVersion() {
return preTaskVersion;
}
public void setPreTaskVersion(int preTaskVersion) {
this.preTaskVersion = preTaskVersion;
}
public int getPostTaskVersion() {
return postTaskVersion;
}
public void setPostTaskVersion(int postTaskVersion) {
this.postTaskVersion = postTaskVersion;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -246,36 +142,18 @@ public class ProcessTaskRelation {
}
ProcessTaskRelation that = (ProcessTaskRelation) o;
return processDefinitionVersion == that.processDefinitionVersion
&& projectCode == that.projectCode
&& processDefinitionCode == that.processDefinitionCode
&& preTaskCode == that.preTaskCode
&& preTaskVersion == that.preTaskVersion
&& postTaskCode == that.postTaskCode
&& postTaskVersion == that.postTaskVersion
&& Objects.equals(name, that.name);
&& projectCode == that.projectCode
&& processDefinitionCode == that.processDefinitionCode
&& preTaskCode == that.preTaskCode
&& preTaskVersion == that.preTaskVersion
&& postTaskCode == that.postTaskCode
&& postTaskVersion == that.postTaskVersion
&& Objects.equals(name, that.name);
}
@Override
public int hashCode() {
return Objects.hash(name, processDefinitionVersion, projectCode, processDefinitionCode, preTaskCode, preTaskVersion, postTaskCode, postTaskVersion);
}
@Override
public String toString() {
return "ProcessTaskRelation{"
+ "id=" + id
+ ", name='" + name + '\''
+ ", processDefinitionVersion=" + processDefinitionVersion
+ ", projectCode=" + projectCode
+ ", processDefinitionCode=" + processDefinitionCode
+ ", preTaskCode=" + preTaskCode
+ ", preTaskVersion=" + preTaskVersion
+ ", postTaskCode=" + postTaskCode
+ ", postTaskVersion=" + postTaskVersion
+ ", conditionType=" + conditionType
+ ", conditionParams='" + conditionParams + '\''
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
return Objects.hash(name, processDefinitionVersion, projectCode, processDefinitionCode, preTaskCode,
preTaskVersion, postTaskCode, postTaskVersion);
}
}

213
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Project.java
View File

@ -19,14 +19,20 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* project
*/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@TableName("t_ds_project")
public class Project {
@ -34,13 +40,13 @@ public class Project {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user id
*/
@TableField("user_id")
private int userId;
private Integer userId;
/**
* user name
@ -91,111 +97,6 @@ public class Project {
@TableField(exist = false)
private int instRunningCount;
public long getCode() {
return code;
}
public void setCode(long code) {
this.code = code;
}
public int getDefCount() {
return defCount;
}
public void setDefCount(int defCount) {
this.defCount = defCount;
}
public int getInstRunningCount() {
return instRunningCount;
}
public void setInstRunningCount(int instRunningCount) {
this.instRunningCount = instRunningCount;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public void setDescription(String description) {
this.description = description;
}
public String getDescription() {
return description;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public int getPerm() {
return perm;
}
public void setPerm(int perm) {
this.perm = perm;
}
@Override
public String toString() {
return "Project{"
+ "id=" + id
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", code=" + code
+ ", name='" + name + '\''
+ ", description='" + description + '\''
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ ", perm=" + perm
+ ", defCount=" + defCount
+ ", instRunningCount=" + instRunningCount
+ '}';
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -220,96 +121,4 @@ public class Project {
result = 31 * result + name.hashCode();
return result;
}
public static Builder newBuilder() {
return new Builder();
}
public static final class Builder {
private int id;
private int userId;
private String userName;
private long code;
private String name;
private String description;
private Date createTime;
private Date updateTime;
private int perm;
private int defCount;
private int instRunningCount;
private Builder() {
}
public Builder code(long code) {
this.code = code;
return this;
}
public Builder id(int id) {
this.id = id;
return this;
}
public Builder userId(int userId) {
this.userId = userId;
return this;
}
public Builder userName(String userName) {
this.userName = userName;
return this;
}
public Builder name(String name) {
this.name = name;
return this;
}
public Builder description(String description) {
this.description = description;
return this;
}
public Builder createTime(Date createTime) {
this.createTime = createTime;
return this;
}
public Builder updateTime(Date updateTime) {
this.updateTime = updateTime;
return this;
}
public Builder perm(int perm) {
this.perm = perm;
return this;
}
public Builder defCount(int defCount) {
this.defCount = defCount;
return this;
}
public Builder instRunningCount(int instRunningCount) {
this.instRunningCount = instRunningCount;
return this;
}
public Project build() {
Project project = new Project();
project.setId(id);
project.setUserId(userId);
project.setCode(code);
project.setUserName(userName);
project.setName(name);
project.setDescription(description);
project.setCreateTime(createTime);
project.setUpdateTime(updateTime);
project.setPerm(perm);
project.setDefCount(defCount);
project.setInstRunningCount(instRunningCount);
return project;
}
}
}

93
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProjectUser.java
View File

@ -19,18 +19,22 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@TableName("t_ds_relation_project_user")
public class ProjectUser {
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
@TableField("user_id")
private int userId;
@ -66,91 +70,4 @@ public class ProjectUser {
@TableField("update_time")
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public int getProjectId() {
return projectId;
}
public void setProjectId(int projectId) {
this.projectId = projectId;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getProjectName() {
return projectName;
}
public void setProjectName(String projectName) {
this.projectName = projectName;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public int getPerm() {
return perm;
}
public void setPerm(int perm) {
this.perm = perm;
}
public long getProjectCode() {
return projectCode;
}
public void setProjectCode(long projectCode) {
this.projectCode = projectCode;
}
@Override
public String toString() {
return "ProjectUser{"
+ "id=" + id
+ ", userId=" + userId
+ ", projectId=" + projectId
+ ", projectCode=" + projectCode
+ ", projectName='" + projectName + '\''
+ ", userName='" + userName + '\''
+ ", perm=" + perm
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
}

65
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Queue.java
View File

@ -16,23 +16,23 @@
*/
package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.util.Date;
/**
* queue
*/
@Data
@TableName("t_ds_queue")
public class Queue {
/**
* id
*/
@TableId(value="id", type=IdType.AUTO)
private int id;
@TableId(value = "id", type = IdType.AUTO)
private Integer id;
/**
* queue name
*/
@ -71,57 +71,6 @@ public class Queue {
this.updateTime = now;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getQueueName() {
return queueName;
}
public void setQueueName(String queueName) {
this.queueName = queueName;
}
public String getQueue() {
return queue;
}
public void setQueue(String queue) {
this.queue = queue;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "Queue{" +
"id=" + id +
", queueName='" + queueName + '\'' +
", queue='" + queue + '\'' +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
@Override
public boolean equals(Object o) {
if (this == o) {

141
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Resource.java
View File

@ -17,22 +17,28 @@
package org.apache.dolphinscheduler.dao.entity;
import com.baomidou.mybatisplus.annotation.TableField;
import org.apache.dolphinscheduler.spi.enums.ResourceType;
import java.util.Date;
import lombok.Data;
import lombok.NoArgsConstructor;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
@Data
@NoArgsConstructor
@TableName("t_ds_resources")
public class Resource {
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* parent id
@ -95,10 +101,6 @@ public class Resource {
@TableField(exist = false)
private String userName;
public Resource() {
}
public Resource(int id, String alias, String fileName, String description, int userId,
ResourceType type, long size,
Date createTime, Date updateTime) {
@ -121,7 +123,8 @@ public class Resource {
this.isDirectory = isDirectory;
}
public Resource(int pid, String alias, String fullName, boolean isDirectory, String description, String fileName, int userId, ResourceType type, long size, Date createTime, Date updateTime) {
public Resource(int pid, String alias, String fullName, boolean isDirectory, String description, String fileName,
int userId, ResourceType type, long size, Date createTime, Date updateTime) {
this.pid = pid;
this.alias = alias;
this.fullName = fullName;
@ -135,130 +138,6 @@ public class Resource {
this.updateTime = updateTime;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getAlias() {
return alias;
}
public void setAlias(String alias) {
this.alias = alias;
}
public int getPid() {
return pid;
}
public void setPid(int pid) {
this.pid = pid;
}
public String getFullName() {
return fullName;
}
public void setFullName(String fullName) {
this.fullName = fullName;
}
public boolean isDirectory() {
return isDirectory;
}
public void setDirectory(boolean directory) {
isDirectory = directory;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public ResourceType getType() {
return type;
}
public void setType(ResourceType type) {
this.type = type;
}
public long getSize() {
return size;
}
public void setSize(long size) {
this.size = size;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
@Override
public String toString() {
return "Resource{" +
"id=" + id +
", pid=" + pid +
", alias='" + alias + '\'' +
", fullName='" + fullName + '\'' +
", isDirectory=" + isDirectory +
", description='" + description + '\'' +
", fileName='" + fileName + '\'' +
", userId=" + userId +
", type=" + type +
", size=" + size +
", createTime=" + createTime +
", updateTime=" + updateTime +
",userName=" + userName +
'}';
}
@Override
public boolean equals(Object o) {
if (this == o) {

120
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ResourcesUser.java
View File

@ -16,106 +16,46 @@
*/
package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.util.Date;
/**
* resource user relation
*/
@Data
@TableName("t_ds_relation_resources_user")
public class ResourcesUser {
/**
* id
*/
@TableId(value="id", type=IdType.AUTO)
private int id;
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private Integer id;
/**
* user id
*/
private int userId;
/**
* user id
*/
private int userId;
/**
* resource id
*/
private int resourcesId;
/**
* resource id
*/
private int resourcesId;
/**
* permission
*/
private int perm;
/**
* permission
*/
private int perm;
/**
* create time
*/
private Date createTime;
/**
* create time
*/
private Date createTime;
/**
* update time
*/
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public int getResourcesId() {
return resourcesId;
}
public void setResourcesId(int resourcesId) {
this.resourcesId = resourcesId;
}
public int getPerm() {
return perm;
}
public void setPerm(int perm) {
this.perm = perm;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "ResourcesUser{" +
"id=" + id +
", userId=" + userId +
", resourcesId=" + resourcesId +
", perm=" + perm +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
/**
* update time
*/
private Date updateTime;
}

199
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Schedule.java
View File

@ -24,20 +24,19 @@ import org.apache.dolphinscheduler.common.enums.WarningType;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* schedule
*
*/
@Data
@TableName("t_ds_schedules")
public class Schedule {
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* process definition code
@ -124,7 +123,6 @@ public class Schedule {
*/
private int warningGroupId;
/**
* process instance priority
*/
@ -139,193 +137,4 @@ public class Schedule {
* environment code
*/
private Long environmentCode;
public int getWarningGroupId() {
return warningGroupId;
}
public void setWarningGroupId(int warningGroupId) {
this.warningGroupId = warningGroupId;
}
public Schedule() {
}
public String getProjectName() {
return projectName;
}
public void setProjectName(String projectName) {
this.projectName = projectName;
}
public Date getStartTime() {
return startTime;
}
public void setStartTime(Date startTime) {
this.startTime = startTime;
}
public Date getEndTime() {
return endTime;
}
public void setEndTime(Date endTime) {
this.endTime = endTime;
}
public String getTimezoneId() {
return timezoneId;
}
public void setTimezoneId(String timezoneId) {
this.timezoneId = timezoneId;
}
public String getCrontab() {
return crontab;
}
public void setCrontab(String crontab) {
this.crontab = crontab;
}
public FailureStrategy getFailureStrategy() {
return failureStrategy;
}
public void setFailureStrategy(FailureStrategy failureStrategy) {
this.failureStrategy = failureStrategy;
}
public WarningType getWarningType() {
return warningType;
}
public void setWarningType(WarningType warningType) {
this.warningType = warningType;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public ReleaseState getReleaseState() {
return releaseState;
}
public void setReleaseState(ReleaseState releaseState) {
this.releaseState = releaseState;
}
public long getProcessDefinitionCode() {
return processDefinitionCode;
}
public void setProcessDefinitionCode(long processDefinitionCode) {
this.processDefinitionCode = processDefinitionCode;
}
public String getProcessDefinitionName() {
return processDefinitionName;
}
public void setProcessDefinitionName(String processDefinitionName) {
this.processDefinitionName = processDefinitionName;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public Priority getProcessInstancePriority() {
return processInstancePriority;
}
public void setProcessInstancePriority(Priority processInstancePriority) {
this.processInstancePriority = processInstancePriority;
}
public String getWorkerGroup() {
return workerGroup;
}
public void setWorkerGroup(String workerGroup) {
this.workerGroup = workerGroup;
}
public Long getEnvironmentCode() {
return this.environmentCode;
}
public void setEnvironmentCode(Long environmentCode) {
this.environmentCode = environmentCode;
}
@Override
public String toString() {
return "Schedule{"
+ "id=" + id
+ ", processDefinitionCode=" + processDefinitionCode
+ ", processDefinitionName='" + processDefinitionName + '\''
+ ", projectName='" + projectName + '\''
+ ", description='" + definitionDescription + '\''
+ ", startTime=" + startTime
+ ", endTime=" + endTime
+ ", timezoneId='" + timezoneId + +'\''
+ ", crontab='" + crontab + '\''
+ ", failureStrategy=" + failureStrategy
+ ", warningType=" + warningType
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ ", userId=" + userId
+ ", userName='" + userName + '\''
+ ", releaseState=" + releaseState
+ ", warningGroupId=" + warningGroupId
+ ", processInstancePriority=" + processInstancePriority
+ ", workerGroup='" + workerGroup + '\''
+ ", environmentCode='" + environmentCode + '\''
+ '}';
}
public String getDefinitionDescription() {
return definitionDescription;
}
public void setDefinitionDescription(String definitionDescription) {
this.definitionDescription = definitionDescription;
}
}

332
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java
View File

@ -21,18 +21,21 @@ import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.common.enums.Flag;
import org.apache.dolphinscheduler.common.enums.Priority;
import org.apache.dolphinscheduler.common.enums.TaskExecuteType;
import org.apache.dolphinscheduler.plugin.task.api.enums.TaskTimeoutStrategy;
import org.apache.dolphinscheduler.common.enums.TimeoutFlag;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.plugin.task.api.enums.TaskTimeoutStrategy;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import org.apache.commons.collections4.CollectionUtils;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.FieldStrategy;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
@ -43,9 +46,7 @@ import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.google.common.base.Strings;
/**
* task definition
*/
@Data
@TableName("t_ds_task_definition")
public class TaskDefinition {
@ -53,7 +54,7 @@ public class TaskDefinition {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* code
@ -225,86 +226,6 @@ public class TaskDefinition {
this.version = version;
}
public int getTaskGroupId() {
return taskGroupId;
}
public void setTaskGroupId(int taskGroupId) {
this.taskGroupId = taskGroupId;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public Flag getFlag() {
return flag;
}
public void setFlag(Flag flag) {
this.flag = flag;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public String getProjectName() {
return projectName;
}
public void setProjectName(String projectName) {
this.projectName = projectName;
}
public String getTaskParams() {
return taskParams;
}
public void setTaskParams(String taskParams) {
this.taskParams = taskParams;
}
public List<Property> getTaskParamList() {
JsonNode localParams = JSONUtils.parseObject(taskParams).findValue("localParams");
if (localParams != null) {
@ -314,19 +235,11 @@ public class TaskDefinition {
return taskParamList;
}
public void setTaskParamList(List<Property> taskParamList) {
this.taskParamList = taskParamList;
}
public void setTaskParamMap(Map<String, String> taskParamMap) {
this.taskParamMap = taskParamMap;
}
public Map<String, String> getTaskParamMap() {
if (taskParamMap == null && !Strings.isNullOrEmpty(taskParams)) {
JsonNode localParams = JSONUtils.parseObject(taskParams).findValue("localParams");
//If a jsonNode is null, not only use !=null, but also it should use the isNull method to be estimated.
// If a jsonNode is null, not only use !=null, but also it should use the isNull method to be estimated.
if (localParams != null && !localParams.isNull()) {
List<Property> propList = JSONUtils.toList(localParams.toString(), Property.class);
@ -341,162 +254,18 @@ public class TaskDefinition {
return taskParamMap;
}
public int getTimeout() {
return timeout;
}
public void setTimeout(int timeout) {
this.timeout = timeout;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public long getCode() {
return code;
}
public void setCode(long code) {
this.code = code;
}
public int getVersion() {
return version;
}
public void setVersion(int version) {
this.version = version;
}
public long getProjectCode() {
return projectCode;
}
public void setProjectCode(long projectCode) {
this.projectCode = projectCode;
}
public String getTaskType() {
return taskType;
}
public void setTaskType(String taskType) {
this.taskType = taskType;
}
public Priority getTaskPriority() {
return taskPriority;
}
public void setTaskPriority(Priority taskPriority) {
this.taskPriority = taskPriority;
}
public String getWorkerGroup() {
return workerGroup;
}
public void setWorkerGroup(String workerGroup) {
this.workerGroup = workerGroup;
}
public int getFailRetryTimes() {
return failRetryTimes;
}
public void setFailRetryTimes(int failRetryTimes) {
this.failRetryTimes = failRetryTimes;
}
public int getFailRetryInterval() {
return failRetryInterval;
}
public void setFailRetryInterval(int failRetryInterval) {
this.failRetryInterval = failRetryInterval;
}
public TaskTimeoutStrategy getTimeoutNotifyStrategy() {
return timeoutNotifyStrategy;
}
public void setTimeoutNotifyStrategy(TaskTimeoutStrategy timeoutNotifyStrategy) {
this.timeoutNotifyStrategy = timeoutNotifyStrategy;
}
public TimeoutFlag getTimeoutFlag() {
return timeoutFlag;
}
public void setTimeoutFlag(TimeoutFlag timeoutFlag) {
this.timeoutFlag = timeoutFlag;
}
public String getResourceIds() {
return resourceIds;
}
public void setResourceIds(String resourceIds) {
this.resourceIds = resourceIds;
}
public int getDelayTime() {
return delayTime;
}
public void setDelayTime(int delayTime) {
this.delayTime = delayTime;
}
public String getDependence() {
return JSONUtils.getNodeString(this.taskParams, Constants.DEPENDENCE);
}
public String getModifyBy() {
return modifyBy;
}
public void setModifyBy(String modifyBy) {
this.modifyBy = modifyBy;
}
public long getEnvironmentCode() {
return this.environmentCode;
}
public void setEnvironmentCode(long environmentCode) {
this.environmentCode = environmentCode;
}
public Integer getCpuQuota() {
return cpuQuota == null ? -1 : cpuQuota;
}
public void setCpuQuota(Integer cpuQuota) {
this.cpuQuota = cpuQuota;
}
public Integer getMemoryMax() {
return memoryMax == null ? -1 : memoryMax;
}
public void setMemoryMax(Integer memoryMax) {
this.memoryMax = memoryMax;
}
public TaskExecuteType getTaskExecuteType() {
return taskExecuteType;
}
public void setTaskExecuteType(TaskExecuteType taskExecuteType) {
this.taskExecuteType = taskExecuteType;
}
@Override
public boolean equals(Object o) {
if (o == null) {
@ -504,71 +273,26 @@ public class TaskDefinition {
}
TaskDefinition that = (TaskDefinition) o;
return failRetryTimes == that.failRetryTimes
&& failRetryInterval == that.failRetryInterval
&& timeout == that.timeout
&& delayTime == that.delayTime
&& Objects.equals(name, that.name)
&& Objects.equals(description, that.description)
&& Objects.equals(taskType, that.taskType)
&& Objects.equals(taskParams, that.taskParams)
&& flag == that.flag
&& taskPriority == that.taskPriority
&& Objects.equals(workerGroup, that.workerGroup)
&& timeoutFlag == that.timeoutFlag
&& timeoutNotifyStrategy == that.timeoutNotifyStrategy
&& (Objects.equals(resourceIds, that.resourceIds)
|| ("".equals(resourceIds) && that.resourceIds == null)
|| ("".equals(that.resourceIds) && resourceIds == null))
&& environmentCode == that.environmentCode
&& taskGroupId == that.taskGroupId
&& taskGroupPriority == that.taskGroupPriority
&& Objects.equals(cpuQuota, that.cpuQuota)
&& Objects.equals(memoryMax, that.memoryMax)
&& Objects.equals(taskExecuteType, that.taskExecuteType);
}
@Override
public String toString() {
return "TaskDefinition{"
+ "id=" + id
+ ", code=" + code
+ ", name='" + name + '\''
+ ", version=" + version
+ ", description='" + description + '\''
+ ", projectCode=" + projectCode
+ ", userId=" + userId
+ ", taskType=" + taskType
+ ", taskParams='" + taskParams + '\''
+ ", taskParamList=" + taskParamList
+ ", taskParamMap=" + taskParamMap
+ ", flag=" + flag
+ ", taskPriority=" + taskPriority
+ ", userName='" + userName + '\''
+ ", projectName='" + projectName + '\''
+ ", workerGroup='" + workerGroup + '\''
+ ", failRetryTimes=" + failRetryTimes
+ ", environmentCode='" + environmentCode + '\''
+ ", taskGroupId='" + taskGroupId + '\''
+ ", taskGroupPriority='" + taskGroupPriority + '\''
+ ", failRetryInterval=" + failRetryInterval
+ ", timeoutFlag=" + timeoutFlag
+ ", timeoutNotifyStrategy=" + timeoutNotifyStrategy
+ ", timeout=" + timeout
+ ", delayTime=" + delayTime
+ ", resourceIds='" + resourceIds + '\''
+ ", cpuQuota=" + cpuQuota
+ ", memoryMax=" + memoryMax
+ ", taskExecuteType=" + taskExecuteType
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
public int getTaskGroupPriority() {
return taskGroupPriority;
}
public void setTaskGroupPriority(int taskGroupPriority) {
this.taskGroupPriority = taskGroupPriority;
&& failRetryInterval == that.failRetryInterval
&& timeout == that.timeout
&& delayTime == that.delayTime
&& Objects.equals(name, that.name)
&& Objects.equals(description, that.description)
&& Objects.equals(taskType, that.taskType)
&& Objects.equals(taskParams, that.taskParams)
&& flag == that.flag
&& taskPriority == that.taskPriority
&& Objects.equals(workerGroup, that.workerGroup)
&& timeoutFlag == that.timeoutFlag
&& timeoutNotifyStrategy == that.timeoutNotifyStrategy
&& (Objects.equals(resourceIds, that.resourceIds)
|| ("".equals(resourceIds) && that.resourceIds == null)
|| ("".equals(that.resourceIds) && resourceIds == null))
&& environmentCode == that.environmentCode
&& taskGroupId == that.taskGroupId
&& taskGroupPriority == that.taskGroupPriority
&& Objects.equals(cpuQuota, that.cpuQuota)
&& Objects.equals(memoryMax, that.memoryMax)
&& Objects.equals(taskExecuteType, that.taskExecuteType);
}
}

104
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskGroup.java
View File

@ -20,15 +20,16 @@ package org.apache.dolphinscheduler.dao.entity;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* Task Group
*/
@Data
@TableName("t_ds_task_group")
public class TaskGroup implements Serializable {
/**
* key
*/
@ -69,7 +70,7 @@ public class TaskGroup implements Serializable {
*/
private long projectCode;
public TaskGroup(String name,long projectCode, String description, int groupSize, int userId,int status) {
public TaskGroup(String name, long projectCode, String description, int groupSize, int userId, int status) {
this.name = name;
this.projectCode = projectCode;
this.description = description;
@ -88,99 +89,4 @@ public class TaskGroup implements Serializable {
this.status = 1;
this.useSize = 0;
}
@Override
public String toString() {
return "TaskGroup{"
+ "id=" + id
+ ", name='" + name + '\''
+ ", description='" + description + '\''
+ ", groupSize=" + groupSize
+ ", useSize=" + useSize
+ ", userId=" + userId
+ ", status=" + status
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public int getGroupSize() {
return groupSize;
}
public void setGroupSize(int groupSize) {
this.groupSize = groupSize;
}
public int getUseSize() {
return useSize;
}
public void setUseSize(int useSize) {
this.useSize = useSize;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public int getStatus() {
return status;
}
public void setStatus(int status) {
this.status = status;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public long getProjectCode() {
return projectCode;
}
public void setProjectCode(long projectCode) {
this.projectCode = projectCode;
}
}

139
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskGroupQueue.java
View File

@ -22,21 +22,22 @@ import org.apache.dolphinscheduler.common.enums.TaskGroupQueueStatus;
import java.io.Serializable;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
/**
* Task Group Queue
*/
@Data
@TableName("t_ds_task_group_queue")
public class TaskGroupQueue implements Serializable {
/**
* key
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* taskInstanceId
*/
@ -99,7 +100,8 @@ public class TaskGroupQueue implements Serializable {
}
public TaskGroupQueue(int taskId, String taskName, int groupId, int processId, int priority, TaskGroupQueueStatus status) {
public TaskGroupQueue(int taskId, String taskName, int groupId, int processId, int priority,
TaskGroupQueueStatus status) {
this.taskId = taskId;
this.taskName = taskName;
this.groupId = groupId;
@ -107,131 +109,4 @@ public class TaskGroupQueue implements Serializable {
this.priority = priority;
this.status = status;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getTaskId() {
return taskId;
}
public void setTaskId(int taskId) {
this.taskId = taskId;
}
public String getTaskName() {
return taskName;
}
public void setTaskName(String taskName) {
this.taskName = taskName;
}
public int getGroupId() {
return groupId;
}
public void setGroupId(int groupId) {
this.groupId = groupId;
}
public int getProcessId() {
return processId;
}
public void setProcessId(int processId) {
this.processId = processId;
}
public int getPriority() {
return priority;
}
public void setPriority(int priority) {
this.priority = priority;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "TaskGroupQueue{"
+ "id=" + id
+ ", taskId=" + taskId
+ ", taskName='" + taskName + '\''
+ ", groupId=" + groupId
+ ", processId=" + processId
+ ", priority=" + priority
+ ", status=" + status
+ ", createTime=" + createTime
+ ", updateTime=" + updateTime
+ '}';
}
public TaskGroupQueueStatus getStatus() {
return status;
}
public void setStatus(TaskGroupQueueStatus status) {
this.status = status;
}
public int getForceStart() {
return forceStart;
}
public void setForceStart(int forceStart) {
this.forceStart = forceStart;
}
public int getInQueue() {
return inQueue;
}
public void setInQueue(int inQueue) {
this.inQueue = inQueue;
}
public String getProjectName() {
return projectName;
}
public void setProjectName(String projectName) {
this.projectName = projectName;
}
public String getProcessInstanceName() {
return processInstanceName;
}
public void setProcessInstanceName(String processInstanceName) {
this.processInstanceName = processInstanceName;
}
public String getProjectCode() {
return projectCode;
}
public void setProjectCode(String projectCode) {
this.projectCode = projectCode;
}
}

7
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskInstance.java
View File

@ -38,14 +38,14 @@ import java.io.Serializable;
import java.util.Date;
import java.util.Map;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.core.type.TypeReference;
import lombok.Data;
/**
* task instance
*/
@ -57,7 +57,7 @@ public class TaskInstance implements Serializable {
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* task name
@ -371,7 +371,6 @@ public class TaskInstance implements Serializable {
return endTime == null;
}
/**
* determine if a task instance can retry
* if subProcess,

93
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/Tenant.java
View File

@ -16,25 +16,25 @@
*/
package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import java.util.Objects;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.util.Date;
import java.util.Objects;
/**
* tenant
*/
@Data
@TableName("t_ds_tenant")
public class Tenant {
/**
* id
*/
@TableId(value="id", type=IdType.AUTO)
private int id;
@TableId(value = "id", type = IdType.AUTO)
private Integer id;
/**
* tenant code
@ -94,83 +94,6 @@ public class Tenant {
this.updateTime = now;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getTenantCode() {
return tenantCode;
}
public void setTenantCode(String tenantCode) {
this.tenantCode = tenantCode;
}
public int getQueueId() {
return queueId;
}
public void setQueueId(int queueId) {
this.queueId = queueId;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getQueueName() {
return queueName;
}
public void setQueueName(String queueName) {
this.queueName = queueName;
}
public String getQueue() {
return queue;
}
public void setQueue(String queue) {
this.queue = queue;
}
@Override
public String toString() {
return "Tenant{" +
"id=" + id +
", tenantCode='" + tenantCode + '\'' +
", queueId=" + queueId +
", queueName='" + queueName + '\'' +
", queue='" + queue + '\'' +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
@Override
public boolean equals(Object o) {
if (this == o) {

120
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/UDFUser.java
View File

@ -16,106 +16,46 @@
*/
package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.util.Date;
/**
* udf user relation
*/
@Data
@TableName("t_ds_relation_udfs_user")
public class UDFUser {
/**
* id
*/
@TableId(value="id", type=IdType.AUTO)
private int id;
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private Integer id;
/**
* id
*/
private int userId;
/**
* id
*/
private int userId;
/**
* udf id
*/
private int udfId;
/**
* udf id
*/
private int udfId;
/**
* permission
*/
private int perm;
/**
* permission
*/
private int perm;
/**
* create time
*/
private Date createTime;
/**
* create time
*/
private Date createTime;
/**
* update time
*/
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public int getUdfId() {
return udfId;
}
public void setUdfId(int udfId) {
this.udfId = udfId;
}
public int getPerm() {
return perm;
}
public void setPerm(int perm) {
this.perm = perm;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
@Override
public String toString() {
return "UDFUser{" +
"id=" + id +
", userId=" + userId +
", udfId=" + udfId +
", perm=" + perm +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
/**
* update time
*/
private Date updateTime;
}

127
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/UdfFunc.java
View File

@ -17,6 +17,14 @@
package org.apache.dolphinscheduler.dao.entity;
import org.apache.dolphinscheduler.common.enums.UdfType;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import java.io.IOException;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
@ -24,22 +32,16 @@ import com.baomidou.mybatisplus.annotation.TableName;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.KeyDeserializer;
import com.google.common.base.Strings;
import org.apache.dolphinscheduler.common.enums.UdfType;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import java.io.IOException;
import java.util.Date;
/**
* udf function
*/
@Data
@TableName("t_ds_udfs")
public class UdfFunc {
/**
* id
*/
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
/**
* user id
*/
@ -111,111 +113,6 @@ public class UdfFunc {
@TableField(exist = false)
private String userName;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getUserId() {
return userId;
}
public void setUserId(int userId) {
this.userId = userId;
}
public String getFuncName() {
return funcName;
}
public void setFuncName(String funcName) {
this.funcName = funcName;
}
public String getClassName() {
return className;
}
public void setClassName(String className) {
this.className = className;
}
public String getArgTypes() {
return argTypes;
}
public void setArgTypes(String argTypes) {
this.argTypes = argTypes;
}
public String getDatabase() {
return database;
}
public void setDatabase(String database) {
this.database = database;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public int getResourceId() {
return resourceId;
}
public void setResourceId(int resourceId) {
this.resourceId = resourceId;
}
public String getResourceName() {
return resourceName;
}
public void setResourceName(String resourceName) {
this.resourceName = resourceName;
}
public UdfType getType() {
return type;
}
public void setType(UdfType type) {
this.type = type;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -246,7 +143,7 @@ public class UdfFunc {
return JSONUtils.toJsonString(this);
}
public static class UdfFuncDeserializer extends KeyDeserializer {
public static class UdfFuncDeserializer extends KeyDeserializer {
@Override
public Object deserializeKey(String key, DeserializationContext ctxt) throws IOException {

161
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/User.java
View File

@ -18,24 +18,25 @@
package org.apache.dolphinscheduler.dao.entity;
import org.apache.dolphinscheduler.common.enums.UserType;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import java.util.Date;
/**
* user
*/
@Data
@TableName("t_ds_user")
public class User {
public class User {
/**
* id
*/
@TableId(value="id", type=IdType.AUTO)
private int id;
@TableId(value = "id", type = IdType.AUTO)
private Integer id;
/**
* user name
@ -110,127 +111,6 @@ public class User {
*/
private Date updateTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public String getUserPassword() {
return userPassword;
}
public void setUserPassword(String userPassword) {
this.userPassword = userPassword;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public UserType getUserType() {
return userType;
}
public void setUserType(UserType userType) {
this.userType = userType;
}
public int getTenantId() {
return tenantId;
}
public void setTenantId(int tenantId) {
this.tenantId = tenantId;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public Date getUpdateTime() {
return updateTime;
}
public void setUpdateTime(Date updateTime) {
this.updateTime = updateTime;
}
public String getPhone() {
return phone;
}
public void setPhone(String phone) {
this.phone = phone;
}
public String getQueueName() {
return queueName;
}
public void setQueueName(String queueName) {
this.queueName = queueName;
}
public String getAlertGroup() {
return alertGroup;
}
public void setAlertGroup(String alertGroup) {
this.alertGroup = alertGroup;
}
public String getTenantCode() {
return tenantCode;
}
public void setTenantCode(String tenantCode) {
this.tenantCode = tenantCode;
}
public String getQueue() {
return queue;
}
public void setQueue(String queue) {
this.queue = queue;
}
public int getState() {
return state;
}
public void setState(int state) {
this.state = state;
}
public String getTimeZone() {
return timeZone;
}
public void setTimeZone(String timeZone) {
this.timeZone = timeZone;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -251,29 +131,8 @@ public class User {
@Override
public int hashCode() {
int result = id;
int result = id == null ? 0 : id;
result = 31 * result + userName.hashCode();
return result;
}
@Override
public String toString() {
return "User{" +
"id=" + id +
", userName='" + userName + '\'' +
", userPassword='" + userPassword + '\'' +
", email='" + email + '\'' +
", phone='" + phone + '\'' +
", userType=" + userType +
", tenantId=" + tenantId +
", state=" + state +
", tenantCode='" + tenantCode + '\'' +
", queueName='" + queueName + '\'' +
", alertGroup='" + alertGroup + '\'' +
", queue='" + queue + '\'' +
", timeZone='" + timeZone + '\'' +
", createTime=" + createTime +
", updateTime=" + updateTime +
'}';
}
}

5
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/WorkerGroup.java
View File

@ -19,11 +19,12 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableField;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
/**
* worker group
@ -33,7 +34,7 @@ import lombok.Data;
public class WorkerGroup {
@TableId(value = "id", type = IdType.AUTO)
private int id;
private Integer id;
private String name;

73
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/WorkerServer.java
View File

@ -19,6 +19,9 @@ package org.apache.dolphinscheduler.dao.entity;
import java.util.Date;
import lombok.Data;
@Data
public class WorkerServer {
/**
@ -36,7 +39,6 @@ public class WorkerServer {
*/
private int port;
/**
* zookeeper directory
*/
@ -56,73 +58,4 @@ public class WorkerServer {
* last heart beat time
*/
private Date lastHeartbeatTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getHost() {
return host;
}
public void setHost(String host) {
this.host = host;
}
public int getPort() {
return port;
}
public void setPort(int port) {
this.port = port;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public String getZkDirectory() {
return zkDirectory;
}
public void setZkDirectory(String zkDirectory) {
this.zkDirectory = zkDirectory;
}
public Date getLastHeartbeatTime() {
return lastHeartbeatTime;
}
public void setLastHeartbeatTime(Date lastHeartbeatTime) {
this.lastHeartbeatTime = lastHeartbeatTime;
}
public String getResInfo() {
return resInfo;
}
public void setResInfo(String resInfo) {
this.resInfo = resInfo;
}
@Override
public String toString() {
return "WorkerServer{" +
"id=" + id +
", host='" + host + '\'' +
", port=" + port +
", zkDirectory='" + zkDirectory + '\'' +
", resInfo='" + resInfo + '\'' +
", createTime=" + createTime +
", lastHeartbeatTime=" + lastHeartbeatTime +
'}';
}
}

2
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.java
View File

@ -250,7 +250,7 @@ public interface ProcessInstanceMapper extends BaseMapper<ProcessInstance> {
List<ProcessInstance> queryByProcessDefineCodeAndProcessDefinitionVersionAndStatusAndNextId(@Param("processDefinitionCode") Long processDefinitionCode,
@Param("processDefinitionVersion") int processDefinitionVersion,
@Param("states") int[] states,
@Param("id") int id);
@Param("id") Integer id);
int updateGlobalParamsById(@Param("globalParams") String globalParams,
@Param("id") int id);

1
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/ProcessInstanceDao.java
View File

@ -19,7 +19,6 @@ package org.apache.dolphinscheduler.dao.repository;
import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
public interface ProcessInstanceDao {
public int insertProcessInstance(ProcessInstance processInstance);

8
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/repository/impl/ProcessInstanceDaoImpl.java
View File

@ -21,12 +21,12 @@ import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
import org.apache.dolphinscheduler.dao.mapper.ProcessInstanceMapper;
import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Repository;
import lombok.NonNull;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Repository;
@Slf4j
@Repository
public class ProcessInstanceDaoImpl implements ProcessInstanceDao {
@ -46,7 +46,7 @@ public class ProcessInstanceDaoImpl implements ProcessInstanceDao {
@Override
public int upsertProcessInstance(@NonNull ProcessInstance processInstance) {
if (processInstance.getId() != 0) {
if (processInstance.getId() != null) {
return updateProcessInstance(processInstance);
} else {
return insertProcessInstance(processInstance);

5
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/utils/DqRuleUtils.java
View File

@ -34,7 +34,8 @@ public class DqRuleUtils {
public static List<DqRuleInputEntry> transformInputEntry(List<DqRuleInputEntry> ruleInputEntryList) {
for (DqRuleInputEntry dqRuleInputEntry : ruleInputEntryList) {
Map<String,Object> valuesMap = JSONUtils.toMap(dqRuleInputEntry.getValuesMap(),String.class,Object.class);
Map<String, Object> valuesMap =
JSONUtils.toMap(dqRuleInputEntry.getValuesMap(), String.class, Object.class);
if (valuesMap != null) {
if (valuesMap.get(dqRuleInputEntry.getField()) != null) {
@ -43,7 +44,7 @@ public class DqRuleUtils {
}
if (valuesMap.get("is_show") != null) {
dqRuleInputEntry.setShow(Boolean.parseBoolean(String.valueOf(valuesMap.get("is_show"))));
dqRuleInputEntry.setIsShow(Boolean.parseBoolean(String.valueOf(valuesMap.get("is_show"))));
}
if (valuesMap.get("can_edit") != null) {

4
dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProcessInstanceMapper.xml
View File

@ -257,7 +257,9 @@
</foreach>
</if>
and next_process_instance_id=0
and id <![CDATA[ < ]]> #{id}
<if test="id != null">
and id <![CDATA[ < ]]> #{id}
</if>
order by id desc
</select>
<select id="loadNextProcess4Serial" resultType="org.apache.dolphinscheduler.dao.entity.ProcessInstance">

57
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/entity/UdfFuncTest.java
View File

@ -18,48 +18,33 @@
package org.apache.dolphinscheduler.dao.entity;
import org.apache.dolphinscheduler.dao.entity.UdfFunc.UdfFuncDeserializer;
import org.junit.Assert;
import org.junit.Test;
import java.io.IOException;
import org.junit.Assert;
import org.junit.Test;
public class UdfFuncTest {
/**
* test to String
*/
@Test
public void testUdfFuncToString() {
UdfFunc udfFunc = new UdfFunc();
udfFunc.setResourceName("dolphin_resource_update");
udfFunc.setResourceId(2);
udfFunc.setClassName("org.apache.dolphinscheduler.test.mrUpdate");
/**
* test UdfFuncDeserializer.deserializeKey
*
* @throws IOException
*/
@Test
public void testUdfFuncDeserializer() throws IOException {
Assert.assertEquals("{\"id\":0,\"userId\":0,\"resourceType\":\"UDF\",\"funcName\":null,\"className\":\"org.apache.dolphinscheduler.test.mrUpdate\",\"argTypes\":null,\"database\":null,"
+ "\"description\":null,\"resourceId\":2,\"resourceName\":\"dolphin_resource_update\",\"type\":null,\"createTime\":null,\"updateTime\":null,\"userName\":null}"
, udfFunc.toString());
}
// UdfFuncDeserializer.deserializeKey key is null
UdfFuncDeserializer udfFuncDeserializer = new UdfFuncDeserializer();
Assert.assertNull(udfFuncDeserializer.deserializeKey(null, null));
/**
* test UdfFuncDeserializer.deserializeKey
*
* @throws IOException
*/
@Test
public void testUdfFuncDeserializer() throws IOException {
//
UdfFunc udfFunc = new UdfFunc();
udfFunc.setResourceName("dolphin_resource_update");
udfFunc.setResourceId(2);
udfFunc.setClassName("org.apache.dolphinscheduler.test.mrUpdate");
// UdfFuncDeserializer.deserializeKey key is null
UdfFuncDeserializer udfFuncDeserializer = new UdfFuncDeserializer();
Assert.assertNull(udfFuncDeserializer.deserializeKey(null, null));
Assert.assertNotNull(udfFuncDeserializer.deserializeKey(udfFunc.toString(), null));
}
//
UdfFunc udfFunc = new UdfFunc();
udfFunc.setResourceName("dolphin_resource_update");
udfFunc.setResourceId(2);
udfFunc.setClassName("org.apache.dolphinscheduler.test.mrUpdate");
Assert.assertNotNull(udfFuncDeserializer.deserializeKey(udfFunc.toString(), null));
}
}
}

44
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/CommandMapperTest.java
View File

@ -64,7 +64,7 @@ public class CommandMapperTest extends BaseDaoTest {
@Test
public void testInsert() {
Command command = createCommand();
assertThat(command.getId(),greaterThan(0));
assertThat(command.getId(), greaterThan(0));
}
/**
@ -73,7 +73,7 @@ public class CommandMapperTest extends BaseDaoTest {
@Test
public void testSelectById() {
Command expectedCommand = createCommand();
//query
// query
Command actualCommand = commandMapper.selectById(expectedCommand.getId());
assertNotNull(actualCommand);
@ -96,7 +96,7 @@ public class CommandMapperTest extends BaseDaoTest {
Command actualCommand = commandMapper.selectById(expectedCommand.getId());
assertNotNull(actualCommand);
assertEquals(expectedCommand.getUpdateTime(),actualCommand.getUpdateTime());
assertEquals(expectedCommand.getUpdateTime(), actualCommand.getUpdateTime());
}
@ -114,8 +114,6 @@ public class CommandMapperTest extends BaseDaoTest {
assertNull(actualCommand);
}
/**
* test query all
*/
@ -140,7 +138,7 @@ public class CommandMapperTest extends BaseDaoTest {
createCommand(CommandType.START_PROCESS, processDefinition.getCode());
List<Command> actualCommand = commandMapper.queryCommandPage(1,0);
List<Command> actualCommand = commandMapper.queryCommandPage(1, 0);
assertNotNull(actualCommand);
}
@ -164,7 +162,7 @@ public class CommandMapperTest extends BaseDaoTest {
List<CommandCount> actualCommandCounts = commandMapper.countCommandState(startTime, endTime, projectCodeArray);
assertThat(actualCommandCounts.size(),greaterThanOrEqualTo(1));
assertThat(actualCommandCounts.size(), greaterThanOrEqualTo(1));
}
/**
@ -175,19 +173,19 @@ public class CommandMapperTest extends BaseDaoTest {
int masterCount = 4;
int thisMasterSlot = 2;
// for hit or miss
toTestQueryCommandPageBySlot(masterCount,thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount,thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount,thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount,thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount, thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount, thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount, thisMasterSlot);
toTestQueryCommandPageBySlot(masterCount, thisMasterSlot);
}
private boolean toTestQueryCommandPageBySlot(int masterCount, int thisMasterSlot) {
Command command = createCommand();
int id = command.getId();
Integer id = command.getId();
boolean hit = id % masterCount == thisMasterSlot;
List<Command> commandList = commandMapper.queryCommandPageBySlot(1, 0, masterCount, thisMasterSlot);
if (hit) {
assertEquals(id,commandList.get(0).getId());
assertEquals(id, commandList.get(0).getId());
} else {
commandList.forEach(o -> {
assertNotEquals(id, o.getId());
@ -197,8 +195,6 @@ public class CommandMapperTest extends BaseDaoTest {
return hit;
}
/**
* create command map
* @param count map count
@ -207,13 +203,13 @@ public class CommandMapperTest extends BaseDaoTest {
* @return command map
*/
private CommandCount createCommandMap(
Integer count,
CommandType commandType,
long processDefinitionCode) {
Integer count,
CommandType commandType,
long processDefinitionCode) {
CommandCount commandCount = new CommandCount();
for (int i = 0;i < count;i++) {
for (int i = 0; i < count; i++) {
createCommand(commandType, processDefinitionCode);
}
commandCount.setCommandType(commandType);
@ -246,12 +242,12 @@ public class CommandMapperTest extends BaseDaoTest {
* @param count map count
* @return command map
*/
private Map<Integer,Command> createCommandMap(Integer count) {
Map<Integer,Command> commandMap = new HashMap<>();
private Map<Integer, Command> createCommandMap(Integer count) {
Map<Integer, Command> commandMap = new HashMap<>();
for (int i = 0;i < count;i++) {
for (int i = 0; i < count; i++) {
Command command = createCommand();
commandMap.put(command.getId(),command);
commandMap.put(command.getId(), command);
}
return commandMap;
}
@ -261,7 +257,7 @@ public class CommandMapperTest extends BaseDaoTest {
* @return
*/
private Command createCommand() {
return createCommand(CommandType.START_PROCESS,1);
return createCommand(CommandType.START_PROCESS, 1);
}
/**

58
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/DataSourceMapperTest.java
View File

@ -18,7 +18,6 @@
package org.apache.dolphinscheduler.dao.mapper;
import static java.util.stream.Collectors.toList;
import static org.hamcrest.Matchers.greaterThan;
import static org.hamcrest.Matchers.greaterThanOrEqualTo;
import static org.junit.Assert.assertEquals;
@ -86,7 +85,6 @@ public class DataSourceMapperTest extends BaseDaoTest {
assertEquals(expectedDataSource, actualDataSource);
}
/**
* test query
*/
@ -108,7 +106,6 @@ public class DataSourceMapperTest extends BaseDaoTest {
assertEquals(expectedDataSource, actualDataSource);
}
/**
* test delete
*/
@ -123,8 +120,6 @@ public class DataSourceMapperTest extends BaseDaoTest {
assertNull(actualDataSource);
}
/**
* test query datasource by type
*/
@ -142,7 +137,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
for (DataSource actualDataSource : actualDataSources) {
DataSource expectedDataSource = datasourceMap.get(actualDataSource.getId());
if (expectedDataSource != null) {
assertEquals(expectedDataSource,actualDataSource);
assertEquals(expectedDataSource, actualDataSource);
}
}
@ -166,7 +161,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
for (DataSource actualDataSource : actualDataSources) {
DataSource expectedDataSource = expectedDataSourceMap.get(actualDataSource.getId());
if (expectedDataSource != null) {
assertEquals(expectedDataSource,actualDataSource);
assertEquals(expectedDataSource, actualDataSource);
}
}
@ -184,7 +179,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
for (DataSource actualDataSource : actualDataSources) {
if (expectedDataSource.getId() == actualDataSource.getId()) {
assertEquals(expectedDataSource,actualDataSource);
assertEquals(expectedDataSource, actualDataSource);
}
}
@ -205,7 +200,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
for (DataSource actualDataSource : actualDataSources) {
DataSource expectedDataSource = expectedDataSourceMap.get(actualDataSource.getId());
if (expectedDataSource != null) {
assertEquals(expectedDataSource,actualDataSource);
assertEquals(expectedDataSource, actualDataSource);
}
}
@ -226,7 +221,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
for (DataSource actualDataSource : actualDataSources) {
DataSource expectedDataSource = expectedDataSourceMap.get(actualDataSource.getId());
if (expectedDataSource != null) {
assertEquals(expectedDataSource,actualDataSource);
assertEquals(expectedDataSource, actualDataSource);
}
}
}
@ -247,48 +242,51 @@ public class DataSourceMapperTest extends BaseDaoTest {
for (DataSource actualDataSource : actualDataSources) {
DataSource expectedDataSource = expectedDataSourceMap.get(actualDataSource.getId());
if (expectedDataSource != null) {
assertEquals(expectedDataSource,actualDataSource);
assertEquals(expectedDataSource, actualDataSource);
}
}
}
@Test
public void testListAuthorizedDataSource() {
//create general user
// create general user
User generalUser1 = createGeneralUser("user1");
User generalUser2 = createGeneralUser("user2");
//create data source
// create data source
DataSource dataSource = createDataSource(generalUser1.getId(), "ds-1");
DataSource unauthorizdDataSource = createDataSource(generalUser2.getId(), "ds-2");
//data source ids
// data source ids
Integer[] dataSourceIds = new Integer[]{dataSource.getId(), unauthorizdDataSource.getId()};
List<DataSource> authorizedDataSource = dataSourceMapper.listAuthorizedDataSource(generalUser1.getId(), dataSourceIds);
List<DataSource> authorizedDataSource =
dataSourceMapper.listAuthorizedDataSource(generalUser1.getId(), dataSourceIds);
assertEquals(generalUser1.getId(), dataSource.getUserId());
Assert.assertNotEquals(generalUser1.getId(), unauthorizdDataSource.getUserId());
Assert.assertFalse(authorizedDataSource.stream().map(t -> t.getId()).collect(toList()).containsAll(Arrays.asList(dataSourceIds)));
assertEquals(generalUser1.getId().intValue(), dataSource.getUserId());
Assert.assertNotEquals(generalUser1.getId().intValue(), unauthorizdDataSource.getUserId());
Assert.assertFalse(authorizedDataSource.stream().map(t -> t.getId()).collect(toList())
.containsAll(Arrays.asList(dataSourceIds)));
//authorize object unauthorizdDataSource to generalUser1
// authorize object unauthorizdDataSource to generalUser1
createUserDataSource(generalUser1, unauthorizdDataSource);
authorizedDataSource = dataSourceMapper.listAuthorizedDataSource(generalUser1.getId(), dataSourceIds);
Assert.assertTrue(authorizedDataSource.stream().map(t -> t.getId()).collect(toList()).containsAll(Arrays.asList(dataSourceIds)));
Assert.assertTrue(authorizedDataSource.stream().map(t -> t.getId()).collect(toList())
.containsAll(Arrays.asList(dataSourceIds)));
}
/**
* create datasource relation
* @param userId
*/
private Map<Integer,DataSource> createDataSourceMap(Integer userId,String name) {
private Map<Integer, DataSource> createDataSourceMap(Integer userId, String name) {
Map<Integer,DataSource> dataSourceMap = new HashMap<>();
Map<Integer, DataSource> dataSourceMap = new HashMap<>();
DataSource dataSource = createDataSource(userId, name);
dataSourceMap.put(dataSource.getId(),dataSource);
dataSourceMap.put(dataSource.getId(), dataSource);
DataSource otherDataSource = createDataSource(userId + 1, name + "1");
@ -312,12 +310,12 @@ public class DataSourceMapperTest extends BaseDaoTest {
* @param count datasource count
* @return datasource map
*/
private Map<Integer,DataSource> createDataSourceMap(Integer count) {
Map<Integer,DataSource> dataSourceMap = new HashMap<>();
private Map<Integer, DataSource> createDataSourceMap(Integer count) {
Map<Integer, DataSource> dataSourceMap = new HashMap<>();
for (int i = 0; i < count; i++) {
DataSource dataSource = createDataSource("test");
dataSourceMap.put(dataSource.getId(),dataSource);
dataSourceMap.put(dataSource.getId(), dataSource);
}
return dataSourceMap;
@ -328,7 +326,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
* @return datasource
*/
private DataSource createDataSource() {
return createDataSource(1,"test");
return createDataSource(1, "test");
}
/**
@ -337,7 +335,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
* @return datasource
*/
private DataSource createDataSource(String name) {
return createDataSource(1,name);
return createDataSource(1, name);
}
/**
@ -346,7 +344,7 @@ public class DataSourceMapperTest extends BaseDaoTest {
* @param name name
* @return datasource
*/
private DataSource createDataSource(Integer userId,String name) {
private DataSource createDataSource(Integer userId, String name) {
Random random = new Random();
DataSource dataSource = new DataSource();
dataSource.setUserId(userId);
@ -399,4 +397,4 @@ public class DataSourceMapperTest extends BaseDaoTest {
return datasourceUser;
}
}
}

18
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/K8sNamespaceMapperTest.java
View File

@ -43,7 +43,7 @@ public class K8sNamespaceMapperTest extends BaseDaoTest {
* @return K8sNamespace
*/
private K8sNamespace insertOne() {
//insertOne
// insertOne
K8sNamespace k8sNamespace = new K8sNamespace();
k8sNamespace.setCode(999L);
k8sNamespace.setNamespace("testNamespace");
@ -78,10 +78,10 @@ public class K8sNamespaceMapperTest extends BaseDaoTest {
*/
@Test
public void testUpdate() {
//insertOne
// insertOne
K8sNamespace k8sNamespace = insertOne();
k8sNamespace.setLimitsMemory(200);
//update
// update
int update = k8sNamespaceMapper.updateById(k8sNamespace);
Assert.assertEquals(update, 1);
}
@ -102,12 +102,11 @@ public class K8sNamespaceMapperTest extends BaseDaoTest {
@Test
public void testQuery() {
insertOne();
//query
// query
List<K8sNamespace> k8sNamespaces = k8sNamespaceMapper.selectList(null);
Assert.assertEquals(k8sNamespaces.size(), 1);
}
/**
* test query k8sNamespaces by id
*/
@ -115,10 +114,9 @@ public class K8sNamespaceMapperTest extends BaseDaoTest {
public void testQueryByK8sNamespaceId() {
K8sNamespace entity = insertOne();
K8sNamespace k8sNamespace = k8sNamespaceMapper.selectById(entity.getId());
Assert.assertEquals(entity.toString(),k8sNamespace.toString());
Assert.assertEquals(entity, k8sNamespace);
}
/**
* test query k8sNamespaces list paging
*/
@ -126,13 +124,13 @@ public class K8sNamespaceMapperTest extends BaseDaoTest {
public void testQueryK8sNamespaceListPaging() {
K8sNamespace entity = insertOne();
Page<K8sNamespace> page = new Page<>(1, 10);
IPage<K8sNamespace> k8sNamespaceIPage = k8sNamespaceMapper.queryK8sNamespacePaging(page,"");
IPage<K8sNamespace> k8sNamespaceIPage = k8sNamespaceMapper.queryK8sNamespacePaging(page, "");
List<K8sNamespace> k8sNamespaceList = k8sNamespaceIPage.getRecords();
Assert.assertEquals(k8sNamespaceList.size(), 1);
k8sNamespaceIPage = k8sNamespaceMapper.queryK8sNamespacePaging(page,"abc");
k8sNamespaceIPage = k8sNamespaceMapper.queryK8sNamespacePaging(page, "abc");
k8sNamespaceList = k8sNamespaceIPage.getRecords();
Assert.assertEquals(k8sNamespaceList.size(), 0);
}
}
}

12
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProcessDefinitionLogMapperTest.java
View File

@ -34,6 +34,7 @@ import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
public class ProcessDefinitionLogMapperTest extends BaseDaoTest {
@Autowired
private UserMapper userMapper;
@ -49,7 +50,7 @@ public class ProcessDefinitionLogMapperTest extends BaseDaoTest {
* @return ProcessDefinition
*/
private ProcessDefinitionLog insertOne() {
//insertOne
// insertOne
ProcessDefinitionLog processDefinitionLog = new ProcessDefinitionLog();
processDefinitionLog.setCode(1L);
processDefinitionLog.setName("def 1");
@ -68,7 +69,7 @@ public class ProcessDefinitionLogMapperTest extends BaseDaoTest {
* @return ProcessDefinition
*/
private ProcessDefinitionLog insertTwo() {
//insertOne
// insertOne
ProcessDefinitionLog processDefinitionLog = new ProcessDefinitionLog();
processDefinitionLog.setCode(1L);
processDefinitionLog.setName("def 2");
@ -85,7 +86,7 @@ public class ProcessDefinitionLogMapperTest extends BaseDaoTest {
@Test
public void testInsert() {
ProcessDefinitionLog processDefinitionLog = insertOne();
Assert.assertNotEquals(processDefinitionLog.getId(), 0);
Assert.assertNotEquals(processDefinitionLog.getId().intValue(), 0);
}
@Test
@ -107,7 +108,7 @@ public class ProcessDefinitionLogMapperTest extends BaseDaoTest {
List<ProcessDefinitionLog> processDefinitionLogs = processDefinitionLogMapper
.queryByDefinitionName(1L, "def 1");
Assert.assertEquals(0, processDefinitionLogs.size());
Assert.assertEquals(1, processDefinitionLogs.size());
}
@ -142,7 +143,8 @@ public class ProcessDefinitionLogMapperTest extends BaseDaoTest {
public void testQueryProcessDefinitionVersionsPaging() {
insertOne();
Page<ProcessDefinitionLog> page = new Page(1, 3);
IPage<ProcessDefinitionLog> processDefinitionLogs = processDefinitionLogMapper.queryProcessDefinitionVersionsPaging(page, 1L,1L);
IPage<ProcessDefinitionLog> processDefinitionLogs =
processDefinitionLogMapper.queryProcessDefinitionVersionsPaging(page, 1L, 1L);
Assert.assertNotEquals(processDefinitionLogs.getTotal(), 0);
}

61
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ResourceMapperTest.java
View File

@ -23,13 +23,13 @@ import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertThat;
import org.apache.dolphinscheduler.common.Constants;
import org.apache.dolphinscheduler.spi.enums.ResourceType;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.dao.BaseDaoTest;
import org.apache.dolphinscheduler.dao.entity.Resource;
import org.apache.dolphinscheduler.dao.entity.ResourcesUser;
import org.apache.dolphinscheduler.dao.entity.Tenant;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.spi.enums.ResourceType;
import org.apache.commons.collections.CollectionUtils;
@ -65,7 +65,7 @@ public class ResourceMapperTest extends BaseDaoTest {
* @return Resource
*/
private Resource insertOne() {
//insertOne
// insertOne
Resource resource = new Resource();
resource.setAlias("ut-resource");
resource.setFullName("/ut-resource");
@ -86,8 +86,9 @@ public class ResourceMapperTest extends BaseDaoTest {
* @param user user
* @return Resource
*/
private Resource createResource(User user, boolean isDirectory, ResourceType resourceType, int pid, String alias, String fullName) {
//insertOne
private Resource createResource(User user, boolean isDirectory, ResourceType resourceType, int pid, String alias,
String fullName) {
// insertOne
Resource resource = new Resource();
resource.setDirectory(isDirectory);
resource.setType(resourceType);
@ -108,7 +109,7 @@ public class ResourceMapperTest extends BaseDaoTest {
* @return Resource
*/
private Resource createResource(User user) {
//insertOne
// insertOne
String alias = String.format("ut-resource-%s", user.getUserName());
String fullName = String.format("/%s", alias);
@ -144,7 +145,7 @@ public class ResourceMapperTest extends BaseDaoTest {
* @return ResourcesUser
*/
private ResourcesUser createResourcesUser(Resource resource, User user) {
//insertOne
// insertOne
ResourcesUser resourcesUser = new ResourcesUser();
resourcesUser.setCreateTime(new Date());
resourcesUser.setUpdateTime(new Date());
@ -167,10 +168,10 @@ public class ResourceMapperTest extends BaseDaoTest {
*/
@Test
public void testUpdate() {
//insertOne
// insertOne
Resource resource = insertOne();
resource.setCreateTime(new Date());
//update
// update
int update = resourceMapper.updateById(resource);
Assert.assertEquals(1, update);
}
@ -191,7 +192,7 @@ public class ResourceMapperTest extends BaseDaoTest {
@Test
public void testQuery() {
Resource resource = insertOne();
//query
// query
List<Resource> resources = resourceMapper.selectList(null);
Assert.assertNotEquals(resources.size(), 0);
}
@ -210,8 +211,7 @@ public class ResourceMapperTest extends BaseDaoTest {
List<Resource> resources = resourceMapper.queryResourceList(
alias,
userId,
type
);
type);
Assert.assertNotEquals(resources.size(), 0);
}
@ -246,15 +246,13 @@ public class ResourceMapperTest extends BaseDaoTest {
-1,
resource.getType().ordinal(),
"",
new ArrayList<>(resource.getId())
);
new ArrayList<>(resource.getId()));
IPage<Resource> resourceIPage1 = resourceMapper.queryResourcePaging(
page,
-1,
resource.getType().ordinal(),
"",
null
);
null);
Assert.assertEquals(resourceIPage.getTotal(), 1);
Assert.assertEquals(resourceIPage1.getTotal(), 1);
@ -267,8 +265,10 @@ public class ResourceMapperTest extends BaseDaoTest {
public void testQueryResourceListAuthored() {
Resource resource = insertOne();
List<Integer> resIds = resourceUserMapper.queryResourcesIdListByUserIdAndPerm(resource.getUserId(), Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> resources = CollectionUtils.isEmpty(resIds) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds);
List<Integer> resIds = resourceUserMapper.queryResourcesIdListByUserIdAndPerm(resource.getUserId(),
Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> resources =
CollectionUtils.isEmpty(resIds) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds);
ResourcesUser resourcesUser = new ResourcesUser();
@ -277,8 +277,10 @@ public class ResourceMapperTest extends BaseDaoTest {
resourcesUser.setPerm(Constants.AUTHORIZE_WRITABLE_PERM);
resourceUserMapper.insert(resourcesUser);
List<Integer> resIds1 = resourceUserMapper.queryResourcesIdListByUserIdAndPerm(1110, Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> resources1 = CollectionUtils.isEmpty(resIds1) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds1);
List<Integer> resIds1 =
resourceUserMapper.queryResourcesIdListByUserIdAndPerm(1110, Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> resources1 =
CollectionUtils.isEmpty(resIds1) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds1);
Assert.assertEquals(0, resources.size());
Assert.assertNotEquals(0, resources1.size());
@ -292,8 +294,10 @@ public class ResourceMapperTest extends BaseDaoTest {
public void testQueryAuthorizedResourceList() {
Resource resource = insertOne();
List<Integer> resIds = resourceUserMapper.queryResourcesIdListByUserIdAndPerm(resource.getUserId(), Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> resources = CollectionUtils.isEmpty(resIds) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds);
List<Integer> resIds = resourceUserMapper.queryResourcesIdListByUserIdAndPerm(resource.getUserId(),
Constants.AUTHORIZE_WRITABLE_PERM);
List<Resource> resources =
CollectionUtils.isEmpty(resIds) ? new ArrayList<>() : resourceMapper.queryResourceListById(resIds);
resourceMapper.deleteById(resource.getId());
Assert.assertEquals(0, resources.size());
@ -306,8 +310,7 @@ public class ResourceMapperTest extends BaseDaoTest {
public void testQueryResourceExceptUserId() {
Resource resource = insertOne();
List<Resource> resources = resourceMapper.queryResourceExceptUserId(
11111
);
11111);
Assert.assertNotEquals(resources.size(), 0);
}
@ -365,13 +368,15 @@ public class ResourceMapperTest extends BaseDaoTest {
List<Resource> resources = resourceMapper.listAuthorizedResource(generalUser2.getId(), resNames);
Assert.assertEquals(generalUser2.getId(), resource.getUserId());
Assert.assertFalse(resources.stream().map(t -> t.getFullName()).collect(toList()).containsAll(Arrays.asList(resNames)));
Assert.assertEquals(generalUser2.getId().intValue(), resource.getUserId());
Assert.assertFalse(
resources.stream().map(t -> t.getFullName()).collect(toList()).containsAll(Arrays.asList(resNames)));
// authorize object unauthorizedResource to generalUser
createResourcesUser(unauthorizedResource, generalUser2);
List<Resource> authorizedResources = resourceMapper.listAuthorizedResource(generalUser2.getId(), resNames);
Assert.assertTrue(authorizedResources.stream().map(t -> t.getFullName()).collect(toList()).containsAll(Arrays.asList(resource.getFullName())));
Assert.assertTrue(authorizedResources.stream().map(t -> t.getFullName()).collect(toList())
.containsAll(Arrays.asList(resource.getFullName())));
}
@ -400,7 +405,8 @@ public class ResourceMapperTest extends BaseDaoTest {
Resource resource = createResource(generalUser1);
createResourcesUser(resource, generalUser2);
List<Resource> resourceList = resourceMapper.queryResourceListAuthored(generalUser2.getId(), ResourceType.FILE.ordinal());
List<Resource> resourceList =
resourceMapper.queryResourceListAuthored(generalUser2.getId(), ResourceType.FILE.ordinal());
Assert.assertNotNull(resourceList);
resourceList = resourceMapper.queryResourceListAuthored(generalUser2.getId(), ResourceType.FILE.ordinal());
@ -435,4 +441,3 @@ public class ResourceMapperTest extends BaseDaoTest {
Assert.assertTrue(resourceMapper.existResource(fullName, type));
}
}

22
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ScheduleMapperTest.java
View File

@ -55,7 +55,7 @@ public class ScheduleMapperTest extends BaseDaoTest {
* @return Schedule
*/
private Schedule insertOne() {
//insertOne
// insertOne
Schedule schedule = new Schedule();
schedule.setStartTime(new Date());
schedule.setEndTime(new Date());
@ -74,10 +74,10 @@ public class ScheduleMapperTest extends BaseDaoTest {
*/
@Test
public void testUpdate() {
//insertOne
// insertOne
Schedule schedule = insertOne();
schedule.setCreateTime(new Date());
//update
// update
int update = scheduleMapper.updateById(schedule);
Assert.assertEquals(update, 1);
}
@ -98,7 +98,7 @@ public class ScheduleMapperTest extends BaseDaoTest {
@Test
public void testQuery() {
Schedule schedule = insertOne();
//query
// query
List<Schedule> schedules = scheduleMapper.selectList(null);
Assert.assertNotEquals(schedules.size(), 0);
}
@ -133,9 +133,9 @@ public class ScheduleMapperTest extends BaseDaoTest {
Schedule schedule = insertOne();
schedule.setUserId(user.getId());
schedule.setProcessDefinitionCode(processDefinition.getCode());
scheduleMapper.insert(schedule);
scheduleMapper.updateById(schedule);
Page<Schedule> page = new Page(1,3);
Page<Schedule> page = new Page(1, 3);
IPage<Schedule> scheduleIPage = scheduleMapper.queryByProcessDefineCodePaging(page,
processDefinition.getCode(), "");
Assert.assertNotEquals(scheduleIPage.getSize(), 0);
@ -171,12 +171,11 @@ public class ScheduleMapperTest extends BaseDaoTest {
Schedule schedule = insertOne();
schedule.setUserId(user.getId());
schedule.setProcessDefinitionCode(processDefinition.getCode());
scheduleMapper.insert(schedule);
scheduleMapper.updateById(schedule);
Page<Schedule> page = new Page(1,3);
Page<Schedule> page = new Page(1, 3);
List<Schedule> schedules = scheduleMapper.querySchedulerListByProjectName(
project.getName()
);
project.getName());
Assert.assertNotEquals(schedules.size(), 0);
}
@ -192,7 +191,8 @@ public class ScheduleMapperTest extends BaseDaoTest {
schedule.setReleaseState(ReleaseState.ONLINE);
scheduleMapper.updateById(schedule);
List<Schedule> schedules = scheduleMapper.selectAllByProcessDefineArray(new long[] {schedule.getProcessDefinitionCode()});
List<Schedule> schedules =
scheduleMapper.selectAllByProcessDefineArray(new long[]{schedule.getProcessDefinitionCode()});
Assert.assertNotEquals(schedules.size(), 0);
}

2
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionLogMapperTest.java
View File

@ -57,7 +57,7 @@ public class TaskDefinitionLogMapperTest extends BaseDaoTest {
@Test
public void testInsert() {
TaskDefinitionLog taskDefinitionLog = insertOne();
Assert.assertNotEquals(taskDefinitionLog.getId(), 0);
Assert.assertNotEquals(taskDefinitionLog.getId().intValue(), 0);
}
@Test

18
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapperTest.java
View File

@ -85,15 +85,15 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
@Test
public void testInsert() {
TaskDefinition taskDefinition = insertOne();
Assert.assertNotEquals(taskDefinition.getId(), 0);
Assert.assertNotEquals(taskDefinition.getId().intValue(), 0);
}
@Test
public void testQueryByDefinitionName() {
TaskDefinition taskDefinition = insertOne();
ProcessTaskRelation processTaskRelation = insertTaskRelation(taskDefinition.getCode());
TaskDefinition result = taskDefinitionMapper.queryByName(taskDefinition.getProjectCode(), processTaskRelation.getProcessDefinitionCode()
, taskDefinition.getName());
TaskDefinition result = taskDefinitionMapper.queryByName(taskDefinition.getProjectCode(),
processTaskRelation.getProcessDefinitionCode(), taskDefinition.getName());
Assert.assertNotNull(result);
}
@ -109,7 +109,8 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
@Test
public void testQueryAllDefinitionList() {
TaskDefinition taskDefinition = insertOne();
List<TaskDefinition> taskDefinitions = taskDefinitionMapper.queryAllDefinitionList(taskDefinition.getProjectCode());
List<TaskDefinition> taskDefinitions =
taskDefinitionMapper.queryAllDefinitionList(taskDefinition.getProjectCode());
Assert.assertNotEquals(taskDefinitions.size(), 0);
}
@ -122,7 +123,8 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
User un = userMapper.queryByUserNameAccurately("un");
TaskDefinition taskDefinition = insertOne(un.getId());
List<DefinitionGroupByUser> users = taskDefinitionMapper.countDefinitionGroupByUser(new Long[]{taskDefinition.getProjectCode()});
List<DefinitionGroupByUser> users =
taskDefinitionMapper.countDefinitionGroupByUser(new Long[]{taskDefinition.getProjectCode()});
Assert.assertNotEquals(users.size(), 0);
}
@ -158,7 +160,8 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
@Test
public void testNullPropertyValueOfLocalParams() {
String definitionJson = "{\"failRetryTimes\":\"0\",\"timeoutNotifyStrategy\":\"\",\"code\":\"5195043558720\",\"flag\":\"YES\",\"environmentCode\":\"-1\",\"taskDefinitionIndex\":2,\"taskPriority\":\"MEDIUM\",\"taskParams\":\"{\\\"preStatements\\\":null,\\\"postStatements\\\":null,\\\"type\\\":\\\"ADB_MYSQL\\\",\\\"database\\\":\\\"lijia\\\",\\\"sql\\\":\\\"create table nation_${random_serial_number} as select * from nation\\\",\\\"localParams\\\":[{\\\"direct\\\":2,\\\"type\\\":3,\\\"prop\\\":\\\"key\\\"}],\\\"Name\\\":\\\"create_table_as_select_nation\\\",\\\"FailRetryTimes\\\":0,\\\"dbClusterId\\\":\\\"amv-bp10o45925jpx959\\\",\\\"sendEmail\\\":false,\\\"displayRows\\\":10,\\\"limit\\\":10000,\\\"agentSource\\\":\\\"Workflow\\\",\\\"agentVersion\\\":\\\"Unkown\\\"}\",\"timeout\":\"0\",\"taskType\":\"ADB_MYSQL\",\"timeoutFlag\":\"CLOSE\",\"projectCode\":\"5191800302720\",\"name\":\"create_table_as_select_nation\",\"delayTime\":\"0\",\"workerGroup\":\"default\"}";
String definitionJson =
"{\"failRetryTimes\":\"0\",\"timeoutNotifyStrategy\":\"\",\"code\":\"5195043558720\",\"flag\":\"YES\",\"environmentCode\":\"-1\",\"taskDefinitionIndex\":2,\"taskPriority\":\"MEDIUM\",\"taskParams\":\"{\\\"preStatements\\\":null,\\\"postStatements\\\":null,\\\"type\\\":\\\"ADB_MYSQL\\\",\\\"database\\\":\\\"lijia\\\",\\\"sql\\\":\\\"create table nation_${random_serial_number} as select * from nation\\\",\\\"localParams\\\":[{\\\"direct\\\":2,\\\"type\\\":3,\\\"prop\\\":\\\"key\\\"}],\\\"Name\\\":\\\"create_table_as_select_nation\\\",\\\"FailRetryTimes\\\":0,\\\"dbClusterId\\\":\\\"amv-bp10o45925jpx959\\\",\\\"sendEmail\\\":false,\\\"displayRows\\\":10,\\\"limit\\\":10000,\\\"agentSource\\\":\\\"Workflow\\\",\\\"agentVersion\\\":\\\"Unkown\\\"}\",\"timeout\":\"0\",\"taskType\":\"ADB_MYSQL\",\"timeoutFlag\":\"CLOSE\",\"projectCode\":\"5191800302720\",\"name\":\"create_table_as_select_nation\",\"delayTime\":\"0\",\"workerGroup\":\"default\"}";
TaskDefinition definition = JSONUtils.parseObject(definitionJson, TaskDefinition.class);
Map<String, String> taskParamsMap = definition.getTaskParamMap();
@ -174,7 +177,8 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
@Test
public void testNullLocalParamsOfTaskParams() {
String definitionJson = "{\"failRetryTimes\":\"0\",\"timeoutNotifyStrategy\":\"\",\"code\":\"5195043558720\",\"flag\":\"YES\",\"environmentCode\":\"-1\",\"taskDefinitionIndex\":2,\"taskPriority\":\"MEDIUM\",\"taskParams\":\"{\\\"preStatements\\\":null,\\\"postStatements\\\":null,\\\"type\\\":\\\"ADB_MYSQL\\\",\\\"database\\\":\\\"lijia\\\",\\\"sql\\\":\\\"create table nation_${random_serial_number} as select * from nation\\\",\\\"localParams\\\":null,\\\"Name\\\":\\\"create_table_as_select_nation\\\",\\\"FailRetryTimes\\\":0,\\\"dbClusterId\\\":\\\"amv-bp10o45925jpx959\\\",\\\"sendEmail\\\":false,\\\"displayRows\\\":10,\\\"limit\\\":10000,\\\"agentSource\\\":\\\"Workflow\\\",\\\"agentVersion\\\":\\\"Unkown\\\"}\",\"timeout\":\"0\",\"taskType\":\"ADB_MYSQL\",\"timeoutFlag\":\"CLOSE\",\"projectCode\":\"5191800302720\",\"name\":\"create_table_as_select_nation\",\"delayTime\":\"0\",\"workerGroup\":\"default\"}";
String definitionJson =
"{\"failRetryTimes\":\"0\",\"timeoutNotifyStrategy\":\"\",\"code\":\"5195043558720\",\"flag\":\"YES\",\"environmentCode\":\"-1\",\"taskDefinitionIndex\":2,\"taskPriority\":\"MEDIUM\",\"taskParams\":\"{\\\"preStatements\\\":null,\\\"postStatements\\\":null,\\\"type\\\":\\\"ADB_MYSQL\\\",\\\"database\\\":\\\"lijia\\\",\\\"sql\\\":\\\"create table nation_${random_serial_number} as select * from nation\\\",\\\"localParams\\\":null,\\\"Name\\\":\\\"create_table_as_select_nation\\\",\\\"FailRetryTimes\\\":0,\\\"dbClusterId\\\":\\\"amv-bp10o45925jpx959\\\",\\\"sendEmail\\\":false,\\\"displayRows\\\":10,\\\"limit\\\":10000,\\\"agentSource\\\":\\\"Workflow\\\",\\\"agentVersion\\\":\\\"Unkown\\\"}\",\"timeout\":\"0\",\"taskType\":\"ADB_MYSQL\",\"timeoutFlag\":\"CLOSE\",\"projectCode\":\"5191800302720\",\"name\":\"create_table_as_select_nation\",\"delayTime\":\"0\",\"workerGroup\":\"default\"}";
TaskDefinition definition = JSONUtils.parseObject(definitionJson, TaskDefinition.class);
Assert.assertNull("Serialize the task definition success", definition.getTaskParamMap());

65
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/UdfFuncMapperTest.java
View File

@ -163,13 +163,13 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testUpdate() {
//insertOne
// insertOne
UdfFunc udfFunc = insertOne("func1");
udfFunc.setResourceName("dolphin_resource_update");
udfFunc.setResourceId(2);
udfFunc.setClassName("org.apache.dolphinscheduler.test.mrUpdate");
udfFunc.setUpdateTime(new Date());
//update
// update
int update = udfFuncMapper.updateById(udfFunc);
Assert.assertEquals(update, 1);
@ -180,9 +180,9 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testDelete() {
//insertOne
// insertOne
UdfFunc udfFunc = insertOne("func2");
//delete
// delete
int delete = udfFuncMapper.deleteById(udfFunc.getId());
Assert.assertEquals(delete, 1);
}
@ -192,12 +192,12 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testQueryUdfByIdStr() {
//insertOne
// insertOne
UdfFunc udfFunc = insertOne("func3");
//insertOne
// insertOne
UdfFunc udfFunc1 = insertOne("func4");
Integer[] idArray = new Integer[]{udfFunc.getId(), udfFunc1.getId()};
//queryUdfByIdStr
// queryUdfByIdStr
List<UdfFunc> udfFuncList = udfFuncMapper.queryUdfByIdStr(idArray, "");
Assert.assertNotEquals(udfFuncList.size(), 0);
}
@ -207,14 +207,15 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testQueryUdfFuncPaging() {
//insertOneUser
// insertOneUser
User user = insertOneUser();
//insertOne
// insertOne
UdfFunc udfFunc = insertOne(user);
//queryUdfFuncPaging
// queryUdfFuncPaging
Page<UdfFunc> page = new Page(1, 3);
IPage<UdfFunc> udfFuncIPage = udfFuncMapper.queryUdfFuncPaging(page, Collections.singletonList(udfFunc.getId()), "");
IPage<UdfFunc> udfFuncIPage =
udfFuncMapper.queryUdfFuncPaging(page, Collections.singletonList(udfFunc.getId()), "");
Assert.assertNotEquals(udfFuncIPage.getTotal(), 0);
}
@ -224,12 +225,13 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testGetUdfFuncByType() {
//insertOneUser
// insertOneUser
User user = insertOneUser();
//insertOne
// insertOne
UdfFunc udfFunc = insertOne(user);
//getUdfFuncByType
List<UdfFunc> udfFuncList = udfFuncMapper.getUdfFuncByType(Collections.singletonList(udfFunc.getId()), udfFunc.getType().ordinal());
// getUdfFuncByType
List<UdfFunc> udfFuncList =
udfFuncMapper.getUdfFuncByType(Collections.singletonList(udfFunc.getId()), udfFunc.getType().ordinal());
Assert.assertNotEquals(udfFuncList.size(), 0);
}
@ -239,10 +241,10 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testQueryUdfFuncExceptUserId() {
//insertOneUser
// insertOneUser
User user1 = insertOneUser();
User user2 = insertOneUser("user2");
//insertOne
// insertOne
UdfFunc udfFunc1 = insertOne(user1);
UdfFunc udfFunc2 = insertOne(user2);
List<UdfFunc> udfFuncList = udfFuncMapper.queryUdfFuncExceptUserId(user1.getId());
@ -255,48 +257,49 @@ public class UdfFuncMapperTest extends BaseDaoTest {
*/
@Test
public void testQueryAuthedUdfFunc() {
//insertOneUser
// insertOneUser
User user = insertOneUser();
//insertOne
// insertOne
UdfFunc udfFunc = insertOne(user);
//insertOneUDFUser
// insertOneUDFUser
UDFUser udfUser = insertOneUDFUser(user, udfFunc);
//queryAuthedUdfFunc
// queryAuthedUdfFunc
List<UdfFunc> udfFuncList = udfFuncMapper.queryAuthedUdfFunc(user.getId());
Assert.assertNotEquals(udfFuncList.size(), 0);
}
@Test
public void testListAuthorizedUdfFunc() {
//create general user
// create general user
User generalUser1 = createGeneralUser("user1");
User generalUser2 = createGeneralUser("user2");
//create udf function
// create udf function
UdfFunc udfFunc = insertOne(generalUser1);
UdfFunc unauthorizdUdfFunc = insertOne(generalUser2);
//udf function ids
// udf function ids
Integer[] udfFuncIds = new Integer[]{udfFunc.getId(), unauthorizdUdfFunc.getId()};
List<UdfFunc> authorizedUdfFunc = udfFuncMapper.listAuthorizedUdfFunc(generalUser1.getId(), udfFuncIds);
Assert.assertEquals(generalUser1.getId(), udfFunc.getUserId());
Assert.assertNotEquals(generalUser1.getId(), unauthorizdUdfFunc.getUserId());
Assert.assertFalse(authorizedUdfFunc.stream().map(t -> t.getId()).collect(toList()).containsAll(Arrays.asList(udfFuncIds)));
Assert.assertEquals(generalUser1.getId().intValue(), udfFunc.getUserId());
Assert.assertNotEquals(generalUser1.getId().intValue(), unauthorizdUdfFunc.getUserId());
Assert.assertFalse(authorizedUdfFunc.stream().map(t -> t.getId()).collect(toList())
.containsAll(Arrays.asList(udfFuncIds)));
//authorize object unauthorizdUdfFunc to generalUser1
// authorize object unauthorizdUdfFunc to generalUser1
insertOneUDFUser(generalUser1, unauthorizdUdfFunc);
authorizedUdfFunc = udfFuncMapper.listAuthorizedUdfFunc(generalUser1.getId(), udfFuncIds);
Assert.assertTrue(authorizedUdfFunc.stream().map(t -> t.getId()).collect(toList()).containsAll(Arrays.asList(udfFuncIds)));
Assert.assertTrue(authorizedUdfFunc.stream().map(t -> t.getId()).collect(toList())
.containsAll(Arrays.asList(udfFuncIds)));
}
@Test
public void batchUpdateUdfFuncTest() {
//create general user
// create general user
User generalUser1 = createGeneralUser("user1");
UdfFunc udfFunc = insertOne(generalUser1);
udfFunc.setResourceName("/updateTest");

2
dolphinscheduler-data-quality/src/test/java/org/apache/dolphinscheduler/data/quality/flow/FlowTestBase.java
View File

@ -28,7 +28,7 @@ import java.util.Properties;
*/
public class FlowTestBase extends SparkApplicationTestBase {
protected String url = "jdbc:h2:mem:test;DB_CLOSE_DELAY=-1";
protected String url = "jdbc:h2:mem:test;MODE=MySQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=true";
protected String driver = "org.h2.Driver";

92
dolphinscheduler-dist/release-docs/LICENSE
View File

@ -218,10 +218,11 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
accessors-smart 2.4.8: https://github.com/netplex/json-smart-v2, Apache 2.0
apacheds-i18n 2.0.0-M15: https://mvnrepository.com/artifact/org.apache.directory.server/apacheds-i18n/2.0.0-M15, Apache 2.0
apacheds-kerberos-codec 2.0.0-M15: https://mvnrepository.com/artifact/org.apache.directory.server/apacheds-kerberos-codec/2.0.0-M15, Apache 2.0
aircompressor 0.3 https://mvnrepository.com/artifact/io.airlift/aircompressor, Apache 2.0
tomcat-embed-el 9.0.65: https://mvnrepository.com/artifact/org.apache.tomcat.embed/tomcat-embed-el/9.0.65, Apache 2.0
api-asn1-api 1.0.0-M20: https://mvnrepository.com/artifact/org.apache.directory.api/api-asn1-api/1.0.0-M20, Apache 2.0
api-util 1.0.0-M20: https://mvnrepository.com/artifact/org.apache.directory.api/api-util/1.0.0-M20, Apache 2.0
audience-annotations 0.5.0: https://mvnrepository.com/artifact/org.apache.yetus/audience-annotations/0.5.0, Apache 2.0
audience-annotations 0.12.0: https://mvnrepository.com/artifact/org.apache.yetus/audience-annotations/0.12.0, Apache 2.0
avro 1.7.4: https://github.com/apache/avro, Apache 2.0
bonecp 0.8.0.RELEASE: https://github.com/wwadge/bonecp, Apache 2.0
byte-buddy 1.9.16: https://mvnrepository.com/artifact/net.bytebuddy/byte-buddy/1.9.16, Apache 2.0
@ -244,43 +245,43 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
commons-math3 3.1.1: https://mvnrepository.com/artifact/org.apache.commons/commons-math3/3.1.1, Apache 2.0
commons-net 3.1: https://github.com/apache/commons-net, Apache 2.0
commons-pool 1.6: https://github.com/apache/commons-pool, Apache 2.0
cron-utils 9.1.3: https://mvnrepository.com/artifact/com.cronutils/cron-utils/9.1.3, Apache 2.0
cron-utils 9.1.6: https://mvnrepository.com/artifact/com.cronutils/cron-utils/9.1.6, Apache 2.0
commons-lang3 3.12.0: https://mvnrepository.com/artifact/org.apache.commons/commons-lang3/3.12.0, Apache 2.0
curator-client 4.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-client/4.3.0, Apache 2.0
curator-framework 4.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-framework/4.3.0, Apache 2.0
curator-recipes 4.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-recipes/4.3.0, Apache 2.0
curator-test 2.12.0: https://mvnrepository.com/artifact/org.apache.curator/curator-test/2.12.0, Apache 2.0
datanucleus-api-jdo 4.2.1: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-api-jdo/4.2.1, Apache 2.0
datanucleus-core 4.1.6: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-core/4.1.6, Apache 2.0
datanucleus-rdbms 4.1.7: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-rdbms/4.1.7, Apache 2.0
curator-client 5.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-client/5.3.0, Apache 2.0
curator-framework 5.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-framework/5.3.0, Apache 2.0
curator-recipes 5.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-recipes/5.3.0, Apache 2.0
curator-test 5.3.0: https://mvnrepository.com/artifact/org.apache.curator/curator-test/5.3.0, Apache 2.0
datanucleus-api-jdo 4.2.4: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-api-jdo/4.2.4, Apache 2.0
datanucleus-core 4.1.17: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-core/4.1.17, Apache 2.0
datanucleus-rdbms 4.1.19: https://mvnrepository.com/artifact/org.datanucleus/datanucleus-rdbms/4.1.19, Apache 2.0
derby 10.14.2.0: https://github.com/apache/derby, Apache 2.0
druid 1.1.14: https://mvnrepository.com/artifact/com.alibaba/druid/1.1.14, Apache 2.0
metrics-core 4.2.11: https://mvnrepository.com/artifact/io.dropwizard.metrics/metrics-core, Apache 2.0
error_prone_annotations 2.1.3 https://mvnrepository.com/artifact/com.google.errorprone/error_prone_annotations/2.1.3, Apache 2.0
gson 2.9.1: https://github.com/google/gson, Apache 2.0
guava 24.1-jre: https://mvnrepository.com/artifact/com.google.guava/guava/24.1-jre, Apache 2.0
guava-retrying 2.0.0: https://mvnrepository.com/artifact/com.github.rholder/guava-retrying/2.0.0, Apache 2.0
hadoop-annotations 2.7.3:https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-annotations/2.7.3, Apache 2.0
hadoop-auth 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-auth/2.7.3, Apache 2.0
hadoop-client 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3, Apache 2.0
hadoop-common 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common/2.7.3, Apache 2.0
hadoop-hdfs 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-hdfs/2.7.3, Apache 2.0
hadoop-mapreduce-client-app 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-app/2.7.3, Apache 2.0
hadoop-mapreduce-client-common 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-common/2.7.3, Apache 2.0
hadoop-mapreduce-client-core 2.7.3: https://mvnrepository.com/artifact/io.hops/hadoop-mapreduce-client-core/2.7.3, Apache 2.0
hadoop-mapreduce-client-jobclient 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-jobclient/2.7.3, Apache 2.0
hadoop-yarn-api 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-api/2.7.3, Apache 2.0
hadoop-yarn-client 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-client/2.7.3, Apache 2.0
hadoop-yarn-common 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-common/2.7.3, Apache 2.0
hadoop-yarn-server-common 2.7.3: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-server-common/2.7.3, Apache 2.0
hadoop-annotations 2.7.7:https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-annotations/2.7.7, Apache 2.0
hadoop-auth 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-auth/2.7.7, Apache 2.0
hadoop-client 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.7, Apache 2.0
hadoop-common 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common/2.7.7, Apache 2.0
hadoop-hdfs 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-hdfs/2.7.7, Apache 2.0
hadoop-mapreduce-client-app 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-app/2.7.7, Apache 2.0
hadoop-mapreduce-client-common 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-common/2.7.7, Apache 2.0
hadoop-mapreduce-client-core 2.7.7: https://mvnrepository.com/artifact/io.hops/hadoop-mapreduce-client-core/2.7.7, Apache 2.0
hadoop-mapreduce-client-jobclient 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-jobclient/2.7.7, Apache 2.0
hadoop-yarn-api 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-api/2.7.7, Apache 2.0
hadoop-yarn-client 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-client/2.7.7, Apache 2.0
hadoop-yarn-common 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-common/2.7.7, Apache 2.0
hadoop-yarn-server-common 2.7.7: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-yarn-server-common/2.7.7, Apache 2.0
HikariCP 4.0.3: https://mvnrepository.com/artifact/com.zaxxer/HikariCP/4.0.3, Apache 2.0
hive-common 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-common/2.1.0, Apache 2.0
hive-jdbc 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/2.1.0, Apache 2.0
hive-metastore 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-metastore/2.1.0, Apache 2.0
hive-orc 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-orc/2.1.0, Apache 2.0
hive-serde 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-serde/2.1.0, Apache 2.0
hive-service 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-service/2.1.0, Apache 2.0
hive-service-rpc 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-service-rpc/2.1.0, Apache 2.0
hive-storage-api 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-storage-api/2.1.0, Apache 2.0
hive-common 2.3.3: https://mvnrepository.com/artifact/org.apache.hive/hive-common/2.3.3, Apache 2.0
hive-jdbc 2.3.3: https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/2.3.3, Apache 2.0
hive-metastore 2.3.3: https://mvnrepository.com/artifact/org.apache.hive/hive-metastore/2.3.3, Apache 2.0
hive-serde 2.3.3: https://mvnrepository.com/artifact/org.apache.hive/hive-serde/2.3.3, Apache 2.0
hive-service 2.3.3: https://mvnrepository.com/artifact/org.apache.hive/hive-service/2.3.3, Apache 2.0
hive-service-rpc 2.3.3: https://mvnrepository.com/artifact/org.apache.hive/hive-service-rpc/2.3.3, Apache 2.0
hive-storage-api 2.4.0: https://mvnrepository.com/artifact/org.apache.hive/hive-storage-api/2.4.0, Apache 2.0
htrace-core 3.1.0-incubating: https://mvnrepository.com/artifact/org.apache.htrace/htrace-core/3.1.0-incubating, Apache 2.0
httpclient 4.5.13: https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient/4.5.13, Apache 2.0
httpcore 4.4.15: https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore/4.4.15, Apache 2.0
@ -307,6 +308,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
jetty-servlet 9.4.48.v20220622: https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-servlet/9.4.48.v20220622, Apache 2.0 and EPL 1.0
jetty-servlets 9.4.48.v20220622: https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-servlets/9.4.48.v20220622, Apache 2.0 and EPL 1.0
jetty-util 6.1.26: https://mvnrepository.com/artifact/org.mortbay.jetty/jetty-util/6.1.26, Apache 2.0 and EPL 1.0
jetty-sslengine 6.1.26: https://mvnrepository.com/artifact/org.mortbay.jetty/jetty-sslengine/6.1.26, Apache 2.0 and EPL 1.0
jetty-util 9.4.48.v20220622: https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-util/9.4.48.v20220622, Apache 2.0 and EPL 1.0
jetty-util-ajax 9.4.48.v20220622: https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-util-ajax/9.4.48.v20220622, Apache 2.0 and EPL 1.0
jetty-webapp 9.4.48.v20220622: https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-webapp/9.4.48.v20220622, Apache 2.0 and EPL 1.0
@ -315,37 +317,37 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
jna-platform 5.10.0: https://mvnrepository.com/artifact/net.java.dev.jna/jna-platform/5.10.0, Apache 2.0 and LGPL 2.1
joda-time 2.10.13: https://github.com/JodaOrg/joda-time, Apache 2.0
jpam 1.1: https://mvnrepository.com/artifact/net.sf.jpam/jpam/1.1, Apache 2.0
json 1.8: https://mvnrepository.com/artifact/com.tdunning/json, Apache 2.0
json-path 2.7.0: https://github.com/json-path/JsonPath, Apache 2.0
json-smart 2.4.8: https://github.com/netplex/json-smart-v2, Apache 2.0
jsqlparser 2.1: https://github.com/JSQLParser/JSqlParser, Apache 2.0 or LGPL 2.1
jsqlparser 4.4: https://github.com/JSQLParser/JSqlParser, Apache 2.0 or LGPL 4.4
jsr305 3.0.0: https://mvnrepository.com/artifact/com.google.code.findbugs/jsr305, Apache 2.0
j2objc-annotations 1.1 https://mvnrepository.com/artifact/com.google.j2objc/j2objc-annotations/1.1, Apache 2.0
libfb303 0.9.3: https://mvnrepository.com/artifact/org.apache.thrift/libfb303/0.9.3, Apache 2.0
libthrift 0.9.3: https://mvnrepository.com/artifact/org.apache.thrift/libthrift/0.9.3, Apache 2.0
log4j-api 2.11.2: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api/2.11.2, Apache 2.0
log4j-core-2.11.2: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core/2.11.2, Apache 2.0
log4j 1.2.17: https://mvnrepository.com/artifact/log4j/log4j/1.2.17, Apache 2.0
log4j-1.2-api 2.17.2: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-1.2-api/2.17.2, Apache 2.0
lz4 1.3.0: https://mvnrepository.com/artifact/net.jpountz.lz4/lz4/1.3.0, Apache 2.0
mapstruct 1.3.1.Final: https://github.com/mapstruct/mapstruct, Apache 2.0
mybatis 3.5.2 https://mvnrepository.com/artifact/org.mybatis/mybatis/3.5.2, Apache 2.0
mybatis-plus 3.2.0: https://github.com/baomidou/mybatis-plus, Apache 2.0
mybatis-plus-annotation 3.2.0: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-annotation/3.2.0, Apache 2.0
mybatis-plus-boot-starter 3.2.0: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-boot-starter/3.2.0, Apache 2.0
mybatis-plus-core 3.2.0: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-core/3.2.0, Apache 2.0
mybatis-plus-extension 3.2.0: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-extension/3.2.0, Apache 2.0
mybatis-spring 2.0.2: https://mvnrepository.com/artifact/org.mybatis/mybatis-spring/2.0.2, Apache 2.0
mybatis 3.5.10 https://mvnrepository.com/artifact/org.mybatis/mybatis/3.5.10, Apache 2.0
mybatis-plus 3.5.2: https://github.com/baomidou/mybatis-plus, Apache 2.0
mybatis-plus-annotation 3.5.2: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-annotation/3.5.2, Apache 2.0
mybatis-plus-boot-starter 3.5.2: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-boot-starter/3.5.2, Apache 2.0
mybatis-plus-core 3.5.2: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-core/3.5.2, Apache 2.0
mybatis-plus-extension 3.5.2: https://mvnrepository.com/artifact/com.baomidou/mybatis-plus-extension/3.5.2, Apache 2.0
mybatis-spring 2.0.7: https://mvnrepository.com/artifact/org.mybatis/mybatis-spring/2.0.7, Apache 2.0
netty 3.6.2.Final: https://github.com/netty/netty, Apache 2.0
netty 4.1.53.Final: https://github.com/netty/netty/blob/netty-4.1.53.Final/LICENSE.txt, Apache 2.0
opencsv 2.3: https://mvnrepository.com/artifact/net.sf.opencsv/opencsv/2.3, Apache 2.0
orc-core 1.3.3 https://mvnrepository.com/artifact/org.apache.orc/orc-core, Apache 2.0
parquet-hadoop-bundle 1.8.1: https://mvnrepository.com/artifact/org.apache.parquet/parquet-hadoop-bundle/1.8.1, Apache 2.0
poi 4.1.2: https://mvnrepository.com/artifact/org.apache.poi/poi/4.1.2, Apache 2.0
poi-ooxml 4.1.2: https://mvnrepository.com/artifact/org.apache.poi/poi-ooxml/4.1.2, Apache 2.0
poi-ooxml-schemas-4.1.2: https://mvnrepository.com/artifact/org.apache.poi/poi-ooxml-schemas/4.1.2, Apache 2.0
quartz 2.3.2: https://mvnrepository.com/artifact/org.quartz-scheduler/quartz/2.3.2, Apache 2.0
snakeyaml 1.30: https://mvnrepository.com/artifact/org.yaml/snakeyaml/1.30, Apache 2.0
snappy 0.2: https://mvnrepository.com/artifact/org.iq80.snappy/snappy/0.2, Apache 2.0
snappy-java 1.0.4.1: https://github.com/xerial/snappy-java, Apache 2.0
snappy-java 1.1.8.4: https://github.com/xerial/snappy-java, Apache 2.0
SparseBitSet 1.2: https://mvnrepository.com/artifact/com.zaxxer/SparseBitSet/1.2, Apache 2.0
spring-aop 5.3.13: https://mvnrepository.com/artifact/org.springframework/spring-aop/5.3.13, Apache 2.0
spring-beans 5.3.19: https://mvnrepository.com/artifact/org.springframework/spring-beans/5.3.19, Apache 2.0
@ -389,7 +391,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
xercesImpl 2.9.1: https://mvnrepository.com/artifact/xerces/xercesImpl/2.9.1, Apache 2.0
xmlbeans 3.1.0: https://mvnrepository.com/artifact/org.apache.xmlbeans/xmlbeans/3.1.0, Apache 2.0
xml-apis 1.3.04: https://mvnrepository.com/artifact/xml-apis/xml-apis/1.3.04, Apache 2.0 and W3C
zookeeper 3.4.14: https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.4.14, Apache 2.0
zookeeper 3.8.0: https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.8.0, Apache 2.0
presto-jdbc 0.238.1 https://mvnrepository.com/artifact/com.facebook.presto/presto-jdbc/0.238.1
protostuff-core 1.7.2: https://github.com/protostuff/protostuff/protostuff-core Apache-2.0
protostuff-runtime 1.7.2: https://github.com/protostuff/protostuff/protostuff-core Apache-2.0
@ -474,9 +476,9 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
click 8.0: https://github.com/pallets/click, BSD 3-Clause
curvesapi 1.06: https://mvnrepository.com/artifact/com.github.virtuald/curvesapi/1.06, BSD 3-clause
javolution 5.5.1: https://mvnrepository.com/artifact/javolution/javolution/5.5.1, BSD
jline 0.9.94: https://github.com/jline/jline3, BSD
jline 2.12: https://github.com/jline/jline3, BSD
jsch 0.1.42: https://mvnrepository.com/artifact/com.jcraft/jsch/0.1.42, BSD
postgresql 42.3.4: https://mvnrepository.com/artifact/org.postgresql/postgresql/42.3.4, BSD 2-clause
postgresql 42.4.1: https://mvnrepository.com/artifact/org.postgresql/postgresql/42.4.1, BSD 2-clause
protobuf-java 2.5.0: https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java/2.5.0, BSD 2-clause
paranamer 2.3: https://mvnrepository.com/artifact/com.thoughtworks.paranamer/paranamer/2.3, BSD
threetenbp 1.3.6: https://mvnrepository.com/artifact/org.threeten/threetenbp/1.3.6, BSD 3-clause
@ -519,7 +521,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
aspectjweaver 1.9.7:https://mvnrepository.com/artifact/org.aspectj/aspectjweaver/1.9.7, EPL 1.0
logback-classic 1.2.11: https://mvnrepository.com/artifact/ch.qos.logback/logback-classic/1.2.11, EPL 1.0 and LGPL 2.1
logback-core 1.2.11: https://mvnrepository.com/artifact/ch.qos.logback/logback-core/1.2.11, EPL 1.0 and LGPL 2.1
h2-1.4.200 https://github.com/h2database/h2database/blob/master/LICENSE.txt, MPL 2.0 or EPL 1.0
h2-2.1.210 https://github.com/h2database/h2database/blob/master/LICENSE.txt, MPL 2.0 or EPL 1.0
========================================================================
MIT licenses

12
dolphinscheduler-dist/release-docs/NOTICE
View File

@ -1680,16 +1680,6 @@ Commons Lang 2.6,
which has the following notices:
* This product includes software from the Spring Framework,under the Apache License 2.0 (see: StringUtils.containsWhitespace())
The binary distribution of this product bundles binaries of
Apache Log4j 1.2.17,
which has the following notices:
* ResolverUtil.java
Copyright 2005-2006 Tim Fennell
Dumbster SMTP test server
Copyright 2004 Jason Paul Kitchen
TypeUtil.java
Copyright 2002-2012 Ramnivas Laddad, Juergen Hoeller, Chris Beams
The binary distribution of this product bundles binaries of
Jetty 6.1.26,
which has the following notices:
@ -1781,7 +1771,7 @@ which has the following notices:
granted provided that the copyright notice appears in all copies./
The binary distribution of this product bundles binaries of
Snappy for Java 1.0.4.1,
Snappy for Java 1.1.8.4,
which has the following notices:
* This product includes software developed by Google
Snappy: http://code.google.com/p/snappy/ (New BSD License)

2
dolphinscheduler-dist/release-docs/licenses/LICENSE-hadoop-annotations.txt
View File

@ -1506,7 +1506,7 @@ following license:
ASM Core 3.2
JSch 0.1.42
ParaNamer Core 2.3
JLine 0.9.94
JLine 2.12
leveldbjni-all 1.8
Hamcrest Core 1.3
xmlenc Library 0.52

2
dolphinscheduler-dist/release-docs/licenses/LICENSE-hadoop-auth.txt
View File

@ -1506,7 +1506,7 @@ following license:
ASM Core 3.2
JSch 0.1.42
ParaNamer Core 2.3
JLine 0.9.94
JLine 2.12
leveldbjni-all 1.8
Hamcrest Core 1.3
xmlenc Library 0.52

2
dolphinscheduler-dist/release-docs/licenses/LICENSE-hadoop-client.txt
View File

@ -1506,7 +1506,7 @@ following license:
ASM Core 3.2
JSch 0.1.42
ParaNamer Core 2.3
JLine 0.9.94
JLine 2.12
leveldbjni-all 1.8
Hamcrest Core 1.3
xmlenc Library 0.52

2
dolphinscheduler-dist/release-docs/licenses/LICENSE-hadoop-common.txt
View File

@ -1506,7 +1506,7 @@ following license:
ASM Core 3.2
JSch 0.1.42
ParaNamer Core 2.3
JLine 0.9.94
JLine 2.12
leveldbjni-all 1.8
Hamcrest Core 1.3
xmlenc Library 0.52

Some files were not shown because too many files have changed in this diff Show More