natas_hw
/
dolphinscheduler
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type (#15967) 

* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* [Improvement][parameter] New data types and type filtering

* [Improvement][parameter] Improvement startup parameters/global parameters data type

* fix api interfaces compatible

* add project parameter data type default value

* improvement project code

* remove useless imports

* remove method onClearSearchTaskType

* add parameter doc

* optimisation logic

* code conflict resolution

* code conflict resolution

* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978)

* update

* test

* add monitor enhance ui

* update

* update

* update doc

* fix spotless

* update

* update

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* update

* fix spotless

* update

---------

Co-authored-by: Wenjun Ruan <wenjun@apache.org>

* [Improvement][Monitor] Add UT for montor (#15998)

* formatting Code
This commit is contained in:
小可耐 2024-05-21 15:39:43 +08:00 committed by GitHub
parent cf400b86ba
commit 0fc2fe971a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
47 changed files with 403 additions and 144 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/docs/en/guide/parameter/global.md
View File

@ -22,7 +22,7 @@ Create a shell task and enter `echo ${dt}` in the script content. In this case,
### Save the workflow and set global parameters
Set global parameter: On the workflow definition page, click the plus sign to the right of "Set Global", after filling in the variable name and value, save it.
Set global parameter: On the workflow definition page, click the plus sign to the right of "Set Global", after filling in the variable name and value, select the appropriate parameter value type, save it.
![global-parameter02](../../../../img/new_ui/dev/parameter/global_parameter02.png)

2
docs/docs/en/guide/parameter/project-parameter.md
View File

@ -8,7 +8,7 @@ Project-level parameters are valid for all task nodes under the entire project.
### Define project-level parameters
On the project page, click Project Parameters and Create Parameters, and fill in the parameter name and parameter value. As shown below:
On the project page, click Project Parameters and Create Parameters, and fill in the parameter name and parameter value, select the appropriate parameter value type. As shown below:
![project-parameter01](../../../../img/new_ui/dev/parameter/project_parameter01.png)

2
docs/docs/en/guide/parameter/startup-parameter.md
View File

@ -6,7 +6,7 @@ Parameters are valid for all task nodes of the entire workflow. It can be config
## Usage
Usage of startup parameters is: at the task launch page, click the '+' below the 'Startup Parameter' and fill in the key and value to save. The workflow will add them into global parameters.
Usage of startup parameters is: at the task launch page, click the '+' below the 'Startup Parameter' and fill in the key and value, select the appropriate parameter value type then to save. The workflow will add them into global parameters.
## Example

2
docs/docs/zh/guide/parameter/global.md
View File

@ -20,7 +20,7 @@
### 保存工作流，并设置全局参数
全局参数配置方式如下：在工作流定义页面，点击“设置全局”右边的加号，填写对应的变量名称和对应的值，保存即可。如下图所示：
全局参数配置方式如下：在工作流定义页面，点击“设置全局”右边的加号，填写对应的变量名称和对应的值，选择相应的参数值类型，保存即可。如下图所示：
![global-parameter02](../../../../img/new_ui/dev/parameter/global_parameter02.png)

2
docs/docs/zh/guide/parameter/project-parameter.md
View File

@ -8,7 +8,7 @@
### 定义项目级别参数
在项目管理页面，点击项目级别参数，点击创建项目级别参数，填写参数名称和参数值。如下图所示：
在项目管理页面，点击项目级别参数，点击创建项目级别参数，填写参数名称和参数值，选择相应的参数值类型。如下图所示：
![project-parameter01](../../../../img/new_ui/dev/parameter/project_parameter01.png)

2
docs/docs/zh/guide/parameter/startup-parameter.md
View File

@ -6,7 +6,7 @@
## 使用方式
启动参数配置方式如下：在启动前参数设置界面，点击“启动参数“下面的加号，填写对应的参数名称和对应的值，点击确定，工作流会将启动参数加入全局参数中。
启动参数配置方式如下：在启动前参数设置界面，点击“启动参数“下面的加号，填写对应的参数名称和对应的值，选择相应的参数值类型，点击确定，工作流会将启动参数加入全局参数中。
## 任务样例

BIN
docs/img/new_ui/dev/parameter/context-subprocess02.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 137 KiB

After

Width:  |  Height:  |  Size: 327 KiB

BIN
docs/img/new_ui/dev/parameter/context_parameter04.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 166 KiB

After

Width:  |  Height:  |  Size: 338 KiB

BIN
docs/img/new_ui/dev/parameter/global_parameter02.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 112 KiB

After

Width:  |  Height:  |  Size: 326 KiB

BIN
docs/img/new_ui/dev/parameter/project_parameter01.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 3.1 MiB

After

Width:  |  Height:  |  Size: 118 KiB

BIN
docs/img/new_ui/dev/parameter/startup_parameter02.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 346 KiB

After

Width:  |  Height:  |  Size: 196 KiB

BIN
docs/img/new_ui/dev/parameter/startup_parameter04.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 347 KiB

After

Width:  |  Height:  |  Size: 195 KiB

18
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java
View File

@ -44,6 +44,8 @@ import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.extract.master.dto.WorkflowExecuteDto;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import org.apache.dolphinscheduler.plugin.task.api.utils.PropertyUtils;
import org.apache.commons.lang3.StringUtils;
@ -163,10 +165,8 @@ public class ExecutorController extends BaseController {
if (timeout == null) {
timeout = Constants.MAX_TASK_TIMEOUT;
}
Map<String, String> startParamMap = null;
if (startParams != null) {
startParamMap = JSONUtils.toMap(startParams);
}
List<Property> startParamList = PropertyUtils.startParamsTransformPropertyList(startParams);
if (complementDependentMode == null) {
complementDependentMode = ComplementDependentMode.OFF_MODE;
@ -175,7 +175,7 @@ public class ExecutorController extends BaseController {
Map<String, Object> result = execService.execProcessInstance(loginUser, projectCode, processDefinitionCode,
scheduleTime, execType, failureStrategy,
startNodeList, taskDependType, warningType, warningGroupId, runMode, processInstancePriority,
workerGroup, tenantCode, environmentCode, timeout, startParamMap, expectedParallelismNumber, dryRun,
workerGroup, tenantCode, environmentCode, timeout, startParamList, expectedParallelismNumber, dryRun,
testFlag,
complementDependentMode, version, allLevelDependent, executionOrder);
return returnDataList(result);
@ -262,10 +262,7 @@ public class ExecutorController extends BaseController {
timeout = Constants.MAX_TASK_TIMEOUT;
}
Map<String, String> startParamMap = null;
if (startParams != null) {
startParamMap = JSONUtils.toMap(startParams);
}
List<Property> startParamList = PropertyUtils.startParamsTransformPropertyList(startParams);
if (complementDependentMode == null) {
log.debug("Parameter complementDependentMode set to {} due to null.", ComplementDependentMode.OFF_MODE);
@ -283,7 +280,8 @@ public class ExecutorController extends BaseController {
result = execService.execProcessInstance(loginUser, projectCode, processDefinitionCode, scheduleTime,
execType, failureStrategy,
startNodeList, taskDependType, warningType, warningGroupId, runMode, processInstancePriority,
workerGroup, tenantCode, environmentCode, timeout, startParamMap, expectedParallelismNumber, dryRun,
workerGroup, tenantCode, environmentCode, timeout, startParamList, expectedParallelismNumber,
dryRun,
testFlag,
complementDependentMode, null, allLevelDependent, executionOrder);

17
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProjectParameterController.java
View File

@ -61,7 +61,8 @@ public class ProjectParameterController extends BaseController {
@Operation(summary = "createProjectParameter", description = "CREATE_PROJECT_PARAMETER_NOTES")
@Parameters({
@Parameter(name = "projectParameterName", description = "PROJECT_PARAMETER_NAME", schema = @Schema(implementation = String.class)),
@Parameter(name = "projectParameterValue", description = "PROJECT_PARAMETER_VALUE", schema = @Schema(implementation = String.class))
@Parameter(name = "projectParameterValue", description = "PROJECT_PARAMETER_VALUE", schema = @Schema(implementation = String.class)),
@Parameter(name = "projectParameterDataType", description = "PROJECT_PARAMETER_DATA_TYPE", schema = @Schema(implementation = String.class))
})
@PostMapping()
@ResponseStatus(HttpStatus.CREATED)
@ -69,9 +70,10 @@ public class ProjectParameterController extends BaseController {
public Result createProjectParameter(@Parameter(hidden = true) @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@Parameter(name = "projectCode", description = "PROJECT_CODE", required = true) @PathVariable long projectCode,
@RequestParam("projectParameterName") String projectParameterName,
@RequestParam(value = "projectParameterValue") String projectParameterValue) {
@RequestParam(value = "projectParameterValue") String projectParameterValue,
@RequestParam(value = "projectParameterDataType", defaultValue = "VARCHAR") String projectParameterDataType) {
return projectParameterService.createProjectParameter(loginUser, projectCode, projectParameterName,
projectParameterValue);
projectParameterValue, projectParameterDataType);
}
@Operation(summary = "updateProjectParameter", description = "UPDATE_PROJECT_PARAMETER_NOTES")
@ -79,6 +81,7 @@ public class ProjectParameterController extends BaseController {
@Parameter(name = "code", description = "PROJECT_PARAMETER_CODE", schema = @Schema(implementation = long.class, example = "123456")),
@Parameter(name = "projectParameterName", description = "PROJECT_PARAMETER_NAME", schema = @Schema(implementation = String.class)),
@Parameter(name = "projectParameterValue", description = "PROJECT_PARAMETER_VALUE", schema = @Schema(implementation = String.class)),
@Parameter(name = "projectParameterDataType", description = "PROJECT_PARAMETER_DATA_TYPE", schema = @Schema(implementation = String.class))
})
@PutMapping(value = "/{code}")
@ResponseStatus(HttpStatus.OK)
@ -87,9 +90,10 @@ public class ProjectParameterController extends BaseController {
@Parameter(name = "projectCode", description = "PROJECT_CODE", required = true) @PathVariable long projectCode,
@PathVariable("code") Long code,
@RequestParam("projectParameterName") String projectParameterName,
@RequestParam(value = "projectParameterValue") String projectParameterValue) {
@RequestParam(value = "projectParameterValue") String projectParameterValue,
@RequestParam(value = "projectParameterDataType") String projectParameterDataType) {
return projectParameterService.updateProjectParameter(loginUser, projectCode, code, projectParameterName,
projectParameterValue);
projectParameterValue, projectParameterDataType);
}
@Operation(summary = "deleteProjectParametersByCode", description = "DELETE_PROJECT_PARAMETER_NOTES")
@ -133,13 +137,14 @@ public class ProjectParameterController extends BaseController {
@Parameter(hidden = true) @RequestAttribute(value = Constants.SESSION_USER) User loginUser,
@Parameter(name = "projectCode", description = "PROJECT_CODE", required = true) @PathVariable long projectCode,
@RequestParam(value = "searchVal", required = false) String searchVal,
@RequestParam(value = "projectParameterDataType", required = false) String projectParameterDataType,
@RequestParam("pageNo") Integer pageNo,
@RequestParam("pageSize") Integer pageSize) {
checkPageParams(pageNo, pageSize);
searchVal = ParameterUtils.handleEscapes(searchVal);
return projectParameterService.queryProjectParameterListPaging(loginUser, projectCode, pageSize, pageNo,
searchVal);
searchVal, projectParameterDataType);
}
@Operation(summary = "queryProjectParameterByCode", description = "QUERY_PROJECT_PARAMETER_NOTES")

6
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ExecutorService.java
View File

@ -30,7 +30,9 @@ import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.dao.entity.ProcessDefinition;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.extract.master.dto.WorkflowExecuteDto;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import java.util.List;
import java.util.Map;
/**
@ -57,7 +59,7 @@ public interface ExecutorService {
* @param environmentCode environment code
* @param runMode run mode
* @param timeout timeout
* @param startParams the global param values which pass to new process instance
* @param startParamList the global param values which pass to new process instance
* @param expectedParallelismNumber the expected parallelism number when execute complement in parallel mode
* @param executionOrder the execution order when complementing data
* @return execute process instance code
@ -71,7 +73,7 @@ public interface ExecutorService {
Priority processInstancePriority, String workerGroup, String tenantCode,
Long environmentCode,
Integer timeout,
Map<String, String> startParams, Integer expectedParallelismNumber,
List<Property> startParamList, Integer expectedParallelismNumber,
int dryRun, int testFlag,
ComplementDependentMode complementDependentMode, Integer version,
boolean allLevelDependent, ExecutionOrder executionOrder);

6
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/ProjectParameterService.java
View File

@ -23,17 +23,17 @@ import org.apache.dolphinscheduler.dao.entity.User;
public interface ProjectParameterService {
Result createProjectParameter(User loginUser, long projectCode, String projectParameterName,
String projectParameterValue);
String projectParameterValue, String projectParameterDataType);
Result updateProjectParameter(User loginUser, long projectCode, long code, String projectParameterName,
String projectParameterValue);
String projectParameterValue, String projectParameterDataType);
Result deleteProjectParametersByCode(User loginUser, long projectCode, long code);
Result batchDeleteProjectParametersByCodes(User loginUser, long projectCode, String codes);
Result queryProjectParameterListPaging(User loginUser, long projectCode, Integer pageSize, Integer pageNo,
String searchVal);
String searchVal, String projectParameterDataType);
Result queryProjectParameterByCode(User loginUser, long projectCode, long code);
}

13
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java
View File

@ -90,6 +90,7 @@ import org.apache.dolphinscheduler.extract.master.transportor.StreamingTaskTrigg
import org.apache.dolphinscheduler.extract.master.transportor.StreamingTaskTriggerResponse;
import org.apache.dolphinscheduler.extract.master.transportor.WorkflowInstanceStateChangeEvent;
import org.apache.dolphinscheduler.plugin.task.api.TaskConstants;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import org.apache.dolphinscheduler.registry.api.enums.RegistryNodeType;
import org.apache.dolphinscheduler.service.command.CommandService;
import org.apache.dolphinscheduler.service.cron.CronUtils;
@ -204,7 +205,7 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
* @param environmentCode environment code
* @param runMode run mode
* @param timeout timeout
* @param startParams the global param values which pass to new process instance
* @param startParamList the global param values which pass to new process instance
* @param expectedParallelismNumber the expected parallelism number when execute complement in parallel mode
* @param testFlag testFlag
* @param executionOrder the execution order when complementing data
@ -220,7 +221,7 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
Priority processInstancePriority, String workerGroup,
String tenantCode,
Long environmentCode, Integer timeout,
Map<String, String> startParams, Integer expectedParallelismNumber,
List<Property> startParamList, Integer expectedParallelismNumber,
int dryRun, int testFlag,
ComplementDependentMode complementDependentMode, Integer version,
boolean allLevelDependent, ExecutionOrder executionOrder) {
@ -270,7 +271,7 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
startNodeList,
cronTime, warningType, loginUser.getId(), warningGroupId, runMode, processInstancePriority,
workerGroup, tenantCode,
environmentCode, startParams, expectedParallelismNumber, dryRun, testFlag,
environmentCode, startParamList, expectedParallelismNumber, dryRun, testFlag,
complementDependentMode, allLevelDependent, executionOrder);
if (create > 0) {
@ -732,7 +733,7 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
WarningType warningType, int executorId, Integer warningGroupId, RunMode runMode,
Priority processInstancePriority, String workerGroup, String tenantCode,
Long environmentCode,
Map<String, String> startParams, Integer expectedParallelismNumber, int dryRun,
List<Property> startParamList, Integer expectedParallelismNumber, int dryRun,
int testFlag, ComplementDependentMode complementDependentMode,
boolean allLevelDependent, ExecutionOrder executionOrder) {
@ -761,8 +762,8 @@ public class ExecutorServiceImpl extends BaseServiceImpl implements ExecutorServ
if (warningType != null) {
command.setWarningType(warningType);
}
if (startParams != null && startParams.size() > 0) {
cmdParam.put(CMD_PARAM_START_PARAMS, JSONUtils.toJsonString(startParams));
if (CollectionUtils.isNotEmpty(startParamList)) {
cmdParam.put(CMD_PARAM_START_PARAMS, JSONUtils.toJsonString(startParamList));
}
command.setCommandParam(JSONUtils.toJsonString(cmdParam));
command.setExecutorId(executorId);

11
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProjectParameterServiceImpl.java
View File

@ -68,7 +68,7 @@ public class ProjectParameterServiceImpl extends BaseServiceImpl implements Proj
@Override
@Transactional
public Result createProjectParameter(User loginUser, long projectCode, String projectParameterName,
String projectParameterValue) {
String projectParameterValue, String projectParameterDataType) {
Result result = new Result();
// check if user have write perm for project
@ -97,6 +97,7 @@ public class ProjectParameterServiceImpl extends BaseServiceImpl implements Proj
.builder()
.paramName(projectParameterName)
.paramValue(projectParameterValue)
.paramDataType(projectParameterDataType)
.code(CodeGenerateUtils.genCode())
.projectCode(projectCode)
.userId(loginUser.getId())
@ -122,7 +123,7 @@ public class ProjectParameterServiceImpl extends BaseServiceImpl implements Proj
@Override
public Result updateProjectParameter(User loginUser, long projectCode, long code, String projectParameterName,
String projectParameterValue) {
String projectParameterValue, String projectParameterDataType) {
Result result = new Result();
// check if user have write perm for project
@ -155,6 +156,7 @@ public class ProjectParameterServiceImpl extends BaseServiceImpl implements Proj
projectParameter.setParamName(projectParameterName);
projectParameter.setParamValue(projectParameterValue);
projectParameter.setParamDataType(projectParameterDataType);
projectParameter.setUpdateTime(new Date());
projectParameter.setOperator(loginUser.getId());
@ -236,7 +238,7 @@ public class ProjectParameterServiceImpl extends BaseServiceImpl implements Proj
@Override
public Result queryProjectParameterListPaging(User loginUser, long projectCode, Integer pageSize, Integer pageNo,
String searchVal) {
String searchVal, String projectParameterDataType) {
Result result = new Result();
Project project = projectMapper.queryByCode(projectCode);
@ -249,7 +251,8 @@ public class ProjectParameterServiceImpl extends BaseServiceImpl implements Proj
Page<ProjectParameter> page = new Page<>(pageNo, pageSize);
IPage<ProjectParameter> iPage =
projectParameterMapper.queryProjectParameterListPaging(page, projectCode, null, searchVal);
projectParameterMapper.queryProjectParameterListPaging(page, projectCode, null, searchVal,
projectParameterDataType);
List<ProjectParameter> projectParameterList = iPage.getRecords();

8
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ExecuteFunctionControllerTest.java
View File

@ -38,8 +38,13 @@ import org.apache.dolphinscheduler.common.enums.RunMode;
import org.apache.dolphinscheduler.common.enums.TaskDependType;
import org.apache.dolphinscheduler.common.enums.WarningType;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.plugin.task.api.enums.DataType;
import org.apache.dolphinscheduler.plugin.task.api.enums.Direct;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
@ -75,7 +80,8 @@ public class ExecuteFunctionControllerTest extends AbstractControllerTest {
final String tenantCode = "root";
final Long environmentCode = 4L;
final Integer timeout = 5;
final ImmutableMap<String, String> startParams = ImmutableMap.of("start", "params");
final List<Property> startParams =
Collections.singletonList(new Property("start", Direct.IN, DataType.VARCHAR, "params"));
final Integer expectedParallelismNumber = 6;
final int dryRun = 7;
final int testFlag = 0;

16
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/controller/ProjectParameterControllerTest.java
View File

@ -22,6 +22,7 @@ import org.apache.dolphinscheduler.api.service.impl.ProjectParameterServiceImpl;
import org.apache.dolphinscheduler.api.utils.Result;
import org.apache.dolphinscheduler.common.enums.UserType;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.plugin.task.api.enums.DataType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
@ -48,8 +49,9 @@ public class ProjectParameterControllerTest {
User loginUser = getGeneralUser();
Mockito.when(projectParameterService.createProjectParameter(Mockito.any(), Mockito.anyLong(), Mockito.any(),
Mockito.any())).thenReturn(getSuccessResult());
Result result = projectParameterController.createProjectParameter(loginUser, 1, "key", "value");
Mockito.any(), Mockito.any())).thenReturn(getSuccessResult());
Result result = projectParameterController.createProjectParameter(loginUser, 1, "key", "value",
DataType.VARCHAR.name());
Assertions.assertEquals(Status.SUCCESS.getCode(), result.getCode());
}
@ -58,8 +60,9 @@ public class ProjectParameterControllerTest {
User loginUser = getGeneralUser();
Mockito.when(projectParameterService.updateProjectParameter(Mockito.any(), Mockito.anyLong(), Mockito.anyLong(),
Mockito.any(), Mockito.any())).thenReturn(getSuccessResult());
Result result = projectParameterController.updateProjectParameter(loginUser, 1, 1L, "key", "value");
Mockito.any(), Mockito.any(), Mockito.any())).thenReturn(getSuccessResult());
Result result = projectParameterController.updateProjectParameter(loginUser, 1, 1L, "key", "value",
DataType.LONG.name());
Assertions.assertEquals(Status.SUCCESS.getCode(), result.getCode());
}
@ -88,8 +91,9 @@ public class ProjectParameterControllerTest {
User loginUser = getGeneralUser();
Mockito.when(projectParameterService.queryProjectParameterListPaging(Mockito.any(), Mockito.anyLong(),
Mockito.anyInt(), Mockito.anyInt(), Mockito.any())).thenReturn(getSuccessResult());
Result result = projectParameterController.queryProjectParameterListPaging(loginUser, 1, "1", 1, 10);
Mockito.anyInt(), Mockito.anyInt(), Mockito.any(), Mockito.any())).thenReturn(getSuccessResult());
Result result = projectParameterController.queryProjectParameterListPaging(loginUser, 1, "1",
DataType.VARCHAR.name(), 1, 10);
Assertions.assertEquals(Status.SUCCESS.getCode(), result.getCode());
}

40
dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/ProjectParameterServiceTest.java
View File

@ -36,6 +36,7 @@ import org.apache.dolphinscheduler.dao.entity.ProjectParameter;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.dao.mapper.ProjectMapper;
import org.apache.dolphinscheduler.dao.mapper.ProjectParameterMapper;
import org.apache.dolphinscheduler.plugin.task.api.enums.DataType;
import java.util.Collections;
@ -76,7 +77,8 @@ public class ProjectParameterServiceTest {
// PERMISSION DENIED
when(projectService.hasProjectAndWritePerm(Mockito.any(), Mockito.any(), Mockito.any(Result.class)))
.thenReturn(false);
Result result = projectParameterService.createProjectParameter(loginUser, projectCode, "key", "value");
Result result = projectParameterService.createProjectParameter(loginUser, projectCode, "key", "value",
DataType.VARCHAR.name());
assertNull(result.getData());
assertNull(result.getCode());
assertNull(result.getMsg());
@ -88,25 +90,29 @@ public class ProjectParameterServiceTest {
try (MockedStatic<CodeGenerateUtils> ignored = Mockito.mockStatic(CodeGenerateUtils.class)) {
when(CodeGenerateUtils.genCode()).thenThrow(CodeGenerateUtils.CodeGenerateException.class);
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key", "value");
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key", "value",
DataType.VARCHAR.name());
assertEquals(Status.CREATE_PROJECT_PARAMETER_ERROR.getCode(), result.getCode());
}
// PROJECT_PARAMETER_ALREADY_EXISTS
when(projectMapper.queryByCode(projectCode)).thenReturn(getProject(projectCode));
when(projectParameterMapper.selectOne(Mockito.any())).thenReturn(getProjectParameter());
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key", "value");
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key", "value",
DataType.VARCHAR.name());
assertEquals(Status.PROJECT_PARAMETER_ALREADY_EXISTS.getCode(), result.getCode());
// INSERT DATA ERROR
when(projectParameterMapper.selectOne(Mockito.any())).thenReturn(null);
when(projectParameterMapper.insert(Mockito.any())).thenReturn(-1);
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key1", "value");
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key1", "value",
DataType.VARCHAR.name());
assertEquals(Status.CREATE_PROJECT_PARAMETER_ERROR.getCode(), result.getCode());
// SUCCESS
when(projectParameterMapper.insert(Mockito.any())).thenReturn(1);
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key1", "value");
result = projectParameterService.createProjectParameter(loginUser, projectCode, "key1", "value",
DataType.VARCHAR.name());
assertEquals(Status.SUCCESS.getCode(), result.getCode());
}
@ -117,7 +123,8 @@ public class ProjectParameterServiceTest {
// NO PERMISSION
when(projectService.hasProjectAndWritePerm(Mockito.any(), Mockito.any(), Mockito.any(Result.class)))
.thenReturn(false);
Result result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key", "value");
Result result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key", "value",
DataType.VARCHAR.name());
assertNull(result.getData());
assertNull(result.getCode());
assertNull(result.getMsg());
@ -127,28 +134,33 @@ public class ProjectParameterServiceTest {
when(projectService.hasProjectAndWritePerm(Mockito.any(), Mockito.any(), Mockito.any(Result.class)))
.thenReturn(true);
when(projectParameterMapper.queryByCode(Mockito.anyLong())).thenReturn(null);
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key", "value");
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key", "value",
DataType.VARCHAR.name());
assertEquals(Status.PROJECT_PARAMETER_NOT_EXISTS.getCode(), result.getCode());
// PROJECT_PARAMETER_ALREADY_EXISTS
when(projectParameterMapper.queryByCode(Mockito.anyLong())).thenReturn(getProjectParameter());
when(projectParameterMapper.selectOne(Mockito.any())).thenReturn(getProjectParameter());
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key", "value");
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key", "value",
DataType.VARCHAR.name());
assertEquals(Status.PROJECT_PARAMETER_ALREADY_EXISTS.getCode(), result.getCode());
// PROJECT_UPDATE_ERROR
when(projectParameterMapper.selectOne(Mockito.any())).thenReturn(null);
when(projectParameterMapper.updateById(Mockito.any())).thenReturn(-1);
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key1", "value");
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key1", "value",
DataType.VARCHAR.name());
assertEquals(Status.UPDATE_PROJECT_PARAMETER_ERROR.getCode(), result.getCode());
// SUCCESS
when(projectParameterMapper.updateById(Mockito.any())).thenReturn(1);
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key1", "value");
result = projectParameterService.updateProjectParameter(loginUser, projectCode, 1, "key1", "value",
DataType.LONG.name());
assertEquals(Status.SUCCESS.getCode(), result.getCode());
ProjectParameter projectParameter = (ProjectParameter) result.getData();
assertNotNull(projectParameter.getOperator());
assertNotNull(projectParameter.getUpdateTime());
assertEquals(DataType.LONG.name(), projectParameter.getParamDataType());
}
@Test
@ -223,7 +235,8 @@ public class ProjectParameterServiceTest {
.thenReturn(false);
Result result =
projectParameterService.queryProjectParameterListPaging(loginUser, projectCode, pageSize, pageNo, null);
projectParameterService.queryProjectParameterListPaging(loginUser, projectCode, pageSize, pageNo, null,
DataType.VARCHAR.name());
assertNull(result.getData());
assertNull(result.getCode());
assertNull(result.getMsg());
@ -235,9 +248,10 @@ public class ProjectParameterServiceTest {
Page<ProjectParameter> page = new Page<>(pageNo, pageSize);
page.setRecords(Collections.singletonList(getProjectParameter()));
when(projectParameterMapper.queryProjectParameterListPaging(any(), anyLong(), any(), any())).thenReturn(page);
when(projectParameterMapper.queryProjectParameterListPaging(any(), anyLong(), any(), any(), any()))
.thenReturn(page);
result = projectParameterService.queryProjectParameterListPaging(loginUser, projectCode, pageSize, pageNo,
null);
null, null);
assertEquals(Status.SUCCESS.getCode(), result.getCode());
}

3
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/ProjectParameter.java
View File

@ -55,6 +55,9 @@ public class ProjectParameter {
@TableField("param_value")
private String paramValue;
@TableField("param_data_type")
private String paramDataType;
private Date createTime;
private Date updateTime;

3
dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ProjectParameterMapper.java
View File

@ -38,7 +38,8 @@ public interface ProjectParameterMapper extends BaseMapper<ProjectParameter> {
IPage<ProjectParameter> queryProjectParameterListPaging(IPage<ProjectParameter> page,
@Param("projectCode") long projectCode,
@Param("projectParameterIds") List<Integer> projectParameterIds,
@Param("searchName") String searchName);
@Param("searchName") String searchName,
@Param("projectParameterDataType") String projectParameterDataType);
List<ProjectParameter> queryByProjectCode(@Param("projectCode") long projectCode);
}

7
dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ProjectParameterMapper.xml
View File

@ -19,7 +19,7 @@
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="org.apache.dolphinscheduler.dao.mapper.ProjectParameterMapper">
<sql id="baseSql">
id, param_name, param_value, code, project_code, user_id, operator, create_time, update_time
id, param_name, param_value, param_data_type, code, project_code, user_id, operator, create_time, update_time
</sql>
<select id="queryByCode" resultType="org.apache.dolphinscheduler.dao.entity.ProjectParameter">
@ -51,7 +51,7 @@
<select id="queryProjectParameterListPaging" resultType="org.apache.dolphinscheduler.dao.entity.ProjectParameter">
select
pp.id, param_name, param_value, code, project_code, user_id, operator, pp.create_time, pp.update_time,
pp.id, param_name, param_value, param_data_type, code, project_code, user_id, operator, pp.create_time, pp.update_time,
u.user_name as create_user,
u2.user_name as modify_user
from t_ds_project_parameter pp
@ -69,6 +69,9 @@
OR param_value LIKE concat('%', #{searchName}, '%')
)
</if>
<if test="projectParameterDataType!=null and projectParameterDataType != ''">
AND param_data_type = #{projectParameterDataType}
</if>
order by pp.update_time desc
</select>

1
dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_h2.sql
View File

@ -667,6 +667,7 @@ CREATE TABLE t_ds_project_parameter
id int(11) NOT NULL AUTO_INCREMENT,
param_name varchar(255) NOT NULL,
param_value text NOT NULL,
param_data_type varchar(50) DEFAULT 'VARCHAR',
code bigint(20) NOT NULL,
project_code bigint(20) NOT NULL,
user_id int(11) DEFAULT NULL,

1
dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
View File

@ -668,6 +668,7 @@ CREATE TABLE `t_ds_project_parameter` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'key',
`param_name` varchar(255) NOT NULL COMMENT 'project parameter name',
`param_value` text NOT NULL COMMENT 'project parameter value',
`param_data_type` varchar(50) DEFAULT 'VARCHAR' COMMENT 'project parameter data type',
`code` bigint(20) NOT NULL COMMENT 'encoding',
`project_code` bigint(20) NOT NULL COMMENT 'project code',
`user_id` int(11) DEFAULT NULL COMMENT 'creator id',

1
dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql
View File

@ -595,6 +595,7 @@ CREATE TABLE t_ds_project_parameter (
id int NOT NULL ,
param_name varchar(255) NOT NULL ,
param_value text NOT NULL ,
param_data_type varchar(50) DEFAULT 'VARCHAR',
code bigint NOT NULL,
project_code bigint NOT NULL,
user_id int DEFAULT NULL ,

4
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.2.2_schema/mysql/dolphinscheduler_ddl.sql
View File

@ -53,4 +53,6 @@ END;
d//
delimiter ;
CALL modify_data_t_ds_audit_log_input_entry;
DROP PROCEDURE modify_data_t_ds_audit_log_input_entry;
DROP PROCEDURE modify_data_t_ds_audit_log_input_entry;
ALTER TABLE t_ds_project_parameter ADD `param_data_type` varchar (50) DEFAULT 'VARCHAR' COMMENT 'project parameter data type';

4
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.2.2_schema/postgresql/dolphinscheduler_ddl.sql
View File

@ -56,4 +56,6 @@ $$ LANGUAGE plpgsql;
d//
select modify_data_t_ds_audit_log_input_entry();
DROP FUNCTION IF EXISTS modify_data_t_ds_audit_log_input_entry();
DROP FUNCTION IF EXISTS modify_data_t_ds_audit_log_input_entry();
ALTER TABLE t_ds_project_parameter ADD COLUMN IF NOT EXISTS param_data_type varchar(50) DEFAULT 'VARCHAR';

2
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/ProjectParameterMapperTest.java
View File

@ -89,7 +89,7 @@ public class ProjectParameterMapperTest extends BaseDaoTest {
insertOne(2, "name2", 2);
Page<ProjectParameter> page = new Page(1, 3);
IPage<ProjectParameter> res = projectParameterMapper.queryProjectParameterListPaging(page, 1, null, null);
IPage<ProjectParameter> res = projectParameterMapper.queryProjectParameterListPaging(page, 1, null, null, null);
Assertions.assertEquals(1, res.getRecords().size());
}
}

23
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringParamsServiceImpl.java
View File

@ -45,7 +45,9 @@ import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
import org.apache.dolphinscheduler.plugin.task.api.utils.MapUtils;
import org.apache.dolphinscheduler.plugin.task.api.utils.ParameterUtils;
import org.apache.dolphinscheduler.plugin.task.api.utils.PropertyUtils;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import java.util.Date;
@ -54,6 +56,7 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
@ -150,9 +153,11 @@ public class CuringParamsServiceImpl implements CuringParamsService {
return new HashMap<>();
}
String startParamJson = cmdParam.get(CommandKeyConstants.CMD_PARAM_START_PARAMS);
Map<String, String> startParamMap = JSONUtils.toMap(startParamJson);
return startParamMap.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey,
entry -> new Property(entry.getKey(), Direct.IN, DataType.VARCHAR, entry.getValue())));
List<Property> propertyList = PropertyUtils.startParamsTransformPropertyList(startParamJson);
if (CollectionUtils.isEmpty(propertyList)) {
return new HashMap<>();
}
return propertyList.stream().collect(Collectors.toMap(Property::getProp, Function.identity()));
}
@Override
@ -181,8 +186,7 @@ public class CuringParamsServiceImpl implements CuringParamsService {
Map<String, Property> prepareParamsMap = new HashMap<>();
// assign value to definedParams here
Map<String, String> globalParamsMap = setGlobalParamsMap(processInstance);
Map<String, Property> globalParams = ParameterUtils.getUserDefParamsMap(globalParamsMap);
Map<String, Property> globalParams = setGlobalParamsMap(processInstance);
// combining local and global parameters
Map<String, Property> localParams = parameters.getInputLocalParametersMap();
@ -287,15 +291,16 @@ public class CuringParamsServiceImpl implements CuringParamsService {
Long.toString(taskInstance.getProcessInstance().getProcessDefinition().getProjectCode()));
return params;
}
private Map<String, String> setGlobalParamsMap(ProcessInstance processInstance) {
Map<String, String> globalParamsMap = new HashMap<>(16);
private Map<String, Property> setGlobalParamsMap(ProcessInstance processInstance) {
Map<String, Property> globalParamsMap = new HashMap<>(16);
// global params string
String globalParamsStr = processInstance.getGlobalParams();
if (globalParamsStr != null) {
List<Property> globalParamsList = JSONUtils.toList(globalParamsStr, Property.class);
globalParamsMap
.putAll(globalParamsList.stream().collect(Collectors.toMap(Property::getProp, Property::getValue)));
.putAll(globalParamsList.stream()
.collect(Collectors.toMap(Property::getProp, Function.identity())));
}
return globalParamsMap;
}
@ -323,7 +328,7 @@ public class CuringParamsServiceImpl implements CuringParamsService {
projectParameterList.forEach(projectParameter -> {
Property property = new Property(projectParameter.getParamName(),
Direct.IN,
DataType.VARCHAR,
Enum.valueOf(DataType.class, projectParameter.getParamDataType()),
projectParameter.getParamValue());
result.put(projectParameter.getParamName(), property);
});

64
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/utils/PropertyUtils.java Normal file
View File

@ -0,0 +1,64 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.task.api.utils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.plugin.task.api.enums.DataType;
import org.apache.dolphinscheduler.plugin.task.api.enums.Direct;
import org.apache.dolphinscheduler.plugin.task.api.model.Property;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
/**
* property utils
*/
public class PropertyUtils {
private PropertyUtils() {
throw new IllegalStateException("PropertyUtils class");
}
/**
* startParams transform propertyList
*
* @param startParams startParams
* @return startParamList
*/
public static List<Property> startParamsTransformPropertyList(String startParams) {
List<Property> startParamList = null;
if (startParams != null) {
JsonElement jsonElement = JsonParser.parseString(startParams);
boolean isJson = jsonElement.isJsonObject();
if (isJson) {
Map<String, String> startParamMap = JSONUtils.toMap(startParams);
startParamList = startParamMap.entrySet().stream()
.map(entry -> new Property(entry.getKey(), Direct.IN, DataType.VARCHAR, entry.getValue()))
.collect(Collectors.toList());
} else {
startParamList = JSONUtils.toList(startParams, Property.class);
}
}
return startParamList;
}
}

2
dolphinscheduler-ui/src/locales/en_US/project.ts
View File

@ -930,12 +930,14 @@ export default {
code: 'Parameter Code',
name: 'Parameter Name',
value: 'Parameter Value',
data_type: 'Parameter Data Type',
create_user: 'Create User',
modify_user: 'Modify User',
create_time: 'Create Time',
update_time: 'Update Time',
name_tips: 'Please enter your parameter name',
value_tips: 'Please enter your parameter value',
data_type_tips: 'Please choose your parameter data type',
operation: 'Operation',
edit: 'Edit',
delete: 'Delete',

2
dolphinscheduler-ui/src/locales/zh_CN/project.ts
View File

@ -900,12 +900,14 @@ export default {
code: '参数编码',
name: '参数名称',
value: '参数值',
data_type: '参数类型',
create_user: '创建用户',
modify_user: '修改用户',
create_time: '创建时间',
update_time: '更新时间',
name_tips: '请输入参数名称',
value_tips: '请输入参数值',
data_type_tips: '请选择参数类型',
operation: '操作',
edit: '编辑',
delete: '删除',

3
dolphinscheduler-ui/src/service/modules/projects-parameter/types.ts
View File

@ -19,6 +19,7 @@ interface ListReq {
pageNo: number
pageSize: number
searchVal?: string
projectParameterDataType?: string
}
interface ProjectParameterCodeReq {
@ -28,6 +29,7 @@ interface ProjectParameterCodeReq {
interface ProjectParameterReq {
projectParameterName: string
projectParameterValue: string
projectParameterDataType: string
}
interface UpdateProjectParameterReq extends ProjectParameterReq {
@ -39,6 +41,7 @@ interface ProjectParameterList {
code: number
name: string
value: string
data_type: string
createTime: string
updateTime: string
}

20
dolphinscheduler-ui/src/views/projects/parameter/components/parameter-modal.tsx
View File

@ -23,9 +23,10 @@ import {
watch
} from 'vue'
import Modal from '@/components/modal'
import { NForm, NFormItem, NInput } from 'naive-ui'
import { NForm, NFormItem, NInput, NSelect } from 'naive-ui'
import { useModal } from './use-modal'
import { useI18n } from 'vue-i18n'
import { DATA_TYPES_MAP, DEFAULT_DATA_TYPE } from "@/views/projects/parameter/data_type"
const ParameterModal = defineComponent({
name: 'ParameterModal',
@ -52,9 +53,11 @@ const ParameterModal = defineComponent({
if (props.statusRef === 0) {
variables.model.projectParameterName = ''
variables.model.projectParameterValue = ''
variables.model.projectParameterDataType = DEFAULT_DATA_TYPE
} else {
variables.model.projectParameterName = props.row.paramName
variables.model.projectParameterValue = props.row.paramValue
variables.model.projectParameterDataType = props.row.paramDataType
}
ctx.emit('cancelModal', props.showModalRef)
}
@ -78,10 +81,12 @@ const ParameterModal = defineComponent({
if (props.statusRef === 0) {
variables.model.projectParameterName = ''
variables.model.projectParameterValue = ''
variables.model.projectParameterDataType = DEFAULT_DATA_TYPE
} else {
variables.model.code = props.row.code
variables.model.projectParameterName = props.row.paramName
variables.model.projectParameterValue = props.row.paramValue
variables.model.projectParameterDataType = props.row.paramDataType
}
}
)
@ -92,6 +97,7 @@ const ParameterModal = defineComponent({
variables.model.code = props.row.code
variables.model.projectParameterName = props.row.paramName
variables.model.projectParameterValue = props.row.paramValue
variables.model.projectParameterDataType = props.row.paramDataType
}
)
@ -112,7 +118,8 @@ const ParameterModal = defineComponent({
onConfirm={this.confirmModal}
confirmDisabled={
!this.model.projectParameterName ||
!this.model.projectParameterValue
!this.model.projectParameterValue ||
!this.model.projectParameterDataType
}
confirmClassName='btn-submit'
cancelClassName='btn-cancel'
@ -135,6 +142,15 @@ const ParameterModal = defineComponent({
v-model={[this.model.projectParameterValue, 'value']}
/>
</NFormItem>
<NFormItem label={t('project.parameter.data_type')} path='data_type'>
<NSelect
placeholder={t('project.parameter.data_type_tips')}
options={Object.keys(DATA_TYPES_MAP).map((item) => {
return { value: item, label: item }
})}
v-model={[this.model.projectParameterDataType, 'value']}
/>
</NFormItem>
</NForm>
)
}}

20
dolphinscheduler-ui/src/views/projects/parameter/components/use-modal.ts
View File

@ -27,6 +27,7 @@ import {
UpdateProjectParameterReq
} from '@/service/modules/projects-parameter/types'
import { useRouter } from 'vue-router'
import { DEFAULT_DATA_TYPE } from "@/views/projects/parameter/data_type";
export function useModal(
props: any,
@ -41,7 +42,8 @@ export function useModal(
model: {
code: ref<number>(-1),
projectParameterName: ref(''),
projectParameterValue: ref('')
projectParameterValue: ref(''),
projectParameterDataType: ref(DEFAULT_DATA_TYPE)
},
saving: false,
rules: {
@ -62,6 +64,15 @@ export function useModal(
return new Error(t('project.parameter.value_tips'))
}
}
},
data_type: {
required: true,
trigger: ['input', 'blur'],
validator() {
if (variables.model.projectParameterDataType === '') {
return new Error(t('project.parameter.data_type_tips'))
}
}
}
}
})
@ -83,12 +94,14 @@ export function useModal(
const submitModal = () => {
const data: ProjectParameterReq = {
projectParameterName: variables.model.projectParameterName,
projectParameterValue: variables.model.projectParameterValue
projectParameterValue: variables.model.projectParameterValue,
projectParameterDataType: variables.model.projectParameterDataType
}
createProjectParameter(data, variables.projectCode).then(() => {
variables.model.projectParameterName = ''
variables.model.projectParameterValue = ''
variables.model.projectParameterDataType = DEFAULT_DATA_TYPE
ctx.emit('confirmModal', props.showModalRef)
})
}
@ -97,7 +110,8 @@ export function useModal(
const data: UpdateProjectParameterReq = {
code: variables.model.code,
projectParameterName: variables.model.projectParameterName,
projectParameterValue: variables.model.projectParameterValue
projectParameterValue: variables.model.projectParameterValue,
projectParameterDataType: variables.model.projectParameterDataType
}
updateProjectParameter(data, variables.projectCode).then(() => {

54
dolphinscheduler-ui/src/views/projects/parameter/data_type.ts Normal file
View File

@ -0,0 +1,54 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
export const DATA_TYPES_MAP = {
VARCHAR: {
alias: 'VARCHAR'
},
INTEGER: {
alias: 'INTEGER'
},
LONG: {
alias: 'LONG'
},
FLOAT: {
alias: 'FLOAT'
},
DOUBLE: {
alias: 'DOUBLE'
},
DATE: {
alias: 'DATE'
},
TIME: {
alias: 'TIME'
},
TIMESTAMP: {
alias: 'TIMESTAMP'
},
BOOLEAN: {
alias: 'BOOLEAN'
},
LIST: {
alias: 'LIST'
},
FILE: {
alias: 'FILE'
}
}
export const DEFAULT_DATA_TYPE = 'VARCHAR'

15
dolphinscheduler-ui/src/views/projects/parameter/index.tsx
View File

@ -21,7 +21,8 @@ import {
NInput,
NPagination,
NSpace,
NButton
NButton,
NSelect
} from 'naive-ui'
import { defineComponent, onMounted, toRefs, watch } from 'vue'
import { useI18n } from 'vue-i18n'
@ -29,6 +30,7 @@ import { useTable } from '@/views/projects/parameter/use-table'
import Card from '@/components/card'
import ParameterModal from '@/views/projects/parameter/components/parameter-modal'
import { SearchOutlined } from '@vicons/antd'
import { DATA_TYPES_MAP } from "@/views/projects/parameter/data_type"
export default defineComponent({
name: 'ProjectParameterList',
@ -40,6 +42,7 @@ export default defineComponent({
pageSize: variables.pageSize,
pageNo: variables.page,
searchVal: variables.searchVal,
projectParameterDataType: variables.projectParameterDataType,
projectCode: variables.projectCode
})
}
@ -116,6 +119,16 @@ export default defineComponent({
v-model={[this.searchVal, 'value']}
placeholder={t('project.parameter.name')}
/>
<NSelect
v-model={[this.projectParameterDataType, 'value']}
size='small'
options={Object.keys(DATA_TYPES_MAP).map((item) => {
return { value: item, label: item }
})}
placeholder={t('project.parameter.data_type_tips')}
style={{ width: '180px' }}
clearable
/>
<NButton size='small' type='primary' onClick={handleSearch}>
<NIcon>
<SearchOutlined />

9
dolphinscheduler-ui/src/views/projects/parameter/use-table.ts
View File

@ -45,6 +45,7 @@ export function useTable() {
page: ref(1),
pageSize: ref(10),
searchVal: ref(),
projectParameterDataType: ref(),
totalPage: ref(1),
showRef: ref(false),
statusRef: ref(0),
@ -69,6 +70,11 @@ export function useTable() {
key: 'paramValue',
...COLUMN_WIDTH_CONFIG['name']
},
{
title: t('project.parameter.data_type'),
key: 'paramDataType',
...COLUMN_WIDTH_CONFIG['name']
},
{
title: t('project.parameter.create_user'),
key: 'createUser',
@ -192,7 +198,8 @@ export function useTable() {
getTableData({
pageSize: variables.pageSize,
pageNo: variables.page,
searchVal: variables.searchVal
searchVal: variables.searchVal,
projectParameterDataType: variables.projectParameterDataType
})
})
}

30
dolphinscheduler-ui/src/views/projects/workflow/components/dag/dag-save-modal.tsx
View File

@ -154,7 +154,8 @@ export default defineComponent({
formValue.value.globalParams = process.globalParamList.map((param) => ({
key: param.prop,
value: param.value,
direct: param.direct
direct: param.direct,
type: param.type
}))
}
}
@ -239,6 +240,7 @@ export default defineComponent({
return {
key: '',
direct: 'IN',
type: 'VARCHAR',
value: ''
}
}}
@ -246,16 +248,16 @@ export default defineComponent({
>
{{
default: (param: {
value: { key: string; direct: string; value: string }
value: { key: string; direct: string; type: string; value: string }
}) => (
<NGrid xGap={12} cols={24}>
<NGridItem span={9}>
<NGridItem span={6}>
<NInput
v-model:value={param.value.key}
placeholder={t('project.dag.key')}
/>
</NGridItem>
<NGridItem span={6}>
<NGridItem span={5}>
<NSelect
options={[
{ value: 'IN', label: 'IN' },
@ -265,7 +267,25 @@ export default defineComponent({
defaultValue={'IN'}
/>
</NGridItem>
<NGridItem span={9}>
<NGridItem span={7}>
<NSelect
options={[
{ value: 'VARCHAR', label: 'VARCHAR' },
{ value: 'INTEGER', label: 'INTEGER' },
{ value: 'LONG', label: 'LONG' },
{ value: 'FLOAT', label: 'FLOAT' },
{ value: 'DOUBLE', label: 'DOUBLE' },
{ value: 'DATE', label: 'DATE' },
{ value: 'TIME', label: 'TIME' },
{ value: 'BOOLEAN', label: 'BOOLEAN' },
{ value: 'LIST', label: 'LIST' },
{ value: 'FILE', label: 'FILE' }
]}
v-model:value={param.value.type}
defaultValue={'VARCHAR'}
/>
</NGridItem>
<NGridItem span={6}>
<NInput
v-model:value={param.value.value}
placeholder={t('project.dag.value')}

1
dolphinscheduler-ui/src/views/projects/workflow/components/dag/types.ts
View File

@ -138,6 +138,7 @@ export interface Coordinate {
export interface GlobalParam {
key: string
direct: string
type: string
value: string
}

115
dolphinscheduler-ui/src/views/projects/workflow/definition/components/start-modal.tsx
View File

@ -34,7 +34,6 @@ import { useModal } from './use-modal'
import {
NForm,
NFormItem,
NButton,
NIcon,
NInput,
NSpace,
@ -45,13 +44,14 @@ import {
NCheckbox,
NDatePicker,
NRadioButton,
NInputNumber
NInputNumber,
NDynamicInput,
NGrid,
NGridItem
} from 'naive-ui'
import {
ArrowDownOutlined,
ArrowUpOutlined,
DeleteOutlined,
PlusCircleOutlined
ArrowUpOutlined
} from '@vicons/antd'
import { IDefinitionData } from '../types'
import styles from '../index.module.scss'
@ -580,50 +580,67 @@ export default defineComponent({
label={t('project.workflow.startup_parameter')}
path='startup_parameter'
>
{this.startParamsList.length === 0 ? (
<NButton text type='primary' onClick={this.addStartParams}>
<NIcon>
<PlusCircleOutlined />
</NIcon>
</NButton>
) : (
<NSpace vertical>
{this.startParamsList.map((item, index) => (
<NSpace class={styles.startup} key={Date.now() + index}>
<NInput
allowInput={this.trim}
pair
separator=':'
placeholder={['prop', 'value']}
defaultValue={[item.prop, item.value]}
onUpdateValue={(param) =>
this.updateParamsList(index, param)
}
/>
<NButton
text
type='error'
onClick={() => this.removeStartParams(index)}
class='btn-delete-custom-parameter'
>
<NIcon>
<DeleteOutlined />
</NIcon>
</NButton>
<NButton
text
type='primary'
onClick={this.addStartParams}
class='btn-create-custom-parameter'
>
<NIcon>
<PlusCircleOutlined />
</NIcon>
</NButton>
</NSpace>
))}
</NSpace>
)}
<NDynamicInput
v-model:value={this.startParamsList}
onCreate={() => {
return {
key: '',
direct: 'IN',
type: 'VARCHAR',
value: ''
}
}}
class='input-startup-params'
>
{{
default: (param: {
value: { prop: string; direct: string; type: string; value: string }
}) => (
<NGrid xGap={12} cols={24}>
<NGridItem span={6}>
<NInput
v-model:value={param.value.prop}
placeholder={t('project.dag.key')}
/>
</NGridItem>
<NGridItem span={5}>
<NSelect
options={[
{ value: 'IN', label: 'IN' },
{ value: 'OUT', label: 'OUT' }
]}
v-model:value={param.value.direct}
defaultValue={'IN'}
/>
</NGridItem>
<NGridItem span={7}>
<NSelect
options={[
{ value: 'VARCHAR', label: 'VARCHAR' },
{ value: 'INTEGER', label: 'INTEGER' },
{ value: 'LONG', label: 'LONG' },
{ value: 'FLOAT', label: 'FLOAT' },
{ value: 'DOUBLE', label: 'DOUBLE' },
{ value: 'DATE', label: 'DATE' },
{ value: 'TIME', label: 'TIME' },
{ value: 'BOOLEAN', label: 'BOOLEAN' },
{ value: 'LIST', label: 'LIST' },
{ value: 'FILE', label: 'FILE' }
]}
v-model:value={param.value.type}
defaultValue={'VARCHAR'}
/>
</NGridItem>
<NGridItem span={6}>
<NInput
v-model:value={param.value.value}
placeholder={t('project.dag.value')}
/>
</NGridItem>
</NGrid>
)
}}
</NDynamicInput>
</NFormItem>
<NFormItem
label={t('project.workflow.whether_dry_run')}

10
dolphinscheduler-ui/src/views/projects/workflow/definition/components/use-modal.ts
View File

@ -119,14 +119,8 @@ export function useModal(
})
}
const startParams = {} as any
for (const item of variables.startParamsList) {
if (item.value !== '') {
startParams[item.prop] = item.value
}
}
params.startParams = !_.isEmpty(startParams)
? JSON.stringify(startParams)
params.startParams = !_.isEmpty(variables.startParamsList)
? JSON.stringify(variables.startParamsList)
: ''
await startProcessInstance(params, variables.projectCode)
window.$message.success(t('project.workflow.success'))

2
dolphinscheduler-ui/src/views/projects/workflow/definition/create/index.tsx
View File

@ -59,7 +59,7 @@ export default defineComponent({
prop: p.key,
value: p.value,
direct: p.direct,
type: 'VARCHAR'
type: p.type
}
})

2
dolphinscheduler-ui/src/views/projects/workflow/definition/detail/index.tsx
View File

@ -85,7 +85,7 @@ export default defineComponent({
prop: p.key,
value: p.value,
direct: p.direct,
type: 'VARCHAR'
type: p.type
}
})

2
dolphinscheduler-ui/src/views/projects/workflow/instance/detail/index.tsx
View File

@ -82,7 +82,7 @@ export default defineComponent({
prop: p.key,
value: p.value,
direct: p.direct,
type: 'VARCHAR'
type: p.type
}
})