0fc2fe971a
[DSIP-39][parameter] Improvement startup parameters/global parameters/project parameters data type ( #15967 )
...
* [Improvement][parameter] New data types and type filtering
* [Improvement][parameter] Improvement startup parameters/global parameters data type
* fix api interfaces compatible
* add project parameter data type default value
* [Improvement][parameter] New data types and type filtering
* [Improvement][parameter] Improvement startup parameters/global parameters data type
* fix api interfaces compatible
* add project parameter data type default value
* improvement project code
* remove useless imports
* remove method onClearSearchTaskType
* add parameter doc
* optimisation logic
* code conflict resolution
* code conflict resolution
* [Improvement][Monitor] Show master && worker Busy Or Normal Status and Show Commands table list (#15978 )
* update
* test
* add monitor enhance ui
* update
* update
* update doc
* fix spotless
* update
* update
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/DataAnalysisController.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/CommandMapper.java
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* Update dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ErrorCommandMapper.xml
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* update
* fix spotless
* update
---------
Co-authored-by: Wenjun Ruan <wenjun@apache.org>
* [Improvement][Monitor] Add UT for montor (#15998 )
* formatting Code
2024-05-21 15:39:43 +08:00
f99d3f1ed3
[Improvement][Audit] Change time to create_time ( #15846 )
...
* change time to create_time
* update
* update
2024-04-15 17:01:06 +08:00
226345512e
[DSIP-26][Audit log] Audit log improvement design ( #15554 )
2024-04-15 13:40:35 +08:00
920ac154cb
[Improvement-15744][parameter] project parameter add update time and update user id ( #15745 )
...
* project parameter add update time and update user id
* project parameter add comment operator user id and UT
* project parameter add ui
2024-04-02 09:42:44 +08:00
5213290262
[Feature-14863][API] Support to manage what worker groups can be used for a project. ( #15600 )
2024-02-20 17:20:00 +08:00
89742332a8
Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time ( #15528 )
2024-02-01 17:13:58 +08:00
e20e066076
[Fix] [Bug] Change default version of Workflow/TaskDefinition to `1` ( #15498 )
2024-01-26 20:51:17 +08:00
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) ( #15355 )
2023-12-23 22:47:59 +08:00
b4cf1ccaac
[Improvement-15009][Parameter] Change project parameter value to text ( #15010 )
2023-11-30 13:56:22 +08:00
5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue ( #14992 )
...
* change data quality field name
* fix-dao
* fix-dao
* fix-dao
* fix-dao
* update
* test
* update
* update
2023-11-13 11:04:30 +08:00
8c01fcf14b
[Improvement][K8S] Remove ResourceQuota ( #14991 )
...
* [Improvement][K8S] Remove ResourceQuota
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: fix pg
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: Clean up unused code
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: naming conventions
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-11-03 15:18:50 +08:00
c0ed68121d
[Feature-14832][Listener]Implementation of Listener Mechanism ( #14981 )
...
* first commit
* 1. sql: sync ddl
2. front-end: change to ternary expression
3. back-end: correct license header in ListenerEvent.java
* test case
* frontend: remove unnecessary console
* fix unit test
* remove log depends on user-provided value
* fix dolphinscheduler_postgresql.sql
* sync database schema
* fix unit test
* fix unit test
* fix some NIT.
* extract GLOBAL_ALERT_GROUP_ID into variable
* fix ddl bug
* add column task_type in t_ds_fav_task in upgrade/3.2.0_schema
* add unit test
2023-10-29 20:28:05 +08:00
25e841a228
chore: Correct db upgrade and add ci avoid regression ( #14620 )
...
add atlasgo to ci
https://atlasgo.io/declarative/diff
related: https://github.com/apache/dolphinscheduler/issues/11286
BTW, it seems some ddl change only exist in 3.1.3、3.1.5、3.1.6 but do not exist in dev which will also cause an error when updating from those version
2023-08-30 11:55:05 +08:00
e2b97c026e
[Improvement][UI] Support to disable or enable the project preferences. ( #14756 )
2023-08-16 20:20:48 +08:00
8505f1878d
Expose swagger.properties to release ( #14590 )
2023-07-19 22:24:42 +08:00
20cf4ad4e6
[Feature-14505][UI] Support to set project-level preferences ( #14573 )
...
* add a few codes about the project preferences
2023-07-19 15:35:54 +08:00
175d976cd2
[Feature][Data Quality] Data Quality Support Choose Database ( #14406 )
...
* add dataquality database api
* change ui
* api change
* update
* fix spotless
* fix h2
* fix pg
* fix-dead-line
* update
* fix-spotless
* update pg sql
* add ut
* fix ut
2023-07-12 09:00:28 +08:00
221df4a278
revert: Datasource need test binding to create prod ( #14381 )
...
We should keep our step as less as possible, for now, we have to
test datasource and binding it to the prod datasource, I think it will
make our users do not thing to use datasource, so I do like to revert
it and keep step as less as possible
related to https://github.com/apache/dolphinscheduler/pull/11670
2023-06-27 10:59:17 +08:00
1c935d901e
[Feature-14214][Parameter] Support CRUD of project-level parameters ( #14264 )
2023-06-12 13:35:38 +08:00
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters ( #14127 )
...
* add dynamic task plugin
* fix ui
* update code
* add DOC
* ADD UT
* fix doc
* fix sanity check
* add upgrade ddl
* fix sql
2023-06-07 16:01:06 +08:00
79cb707b57
[Improvement-13932][API] Tenant parameters and Worker group parameters are put together in the task startup parameters ( #13935 )
...
* [Improvement][API] Tenant parameters and Worker group parameters are put together in the task startup parameters
2023-04-18 14:56:38 +08:00
9df89cf0cf
Use bootstrap user as the default tenant ( #13825 )
2023-03-30 14:11:23 +08:00
0bfd7d66d9
db: Modify name len to 255 ( #13788 )
2023-03-24 09:40:09 +08:00
8895921d87
[Improvement-13651] Remove useless resources_task ( #13721 )
2023-03-23 22:52:23 +08:00
133d1733eb
Alter title in t_ds_alert to 512 ( #13535 )
2023-02-09 09:41:07 +08:00
8be32d4145
[Feature][Api] When use api to run a process we want get processInstanceId ( #13184 )
...
* add sql
* add mapper
* add dao
* add excutor
Co-authored-by: qianl4 <qianl4@cicso.com>
2023-01-18 17:58:32 +08:00
8a479927f3
Add projectCode in t_ds_process_instance and t_ds_task_instance to remove join ( #13284 )
2023-01-03 09:52:28 +08:00
66e20271ad
[Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks ( #13194 )
...
* Supports task instance cache operation
* add task plugin cache
* use SHA-256 to generate key
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Update dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
* Optimizing database Scripts
* Optimize clear cache operation
Co-authored-by: Jay Chung <zhongjiajie955@gmail.com>
2022-12-18 18:17:09 +08:00
d9f43adafd
Prevent default alert group from binding alert plugin instances with id 1 and 2 automatically ( #13114 )
2022-12-06 23:10:48 +08:00
4b4d0b92a3
[FIX][DAO-12277] change t_ds_fav_task column task_name to task_type and related code ( #12276 )
...
* change t_ds_fav_task column task_name to task_type and related code
* change task_name to task_type
* change FavTask taskName to taskType, taskType to taskCategory
Co-authored-by: labbomb <739955946@qq.com>
2022-10-27 09:36:26 +08:00
489e7fe4e2
[Feature-10495][Resource Center] Resource Center Refactor ( #12076 )
...
* resource center refactor - S3 services connection
Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
2022-10-26 13:53:44 +08:00
97a0771f21
[fix][sql] Add unique key to process_definition_log avoid TooManyResultExpection ( #12503 )
...
* Add unique key to t_ds_process_definition_log
2022-10-25 14:20:37 +08:00
64f4cb4f3e
refactor: dag-menu refactor ( #11965 )
...
* refactor: dag-menu refactor
* feat: add i18n support for the dag-menu
* feat: add e2e class
Co-authored-by: devosend <devosend@gmail.com>
Co-authored-by: Tq <tianqitobethefirst@gmail.com>
2022-09-26 11:06:39 +08:00
5b384f3fab
[Feature-11473][Task]Support test task ( #11670 )
...
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* [improve] support test tasks
* Update TaskExecuteThread.java
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* try solve e2e q
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update DataSource.java
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages_zh_CN.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* Update messages.properties
* Update messages_en_US.properties
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
* [Feature] support test tasks
2022-09-17 13:40:56 +08:00
f8d46a26c1
[Feature-11530] add state history for process instance ( #11757 )
...
* [Feature] add state history for process instance (#97 )
* add state history for process instance
* upsertProcessInstance
* remove unuse method
* fix UT
Co-authored-by: caishunfeng <534328519@qq.com>
2022-09-03 10:14:05 +08:00
b3cc8a55b9
[feature][API]feature: add task type list and enable user to add task type to fav ( #11727 )
...
* feature: add task type list and enable user to add task type to fav
* make DependentExecute.java variable protected
* add standalone server fileSet
2022-09-02 11:40:23 +08:00
abfef1a929
[improvement] Add two parameters in workergroup, and support the application of description display and other parameters ( #11542 )
...
* [Improvement] Improvement default worker group
* [Improvement] Improvement default worker group
* [Improvement] Improvement default worker group
* [Improvement] Improvement default worker group
* [Improvement] Improvement default worker group
* [improvement] The default worker grouping supports editing
* [improvement] The default worker grouping supports editing
* Update WorkerGroupServiceImpl.java
* Update WorkerGroupServiceImpl.java
* The default worker grouping supports editing
* update sql file
* update
* update
* update
* update
2022-08-23 10:54:17 +08:00
2862f5b67c
Set process instance and task priority default value as 2 ( #11539 )
2022-08-18 16:21:12 +08:00
5983e41f4b
[fix] Database change not sync ( #11258 )
...
* [fix] Database change not sync
* Make del sql upgrade same as release version
* Correct data quality dml sql
* Upgrade database DDL to avoid log path too long error
* Add missing if exists
2022-08-17 10:50:34 +08:00
8217784044
[Fix-1360]: Fix Description length check ( #11371 )
...
* [Fix-1360]: Fix Description length check
* Update dolphinscheduler_h2.sql
h2 Increase the length of some fields
* Update dolphinscheduler_ddl.sql
2022-08-11 13:26:50 +08:00
0464123c2b
[Feature-11223] support stream task ( #11350 )
...
* add task execute type
* update task definition list paging
* update task instance list paging
* stream task start
* [Feature][UI] Some changes to execute task.
* Set the connection edge to dashed line.
* Add FLINK_STREAM task.
* add stream task
* flink savepoint and cancel
* fix query bug
* add stream task definition
* add task instance for stream task
* delete stream task definition state
* update api for stream task definition edit
* modify search for stream task instance
* add language
* delete task type search for stream task definition
* change task type search for stream task instance
* add jump button
* add savepoint
* add down log for stream task instance
* ui test
* stream task start
* run DAG
* [Fix][UI] Fix the stream task edgs not to be dashed when filling back.
* [Feature][UI] Remove some fields for FLINK_STREAM.
* add start modal
* add dryRun column for stream task instance
* fix duration
* fix pon
* fix build error
* Add success tip
* add auto sync for stream task instance
* remove forgien key for task instance
* license header
* UT fix
* modify locales
* recover common config
* fix UT
* add doc
Co-authored-by: Amy <amywang0104@163.com>
Co-authored-by: devosend <devosend@gmail.com>
2022-08-10 21:44:43 +08:00
1e48bbc5c6
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster ( #10650 )
...
* server code
* namespace ui
* fix dao test
* fix test
* fix
* docs and image update
Co-authored-by: qianl4 <qianl4@cicso.com>
2022-07-07 16:05:52 +08:00
b8a90d801e
[Improvement] Resource authority module database table unique index increased. ( #10807 )
...
* Resource authority module database table unique index increased.
* add unique in sql file
* rerun
* ut fix
* e2e rerun
* dao un fix
2022-07-07 14:35:50 +08:00
ff065d8e5b
[Feature][Improvement] Support multi cluster environments - k8s type ( #10096 )
...
* service code
* [Feature][UI] Add front-end for cluster manage
* fix e2e
* remove comment on cluster controller
* doc
* img
* setting e2e.yaml
* test
* rerun e2e
* fix bug from comment
* Update index.tsx
use Nspace instead of css.
* Update index.tsx
Remove the style.
* Delete index.module.scss
Remove the useless file.
Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>
2022-06-15 13:39:20 +08:00
b1fb17119b
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup ( #10373 )
2022-06-08 19:42:05 +08:00
c837580c68
[bug][db] Table relation_project_user have duplicate record ( #9536 )
...
* update sql ddl
* fix syntax err
2022-04-27 12:04:08 +08:00
165d7aa51f
[Feature][Task] K8s namespace auth manager ( #9303 )
...
* k8s auth
* remove log
* fix test
* use constants
* use constants K8S_LOCAL_TEST_CLUSTER
* simple auth get
* change test
* add namespace authorize in user page
* prettier code
* change test data
Co-authored-by: qianl4 <qianl4@cicso.com>
Co-authored-by: William Tong <weitong@cisco.com>
2022-04-20 18:23:23 +08:00
706cdb6a8e
[Fix-9174] [Alert] Fix deduplication of alarm information ( #9371 )
...
* feat(issue #9174 ):
Fix-9174
2022-04-14 10:30:25 +08:00
e2759a8f42
[Feature] [ALERT-9406]add new properties to alert class ( #9408 )
...
* add new properties to Alert.java and do minor changes to comments
* fix Integer to int
* fix Integer to int
* fix sql files
* fix not null properties to default null
2022-04-08 23:54:34 +08:00
2bab12f2c8
[Feature-9204][alert] Implement alert send status ( #9208 )
...
* [DS-9204][feat][alert,dao] Implement alert send status
- implement alert send status
- add alert send status entity、mapper
- modify alert dao
- modify alert sender
- add test
- add sql
This closes #9204
* [DS-9204][feat][alert,dao] Implement alert send status
- add license header
This closes #9204
2022-04-06 18:08:00 +08:00
小可耐