68065d3f24
fix UT test ( #15684 )
2024-03-08 17:21:23 +08:00
502b49a1cb
[TEST] fix log related test ( #15659 )
...
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
Co-authored-by: xiangzihao <460888207@qq.com>
2024-03-04 16:22:18 +08:00
88a8f06b1d
Fix can modify file which is not under resource path ( #15652 )
2024-03-01 19:39:34 +08:00
b278df1311
[Improvement][API] add PageInfo UT ( #15622 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
2024-02-26 15:24:32 +08:00
380d108133
[TEST] add task definition log service test ( #15631 )
...
Co-authored-by: abzymeinsjtu <abzymeinsjtu@B-54Q8MD6R-0244.local>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2024-02-25 21:57:56 +08:00
eb2a75b3b7
[Bug][API] list paging missing totalpage ( #15619 )
...
Signed-off-by: Gallardot <gallardot@apache.org>
2024-02-23 13:57:54 +08:00
5213290262
[Feature-14863][API] Support to manage what worker groups can be used for a project. ( #15600 )
2024-02-20 17:20:00 +08:00
718a01bb42
[Improvement][UT] Add UT for end time check when force success task ( #15562 )
2024-02-08 09:19:56 +08:00
3526b42df6
Add Server HealthIndicator ( #15274 )
2024-02-07 14:07:30 +08:00
73a5a77db2
[Fix] Fix WorkflowInstance batch start failed will throw incorrect exception. ( #15577 )
2024-02-06 13:16:25 +08:00
43a06525a2
Fix Recover WorkflowInstance will casue workflow Instance state is success but task insatnce is killed/paused ( #15574 )
2024-02-06 10:09:30 +08:00
e5a208f363
[Bug][force-success] force success add end time ( #15144 )
...
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-02-05 14:23:16 +08:00
89742332a8
Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time ( #15528 )
2024-02-01 17:13:58 +08:00
2f66a667ba
refactor comments & function name for confuse ( #15546 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-02-01 10:39:54 +08:00
25810a81dc
Exit JVM when OOM ( #15538 )
2024-01-31 13:59:30 +08:00
95ae0036f6
[improvement][api] Fix typo for controllers ( #15438 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-24 10:39:35 +08:00
308e4fb4a6
Directly Throw exception when taskInstancy log path is empty which log need to be queried ( #15511 )
2024-01-20 16:40:30 +08:00
0dc373df2a
Set TaskGroupQueue updateTime when force start ( #15510 )
2024-01-18 22:11:50 +08:00
16c97ef44d
[DS-15489][style]rename the vo object suffix ( #15504 )
...
- the naming suffix of the objects under the package dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/vo is not unified, such as AlertPluginInstanceVO and ScheduleVO, etc.
- this issue solve it, and uniformly modified to the VO suffix
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 17:13:53 +08:00
64e1e67581
Use DefaultUncaughtExceptionHandler to log the uncached exception ( #15496 )
...
Co-authored-by: fuchanghai <changhaifu@apache.org>
2024-01-18 14:59:17 +08:00
e1fcd4e485
[Bug][Api] Fix NPE when deleting a workflow instance ( #15485 ) ( #15488 )
2024-01-17 22:03:33 +08:00
cb5317f7ca
[bug] Exception when using host in ipv6 format ( #14040 )
...
* DB2 edit again, no username.
* [bug] db2's jdbcUrl should be like this "jdbc:db2://localhost:50000/sample:currentSchema=DB2INST1;characterEncoding=UTF-8;".should be ":"
* Update Db2DataSourceProcessor.java
* Update Db2DataSourceProcessorTest.java
* 【Bug】OracleSource edit again, ConnectionType is null
* when user inputs ipv6,checkhost will return "!IPV4_PATTERN.matcher(host).matches()";
I think we should use isInetAddress to check the host is isValid
* Update AbstractDataSourceProcessor.java
* Update DatabendDataSourceProcessorTest.java
last week other modified this
* Update MySQLDataSourceProcessorTest.java
* Update DataSourceUtilsTest.java
* Update MySQLDataSourceProcessorTest.java
* Update KyuubiDataSourceProcessorTest.java
* Update AbstractDataSourceProcessor.java
if host is ipv6 or ipv4 will OK; if not check other
* Update AbstractDataSourceProcessor.java
* Update AbstractDataSourceProcessor.java
* Update DataSourceUtilsTest.java
增加ipv6数据源验证
* [Bug] [dolphinscheduler-api] queryDataSource'password should be hidden.
---------
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Rick Cheng <rickchengx@gmail.com>
2024-01-15 09:33:08 +08:00
0f7081be10
[Feature] timed scheduler Improvement ( #15449 ) ( #15450 )
2024-01-11 17:25:31 +08:00
d6dea4633c
Optimize server startup log ( #15362 )
2023-12-27 20:20:21 +08:00
5b6b0ceb31
Fix resource file usage(Delete Resource/ResourceUser which is deprecated) ( #15355 )
2023-12-23 22:47:59 +08:00
43f5f24529
Add dolphinscheduler-extract-common module ( #15266 )
2023-12-05 21:06:39 +08:00
7bfc6dc3cf
[Bug-15215][Api] non-admin should not modify tenantId and queue ( #15254 )
...
* bugfix-15215：Users are not allowed to modify the default tenant and queue through the update API
* fix : #15215
2023-12-04 10:20:34 +08:00
12f8138167
Expire session when update user password ( #15219 )
2023-11-26 18:53:44 +08:00
1d23e4fe5d
Remove spring cache for dao ( #15184 )
2023-11-24 14:54:05 +08:00
9c7ad6df0a
[Improvement] Clean up Scheduler logic ( #15198 )
...
* Clean up Scheduler logic
* Update dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
Co-authored-by: 旺阳 <wang@lqwang.net>
---------
Co-authored-by: 旺阳 <wang@lqwang.net>
2023-11-24 10:53:23 +08:00
df656a7671
Fix home page workflow instance miss status ( #15193 )
2023-11-22 10:33:24 +08:00
7308888c70
fix security issue ( #15192 )
2023-11-21 11:07:13 +08:00
9be81be328
Remove API Result in Service ( #15181 )
2023-11-20 17:04:03 +08:00
6aa6e114a5
[Improvement][Alert] Add a test send feature when creating an alert instance ( #15163 )
...
* add alert test send feature
* update doc
* add alarm instance test send ui
* update
* fix mvn
* fix test
* update
* update
* change to rpc
* fix ut
* fix ut
* update
* update
* change result
* update
* Update docs/docs/en/guide/alert/alert_plugin_user_guide.md
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
---------
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-11-20 10:03:37 +08:00
5afd94136c
[Fix][Data Quality] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue ( #14992 )
...
* change data quality field name
* fix-dao
* fix-dao
* fix-dao
* fix-dao
* update
* test
* update
* update
2023-11-13 11:04:30 +08:00
8c01fcf14b
[Improvement][K8S] Remove ResourceQuota ( #14991 )
...
* [Improvement][K8S] Remove ResourceQuota
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: fix pg
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: Clean up unused code
Signed-off-by: Gallardot <gallardot@apache.org>
* chore: naming conventions
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-11-03 15:18:50 +08:00
dc503b0e00
[Fix-15072][Resource Center] Non-admin user can not query resource recursively ( #15097 )
2023-11-02 13:34:30 +08:00
c0ed68121d
[Feature-14832][Listener]Implementation of Listener Mechanism ( #14981 )
...
* first commit
* 1. sql: sync ddl
2. front-end: change to ternary expression
3. back-end: correct license header in ListenerEvent.java
* test case
* frontend: remove unnecessary console
* fix unit test
* remove log depends on user-provided value
* fix dolphinscheduler_postgresql.sql
* sync database schema
* fix unit test
* fix unit test
* fix some NIT.
* extract GLOBAL_ALERT_GROUP_ID into variable
* fix ddl bug
* add column task_type in t_ds_fav_task in upgrade/3.2.0_schema
* add unit test
2023-10-29 20:28:05 +08:00
a5284e4024
Remove mapper usage in tools ( #15073 )
2023-10-25 22:58:47 +08:00
ae847ba331
[Fix-15036] [API] Fix task definition edit doesn't work ( #14801 )
...
* [Bug] [task definition] 修复任务定义修改,工作流无法生效问题
* [Bug] [API] Fix task check apply
* [BUG][API] fix updateTaskDefinition method ut
* [BUG][API] fix updateTaskDefinition method code specification
* [BUG][API] fix one task corresponds to multiple workflow problems
* [BUG][API] queryProcessTaskRelationByTaskCodeAndTaskVersion method adjust
* [BUG][API] fix updateTaskDefinition method update
---------
Co-authored-by: David Zollo <dailidong66@gmail.com>
Co-authored-by: 旺阳 <wang@lqwang.net>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-10-25 20:25:02 +08:00
33084d281f
Add dolphinscheduler-dao-plugin module ( #15019 )
2023-10-14 17:58:40 +08:00
1f67e68400
Revert "[Bug] [Resource] fix resource delete bug ( #15003 )" ( #15020 )
...
This reverts commit d6a900667a
2023-10-12 04:28:11 -05:00
d6a900667a
[Bug] [Resource] fix resource delete bug ( #15003 )
2023-10-12 03:28:23 -05:00
8cb765bdcb
[Bug] [Api] Logback file STDOUT duplicate ( #14966 )
2023-09-26 20:19:43 +08:00
b303648752
[Improvement][Api]Change default zookeeper timeout setting ( #14940 )
...
* Local running api project connection remote zookeeper always reported connection timeout #14939 Adjust the connection zookeeper timeout time
* Local running api project connection remote zookeeper always reported connection timeout #14939 Adjust the connection zookeeper timeout time
* Local running api project connection remote zookeeper always reported connection timeout #14939 Adjust the connection zookeeper timeout time
* reported connection timeout #14939 Adjust the connection timeout time use the curator default setting
---------
Co-authored-by: AmriStrong <wanggang1@baseus.com>
2023-09-21 15:50:22 +08:00
2f2884fd6b
[Fix-14895][API] Keep printing null if task log file not exist ( #14912 )
2023-09-19 18:29:33 +08:00
4938470d20
[Fix-14907]Fix click to view the history in the task node of the workflow instance ( #14909 )
2023-09-13 11:17:10 +08:00
a070aa93a7
[Improvement][API] Support to check if the worker group has been used by any tasks or schedulers when users delete or rename it. ( #14893 )
2023-09-13 10:34:59 +08:00
5062aa12da
[Fix-14811][fix] Fix when using Casdoor SSO, ldap, OAuth2 to log in, the tenant problem of not specifying the currently logged in user ( #14812 )
2023-09-07 19:56:38 +08:00
b12df72a0c
[Fix][Resource Center] Remove NONE storage type and set LOCAL type as default ( #14661 )
2023-09-07 14:09:15 +08:00
bc4326125d
Fix session/thread leak at ssh datasource ( #14859 )
2023-09-07 11:55:31 +08:00
11c8e0bdf1
[Improve] Fix placeholders should match arguments in log statement ( #14850 )
2023-09-06 13:32:26 +08:00
974d700984
Revert "fix: Local resource error in some entrypoint ( #14826 )" ( #14838 )
...
This reverts commit 9e60632201
2023-09-05 12:41:04 +08:00
adf49fd23f
[Feature-14802][api] Use Casdoor SSO to log in, add admin user configuration ( #14814 )
...
* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration
Use Casdoor SSO to log in, add admin user configuration
This closes #14802
* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration
Use Casdoor SSO to log in, add admin user configuration
This closes #14802
* update doc
* [Feature-14802][feat] Use Casdoor SSO to log in, add admin user configuration
Use Casdoor SSO to log in, add admin user configuration
This closes #14802
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
Co-authored-by: 旺阳 <qingwli@cisco.com>
2023-09-04 13:25:40 +08:00
9e60632201
fix: Local resource error in some entrypoint ( #14826 )
...
Fix createResource, queryResourceListPaging, onlineCreateResource
resource error in local resource
2023-08-28 21:16:16 +08:00
d223d654cc
Add rpc benchmark test ( #14797 )
2023-08-27 20:59:55 +08:00
d62ce25a01
Remove `@AccessLogAnnotation` annotation, replace with built-in access log ( #14803 )
2023-08-24 15:19:01 +08:00
55cc77226f
[RPC] Refactor the remote module. ( #14776 )
...
* Add dolphinscheduler-extract-base
* Add dolphinscheduler-extract-master
* Add dolphinscheduler-extract-worker
* Add dolphinscheduler-extract-alert
2023-08-23 09:43:17 +08:00
70731a1617
[Feature][Authentication] support oauth2 login ( #14743 )
...
* support oauth2 login
---------
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-08-21 15:57:30 +08:00
e2b97c026e
[Improvement][UI] Support to disable or enable the project preferences. ( #14756 )
2023-08-16 20:20:48 +08:00
a826ed2c1a
[Improvement][security] get-user-info API endpoint should not return the user's password in MD5 format ( #14680 )
...
* [Improvement][security] get-user-info API endpoint should not return the user's password in MD5 format
Signed-off-by: Gallardot <gallardot@apache.org>
* [Improvement][security] list-paging API endpoint should not return the user's password in MD5 format
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Aaron Wang <wangweirao16@gmail.com>
2023-08-14 18:28:11 +08:00
9ce8871158
Support get property from env/jvm/common.properties ( #14645 )
2023-07-28 14:41:58 +08:00
5ccc4cc889
[Improvement-4375][api] cannot delete yarn queue ( #13046 )
...
* [Bug-12833][api]
delete api
* [Bug-12833][api]
delete yarn queue
* [Bug-12833][api]
add Unit Test
* Repair the failed CI
* Repair the failed UT
* Repair the failed Docs
* Repair the failed Doc
---------
Co-authored-by: amao <Guoqing.Mao@Yumchina.com>
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-26 11:52:57 +08:00
4aab0b234f
Use AdHoc datasource client in sqlTask ( #14631 )
...
* Use AdHoc datasource client in sqlTask
* Add method in DataSourceChannel to create PooledDataSourceClient and AdHocDataSourceClient
2023-07-26 08:58:50 +08:00
759756ae80
[Improvement-14269][API] Bind task group with project ( #14392 )
...
* [Improvement-14269][API] Bind task group with project
* remove comment
* add project permission check for task group operation
* add doc
2023-07-23 12:11:37 +08:00
ca760aabd4
[Fix-14512][BOM] kubernetes-client version conflict ( #14561 )
2023-07-21 18:17:53 +08:00
1e31327418
[Improvement][K8S] Disable delete namespaces in the K8S cluster ( #14597 )
...
* [Improvement][K8S] Disable all namespaces operations in the K8S cluster
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-07-21 10:28:20 +08:00
3d033da55d
[Improvement] Dataquality code style enhance ( #14592 )
...
* code style enhance
* update
---------
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-20 20:02:51 +08:00
de2d215aee
impv: Correct datax execute and python execute script name ( #14523 )
...
* impv: Correct datax execute and python execute script name
we use PYTHON_LAUNCHER for python script execute and
DATAX_LAUNCHER for datax script name
* Add pr number
* fix ut
* style
---------
Co-authored-by: xiangzihao <460888207@qq.com>
2023-07-20 09:13:55 +08:00
8505f1878d
Expose swagger.properties to release ( #14590 )
2023-07-19 22:24:42 +08:00
93c3871925
Support parse ipv6 ( #14584 )
2023-07-19 17:25:58 +08:00
20cf4ad4e6
[Feature-14505][UI] Support to set project-level preferences ( #14573 )
...
* add a few codes about the project preferences
2023-07-19 15:35:54 +08:00
804b715643
[Bug][dolphinscheduler-api]validated for duplicate username when creating or registering a user ( #14577 )
...
* [Bug][dolphinscheduler-api]validated for duplicate username when creating or registering a user
Signed-off-by: Gallardot <gallardot@apache.org>
* [Bug][dolphinscheduler-api]validated for duplicate username when creating or registering a user
Signed-off-by: Gallardot <gallardot@apache.org>
---------
Signed-off-by: Gallardot <gallardot@apache.org>
Co-authored-by: xiangzihao <460888207@qq.com>
Co-authored-by: Eric Gao <ericgao.apache@gmail.com>
2023-07-19 12:58:00 +08:00
1ba0aecc71
[Bug][api] An error occurred while trying to access the Postgres database ( #14574 )
...
Signed-off-by: hoey94 <cnnqjban521@gmail.com>
2023-07-18 17:22:27 +08:00
e6d94632b9
Add WorkflowExecuteContext ( #14544 )
2023-07-14 18:40:01 +08:00
484f8419c4
[FIX-14299][api]: check schedule exists ( #14430 )
...
* fix: check schedule exists
* put error message in result
---------
Co-authored-by: lutong <lutongtong211027@credithc.com>
2023-07-13 16:21:07 +08:00
175d976cd2
[Feature][Data Quality] Data Quality Support Choose Database ( #14406 )
...
* add dataquality database api
* change ui
* api change
* update
* fix spotless
* fix h2
* fix pg
* fix-dead-line
* update
* fix-spotless
* update pg sql
* add ut
* fix ut
2023-07-12 09:00:28 +08:00
0246327083
Polish config, expose rpc config in application.yml ( #14501 )
2023-07-11 09:29:31 +08:00
406aeaf659
Use jvm_args_env.sh to set JVM args ( #14487 )
2023-07-10 17:39:57 +08:00
3de0b5a38c
[Feature-14339][Parameter] Add parameter combining of project-level parameters ( #14452 )
2023-07-10 10:17:46 +08:00
d6735e8ecc
[Improvement][API] Fix some nits in apis ( #14481 )
...
* Fix some nits in apis
2023-07-07 18:19:01 +08:00
68a3a75211
[Improvement-14457][UT] Improve the expected and actual value of Assertions ( #14459 )
2023-07-06 13:12:48 +08:00
76b1eefb68
[Feature-14321][API] Support to complement data in descending or ascending order of date ( #14450 )
2023-07-05 14:35:18 +08:00
d8cf0f8b85
fix issue#14441 ( #14441 ) ( #14444 )
...
alert-plugin-instances api, if no alert-plugin-instances, it will response an empty list, but not null
Co-authored-by: ligen60 <ligen60@jd.com>
2023-07-04 16:20:14 +08:00
f186cd5dab
[Feature][Api] LDAP SSL supports&& Add LDAP K8s configuration ( #14349 )
2023-06-29 16:14:37 +08:00
bcd1df19b8
[Improvement-14393][Resource Center] Remove deprecated resource description in api and UI page ( #14394 )
2023-06-28 14:13:20 +08:00
5250b253eb
[Improvement-14311][Permission]Remove duplicate checkProjectAndAuth ( #14313 )
2023-06-27 14:01:37 +08:00
221df4a278
revert: Datasource need test binding to create prod ( #14381 )
...
We should keep our step as less as possible, for now, we have to
test datasource and binding it to the prod datasource, I think it will
make our users do not thing to use datasource, so I do like to revert
it and keep step as less as possible
related to https://github.com/apache/dolphinscheduler/pull/11670
2023-06-27 10:59:17 +08:00
d4921fa2dd
[Feature][UI] Support CRUD of project-level parameters. ( #14344 )
2023-06-13 18:54:41 +08:00
61118d4467
[fix-14123] remove the equals and hashcode in Project pojo ( #14167 )
...
* [fix] remove the equals and hashcode in Project pojo
* [fix]fix the Strict Problem in Mockito by using `doReturn()` instead of `when()` functionw
---------
Co-authored-by: xuankun zheng <zhxuankun@gmail.com>
2023-06-13 11:19:04 +08:00
1c935d901e
[Feature-14214][Parameter] Support CRUD of project-level parameters ( #14264 )
2023-06-12 13:35:38 +08:00
aa2d9a7f05
Add baseDao ( #14316 )
2023-06-09 19:02:50 +08:00
e7cdc7c136
[Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters ( #14127 )
...
* add dynamic task plugin
* fix ui
* update code
* add DOC
* ADD UT
* fix doc
* fix sanity check
* add upgrade ddl
* fix sql
2023-06-07 16:01:06 +08:00
767331281a
[Improvement-14255][Project] Remove userName when update project ( #14256 )
2023-06-02 09:58:22 +08:00
d67fe52c33
Fix the perm setting for the process definition import function ( #14239 )
...
* Fix the perm setting for the process definition import function
* fix the style issue
2023-06-01 15:59:02 +08:00
8aec3846b4
[Improvement-14243][Metrics] Add user tag of response time of api server ( #14248 )
2023-06-01 15:13:28 +08:00
b8748e29ae
Fix TreeView "Task Status" not show bug and compatibility with older versions ( #14207 )
...
use WorkflowExecutionStatus.name()
2023-05-29 13:39:38 +08:00
4c2e57cfb3
[Feature][UI] Support to view and manage all timing settings of a project. ( #14178 )
2023-05-24 08:50:53 +08:00
5c4ba4105d
[Feature-14138][Metrics] Add metrics for api server ( #14177 )
2023-05-23 12:22:49 +08:00
428f1559a3
[Improv-14083][Resource Center] Handle deleteSource in OSS / S3 / GCS ( #14084 )
2023-05-22 20:44:43 +08:00
caishunfeng