Commit Graph

26 Commits

Author SHA1 Message Date
chenrj e1ac0e2605
[python] Add new resource plugin: gitlab, OSS and S3 (#12025)
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-10-10 18:07:57 +08:00
chenrj b1b57c8fb7
[feat][python] new resource plugin GitHub and gitlab (#11831)
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
Co-authored-by: JieguangZhou <jieguang_zhou@163.com>
Co-authored-by: Devosend <devosend@gmail.com>
Co-authored-by: juzimao <578961953@qq.com>
2022-09-22 14:43:00 +08:00
JieguangZhou ad683c3c42
[feat][python] Support MLflow task in python api (#11962) 2022-09-18 16:28:18 +08:00
JieguangZhou bbc228e93f
[Feature][PyDolphinScheduler] Support DVC task in pyds #11922 (#11941)
* add dvc task in pyds

* add BaseDVC class
2022-09-17 16:55:51 +08:00
JieguangZhou e50a32d9cc
[feat][python] Support OpenMLDB task in python api (#11944)
Using the whole word zookeeper instead of short cut zk
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-09-17 15:24:17 +08:00
JieguangZhou e05f14ba23
[feat][python] Add sagemaker yaml example (#11925) 2022-09-17 10:34:06 +08:00
JieguangZhou 5202e5cfc6
[feat][python] Support Pytorch task in python api (#11975)
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-09-16 20:24:38 +08:00
JieguangZhou c41fa5a8b1
fix pydolphin yaml doc link (#11817) 2022-09-09 10:33:17 +08:00
chenrj 6d460a3ca2
[DSIP-13][python] New mechanism file plugins to Python API (#11360) 2022-09-07 10:02:37 +08:00
JieguangZhou 38ee91fb1e
[DSIP-11][python] create workflows from YAML configuration (#11611) 2022-09-06 09:28:16 +08:00
Jiajie Zhong 26afd655cb
[python] Add multiple version of document (#11391)
* Using sphinx-multiversion to support multiple version
of python api document
* change python api sidebar link
* fix whitelist_externals deprecated and add git

close: #8227
2022-08-11 12:02:29 +08:00
Jiajie Zhong 9e692f919a
[python] Add directly install without clone source code (#11179)
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2022-07-28 21:40:20 +08:00
Jiajie Zhong 7a766cbcf2
[python] Refactor structure to avoid cycle import (#11167)
Currently, our core and side module dependent on each others. and will cause
cycle import in our codebase, especially in issue #10905, we try to refactor
to solve this problem.

This patch do the following change:

* Rename module `side` to `models`
* Move `core/base` and `core/sidebase` to dir `modules`
* Move `configuration` and `default_config.yaml` to the root of pydolphinscheduler
2022-07-28 19:02:13 +08:00
JieguangZhou 952aee80e7
[fix] py sagemaker doc (#11109) 2022-07-22 22:57:50 +08:00
JieguangZhou cc088e0f54
[python] Support SageMaker task type (#11002) 2022-07-22 10:47:08 +08:00
Jiajie Zhong 113a67122f
[python] fix doc build error (#10810) 2022-07-06 22:02:08 +08:00
Jiajie Zhong b4eee7edb6
[python] Correct some docs for python api (#10772)
* Remove add leading `$` from command
* Add copy button from code block
* Using `python -m pip` to install package avoid error
2022-07-05 18:34:31 +08:00
Jiajie Zhong 56e0ea802d
[python] Fix process definition attr release state not work (#10151)
This patch fix the error release state not work when it set to
offline and submit it to Java gateway, it error because we do
not pass the attribute to Java gateway function
`createOrUpdateProcessDefinition`

close: #9779
2022-05-24 10:25:19 +08:00
陈家名 828034f8b9
[improve][python] Specify warning group when deploy process-definition (#9773) 2022-04-27 22:02:14 +08:00
Jiajie Zhong 1f48601c75
[python] Add task decorator for python function (#9496)
* [python] Add task decorator for python function

* Add decorator `@task`
* Add a tutorial about it
* Change tutorial doc and combine into traditional docs
  * Add sphinx-inline-tab for better view

* revert not need change

* Correct python function indent

* Correct integration test
2022-04-15 15:50:52 +08:00
Jiajie Zhong 59a026d897
[python] Support read config in env variable (#9517)
Add a new method to get config from environment variables
and for now, we have three ways to get config and the
priority is `env-var > custom-config-file > built-in-config-file`.

Environment config setting does not work in CLI, because it will
confuse users when they get config value is `var-env` but value
in the configuration file is `var-in-file`, they may not find
the way how to change it

* Add documentation
* Add it to UPDATING.md

close: #8344
2022-04-15 15:46:44 +08:00
Jiajie Zhong aaf2042ec4
[python] Add missing doc about config and connect remote server (#9443)
which includes `configuration`, `run example`
`how to connect remote server`

close: #9286, #9284, #8917
2022-04-12 09:42:28 +08:00
Jiajie Zhong 3457cee960
[python] Migrate pythonGatewayServer into api server (#9372)
Currently the size of our distribute package is up to
800MB, this patch is migrate python gateway server into
api server

The distribute package size before and after this patch is:

```sh
# before
796M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz

# after
647M   apache-dolphinscheduler-2.0.4-SNAPSHOT-bin.tar.gz
```
2022-04-07 14:41:15 +08:00
Jiajie Zhong 3025f67d37
[python] Add mechanism cli only with version as subcommand (#8516)
* Add basic cli mechanism with Click, for now just including
  one single subcommand `version`
* Add general and easy test class in tests/testing/cli, and
  test to version
* Add sphinx-click to general cli docs basic on click
2022-02-24 16:05:55 +08:00
Eric Gao 49a1270ef5
[Feature-8222][python] move examples into the scope of source package (#8340) 2022-02-14 15:07:00 +08:00
Jiajie Zhong 35578efd08
[python] Initiate document for pydolphinscheduler (#8005) 2022-01-26 12:37:57 +08:00