natas_hw
/
dolphinscheduler
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[python] Correct some docs for python api (#10772) 

* Remove add leading `$` from command
* Add copy button from code block
* Using `python -m pip` to install package avoid error
This commit is contained in:
Jiajie Zhong 2022-07-05 18:34:31 +08:00 committed by GitHub
parent 43d7cc8a98
commit b4eee7edb6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 69 additions and 57 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
dolphinscheduler-python/pydolphinscheduler/DEVELOP.md
View File

@ -34,7 +34,7 @@ Now, we should install all dependence to make sure we could run test or check co
```shell
cd dolphinscheduler/dolphinscheduler-python/pydolphinscheduler
pip install .[dev]
python -m pip install .[dev]
```
Next, we have to open pydolphinscheduler project in you editor. We recommend you use [pycharm][pycharm]
@ -154,7 +154,7 @@ GitHub. You may locally ensure docs could be built suceessfully in case the fail
To build docs locally, install sphinx and related python modules first via:
```shell
pip install '.[doc]'
python -m pip install '.[doc]'
```
Then

6
dolphinscheduler-python/pydolphinscheduler/README.md
View File

@ -38,11 +38,11 @@ your workflow by python code, aka workflow-as-codes.
```shell
# Install
$ pip install apache-dolphinscheduler
python -m pip install apache-dolphinscheduler
# Verify installation is successful, it will show the version of apache-dolphinscheduler, here we use 0.1.0 as example
$ pydolphinscheduler version
0.1.0
pydolphinscheduler version
# 0.1.0
```
Here we show you how to install and run a simple example of pydolphinscheduler

1
dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py
View File

@ -59,6 +59,7 @@ extensions = [
"sphinx_click.ext",
# Add inline tabbed content
"sphinx_inline_tabs",
"sphinx_copybutton",
]
# Add any paths that contain templates here, relative to this directory.

59
dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst
View File

@ -47,10 +47,10 @@ how to change them by Bash.
.. code-block:: bash
# Modify Java Gateway Address
$ export PYDS_JAVA_GATEWAY_ADDRESS="192.168.1.1"
export PYDS_JAVA_GATEWAY_ADDRESS="192.168.1.1"
# Modify Workflow Default User
$ export PYDS_WORKFLOW_USER="custom-user"
export PYDS_WORKFLOW_USER="custom-user"
After executing the commands above, both ``PYDS_JAVA_GATEWAY_ADDRESS`` and ``PYDS_WORKFLOW_USER`` will be changed.
The next time you execute and submit your workflow, it will submit to host `192.168.1.1`, and with workflow's user
@ -136,15 +136,15 @@ integrated built-in configurations in its package, but you could also export it
.. code-block:: bash
$ pydolphinscheduler config --init
pydolphinscheduler config --init
And it will create a new YAML file in the path `~/pydolphinscheduler/config.yaml` by default. If you want to export
it to another path, you should set `PYDS_HOME` before you run command :code:`pydolphinscheduler config --init`.
.. code-block:: bash
$ export PYDS_HOME=<CUSTOM_PATH>
$ pydolphinscheduler config --init
export PYDS_HOME=<CUSTOM_PATH>
pydolphinscheduler config --init
After that, your configuration file will export into `<CUSTOM_PATH>/config.yaml` instead of the default path.
@ -161,36 +161,37 @@ node of the configuration but could get the parent configuration, there are simp
.. code-block:: bash
# Get single configuration in the leaf node
$ pydolphinscheduler config --get java_gateway.address
The configuration query as below:
# Get single configuration in the leaf node,
# The output look like below:
# java_gateway.address = 127.0.0.1
pydolphinscheduler config --get java_gateway.address
java_gateway.address = 127.0.0.1
# Get multiple configuration in the leaf node,
# The output look like below:
# java_gateway.address = 127.0.0.1
# java_gateway.port = 25333
pydolphinscheduler config --get java_gateway.address --get java_gateway.port
# Get multiple configuration in the leaf node
$ pydolphinscheduler config --get java_gateway.address --get java_gateway.port
The configuration query as below:
java_gateway.address = 127.0.0.1
java_gateway.port = 25333
# Get parent configuration which contain multiple leaf nodes,
# The output look like below:
# java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)])
pydolphinscheduler config --get java_gateway
# Get parent configuration which contain multiple leaf nodes
$ pydolphinscheduler config --get java_gateway
The configuration query as below:
# Set single configuration,
# The output look like below:
# Set configuration done.
pydolphinscheduler config --set java_gateway.address 192.168.1.1
java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)])
# Set multiple configuration
# The output look like below:
# Set configuration done.
pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334
# Set single configuration
$ pydolphinscheduler config --set java_gateway.address 192.168.1.1
Set configuration done.
# Set multiple configuration
$ pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334
Set configuration done.
# Set configuration not in leaf node will fail
$ pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True
Raise error.
# Set configuration not in leaf node will fail
# The output look like below:
# Raise error.
pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True
For more information about our CLI, you could see document :doc:`cli`.

6
dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst
View File

@ -28,7 +28,7 @@ Export Configuration File
.. code-block:: bash
$ pydolphinscheduler config --init
pydolphinscheduler config --init
your could find more detail in :ref:`configuration exporting <config:export configuration file>`
@ -37,7 +37,7 @@ Run API Server in Other Host
.. code-block:: bash
$ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
your could find more detail in :ref:`configuration setting <config:change configuration>`
@ -46,6 +46,6 @@ Run API Server in Other Port
.. code-block:: bash
$ pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port>
pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port>
your could find more detail in :ref:`configuration setting <config:change configuration>`

49
dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst
View File

@ -41,8 +41,9 @@ without error(here is a example after Python 3.8.7 installed)
.. code-block:: bash
$ python --version
Python 3.8.7
python --version
Will see detail of Python version, such as *Python 3.8.7*
Installing PyDolphinScheduler
-----------------------------
@ -52,7 +53,7 @@ After Python is already installed on your machine following section
.. code-block:: bash
$ pip install apache-dolphinscheduler
python -m pip install apache-dolphinscheduler
The latest version of *PyDolphinScheduler* would be installed after you run above
command in your terminal. You could go and `start Python Gateway Service`_ to finish
@ -60,8 +61,16 @@ the prepare, and then go to :doc:`tutorial` to make your hand dirty. But if you
want to install the unreleased version of *PyDolphinScheduler*, you could go and see
section `installing PyDolphinScheduler in dev`_ for more detail.
Installing PyDolphinScheduler In Dev
------------------------------------
.. note::
Currently, we released multiple pre-release package in PyPI, you can see all released package
including pre-release in `release history <https://pypi.org/project/apache-dolphinscheduler/#history>`_.
You can fix the the package version if you want to install pre-release package, for example if
you want to install version `3.0.0-beta-2` package, you can run command
:code:`python -m pip install apache-dolphinscheduler==3.0.0b2`.
Installing PyDolphinScheduler In DEV Branch
-------------------------------------------
Because the project is developing and some of the features still not release.
If you want to try some thing unreleased you could install from the source code
@ -69,10 +78,10 @@ which we hold in GitHub
.. code-block:: bash
# Clone Apache DolphinScheduler repository
$ git clone git@github.com:apache/dolphinscheduler.git
# Install PyDolphinScheduler in develop mode
$ cd dolphinscheduler-python/pydolphinscheduler && pip install -e .
# Clone Apache DolphinScheduler repository
git clone git@github.com:apache/dolphinscheduler.git
# Install PyDolphinScheduler in develop mode
cd dolphinscheduler-python/pydolphinscheduler && python -m pip install -e .
After you installed *PyDolphinScheduler*, please remember `start Python Gateway Service`_
which waiting for *PyDolphinScheduler*'s workflow definition require.
@ -88,18 +97,18 @@ go `install Apache DolphinScheduler`_ for more detail
.. code-block:: bash
# Start DolphinScheduler api-server which including python gateway service
$ ./bin/dolphinscheduler-daemon.sh start api-server
# Start DolphinScheduler api-server which including python gateway service
./bin/dolphinscheduler-daemon.sh start api-server
To check whether the server is alive or not, you could run :code:`jps`. And
the server is health if keyword `ApiApplicationServer` in the console.
.. code-block:: bash
$ jps
....
201472 ApiApplicationServer
....
jps
# ....
# 201472 ApiApplicationServer
# ....
.. note::
@ -115,23 +124,23 @@ single bash command to get it
.. code-block:: bash
$ wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
or you could copy-paste the content from `tutorial source code`_. And then you could run the example in your
terminal
.. code-block:: bash
$ python tutorial.py
python tutorial.py
If you want to submit your workflow to a remote API server, which means that your workflow script is different
from the API server, you should first change pydolphinscheduler configuration and then submit the workflow script
.. code-block:: bash
$ pydolphinscheduler config --init
$ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
$ python tutorial.py
pydolphinscheduler config --init
pydolphinscheduler config --set java_gateway.address <YOUR-API-SERVER-IP-OR-HOSTNAME>
python tutorial.py
.. note::

1
dolphinscheduler-python/pydolphinscheduler/setup.py
View File

@ -52,6 +52,7 @@ doc = [
"sphinx_rtd_theme>=1.0",
"sphinx-click>=3.0",
"sphinx-inline-tabs",
"sphinx-copybutton>=0.4.0",
]
test = [