Docs port otx and datumaro changes (#18005)
port and adjust: https://github.com/openvinotoolkit/openvino/pull/17944 https://github.com/openvinotoolkit/openvino/pull/17269
This commit is contained in:
Karol Blaszczak
GitHub
parent
b10cf93e7e
commit
04bcae3f49
|
|
@ -0,0 +1,71 @@
|
|||
# Datumaro {#datumaro_documentation}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
Datumaro provides a suite of basic data import/export (IE) for more than 35 public vision data
|
||||
formats and manipulation functionalities such as validation, correction, filtration, and some
|
||||
transformations. To achieve the web-scale training, this further aims to merge multiple
|
||||
heterogeneous datasets through comparator and merger. Datumaro is integrated into Geti™, OpenVINO™
|
||||
Training Extensions, and CVAT for the ease of data preparation. Datumaro is open-sourced and
|
||||
available on `GitHub <https://github.com/openvinotoolkit/datumaro>`__.
|
||||
Refer to the official `documentation <https://openvinotoolkit.github.io/datumaro/stable/docs/get-started/introduction.html>`__ to learn more.
|
||||
Plus, enjoy `Jupyter notebooks <https://github.com/openvinotoolkit/datumaro/tree/develop/notebooks>`__ for the real Datumaro practices.
|
||||
|
||||
Detailed Workflow
|
||||
#################
|
||||
|
||||
.. image:: ./_static/images/datumaro.png
|
||||
|
||||
1. To start working with Datumaro, download public datasets or prepare your own annotated dataset.
|
||||
|
||||
.. note::
|
||||
Datumaro provides a CLI `datum download` for downloading `TensorFlow Datasets <https://www.tensorflow.org/datasets>`__.
|
||||
|
||||
2. Import data into Datumaro and manipulate the dataset for the data quality using `Validator`, `Corrector`, and `Filter`.
|
||||
|
||||
3. Compare two datasets and transform the label schemas (category information) before merging them.
|
||||
|
||||
4. Merge two datasets to a large-scale dataset.
|
||||
|
||||
.. note::
|
||||
There are some choices of merger, i.e., `ExactMerger`, `IntersectMerger`, and `UnionMerger`.
|
||||
|
||||
5. Split the unified dataset into subsets, e.g., `train`, `valid`, and `test` through `Splitter`.
|
||||
|
||||
.. note::
|
||||
We can split data with a given ratio of subsets according to both the number of samples or
|
||||
annotations. Please see `SplitTask` for the task-specific split.
|
||||
|
||||
6. Export the cleaned and unified dataset for follow-up workflows such as model training.
|
||||
Go to :doc:`OpenVINO™ Training Extensions <ote_documentation>`.
|
||||
|
||||
If the results are unsatisfactory, add datasets and perform the same steps, starting with dataset annotation.
|
||||
|
||||
Datumaro Components
|
||||
###################
|
||||
|
||||
* `Datumaro CLIs <https://openvinotoolkit.github.io/datumaro/stable/docs/command-reference/overview.html>`__
|
||||
* `Datumaro APIs <https://openvinotoolkit.github.io/datumaro/stable/docs/reference/datumaro_module.html>`__
|
||||
* `Datumaro data format <https://openvinotoolkit.github.io/datumaro/stable/docs/data-formats/datumaro_format.html>`__
|
||||
* `Supported data formats <https://openvinotoolkit.github.io/datumaro/stable/docs/data-formats/formats/index.html>`__
|
||||
|
||||
Tutorials
|
||||
#########
|
||||
|
||||
* `Basic skills <https://openvinotoolkit.github.io/datumaro/stable/docs/level-up/basic_skills/index.html>`__
|
||||
* `Intermediate skills <https://openvinotoolkit.github.io/datumaro/stable/docs/level-up/intermediate_skills/index.html>`__
|
||||
* `Advanced skills <https://openvinotoolkit.github.io/datumaro/stable/docs/level-up/advanced_skills/index.html>`__
|
||||
|
||||
Python Hands-on Examples
|
||||
########################
|
||||
|
||||
* `Data IE <https://openvinotoolkit.github.io/datumaro/stable/docs/jupyter_notebook_examples/dataset_IO.html>`__
|
||||
* `Data manipulation <https://openvinotoolkit.github.io/datumaro/stable/docs/jupyter_notebook_examples/manipulate.html>`__
|
||||
* `Data exploration <https://openvinotoolkit.github.io/datumaro/stable/docs/jupyter_notebook_examples/explore.html>`__
|
||||
* `Data refinement <https://openvinotoolkit.github.io/datumaro/stable/docs/jupyter_notebook_examples/refine.html>`__
|
||||
* `Data transformation <https://openvinotoolkit.github.io/datumaro/stable/docs/jupyter_notebook_examples/transform.html>`__
|
||||
* `Deep learning end-to-end use-cases <https://openvinotoolkit.github.io/datumaro/stable/docs/jupyter_notebook_examples/e2e_example.html>`__
|
||||
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
|
@ -33,5 +33,4 @@ Once you have a model that meets both OpenVINO™ and your requirements, you can
|
|||
@endsphinxdirective
|
||||
|
||||
|
||||
|
||||
Apart from the default deployment options, you may also [deploy your application for the TensorFlow framework with OpenVINO Integration](./openvino_ecosystem_ovtf.md).
|
||||
OpenVINO 2023.0 provides more options, providing inference of TensorFlow models with no additional conversion.
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
:hidden:
|
||||
|
||||
ote_documentation
|
||||
ovtf_integration
|
||||
datumaro_documentation
|
||||
ovsa_get_started
|
||||
openvino_inference_engine_tools_compile_tool_README
|
||||
openvino_docs_tuning_utilities
|
||||
|
|
@ -36,6 +36,16 @@ More resources:
|
|||
* [GitHub](https://github.com/openvinotoolkit/training_extensions)
|
||||
* [Documentation](https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/introduction.html)
|
||||
|
||||
### Dataset Management Framework (Datumaro)
|
||||
A framework and CLI tool to build, transform, and analyze datasets.
|
||||
|
||||
More resources:
|
||||
|
||||
* [Overview](@ref datumaro_documentation)
|
||||
* [PyPI](https://pypi.org/project/datumaro/)
|
||||
* [GitHub](https://github.com/openvinotoolkit/datumaro)
|
||||
* [Documentation](https://openvinotoolkit.github.io/datumaro/stable/docs/get-started/introduction.html)
|
||||
|
||||
### OpenVINO™ Security Add-on
|
||||
A solution for Model Developers and Independent Software Vendors to use secure packaging and secure model execution.
|
||||
|
||||
|
|
@ -47,6 +57,8 @@ More resources:
|
|||
### OpenVINO™ integration with TensorFlow (OVTF)
|
||||
A solution empowering TensorFlow developers with OpenVINO's optimization capabilities. With just two lines of code in your application, you can offload inference to OpenVINO, while keeping the TensorFlow API.
|
||||
|
||||
OpenVINO™ Integration with TensorFlow will no longer be supported as of OpenVINO release 2023.0. As part of the 2023.0 release, OpenVINO will feature a significantly enhanced TensorFlow user experience within native OpenVINO without needing offline model conversions.
|
||||
|
||||
More resources:
|
||||
* [documentation](https://github.com/openvinotoolkit/openvino_tensorflow)
|
||||
* [PyPI](https://pypi.org/project/openvino-tensorflow/)
|
||||
|
|
@ -77,11 +89,3 @@ More resources:
|
|||
* [Docker Hub](https://hub.docker.com/r/openvino/cvat_server)
|
||||
* [GitHub](https://github.com/openvinotoolkit/cvat)
|
||||
|
||||
### Dataset Management Framework (Datumaro)
|
||||
A framework and CLI tool to build, transform, and analyze datasets.
|
||||
|
||||
More resources:
|
||||
* [documentation on GitHub](https://openvinotoolkit.github.io/datumaro/docs/)
|
||||
* [PyPI](https://pypi.org/project/datumaro/)
|
||||
* [GitHub](https://github.com/openvinotoolkit/datumaro)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,44 +0,0 @@
|
|||
# OpenVINO™ integration with TensorFlow {#ovtf_integration}
|
||||
|
||||
**OpenVINO™ integration with TensorFlow** is a solution for TensorFlow developers who want to get started with OpenVINO™ in their inferencing applications. By adding just two lines of code you can now take advantage of OpenVINO™ toolkit optimizations with TensorFlow inference applications across a range of Intel® computation devices.
|
||||
|
||||
This is all you need:
|
||||
```bash
|
||||
import openvino_tensorflow
|
||||
openvino_tensorflow.set_backend('<backend_name>')
|
||||
```
|
||||
|
||||
**OpenVINO™ integration with TensorFlow** accelerates inference across many AI models on a variety of Intel® technologies, such as:
|
||||
- Intel® CPUs
|
||||
- Intel® integrated GPUs
|
||||
- Intel® Movidius™ Vision Processing Units - referred to as VPU
|
||||
- Intel® Vision Accelerator Design with 8 Intel Movidius™ MyriadX VPUs - referred to as VAD-M or HDDL
|
||||
|
||||
> **NOTE**: For maximum performance, efficiency, tooling customization, and hardware control, we recommend developers to adopt native OpenVINO™ solutions.
|
||||
To find out more about the product itself, as well as learn how to use it in your project, check its dedicated [GitHub repository](https://github.com/openvinotoolkit/openvino_tensorflow/tree/master/docs).
|
||||
|
||||
|
||||
To see what you can do with **OpenVINO™ integration with TensorFlow**, explore the demos located in the [examples folder](https://github.com/openvinotoolkit/openvino_tensorflow/tree/master/examples) in our GitHub repository.
|
||||
|
||||
Sample tutorials are also hosted on [Intel® DevCloud](https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/build/ovtfoverview.html). The demo applications are implemented using Jupyter Notebooks. You can interactively execute them on Intel® DevCloud nodes, compare the results of **OpenVINO™ integration with TensorFlow**, native TensorFlow, and OpenVINO™.
|
||||
|
||||
## License
|
||||
**OpenVINO™ integration with TensorFlow** is licensed under [Apache License Version 2.0](https://github.com/openvinotoolkit/openvino_tensorflow/blob/master/LICENSE).
|
||||
By contributing to the project, you agree to the license and copyright terms therein
|
||||
and release your contribution under these terms.
|
||||
|
||||
## Support
|
||||
|
||||
Submit your questions, feature requests and bug reports via [GitHub issues](https://github.com/openvinotoolkit/openvino_tensorflow/issues).
|
||||
|
||||
## How to Contribute
|
||||
|
||||
We welcome community contributions to **OpenVINO™ integration with TensorFlow**. If you have an idea for improvement:
|
||||
|
||||
* Share your proposal via [GitHub issues](https://github.com/openvinotoolkit/openvino_tensorflow/issues).
|
||||
* Submit a [pull request](https://github.com/openvinotoolkit/openvino_tensorflow/pulls).
|
||||
|
||||
We will review your contribution as soon as possible. If any additional fixes or modifications are necessary, we will guide you and provide feedback. Before you make your contribution, make sure you can build **OpenVINO™ integration with TensorFlow** and run all the examples with your fix/patch. If you want to introduce a large feature, create test cases for your feature. Upon our verification of your pull request, we will merge it to the repository provided that the pull request has met the above mentioned requirements and proved acceptable.
|
||||
|
||||
---
|
||||
\* Other names and brands may be claimed as the property of others.
|
||||
|
|
@ -26,14 +26,15 @@ If the results are unsatisfactory, add datasets and perform the same steps, star
|
|||
OpenVINO Training Extensions Components
|
||||
#######################################
|
||||
|
||||
- `OpenVINO Training Extensions SDK <https://github.com/openvinotoolkit/training_extensions/tree/master/ote_sdk>`__
|
||||
- `OpenVINO Training Extensions CLI <https://github.com/openvinotoolkit/training_extensions/tree/master/ote_cli>`__
|
||||
- `OpenVINO Training Extensions Algorithms <https://github.com/openvinotoolkit/training_extensions/tree/master/external>`__
|
||||
* `OpenVINO Training Extensions API <https://github.com/openvinotoolkit/training_extensions/tree/develop/otx/api>`__
|
||||
* `OpenVINO Training Extensions CLI <https://github.com/openvinotoolkit/training_extensions/tree/develop/otx/cli>`__
|
||||
* `OpenVINO Training Extensions Algorithms <https://github.com/openvinotoolkit/training_extensions/tree/develop/otx/algorithms>`__
|
||||
|
||||
Tutorials
|
||||
#########
|
||||
|
||||
`Object Detection <https://github.com/openvinotoolkit/training_extensions/blob/master/ote_cli/notebooks/train.ipynb>`__
|
||||
* `Base tutorial <https://openvinotoolkit.github.io/training_extensions/stable/guide/tutorials/base/index.html>`__
|
||||
* `Advanced tutorial <https://openvinotoolkit.github.io/training_extensions/stable/guide/tutorials/advanced/index.html>`__
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3e91bdf5dc737b1f56e6920eb9f6cbecca9b3186dc7aa049827ced5f12bd598c
|
||||
size 102296
|
||||
|
|
@ -17,7 +17,7 @@ before every release. These models are considered officially supported.
|
|||
| If your model is not included but is similar to those that are, it is still very likely to work.
|
||||
If your model fails to execute properly there are a few options available:
|
||||
|
||||
* If the model originates from a framework like TensorFlow or PyTorch, OpenVINO™ offers a hybrid solution. The original model can be run without explicit conversion into the OpenVINO format. For more information, see :ref:`OpenVINO TensorFlow Integration <https://docs.openvino.ai/latest/ovtf_integration.html>`.
|
||||
* If the model originates from a framework like TensorFlow or PyTorch, OpenVINO™ offers a hybrid solution. The original model can be run without explicit conversion into the OpenVINO format (option available from OpenVINO 2023.0).
|
||||
* You can create a GitHub request for the operation(s) that are missing. These requests are reviewed regularly. You will be informed if and how the request will be accommodated. Additionally, your request may trigger a reply from someone in the community who can help.
|
||||
* As OpenVINO™ is open source you can enhance it with your own contribution to the GitHub repository. To learn more, see the articles on :ref:`OpenVINO Extensibility<https://docs.openvino.ai/latest/openvino_docs_Extensibility_UG_Intro.html>`.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue