[DOCS] Links fixes (#23172)
### Details: - *item1* - *...* ### Tickets: - *ticket-id*
This commit is contained in:
Tatiana Savina
GitHub
parent
79c8d38174
commit
81d061fc70
|
|
@ -26,10 +26,10 @@ A suite of advanced algorithms for Neural Network inference optimization with mi
|
|||
|
||||
More resources:
|
||||
|
||||
* :doc:`Documentation <../openvino-workflow/model-optimization-guide/compressing-models-during-training>`
|
||||
* `GitHub <https://github.com/openvinotoolkit/nncf>`__
|
||||
* `PyPI <https://pypi.org/project/nncf/>`__
|
||||
* `Conda Forge <https://anaconda.org/conda-forge/nncf/>`__
|
||||
* :doc:`Documentation <../openvino-workflow/model-optimization-guide/compressing-models-during-training>`
|
||||
* `GitHub <https://github.com/openvinotoolkit/nncf>`__
|
||||
* `PyPI <https://pypi.org/project/nncf/>`__
|
||||
* `Conda Forge <https://anaconda.org/conda-forge/nncf/>`__
|
||||
|
||||
|
||||
**OpenVINO™ Training Extensions**
|
||||
|
|
@ -40,27 +40,27 @@ More resources:
|
|||
|
||||
* :doc:`Overview <openvino-ecosystem/openvino-training-extensions>`
|
||||
* `GitHub <https://github.com/openvinotoolkit/training_extensions>`__
|
||||
* `Documentation <https://openvinotoolkit.github.io/training_extensions/stable/guide/../get-started/introduction.html>`__
|
||||
* `Documentation <https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/introduction.html>`__
|
||||
|
||||
**OpenVINO™ Security Add-on**
|
||||
|
||||
A solution for Model Developers and Independent Software Vendors to use secure packaging and secure model execution.
|
||||
A solution for Model Developers and Independent Software Vendors to use secure packaging and secure model execution.
|
||||
|
||||
More resources:
|
||||
|
||||
* :doc:`Documentation <openvino-ecosystem/openvino-security-add-on>`
|
||||
* `GitHub <https://github.com/openvinotoolkit/security_addon>`__
|
||||
* `GitHub <https://github.com/openvinotoolkit/security_addon>`__
|
||||
|
||||
**Dataset Management Framework (Datumaro)**
|
||||
|
||||
A framework and CLI tool to build, transform, and analyze datasets.
|
||||
|
||||
More resources:
|
||||
|
||||
|
||||
* :doc:`Overview <openvino-ecosystem/datumaro>`
|
||||
* `PyPI <https://pypi.org/project/datumaro/>`__
|
||||
* `GitHub <https://github.com/openvinotoolkit/datumaro>`__
|
||||
* `Documentation <https://openvinotoolkit.github.io/datumaro/stable/docs/get-started/introduction.html>`__
|
||||
* `PyPI <https://pypi.org/project/datumaro/>`__
|
||||
* `GitHub <https://github.com/openvinotoolkit/datumaro>`__
|
||||
* `Documentation <https://openvinotoolkit.github.io/datumaro/stable/docs/get-started/introduction.html>`__
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -5,15 +5,15 @@ OpenVINO™ Training Extensions
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: OpenVINO™ Training Extensions include advanced algorithms used
|
||||
to create, train and convert deep learning models with OpenVINO
|
||||
:description: OpenVINO™ Training Extensions include advanced algorithms used
|
||||
to create, train and convert deep learning models with OpenVINO
|
||||
Toolkit for optimized inference.
|
||||
|
||||
|
||||
OpenVINO™ Training Extensions provide a suite of advanced algorithms to train
|
||||
Deep Learning models and convert them using the `OpenVINO™
|
||||
toolkit <https://software.intel.com/en-us/openvino-toolkit>`__ for optimized
|
||||
inference. It allows you to export and convert the models to the needed format. OpenVINO Training Extensions independently create and train the model. It is open-sourced and available on `GitHub <https://github.com/openvinotoolkit/training_extensions>`__. Read the OpenVINO Training Extensions `../../documentation <https://openvinotoolkit.github.io/training_extensions/stable/guide/../../get-started/introduction.html>`__ to learn more.
|
||||
inference. It allows you to export and convert the models to the needed format. OpenVINO Training Extensions independently create and train the model. It is open-sourced and available on `GitHub <https://github.com/openvinotoolkit/training_extensions>`__. Read the OpenVINO Training Extensions `documentation <https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/introduction.html>`__ to learn more.
|
||||
|
||||
Detailed Workflow
|
||||
#################
|
||||
|
|
@ -24,7 +24,7 @@ Detailed Workflow
|
|||
|
||||
2. OpenVINO Training Extensions train the model, using training interface, and evaluate the model quality on your dataset, using evaluation and inference interfaces.
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
Prepare a separate dataset or split the dataset you have for more accurate quality evaluation.
|
||||
|
||||
3. Having successful evaluation results received, you have an opportunity to deploy your model or continue optimizing it, using NNCF. For more information about these frameworks, go to :doc:`Optimization Guide <../../openvino-workflow/model-optimization>`.
|
||||
|
|
@ -34,9 +34,9 @@ If the results are unsatisfactory, add datasets and perform the same steps, star
|
|||
OpenVINO Training Extensions Components
|
||||
#######################################
|
||||
|
||||
* `OpenVINO Training Extensions API <https://github.com/openvinotoolkit/training_extensions/tree/develop/otx/api>`__
|
||||
* `OpenVINO Training Extensions CLI <https://github.com/openvinotoolkit/training_extensions/tree/develop/otx/cli>`__
|
||||
* `OpenVINO Training Extensions Algorithms <https://github.com/openvinotoolkit/training_extensions/tree/develop/otx/algorithms>`__
|
||||
* `OpenVINO Training Extensions API <https://github.com/openvinotoolkit/training_extensions/tree/develop/src/otx/api>`__
|
||||
* `OpenVINO Training Extensions CLI <https://github.com/openvinotoolkit/training_extensions/tree/develop/src/otx/cli>`__
|
||||
* `OpenVINO Training Extensions Algorithms <https://github.com/openvinotoolkit/training_extensions/tree/develop/src/otx/algorithms>`__
|
||||
|
||||
Tutorials
|
||||
#########
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ The granularity of OpenVINO packages may vary for different distribution types.
|
|||
|
||||
- The main library ``openvino`` is used by users' C++ applications to link against with. For C language applications, ``openvino_c`` is additionally required for distribution. The library includes OpenVINO API 2.0.
|
||||
- The "optional" plugin libraries like ``openvino_intel_cpu_plugin`` (matching the ``openvino_.+_plugin`` pattern) are used to provide inference capabilities on specific devices or additional capabilities like :doc:`Hetero Execution <running-inference/inference-devices-and-modes/hetero-execution>` and :doc:`Multi-Device Execution <running-inference/inference-devices-and-modes/multi-device>`.
|
||||
- The "optional" plugin libraries like ``../documentation/openvino-ir-format_frontend`` (matching ``openvino_.+_frontend``) are used to provide capabilities to read models of different file formats such as OpenVINO IR, TensorFlow, ONNX, and PaddlePaddle.
|
||||
- The "optional" plugin libraries like ``openvino_ir_frontend`` (matching ``openvino_.+_frontend``) are used to provide capabilities to read models of different file formats such as OpenVINO IR, TensorFlow, ONNX, and PaddlePaddle.
|
||||
|
||||
Here the term "optional" means that if the application does not use the capability enabled by the plugin, the plugin library or a package with the plugin is not needed in the final distribution.
|
||||
|
||||
|
|
|
|||
|
|
@ -114,17 +114,16 @@ Libraries for Execution Modes
|
|||
The ``HETERO``, ``MULTI``, ``BATCH``, and ``AUTO`` execution modes can also be used by the application explicitly or implicitly.
|
||||
Use the following recommendation scheme to decide whether to add the appropriate libraries to the distribution package:
|
||||
|
||||
- If :doc:`AUTO <../running-inference/inference-devices-and-modes/auto-device-selection>` is used explicitly in the application or `ov::Core::compile_model <classov_1_1Core.html#doxid-classov-1-1-core-1a46555f0803e8c29524626be08e7f5c5a>`__ is used without specifying a device, put ``openvino_auto_plugin`` to the distribution.
|
||||
- If :doc:`AUTO <../running-inference/inference-devices-and-modes/auto-device-selection>` is used explicitly in the application or ov::Core::compile_model is used without specifying a device, put ``openvino_auto_plugin`` to the distribution.
|
||||
|
||||
.. note::
|
||||
|
||||
Automatic Device Selection relies on :doc:`inference device plugins <../running-inference/inference-devices-and-modes>`.
|
||||
If you are not sure which inference devices are available on the target system, put all inference plugin libraries in the distribution.
|
||||
If `ov::device::priorities <groupov_runtime_cpp_prop_api.html#doxid-group-ov-runtime-cpp-prop-api-1gae88af90a18871677f39739cb0ef0101e>`__
|
||||
is used for `AUTO` to specify a limited device list, grab the corresponding device plugins only.
|
||||
If ov::device::priorities is used for `AUTO` to specify a limited device list, grab the corresponding device plugins only.
|
||||
|
||||
- If :doc:`MULTI <../running-inference/inference-devices-and-modes/multi-device>` is used explicitly, put ``openvino_auto_plugin`` in the distribution.
|
||||
- If :doc:`HETERO <../running-inference/inference-devices-and-modes/hetero-execution>` is either used explicitly or `ov::hint::performance_mode <groupov_runtime_cpp_prop_api.html#doxid-group-ov-runtime-cpp-prop-api-1ga2691fe27acc8aa1d1700ad40b6da3ba2>`__ is used with GPU, put ``openvino_hetero_plugin`` in the distribution.
|
||||
- If :doc:`HETERO <../running-inference/inference-devices-and-modes/hetero-execution>` is either used explicitly or ov::hint::performance_mode is used with GPU, put ``openvino_hetero_plugin`` in the distribution.
|
||||
- If :doc:`BATCH <../running-inference/inference-devices-and-modes/automatic-batching>` is either used explicitly or ``ov::hint::performance_mode`` is used with GPU, put ``openvino_batch_plugin`` in the distribution.
|
||||
|
||||
Frontend Libraries for Reading Models
|
||||
|
|
@ -132,18 +131,18 @@ Frontend Libraries for Reading Models
|
|||
|
||||
OpenVINO Runtime uses frontend libraries dynamically to read models in different formats:
|
||||
|
||||
- ``../../documentation/openvino-ir-format_frontend`` is used to read OpenVINO IR.
|
||||
- ``openvino_ir_frontend`` is used to read OpenVINO IR.
|
||||
- ``openvino_tensorflow_frontend`` is used to read the TensorFlow file format.
|
||||
- ``openvino_tensorflow_lite_frontend`` is used to read the TensorFlow Lite file format.
|
||||
- ``openvino_onnx_frontend`` is used to read the ONNX file format.
|
||||
- ``openvino_paddle_frontend`` is used to read the Paddle file format.
|
||||
- ``openvino_pytorch_frontend`` is used to convert PyTorch model via ``openvino.convert_model`` API.
|
||||
|
||||
Depending on the model format types that are used in the application in `ov::Core::read_model <classov_1_1Core.html#doxid-classov-1-1-core-1ae0576a95f841c3a6f5e46e4802716981>`__, select the appropriate libraries.
|
||||
Depending on the model format types that are used in the application in ov::Core::read_model, select the appropriate libraries.
|
||||
|
||||
.. note::
|
||||
|
||||
To optimize the size of the final distribution package, it is recommended to convert models to OpenVINO IR by using :doc:`model conversion API <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api>`. This way you do not have to keep TensorFlow, TensorFlow Lite, ONNX, PaddlePaddle, and other frontend libraries in the distribution package.
|
||||
To optimize the size of the final distribution package, it is recommended to convert models to OpenVINO IR by using :doc:`model conversion API <../model-preparation>`. This way you do not have to keep TensorFlow, TensorFlow Lite, ONNX, PaddlePaddle, and other frontend libraries in the distribution package.
|
||||
|
||||
Examples
|
||||
####################
|
||||
|
|
@ -152,16 +151,16 @@ Examples
|
|||
|
||||
In this example, the application is written in C, performs inference on CPU, and reads models stored in the OpenVINO IR format.
|
||||
|
||||
The following libraries are used: ``openvino_c``, ``openvino``, ``openvino_intel_cpu_plugin``, and ``../../documentation/openvino-ir-format_frontend``.
|
||||
The following libraries are used: ``openvino_c``, ``openvino``, ``openvino_intel_cpu_plugin``, and ``openvino_ir_frontend``.
|
||||
|
||||
- The ``openvino_c`` library is a main dependency of the application. The app links against this library.
|
||||
- The ``openvino`` library is used as a private dependency for ``openvino_c`` and is also used in the deployment.
|
||||
- ``openvino_intel_cpu_plugin`` is used for inference.
|
||||
- ``../../documentation/openvino-ir-format_frontend`` is used to read source models.
|
||||
- ``openvino_ir_frontend`` is used to read source models.
|
||||
|
||||
**MULTI execution on GPU and CPU in `tput` mode**
|
||||
|
||||
In this example, the application is written in C++, performs inference :doc:`simultaneously on GPU and CPU devices <../running-inference/inference-devices-and-modes/multi-device>` with the `ov::hint::PerformanceMode::THROUGHPUT <enumov_1_1hint_1_1PerformanceMode.html#doxid-group-ov-runtime-cpp-prop-api-1gga032aa530efa40760b79af14913d48d73a50f9b1f40c078d242af7ec323ace44b3>`__ property set, and reads models stored in the ONNX format.
|
||||
In this example, the application is written in C++, performs inference :doc:`simultaneously on GPU and CPU devices <../running-inference/inference-devices-and-modes/multi-device>` with the ov::hint::PerformanceMode::THROUGHPUT property set, and reads models stored in the ONNX format.
|
||||
|
||||
The following libraries are used: ``openvino``, ``openvino_intel_gpu_plugin``, ``openvino_intel_cpu_plugin``, ``openvino_auto_plugin``, ``openvino_auto_batch_plugin``, and ``openvino_onnx_frontend``.
|
||||
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ Put all CSV files together for ``stage 2`` to generate resulting OpenVINO binari
|
|||
|
||||
Device-agnostic Conditional Compilation (POC)
|
||||
#############################################
|
||||
In some cases, adopting conditional compilation is necessary to support multiple different ``SKUs``(Stock Keeping Unit: is usually a string of numbers and alphabets used by the manufacturer to identify their product), but there may be limitations in collecting statistics information for every target hardware. To achieve this, conditional compilation must be capable of running a model on an accelerator with all previous SKUs.
|
||||
In some cases, adopting conditional compilation is necessary to support multiple different ``SKUs`` (Stock Keeping Unit: is usually a string of numbers and alphabets used by the manufacturer to identify their product), but there may be limitations in collecting statistics information for every target hardware. To achieve this, conditional compilation must be capable of running a model on an accelerator with all previous SKUs.
|
||||
|
||||
Conditional compilation requires the initial collection of statistical information to exclude unused code regions, such as ops and kernels. To do this, all included ops and kernels must be executed at least once. For multiple SKUs, it is necessary for all ops and kernels that will be used by any of the SKUs to be encountered at least once in the profiling data. If the profiling is done on a CPU platform, it is impossible without using an emulator.
|
||||
|
||||
|
|
@ -163,11 +163,7 @@ Considering that JIT kernels can be affected by L1/L2/L3 cache size and the numb
|
|||
|
||||
- L2/L3 cache emulation
|
||||
|
||||
Hack the function of get cache size
|
||||
|
||||
``unsigned int dnnl::impl::cpu::platform::get_per_core_cache_size(int level)``
|
||||
|
||||
to make it return emulated cache size in analyzed stage, the simplest way is to leverage environment variable to pass the emulated cache size, for example:
|
||||
Hack the function of get cache size ``unsigned int dnnl::impl::cpu::platform::get_per_core_cache_size(int level)`` to make it return emulated cache size in analyzed stage, the simplest way is to leverage environment variable to pass the emulated cache size, for example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
|
|
|
|||
|
|
@ -170,15 +170,6 @@ For more details on compression options, refer to the :doc:`weight compression g
|
|||
OpenVINO also supports 4-bit models from Hugging Face `Transformers <https://github.com/huggingface/transformers>`__ library optimized
|
||||
with `GPTQ <https://github.com/PanQiWei/AutoGPTQ>`__. In this case, there is no need for an additional model optimization step because model conversion will automatically preserve the INT4 optimization results, allowing model inference to benefit from it.
|
||||
|
||||
Another optimization that is applied by default when using ``OVModelForCausalLM`` class is transformation of the model to a stateful form.
|
||||
This transformation further improves inference performance and decreases amount of allocated runtime memory in long running text generation scenarious.
|
||||
It is achieved by hiding inputs and outputs of the model that represent past KV-cache tensors, and handling them inside the model in a more efficient way.
|
||||
This feature is activated automatically for a wide range of supported text generation models, keeping not supported models in a regular, stateless form.
|
||||
|
||||
Model usage are identical for stateful and stateless models as long as Optimum-Intel API is used because KV-cache handling is an internal detail of the text-generation API of Transformers library.
|
||||
But a form of a model matterns in case when exported from Optimum-Intel OpenVINO model IR is used in an application implemented with native OpenVINO API, because stateful and stateless models have different number of inputs and outputs.
|
||||
Please refer to a dedicated section of this document below for more information about using native OpenVINO API.
|
||||
|
||||
Below are some examples of using Optimum-Intel for model conversion and inference:
|
||||
|
||||
* `Stable Diffusion v2.1 using Optimum-Intel OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/236-stable-diffusion-v2/236-stable-diffusion-v2-optimum-demo.ipynb>`__
|
||||
|
|
|
|||
|
|
@ -30,8 +30,8 @@ NNCF Quick Start Examples
|
|||
|
||||
See the following Jupyter Notebooks for step-by-step examples showing how to add model compression to a PyTorch or Tensorflow training pipeline with NNCF:
|
||||
|
||||
- `Quantization Aware Training with NNCF and PyTorch <notebooks/302-pytorch-quantization-aware-training-with-output.html>`__.
|
||||
- `Quantization Aware Training with NNCF and TensorFlow <notebooks/305-tensorflow-quantization-aware-training-with-output.html>`__.
|
||||
- `Quantization Aware Training with NNCF and PyTorch <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/302-pytorch-quantization-aware-training>`__.
|
||||
- `Quantization Aware Training with NNCF and TensorFlow <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/305-tensorflow-quantization-aware-training>`__.
|
||||
|
||||
Installation
|
||||
####################
|
||||
|
|
@ -111,6 +111,6 @@ Additional Resources
|
|||
- :doc:`Quantizing Models Post-training <quantizing-models-post-training>`
|
||||
- `NNCF GitHub repository <https://github.com/openvinotoolkit/nncf>`__
|
||||
- `NNCF FAQ <https://github.com/openvinotoolkit/nncf/blob/develop/docs/FAQ.md>`__
|
||||
- `Quantization Aware Training with NNCF and PyTorch <notebooks/302-pytorch-quantization-aware-training-with-output.html>`__
|
||||
- `Quantization Aware Training with NNCF and TensorFlow <notebooks/305-tensorflow-quantization-aware-training-with-output.html>`__.
|
||||
- `Quantization Aware Training with NNCF and PyTorch <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/302-pytorch-quantization-aware-training>`__.
|
||||
- `Quantization Aware Training with NNCF and TensorFlow <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/305-tensorflow-quantization-aware-training>`__.
|
||||
|
||||
|
|
|
|||
|
|
@ -7,23 +7,23 @@ Filter Pruning of Convolutional Models
|
|||
Introduction
|
||||
####################
|
||||
|
||||
Filter pruning is an advanced optimization method that allows reducing the computational complexity of the model by removing
|
||||
redundant or unimportant filters from the convolutional operations of the model. This removal is done in two steps:
|
||||
Filter pruning is an advanced optimization method that allows reducing the computational complexity of the model by removing
|
||||
redundant or unimportant filters from the convolutional operations of the model. This removal is done in two steps:
|
||||
|
||||
1. Unimportant filters are zeroed out by the NNCF optimization with fine-tuning.
|
||||
|
||||
2. Zero filters are removed from the model during the export to OpenVINO Intermediate Representation (IR).
|
||||
|
||||
|
||||
Filter Pruning method from the NNCF can be used stand-alone but we usually recommend stacking it with 8-bit quantization for
|
||||
two reasons. First, 8-bit quantization is the best method in terms of achieving the highest accuracy-performance trade-offs so
|
||||
stacking it with filter pruning can give even better performance results. Second, applying quantization along with filter
|
||||
pruning does not hurt accuracy a lot since filter pruning removes noisy filters from the model which narrows down values
|
||||
Filter Pruning method from the NNCF can be used stand-alone but we usually recommend stacking it with 8-bit quantization for
|
||||
two reasons. First, 8-bit quantization is the best method in terms of achieving the highest accuracy-performance trade-offs so
|
||||
stacking it with filter pruning can give even better performance results. Second, applying quantization along with filter
|
||||
pruning does not hurt accuracy a lot since filter pruning removes noisy filters from the model which narrows down values
|
||||
ranges of weights and activations and helps to reduce overall quantization error.
|
||||
|
||||
.. note::
|
||||
Filter Pruning usually requires a long fine-tuning or retraining of the model which can be comparable to training the
|
||||
model from scratch. Otherwise, a large accuracy degradation can be caused. Therefore, the training schedule should be
|
||||
Filter Pruning usually requires a long fine-tuning or retraining of the model which can be comparable to training the
|
||||
model from scratch. Otherwise, a large accuracy degradation can be caused. Therefore, the training schedule should be
|
||||
adjusted accordingly when applying this method.
|
||||
|
||||
|
||||
|
|
@ -43,13 +43,13 @@ In this step, NNCF-related imports are added in the beginning of the training sc
|
|||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [imports]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -58,26 +58,26 @@ In this step, NNCF-related imports are added in the beginning of the training sc
|
|||
2. Create NNCF configuration
|
||||
++++++++++++++++++++++++++++
|
||||
|
||||
Here, you should define NNCF configuration which consists of model-related parameters (`"input_info"` section) and parameters
|
||||
Here, you should define NNCF configuration which consists of model-related parameters (`"input_info"` section) and parameters
|
||||
of optimization methods (`"compression"` section).
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [nncf_congig]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
:fragment: [nncf_congig]
|
||||
|
||||
Here is a brief description of the required parameters of the Filter Pruning method. For a full description refer to the
|
||||
|
||||
Here is a brief description of the required parameters of the Filter Pruning method. For a full description refer to the
|
||||
`GitHub <https://github.com/openvinotoolkit/nncf/blob/develop/docs/compression_algorithms/Pruning.md>`__ page.
|
||||
|
||||
* ``pruning_init`` - initial pruning rate target. For example, value ``0.1`` means that at the begging of training, convolutions that can be pruned will have 10% of their filters set to zero.
|
||||
|
|
@ -90,23 +90,23 @@ Here is a brief description of the required parameters of the Filter Pruning met
|
|||
3. Apply optimization methods
|
||||
+++++++++++++++++++++++++++++
|
||||
|
||||
In the next step, the original model is wrapped by the NNCF object using the ``create_compressed_model()`` API using the
|
||||
configuration defined in the previous step. This method returns a so-called compression controller and the wrapped model
|
||||
that can be used the same way as the original model. It is worth noting that optimization methods are applied at this step
|
||||
so that the model undergoes a set of corresponding transformations and can contain additional operations required for the
|
||||
In the next step, the original model is wrapped by the NNCF object using the ``create_compressed_model()`` API using the
|
||||
configuration defined in the previous step. This method returns a so-called compression controller and the wrapped model
|
||||
that can be used the same way as the original model. It is worth noting that optimization methods are applied at this step
|
||||
so that the model undergoes a set of corresponding transformations and can contain additional operations required for the
|
||||
optimization.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [wrap_model]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -115,21 +115,21 @@ optimization.
|
|||
4. Fine-tune the model
|
||||
++++++++++++++++++++++
|
||||
|
||||
This step assumes that you will apply fine-tuning to the model the same way as it is done for the baseline model. In the case
|
||||
of Filter Pruning method we recommend using the training schedule and learning rate similar to what was used for the training
|
||||
This step assumes that you will apply fine-tuning to the model the same way as it is done for the baseline model. In the case
|
||||
of Filter Pruning method we recommend using the training schedule and learning rate similar to what was used for the training
|
||||
of the original model.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [tune_model]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -139,49 +139,49 @@ of the original model.
|
|||
5. Multi-GPU distributed training
|
||||
+++++++++++++++++++++++++++++++++
|
||||
|
||||
In the case of distributed multi-GPU training (not DataParallel), you should call ``compression_ctrl.distributed()`` before the
|
||||
In the case of distributed multi-GPU training (not DataParallel), you should call ``compression_ctrl.distributed()`` before the
|
||||
fine-tuning that will inform optimization methods to do some adjustments to function in the distributed mode.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [distributed]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
:fragment: [distributed]
|
||||
|
||||
|
||||
6. Export quantized model
|
||||
+++++++++++++++++++++++++
|
||||
|
||||
When fine-tuning finishes, the quantized model can be exported to the corresponding format for further inference: ONNX in
|
||||
When fine-tuning finishes, the quantized model can be exported to the corresponding format for further inference: ONNX in
|
||||
the case of PyTorch and frozen graph - for TensorFlow 2.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [export]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
:fragment: [export]
|
||||
|
||||
|
||||
These were the basic steps to applying the QAT method from the NNCF. However, it is required in some cases to save/load model
|
||||
These were the basic steps to applying the QAT method from the NNCF. However, it is required in some cases to save/load model
|
||||
checkpoints during the training. Since NNCF wraps the original model with its own object it provides an API for these needs.
|
||||
|
||||
|
||||
|
|
@ -194,18 +194,18 @@ To save model checkpoint use the following API:
|
|||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [save_checkpoint]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
:fragment: [save_checkpoint]
|
||||
|
||||
|
||||
|
||||
8. (Optional) Restore from checkpoint
|
||||
+++++++++++++++++++++++++++++++++++++
|
||||
|
|
@ -216,31 +216,31 @@ To restore the model from checkpoint you should use the following API:
|
|||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_torch.py
|
||||
:language: python
|
||||
:fragment: [load_checkpoint]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
:fragment: [load_checkpoint]
|
||||
:fragment: [load_checkpoint]
|
||||
|
||||
For more details on saving/loading checkpoints in the NNCF, see the following
|
||||
`../../../documentation <https://github.com/openvinotoolkit/nncf/blob/develop/docs/Usage.md#saving-and-loading-compressed-models>`__.
|
||||
For more details on saving/loading checkpoints in the NNCF, see the following
|
||||
`documentation <https://github.com/openvinotoolkit/nncf/blob/develop/docs/Usage.md#saving-and-loading-compressed-models>`__.
|
||||
|
||||
Deploying pruned model
|
||||
######################
|
||||
|
||||
The pruned model requres an extra step that should be done to get performance improvement. This step involves removal of the
|
||||
zero filters from the model. This is done at the model conversion step using :doc:`model conversion API <../../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api>` tool when model is converted from the framework representation (ONNX, TensorFlow, etc.) to OpenVINO Intermediate Representation.
|
||||
The pruned model requres an extra step that should be done to get performance improvement. This step involves removal of the
|
||||
zero filters from the model. This is done at the model conversion step using :doc:`model conversion API <../../model-preparation>`>` tool when model is converted from the framework representation (ONNX, TensorFlow, etc.) to OpenVINO Intermediate Representation.
|
||||
|
||||
* To remove zero filters from the pruned model add the following parameter to the model conversion command: ``transform=Pruning``
|
||||
|
||||
After that, the model can be deployed with OpenVINO in the same way as the baseline model.
|
||||
For more details about model deployment with OpenVINO, see the corresponding :doc:`../../../documentation <../../running-inference>`.
|
||||
For more details about model deployment with OpenVINO, see the corresponding :doc:`documentation <../../running-inference>`.
|
||||
|
||||
|
||||
Examples
|
||||
|
|
|
|||
|
|
@ -7,20 +7,20 @@ Quantization-aware Training (QAT)
|
|||
Introduction
|
||||
####################
|
||||
|
||||
Quantization-aware Training is a popular method that allows quantizing a model and applying fine-tuning to restore accuracy
|
||||
degradation caused by quantization. In fact, this is the most accurate quantization method. This document describes how to
|
||||
apply QAT from the Neural Network Compression Framework (NNCF) to get 8-bit quantized models. This assumes that you are
|
||||
Quantization-aware Training is a popular method that allows quantizing a model and applying fine-tuning to restore accuracy
|
||||
degradation caused by quantization. In fact, this is the most accurate quantization method. This document describes how to
|
||||
apply QAT from the Neural Network Compression Framework (NNCF) to get 8-bit quantized models. This assumes that you are
|
||||
knowledgeable in Python programming and familiar with the training code for the model in the source DL framework.
|
||||
|
||||
Using NNCF QAT
|
||||
####################
|
||||
|
||||
Here, we provide the steps that are required to integrate QAT from NNCF into the training script written with
|
||||
Here, we provide the steps that are required to integrate QAT from NNCF into the training script written with
|
||||
PyTorch or TensorFlow 2:
|
||||
|
||||
.. note::
|
||||
Currently, NNCF for TensorFlow 2 supports optimization of the models created using Keras
|
||||
`Sequential API <https://www.tensorflow.org/guide/keras/sequential_model>`__ or
|
||||
Currently, NNCF for TensorFlow 2 supports optimization of the models created using Keras
|
||||
`Sequential API <https://www.tensorflow.org/guide/keras/sequential_model>`__ or
|
||||
`Functional API <https://www.tensorflow.org/guide/keras/functional>`__.
|
||||
|
||||
1. Import NNCF API
|
||||
|
|
@ -32,145 +32,145 @@ In this step, you add NNCF-related imports in the beginning of the training scri
|
|||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [imports]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [imports]
|
||||
:fragment: [imports]
|
||||
|
||||
2. Create NNCF configuration
|
||||
++++++++++++++++++++++++++++
|
||||
|
||||
Here, you should define NNCF configuration which consists of model-related parameters (``"input_info"`` section) and parameters
|
||||
of optimization methods (``"compression"`` section). For faster convergence, it is also recommended to register a dataset object
|
||||
Here, you should define NNCF configuration which consists of model-related parameters (``"input_info"`` section) and parameters
|
||||
of optimization methods (``"compression"`` section). For faster convergence, it is also recommended to register a dataset object
|
||||
specific to the DL framework. It will be used at the model creation step to initialize quantization parameters.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [nncf_congig]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [nncf_congig]
|
||||
|
||||
:fragment: [nncf_congig]
|
||||
|
||||
|
||||
3. Apply optimization methods
|
||||
+++++++++++++++++++++++++++++
|
||||
|
||||
In the next step, you need to wrap the original model object with the ``create_compressed_model()`` API using the configuration
|
||||
defined in the previous step. This method returns a so-called compression controller and a wrapped model that can be used the
|
||||
same way as the original model. It is worth noting that optimization methods are applied at this step so that the model
|
||||
undergoes a set of corresponding transformations and can contain additional operations required for the optimization. In
|
||||
the case of QAT, the compression controller object is used for model export and, optionally, in distributed training as it
|
||||
In the next step, you need to wrap the original model object with the ``create_compressed_model()`` API using the configuration
|
||||
defined in the previous step. This method returns a so-called compression controller and a wrapped model that can be used the
|
||||
same way as the original model. It is worth noting that optimization methods are applied at this step so that the model
|
||||
undergoes a set of corresponding transformations and can contain additional operations required for the optimization. In
|
||||
the case of QAT, the compression controller object is used for model export and, optionally, in distributed training as it
|
||||
will be shown below.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [wrap_model]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [wrap_model]
|
||||
|
||||
|
||||
|
||||
4. Fine-tune the model
|
||||
++++++++++++++++++++++
|
||||
|
||||
This step assumes that you will apply fine-tuning to the model the same way as it is done for the baseline model. In the
|
||||
case of QAT, it is required to train the model for a few epochs with a small learning rate, for example, 10e-5. In principle,
|
||||
This step assumes that you will apply fine-tuning to the model the same way as it is done for the baseline model. In the
|
||||
case of QAT, it is required to train the model for a few epochs with a small learning rate, for example, 10e-5. In principle,
|
||||
you can skip this step which means that the post-training optimization will be applied to the model.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [tune_model]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [tune_model]
|
||||
|
||||
|
||||
|
||||
|
||||
5. Multi-GPU distributed training
|
||||
+++++++++++++++++++++++++++++++++
|
||||
|
||||
In the case of distributed multi-GPU training (not DataParallel), you should call ``compression_ctrl.distributed()`` before
|
||||
In the case of distributed multi-GPU training (not DataParallel), you should call ``compression_ctrl.distributed()`` before
|
||||
the fine-tuning that will inform optimization methods to do some adjustments to function in the distributed mode.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [distributed]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [distributed]
|
||||
|
||||
|
||||
6. Export quantized model
|
||||
+++++++++++++++++++++++++
|
||||
|
||||
When fine-tuning finishes, the quantized model can be exported to the corresponding format for further inference: ONNX in
|
||||
When fine-tuning finishes, the quantized model can be exported to the corresponding format for further inference: ONNX in
|
||||
the case of PyTorch and frozen graph - for TensorFlow 2.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [export]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [export]
|
||||
|
||||
|
||||
|
||||
.. note::
|
||||
The precision of weights gets INT8 only after the step of model conversion to OpenVINO Intermediate Representation.
|
||||
The precision of weights gets INT8 only after the step of model conversion to OpenVINO Intermediate Representation.
|
||||
You can expect the model footprint reduction only for that format.
|
||||
|
||||
|
||||
These were the basic steps to applying the QAT method from the NNCF. However, it is required in some cases to save/load model
|
||||
These were the basic steps to applying the QAT method from the NNCF. However, it is required in some cases to save/load model
|
||||
checkpoints during the training. Since NNCF wraps the original model with its own object it provides an API for these needs.
|
||||
|
||||
7. (Optional) Save checkpoint
|
||||
|
|
@ -182,13 +182,13 @@ To save model checkpoint use the following API:
|
|||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [save_checkpoint]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -204,26 +204,26 @@ To restore the model from checkpoint you should use the following API:
|
|||
|
||||
.. tab-item:: PyTorch
|
||||
:sync: pytorch
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_torch.py
|
||||
:language: python
|
||||
:fragment: [load_checkpoint]
|
||||
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tensorflow-2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
:fragment: [load_checkpoint]
|
||||
|
||||
|
||||
For more details on saving/loading checkpoints in the NNCF, see the following `../../../documentation <https://github.com/openvinotoolkit/nncf/blob/develop/docs/Usage.md#saving-and-loading-compressed-models>`__.
|
||||
|
||||
For more details on saving/loading checkpoints in the NNCF, see the following `documentation <https://github.com/openvinotoolkit/nncf/blob/develop/docs/Usage.md#saving-and-loading-compressed-models>`__.
|
||||
|
||||
Deploying quantized model
|
||||
#########################
|
||||
|
||||
The quantized model can be deployed with OpenVINO in the same way as the baseline model. No extra steps or options are
|
||||
required in this case. For more details, see the corresponding :doc:`../../../documentation <../../running-inference>`.
|
||||
The quantized model can be deployed with OpenVINO in the same way as the baseline model. No extra steps or options are
|
||||
required in this case. For more details, see the corresponding :doc:`documentation <../../running-inference>`.
|
||||
|
||||
Examples
|
||||
####################
|
||||
|
|
|
|||
|
|
@ -96,8 +96,7 @@ Convert a Model with Python: ``convert_model``
|
|||
##############################################
|
||||
|
||||
Model Conversion API in Python uses the ``openvino.convert_model`` function, to turn a model
|
||||
into the `openvino.Model <api/ie_python_api/_autosummary/openvino.runtime.Model.html>`__
|
||||
object and load it to memory. Now it can be: saved to a drive with ``openvino.save_model``
|
||||
into the openvino.Model object and load it to memory. Now it can be: saved to a drive with ``openvino.save_model``
|
||||
or further :doc:`optimized with NNCF <model-optimization>` before saving.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ Conversion Parameters
|
|||
.. meta::
|
||||
:description: Model Conversion API provides several parameters to adjust model conversion.
|
||||
|
||||
This document describes all available parameters for ``openvino.convert_model``, ``ovc``, and ``openvino.save_model`` without focusing on a particular framework model format. Use this information for your reference as a common description of the conversion API capabilities in general. Part of the options can be not relevant to some specific frameworks. Use :doc:`Supported Model Formats <../model-preparation>` page for more dedicated framework-dependent ../../learn-openvino/interactive-tutorials-python.
|
||||
This document describes all available parameters for ``openvino.convert_model``, ``ovc``, and ``openvino.save_model`` without focusing on a particular framework model format. Use this information for your reference as a common description of the conversion API capabilities in general. Part of the options can be not relevant to some specific frameworks. Use :doc:`Supported Model Formats <../model-preparation>` page for more dedicated framework-dependent tutorials.
|
||||
|
||||
In most cases when it is required to convert a model the following simple syntax can be used:
|
||||
|
||||
|
|
|
|||
|
|
@ -144,5 +144,5 @@ For the list of supported standard layers, refer to the :doc:`Supported Operatio
|
|||
Additional Resources
|
||||
####################
|
||||
|
||||
Check out more examples of model conversion in :doc:`interactive Python ../../learn-openvino/interactive-tutorials-python <../../learn-openvino/interactive-tutorials-python>`.
|
||||
Check out more examples of model conversion in :doc:`interactive Python tutorials <../../learn-openvino/interactive-tutorials-python>`.
|
||||
|
||||
|
|
|
|||
|
|
@ -84,7 +84,7 @@ In practice, the code to evaluate or test the PyTorch model is usually provided
|
|||
category_name = weights.meta["categories"][class_id]
|
||||
print(f"{category_name}: {100 * score:.1f}% (with OpenVINO)")
|
||||
|
||||
Check out more examples in :doc:`interactive Python ../../learn-openvino/interactive-tutorials-python <../../learn-openvino/interactive-tutorials-python>`.
|
||||
Check out more examples in :doc:`interactive Python tutorials <../../learn-openvino/interactive-tutorials-python>`.
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -121,10 +121,10 @@ If your model has a ``dict`` input, such as, ``{"x": a, "y": b, "z": c}``, it wi
|
|||
|
||||
An important consequence of flattening is that only ``tuple`` and ``dict`` with a fixed number of elements and key values are supported. The structure of such inputs should be fully described in the ``example_input`` parameter of ``convert_model``. The flattening on outputs should be reproduced with the given ``example_input`` and cannot be changed once the conversion is done.
|
||||
|
||||
Check out more examples of model conversion with non-tensor data types in the following ../../learn-openvino/interactive-tutorials-python:
|
||||
Check out more examples of model conversion with non-tensor data types in the following tutorials:
|
||||
|
||||
* `Video Subtitle Generation using Whisper and OpenVINO™ <notebooks/227-whisper-subtitles-generation-with-output.html>`__
|
||||
* `Visual Question Answering and Image Captioning using BLIP and OpenVINO <notebooks/233-blip-visual-language-processing-with-output.html>`__
|
||||
* `Video Subtitle Generation using Whisper and OpenVINO™ <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/227-whisper-subtitles-generation>`__
|
||||
* `Visual Question Answering and Image Captioning using BLIP and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/233-blip-visual-language-processing>`__
|
||||
|
||||
|
||||
Exporting a PyTorch Model to ONNX Format
|
||||
|
|
|
|||
|
|
@ -56,8 +56,8 @@ Here are code examples of how to use these methods with different model formats:
|
|||
compiled_model = core.compile_model(ov_model, "AUTO")
|
||||
|
||||
For more details on conversion, refer to the
|
||||
:doc:`guide <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-pytorch>`
|
||||
and an example `tutorial <https://docs.openvino.ai/nightly/notebooks/102-pytorch-onnx-to-openvino-with-output.html>`__
|
||||
:doc:`guide <convert-model-pytorch>`
|
||||
and an example `tutorial <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/102-pytorch-to-openvino/102-pytorch-onnx-to-openvino.ipynb>`__
|
||||
on this topic.
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
|
|
@ -98,8 +98,8 @@ Here are code examples of how to use these methods with different model formats:
|
|||
compiled_model = core.compile_model(ov_model, "AUTO")
|
||||
|
||||
For more details on conversion, refer to the
|
||||
:doc:`guide <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-tensorflow>`
|
||||
and an example `tutorial <https://docs.openvino.ai/nightly/notebooks/101-tensorflow-to-openvino-with-output.html>`__
|
||||
:doc:`guide <convert-model-tensorflow>`
|
||||
and an example `tutorial <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/101-tensorflow-classification-to-openvino>`__
|
||||
on this topic.
|
||||
|
||||
* The ``read_model()`` and ``compile_model()`` methods:
|
||||
|
|
@ -172,7 +172,7 @@ Here are code examples of how to use these methods with different model formats:
|
|||
ovc <INPUT_MODEL>.pb
|
||||
|
||||
For details on the conversion, refer to the
|
||||
:doc:`article <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-tensorflow>`.
|
||||
:doc:`article <../model-preparation>`.
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
:sync: tflite
|
||||
|
|
@ -199,8 +199,8 @@ Here are code examples of how to use these methods with different model formats:
|
|||
compiled_model = core.compile_model(ov_model, "AUTO")
|
||||
|
||||
For more details on conversion, refer to the
|
||||
:doc:`guide <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-tensorflow>`
|
||||
and an example `tutorial <https://docs.openvino.ai/nightly/notebooks/119-tflite-to-openvino-with-output.html>`__
|
||||
:doc:`guide <convert-model-tensorflow-lite>`
|
||||
and an example `tutorial <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/119-tflite-to-openvino/119-tflite-to-openvino.ipynb>`__
|
||||
on this topic.
|
||||
|
||||
|
||||
|
|
@ -290,7 +290,7 @@ Here are code examples of how to use these methods with different model formats:
|
|||
ovc <INPUT_MODEL>.tflite
|
||||
|
||||
For details on the conversion, refer to the
|
||||
:doc:`article <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-tensorflow-lite>`.
|
||||
:doc:`article <convert-model-tensorflow-lite>`.
|
||||
|
||||
.. tab-item:: ONNX
|
||||
:sync: onnx
|
||||
|
|
@ -317,8 +317,8 @@ Here are code examples of how to use these methods with different model formats:
|
|||
compiled_model = core.compile_model(ov_model, "AUTO")
|
||||
|
||||
For more details on conversion, refer to the
|
||||
:doc:`guide <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-onnx>`
|
||||
and an example `tutorial <https://docs.openvino.ai/nightly/notebooks/102-pytorch-onnx-to-openvino-with-output.html>`__
|
||||
:doc:`guide <convert-model-onnx>`
|
||||
and an example `tutorial <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/102-pytorch-to-openvino/102-pytorch-onnx-to-openvino.ipynb>`__
|
||||
on this topic.
|
||||
|
||||
|
||||
|
|
@ -385,7 +385,7 @@ Here are code examples of how to use these methods with different model formats:
|
|||
ov_compiled_model_t* compiled_model = NULL;
|
||||
ov_core_compile_model_from_file(core, "<INPUT_MODEL>.onnx", "AUTO", 0, &compiled_model);
|
||||
|
||||
For details on the conversion, refer to the :doc:`article <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-onnx>`
|
||||
For details on the conversion, refer to the :doc:`article <convert-model-onnx>`
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli
|
||||
|
|
@ -405,7 +405,7 @@ Here are code examples of how to use these methods with different model formats:
|
|||
ovc <INPUT_MODEL>.onnx
|
||||
|
||||
For details on the conversion, refer to the
|
||||
:doc:`article <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-onnx>`
|
||||
:doc:`article <convert-model-onnx>`
|
||||
|
||||
.. tab-item:: PaddlePaddle
|
||||
:sync: pdpd
|
||||
|
|
@ -438,8 +438,8 @@ Here are code examples of how to use these methods with different model formats:
|
|||
compiled_model = core.compile_model(ov_model, "AUTO")
|
||||
|
||||
For more details on conversion, refer to the
|
||||
:doc:`guide <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-paddle>`
|
||||
and an example `tutorial <https://docs.openvino.ai/nightly/notebooks/103-paddle-to-openvino-classification-with-output.html>`__
|
||||
:doc:`guide <convert-model-paddle>`
|
||||
and an example `tutorial <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/103-paddle-to-openvino/103-paddle-to-openvino-classification.ipynb>`__
|
||||
on this topic.
|
||||
|
||||
* The ``read_model()`` method:
|
||||
|
|
@ -527,7 +527,7 @@ Here are code examples of how to use these methods with different model formats:
|
|||
ovc <INPUT_MODEL>.pdmodel
|
||||
|
||||
For details on the conversion, refer to the
|
||||
:doc:`article <../../documentation/legacy-features/transition-legacy-conversion-api/legacy-conversion-api/[legacy]-supported-model-formats/[legacy]-convert-paddle>`.
|
||||
:doc:`article <convert-model-paddle>`.
|
||||
|
||||
|
||||
* :doc:`How to convert PyTorch <convert-model-pytorch>`
|
||||
|
|
|
|||
Loading…
Reference in New Issue