natas_hw
/
openvino
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[DOCS] `convert_model()` as a default conversion path (#17454)

This commit is contained in:
Sebastian Golebiewski 2023-05-31 17:53:22 +02:00 committed by GitHub
parent b779dc3246
commit 6c287986b7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
93 changed files with 1220 additions and 907 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
docs/Documentation/model_introduction.md
View File

@ -10,22 +10,48 @@
omz_tools_downloader
Every deep learning workflow begins with obtaining a model. You can choose to prepare a custom one, use a ready-made solution and adjust it to your needs, or even download and run a pre-trained network from an online database, such as OpenVINO's :doc:`Open Model Zoo <model_zoo>`.
Every deep learning workflow begins with obtaining a model. You can choose to prepare a custom one, use a ready-made solution and adjust it to your needs, or even download and run a pre-trained network from an online database, such as `TensorFlow Hub <https://tfhub.dev/>`__, `Hugging Face <https://huggingface.co/>`__, `Torchvision models <https://pytorch.org/hub/>`__.
:doc:`OpenVINO™ supports several model formats <Supported_Model_Formats>` and allows to convert them to it's own, OpenVINO IR, providing a tool dedicated to this task.
:doc:`OpenVINO™ supports several model formats <Supported_Model_Formats>` and allows converting them to it's own, `openvino.runtime.Model <api/ie_python_api/_autosummary/openvino.runtime.Model.html>`__ (`ov.Model <api/ie_python_api/_autosummary/openvino.runtime.Model.html>`__ ), providing a tool dedicated to this task.
:doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` reads the original model and creates the OpenVINO IR model (.xml and .bin files) so that inference can ultimately be performed without delays due to format conversion. Optionally, Model Optimizer can adjust the model to be more suitable for inference, for example, by :doc:`alternating input shapes <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`, :doc:`embedding preprocessing <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>` and :doc:`cutting training parts off <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>`.
There are several options to convert a model from original framework to OpenVINO model format (``ov.Model``).
The approach to fully convert a model is considered the default choice, as it allows the full extent of OpenVINO features. The OpenVINO IR model format is used by other conversion and preparation tools, such as the Post-Training Optimization Tool, for further optimization of the converted model.
The ``read_model()`` method reads a model from a file and produces ``ov.Model``. If the file is in one of the supported original framework file formats, it is converted automatically to OpenVINO Intermediate Representation. If the file is already in the OpenVINO IR format, it is read "as-is", without any conversion involved. ``ov.Model`` can be serialized to IR using the ``ov.serialize()`` method. The serialized IR can be further optimized using :doc:`Post-Training Optimization tool <pot_introduction>` that applies post-training quantization methods.
Conversion is not required for ONNX, PaddlePaddle, TensorFlow Lite and TensorFlow models (check :doc:`TensorFlow Frontend Capabilities and Limitations <openvino_docs_MO_DG_TensorFlow_Frontend>`), as OpenVINO provides C++ and Python APIs for importing them to OpenVINO Runtime directly. It provides a convenient way to quickly switch from framework-based code to OpenVINO-based code in your inference application.
Convert a model in Python
######################################
Model conversion API, specifically, the ``mo.convert_model()`` method converts a model from original framework to ``ov.Model``. ``mo.convert_model()`` returns ``ov.Model`` object in memory so the ``read_model()`` method is not required. The resulting ``ov.Model`` can be inferred in the same training environment (python script or Jupiter Notebook). ``mo.convert_model()`` provides a convenient way to quickly switch from framework-based code to OpenVINO-based code in your inference application. In addition to model files, ``mo.convert_model()`` can take OpenVINO extension objects constructed directly in Python for easier conversion of operations that are not supported in OpenVINO. The ``mo.convert_model()`` method also has a set of parameters to :doc:`cut the model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>`, :doc:`set input shapes or layout <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`, :doc:`add preprocessing <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>`, etc.
.. image:: _static/images/model_conversion_diagram.svg
:alt: model conversion diagram
Convert a model with ``mo`` command-line tool
#############################################
Another option to convert a model is to use ``mo`` command-line tool. ``mo`` is a cross-platform tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices in the same measure, as the ``mo.convert_model`` method.
``mo`` requires the use of a pre-trained deep learning model in one of the supported formats: TensorFlow, TensorFlow Lite, PaddlePaddle, or ONNX. ``mo`` converts the model to the OpenVINO Intermediate Representation format (IR), which needs to be read with the ``ov.read_model()`` method. Then, you can compile and infer the ``ov.Model`` later with :doc:`OpenVINO™ Runtime <openvino_docs_OV_UG_OV_Runtime_User_Guide>`.
The figure below illustrates the typical workflow for deploying a trained deep learning model:
.. image:: _static/images/BASIC_FLOW_MO_simplified.svg
where IR is a pair of files describing the model:
* ``.xml`` - Describes the network topology.
* ``.bin`` - Contains the weights and biases binary data.
Model files (not Python objects) from ONNX, PaddlePaddle, TensorFlow and TensorFlow Lite (check :doc:`TensorFlow Frontend Capabilities and Limitations <openvino_docs_MO_DG_TensorFlow_Frontend>`) do not require a separate step for model conversion, that is ``mo.convert_model``. OpenVINO provides C++ and Python APIs for importing the models to OpenVINO Runtime directly by just calling the ``read_model`` method.
The results of both ``mo`` and ``mo.convert_model()`` conversion methods described above are the same. You can choose one of them, depending on what is most convenient for you. Keep in mind that there should not be any differences in the results of model conversion if the same set of parameters is used.
This section describes how to obtain and prepare your model for work with OpenVINO to get the best inference results:
* :doc:`See the supported formats and how to use them in your project <Supported_Model_Formats>`.
* :doc:`Convert different model formats to the OpenVINO IR format <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
* :doc:`Automate model-related tasks with Model Downloader and additional OMZ Tools <omz_tools_downloader>`.
To begin with, you may want to :doc:`browse a database of models for use in your projects <model_zoo>`.
* :doc:`Convert different model formats to the ov.Model format <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
@endsphinxdirective

2
docs/Documentation/openvino_workflow.md
View File

@ -15,7 +15,7 @@
| :doc:`Model Preparation <openvino_docs_model_processing_introduction>`
| With Model Downloader and Model Optimizer guides, you will learn to download pre-trained models and convert them for use with OpenVINO™. You can use your own models or choose some from a broad selection provided in the Open Model Zoo.
| With model conversion API guide, you will learn to convert pre-trained models for use with OpenVINO™. You can use your own models or choose some from a broad selection in online databases, such as `TensorFlow Hub <https://tfhub.dev/>`__, `Hugging Face <https://huggingface.co/>`__, `Torchvision models <https://pytorch.org/hub/>`__..
| :doc:`Model Optimization and Compression <openvino_docs_model_optimization_guide>`
| In this section you will find out how to optimize a model to achieve better inference performance. It describes multiple optimization methods for both the training and post-training stages.

195
docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md
View File

@ -1,4 +1,4 @@
# Model Optimizer Usage {#openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide}
# Convert a Model {#openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide}
@sphinxdirective
@ -8,7 +8,6 @@
:maxdepth: 1
:hidden:
openvino_docs_model_inputs_outputs
openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model
openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model
openvino_docs_MO_DG_Additional_Optimization_Use_Cases
@ -17,86 +16,126 @@
openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ
Model Optimizer is a cross-platform command-line tool that facilitates the transition between training and deployment environments,
performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
To convert a model to OpenVINO model format (``ov.Model``), you can use the following command:
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model(INPUT_MODEL)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model INPUT_MODEL
To use it, you need a pre-trained deep learning model in one of the supported formats:
TensorFlow, PyTorch, ONNX, TensorFlow Lite, and PaddlePaddle (OpenVINO support for Apache MXNet, Caffe, and Kaldi is currently
being deprecated and will be removed entirely in the future). Model Optimizer converts the model to the OpenVINO Intermediate Representation format (IR),
which you can infer later with :doc:`OpenVINO™ Runtime <openvino_docs_OV_UG_OV_Runtime_User_Guide>`.
If the out-of-the-box conversion (only the ``input_model`` parameter is specified) is not successful, use the parameters mentioned below to override input shapes and cut the model:
Note that Model Optimizer does not infer models.
The figure below illustrates the typical workflow for deploying a trained deep learning model:
.. image:: _static/images/BASIC_FLOW_MO_simplified.svg
where IR is a pair of files describing the model:
* ``.xml`` - Describes the network topology.
* ``.bin`` - Contains the weights and biases binary data.
The OpenVINO IR can be additionally optimized for inference by :doc:`Post-training optimization <pot_introduction>` that applies post-training quantization methods.
How to Run Model Optimizer
##########################
To convert a model to IR, you can run Model Optimizer by using the following command:
.. code-block:: sh
mo --input_model INPUT_MODEL
If the out-of-the-box conversion (only the ``--input_model`` parameter is specified) is not successful, use the parameters mentioned below to override input shapes and cut the model:
- Model Optimizer provides two parameters to override original input shapes for model conversion: ``--input`` and ``--input_shape``.
- model conversion API provides two parameters to override original input shapes for model conversion: ``input`` and ``input_shape``.
For more information about these parameters, refer to the :doc:`Setting Input Shapes <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
- To cut off unwanted parts of a model (such as unsupported operations and training sub-graphs),
use the ``--input`` and ``--output`` parameters to define new inputs and outputs of the converted model.
use the ``input`` and ``output`` parameters to define new inputs and outputs of the converted model.
For a more detailed description, refer to the :doc:`Cutting Off Parts of a Model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>` guide.
You can also insert additional input pre-processing sub-graphs into the converted model by using
the ``--mean_values``, ``scales_values``, ``--layout``, and other parameters described
the ``mean_values``, ``scales_values``, ``layout``, and other parameters described
in the :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>` article.
The ``--compress_to_fp16`` compression parameter in Model Optimizer allows generating IR with constants (for example, weights for convolutions and matrix multiplications) compressed to ``FP16`` data type. For more details, refer to the :doc:`Compression of a Model to FP16 <openvino_docs_MO_DG_FP16_Compression>` guide.
The ``compress_to_fp16`` compression parameter in ``mo`` command-line tool allows generating IR with constants (for example, weights for convolutions and matrix multiplications) compressed to ``FP16`` data type. For more details, refer to the :doc:`Compression of a Model to FP16 <openvino_docs_MO_DG_FP16_Compression>` guide.
To get the full list of conversion parameters available in Model Optimizer, run the following command:
To get the full list of conversion parameters, run the following command:
.. code-block:: sh
.. tab-set::
mo --help
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model(help=True)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --help
Examples of CLI Commands
########################
Examples of model conversion parameters
#######################################
Below is a list of separate examples for different frameworks and Model Optimizer parameters:
Below is a list of separate examples for different frameworks and model conversion parameters:
1. Launch Model Optimizer for a TensorFlow MobileNet model in the binary protobuf format:
1. Launch model conversion for a TensorFlow MobileNet model in the binary protobuf format:
.. code-block:: sh
.. tab-set::
mo --input_model MobileNet.pb
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("MobileNet.pb")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model MobileNet.pb
Launch Model Optimizer for a TensorFlow BERT model in the SavedModel format with three inputs. Specify input shapes explicitly where the batch size and the sequence length equal 2 and 30 respectively:
Launch model conversion for a TensorFlow BERT model in the SavedModel format with three inputs. Specify input shapes explicitly where the batch size and the sequence length equal 2 and 30 respectively:
.. code-block:: sh
.. tab-set::
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
.. tab-item:: Python
:sync: mo-python-api
For more information, refer to the :doc:`Converting a TensorFlow Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
.. code-block:: python
2. Launch Model Optimizer for an ONNX OCR model and specify new output explicitly:
from openvino.tools.mo import convert_model
ov_model = convert_model("BERT", input_shape=[[2,30],[2,30],[2,30]])
.. code-block:: sh
.. tab-item:: CLI
:sync: cli-tool
mo --input_model ocr.onnx --output probabilities
.. code-block:: sh
mo --saved_model_dir BERT --input_shape [2,30],[2,30],[2,30]
For more information, refer to the :doc:`Converting a TensorFlow Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
2. Launch model conversion for an ONNX OCR model and specify new output explicitly:
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("ocr.onnx", output="probabilities")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model ocr.onnx --output probabilities
For more information, refer to the :doc:`Converting an ONNX Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX>` guide.
@ -105,43 +144,29 @@ Below is a list of separate examples for different frameworks and Model Optimize
PyTorch models must be exported to the ONNX format before conversion into IR. More information can be found in :doc:`Converting a PyTorch Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch>`.
3. Launch Model Optimizer for a PaddlePaddle UNet model and apply mean-scale normalization to the input:
3. Launch model conversion for a PaddlePaddle UNet model and apply mean-scale normalization to the input:
.. code-block:: sh
.. tab-set::
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("unet.pdmodel", mean_values=[123,117,104], scale=255)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
For more information, refer to the :doc:`Converting a PaddlePaddle Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Paddle>` guide.
4. Launch Model Optimizer for an Apache MXNet SSD Inception V3 model and specify first-channel layout for the input:
.. code-block:: sh
mo --input_model ssd_inception_v3-0000.params --layout NCHW
For more information, refer to the :doc:`Converting an Apache MXNet Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet>` guide.
5. Launch Model Optimizer for a Caffe AlexNet model with input channels in the RGB format which needs to be reversed:
.. code-block:: sh
mo --input_model alexnet.caffemodel --reverse_input_channels
For more information, refer to the :doc:`Converting a Caffe Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe>` guide.
6. Launch Model Optimizer for a Kaldi LibriSpeech nnet2 model:
.. code-block:: sh
mo --input_model librispeech_nnet2.mdl --input_shape [1,140]
For more information, refer to the :doc:`Converting a Kaldi Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi>` guide.
- To get conversion recipes for specific TensorFlow, ONNX, PyTorch, Apache MXNet, and Kaldi models, refer to the :doc:`Model Conversion Tutorials <openvino_docs_MO_DG_prepare_model_convert_model_tutorials>`.
- To get conversion recipes for specific TensorFlow, ONNX, and PyTorch models, refer to the :doc:`Model Conversion Tutorials <openvino_docs_MO_DG_prepare_model_convert_model_tutorials>`.
- For more information about IR, see :doc:`Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™ <openvino_docs_MO_DG_IR_and_opsets>`.
- For more information about support of neural network models trained with various frameworks, see :doc:`OpenVINO Extensibility Mechanism <openvino_docs_Extensibility_UG_Intro>`

217
docs/MO_DG/prepare_model/Additional_Optimizations.md
View File

@ -2,149 +2,237 @@
@sphinxdirective
Input data for inference can be different from the training dataset and requires
additional preprocessing before inference. To accelerate the whole pipeline including
preprocessing and inference, Model Optimizer provides special parameters such as ``--mean_values``,
Input data for inference can be different from the training dataset and requires
additional preprocessing before inference. To accelerate the whole pipeline including
preprocessing and inference, model conversion API provides special parameters such as ``mean_values``,
``--scale_values``, ``--reverse_input_channels``, and ``--layout``. Based on these
parameters, Model Optimizer generates OpenVINO IR with additionally inserted sub-graphs
to perform the defined preprocessing. This preprocessing block can perform mean-scale
normalization of input data, reverting data along channel dimension, and changing
the data layout. See the following sections for details on the parameters, or the
:doc:`Overview of Preprocessing API <openvino_docs_OV_UG_Preprocessing_Overview>`
``scale_values``, ``reverse_input_channels``, and ``layout``. Based on these
parameters, model conversion API generates OpenVINO IR with additionally inserted sub-graphs
to perform the defined preprocessing. This preprocessing block can perform mean-scale
normalization of input data, reverting data along channel dimension, and changing
the data layout. See the following sections for details on the parameters, or the
:doc:`Overview of Preprocessing API <openvino_docs_OV_UG_Preprocessing_Overview>`
for the same functionality in OpenVINO Runtime.
Specifying Layout
#################
You may need to set input layouts, as it is required by some preprocessing, for
You may need to set input layouts, as it is required by some preprocessing, for
example, setting a batch, applying mean or scales, and reversing input channels (BGR<->RGB).
Layout defines the meaning of dimensions in shape and can be specified for both
inputs and outputs. Some preprocessing requires to set input layouts, for example,
Layout defines the meaning of dimensions in shape and can be specified for both
inputs and outputs. Some preprocessing requires to set input layouts, for example,
setting a batch, applying mean or scales, and reversing input channels (BGR<->RGB).
For the layout syntax, check the :doc:`Layout API overview <openvino_docs_OV_UG_Layout_Overview>`.
To specify the layout, you can use the ``--layout`` option followed by the layout value.
To specify the layout, you can use the ``layout`` option followed by the layout value.
For example, the following command specifies the ``NHWC`` layout for a Tensorflow
For example, the following command specifies the ``NHWC`` layout for a Tensorflow
``nasnet_large`` model that was exported to the ONNX format:
.. code-block:: sh
mo --input_model tf_nasnet_large.onnx --layout nhwc
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("tf_nasnet_large.onnx", layout="nhwc")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model tf_nasnet_large.onnx --layout nhwc
Additionally, if a model has more than one input or needs both input and output
Additionally, if a model has more than one input or needs both input and output
layouts specified, you need to provide the name of each input or output to apply the layout.
For example, the following command specifies the layout for an ONNX ``Yolo v3 Tiny``
model with its first input ``input_1`` in ``NCHW`` layout and second input ``image_shape``
For example, the following command specifies the layout for an ONNX ``Yolo v3 Tiny``
model with its first input ``input_1`` in ``NCHW`` layout and second input ``image_shape``
having two dimensions: batch and size of the image expressed as the ``N?`` layout:
.. code-block:: sh
.. tab-set::
mo --input_model yolov3-tiny.onnx --layout input_1(nchw),image_shape(n?)
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("yolov3-tiny.onnx", layout={"input_1": "nchw", "image_shape": "n?"})
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model yolov3-tiny.onnx --layout input_1(nchw),image_shape(n?)
Changing Model Layout
#####################
Changing the model layout may be necessary if it differs from the one presented by input data.
Use either ``--layout`` or ``--source_layout`` with ``--target_layout`` to change the layout.
Changing the model layout may be necessary if it differs from the one presented by input data.
Use either ``layout`` or ``source_layout`` with ``target_layout`` to change the layout.
For example, for the same ``nasnet_large`` model mentioned previously, you can use
For example, for the same ``nasnet_large`` model mentioned previously, you can use
the following commands to provide data in the ``NCHW`` layout:
.. code-block:: sh
mo --input_model tf_nasnet_large.onnx --source_layout nhwc --target_layout nchw
mo --input_model tf_nasnet_large.onnx --layout "nhwc->nchw"
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("tf_nasnet_large.onnx", source_layout="nhwc", target_layout="nchw")
ov_model = convert_model("tf_nasnet_large.onnx", layout="nhwc->nchw")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model tf_nasnet_large.onnx --source_layout nhwc --target_layout nchw
mo --input_model tf_nasnet_large.onnx --layout "nhwc->nchw"
Again, if a model has more than one input or needs both input and output layouts
Again, if a model has more than one input or needs both input and output layouts
specified, you need to provide the name of each input or output to apply the layout.
For example, to provide data in the ``NHWC`` layout for the `Yolo v3 Tiny` model
For example, to provide data in the ``NHWC`` layout for the `Yolo v3 Tiny` model
mentioned earlier, use the following commands:
.. code-block:: sh
.. tab-set::
mo --input_model yolov3-tiny.onnx --source_layout "input_1(nchw),image_shape(n?)" --target_layout "input_1(nhwc)"
mo --input_model yolov3-tiny.onnx --layout "input_1(nchw->nhwc),image_shape(n?)"
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("yolov3-tiny.onnx", source_layout={"input_1": "nchw", "image_shape": "n?"}, target_layout={"input_1": "nhwc"})
ov_model = convert_model("yolov3-tiny.onnx", layout={"input_1": "nchw->nhwc", "image_shape": "n?"}
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model yolov3-tiny.onnx --source_layout "input_1(nchw),image_shape(n?)" --target_layout "input_1(nhwc)"
mo --input_model yolov3-tiny.onnx --layout "input_1(nchw->nhwc),image_shape(n?)"
Specifying Mean and Scale Values
################################
Neural network models are usually trained with the normalized input data. This
means that the input data values are converted to be in a specific range, for example,
``[0, 1]`` or ``[-1, 1]``. Sometimes, the mean values (mean images) are subtracted
Neural network models are usually trained with the normalized input data. This
means that the input data values are converted to be in a specific range, for example,
``[0, 1]`` or ``[-1, 1]``. Sometimes, the mean values (mean images) are subtracted
from the input data values as part of the preprocessing.
There are two cases of how the input data preprocessing is implemented.
* The input preprocessing operations are a part of a model.
In this case, the application does not perform a separate preprocessing step:
everything is embedded into the model itself. Model Optimizer will generate the
OpenVINO IR format with required preprocessing operations, and no ``mean`` and
In this case, the application does not perform a separate preprocessing step:
everything is embedded into the model itself. ``convert_model()`` will generate the
ov.Model with required preprocessing operations, and no ``mean`` and
``scale`` parameters are required.
* The input preprocessing operations are not a part of a model and the preprocessing
* The input preprocessing operations are not a part of a model and the preprocessing
is performed within the application which feeds the model with input data.
In this case, information about mean/scale values should be provided to Model
Optimizer to embed it to the generated OpenVINO IR format.
In this case, information about mean/scale values should be provided to ``convert_model()``
to embed it to the generated ``ov.Model``.
Model Optimizer provides command-line parameters to specify the values: ``--mean_values``,
``--scale_values``, ``--scale``. Using these parameters, Model Optimizer embeds the
corresponding preprocessing block for mean-value normalization of the input data
and optimizes this block so that the preprocessing takes negligible time for inference.
Model conversion API represented by ``convert_model()`` provides command-line parameters
to specify the values: ``mean_values``, ``scale_values``, ``scale``. Using these parameters,
model conversion API embeds the corresponding preprocessing block for mean-value
normalization of the input data and optimizes this block so that the preprocessing
takes negligible time for inference.
For example, the following command runs Model Optimizer for the PaddlePaddle UNet
For example, the following command runs model conversion for the PaddlePaddle UNet
model and applies mean-scale normalization to the input data:
.. code-block:: sh
.. tab-set::
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("unet.pdmodel", mean_values=[123,117,104], scale=255)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
Reversing Input Channels
########################
Sometimes, input images for your application can be of the RGB (or BGR) format
and the model is trained on images of the BGR (or RGB) format, which is in the
opposite order of color channels. In this case, it is important to preprocess the
Sometimes, input images for your application can be of the RGB (or BGR) format
and the model is trained on images of the BGR (or RGB) format, which is in the
opposite order of color channels. In this case, it is important to preprocess the
input images by reverting the color channels before inference.
To embed this preprocessing step into OpenVINO IR, Model Optimizer provides the
``--reverse_input_channels`` command-line parameter to shuffle the color channels.
To embed this preprocessing step into ``ov.Model``, model conversion API provides the
``reverse_input_channels`` command-line parameter to shuffle the color channels.
The ``--reverse_input_channels`` parameter can be used to preprocess the model
The ``reverse_input_channels`` parameter can be used to preprocess the model
input in the following cases:
* Only one dimension in the input shape has a size equal to ``3``.
* One dimension has an undefined size and is marked as ``C`` channel using ``layout`` parameters.
Using the ``--reverse_input_channels`` parameter, Model Optimizer embeds the corresponding
preprocessing block for reverting the input data along channel dimension and optimizes
Using the ``reverse_input_channels`` parameter, model conversion API embeds the corresponding
preprocessing block for reverting the input data along channel dimension and optimizes
this block so that the preprocessing takes only negligible time for inference.
For example, the following command launches Model Optimizer for the TensorFlow AlexNet
For example, the following command launches model conversion for the TensorFlow AlexNet
model and embeds the ``reverse_input_channel`` preprocessing block into OpenVINO IR:
.. code-block:: sh
mo --input_model alexnet.pb --reverse_input_channels
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("alexnet.pb", reverse_input_channels=True)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model alexnet.pb --reverse_input_channels
.. note::
If both mean and scale values are specified, the mean is subtracted first and
then the scale is applied regardless of the order of options in the command-line.
Input values are *divided* by the scale value(s). If the ``--reverse_input_channels``
option is also used, ``reverse_input_channels`` will be applied first, then ``mean``
and after that ``scale``. The data flow in the model looks as follows:
If both mean and scale values are specified, the mean is subtracted first and
then the scale is applied regardless of the order of options in the command-line.
Input values are *divided* by the scale value(s). If the ``reverse_input_channels``
option is also used, ``reverse_input_channels`` will be applied first, then ``mean``
and after that ``scale``. The data flow in the model looks as follows:
``Parameter -> ReverseInputChannels -> Mean apply-> Scale apply -> the original body of the model``.
Additional Resources
@ -153,4 +241,3 @@ Additional Resources
* :doc:`Overview of Preprocessing API <openvino_docs_OV_UG_Preprocessing_Overview>`
@endsphinxdirective

23
docs/MO_DG/prepare_model/FP16_Compression.md
View File

@ -2,16 +2,29 @@
@sphinxdirective
Model Optimizer can convert all floating-point weights to the ``FP16`` data type.
Optionally all relevant floating-point weights can be compressed to ``FP16`` data type during the model conversion.
It results in creating a "compressed ``FP16`` model", which occupies about half of
the original space in the file system. The compression may introduce a drop in accuracy.
but it is negligible for most models.
To compress the model, use the `--compress_to_fp16` or `--compress_to_fp16=True` option:
To compress the model, use the ``compress_to_fp16=True`` option:
.. code-block:: sh
.. tab-set::
mo --input_model INPUT_MODEL --compress_to_fp16
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model(INPUT_MODEL, compress_to_fp16=False)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model INPUT_MODEL --compress_to_fp16=False
For details on how plugins handle compressed ``FP16`` models, see
@ -28,7 +41,7 @@ For details on how plugins handle compressed ``FP16`` models, see
Some large models (larger than a few GB) when compressed to ``FP16`` may consume enormous amount of RAM on the loading
phase of the inference. In case if you are facing such problems, please try to convert them without compression:
`mo --input_model INPUT_MODEL --compress_to_fp16=False`
``convert_model(INPUT_MODEL, compress_to_fp16=False)`` or ``convert_model(INPUT_MODEL)``
@endsphinxdirective

11
docs/MO_DG/prepare_model/MO_Python_API.md
View File

@ -1,10 +1,15 @@
# Model Optimizer Python API {#openvino_docs_MO_DG_Python_API}
# Convert Models Represented as Python Objects {#openvino_docs_MO_DG_Python_API}
@sphinxdirective
Model Optimizer (MO) has a Python API for model conversion, which is represented by the ``convert_model()`` method in the openvino.tools.mo namespace.
Model conversion API is represented by ``convert_model()`` method in openvino.tools.mo namespace. ``convert_model()`` is compatible with types from openvino.runtime, like PartialShape, Layout, Type, etc.
``convert_model()`` has the ability available from the command-line tool, plus the ability to pass Python model objects, such as a Pytorch model or TensorFlow Keras model directly, without saving them into files and without leaving the training environment (Jupyter Notebook or training scripts). In addition to input models consumed directly from Python, ``convert_model`` can take OpenVINO extension objects constructed directly in Python for easier conversion of operations that are not supported in OpenVINO.
.. note::
Model conversion can be performed by the ``convert_model()`` method and MO command line tool. The functionality from this article is applicable for ``convert_model()`` only and it is not present in command line tool.
``convert_model()`` has all the functionality available from the command-line tool, plus the ability to pass Python model objects, such as a Pytorch model or TensorFlow Keras model directly, without saving them into files and without leaving the training environment (Jupyter Notebook or training scripts). In addition to input models consumed directly from Python, ``convert_model`` can take OpenVINO extension objects constructed directly in Python for easier conversion of operations that are not supported in OpenVINO.
``convert_model()`` returns an openvino.runtime.Model object which can be compiled and inferred or serialized to IR.

5
docs/MO_DG/prepare_model/Model_Optimizer_FAQ.md
View File

@ -2,6 +2,10 @@
@sphinxdirective
.. important::
All of the issues below refer to :doc:`legacy functionalities <openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer>`.
If your question is not covered by the topics below, use the
`OpenVINO Support page <https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit>`__,
where you can participate in a free forum discussion.
@ -11,7 +15,6 @@ where you can participate in a free forum discussion.
Note that OpenVINO support for Apache MXNet, Caffe, and Kaldi is currently being deprecated.
As legacy formats, they will not be supported as actively as the main frontends and will be removed entirely in the future.
.. _question-1:
Q1. What does the message "[ ERROR ]: Current caffe.proto does not contain field" mean?

17
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Caffe.md
View File

@ -7,8 +7,7 @@
Note that OpenVINO support for Caffe is currently being deprecated and will be removed entirely in the future.
To convert a Caffe model, run Model Optimizer with the path to the input model ``.caffemodel`` file:
To convert a Caffe model, run ``mo`` with the path to the input model ``.caffemodel`` file:
.. code-block:: cpp
@ -44,13 +43,13 @@ The following list provides the Caffe-specific parameters.
CLI Examples Using Caffe-Specific Parameters
++++++++++++++++++++++++++++++++++++++++++++
* Launching Model Optimizer for `bvlc_alexnet.caffemodel <https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet>`__ with a specified `prototxt` file. This is needed when the name of the Caffe model and the `.prototxt` file are different or are placed in different directories. Otherwise, it is enough to provide only the path to the input `model.caffemodel` file.
* Launching model conversion for `bvlc_alexnet.caffemodel <https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet>`__ with a specified `prototxt` file. This is needed when the name of the Caffe model and the `.prototxt` file are different or are placed in different directories. Otherwise, it is enough to provide only the path to the input `model.caffemodel` file.
.. code-block:: cpp
mo --input_model bvlc_alexnet.caffemodel --input_proto bvlc_alexnet.prototxt
* Launching Model Optimizer for `bvlc_alexnet.caffemodel <https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet>`__ with a specified `CustomLayersMapping` file. This is the legacy method of quickly enabling model conversion if your model has custom layers. This requires the Caffe system on the computer. Example of ``CustomLayersMapping.xml`` can be found in ``<OPENVINO_INSTALLATION_DIR>/mo/front/caffe/CustomLayersMapping.xml.example``. The optional parameters without default values and not specified by the user in the ``.prototxt`` file are removed from the Intermediate Representation, and nested parameters are flattened:
* Launching model conversion for `bvlc_alexnet.caffemodel <https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet>`__ with a specified `CustomLayersMapping` file. This is the legacy method of quickly enabling model conversion if your model has custom layers. This requires the Caffe system on the computer. Example of ``CustomLayersMapping.xml`` can be found in ``<OPENVINO_INSTALLATION_DIR>/mo/front/caffe/CustomLayersMapping.xml.example``. The optional parameters without default values and not specified by the user in the ``.prototxt`` file are removed from the Intermediate Representation, and nested parameters are flattened:
.. code-block:: cpp
@ -77,7 +76,7 @@ CLI Examples Using Caffe-Specific Parameters
}
}
* Launching the Model Optimizer for a multi-input model with two inputs and providing a new shape for each input in the order they are passed to the Model Optimizer. In particular, for data, set the shape to ``1,3,227,227``. For rois, set the shape to ``1,6,1,1``:
* Launching model conversion for a multi-input model with two inputs and providing a new shape for each input in the order they are passed to the model conversion API. In particular, for data, set the shape to ``1,3,227,227``. For rois, set the shape to ``1,6,1,1``:
.. code-block:: cpp
@ -86,7 +85,7 @@ CLI Examples Using Caffe-Specific Parameters
Custom Layer Definition
########################
Internally, when you run Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list. If your topology contains such kind of layers, Model Optimizer classifies them as custom.
For the definition of custom layers, refer to the :doc:`Cutting Off Parts of a Model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>` page.
Supported Caffe Layers
#######################
@ -96,16 +95,16 @@ For the list of supported standard layers, refer to the :doc:`Supported Operatio
Frequently Asked Questions (FAQ)
################################
Model Optimizer provides explanatory messages when it is unable to complete conversions due to typographical errors, incorrectly used options, or other issues. A message describes the potential cause of the problem and gives a link to :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>` which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections to help you understand what went wrong.
Model conversion API provides explanatory messages when it is unable to complete conversions due to typographical errors, incorrectly used options, or other issues. A message describes the potential cause of the problem and gives a link to :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>` which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`to help you understand what went wrong.
Summary
#######
In this document, you learned:
* Basic information about how the Model Optimizer works with Caffe models.
* Basic information about how model conversion works with Caffe models.
* Which Caffe models are supported.
* How to convert a trained Caffe model by using Model Optimizer with both framework-agnostic and Caffe-specific command-line options.
* How to convert a trained Caffe model by using model conversion API with both framework-agnostic and Caffe-specific command-line parameters.
Additional Resources
####################

20
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md
View File

@ -9,9 +9,9 @@
.. note::
Model Optimizer supports the `nnet1 <http://kaldi-asr.org/doc/dnn1.html>`__ and `nnet2 <http://kaldi-asr.org/doc/dnn2.html>`__ formats of Kaldi models. The support of the `nnet3 <http://kaldi-asr.org/doc/dnn3.html>`__ format is limited.
Model conversion API supports the `nnet1 <http://kaldi-asr.org/doc/dnn1.html>`__ and `nnet2 <http://kaldi-asr.org/doc/dnn2.html>`__ formats of Kaldi models. The support of the `nnet3 <http://kaldi-asr.org/doc/dnn3.html>`__ format is limited.
To convert a Kaldi model, run Model Optimizer with the path to the input model ``.nnet`` or ``.mdl`` file:
To convert a Kaldi model, run model conversion with the path to the input model ``.nnet`` or ``.mdl`` file:
.. code-block:: cpp
@ -33,20 +33,20 @@ The following list provides the Kaldi-specific parameters.
Examples of CLI Commands
########################
* To launch Model Optimizer for the ``wsj_dnn5b_smbr`` model with the specified ``.nnet`` file:
* To launch model conversion for the ``wsj_dnn5b_smbr`` model with the specified ``.nnet`` file:
.. code-block:: cpp
mo --input_model wsj_dnn5b_smbr.nnet
* To launch Model Optimizer for the ``wsj_dnn5b_smbr`` model with the existing file that contains counts for the last layer with biases:
* To launch model conversion for the ``wsj_dnn5b_smbr`` model with the existing file that contains counts for the last layer with biases:
.. code-block:: cpp
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts
* The Model Optimizer normalizes сounts in the following way:
* The model conversion normalizes сounts in the following way:
.. math::
@ -60,19 +60,19 @@ Examples of CLI Commands
* The normalized counts are subtracted from biases of the last or next to last layer (if last layer is SoftMax).
.. note:: Model Optimizer will show a warning if a model contains values of counts and the `--counts` option is not used.
.. note:: Model conversion API will show a warning if a model contains values of counts and the ``counts`` option is not used.
* If you want to remove the last SoftMax layer in the topology, launch the Model Optimizer with the `--remove_output_softmax` flag:
* If you want to remove the last SoftMax layer in the topology, launch the model conversion with the ``remove_output_softmax`` flag:
.. code-block:: cpp
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --remove_output_softmax
The Model Optimizer finds the last layer of the topology and removes this layer only if it is a SoftMax layer.
Model conversion API finds the last layer of the topology and removes this layer only if it is a SoftMax layer.
.. note:: Model Optimizer can remove SoftMax layer only if the topology has one output.
.. note:: Model conversion can remove SoftMax layer only if the topology has one output.
* You can use the *OpenVINO Speech Recognition* sample application for the sample inference of Kaldi models. This sample supports models with only one output. If your model has several outputs, specify the desired one with the ``--output`` option.
* You can use the *OpenVINO Speech Recognition* sample application for the sample inference of Kaldi models. This sample supports models with only one output. If your model has several outputs, specify the desired one with the ``output`` option.
Supported Kaldi Layers
######################

11
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_MxNet.md
View File

@ -2,6 +2,7 @@
@sphinxdirective
.. warning::
Note that OpenVINO support for Apache MXNet is currently being deprecated and will be removed entirely in the future.
@ -41,12 +42,12 @@ The following list provides the MXNet-specific parameters.
.. note::
By default, Model Optimizer does not use the Apache MXNet loader. It transforms the topology to another format which is compatible with the latest version of Apache MXNet. However, the Apache MXNet loader is required for models trained with lower version of Apache MXNet. If your model was trained with an Apache MXNet version lower than 1.0.0, specify the ``--legacy_mxnet_model`` key to enable the Apache MXNet loader. Note that the loader does not support models with custom layers. In this case, you must manually recompile Apache MXNet with custom layers and install it in your environment.
By default, model conversion API does not use the Apache MXNet loader. It transforms the topology to another format which is compatible with the latest version of Apache MXNet. However, the Apache MXNet loader is required for models trained with lower version of Apache MXNet. If your model was trained with an Apache MXNet version lower than 1.0.0, specify the ``--legacy_mxnet_model`` key to enable the Apache MXNet loader. Note that the loader does not support models with custom layers. In this case, you must manually recompile Apache MXNet with custom layers and install it in your environment.
Custom Layer Definition
#######################
Internally, when you run Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list. If your topology contains such kind of layers, Model Optimizer classifies them as custom.
For the definition of custom layers, refer to the :doc:`Cutting Off Parts of a Model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>` page.
Supported MXNet Layers
#######################
@ -56,16 +57,16 @@ For the list of supported standard layers, refer to the :doc:`Supported Operatio
Frequently Asked Questions (FAQ)
################################
Model Optimizer provides explanatory messages when it is unable to complete conversions due to typographical errors, incorrectly used options, or other issues. A message describes the potential cause of the problem and gives a link to :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>` which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections to help you understand what went wrong.
Model conversion API provides explanatory messages when it is unable to complete conversions due to typographical errors, incorrectly used options, or other issues. A message describes the potential cause of the problem and gives a link to :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>` which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` to help you understand what went wrong.
Summary
########
In this document, you learned:
* Basic information about how Model Optimizer works with MXNet models.
* Basic information about how model conversion API works with MXNet models.
* Which MXNet models are supported.
* How to convert a trained MXNet model by using the Model Optimizer with both framework-agnostic and MXNet-specific command-line options.
* How to convert a trained MXNet model by using model conversion API with both framework-agnostic and MXNet-specific command-line parameters.
Additional Resources
####################

6
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_ONNX.md
View File

@ -10,11 +10,11 @@ Introduction to ONNX
Converting an ONNX Model
########################
This page provides instructions on how to convert a model from the ONNX format to the OpenVINO IR format using Model Optimizer. To use Model Optimizer, install OpenVINO Development Tools by following the :doc:`installation instructions <openvino_docs_install_guides_install_dev_tools>`.
This page provides instructions on model conversion from the ONNX format to the OpenVINO IR format. To use model conversion API, install OpenVINO Development Tools by following the :doc:`installation instructions <openvino_docs_install_guides_install_dev_tools>`.
The Model Optimizer process assumes you have an ONNX model that was directly downloaded from a public repository or converted from any framework that supports exporting to the ONNX format.
Model conversion process assumes you have an ONNX model that was directly downloaded from a public repository or converted from any framework that supports exporting to the ONNX format.
To convert an ONNX model, run Model Optimizer with the path to the input model ``.onnx`` file:
To convert an ONNX model, run model conversion with the path to the input model ``.onnx`` file:
.. code-block:: sh

4
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Paddle.md
View File

@ -24,7 +24,7 @@ To convert a PaddlePaddle model, use the ``mo`` script and specify the path to t
Converting PaddlePaddle Model From Memory Using Python API
##########################################################
MO Python API supports passing PaddlePaddle models directly from memory.
Model conversion API supports passing PaddlePaddle models directly from memory.
Following PaddlePaddle model formats are supported:
@ -162,7 +162,7 @@ The following PaddlePaddle models have been officially validated and confirmed t
Frequently Asked Questions (FAQ)
################################
When Model Optimizer is unable to run to completion due to typographical errors, incorrectly used options, or other issues, it provides explanatory messages. They describe the potential cause of the problem and give a link to the :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>`, which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
When model conversion API is unable to run to completion due to typographical errors, incorrectly used options, or other issues, it provides explanatory messages. They describe the potential cause of the problem and give a link to the :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>`, which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` to help you understand what went wrong.
Additional Resources
####################

40
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md
View File

@ -2,9 +2,9 @@
@sphinxdirective
This page provides general instructions on how to convert a model from a TensorFlow format to the OpenVINO IR format using Model Optimizer. The instructions are different depending on whether your model was created with TensorFlow v1.X or TensorFlow v2.X.
This page provides general instructions on how to run model conversion from a TensorFlow format to the OpenVINO IR format. The instructions are different depending on whether your model was created with TensorFlow v1.X or TensorFlow v2.X.
To use Model Optimizer, install OpenVINO Development Tools by following the :doc:`installation instructions <openvino_docs_install_guides_install_dev_tools>`.
To use model conversion API, install OpenVINO Development Tools by following the :doc:`installation instructions <openvino_docs_install_guides_install_dev_tools>`.
Converting TensorFlow 1 Models
###############################
@ -14,7 +14,7 @@ Converting Frozen Model Format
To convert a TensorFlow model, use the ``*mo*`` script to simply convert a model with a path to the input model ``*.pb*`` file:
.. code-block:: cpp
.. code-block:: sh
mo --input_model <INPUT_MODEL>.pb
@ -22,19 +22,19 @@ To convert a TensorFlow model, use the ``*mo*`` script to simply convert a model
Converting Non-Frozen Model Formats
+++++++++++++++++++++++++++++++++++
There are three ways to store non-frozen TensorFlow models and convert them by Model Optimizer:
There are three ways to store non-frozen TensorFlow models and convert them by model conversion API:
1. **Checkpoint**. In this case, a model consists of two files: ``inference_graph.pb`` (or ``inference_graph.pbtxt``) and ``checkpoint_file.ckpt``.
If you do not have an inference graph file, refer to the `Freezing Custom Models in Python <#Freezing-Custom-Models-in-Python>`__ section.
To convert the model with the inference graph in ``.pb`` format, run the `mo` script with a path to the checkpoint file:
.. code-block:: cpp
.. code-block:: sh
mo --input_model <INFERENCE_GRAPH>.pb --input_checkpoint <INPUT_CHECKPOINT>
To convert the model with the inference graph in ``.pbtxt`` format, run the ``mo`` script with a path to the checkpoint file:
.. code-block:: cpp
.. code-block:: sh
mo --input_model <INFERENCE_GRAPH>.pbtxt --input_checkpoint <INPUT_CHECKPOINT> --input_model_is_text
@ -43,7 +43,7 @@ To convert the model with the inference graph in ``.pbtxt`` format, run the ``mo
``model_name.data-00000-of-00001`` (the numbers may vary), and ``checkpoint`` (optional).
To convert such TensorFlow model, run the `mo` script with a path to the MetaGraph ``.meta`` file:
.. code-block:: cpp
.. code-block:: sh
mo --input_meta_graph <INPUT_META_GRAPH>.meta
@ -52,7 +52,7 @@ To convert such TensorFlow model, run the `mo` script with a path to the MetaGra
and several subfolders: ``variables``, ``assets``, and ``assets.extra``. For more information about the SavedModel directory, refer to the `README <https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/saved_model#components>`__ file in the TensorFlow repository.
To convert such TensorFlow model, run the ``mo`` script with a path to the SavedModel directory:
.. code-block:: cpp
.. code-block:: sh
mo --saved_model_dir <SAVED_MODEL_DIRECTORY>
@ -67,7 +67,7 @@ Freezing Custom Models in Python
When a network is defined in Python code, you have to create an inference graph file. Graphs are usually built in a form
that allows model training. That means all trainable parameters are represented as variables in the graph.
To be able to use such graph with Model Optimizer, it should be frozen and dumped to a file with the following code:
To be able to use such graph with model conversion API, it should be frozen and dumped to a file with the following code:
.. code-block:: python
@ -97,7 +97,7 @@ SavedModel Format
A model in the SavedModel format consists of a directory with a ``saved_model.pb`` file and two subfolders: ``variables`` and ``assets``.
To convert such a model, run the `mo` script with a path to the SavedModel directory:
.. code-block:: cpp
.. code-block:: sh
mo --saved_model_dir <SAVED_MODEL_DIRECTORY>
@ -162,30 +162,30 @@ Then follow the above instructions for the SavedModel format.
Command-Line Interface (CLI) Examples Using TensorFlow-Specific Parameters
##########################################################################
* Launching the Model Optimizer for Inception V1 frozen model when model file is a plain text protobuf:
* Launching model conversion for Inception V1 frozen model when model file is a plain text protobuf:
.. code-block:: cpp
.. code-block:: sh
mo --input_model inception_v1.pbtxt --input_model_is_text -b 1
* Launching the Model Optimizer for Inception V1 frozen model and dump information about the graph to TensorBoard log dir ``/tmp/log_dir``
* Launching model conversion for Inception V1 frozen model and dump information about the graph to TensorBoard log dir ``/tmp/log_dir``
.. code-block:: cpp
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --tensorboard_logdir /tmp/log_dir
* Launching the Model Optimizer for BERT model in the SavedModel format, with three inputs. Specify explicitly the input shapes where the batch size and the sequence length equal 2 and 30 respectively.
* Launching model conversion for BERT model in the SavedModel format, with three inputs. Specify explicitly the input shapes where the batch size and the sequence length equal 2 and 30 respectively.
.. code-block:: cpp
.. code-block:: sh
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
Conversion of TensorFlow models from memory using Python API
############################################################
MO Python API supports passing TensorFlow/TensorFlow2 models directly from memory.
Model conversion API supports passing TensorFlow/TensorFlow2 models directly from memory.
* ``tf.keras.Model``
@ -288,17 +288,17 @@ For the list of supported standard layers, refer to the :doc:`Supported Operatio
Frequently Asked Questions (FAQ)
################################
The Model Optimizer provides explanatory messages if it is unable to run to completion due to typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>`. The FAQ provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
The model conversion API provides explanatory messages if it is unable to run to completion due to typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the :doc:`Model Optimizer FAQ <openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ>`. The FAQ provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` to help you understand what went wrong.
Summary
#######
In this document, you learned:
* Basic information about how the Model Optimizer works with TensorFlow models.
* Basic information about how the model conversion API works with TensorFlow models.
* Which TensorFlow models are supported.
* How to freeze a TensorFlow model.
* How to convert a trained TensorFlow model using the Model Optimizer with both framework-agnostic and TensorFlow-specific command-line options.
* How to convert a trained TensorFlow model using model conversion API with both framework-agnostic and TensorFlow-specific command-line parameters.
Additional Resources
####################

145
docs/MO_DG/prepare_model/convert_model/Converting_Model.md
View File

@ -1,98 +1,145 @@
# Setting Input Shapes {#openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model}
With Model Optimizer you can increase your model's efficiency by providing an additional shape definition, with these two parameters: `--input_shape` and `--static_shape`.
With model conversion API you can increase your model's efficiency by providing an additional shape definition, with these two parameters: `input_shape` and `static_shape`.
@sphinxdirective
.. _when_to_specify_input_shapes:
Specifying --input_shape Command-line Parameter
###############################################
Specifying input_shape parameter
################################
Model Optimizer supports conversion of models with dynamic input shapes that contain undefined dimensions.
``convert_model()`` supports conversion of models with dynamic input shapes that contain undefined dimensions.
However, if the shape of data is not going to change from one inference request to another,
it is recommended to set up static shapes (when all dimensions are fully defined) for the inputs.
Doing it at this stage, instead of during inference in runtime, can be beneficial in terms of performance and memory consumption.
To set up static shapes, Model Optimizer provides the ``--input_shape`` parameter.
To set up static shapes, model conversion API provides the ``input_shape`` parameter.
For more information on input shapes under runtime, refer to the :doc:`Changing input shapes <openvino_docs_OV_UG_ShapeInference>` guide.
To learn more about dynamic shapes in runtime, refer to the :doc:`Dynamic Shapes <openvino_docs_OV_UG_DynamicShapes>` guide.
The OpenVINO Runtime API may present certain limitations in inferring models with undefined dimensions on some hardware. See the :doc:`Features support matrix <openvino_docs_OV_UG_Working_with_devices>` for reference.
In this case, the ``--input_shape`` parameter and the :doc:`reshape method <openvino_docs_OV_UG_ShapeInference>` can help to resolve undefined dimensions.
In this case, the ``input_shape`` parameter and the :doc:`reshape method <openvino_docs_OV_UG_ShapeInference>` can help to resolve undefined dimensions.
Sometimes, Model Optimizer is unable to convert models out-of-the-box (only the ``--input_model`` parameter is specified).
Such problem can relate to models with inputs of undefined ranks and a case of cutting off parts of a model.
In this case, input shapes must be specified explicitly with the ``--input_shape`` parameter.
For example, run Model Optimizer for the TensorFlow MobileNet model with the single input
For example, run model conversion for the TensorFlow MobileNet model with the single input
and specify the input shape of ``[2,300,300,3]``:
.. code-block:: sh
.. tab-set::
mo --input_model MobileNet.pb --input_shape [2,300,300,3]
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("MobileNet.pb", input_shape=[2,300,300,3])
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model MobileNet.pb --input_shape [2,300,300,3]
If a model has multiple inputs, ``--input_shape`` must be used in conjunction with ``--input`` parameter.
The ``--input`` parameter contains a list of input names, for which shapes in the same order are defined via ``--input_shape``.
For example, launch Model Optimizer for the ONNX OCR model with a pair of inputs ``data`` and ``seq_len``
If a model has multiple inputs, ``input_shape`` must be used in conjunction with ``input`` parameter.
The ``input`` parameter contains a list of input names, for which shapes in the same order are defined via ``input_shape``.
For example, launch model conversion for the ONNX OCR model with a pair of inputs ``data`` and ``seq_len``
and specify shapes ``[3,150,200,1]`` and ``[3]`` for them:
.. code-block:: sh
.. tab-set::
mo --input_model ocr.onnx --input data,seq_len --input_shape [3,150,200,1],[3]
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("ocr.onnx", input=["data","seq_len"], input_shape=[[3,150,200,1],[3]])
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [3,150,200,1],[3]
Alternatively, specify input shapes, using the ``--input`` parameter as follows:
Alternatively, specify input shapes, using the ``input`` parameter as follows:
.. code-block:: sh
.. tab-set::
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3]
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("ocr.onnx", input=[("data",[3,150,200,1]),("seq_len",[3])])
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3]
The ``--input_shape`` parameter allows overriding original input shapes to ones compatible with a given model.
The ``input_shape`` parameter allows overriding original input shapes to ones compatible with a given model.
Dynamic shapes, i.e. with dynamic dimensions, can be replaced in the original model with static shapes for the converted model, and vice versa.
The dynamic dimension can be marked in Model Optimizer command-line as ``-1``* or *``?``.
For example, launch Model Optimizer for the ONNX OCR model and specify dynamic batch dimension for inputs:
The dynamic dimension can be marked in model conversion API parameter as ``-1`` or ``?``.
For example, launch model conversion for the ONNX OCR model and specify dynamic batch dimension for inputs:
.. code-block:: sh
.. tab-set::
mo --input_model ocr.onnx --input data,seq_len --input_shape [-1,150,200,1],[-1]
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("ocr.onnx", input=["data","seq_len"], input_shape=[[-1,150,200,1],[-1]]
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [-1,150,200,1],[-1]
To optimize memory consumption for models with undefined dimensions in run-time, Model Optimizer provides the capability to define boundaries of dimensions.
To optimize memory consumption for models with undefined dimensions in run-time, model conversion API provides the capability to define boundaries of dimensions.
The boundaries of undefined dimension can be specified with ellipsis.
For example, launch Model Optimizer for the ONNX OCR model and specify a boundary for the batch dimension:
For example, launch model conversion for the ONNX OCR model and specify a boundary for the batch dimension:
.. code-block:: sh
.. tab-set::
mo --input_model ocr.onnx --input data,seq_len --input_shape [1..3,150,200,1],[1..3]
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
from openvino.runtime import Dimension
ov_model = convert_model("ocr.onnx", input=["data","seq_len"], input_shape=[[Dimension(1,3),150,200,1],[Dimension(1,3)]]
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [1..3,150,200,1],[1..3]
Practically, some models are not ready for input shapes change.
In this case, a new input shape cannot be set via Model Optimizer.
For more information about shape follow the :doc:`inference troubleshooting <troubleshooting_reshape_errors>`
In this case, a new input shape cannot be set via model conversion API.
For more information about shape follow the :doc:`inference troubleshooting <troubleshooting_reshape_errors>`
and :ref:`ways to relax shape inference flow <how-to-fix-non-reshape-able-model>` guides.
Specifying --static_shape Command-line Parameter
################################################
Model Optimizer provides the ``--static_shape`` parameter that allows evaluating shapes of all operations in the model for fixed input shapes
and folding shape computing sub-graphs into constants. The resulting IR may be more compact in size and the loading time for such IR may decrease.
However, the resulting IR will not be reshape-able with the help of the :doc:`reshape method <openvino_docs_OV_UG_ShapeInference>` from OpenVINO Runtime API.
It is worth noting that the ``--input_shape`` parameter does not affect reshapeability of the model.
For example, launch Model Optimizer for the ONNX OCR model using ``--static_shape``:
.. code-block:: sh
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3] --static_shape
Additional Resources
####################
* :doc:`Introduction to converting models with Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Cutting Off Parts of a Model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>`
@endsphinxdirective
@endsphinxdirective

304
docs/MO_DG/prepare_model/convert_model/Cutting_Model.md
View File

@ -2,7 +2,7 @@
@sphinxdirective
Sometimes, it is necessary to remove parts of a model when converting it to OpenVINO IR. This chapter describes how to do it, using Model Optimizer command-line options. Model cutting applies mostly to TensorFlow models, which is why TensorFlow will be used in this chapter's examples, but it may be also useful for other frameworks.
Sometimes, it is necessary to remove parts of a model when converting it to OpenVINO IR. This chapter describes how to do it, using model conversion API parameters. Model cutting applies mostly to TensorFlow models, which is why TensorFlow will be used in this chapter's examples, but it may be also useful for other frameworks.
Purpose of Model Cutting
########################
@ -12,25 +12,29 @@ The following examples are the situations when model cutting is useful or even r
* A model has pre- or post-processing parts that cannot be translated to existing OpenVINO operations.
* A model has a training part that is convenient to be kept in the model but not used during inference.
* A model is too complex be converted at once, because it contains a lot of unsupported operations that cannot be easily implemented as custom layers.
* A problem occurs with model conversion in Model Optimizer or inference in OpenVINO™ Runtime. To identify the issue, limit the conversion scope by iterative search for problematic areas in the model.
* A problem occurs with model conversion or inference in OpenVINO™ Runtime. To identify the issue, limit the conversion scope by iterative search for problematic areas in the model.
* A single custom layer or a combination of custom layers is isolated for debugging purposes.
Command-Line Options
####################
.. note::
Model Optimizer provides command line options ``--input`` and ``--output`` to specify new entry and exit nodes, while ignoring the rest of the model:
Internally, when you run model conversion API, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list. If your topology contains such kind of layers, model conversion API classifies them as custom.
* ``--input`` option accepts a comma-separated list of layer names of the input model that should be treated as new entry points to the model.
* ``--output`` option accepts a comma-separated list of layer names of the input model that should be treated as new exit points from the model.
Model conversion API parameters
###############################
The ``--input`` option is required for cases unrelated to model cutting. For example, when the model contains several inputs and ``--input_shape`` or ``--mean_values`` options are used, the ``--input`` option specifies the order of input nodes for correct mapping between multiple items provided in ``--input_shape`` and ``--mean_values`` and the inputs in the model.
Model conversion API provides command line options ``input`` and ``output`` to specify new entry and exit nodes, while ignoring the rest of the model:
Model cutting is illustrated with the Inception V1 model, found in the ``models/research/slim`` repository. To proceed with this chapter, make sure you do the necessary steps to :doc:`prepare the model for Model Optimizer <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
* ``input`` option accepts a list of layer names of the input model that should be treated as new entry points to the model. See the full list of accepted types for input on :doc:`Model Conversion Python API <openvino_docs_MO_DG_Python_API>` page.
* ``output`` option accepts a list of layer names of the input model that should be treated as new exit points from the model.
Default Behavior without --input and --output
#############################################
The ``input`` option is required for cases unrelated to model cutting. For example, when the model contains several inputs and ``input_shape`` or ``mean_values`` options are used, the ``input`` option specifies the order of input nodes for correct mapping between multiple items provided in ``input_shape`` and ``mean_values`` and the inputs in the model.
The input model is converted as a whole if neither ``--input`` nor ``--output`` command line options are used. All ``Placeholder`` operations in a TensorFlow graph are automatically identified as entry points. The ``Input`` layer type is generated for each of them. All nodes that have no consumers are automatically identified as exit points.
Model cutting is illustrated with the Inception V1 model, found in the ``models/research/slim`` repository. To proceed with this chapter, make sure you do the necessary steps to :doc:`prepare the model for model conversion <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
Default Behavior without input and output
#########################################
The input model is converted as a whole if neither ``input`` nor ``output`` command line options are used. All ``Placeholder`` operations in a TensorFlow graph are automatically identified as entry points. The ``Input`` layer type is generated for each of them. All nodes that have no consumers are automatically identified as exit points.
For Inception_V1, there is one ``Placeholder``: input. If the model is viewed in TensorBoard, the input operation is easy to find:
@ -44,16 +48,28 @@ In TensorBoard, along with some of its predecessors, it looks as follows:
.. image:: _static/images/inception_v1_std_output.svg
:alt: TensorBoard with predecessors
Convert this model and put the results in a writable output directory:
Convert this model to ``ov.Model``:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb -b 1 --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --output_dir <OUTPUT_MODEL_DIR>
(The other examples on this page assume that you first go to the ``model_optimizer`` directory and add the ``--output_dir`` argument with a directory where you have read/write permissions.)
The output ``.xml`` file with an Intermediate Representation contains the ``Input`` layer among other layers in the model:
``ov.Model`` can be serialized with the ``ov.serialize()`` method to Intermediate Representation which can be used for model structure exploring.
In IR, the structure of a model has the following layers:
.. code-block:: xml
@ -94,13 +110,28 @@ The last layer in the model is ``InceptionV1/Logits/Predictions/Reshape_1``, whi
</layer>
Due to automatic identification of inputs and outputs, providing the ``--input`` and ``--output`` options to convert the whole model is not required. The following commands are equivalent for the Inception V1 model:
Due to automatic identification of inputs and outputs, providing the ``input`` and ``output`` options to convert the whole model is not required. The following commands are equivalent for the Inception V1 model:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb -b 1 --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
mo --input_model inception_v1.pb -b 1 --input input --output InceptionV1/Logits/Predictions/Reshape_1 --output_dir <OUTPUT_MODEL_DIR>
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1)
ov_model = convert_model("inception_v1.pb", batch=1, input="input", output="InceptionV1/Logits/Predictions/Reshape_1")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --output_dir <OUTPUT_MODEL_DIR>
mo --input_model inception_v1.pb -b 1 --input input --output InceptionV1/Logits/Predictions/Reshape_1 --output_dir <OUTPUT_MODEL_DIR>
The Intermediate Representations are identical for both conversions. The same is true if the model has multiple inputs and/or outputs.
@ -120,9 +151,22 @@ If you want to cut your model at the end, you have the following options:
1. The following command cuts off the rest of the model after the ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu``, making this node the last in the model:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb -b 1 --output=InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --output=InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
The resulting Intermediate Representation has three layers:
@ -166,13 +210,26 @@ If you want to cut your model at the end, you have the following options:
</net>
As shown in the TensorBoard picture, the original model has more nodes than its Intermediate Representation. Model Optimizer has fused batch normalization ``InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm`` with convolution ``InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution``, which is why it is not present in the final model. This is not an effect of the ``--output`` option, it is the typical behavior of Model Optimizer for batch normalizations and convolutions. The effect of the ``--output`` is that the ``ReLU`` layer becomes the last one in the converted model.
As shown in the TensorBoard picture, the original model has more nodes than its Intermediate Representation. Model conversion, using ``convert_model()``, consists of a set of model transformations, including fusing of batch normalization ``InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm`` with convolution ``InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution``, which is why it is not present in the final model. This is not an effect of the ``output`` option, it is the typical behavior of model conversion API for batch normalizations and convolutions. The effect of the ``output`` is that the ``ReLU`` layer becomes the last one in the converted model.
2. The following command cuts the edge that comes from 0 output port of the ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` and the rest of the model, making this node the last one in the model:
.. code-block::
.. tab-set::
mo --input_model inception_v1.pb -b 1 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu:0 --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu:0")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu:0 --output_dir <OUTPUT_MODEL_DIR>
The resulting Intermediate Representation has three layers, which are the same as in the previous case:
@ -220,9 +277,22 @@ If you want to cut your model at the end, you have the following options:
3. The following command cuts the edge that comes to 0 input port of the ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` and the rest of the model including ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu``, deleting this node and making the previous node ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Conv2D`` the last in the model:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb -b 1 --output=0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, output="0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --output=0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
The resulting Intermediate Representation has two layers, which are the same as the first two layers in the previous case:
@ -262,11 +332,25 @@ Cutting from the Beginning
If you want to go further and cut the beginning of the model, leaving only the ``ReLU`` layer, you have the following options:
1. Use the following command line, where ``--input`` and ``--output`` specify the same node in the graph:
1. Use the following parameters, where ``input`` and ``output`` specify the same node in the graph:
.. code-block:: sh
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu", input="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model=inception_v1.pb -b 1 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --input InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
mo --input_model=inception_v1.pb -b 1 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --input InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
The resulting Intermediate Representation looks as follows:
@ -295,15 +379,29 @@ If you want to go further and cut the beginning of the model, leaving only the `
</net>
``Input`` layer is automatically created to feed the layer that is converted from the node specified in ``--input``, which is ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` in this case. Model Optimizer does not replace the ``ReLU`` node by the ``Input`` layer. It produces such Intermediate Representation to make the node the first executable node in the final Intermediate Representation. Therefore, Model Optimizer creates enough ``Inputs`` to feed all input ports of the node that is passed in ``--input``.
``Input`` layer is automatically created to feed the layer that is converted from the node specified in ``input``, which is ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` in this case. ``convert_model()`` does not replace the ``ReLU`` node by the ``Input`` layer. It produces such ``ov.Model`` to make the node the first executable node in the final Intermediate Representation. Therefore, model conversion creates enough ``Inputs`` to feed all input ports of the node that is passed in ``input``.
Even though ``--input_shape`` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow model to the point, at which the new input is defined. It has the same shape ``[1,64,112,112]`` as the model converted as a whole or without cutting off the beginning.
Even though ``input_shape`` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow model to the point, at which the new input is defined. It has the same shape ``[1,64,112,112]`` as the model converted as a whole or without cutting off the beginning.
2. Cut the edge incoming to layer by port number. To specify the incoming port, use the following notation ``--input=port:input_node``. To cut everything before ``ReLU`` layer, cut the edge incoming to port 0 of ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` node:
2. Cut the edge incoming to layer by port number. To specify the incoming port, use the following notation ``input=port:input_node``. To cut everything before ``ReLU`` layer, cut the edge incoming to port 0 of ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` node:
.. code-block:: sh
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, input="0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu", output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --input 0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
mo --input_model inception_v1.pb -b 1 --input 0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
The resulting Intermediate Representation looks as follows:
@ -332,15 +430,28 @@ If you want to go further and cut the beginning of the model, leaving only the `
</net>
``Input`` layer is automatically created to feed the layer that is converted from the node specified in ``--input``, which is ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` in this case. Model Optimizer does not replace the ``ReLU`` node by the ``Input`` layer, it produces such Intermediate Representation to make the node be the first executable node in the final Intermediate Representation. Therefore, Model Optimizer creates enough ``Inputs`` to feed all input ports of the node that is passed in ``--input``.
``Input`` layer is automatically created to feed the layer that is converted from the node specified in ``input``, which is ``InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`` in this case. ``convert_model()`` does not replace the ``ReLU`` node by the ``Input`` layer, it produces such ``ov.Model`` to make the node be the first executable node in the final Intermediate Representation. Therefore, ``convert_model()`` creates enough ``Inputs`` to feed all input ports of the node that is passed in ``input``.
Even though ``--input_shape`` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow model to the point, at which the new input is defined. It has the same shape ``[1,64,112,112]`` as the model converted as a whole or without cutting off the beginning.
Even though ``input_shape`` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow model to the point, at which the new input is defined. It has the same shape ``[1,64,112,112]`` as the model converted as a whole or without cutting off the beginning.
3. Cut edge outcoming from layer by port number. To specify the outcoming port, use the following notation ``--input=input_node:port``. To cut everything before ``ReLU`` layer, cut edge from ``InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1`` node to ``ReLU``:
3. Cut edge outcoming from layer by port number. To specify the outcoming port, use the following notation ``input=input_node:port``. To cut everything before ``ReLU`` layer, cut edge from ``InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1`` node to ``ReLU``:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb -b 1 --input InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1:0 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, input="InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1:0", output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --input InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1:0 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
The resulting Intermediate Representation looks as follows:
@ -370,79 +481,57 @@ If you want to go further and cut the beginning of the model, leaving only the `
</net>
Shape Override for New Inputs
#############################
The input shape can be overridden with ``--input_shape``. In this case, the shape is applied to the node referenced in ``--input``, not to the original ``Placeholder`` in the model. For example, the command below:
.. code-block:: sh
mo --input_model inception_v1.pb --input_shape=[1,5,10,20] --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --input InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
gives the following shapes in the ``Input`` and ``ReLU`` layers:
.. code-block:: xml
<layer id="0" name="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu/placeholder_port_0" precision="FP32" type="Input">
<output>
<port id="0">
<dim>1</dim>
<dim>20</dim>
<dim>5</dim>
<dim>10</dim>
</port>
</output>
</layer>
<layer id="3" name="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu" precision="FP32" type="ReLU">
<input>
<port id="0">
<dim>1</dim>
<dim>20</dim>
<dim>5</dim>
<dim>10</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>20</dim>
<dim>5</dim>
<dim>10</dim>
</port>
</output>
</layer>
An input shape ``[1,20,5,10]`` in the final Intermediate Representation differs from the shape ``[1,5,10,20]`` specified in the command line, because the original TensorFlow model uses NHWC layout, but the Intermediate Representation uses NCHW layout. Thus, usual NHWC to NCHW layout conversion occurred.
When ``--input_shape`` is specified, shape inference inside Model Optimizer is not performed for the nodes in the beginning of the model that are not included in the translated region. It differs from the case when ``--input_shape`` is not specified as noted in the previous section, where the shape inference is still performed for such nodes to deduce shape for the layers that should fall into the final Intermediate Representation. Therefore, ``--input_shape`` should be used for a model with a complex graph with loops, which are not supported by Model Optimizer, to exclude such parts from the Model Optimizer shape inference process completely.
Inputs with Multiple Input Ports
################################
There are operations that contain more than one input port. In the example considered here, the convolution ``InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`` is such operation. When ``--input_shape`` is not provided, a new ``Input`` layer is created for each dynamic input port for the node. If a port is evaluated to a constant blob, this constant remains in the model and a corresponding input layer is not created. TensorFlow convolution used in this model contains two ports:
There are operations that contain more than one input port. In the example considered here, the convolution ``InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`` is such operation. When ``input_shape`` is not provided, a new ``Input`` layer is created for each dynamic input port for the node. If a port is evaluated to a constant blob, this constant remains in the model and a corresponding input layer is not created. TensorFlow convolution used in this model contains two ports:
* port 0: input tensor for convolution (dynamic)
* port 1: convolution weights (constant)
Following this behavior, Model Optimizer creates an ``Input`` layer for port 0 only, leaving port 1 as a constant. Thus, the result of:
Following this behavior, ``convert_model()`` creates an ``Input`` layer for port 0 only, leaving port 1 as a constant. Thus, the result of:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb -b 1 --input InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", batch=1, input="InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution")
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb -b 1 --input InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --output_dir <OUTPUT_MODEL_DIR>
is identical to the result of conversion of the model as a whole, because this convolution is the first executable operation in Inception V1.
Different behavior occurs when ``--input_shape`` is also used as an attempt to override the input shape:
Different behavior occurs when ``input_shape`` is also used as an attempt to override the input shape:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb--input=InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --input_shape [1,224,224,3] --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", input="InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution", input_shape=[1,224,224,3])
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb--input=InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --input_shape [1,224,224,3] --output_dir <OUTPUT_MODEL_DIR>
An error occurs (for more information, see the :ref:`Model Optimizer FAQ <question-30>`):
An error occurs (for more information, see the :ref:`Model Conversion FAQ <question-30>`):
.. code-block:: sh
@ -450,13 +539,26 @@ An error occurs (for more information, see the :ref:`Model Optimizer FAQ <questi
Try not to provide input shapes or specify input port with PORT:NODE notation, where PORT is an integer.
For more information, see FAQ #30
When ``--input_shape`` is specified and the node contains multiple input ports, you need to provide an input port index together with an input node name. The input port index is specified in front of the node name with ``‘:’`` as a separator (``PORT:NODE``). In this case, the port index 0 of the node ``InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`` should be specified as ``0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution``.
When ``input_shape`` is specified and the node contains multiple input ports, you need to provide an input port index together with an input node name. The input port index is specified in front of the node name with ``‘:’`` as a separator (``PORT:NODE``). In this case, the port index 0 of the node ``InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`` should be specified as ``0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution``.
The correct command line is:
.. code-block:: sh
.. tab-set::
mo --input_model inception_v1.pb --input 0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --input_shape=[1,224,224,3] --output_dir <OUTPUT_MODEL_DIR>
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model("inception_v1.pb", input="0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution", input_shape=[1,224,224,3])
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --input_model inception_v1.pb --input 0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --input_shape=[1,224,224,3] --output_dir <OUTPUT_MODEL_DIR>
@endsphinxdirective

4
docs/MO_DG/prepare_model/convert_model/IR_suitable_for_INT8_inference.md
View File

@ -13,7 +13,7 @@ Intermediate Representation should be specifically formed to be suitable for low
Such a model is called a Low Precision IR and can be generated in two ways:
* By :doc:`quantizing regular IR with the Post-Training Optimization tool <pot_introduction>`
* Using Model Optimizer for a model pre-trained for Low Precision inference: TensorFlow models (``.pb`` model file with ``FakeQuantize`` operations), quantized TensorFlow Lite models and ONNX quantized models.
* Using model conversion of a model pre-trained for Low Precision inference: TensorFlow models (``.pb`` model file with ``FakeQuantize`` operations), quantized TensorFlow Lite models and ONNX quantized models.
TensorFlow and ONNX quantized models can be prepared by `Neural Network Compression Framework <https://github.com/openvinotoolkit/nncf/blob/develop/README.md>`__.
For an operation to be executed in INT8, it must have `FakeQuantize` operations as inputs.
@ -43,6 +43,6 @@ See the visualization of `Convolution` with the compressed weights:
.. image:: _static/images/compressed_int8_Convolution_weights.png
Both Model Optimizer and Post-Training Optimization tool generate a compressed IR by default.
Both model conversion API and Post-Training Optimization tool generate a compressed IR by default.
@endsphinxdirective

5
docs/MO_DG/prepare_model/convert_model/kaldi_specific/Aspire_Tdnn_Model.md
View File

@ -13,7 +13,7 @@ for the ASpIRE Chain Time Delay Neural Network (TDNN) from the Kaldi project off
Converting an ASpIRE Chain TDNN Model to IR
###########################################
Generate the Intermediate Representation of the model by running Model Optimizer with the following parameters:
Generate the Intermediate Representation of the model by running model conversion with the following parameters:
.. code-block:: sh
@ -101,8 +101,7 @@ Prepare ivectors for the Speech Recognition sample:
<path_to_kaldi_repo>/src/featbin/copy-feats --binary=False ark:ivector_online.1.ark ark,t:ivector_online.1.ark.txt
5. For the Speech Recognition sample, the ``.ark`` file must contain an ivector
for each frame. Copy the ivector ``frame_count`` times by running the below script in the Python command prompt:
5. For the Speech Recognition sample, the ``.ark`` file must contain an ivector for each frame. Copy the ivector ``frame_count`` times by running the below script in the Python command prompt:
.. code-block:: python

5
docs/MO_DG/prepare_model/convert_model/mxnet_specific/Convert_GluonCV_Models.md
View File

@ -6,8 +6,7 @@
Note that OpenVINO support for Apache MXNet is currently being deprecated and will be removed entirely in the future.
This article provides the instructions and examples on how to use Model Optimizer to convert `GluonCV SSD and YOLO-v3 models <https://gluon-cv.mxnet.io/model_zoo/detection.html>`__ to IR.
This article provides the instructions and examples on how to convert `GluonCV SSD and YOLO-v3 models <https://gluon-cv.mxnet.io/model_zoo/detection.html>`__ to IR.
1. Choose the topology available from the `GluonCV Model Zoo <https://gluon-cv.mxnet.io/model_zoo/detection.html>`__ and export to the MXNet format using the GluonCV API. For example, for the ``ssd_512_mobilenet1.0`` topology:
@ -20,7 +19,7 @@ This article provides the instructions and examples on how to use Model Optimize
As a result, you will get an MXNet model representation in ``ssd_512_mobilenet1.0.params`` and ``ssd_512_mobilenet1.0.json`` files generated in the current directory.
2. Run the Model Optimizer tool, specifying the ``--enable_ssd_gluoncv`` option. Make sure the ``--input_shape`` parameter is set to the input shape layout of your model (NHWC or NCHW). The examples below illustrate running the Model Optimizer for the SSD and YOLO-v3 models trained with the NHWC layout and located in the ``<model_directory>``:
2. Run model conversion API, specifying the ``enable_ssd_gluoncv`` option. Make sure the ``input_shape`` parameter is set to the input shape layout of your model (NHWC or NCHW). The examples below illustrate running model conversion for the SSD and YOLO-v3 models trained with the NHWC layout and located in the ``<model_directory>``:
* **For GluonCV SSD topologies:**

4
docs/MO_DG/prepare_model/convert_model/mxnet_specific/Convert_Style_Transfer_From_MXNet.md
View File

@ -141,7 +141,7 @@ The ``models/13`` string in the code above is composed of the following substrin
Any style can be selected from `collection of pre-trained weights <https://pan.baidu.com/s/1skMHqYp>`__. On the Chinese-language page, click the down arrow next to a size in megabytes. Then wait for an overlay box to appear, and click the blue button in it to download. The ``generate()`` function generates ``nst_vgg19-symbol.json`` and ``vgg19-symbol.json`` files for the specified shape. In the code, it is ``[1024 x 768]`` for a 4:3 ratio. You can specify another, for example, ``[224,224]`` for a square ratio.
**Step 6**: Run the Model Optimizer to generate an Intermediate Representation (IR):
**Step 6**: Run model conversion to generate an Intermediate Representation (IR):
1. Create a new directory. For example:
@ -166,7 +166,7 @@ Any style can be selected from `collection of pre-trained weights <https://pan.b
Make sure that all the ``.params`` and ``.json`` files are in the same directory as the ``.nd`` files. Otherwise, the conversion process fails.
3. Run the Model Optimizer for Apache MXNet. Use the ``--nd_prefix_name`` option to specify the decoder prefix and ``--input_shape`` to specify input shapes in ``[N,C,W,H]`` order. For example:
3. Run model conversion for Apache MXNet. Use the ``--nd_prefix_name`` option to specify the decoder prefix and ``input_shape`` to specify input shapes in ``[N,C,W,H]`` order. For example:
.. code-block:: sh

18
docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md
View File

@ -6,19 +6,19 @@ The instructions below are applicable **only** to the Faster R-CNN model convert
1. Download the pretrained model file from `onnx/models <https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn>`__ (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117).
2. Generate the Intermediate Representation of the model, by changing your current working directory to the Model Optimizer installation directory, and running Model Optimizer with the following parameters:
2. Generate the Intermediate Representation of the model, by changing your current working directory to the model conversion API installation directory, and running model conversion with the following parameters:
.. code-block:: sh
.. code-block:: sh
mo \
--input_model FasterRCNN-10.onnx \
--input_shape [1,3,800,800] \
--input 0:2 \
--mean_values [102.9801,115.9465,122.7717] \
--transformations_config front/onnx/faster_rcnn.json
mo \
--input_model FasterRCNN-10.onnx \
--input_shape [1,3,800,800] \
--input 0:2 \
--mean_values [102.9801,115.9465,122.7717] \
--transformations_config front/onnx/faster_rcnn.json
Be aware that the height and width specified with the ``input_shape`` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the `Faster R-CNN article <https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn>`__.
Be aware that the height and width specified with the ``input_shape`` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the `Faster R-CNN article <https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/ faster-rcnn>`__.
3. Interpret the outputs of the generated IR: class indices, probabilities and box coordinates. Below are the outputs from the ``DetectionOutput`` layer:

2
docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_GPT2.md
View File

@ -15,7 +15,7 @@ To download the model and sample test data, go to `this model <https://github.co
Converting an ONNX GPT-2 Model to IR
####################################
Generate the Intermediate Representation of the model GPT-2 by running Model Optimizer with the following parameters:
Generate the Intermediate Representation of the model GPT-2 by running model conversion with the following parameters:
.. code-block:: sh

20
docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Mask_RCNN.md
View File

@ -6,26 +6,26 @@ The instructions below are applicable **only** to the Mask R-CNN model converted
1. Download the pretrained model file from `onnx/models <https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn>`__ (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117).
2. Generate the Intermediate Representation of the model by changing your current working directory to the Model Optimizer installation directory and running Model Optimizer with the following parameters:
2. Generate the Intermediate Representation of the model by changing your current working directory to the model conversion API installation directory and running model conversion with the following parameters:
.. code-block:: sh
.. code-block:: sh
mo \
--input_model mask_rcnn_R_50_FPN_1x.onnx \
--input "0:2" \
--input_shape [1,3,800,800] \
--mean_values [102.9801,115.9465,122.7717] \
--transformations_config front/onnx/mask_rcnn.json
mo \
--input_model mask_rcnn_R_50_FPN_1x.onnx \
--input "0:2" \
--input_shape [1,3,800,800] \
--mean_values [102.9801,115.9465,122.7717] \
--transformations_config front/onnx/mask_rcnn.json
Be aware that the height and width specified with the ``input_shape`` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the `documentation <https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn>`__.
Be aware that the height and width specified with the ``input_shape`` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the `documentation <https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn>`__.
3. Interpret the outputs of the generated IR file: masks, class indices, probabilities and box coordinates:
* masks
* class indices
* probabilities
* box coordinates
* box coordinates
The first one is a layer with the name ``6849/sink_port_0``, and rest are outputs from the ``DetectionOutput`` layer.

10
docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_Cascade_RCNN_res101.md
View File

@ -9,15 +9,15 @@ Downloading and Converting Model to ONNX
* Clone the `repository <https://github.com/open-mmlab/mmdetection>`__ :
.. code-block:: sh
.. code-block:: sh
git clone https://github.com/open-mmlab/mmdetection
cd mmdetection
git clone https://github.com/open-mmlab/mmdetection
cd mmdetection
.. note::
.. note::
To set up an environment, refer to the `instructions <https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md#installation>`__.
To set up an environment, refer to the `instructions <https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md#installation>`__.
* Download the pre-trained `model <https://download.openmmlab.com/mmdetection/v2.0/cascade_rcnn/cascade_rcnn_r101_fpn_1x_coco/cascade_rcnn_r101_fpn_1x_coco_20200317-0b6a2fbf.pth>`__. The model is also available `here <https://github.com/open-mmlab/mmdetection/blob/master/configs/cascade_rcnn/README.md>`__.

24
docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_QuartzNet.md
View File

@ -14,20 +14,20 @@ Here are the instructions on how to obtain QuartzNet in ONNX format.
2. Run the following code:
.. code-block:: python
import nemo
import nemo.collections.asr as nemo_asr
quartznet = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En")
# Export QuartzNet model to ONNX format
quartznet.decoder.export('decoder_qn.onnx')
quartznet.encoder.export('encoder_qn.onnx')
quartznet.export('qn.onnx')
.. code-block:: python
import nemo
import nemo.collections.asr as nemo_asr
quartznet = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En")
# Export QuartzNet model to ONNX format
quartznet.decoder.export('decoder_qn.onnx')
quartznet.encoder.export('encoder_qn.onnx')
quartznet.export('qn.onnx')
This code produces 3 ONNX model files: ``encoder_qn.onnx``, ``decoder_qn.onnx``, ``qn.onnx``.
They are ``decoder``, ``encoder``, and a combined ``decoder(encoder(x))`` models, respectively.
This code produces 3 ONNX model files: ``encoder_qn.onnx``, ``decoder_qn.onnx``, ``qn.onnx``.
They are ``decoder``, ``encoder``, and a combined ``decoder(encoder(x))`` models, respectively.
Converting an ONNX QuartzNet model to IR
########################################

2
docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_YOLACT.md
View File

@ -186,7 +186,7 @@ Converting a YOLACT Model to the OpenVINO IR format
**Step 4**. Embed input preprocessing into the IR:
To get performance gain by offloading to the OpenVINO application of mean/scale values and RGB->BGR conversion, use the following options of the Model Optimizer (MO):
To get performance gain by offloading to the OpenVINO application of mean/scale values and RGB->BGR conversion, use the following model conversion API parameters:
* If the backbone of the model is Resnet50-FPN or Resnet101-FPN, use the following MO command line:

3
docs/MO_DG/prepare_model/convert_model/supported_model_formats.md
View File

@ -24,9 +24,10 @@ OpenVINO Runtime without any prior conversion. For a guide on how to run inferen
see how to :doc:`Integrate OpenVINO™ with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`.
**MXNet, Caffe, Kaldi** - legacy formats that need to be converted to OpenVINO IR before running inference.
The conversion is done with Model Optimizer and in some cases may involve intermediate steps. OpenVINO is currently proceeding
The model conversion in some cases may involve intermediate steps. OpenVINO is currently proceeding
**to deprecate these formats** and **remove their support entirely in the future**.
Refer to the following articles for details on conversion for different formats and models:
* :doc:`How to convert ONNX <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX>`

2
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_AttentionOCR_From_Tensorflow.md
View File

@ -29,7 +29,7 @@ The original AOCR model includes the preprocessing data, which contains:
* Decoding input data to binary format where input data is an image represented as a string.
* Resizing binary image to working resolution.
The resized image is sent to the convolution neural network (CNN). Because Model Optimizer does not support image decoding, the preprocessing part of the model should be cut off, using the ``--input`` command-line parameter.
The resized image is sent to the convolution neural network (CNN). Because model conversion API does not support image decoding, the preprocessing part of the model should be cut off, using the ``input`` command-line parameter.
.. code-block:: sh

92
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md
View File

@ -38,7 +38,7 @@ Pretrained model meta-graph files are ``bert_model.ckpt.*``.
Converting a TensorFlow BERT Model to IR
#########################################
To generate the BERT Intermediate Representation (IR) of the model, run Model Optimizer with the following parameters:
To generate the BERT Intermediate Representation (IR) of the model, run model conversion with the following parameters:
.. code-block:: sh
@ -59,21 +59,21 @@ Follow these steps to make a pretrained TensorFlow BERT model reshapable over ba
2. Clone google-research/bert git repository:
.. code-block:: sh
.. code-block:: sh
https://github.com/google-research/bert.git
https://github.com/google-research/bert.git
3. Go to the root directory of the cloned repository:
.. code-block:: sh
.. code-block:: sh
cd bert
cd bert
4. (Optional) Checkout to the commit that the conversion was tested on:
.. code-block:: sh
.. code-block:: sh
git checkout eedf5716c
git checkout eedf5716c
5. Download script to load GLUE data:
@ -89,66 +89,66 @@ Follow these steps to make a pretrained TensorFlow BERT model reshapable over ba
6. Download GLUE data by running:
.. code-block:: sh
.. code-block:: sh
python3 download_glue_data.py --tasks MRPC
python3 download_glue_data.py --tasks MRPC
7. Open the file ``modeling.py`` in the text editor and delete lines 923-924. They should look like this:
.. code-block:: python
.. code-block:: python
if not non_static_indexes:
return shape
if not non_static_indexes:
return shape
8. Open the file ``run_classifier.py`` and insert the following code after the line 645:
.. code-block:: python
.. code-block:: python
import os, sys
import tensorflow as tf
from tensorflow.python.framework import graph_io
with tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph()) as sess:
(assignment_map, initialized_variable_names) = \
modeling.get_assignment_map_from_checkpoint(tf.compat.v1.trainable_variables(), init_checkpoint)
tf.compat.v1.train.init_from_checkpoint(init_checkpoint, assignment_map)
sess.run(tf.compat.v1.global_variables_initializer())
frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["bert/pooler/dense/Tanh"])
graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)
print('BERT frozen model path {}'.format(os.path.join(os.path.dirname(__file__), 'inference_graph.pb')))
sys.exit(0)
import os, sys
import tensorflow as tf
from tensorflow.python.framework import graph_io
with tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph()) as sess:
(assignment_map, initialized_variable_names) = \
modeling.get_assignment_map_from_checkpoint(tf.compat.v1.trainable_variables(), init_checkpoint)
tf.compat.v1.train.init_from_checkpoint(init_checkpoint, assignment_map)
sess.run(tf.compat.v1.global_variables_initializer())
frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["bert/pooler/dense/Tanh"])
graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)
print('BERT frozen model path {}'.format(os.path.join(os.path.dirname(__file__), 'inference_graph.pb')))
sys.exit(0)
Lines before the inserted code should look like this:
Lines before the inserted code should look like this:
.. code-block:: python
.. code-block:: python
(total_loss, per_example_loss, logits, probabilities) = create_model(
bert_config, is_training, input_ids, input_mask, segment_ids, label_ids,
num_labels, use_one_hot_embeddings)
(total_loss, per_example_loss, logits, probabilities) = create_model(
bert_config, is_training, input_ids, input_mask, segment_ids, label_ids,
num_labels, use_one_hot_embeddings)
9. Set environment variables ``BERT_BASE_DIR``, ``BERT_REPO_DIR`` and run the script ``run_classifier.py`` to create ``inference_graph.pb`` file in the root of the cloned BERT repository.
.. code-block:: sh
.. code-block:: sh
export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
export BERT_REPO_DIR=/current/working/directory
export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
export BERT_REPO_DIR=/current/working/directory
python3 run_classifier.py \
--task_name=MRPC \
--do_eval=true \
--data_dir=$BERT_REPO_DIR/glue_data/MRPC \
--vocab_file=$BERT_BASE_DIR/vocab.txt \
--bert_config_file=$BERT_BASE_DIR/bert_config.json \
--init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
--output_dir=./
python3 run_classifier.py \
--task_name=MRPC \
--do_eval=true \
--data_dir=$BERT_REPO_DIR/glue_data/MRPC \
--vocab_file=$BERT_BASE_DIR/vocab.txt \
--bert_config_file=$BERT_BASE_DIR/bert_config.json \
--init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
--output_dir=./
Run Model Optimizer with the following command line parameters to generate reshape-able BERT Intermediate Representation (IR):
Run model conversion with the following command line parameters to generate reshape-able BERT Intermediate Representation (IR):
.. code-block:: sh
.. code-block:: sh
mo \
--input_model inference_graph.pb \
--input "IteratorGetNext:0{i32}[1,128],IteratorGetNext:1{i32}[1,128],IteratorGetNext:4{i32}[1,128]"
mo \
--input_model inference_graph.pb \
--input "IteratorGetNext:0{i32}[1,128],IteratorGetNext:1{i32}[1,128],IteratorGetNext:4{i32}[1,128]"
For other applicable parameters, refer to the :doc:`Convert Model from TensorFlow <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.

38
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md
View File

@ -6,7 +6,7 @@ This tutorial explains how to convert a CRNN model to OpenVINO™ Intermediate R
There are several public versions of TensorFlow CRNN model implementation available on GitHub. This tutorial explains how to convert the model from
the `CRNN Tensorflow <https://github.com/MaybeShewill-CV/CRNN_Tensorflow>`__ repository to IR, and is validated with Python 3.7, TensorFlow 1.15.0, and protobuf 3.19.0.
If you have another implementation of CRNN model, it can be converted to OpenVINO IR in a similar way. You need to get inference graph and run Model Optimizer on it.
If you have another implementation of CRNN model, it can be converted to OpenVINO IR in a similar way. You need to get inference graph and run model conversion of it.
**To convert the model to IR:**
@ -14,21 +14,21 @@ If you have another implementation of CRNN model, it can be converted to OpenVIN
1. Clone the repository:
.. code-block:: sh
git clone https://github.com/MaybeShewill-CV/CRNN_Tensorflow.git
.. code-block:: sh
git clone https://github.com/MaybeShewill-CV/CRNN_Tensorflow.git
2. Go to the ``CRNN_Tensorflow`` directory of the cloned repository:
.. code-block:: sh
cd path/to/CRNN_Tensorflow
.. code-block:: sh
cd path/to/CRNN_Tensorflow
3. Check out the necessary commit:
.. code-block:: sh
git checkout 64f1f1867bffaacfeacc7a80eebf5834a5726122
.. code-block:: sh
git checkout 64f1f1867bffaacfeacc7a80eebf5834a5726122
**Step 2.** Train the model using the framework or the pretrained checkpoint provided in this repository.
@ -51,20 +51,20 @@ If you have another implementation of CRNN model, it can be converted to OpenVIN
2. Edit the ``tools/demo_shadownet.py`` script. After ``saver.restore(sess=sess, save_path=weights_path)`` line, add the following code:
.. code-block:: python
from tensorflow.python.framework import graph_io
frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['shadow/LSTMLayers/transpose_time_major'])
graph_io.write_graph(frozen, '.', 'frozen_graph.pb', as_text=False)
.. code-block:: python
from tensorflow.python.framework import graph_io
frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['shadow/LSTMLayers/transpose_time_major'])
graph_io.write_graph(frozen, '.', 'frozen_graph.pb', as_text=False)
3. Run the demo with the following command:
.. code-block:: sh
python tools/demo_shadownet.py --image_path data/test_images/test_01.jpg --weights_path model/shadownet/shadownet_2017-10-17-11-47-46.ckpt-199999
.. code-block:: sh
python tools/demo_shadownet.py --image_path data/test_images/test_01.jpg --weights_path model/shadownet/shadownet_2017-10-17-11-47-46.ckpt-199999
If you want to use your checkpoint, replace the path in the ``--weights_path`` parameter with a path to your checkpoint.
If you want to use your checkpoint, replace the path in the ``--weights_path`` parameter with a path to your checkpoint.
4. In the ``CRNN_Tensorflow`` directory, you will find the inference CRNN graph ``frozen_graph.pb``. You can use this graph with OpenVINO to convert the model to IR and then run inference.

6
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md
View File

@ -73,14 +73,14 @@ are assigned to cell state and hidden state, which are these two variables.
Converting the Main Part of DeepSpeech Model into OpenVINO IR
#############################################################
Model Optimizer assumes that the output model is for inference only. That is why you should cut ``previous_state_c`` and ``previous_state_h`` variables off and resolve keeping cell and hidden states on the application level.
Model conversion API assumes that the output model is for inference only. That is why you should cut ``previous_state_c`` and ``previous_state_h`` variables off and resolve keeping cell and hidden states on the application level.
There are certain limitations for the model conversion:
* Time length (``time_len``) and sequence length (``seq_len``) are equal.
* Original model cannot be reshaped, so you should keep original shapes.
To generate the IR, run Model Optimizer with the following parameters:
To generate the IR, run model conversion with the following parameters:
.. code-block:: sh
@ -94,6 +94,6 @@ Where:
* ``input_lengths->[16]`` Replaces the input node with name "input_lengths" with a constant tensor of shape [1] with a single integer value of 16. This means that the model now can consume input sequences of length 16 only.
* ``input_node[1 16 19 26],previous_state_h[1 2048],previous_state_c[1 2048]`` replaces the variables with a placeholder.
* ``--output ".../GatherNd_1,.../GatherNd,logits"`` output node names.
* ``output ".../GatherNd_1,.../GatherNd,logits"`` output node names.
@endsphinxdirective

10
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_FaceNet_From_Tensorflow.md
View File

@ -14,7 +14,7 @@ There are two inputs in this network: boolean ``phase_train`` which manages stat
Converting a TensorFlow FaceNet Model to the IR
###############################################
To generate a FaceNet OpenVINO model, feed a TensorFlow FaceNet model to Model Optimizer with the following parameters:
To generate a FaceNet OpenVINO model, feed a TensorFlow FaceNet model to model conversion API with the following parameters:
.. code-block:: sh
@ -23,11 +23,11 @@ To generate a FaceNet OpenVINO model, feed a TensorFlow FaceNet model to Model O
--freeze_placeholder_with_value "phase_train->False"
The batch joining pattern transforms to a placeholder with the model default shape if ``--input_shape`` or ``--batch`*/*`-b`` are not provided. Otherwise, the placeholder shape has custom parameters.
The batch joining pattern transforms to a placeholder with the model default shape if ``input_shape`` or ``batch`*/*`-b`` are not provided. Otherwise, the placeholder shape has custom parameters.
* ``--freeze_placeholder_with_value "phase_train->False"`` to switch graph to inference mode
* ``--batch`*/*`-b`` is applicable to override original network batch
* ``--input_shape`` is applicable with or without ``--input``
* ``freeze_placeholder_with_value "phase_train->False"`` to switch graph to inference mode
* ``batch`*/*`-b`` is applicable to override original network batch
* ``input_shape`` is applicable with or without ``input``
* other options are applicable
@endsphinxdirective

302
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_GNMT_From_Tensorflow.md
View File

@ -6,7 +6,7 @@ This tutorial explains how to convert Google Neural Machine Translation (GNMT) m
There are several public versions of TensorFlow GNMT model implementation available on GitHub. This tutorial explains how to convert the GNMT model from the `TensorFlow Neural Machine Translation (NMT) repository <https://github.com/tensorflow/nmt>`__ to the IR.
Creating a Patch File
Creating a Patch File
#####################
Before converting the model, you need to create a patch file for the repository. The patch modifies the framework code by adding a special command-line argument to the framework options that enables inference graph dumping:
@ -14,130 +14,130 @@ Before converting the model, you need to create a patch file for the repository.
1. Go to a writable directory and create a ``GNMT_inference.patch`` file.
2. Copy the following diff code to the file:
.. code-block:: cpp
.. code-block:: cpp
diff --git a/nmt/inference.py b/nmt/inference.py
index 2cbef07..e185490 100644
--- a/nmt/inference.py
+++ b/nmt/inference.py
@@ -17,9 +17,11 @@
from __future__ import print_function
diff --git a/nmt/inference.py b/nmt/inference.py
index 2cbef07..e185490 100644
--- a/nmt/inference.py
+++ b/nmt/inference.py
@@ -17,9 +17,11 @@
from __future__ import print_function
import codecs
+import os
import time
import codecs
+import os
import time
import tensorflow as tf
+from tensorflow.python.framework import graph_io
import tensorflow as tf
+from tensorflow.python.framework import graph_io
from . import attention_model
from . import gnmt_model
@@ -105,6 +107,29 @@ def start_sess_and_load_model(infer_model, ckpt_path):
return sess, loaded_infer_model
from . import attention_model
from . import gnmt_model
@@ -105,6 +107,29 @@ def start_sess_and_load_model(infer_model, ckpt_path):
return sess, loaded_infer_model
+def inference_dump_graph(ckpt_path, path_to_dump, hparams, scope=None):
+ model_creator = get_model_creator(hparams)
+ infer_model = model_helper.create_infer_model(model_creator, hparams, scope)
+ sess = tf.Session(
+ graph=infer_model.graph, config=utils.get_config_proto())
+ with infer_model.graph.as_default():
+ loaded_infer_model = model_helper.load_model(
+ infer_model.model, ckpt_path, sess, "infer")
+ utils.print_out("Dumping inference graph to {}".format(path_to_dump))
+ loaded_infer_model.saver.save(
+ sess,
+ os.path.join(path_to_dump + 'inference_GNMT_graph')
+ )
+ utils.print_out("Dumping done!")
+
+ output_node_name = 'index_to_string_Lookup'
+ utils.print_out("Freezing GNMT graph with output node {}...".format(output_node_name))
+ frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def,
+ [output_node_name])
+ graph_io.write_graph(frozen, '.', os.path.join(path_to_dump, 'frozen_GNMT_inference_graph.pb'), as_text=False)
+ utils.print_out("Freezing done. Freezed model frozen_GNMT_inference_graph.pb saved to {}".format(path_to_dump))
+
+
def inference(ckpt_path,
inference_input_file,
inference_output_file,
diff --git a/nmt/nmt.py b/nmt/nmt.py
index f5823d8..a733748 100644
--- a/nmt/nmt.py
+++ b/nmt/nmt.py
@@ -310,6 +310,13 @@ def add_arguments(parser):
parser.add_argument("--num_intra_threads", type=int, default=0,
help="number of intra_op_parallelism_threads")
+def inference_dump_graph(ckpt_path, path_to_dump, hparams, scope=None):
+ model_creator = get_model_creator(hparams)
+ infer_model = model_helper.create_infer_model(model_creator, hparams, scope)
+ sess = tf.Session(
+ graph=infer_model.graph, config=utils.get_config_proto())
+ with infer_model.graph.as_default():
+ loaded_infer_model = model_helper.load_model(
+ infer_model.model, ckpt_path, sess, "infer")
+ utils.print_out("Dumping inference graph to {}".format(path_to_dump))
+ loaded_infer_model.saver.save(
+ sess,
+ os.path.join(path_to_dump + 'inference_GNMT_graph')
+ )
+ utils.print_out("Dumping done!")
+
+ output_node_name = 'index_to_string_Lookup'
+ utils.print_out("Freezing GNMT graph with output node {}...".format(output_node_name))
+ frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def,
+ [output_node_name])
+ graph_io.write_graph(frozen, '.', os.path.join(path_to_dump, 'frozen_GNMT_inference_graph.pb'), as_text=False)
+ utils.print_out("Freezing done. Freezed model frozen_GNMT_inference_graph.pb saved to {}".format(path_to_dump))
+
+
def inference(ckpt_path,
inference_input_file,
inference_output_file,
diff --git a/nmt/nmt.py b/nmt/nmt.py
index f5823d8..a733748 100644
--- a/nmt/nmt.py
+++ b/nmt/nmt.py
@@ -310,6 +310,13 @@ def add_arguments(parser):
parser.add_argument("--num_intra_threads", type=int, default=0,
help="number of intra_op_parallelism_threads")
+ # Special argument for inference model dumping without inference
+ parser.add_argument("--dump_inference_model", type="bool", nargs="?",
+ const=True, default=False,
+ help="Argument for dump inference graph for specified trained ckpt")
+
+ parser.add_argument("--path_to_dump", type=str, default="",
+ help="Path to dump inference graph.")
+ # Special argument for inference model dumping without inference
+ parser.add_argument("--dump_inference_model", type="bool", nargs="?",
+ const=True, default=False,
+ help="Argument for dump inference graph for specified trained ckpt")
+
+ parser.add_argument("--path_to_dump", type=str, default="",
+ help="Path to dump inference graph.")
def create_hparams(flags):
"""Create training hparams."""
@@ -396,6 +403,9 @@ def create_hparams(flags):
language_model=flags.language_model,
num_intra_threads=flags.num_intra_threads,
num_inter_threads=flags.num_inter_threads,
+
+ dump_inference_model=flags.dump_inference_model,
+ path_to_dump=flags.path_to_dump,
)
def create_hparams(flags):
"""Create training hparams."""
@@ -396,6 +403,9 @@ def create_hparams(flags):
language_model=flags.language_model,
num_intra_threads=flags.num_intra_threads,
num_inter_threads=flags.num_inter_threads,
+
+ dump_inference_model=flags.dump_inference_model,
+ path_to_dump=flags.path_to_dump,
)
@@ -613,7 +623,7 @@ def create_or_load_hparams(
return hparams
@@ -613,7 +623,7 @@ def create_or_load_hparams(
return hparams
-def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""):
+def run_main(flags, default_hparams, train_fn, inference_fn, inference_dump, target_session=""):
"""Run main."""
# Job
jobid = flags.jobid
@@ -653,8 +663,26 @@ def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""):
out_dir, default_hparams, flags.hparams_path,
save_hparams=(jobid == 0))
-def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""):
+def run_main(flags, default_hparams, train_fn, inference_fn, inference_dump, target_session=""):
"""Run main."""
# Job
jobid = flags.jobid
@@ -653,8 +663,26 @@ def run_main(flags, default_hparams, train_fn, inference_fn, target_session=""):
out_dir, default_hparams, flags.hparams_path,
save_hparams=(jobid == 0))
- ## Train / Decode
- if flags.inference_input_file:
+ # Dumping inference model
+ if flags.dump_inference_model:
+ # Inference indices
+ hparams.inference_indices = None
+ if flags.inference_list:
+ (hparams.inference_indices) = (
+ [int(token) for token in flags.inference_list.split(",")])
+
+ # Ckpt
+ ckpt = flags.ckpt
+ if not ckpt:
+ ckpt = tf.train.latest_checkpoint(out_dir)
+
+ # Path to dump graph
+ assert flags.path_to_dump != "", "Please, specify path_to_dump model."
+ path_to_dump = flags.path_to_dump
+ if not tf.gfile.Exists(path_to_dump): tf.gfile.MakeDirs(path_to_dump)
+
+ inference_dump(ckpt, path_to_dump, hparams)
+ elif flags.inference_input_file:
# Inference output directory
trans_file = flags.inference_output_file
assert trans_file
@@ -693,7 +721,8 @@ def main(unused_argv):
default_hparams = create_hparams(FLAGS)
train_fn = train.train
inference_fn = inference.inference
- run_main(FLAGS, default_hparams, train_fn, inference_fn)
+ inference_dump = inference.inference_dump_graph
+ run_main(FLAGS, default_hparams, train_fn, inference_fn, inference_dump)
- ## Train / Decode
- if flags.inference_input_file:
+ # Dumping inference model
+ if flags.dump_inference_model:
+ # Inference indices
+ hparams.inference_indices = None
+ if flags.inference_list:
+ (hparams.inference_indices) = (
+ [int(token) for token in flags.inference_list.split(",")])
+
+ # Ckpt
+ ckpt = flags.ckpt
+ if not ckpt:
+ ckpt = tf.train.latest_checkpoint(out_dir)
+
+ # Path to dump graph
+ assert flags.path_to_dump != "", "Please, specify path_to_dump model."
+ path_to_dump = flags.path_to_dump
+ if not tf.gfile.Exists(path_to_dump): tf.gfile.MakeDirs(path_to_dump)
+
+ inference_dump(ckpt, path_to_dump, hparams)
+ elif flags.inference_input_file:
# Inference output directory
trans_file = flags.inference_output_file
assert trans_file
@@ -693,7 +721,8 @@ def main(unused_argv):
default_hparams = create_hparams(FLAGS)
train_fn = train.train
inference_fn = inference.inference
- run_main(FLAGS, default_hparams, train_fn, inference_fn)
+ inference_dump = inference.inference_dump_graph
+ run_main(FLAGS, default_hparams, train_fn, inference_fn, inference_dump)
if __name__ == "__main__":
if __name__ == "__main__":
3. Save and close the file.
@ -151,15 +151,15 @@ Converting a GNMT Model to the IR
1. Clone the NMT repository:
.. code-block:: sh
.. code-block:: sh
git clone https://github.com/tensorflow/nmt.git
git clone https://github.com/tensorflow/nmt.git
2. Check out the necessary commit:
.. code-block:: sh
.. code-block:: sh
git checkout b278487980832417ad8ac701c672b5c3dc7fa553
git checkout b278487980832417ad8ac701c672b5c3dc7fa553
**Step 2**. Get a trained model. You have two options:
@ -176,25 +176,25 @@ For the GNMT model, the training graph and the inference graph have different de
1. Apply the ``GNMT_inference.patch`` patch to the repository. `Create a Patch File <#Creating-a-Patch-File>`__ instructions if you do not have it:
.. code-block:: sh
.. code-block:: sh
git apply /path/to/patch/GNMT_inference.patch
git apply /path/to/patch/GNMT_inference.patch
2. Run the NMT framework to dump the inference model:
.. code-block:: sh
.. code-block:: sh
python -m nmt.nmt
--src=de
--tgt=en
--ckpt=/path/to/ckpt/translate.ckpt
--hparams_path=/path/to/repository/nmt/nmt/standard_hparams/wmt16_gnmt_4_layer.json
--vocab_prefix=/path/to/vocab/vocab.bpe.32000
--out_dir=""
--dump_inference_model
--infer_mode beam_search
--path_to_dump /path/to/dump/model/
python -m nmt.nmt
--src=de
--tgt=en
--ckpt=/path/to/ckpt/translate.ckpt
--hparams_path=/path/to/repository/nmt/nmt/standard_hparams/wmt16_gnmt_4_layer.json
--vocab_prefix=/path/to/vocab/vocab.bpe.32000
--out_dir=""
--dump_inference_model
--infer_mode beam_search
--path_to_dump /path/to/dump/model/
If you use different checkpoints, use the corresponding values for the ``src``, ``tgt``, ``ckpt``, ``hparams_path``, and ``vocab_prefix`` parameters.
@ -253,50 +253,50 @@ Outputs of the model:
that contains ``beam_size`` best translations for every sentence from input (also decoded as indices of words in
vocabulary).
.. note::
.. note::
The shape of this tensor in TensorFlow can be different: instead of ``max_sequence_length * 2``, it can be any value less than that, because OpenVINO does not support dynamic shapes of outputs, while TensorFlow can stop decoding iterations when ``eos`` symbol is generated.
Running GNMT IR
Running GNMT IR
---------------
1. With benchmark app:
.. code-block:: sh
.. code-block:: sh
benchmark_app -m <path to the generated GNMT IR> -d CPU
benchmark_app -m <path to the generated GNMT IR> -d CPU
2. With OpenVINO Runtime Python API:
.. note::
.. note::
Before running the example, insert a path to your GNMT ``.xml`` and ``.bin`` files into ``MODEL_PATH`` and ``WEIGHTS_PATH``, and fill ``input_data_tensor`` and ``seq_lengths`` tensors according to your input data.
Before running the example, insert a path to your GNMT ``.xml`` and ``.bin`` files into ``MODEL_PATH`` and ``WEIGHTS_PATH``, and fill ``input_data_tensor`` and ``seq_lengths`` tensors according to your input data.
.. code-block:: py
.. code-block:: py
from openvino.inference_engine import IENetwork, IECore
from openvino.inference_engine import IENetwork, IECore
MODEL_PATH = '/path/to/IR/frozen_GNMT_inference_graph.xml'
WEIGHTS_PATH = '/path/to/IR/frozen_GNMT_inference_graph.bin'
MODEL_PATH = '/path/to/IR/frozen_GNMT_inference_graph.xml'
WEIGHTS_PATH = '/path/to/IR/frozen_GNMT_inference_graph.bin'
# Creating network
net = IENetwork(
model=MODEL_PATH,
weights=WEIGHTS_PATH)
# Creating network
net = IENetwork(
model=MODEL_PATH,
weights=WEIGHTS_PATH)
# Creating input data
input_data = {'IteratorGetNext/placeholder_out_port_0': input_data_tensor,
'IteratorGetNext/placeholder_out_port_1': seq_lengths}
# Creating input data
input_data = {'IteratorGetNext/placeholder_out_port_0': input_data_tensor,
'IteratorGetNext/placeholder_out_port_1': seq_lengths}
# Creating plugin and loading extensions
ie = IECore()
ie.add_extension(extension_path="libcpu_extension.so", device_name="CPU")
# Creating plugin and loading extensions
ie = IECore()
ie.add_extension(extension_path="libcpu_extension.so", device_name="CPU")
# Loading network
exec_net = ie.load_network(network=net, device_name="CPU")
# Loading network
exec_net = ie.load_network(network=net, device_name="CPU")
# Run inference
result_ie = exec_net.infer(input_data)
# Run inference
result_ie = exec_net.infer(input_data)
For more information about Python API, refer to the :doc:`OpenVINO Runtime Python API <api/ie_python_api/api>` guide.

44
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md
View File

@ -10,44 +10,44 @@ This tutorial explains how to convert Neural Collaborative Filtering (NCF) model
2. Freeze the inference graph you get in the previous step in ``model_dir``, following the instructions from the **Freezing Custom Models in Python** section of the :doc:`Converting a TensorFlow Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
Run the following commands:
Run the following commands:
.. code-block:: py
.. code-block:: py
import tensorflow as tf
from tensorflow.python.framework import graph_io
import tensorflow as tf
from tensorflow.python.framework import graph_io
sess = tf.compat.v1.Session()
saver = tf.compat.v1.train.import_meta_graph("/path/to/model/model.meta")
saver.restore(sess, tf.train.latest_checkpoint('/path/to/model/'))
sess = tf.compat.v1.Session()
saver = tf.compat.v1.train.import_meta_graph("/path/to/model/model.meta")
saver.restore(sess, tf.train.latest_checkpoint('/path/to/model/'))
frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, \
["rating/BiasAdd"])
graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)
frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, \
["rating/BiasAdd"])
graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)
where ``rating/BiasAdd`` is an output node.
where ``rating/BiasAdd`` is an output node.
3. Convert the model to the OpenVINO format. If you look at your frozen model, you can see that
it has one input that is split into four ``ResourceGather`` layers. (Click image to zoom in.)
.. image:: ./_static/images/NCF_start.svg
.. image:: ./_static/images/NCF_start.svg
However, as the Model Optimizer does not support such data feeding, you should skip it. Cut
the edges incoming in ``ResourceGather`` port 1:
However, as the model conversion API does not support such data feeding, you should skip it. Cut
the edges incoming in ``ResourceGather`` port 1:
.. code-block:: shell
.. code-block:: sh
mo --input_model inference_graph.pb \
--input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \
1:embedding_2/embedding_lookup,1:embedding_3/embedding_lookup \
--input_shape [256],[256],[256],[256] \
--output_dir <OUTPUT_MODEL_DIR>
mo --input_model inference_graph.pb \
--input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \
1:embedding_2/embedding_lookup,1:embedding_3/embedding_lookup \
--input_shape [256],[256],[256],[256] \
--output_dir <OUTPUT_MODEL_DIR>
In the ``input_shape`` parameter, 256 specifies the ``batch_size`` for your model.
In the ``input_shape`` parameter, 256 specifies the ``batch_size`` for your model.
Alternatively, you can do steps 2 and 3 in one command line:
.. code-block:: shell
.. code-block:: sh
mo --input_meta_graph /path/to/model/model.meta \
--input 1:embedding/embedding_lookup,1:embedding_1/embedding_lookup, \

86
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Object_Detection_API_Models.md
View File

@ -2,9 +2,9 @@
@sphinxdirective
* Starting with the 2022.1 release, Model Optimizer can convert the TensorFlow Object Detection API Faster and Mask RCNNs topologies differently. By default, Model Optimizer adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the preprocessing applied to the input image (refer to the :doc:`Proposal <openvino_docs_ops_detection_Proposal_4>` operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model Optimizer can generate IR for such models and insert operation :doc:`DetectionOutput <openvino_docs_ops_detection_DetectionOutput_1>` instead of ``Proposal``. The `DetectionOutput` operation does not require additional model input "image_info". Moreover, for some models the produced inference results are closer to the original TensorFlow model. In order to trigger new behavior, the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal".
* Starting with the 2021.1 release, Model Optimizer converts the TensorFlow Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the OpenVINO Runtime using dedicated reshape API. Refer to the :doc:`Using Shape Inference <openvino_docs_OV_UG_ShapeInference>` guide for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size.
* To generate IRs for TF 1 SSD topologies, Model Optimizer creates a number of ``PriorBoxClustered`` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the OpenVINO Runtime using dedicated API. The reshaping is supported for all SSD topologies except FPNs, which contain hardcoded shapes for some operations preventing from changing topology input shape.
* Starting with the 2022.1 release, model conversion API can convert the TensorFlow Object Detection API Faster and Mask RCNNs topologies differently. By default, model conversion adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the preprocessing applied to the input image (refer to the :doc:`Proposal <openvino_docs_ops_detection_Proposal_4>` operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model conversion API can generate IR for such models and insert operation :doc:`DetectionOutput <openvino_docs_ops_detection_DetectionOutput_1>` instead of ``Proposal``. The `DetectionOutput` operation does not require additional model input "image_info". Moreover, for some models the produced inference results are closer to the original TensorFlow model. In order to trigger new behavior, the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal".
* Starting with the 2021.1 release, model conversion API converts the TensorFlow Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the OpenVINO Runtime using dedicated reshape API. Refer to the :doc:`Using Shape Inference <openvino_docs_OV_UG_ShapeInference>` guide for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size.
* To generate IRs for TF 1 SSD topologies, model conversion API creates a number of ``PriorBoxClustered`` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the OpenVINO Runtime using dedicated API. The reshaping is supported for all SSD topologies except FPNs, which contain hardcoded shapes for some operations preventing from changing topology input shape.
Converting a Model
##################
@ -13,12 +13,12 @@ You can download TensorFlow Object Detection API models from the `TensorFlow 1 D
.. note::
Before converting, make sure you have configured Model Optimizer. For configuration steps, refer to the :doc:`Configuring Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
Before converting, make sure you have configured model conversion API. For configuration steps, refer to the :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
To convert a TensorFlow Object Detection API model, run the ``mo`` command with the following required parameters:
* ``--input_model <path_to_frozen.pb>`` - File with a pretrained model (binary or text .pb file after freezing) OR ``--saved_model_dir <path_to_saved_model>`` for the TensorFlow 2 models
* ``--transformations_config <path_to_subgraph_replacement_configuration_file.json>`` - A subgraph replacement configuration file with transformations description. For the models downloaded from the TensorFlow Object Detection API zoo, you can find the configuration files in the ``<PYTHON_SITE_PACKAGES>/openvino/tools/mo/front/tf`` directory. Use:
* ``input_model <path_to_frozen.pb>`` - File with a pretrained model (binary or text .pb file after freezing) OR ``saved_model_dir <path_to_saved_model>`` for the TensorFlow 2 models
* ``transformations_config <path_to_subgraph_replacement_configuration_file.json>`` - A subgraph replacement configuration file with transformations description. For the models downloaded from the TensorFlow Object Detection API zoo, you can find the configuration files in the ``<PYTHON_SITE_PACKAGES>/openvino/tools/mo/front/tf`` directory. Use:
* ``ssd_v2_support.json`` - for frozen SSD topologies from the models zoo version up to 1.13.X inclusively
* ``ssd_support_api_v.1.14.json`` - for SSD topologies trained using the TensorFlow Object Detection API version 1.14 up to 1.14.X inclusively
@ -48,12 +48,12 @@ To convert a TensorFlow Object Detection API model, run the ``mo`` command with
* ``rfcn_support_api_v1.13.json`` - for RFCN topology from the models zoo frozen with TensorFlow version 1.13.X
* ``rfcn_support_api_v1.14.json`` - for RFCN topology from the models zoo frozen with TensorFlow version 1.14.0 or higher
* ``--tensorflow_object_detection_api_pipeline_config <path_to_pipeline.config>`` - A special configuration file that describes the topology hyper-parameters and structure of the TensorFlow Object Detection API model. For the models downloaded from the TensorFlow Object Detection API zoo, the configuration file is named ``pipeline.config``. If you plan to train a model yourself, you can find templates for these files in the `models repository <https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs>`__.
* ``--input_shape`` (optional) - A custom input image shape. For more information how the ``--input_shape`` parameter is handled for the TensorFlow Object Detection API models, refer to the `Custom Input Shape <#Custom-Input-Shape>`__ guide.
* ``tensorflow_object_detection_api_pipeline_config <path_to_pipeline.config>`` - A special configuration file that describes the topology hyper-parameters and structure of the TensorFlow Object Detection API model. For the models downloaded from the TensorFlow Object Detection API zoo, the configuration file is named ``pipeline.config``. If you plan to train a model yourself, you can find templates for these files in the `models repository <https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs>`__.
* ``input_shape`` (optional) - A custom input image shape. For more information how the ``input_shape`` parameter is handled for the TensorFlow Object Detection API models, refer to the `Custom Input Shape <#Custom-Input-Shape>`__ guide.
.. note::
The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``--reverse_input_channels``. Otherwise, inference results may be incorrect. If you convert a TensorFlow Object Detection API model to use with the OpenVINO sample applications, you must specify the ``--reverse_input_channels`` parameter. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``reverse_input_channels``. Otherwise, inference results may be incorrect. If you convert a TensorFlow Object Detection API model to use with the OpenVINO sample applications, you must specify the ``reverse_input_channels`` parameter. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
Additionally to the mandatory parameters listed above you can use optional conversion parameters if needed. A full list of parameters is available in the :doc:`Converting a TensorFlow Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
@ -84,82 +84,82 @@ There are several important notes about feeding input images to the samples:
2. TensorFlow implementation of image resize may be different from the one implemented in the sample. Even reading input image from compressed format (like ``.jpg``) could give different results in the sample and TensorFlow. If it is necessary to compare accuracy between the TensorFlow and the OpenVINO, it is recommended to pass pre-resized input image in a non-compressed format (like ``.bmp``).
3. If you want to infer the model with the OpenVINO samples, convert the model specifying the ``--reverse_input_channels`` command line parameter. The samples load images in BGR channels order, while TensorFlow models were trained with images in RGB order. When the ``--reverse_input_channels`` command line parameter is specified, Model Optimizer performs first convolution or other channel dependent operation weights modification so the output will be like the image is passed with RGB channels order.
3. If you want to infer the model with the OpenVINO samples, convert the model specifying the ``reverse_input_channels`` command line parameter. The samples load images in BGR channels order, while TensorFlow models were trained with images in RGB order. When the ``reverse_input_channels`` command line parameter is specified, model conversion API performs first convolution or other channel dependent operation weights modification so the output will be like the image is passed with RGB channels order.
4. Read carefully the messages printed by Model Optimizer during a model conversion. They contain important instructions on how to prepare input data before running the inference and how to interpret the output.
4. Read carefully the messages printed by model conversion API. They contain important instructions on how to prepare input data before running the inference and how to interpret the output.
Custom Input Shape
##################
Model Optimizer handles the command line parameter ``--input_shape`` for TensorFlow Object Detection API models in a special way depending on the image resizer type defined in the ``pipeline.config`` file. TensorFlow Object Detection API generates different ``Preprocessor`` sub-graph based on the image resizer type. Model Optimizer supports two types of image resizer:
Model conversion handles the command line parameter ``input_shape`` for TensorFlow Object Detection API models in a special way depending on the image resizer type defined in the ``pipeline.config`` file. TensorFlow Object Detection API generates different ``Preprocessor`` sub-graph based on the image resizer type. Model conversion API supports two types of image resizer:
* ``fixed_shape_resizer`` --- *Stretches* input image to the specific height and width. The ``pipeline.config`` snippet below shows a ``fixed_shape_resizer`` sample definition:
.. code-block:: sh
.. code-block:: sh
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
}
* ``keep_aspect_ratio_resizer`` --- Resizes the input image *keeping aspect ratio* to satisfy the minimum and maximum size constraints. The ``pipeline.config`` snippet below shows a ``keep_aspect_ratio_resizer`` sample definition:
.. code-block:: sh
.. code-block:: sh
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 600
max_dimension: 1024
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 600
max_dimension: 1024
}
}
}
If an additional parameter "pad_to_max_dimension" is equal to "true", then the resized image will be padded with 0s to the square image of size "max_dimension".
Fixed Shape Resizer Replacement
+++++++++++++++++++++++++++++++
* If the ``--input_shape`` command line parameter is not specified, Model Optimizer generates an input operation with the height and width as defined in the ``pipeline.config``.
* If the ``input_shape`` command line parameter is not specified, model conversion generates an input operation with the height and width as defined in the ``pipeline.config``.
* If the ``--input_shape [1, H, W, 3]`` command line parameter is specified, Model Optimizer sets the input operation height to ``H`` and width to ``W`` and convert the model. However, the conversion may fail because of the following reasons:
* If the ``input_shape [1, H, W, 3]`` command line parameter is specified, model conversion sets the input operation height to ``H`` and width to ``W`` and convert the model. However, the conversion may fail because of the following reasons:
* The model is not reshape-able, meaning that it's not possible to change the size of the model input image. For example, SSD FPN models have ``Reshape`` operations with hard-coded output shapes, but the input size to these ``Reshape`` instances depends on the input image size. In this case, Model Optimizer shows an error during the shape inference phase. Run Model Optimizer with ``--log_level DEBUG`` to see the inferred operations output shapes to see the mismatch.
* Custom input shape is too small. For example, if you specify ``--input_shape [1,100,100,3]`` to convert a SSD Inception V2 model, one of convolution or pooling nodes decreases input tensor spatial dimensions to non-positive values. In this case, Model Optimizer shows error message like this: '[ ERROR ] Shape [ 1 -1 -1 256] is not fully defined for output X of "node_name".'
* The model is not reshape-able, meaning that it's not possible to change the size of the model input image. For example, SSD FPN models have ``Reshape`` operations with hard-coded output shapes, but the input size to these ``Reshape`` instances depends on the input image size. In this case, model conversion API shows an error during the shape inference phase. Run model conversion with ``log_level DEBUG`` to see the inferred operations output shapes to see the mismatch.
* Custom input shape is too small. For example, if you specify ``input_shape [1,100,100,3]`` to convert a SSD Inception V2 model, one of convolution or pooling nodes decreases input tensor spatial dimensions to non-positive values. In this case, model conversion API shows error message like this: '[ ERROR ] Shape [ 1 -1 -1 256] is not fully defined for output X of "node_name".'
Keeping Aspect Ratio Resizer Replacement
++++++++++++++++++++++++++++++++++++++++
* If the ``--input_shape`` command line parameter is not specified, Model Optimizer generates an input operation with both height and width equal to the value of parameter ``min_dimension`` in the ``keep_aspect_ratio_resizer``.
* If the ``input_shape`` command line parameter is not specified, model conversion API generates an input operation with both height and width equal to the value of parameter ``min_dimension`` in the ``keep_aspect_ratio_resizer``.
* If the ``--input_shape [1, H, W, 3]`` command line parameter is specified, Model Optimizer scales the specified input image height ``H`` and width ``W`` to satisfy the ``min_dimension`` and ``max_dimension`` constraints defined in the ``keep_aspect_ratio_resizer``. The following function calculates the input operation height and width:
* If the ``input_shape [1, H, W, 3]`` command line parameter is specified, model conversion API scales the specified input image height ``H`` and width ``W`` to satisfy the ``min_dimension`` and ``max_dimension`` constraints defined in the ``keep_aspect_ratio_resizer``. The following function calculates the input operation height and width:
.. code-block:: py
.. code-block:: py
def calculate_shape_keeping_aspect_ratio(H: int, W: int, min_dimension: int, max_dimension: int):
ratio_min = min_dimension / min(H, W)
ratio_max = max_dimension / max(H, W)
ratio = min(ratio_min, ratio_max)
return int(round(H * ratio)), int(round(W * ratio))
def calculate_shape_keeping_aspect_ratio(H: int, W: int, min_dimension: int, max_dimension: int):
ratio_min = min_dimension / min(H, W)
ratio_max = max_dimension / max(H, W)
ratio = min(ratio_min, ratio_max)
return int(round(H * ratio)), int(round(W * ratio))
The ``--input_shape`` command line parameter should be specified only if the "pad_to_max_dimension" does not exist of is set to "false" in the ``keep_aspect_ratio_resizer``.
The ``input_shape`` command line parameter should be specified only if the "pad_to_max_dimension" does not exist of is set to "false" in the ``keep_aspect_ratio_resizer``.
Models with ``keep_aspect_ratio_resizer`` were trained to recognize object in real aspect ratio, in contrast with most of the classification topologies trained to recognize objects stretched vertically and horizontally as well. By default, Model Optimizer converts topologies with ``keep_aspect_ratio_resizer`` to consume a square input image. If the non-square image is provided as input, it is stretched without keeping aspect ratio that results to object detection quality decrease.
Models with ``keep_aspect_ratio_resizer`` were trained to recognize object in real aspect ratio, in contrast with most of the classification topologies trained to recognize objects stretched vertically and horizontally as well. By default, topologies are converted with ``keep_aspect_ratio_resizer`` to consume a square input image. If the non-square image is provided as input, it is stretched without keeping aspect ratio that results to object detection quality decrease.
.. note::
It is highly recommended to specify the ``--input_shape`` command line parameter for the models with ``keep_aspect_ratio_resizer``, if the input image dimensions are known in advance.
It is highly recommended to specify the ``input_shape`` command line parameter for the models with ``keep_aspect_ratio_resizer``, if the input image dimensions are known in advance.
Model Conversion Process in Detail
##################################
This section is intended for users who want to understand how Model Optimizer performs Object Detection API models conversion in details. The information in this section is also useful for users having complex models that are not converted with Model Optimizer out of the box. It is highly recommended to read the **Graph Transformation Extensions** section in the :doc:`[Legacy] Model Optimizer Extensibility <openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Model_Optimizer_Extensions_Model_Optimizer_Transformation_Extensions>` documentation first to understand sub-graph replacement concepts which are used here.
This section is intended for users who want to understand how model conversion API performs Object Detection API models conversion in details. The information in this section is also useful for users having complex models that are not converted with model conversion API out of the box. It is highly recommended to read the **Graph Transformation Extensions** section in the :doc:`[Legacy] Model Optimizer Extensibility <openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer>` documentation first to understand sub-graph replacement concepts which are used here.
It is also important to open the model in the `TensorBoard <https://www.tensorflow.org/guide/summaries_and_tensorboard>`__ to see the topology structure. Model Optimizer can create an event file that can be then fed to the TensorBoard tool. Run Model Optimizer, providing two command line parameters:
It is also important to open the model in the `TensorBoard <https://www.tensorflow.org/guide/summaries_and_tensorboard>`__ to see the topology structure. Model conversion API can create an event file that can be then fed to the TensorBoard tool. Run model conversion, providing two command line parameters:
* ``--input_model <path_to_frozen.pb>`` --- Path to the frozen model.
* ``--tensorboard_logdir`` --- Path to the directory where TensorBoard looks for the event files.
* ``input_model <path_to_frozen.pb>`` --- Path to the frozen model.
* ``tensorboard_logdir`` --- Path to the directory where TensorBoard looks for the event files.
Implementation of the transformations for Object Detection API models is located in the `file <https://github.com/openvinotoolkit/openvino/blob/releases/2022/1/tools/mo/openvino/tools/mo/front/tf/ObjectDetectionAPI.py>`__. Refer to the code in this file to understand the details of the conversion process.

4
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_RetinaNet_From_Tensorflow.md
View File

@ -7,14 +7,14 @@ This tutorial explains how to convert a RetinaNet model to the Intermediate Repr
`Public RetinaNet model <https://github.com/fizyr/keras-retinanet>`__ does not contain pretrained TensorFlow weights.
To convert this model to the TensorFlow format, follow the `Reproduce Keras to TensorFlow Conversion tutorial <https://docs.openvino.ai/2023.0/omz_models_model_retinanet_tf.html>`__.
After converting the model to TensorFlow format, run the Model Optimizer command below:
After converting the model to TensorFlow format, run the following command:
.. code-block:: sh
mo --input "input_1[1,1333,1333,3]" --input_model retinanet_resnet50_coco_best_v2.1.0.pb --transformations_config front/tf/retinanet.json
Where ``transformations_config`` command-line parameter specifies the configuration json file containing model conversion hints for the Model Optimizer.
Where ``transformations_config`` command-line parameter specifies the configuration json file containing model conversion hints for model conversion API.
The json file contains some parameters that need to be changed if you train the model yourself. It also contains information on how to match endpoints
to replace the subgraph nodes. After the model is converted to the OpenVINO IR format, the output nodes will be replaced with DetectionOutput layer.

10
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_Slim_Library_Models.md
View File

@ -7,7 +7,7 @@
1. Download the TensorFlow-Slim models `git repository <https://github.com/tensorflow/models>`__.
2. Download the pre-trained model `checkpoint <https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models>`__.
3. Export the inference graph.
4. Convert the model using the Model Optimizer.
4. Convert the model using model conversion API.
The `Example of an Inception V1 Model Conversion <#example_of_an_inception_v1_model_conversion>`__ below illustrates the process of converting an Inception V1 Model.
@ -46,7 +46,7 @@ This example demonstrates how to convert the model on Linux OSes, but it could b
--output_file inception_v1_inference_graph.pb
Model Optimizer comes with the summarize graph utility, which identifies graph input and output nodes. Run the utility to determine input/output nodes of the Inception V1 model:
Model conversion API comes with the summarize graph utility, which identifies graph input and output nodes. Run the utility to determine input/output nodes of the Inception V1 model:
.. code-block:: sh
@ -63,21 +63,21 @@ The output looks as follows:
The tool finds one input node with name ``input``, type ``float32``, fixed image size ``(224,224,3)`` and undefined batch size ``-1``. The output node name is ``InceptionV1/Logits/Predictions/Reshape_1``.
**Step 4**. Convert the model with the Model Optimizer:
**Step 4**. Convert the model with the model conversion API:
.. code-block:: sh
mo --input_model ./inception_v1_inference_graph.pb --input_checkpoint ./inception_v1.ckpt -b 1 --mean_value [127.5,127.5,127.5] --scale 127.5
The ``-b`` command line parameter is required because the Model Optimizer cannot convert a model with undefined input size.
The ``-b`` command line parameter is required because model conversion API cannot convert a model with undefined input size.
For the information on why ``--mean_values`` and ``--scale`` command-line parameters are used, refer to the `Mean and Scale Values for TensorFlow-Slim Models <#Mean-and-Scale-Values-for-TensorFlow-Slim-Models>`__.
Mean and Scale Values for TensorFlow-Slim Models
#################################################
The TensorFlow-Slim Models were trained with normalized input data. There are several different normalization algorithms used in the Slim library. OpenVINO classification sample does not perform image pre-processing except resizing to the input layer size. It is necessary to pass mean and scale values to the Model Optimizer so they are embedded into the generated IR in order to get correct classification results.
The TensorFlow-Slim Models were trained with normalized input data. There are several different normalization algorithms used in the Slim library. OpenVINO classification sample does not perform image pre-processing except resizing to the input layer size. It is necessary to pass mean and scale values to model conversion API so they are embedded into the generated IR in order to get correct classification results.
The file `preprocessing_factory.py <https://github.com/tensorflow/models/blob/master/research/slim/preprocessing/preprocessing_factory.py>`__ contains a dictionary variable ``preprocessing_fn_map`` defining mapping between the model type and pre-processing function to be used. The function code should be analyzed to figure out the mean/scale values.

2
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_XLNet_From_Tensorflow.md
View File

@ -187,7 +187,7 @@ The script should save into ``~/XLNet-Large/xlnet``.
Converting a frozen TensorFlow XLNet Model to IR
#################################################
To generate the XLNet Intermediate Representation (IR) of the model, run Model Optimizer with the following parameters:
To generate the XLNet Intermediate Representation (IR) of the model, run model conversion with the following parameters:
.. code-block:: sh

24
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_YOLO_From_Tensorflow.md
View File

@ -7,7 +7,7 @@ This document explains how to convert real-time object detection YOLOv1, YOLOv2,
* The ``.cfg`` file with model configurations
* The ``.weights`` file with model weights
Depending on a YOLO model version, the Model Optimizer converts it differently:
Depending on a YOLO model version, the ``convert_model()`` method converts it differently:
- YOLOv4 must be first converted from Keras to TensorFlow 2.
- YOLOv3 has several implementations. This tutorial uses a TensorFlow implementation of YOLOv3 model, which can be directly converted to an IR.
@ -46,11 +46,11 @@ This section explains how to convert the YOLOv4 Keras model from the `repository
python keras-YOLOv3-model-set/tools/model_converter/convert.py <path_to_cfg_file>/yolov4-tiny.cfg <path_to_weights>/yolov4-tiny.weights <saved_model_dir>
4. Run Model Optimizer to converter the model from the TensorFlow 2 format to an IR:
4. Run model conversion for from the TensorFlow 2 format to an IR:
.. note::
Before you run the conversion, make sure you have installed all the Model Optimizer dependencies for TensorFlow 2.
Before you run the conversion, make sure you have installed all the model conversion API dependencies for TensorFlow 2.
.. code-block:: sh
@ -162,7 +162,7 @@ where:
- ``id`` and ``match_kind`` are parameters that you cannot change.
- ``custom_attributes`` is a parameter that stores all the YOLOv3 specific attributes:
- ``classes``, ``coords``, ``num``, and ``masks`` are attributes that you should copy from the configuration file that was used for model training. If you used DarkNet officially shared weights, you can use ``yolov3.cfg`` or ``yolov3-tiny.cfg`` configuration file from `GitHub repository <https://github.com/david8862/keras-YOLOv3-model-set/tree/master/cfg>`__. eplace the default values in ``custom_attributes`` with the parameters that follow the ``[yolo]`` titles in the configuration file.
- ``classes``, ``coords``, ``num``, and ``masks`` are attributes that you should copy from the configuration file that was used for model training. If you used DarkNet officially shared weights, you can use ``yolov3.cfg`` or ``yolov3-tiny.cfg`` configuration file from `GitHub repository <https://github.com/david8862/keras-YOLOv3-model-set/tree/master/cfg>`__. Replace the default values in ``custom_attributes`` with the parameters that follow the ``[yolo]`` titles in the configuration file.
- ``anchors`` is an optional parameter that is not used while inference of the model, but it used in a demo to parse ``Region`` layer output
- ``entry_points`` is a node name list to cut off the model and append the ``Region`` layer with custom attributes specified above.
@ -191,12 +191,12 @@ To generate an IR of the YOLOv3-tiny TensorFlow model, run:
where:
* ``--batch`` defines shape of model input. In the example, ``--batch`` is equal to 1, but you can also specify other integers larger than 1.
* ``--transformations_config`` adds missing ``Region`` layers to the model. In the IR, the ``Region`` layer has name ``RegionYolo``.
* ``batch`` defines shape of model input. In the example, ``batch`` is equal to 1, but you can also specify other integers larger than 1.
* ``transformations_config`` adds missing ``Region`` layers to the model. In the IR, the ``Region`` layer has name ``RegionYolo``.
.. note::
The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``--reverse_input_channels``. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``reverse_input_channels``. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
OpenVINO toolkit provides a demo that uses YOLOv3 model. Refer to the :doc:`Object Detection C++ Demo <omz_demos_object_detection_demo_cpp>` for more information.
@ -281,7 +281,7 @@ To recreate the original model structure, use the corresponding yolo ``.json`` c
If chosen model has specific values of these parameters, create another configuration file with custom operations and use it for conversion.
To generate the IR of the YOLOv1 model, provide TensorFlow YOLOv1 or YOLOv2 model to Model Optimizer with the following parameters:
To generate the IR of the YOLOv1 model, provide TensorFlow YOLOv1 or YOLOv2 model to model conversion API with the following parameters:
.. code-block:: sh
@ -294,13 +294,13 @@ To generate the IR of the YOLOv1 model, provide TensorFlow YOLOv1 or YOLOv2 mode
where:
* ``--batch`` defines shape of model input. In the example, ``--batch`` is equal to 1, but you can also specify other integers larger than 1.
* ``--scale`` specifies scale factor that input values will be divided by. The model was trained with input values in the range ``[0,1]``. OpenVINO toolkit samples read input images as values in ``[0,255]`` range, so the scale 255 must be applied.
* ``--transformations_config`` adds missing ``Region`` layers to the model. In the IR, the ``Region`` layer has name ``RegionYolo``. For other applicable parameters, refer to the :doc:`Convert Model from TensorFlow <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
* ``batch`` defines shape of model input. In the example, ``batch`` is equal to 1, but you can also specify other integers larger than 1.
* ``scale`` specifies scale factor that input values will be divided by. The model was trained with input values in the range ``[0,1]``. OpenVINO toolkit samples read input images as values in ``[0,255]`` range, so the scale 255 must be applied.
* ``transformations_config`` adds missing ``Region`` layers to the model. In the IR, the ``Region`` layer has name ``RegionYolo``. For other applicable parameters, refer to the :doc:`Convert Model from TensorFlow <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
.. note::
The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``--reverse_input_channels``. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the ``RGB<->BGR`` conversion specifying the command-line parameter: ``reverse_input_channels``. Otherwise, inference results may be incorrect. For more information about the parameter, refer to the **When to Reverse Input Channels** section of the :doc:`Converting a Model to Intermediate Representation (IR) <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
@endsphinxdirective

52
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_lm_1b_From_Tensorflow.md
View File

@ -11,55 +11,55 @@ To download the model for IR conversion, follow the instructions:
1. Create new directory to store the model:
.. code-block:: shell
.. code-block:: sh
mkdir lm_1b
mkdir lm_1b
2. Go to the ``lm_1b`` directory:
.. code-block:: shell
.. code-block:: sh
cd lm_1b
cd lm_1b
3. Download the model GraphDef file:
.. code-block:: shell
.. code-block:: sh
wget http://download.tensorflow.org/models/LM_LSTM_CNN/graph-2016-09-10.pbtxt
wget http://download.tensorflow.org/models/LM_LSTM_CNN/graph-2016-09-10.pbtxt
4. Create new directory to store 12 checkpoint shared files:
.. code-block:: shell
.. code-block:: sh
mkdir ckpt
mkdir ckpt
5. Go to the ``ckpt`` directory:
.. code-block:: shell
.. code-block:: sh
cd ckpt
cd ckpt
6. Download 12 checkpoint shared files:
.. code-block:: shell
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-base
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-char-embedding
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-lstm
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax0
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax1
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax2
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax3
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax4
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax5
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax6
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax7
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax8
.. code-block:: sh
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-base
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-char-embedding
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-lstm
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax0
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax1
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax2
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax3
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax4
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax5
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax6
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax7
wget http://download.tensorflow.org/models/LM_LSTM_CNN/all_shards-2016-09-10/ckpt-softmax8
Once you have downloaded the pretrained model files, you will have the ``lm_1b`` directory with the following hierarchy:
.. code-block:: shell
.. code-block:: sh
lm_1b/
graph-2016-09-10.pbtxt
@ -103,7 +103,7 @@ There is a certain limitation for the model conversion: the original model canno
To generate the ``lm_1b`` Intermediate Representation (IR), provide TensorFlow ``lm_1b`` model to the
Model Optimizer with parameters:
.. code-block:: shell
.. code-block:: sh
mo
--input_model lm_1b/graph-2016-09-10.pbtxt \

17
docs/MO_DG/prepare_model/model_inputs_outputs.md
View File

@ -1,17 +0,0 @@
# Model Inputs and Outputs, Shapes and Layouts {#openvino_docs_model_inputs_outputs}
@sphinxdirective
Users interact with a model by passing data to its *inputs* before the inference and retrieving data from its *outputs* after the inference. A model may have one or multiple inputs and outputs. Normally, in OpenVINO™ toolkit, all inputs and outputs in the converted model are identified in the same way as in the original framework model.
OpenVINO uses the *names of tensors* for identification. Depending on the framework, the names of tensors are formed differently.
A model accepts inputs and produces outputs of some *shape*. Shape defines the number of dimensions in a tensor and their order. For example, an image classification model can accept tensor of shape [1, 3, 240, 240] and produces tensor of shape [1, 1000].
The meaning of each dimension in the shape is specified by its *layout*. Layout is an interpretation of shape dimensions. OpenVINO toolkit conversion tools and APIs keep all dimensions and their order unchanged and aligned with the original framework model. Usually, original models do not contain layout information explicitly, but in various pre-processing and post-processing scenarios in the OpenVINO Runtime API, sometimes it is required to have the layout specified explicitly. We recommend specifying layouts for inputs/outputs during the model conversion.
OpenVINO also supports *partially defined shapes*, where part of the dimensions is undefined. Undefined dimensions are also kept intact in the final IR file and you can define them later, during runtime. Undefined dimensions can be used as :doc:`dynamic dimensions <openvino_docs_OV_UG_DynamicShapes>` for certain hardware and models, which enables you to change shapes of input data dynamically in each infer request. For example, the sequence length dimension in the BERT model can be left undefined and variously sized data along this dimension can be fed on the CPU.
To learn about how the model is represented in OpenVINO™ Runtime, see the :doc:`Model Representation in OpenVINO™ Runtime <openvino_docs_OV_UG_Model_Representation>`.
@endsphinxdirective

4
docs/OV_Runtime_UG/ShapeInference.md
View File

@ -16,8 +16,8 @@ The following instructions are for cases where you need to change the model inpu
.. note::
If you need to do this only once, prepare a model with updated shapes via
:doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
For more information, refer to the :ref:`Specifying --input_shape Command-line Parameter <when_to_specify_input_shapes>` article.
:doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
For more information, refer to the :ref:`Specifying input_shape Parameter <when_to_specify_input_shapes>` article.
The reshape method

4
docs/OV_Runtime_UG/Troubleshooting_ReshapeMethod.md
View File

@ -25,7 +25,7 @@ How To Fix Non-Reshape-able Model
To fix some operators which prevent normal shape propagation:
* see if the issue can be fixed via changing the values of some operators' input. For example, the most common problem of non-reshape-able models is a ``Reshape`` operator with a hard-coded output shape. You can cut-off the hard-coded second input of ``Reshape`` and fill it in with relaxed values. For the following example in the diagram below, the Model Optimizer CLI should read:
* see if the issue can be fixed via changing the values of some operators' input. For example, the most common problem of non-reshape-able models is a ``Reshape`` operator with a hard-coded output shape. You can cut-off the hard-coded second input of ``Reshape`` and fill it in with relaxed values. For the following example in the diagram below, the model conversion API command line should read:
.. code-block:: sh
@ -38,7 +38,7 @@ To fix some operators which prevent normal shape propagation:
.. image:: _static/images/batch_relaxation.png
* transform the model during Model Optimizer conversion on the back phase. For more information, see the :doc:`Model Optimizer extension <openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer>`,
* transform the model conversion on the back phase. For more information, see the :doc:`How to Convert a Model <openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer>`,
* transform OpenVINO Model during the runtime. For more information, see :doc:`OpenVINO Runtime Transformations <openvino_docs_transformations>`,
* modify the original model with the help of the original framework.

2
docs/OV_Runtime_UG/deployment/local-distribution.md
View File

@ -120,7 +120,7 @@ Depending on the model format types that are used in the application in `ov::Cor
.. note::
To optimize the size of final distribution package, you are recommended to convert models to OpenVINO IR by using :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`. This way you don't have to keep TensorFlow, TensorFlow Lite, ONNX, PaddlePaddle, and other frontend libraries in the distribution package.
To optimize the size of final distribution package, you are recommended to convert models to OpenVINO IR by using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`. This way you don't have to keep TensorFlow, TensorFlow Lite, ONNX, PaddlePaddle, and other frontend libraries in the distribution package.
(Legacy) Preprocessing via G-API
++++++++++++++++++++++++++++++++

14
docs/OV_Runtime_UG/migration_ov_2_0/intro.md
View File

@ -20,14 +20,14 @@ Introduction of API 2.0
Versions of OpenVINO prior to 2022.1 required changes in the application logic when migrating an app from other frameworks, such as TensorFlow, ONNX Runtime, PyTorch, PaddlePaddle, etc. The changes were required because:
- Model Optimizer changed input precisions for some inputs. For example, neural language processing models with ``I64`` inputs were changed to include ``I32`` ones.
- Model Optimizer changed layouts for TensorFlow models (see the :doc:`Layouts in OpenVINO <openvino_docs_OV_UG_Layout_Overview>`). It lead to unusual requirement of using the input data with a different layout than that of the framework:
- Model conversion API changed input precisions for some inputs. For example, neural language processing models with ``I64`` inputs were changed to include ``I32`` ones.
- Model conversion API changed layouts for TensorFlow models (see the :doc:`Layouts in OpenVINO <openvino_docs_OV_UG_Layout_Overview>`). It lead to unusual requirement of using the input data with a different layout than that of the framework:
.. image:: _static/images/tf_openvino.svg
:alt: tf_openvino
- Inference Engine API (`InferenceEngine::CNNNetwork <classInferenceEngine_1_1CNNNetwork.html#doxid-class-inference-engine-1-1-c-n-n-network>`__) applied some conversion rules for input and output precisions due to limitations in device plugins.
- Users needed to specify input shapes during model conversions in Model Optimizer, and work with static shapes in the application.
- Users needed to specify input shapes during model conversions in model conversion API, and work with static shapes in the application.
OpenVINO™ 2022.1 has introduced API 2.0 (also called OpenVINO API v2) to align the logic of working with models as it is done in their origin frameworks - no layout and precision changes, operating with tensor names and indices to address inputs and outputs. OpenVINO Runtime has combined Inference Engine API used for inference and nGraph API targeted to work with models and operations. API 2.0 has a common structure, naming convention styles, namespaces, and removes duplicated structures. For more details, see the :doc:`Changes to Inference Pipeline in OpenVINO API v2 <openvino_2_0_inference_pipeline>`.
@ -39,7 +39,7 @@ OpenVINO™ 2022.1 has introduced API 2.0 (also called OpenVINO API v2) to align
The New OpenVINO IR v11
#######################
To support these features, OpenVINO has introduced OpenVINO IR v11, which is now the default version for Model Optimizer. The model represented in OpenVINO IR v11 fully matches the original model in the original framework format in terms of inputs and outputs. It is also not required to specify input shapes during conversion, which results in OpenVINO IR v11 containing ``-1`` to denote undefined dimensions. For more details on how to fully utilize this feature, see :doc:`Working with dynamic shapes <openvino_docs_OV_UG_DynamicShapes>`. For information on how to reshape to static shapes in application, see :doc:`Changing input shapes <openvino_docs_OV_UG_ShapeInference>`.
To support these features, OpenVINO has introduced OpenVINO IR v11, which is now the default version for model conversion API. The model represented in OpenVINO IR v11 fully matches the original model in the original framework format in terms of inputs and outputs. It is also not required to specify input shapes during conversion, which results in OpenVINO IR v11 containing ``-1`` to denote undefined dimensions. For more details on how to fully utilize this feature, see :doc:`Working with dynamic shapes <openvino_docs_OV_UG_DynamicShapes>`. For information on how to reshape to static shapes in application, see :doc:`Changing input shapes <openvino_docs_OV_UG_ShapeInference>`.
OpenVINO IR v11 is fully compatible with applications written with the Inference Engine API used by older versions of OpenVINO. This backward compatibility is allowed thanks to additional runtime information included in OpenVINO IR v11. This means that when OpenVINO IR v11 is read by an application based on Inference Engine, it is internally converted to OpenVINO IR v10.
@ -55,7 +55,7 @@ Some of the OpenVINO Development Tools also support both OpenVINO IR v10 and v11
- Accuracy checker uses API 2.0 for model accuracy measurement by default. It also supports switching to the old API by using the ``--use_new_api False`` command-line parameter. Both launchers accept OpenVINO IR v10 and v11, but in some cases configuration files should be updated. For more details, see the `Accuracy Checker documentation <https://github.com/openvinotoolkit/open_model_zoo/blob/master/tools/accuracy_checker/openvino/tools/accuracy_checker/launcher/openvino_launcher_readme.md>`__.
- :doc:`Compile tool <openvino_ecosystem>` compiles the model to be used in API 2.0 by default. To use the resulting compiled blob under the Inference Engine API, the additional ``ov_api_1_0`` option should be passed.
However, Post-Training Optimization Tool of OpenVINO 2022.1 does not support OpenVINO IR v10. They require the latest version of Model Optimizer to generate OpenVINO IR v11 files.
However, Post-Training Optimization Tool of OpenVINO 2022.1 does not support OpenVINO IR v10. They require the latest version of model conversion API to generate OpenVINO IR v11 files.
.. note::
@ -76,14 +76,14 @@ To understand the differences between Inference Engine API and API 2.0, see the
- **Old behavior** of OpenVINO assumes that:
- Model Optimizer can change input element types and order of dimensions (layouts) for the model from the original framework.
- Model Conversion API can change input element types and order of dimensions (layouts) for the model from the original framework.
- Inference Engine can override input and output element types.
- Inference Engine API uses operation names to address inputs and outputs (e.g. `InferenceEngine::InferRequest::GetBlob <classInferenceEngine_1_1InferRequest.html#doxid-class-inference-engine-1-1-infer-request-1a9601a4cda3f309181af34feedf1b914c>`__).
- Inference Engine API does not support compiling of models with dynamic input shapes.
- **New behavior** implemented in 2022.1 assumes full model alignment with the framework:
- Model Optimizer preserves input element types and order of dimensions (layouts), and stores tensor names from the original models.
- Model Conversion API preserves input element types and order of dimensions (layouts), and stores tensor names from the original models.
- OpenVINO Runtime 2022.1 reads models in any format (OpenVINO IR v10, OpenVINO IR v11, TensorFlow (check :doc:`TensorFlow Frontend Capabilities and Limitations <openvino_docs_MO_DG_TensorFlow_Frontend>`), ONNX, PaddlePaddle, etc.).
- API 2.0 uses tensor names for addressing, which is the standard approach among the compatible model frameworks.
- API 2.0 can also address input and output tensors by the index. Some model formats like ONNX are sensitive to the input and output order, which is preserved by OpenVINO 2022.1.

2
docs/OV_Runtime_UG/migration_ov_2_0/preprocessing.md
View File

@ -23,7 +23,7 @@ More importantly, API 2.0 does not assume any default layouts as Inference Engin
.. note::
Use Model Optimizer preprocessing capabilities to insert preprocessing operations in your model for optimization. Thus, the application does not need to read the model and set preprocessing repeatedly. You can use the :doc:`model caching feature <openvino_docs_OV_UG_Model_caching_overview>` to improve the time-to-inference.
Use model conversion API preprocessing capabilities to insert preprocessing operations in your model for optimization. Thus, the application does not need to read the model and set preprocessing repeatedly. You can use the :doc:`model caching feature <openvino_docs_OV_UG_Model_caching_overview>` to improve the time-to-inference.
The following sections demonstrate how to migrate preprocessing scenarios from Inference Engine API to API 2.0.
The snippets assume that you need to preprocess a model input with the ``tensor_name`` in Inference Engine API, using ``operation_name`` to address the data.

6
docs/OV_Runtime_UG/model_state_intro.md
View File

@ -26,7 +26,7 @@ OpenVINO State Representation
OpenVINO contains the ``Variable``, a special abstraction to represent a state in a model. There are two operations: :doc:`Assign <openvino_docs_ops_infrastructure_Assign_3>` - to save a value in a state and :doc:`ReadValue <openvino_docs_ops_infrastructure_ReadValue_3>` - to read a value saved on previous iteration.
To get a model with states ready for inference, convert a model from another framework to OpenVINO IR with Model Optimizer or create an OpenVINO model.
To get a model with states ready for inference, convert a model from another framework to OpenVINO IR with model conversion API or create an OpenVINO model.
(For more information, refer to the :doc:`Build OpenVINO Model section <openvino_docs_OV_UG_Model_Representation>`).
Below is the graph in both forms:
@ -206,7 +206,7 @@ If the original framework does not have a special API for working with states, O
TensorIterator/Loop operations
++++++++++++++++++++++++++++++
You can get the TensorIterator/Loop operations from different frameworks via Model Optimizer.
You can get the TensorIterator/Loop operations from different frameworks via model conversion API.
* **ONNX and frameworks supported via ONNX format** - ``LSTM``, ``RNN``, and ``GRU`` original layers are converted to the ``TensorIterator`` operation. The ``TensorIterator`` body contains ``LSTM``/``RNN``/``GRU Cell``. The ``Peepholes`` and ``InputForget`` modifications are not supported, while the ``sequence_lengths`` optional input is.
``ONNX Loop`` layer is converted to the OpenVINO :doc:`Loop <openvino_docs_ops_infrastructure_Loop_5>` operation.
@ -214,7 +214,7 @@ You can get the TensorIterator/Loop operations from different frameworks via Mod
* **Apache MXNet** - ``LSTM``, ``RNN``, ``GRU`` original layers are converted to ``TensorIterator`` operation, which body contains ``LSTM``/``RNN``/``GRU Cell`` operations.
* **TensorFlow** - ``BlockLSTM`` is converted to ``TensorIterator`` operation. The ``TensorIterator`` body contains ``LSTM Cell`` operation, whereas ``Peepholes`` and ``InputForget`` modifications are not supported.
The ``While`` layer is converted to ``TensorIterator``, which body can contain any supported operations. However, when count of iterations cannot be calculated in shape inference (Model Optimizer conversion) time, the dynamic cases are not supported.
The ``While`` layer is converted to ``TensorIterator``, which body can contain any supported operations. However, when count of iterations cannot be calculated in shape inference (model conversion) time, the dynamic cases are not supported.
* **TensorFlow2** - ``While`` layer is converted to ``Loop`` operation, which body can contain any supported operations.

6
docs/OV_Runtime_UG/ov_dynamic_shapes.md
View File

@ -55,7 +55,7 @@ Model input dimensions can be specified as dynamic using the model.reshape metho
.. note::
Some models may already have dynamic shapes out of the box and do not require additional configuration. This can either be because it was generated with dynamic shapes from the source framework, or because it was converted with Model Optimizer to use dynamic shapes. For more information, see the Dynamic Dimensions “Out of the Box” section.
Some models may already have dynamic shapes out of the box and do not require additional configuration. This can either be because it was generated with dynamic shapes from the source framework, or because it was converted with Model Conversion API to use dynamic shapes. For more information, see the Dynamic Dimensions “Out of the Box” section.
The examples below show how to set dynamic dimensions with a model that has a static ``[1, 3, 224, 224]`` input shape (such as `mobilenet-v2 <https://docs.openvino.ai/2023.0/omz_models_model_mobilenet_v2.html>`__). The first example shows how to change the first dimension (batch size) to be dynamic. In the second example, the third and fourth dimensions (height and width) are set as dynamic.
@ -111,7 +111,7 @@ For more examples of how to change multiple input layers, see :doc:`Changing Inp
Undefined Dimensions "Out Of the Box"
-------------------------------------
Many DL frameworks support generating models with dynamic (or undefined) dimensions. If such a model is converted with Model Optimizer or read directly by ``Core::read_model``, its dynamic dimensions are preserved. These models do not need any additional configuration to use them with dynamic shapes.
Many DL frameworks support generating models with dynamic (or undefined) dimensions. If such a model is converted with ``convert_model()`` or read directly by ``Core::read_model``, its dynamic dimensions are preserved. These models do not need any additional configuration to use them with dynamic shapes.
To check if a model already has dynamic dimensions, first load it with the ``read_model()`` method, then check the ``partial_shape`` property of each layer. If the model has any dynamic dimensions, they will be reported as ``?``. For example, the following code will print the name and dimensions of each input layer:
@ -134,7 +134,7 @@ To check if a model already has dynamic dimensions, first load it with the ``rea
If the input model already has dynamic dimensions, that will not change during inference. If the inputs will not be used dynamically, it is recommended to set them to static values using the ``reshape`` method to save application memory and potentially improve inference speed. The OpenVINO API supports any combination of static and dynamic dimensions.
Static and dynamic dimensions can also be set when converting the model with Model Optimizer. It has identical capabilities to the ``reshape`` method, so you can save time by converting the model with dynamic shapes beforehand rather than in the application code. To get information about setting input shapes using Model Optimizer, refer to :doc:`Setting Input Shapes <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
Static and dynamic dimensions can also be set when converting the model with ``convert_model()``. It has identical capabilities to the ``reshape`` method, so you can save time by converting the model with dynamic shapes beforehand rather than in the application code. To get information about setting input shapes using ``convert_model()``, refer to :doc:`Setting Input Shapes <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
Dimension Bounds
----------------

4
docs/OV_Runtime_UG/protecting_model_guide.md
View File

@ -11,7 +11,7 @@ This guide presents how to use OpenVINO securely with protected models.
Secure Model Deployment
#######################
After a model is optimized by the OpenVINO Model Optimizer, it's deployed to target devices in the OpenVINO Intermediate Representation (OpenVINO IR) format. An optimized model is stored on edge device and is executed by the OpenVINO Runtime. TensorFlow, TensorFlow Lite, ONNX and PaddlePaddle models can be read natively by OpenVINO Runtime as well.
After a model is optimized by model conversion API, it's deployed to target devices in the OpenVINO Intermediate Representation (OpenVINO IR) format. An optimized model is stored on edge device and is executed by the OpenVINO Runtime. TensorFlow, TensorFlow Lite, ONNX and PaddlePaddle models can be read natively by OpenVINO Runtime as well.
Encrypting and optimizing model before deploying it to the edge device can be used to protect deep-learning models. The edge device should keep the stored model protected all the time and have the model decrypted **in runtime only** for use by the OpenVINO Runtime.
@ -40,7 +40,7 @@ Additional Resources
####################
- Intel® Distribution of OpenVINO™ toolkit `home page <https://software.intel.com/en-us/openvino-toolkit>`__.
- Model Optimizer :doc:`Developer Guide <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- :doc:`OpenVINO™ Runtime User Guide <openvino_docs_OV_UG_OV_Runtime_User_Guide>`.
- For more information on Sample Applications, see the :doc:`OpenVINO Samples Overview <openvino_docs_OV_UG_Samples_Overview>`
- For information on a set of pre-trained models, see the :doc:`Overview of OpenVINO™ Toolkit Pre-Trained Models <omz_models_group_intel>`.

2
docs/OV_Runtime_UG/supported_plugins/GNA.md
View File

@ -214,7 +214,7 @@ GNA plugin natively supports stateful models. For more details on such models, r
.. note::
The GNA is typically used in streaming scenarios when minimizing latency is important. Taking into account that POT does not
support the ``TensorIterator`` operation, the recommendation is to use the ``--transform`` option of the Model Optimizer
support the ``TensorIterator`` operation, the recommendation is to use the ``transform`` option of model conversion API
to apply ``LowLatency2`` transformation when converting an original model.
Profiling

2
docs/OV_Runtime_UG/supported_plugins/GPU.md
View File

@ -450,7 +450,7 @@ GPU Performance Checklist: Summary
Since OpenVINO relies on the OpenCL kernels for the GPU implementation, many general OpenCL tips apply:
- Prefer ``FP16`` inference precision over ``FP32``, as Model Optimizer can generate both variants, and the ``FP32`` is the default. To learn about optimization options, see :doc:`Optimization Guide<openvino_docs_model_optimization_guide>`.
- Prefer ``FP16`` inference precision over ``FP32``, as Model Conversion API can generate both variants, and the ``FP32`` is the default. To learn about optimization options, see :doc:`Optimization Guide<openvino_docs_model_optimization_guide>`.
- Try to group individual infer jobs by using :doc:`automatic batching <openvino_docs_OV_UG_Automatic_Batching>`.
- Consider :doc:`caching <openvino_docs_OV_UG_Model_caching_overview>` to minimize model load time.
- If your application performs inference on the CPU alongside the GPU, or otherwise loads the host heavily, make sure that the OpenCL driver threads do not starve. :doc:`CPU configuration options <openvino_docs_OV_UG_supported_plugins_CPU>` can be used to limit the number of inference threads for the CPU plugin.

4
docs/_static/images/BASIC_FLOW_MO_simplified.svg vendored
View File

@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bc85deaa1a051454adb6efa60d8cdc54f715b57133ebc90933fbbd3a684cfd94
size 38386
oid sha256:53f6f30af6d39d91d7f3f4e3bbd086e3dbc94e5ac97233d56a90826579759e7f
size 104225

3
docs/_static/images/model_conversion_diagram.svg vendored Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9eaab59dc35a5e40bef352cf140f1a6e37b9f9bb10ae64b1eafc1b9060c361ca
size 168807

14
docs/get_started/get_started_demos.md
View File

@ -9,7 +9,7 @@ To get started, you must first install OpenVINO Runtime, install OpenVINO Develo
Once the prerequisites have been installed, perform the following steps:
1. :ref:`Use Model Downloader to download a suitable model <download-models>`.
2. :ref:`Convert the model with Model Optimizer <convert-models-to-intermediate-representation>`.
2. :ref:`Convert the model with mo <convert-models-to-intermediate-representation>`.
3. :ref:`Download media files to run inference <download-media>`.
4. :ref:`Run inference with the Image Classification sample application and see the results <run-image-classification>`.
@ -168,10 +168,10 @@ This guide used the following model to run the Image Classification Sample:
.. _convert-models-to-intermediate-representation:
Step 2: Convert the Model with Model Optimizer
##############################################
Step 2: Convert the Model with ``mo``
#####################################
In this step, your trained models are ready to run through the Model Optimizer to convert them to the IR (Intermediate Representation) format. For most model types, this is required before using OpenVINO Runtime with the model.
In this step, your trained models are ready for conversion with ``mo`` to the OpenVINO IR (Intermediate Representation) format. For most model types, this is required before using OpenVINO Runtime with the model.
Models in the IR format always include an ``.xml`` and ``.bin`` file and may also include other files such as ``.json`` or ``.mapping``. Make sure you have these files together in a single directory so OpenVINO Runtime can find them.
@ -181,7 +181,7 @@ OPTIONAL: ``model_name.json``, ``model_name.mapping``, etc.
This tutorial uses the public GoogleNet v1 Caffe model to run the Image Classification Sample. See the example in the Download Models section of this page to learn how to download this model.
The googlenet-v1 model is downloaded in the Caffe format. You must use Model Optimizer to convert the model to IR.
The googlenet-v1 model is downloaded in the Caffe format. You must use ``mo`` to convert the model to IR.
Create an ``<ir_dir>`` directory to contain the model's Intermediate Representation (IR).
@ -203,9 +203,9 @@ Create an ``<ir_dir>`` directory to contain the model's Intermediate Representat
mkdir ~/ir
To save disk space for your IR file, you can apply :doc:`weights compression to FP16 <openvino_docs_MO_DG_FP16_Compression>`. To generate an IR with FP16 weights, run Model Optimizer with the ``--compress_to_fp16`` option.
To save disk space for your IR file, you can apply :doc:`weights compression to FP16 <openvino_docs_MO_DG_FP16_Compression>`. To generate an IR with FP16 weights, run model conversion with the ``--compress_to_fp16`` option.
Generic Model Optimizer script:
Generic model conversion script:
.. code-block:: sh

13
docs/glossary.md
View File

@ -6,9 +6,9 @@
Acronyms and Abbreviations
#################################################
================== ==================================================
================== ===========================================================================
Abbreviation Description
================== ==================================================
================== ===========================================================================
API Application Programming Interface
AVX Advanced Vector Extensions
clDNN Compute Library for Deep Neural Networks
@ -32,7 +32,7 @@ Acronyms and Abbreviations
LRN Local Response Normalization
mAP Mean Average Precision
Intel® OneDNN Intel® OneAPI Deep Neural Network Library
MO Model Optimizer
`mo` Command-line tool for model conversion, CLI for ``tools.mo.convert_model``
MVN Mean Variance Normalization
NCDHW Number of images, Channels, Depth, Height, Width
NCHW Number of images, Channels, Height, Width
@ -55,7 +55,7 @@ Acronyms and Abbreviations
VGG Visual Geometry Group
VOC Visual Object Classes
WINAPI Windows Application Programming Interface
================== ==================================================
================== ===========================================================================
Terms
@ -71,11 +71,14 @@ Glossary of terms used in OpenVINO™
| A preferred hardware device to run inference (CPU, GPU, GNA, etc.).
| *Extensibility mechanism, Custom layers*
| The mechanism that provides you with capabilities to extend the OpenVINO™ Runtime and Model Optimizer so that they can work with models containing operations that are not yet supported.
| The mechanism that provides you with capabilities to extend the OpenVINO™ Runtime and model conversion API so that they can work with models containing operations that are not yet supported.
| *layer / operation*
| In OpenVINO, both terms are treated synonymously. To avoid confusion, "layer" is being pushed out and "operation" is the currently accepted term.
| *Model conversion API*
| A component of OpenVINO Development Tools. The API is used to import, convert, and optimize models trained in popular frameworks to a format usable by other OpenVINO components. In ``openvino.tools.mo`` namespace, model conversion API is represented by a Python ``mo.convert_model()`` method and ``mo`` command-line tool.
| *OpenVINO™ <code>Core</code>*
| OpenVINO™ Core is a software component that manages inference on certain Intel(R) hardware devices: CPU, GPU, GNA, etc.

10
docs/install_guides/installing-model-dev-tools.md
View File

@ -4,7 +4,7 @@
OpenVINO Development Tools is a set of utilities that make it easy to develop and optimize models and applications for OpenVINO. It provides the following tools:
* Model Optimizer
* Model conversion API
* Benchmark Tool
* Accuracy Checker and Annotation Converter
* Post-Training Optimization Tool
@ -31,7 +31,7 @@ For C++ Developers
If you are a C++ developer, you must first install OpenVINO Runtime separately to set up the C++ libraries, sample code, and dependencies for building applications with OpenVINO. These files are not included with the PyPI distribution. See the :doc:`Install OpenVINO Runtime <openvino_docs_install_guides_install_runtime>` page to install OpenVINO Runtime from an archive file for your operating system.
Once OpenVINO Runtime is installed, you may install OpenVINO Development Tools for access to tools like Model Optimizer, Model Downloader, Benchmark Tool, and other utilities that will help you optimize your model and develop your application. Follow the steps in the :ref:`Installing OpenVINO Development Tools <install_dev_tools>` section on this page to install it.
Once OpenVINO Runtime is installed, you may install OpenVINO Development Tools for access to tools like ``mo``, Model Downloader, Benchmark Tool, and other utilities that will help you optimize your model and develop your application. Follow the steps in the :ref:`Installing OpenVINO Development Tools <install_dev_tools>` section on this page to install it.
.. _install_dev_tools:
@ -134,7 +134,7 @@ For example, to install and configure dependencies required for working with Ten
.. note::
Model Optimizer support for TensorFlow 1.x environment has been deprecated. Use the ``tensorflow2`` parameter to install a TensorFlow 2.x environment that can convert both TensorFlow 1.x and 2.x models. If your model isn't compatible with the TensorFlow 2.x environment, use the `tensorflow` parameter to install the TensorFlow 1.x environment. The TF 1.x environment is provided only for legacy compatibility reasons.
Model conversion API support for TensorFlow 1.x environment has been deprecated. Use the ``tensorflow2`` parameter to install a TensorFlow 2.x environment that can convert both TensorFlow 1.x and 2.x models. If your model isn't compatible with the TensorFlow 2.x environment, use the `tensorflow` parameter to install the TensorFlow 1.x environment. The TF 1.x environment is provided only for legacy compatibility reasons.
For more details on the openvino-dev PyPI package, see `pypi.org <https://pypi.org/project/openvino-dev/>`__ .
@ -147,7 +147,7 @@ To verify the package is properly installed, run the command below (this may tak
mo -h
You will see the help message for Model Optimizer if installation finished successfully. If you get an error, refer to the :doc:`Troubleshooting Guide <openvino_docs_get_started_guide_troubleshooting>` for possible solutions.
You will see the help message for ``mo`` if installation finished successfully. If you get an error, refer to the :doc:`Troubleshooting Guide <openvino_docs_get_started_guide_troubleshooting>` for possible solutions.
Congratulations! You finished installing OpenVINO Development Tools with C++ capability. Now you can start exploring OpenVINO's functionality through example C++ applications. See the "What's Next?" section to learn more!
@ -188,7 +188,7 @@ Learn OpenVINO Development Tools
++++++++++++++++++++++++++++++++
* Explore a variety of pre-trained deep learning models in the :doc:`Open Model Zoo <model_zoo>` and deploy them in demo applications to see how they work.
* Want to import a model from another framework and optimize its performance with OpenVINO? Visit the :doc:`Model Optimizer Developer Guide <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
* Want to import a model from another framework and optimize its performance with OpenVINO? Visit the :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` page.
* Accelerate your model's speed even further with quantization and other compression techniques using :doc:`Post-Training Optimization Tool <pot_introduction>`.
* Benchmark your model's inference speed with one simple command using the :doc:`Benchmark Tool <openvino_inference_engine_tools_benchmark_tool_README>`.

6
docs/install_guides/installing-openvino-from-archive-linux.md
View File

@ -11,7 +11,7 @@ page for instructions on how to install OpenVINO Runtime for Python using PyPI.
.. note::
The following development tools can be installed via `pypi.org <https://pypi.org/project/openvino-dev/>`__ only:
Model Optimizer, Post-Training Optimization Tool, Model Downloader and other Open Model Zoo tools,
model conversion API, Post-Training Optimization Tool, Model Downloader and other Open Model Zoo tools,
Accuracy Checker, and Annotation Converter.
See the `Release Notes <https://software.intel.com/en-us/articles/OpenVINO-RelNotes>`__ for more information on updates in the latest release.
@ -183,7 +183,7 @@ Step 3 (Optional): Install Additional Components
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
OpenVINO Development Tools is a set of utilities for working with OpenVINO and OpenVINO models.
It provides tools like Model Optimizer, Benchmark Tool, Post-Training Optimization Tool, and Open Model Zoo Downloader.
It provides tools like model conversion API, Benchmark Tool, Post-Training Optimization Tool, and Open Model Zoo Downloader.
If you install OpenVINO Runtime using archive files, OpenVINO Development Tools must be installed separately.
See the :doc:`Install OpenVINO Development Tools <openvino_docs_install_guides_install_dev_tools>`
@ -257,7 +257,7 @@ Additional Resources
###########################################################
* :doc:`Troubleshooting Guide for OpenVINO Installation & Configuration <openvino_docs_get_started_guide_troubleshooting>`
* Converting models for use with OpenVINO™: :doc:`Model Optimizer User Guide <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* Converting models for use with OpenVINO™: :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* Writing your own OpenVINO™ applications: :doc:`OpenVINO™ Runtime User Guide <openvino_docs_OV_UG_OV_Runtime_User_Guide>`
* Sample applications: :doc:`OpenVINO™ Toolkit Samples Overview <openvino_docs_OV_UG_Samples_Overview>`
* Pre-trained deep learning models: :doc:`Overview of OpenVINO™ Toolkit Pre-Trained Models <model_zoo>`

4
docs/install_guides/installing-openvino-raspbian.md
View File

@ -134,9 +134,9 @@ If you want to use your model for inference, the model must be converted to the
* OpenCV is necessary to run demos from Open Model Zoo (OMZ). Some OpenVINO samples can also extend their capabilities when compiled with OpenCV as a dependency. To install OpenCV for OpenVINO, see the `instructions on Github <https://github.com/opencv/opencv/wiki/BuildOpenCV4OpenVINO)>`_.
* Convert the models using the Model Optimizer. Model Optimizer is provided with OpenVINO Development Tools.
* Convert the models using the model conversion API, which is included in OpenVINO Development Tools.
* OpenVINO Development Tools is a set of utilities for working with OpenVINO and OpenVINO models. It provides tools like Model Optimizer, Benchmark Tool, Post-Training Optimization Tool, and Open Model Zoo Downloader. See the :doc:`Install OpenVINO Development Tools <openvino_docs_install_guides_install_dev_tools>` page for step-by-step installation instructions.
* OpenVINO Development Tools is a set of utilities for working with OpenVINO and OpenVINO models. It provides tools like model conversion API, Benchmark Tool, Post-Training Optimization Tool, and Open Model Zoo Downloader. See the :doc:`Install OpenVINO Development Tools <openvino_docs_install_guides_install_dev_tools>` page for step-by-step installation instructions.
What's Next?
####################

2
docs/install_guides/installing-openvino-yocto.md
View File

@ -72,7 +72,7 @@ Step 1: Set Up Environment
# Include OpenVINO Python API package in the target image.
CORE_IMAGE_EXTRA_INSTALL:append = " openvino-inference-engine-python3"
# Include Model Optimizer in the target image.
# Include model conversion API in the target image.
CORE_IMAGE_EXTRA_INSTALL:append = " openvino-model-optimizer"

14
docs/install_guides/pypi-openvino-dev.md
View File

@ -91,7 +91,7 @@ For example, to install and configure the components for working with TensorFlow
```sh
pip install openvino-dev[tensorflow2,onnx]
```
> **NOTE**: Model Optimizer support for TensorFlow 1.x environment has been deprecated. Use TensorFlow 2.x environment to convert both TensorFlow 1.x and 2.x models.
> **NOTE**: Model conversion API support for TensorFlow 1.x environment has been deprecated. Use TensorFlow 2.x environment to convert both TensorFlow 1.x and 2.x models.
> **NOTE**: On macOS, you may need to enclose the package name in quotes: `pip install "openvino-dev[extras]"`.
@ -101,7 +101,7 @@ For example, to install and configure the components for working with TensorFlow
```sh
mo -h
```
You will see the help message for Model Optimizer if installation finished successfully.
You will see the help message for ``mo`` if installation finished successfully.
- To verify that OpenVINO Runtime from the **runtime package** is available, run the command below:
```sh
@ -119,11 +119,11 @@ For example, to install and configure the components for working with TensorFlow
| Component | Console Script | Description |
|------------------|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Model Optimizer](https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) | `mo` |**Model Optimizer** imports, converts, and optimizes models that were trained in popular frameworks to a format usable by OpenVINO components. <br>Supported frameworks include Caffe\*, TensorFlow\*, MXNet\*, PaddlePaddle\*, and ONNX\*. |
| [Benchmark Tool](https://docs.openvino.ai/2023.0/openvino_inference_engine_tools_benchmark_tool_README.html)| `benchmark_app` | **Benchmark Application** allows you to estimate deep learning inference performance on supported devices for synchronous and asynchronous modes. |
| [Accuracy Checker](https://docs.openvino.ai/2023.0/omz_tools_accuracy_checker.html) and <br> [Annotation Converter](https://docs.openvino.ai/2023.0/omz_tools_accuracy_checker_annotation_converters.html) | `accuracy_check` <br> `convert_annotation` |**Accuracy Checker** is a deep learning accuracy validation tool that allows you to collect accuracy metrics against popular datasets. The main advantages of the tool are the flexibility of configuration and a set of supported datasets, preprocessing, postprocessing, and metrics. <br> **Annotation Converter** is a utility that prepares datasets for evaluation with Accuracy Checker. |
| [Post-Training Optimization Tool](https://docs.openvino.ai/2023.0/pot_introduction.html)| `pot` |**Post-Training Optimization Tool** allows you to optimize trained models with advanced capabilities, such as quantization and low-precision optimizations, without the need to retrain or fine-tune models. |
| [Model Downloader and other Open Model Zoo tools](https://docs.openvino.ai/2023.0/omz_tools_downloader.html)| `omz_downloader` <br> `omz_converter` <br> `omz_quantizer` <br> `omz_info_dumper`| **Model Downloader** is a tool for getting access to the collection of high-quality and extremely fast pre-trained deep learning [public](@ref omz_models_group_public) and [Intel](@ref omz_models_group_intel)-trained models. These free pre-trained models can be used to speed up the development and production deployment process without training your own models. The tool downloads model files from online sources and, if necessary, patches them to make them more usable with Model Optimizer. A number of additional tools are also provided to automate the process of working with downloaded models:<br> **Model Converter** is a tool for converting Open Model Zoo models that are stored in an original deep learning framework format into the OpenVINO Intermediate Representation (IR) using Model Optimizer. <br> **Model Quantizer** is a tool for automatic quantization of full-precision models in the IR format into low-precision versions using the Post-Training Optimization Tool. <br> **Model Information Dumper** is a helper utility for dumping information about the models to a stable, machine-readable format. |
| [Model conversion API](https://docs.openvino.ai/nightly/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) | `mo` |**Model conversion API** imports, converts, and optimizes models that were trained in popular frameworks to a format usable by OpenVINO components. <br>Supported frameworks include Caffe\*, TensorFlow\*, MXNet\*, PaddlePaddle\*, and ONNX\*. |
| [Benchmark Tool](https://docs.openvino.ai/nightly/openvino_inference_engine_tools_benchmark_tool_README.html)| `benchmark_app` | **Benchmark Application** allows you to estimate deep learning inference performance on supported devices for synchronous and asynchronous modes. |
| [Accuracy Checker](https://docs.openvino.ai/nightly/omz_tools_accuracy_checker.html) and <br> [Annotation Converter](https://docs.openvino.ai/nightly/omz_tools_accuracy_checker_annotation_converters.html) | `accuracy_check` <br> `convert_annotation` |**Accuracy Checker** is a deep learning accuracy validation tool that allows you to collect accuracy metrics against popular datasets. The main advantages of the tool are the flexibility of configuration and a set of supported datasets, preprocessing, postprocessing, and metrics. <br> **Annotation Converter** is a utility that prepares datasets for evaluation with Accuracy Checker. |
| [Post-Training Optimization Tool](https://docs.openvino.ai/nightly/pot_introduction.html)| `pot` |**Post-Training Optimization Tool** allows you to optimize trained models with advanced capabilities, such as quantization and low-precision optimizations, without the need to retrain or fine-tune models. |
| [Model Downloader and other Open Model Zoo tools](https://docs.openvino.ai/nightly/omz_tools_downloader.html)| `omz_downloader` <br> `omz_converter` <br> `omz_quantizer` <br> `omz_info_dumper`| **Model Downloader** is a tool for getting access to the collection of high-quality and extremely fast pre-trained deep learning [public](@ref omz_models_group_public) and [Intel](@ref omz_models_group_intel)-trained models. These free pre-trained models can be used to speed up the development and production deployment process without training your own models. The tool downloads model files from online sources and, if necessary, patches them to make them more usable with model conversion API. A number of additional tools are also provided to automate the process of working with downloaded models:<br> **Model Converter** is a tool for converting Open Model Zoo models that are stored in an original deep learning framework format into the OpenVINO Intermediate Representation (IR) using model conversion API. <br> **Model Quantizer** is a tool for automatic quantization of full-precision models in the IR format into low-precision versions using the Post-Training Optimization Tool. <br> **Model Information Dumper** is a helper utility for dumping information about the models to a stable, machine-readable format. |
## Troubleshooting

25
docs/install_guides/troubleshooting-steps.md
View File

@ -8,11 +8,26 @@ Check the versions of OpenVINO Runtime and Development Tools
#############################################################
* To check the version of OpenVINO Development Tools, use the following command:
.. code-block:: sh
mo --version
.. tab-set::
.. tab-item:: Python
:sync: mo-python-api
.. code-block:: python
from openvino.tools.mo import convert_model
ov_model = convert_model(version=True)
.. tab-item:: CLI
:sync: cli-tool
.. code-block:: sh
mo --version
* To check the version of OpenVINO Runtime, use the following code:
.. code-block:: sh

2
docs/optimization_guide/dldt_deployment_optimization_common.md
View File

@ -12,7 +12,7 @@ Inputs Pre-Processing with OpenVINO
In many cases, a network expects a pre-processed image. It is advised not to perform any unnecessary steps in the code:
* Model Optimizer can efficiently incorporate the mean and normalization (scale) values into a model (for example, to the weights of the first convolution). For more details, see the :doc:`relevant Model Optimizer command-line options <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>`.
* Model conversion API can efficiently incorporate the mean and normalization (scale) values into a model (for example, to the weights of the first convolution). For more details, see the :doc:`relevant model conversion API command-line parameters <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>`.
* Let OpenVINO accelerate other means of :doc:`Image Pre-processing and Conversion <openvino_docs_OV_UG_Preprocessing_Overview>`
* Data which is already in the "on-device" memory can be input directly by using the :doc:`remote tensors API of the GPU Plugin <openvino_docs_OV_UG_supported_plugins_GPU_RemoteTensor_API>`.

2
docs/optimization_guide/model_optimization_guide.md
View File

@ -12,6 +12,8 @@
Model optimization is an optional offline step of improving the final model performance and reducing the model size by applying special optimization methods, such as 8-bit quantization, pruning, etc. OpenVINO offers two optimization paths implemented in `Neural Network Compression Framework (NNCF) <https://github.com/openvinotoolkit/nncf>`__:
- :doc:`Model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` implements most of the optimization parameters to a model by default. Yet, you are free to configure mean/scale values, batch size, RGB vs BGR input channels, and other parameters to speed up preprocess of a model (:doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>`).
- :doc:`Post-training Quantization <ptq_introduction>` is designed to optimize the inference of deep learning models by applying the post-training 8-bit integer quantization that does not require model retraining or fine-tuning.
- :doc:`Training-time Optimization <tmo_introduction>`, a suite of advanced methods for training-time model optimization within the DL framework, such as PyTorch and TensorFlow 2.x. It supports methods like Quantization-aware Training, Structured and Unstructured Pruning, etc.

6
docs/optimization_guide/nncf/filter_pruning.md
View File

@ -232,10 +232,10 @@ For more details on saving/loading checkpoints in the NNCF, see the following
Deploying pruned model
######################
The pruned model requires an extra step that should be done to get a performance improvement. This step involves the removal of the
zero filters from the model. This is done at the model conversion step using :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` tool when the model is converted from the framework representation (ONNX, TensorFlow, etc.) to OpenVINO Intermediate Representation.
The pruned model requres an extra step that should be done to get performance improvement. This step involves removal of the
zero filters from the model. This is done at the model conversion step using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` tool when model is converted from the framework representation (ONNX, TensorFlow, etc.) to OpenVINO Intermediate Representation.
* To remove zero filters from the pruned model add the following parameter to the model conversion command: ``--transform=Pruning``
* To remove zero filters from the pruned model add the following parameter to the model conversion command: ``transform=Pruning``
After that, the model can be deployed with OpenVINO in the same way as the baseline model.
For more details about model deployment with OpenVINO, see the corresponding :doc:`documentation <openvino_docs_OV_UG_OV_Runtime_User_Guide>`.

2
docs/resources/openvino_ir.md
View File

@ -11,7 +11,7 @@
The models, built and trained using various frameworks, can be large and architecture-dependent. To successfully run inference from any device and maximize the benefits of OpenVINO tools, you can convert the model to the OpenVINO Intermediate Representation (IR) format.
OpenVINO IR is the proprietary model format of OpenVINO. It is produced after converting a model with the Model Optimizer tool. Model Optimizer translates the frequently used deep learning operations to their respective similar representation in OpenVINO and tunes them with the associated weights and biases from the trained model. The resulting IR contains two files:
OpenVINO IR is the proprietary model format of OpenVINO. It is produced after converting a model with model conversion API. Model conversion API translates the frequently used deep learning operations to their respective similar representation in OpenVINO and tunes them with the associated weights and biases from the trained model. The resulting IR contains two files:
* ``.xml`` - Describes the model topology.
* ``.bin`` - Contains the weights and binary data.

6
docs/resources/prerelease_information.md
View File

@ -103,10 +103,10 @@ Please file a github Issue on these with the label “pre-release” so we can g
OpenVINO™ repository tag: `2023.0.0.dev20230217 <https://github.com/openvinotoolkit/openvino/releases/tag/2023.0.0.dev20230217>`__
* Enabled PaddlePaddle Framework 2.4
* Preview of TensorFlow Lite Front End – Load models directly via “read_model” into OpenVINO Runtime and export OpenVINO IR format using Model Optimizer or “convert_model”
* Preview of TensorFlow Lite Frontend – Load models directly via “read_model” into OpenVINO Runtime and export OpenVINO IR format using model conversion API or “convert_model”
* PyTorch Frontend is available as an experimental feature which will allow you to convert PyTorch models, using convert_model Python API directly from your code without the need to export to the ONNX format. Model coverage is continuously increasing. Feel free to start using the option and give us feedback.
* Model Optimizer now uses the TensorFlow Frontend as the default path for conversion to IR. Known limitations compared to the legacy approach are: TF1 Loop, Complex types, models requiring config files and old python extensions. The solution detects unsupported functionalities and provides fallback. To force using the legacy frontend ``--use_legacy_fronted`` can be specified.
* Model Optimizer now supports out-of-the-box conversion of TF2 Object Detection models. At this point, same performance experience is guaranteed only on CPU devices. Feel free to start enjoying TF2 Object Detection models without config files!
* Model conversion API now uses the TensorFlow Frontend as the default path for conversion to IR. Known limitations compared to the legacy approach are: TF1 Loop, Complex types, models requiring config files and old python extensions. The solution detects unsupported functionalities and provides fallback. To force using the legacy frontend ``use_legacy_fronted`` can be specified.
* Model conversion API now supports out-of-the-box conversion of TF2 Object Detection models. At this point, same performance experience is guaranteed only on CPU devices. Feel free to start enjoying TF2 Object Detection models without config files!
* Introduced new option ov::auto::enable_startup_fallback / ENABLE_STARTUP_FALLBACK to control whether to use CPU to accelerate first inference latency for accelerator HW devices like GPU.
* New FrontEndManager register_front_end(name, lib_path) interface added, to remove “OV_FRONTEND_PATH” env var (a way to load non-default frontends).

2
docs/resources/telemetry_information.md
View File

@ -63,7 +63,7 @@ Telemetry Data Collection Details
.. tab:: Tools Collecting Data
* Model Optimizer
* Model conversion API
* Model Downloader
* Accuracy Checker
* Post-Training Optimization Toolkit

6
samples/c/hello_classification/README.md
View File

@ -75,8 +75,8 @@ To run the sample, you need specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using ``mo`` with `reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (\*.onnx) that do not require preprocessing.
Example
@ -132,7 +132,7 @@ See Also
- :doc:`Integrate OpenVINO™ into Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`C API Reference <pot_compression_api_README>`
@endsphinxdirective

8
samples/c/hello_nv12_input_classification/README.md
View File

@ -65,8 +65,8 @@ The sample accepts an uncompressed image in the NV12 color format. To run the sa
.. note::
- Because the sample reads raw image files, you should provide a correct image size along with the image path. The sample expects the logical size of the image, not the buffer size. For example, for 640x480 BGR/RGB image the corresponding NV12 logical image size is also 640x480, whereas the buffer size is 640x720.
- By default, this sample expects that network input has BGR channels order. If you trained your model to work with RGB order, you need to reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, this sample expects that network input has BGR channels order. If you trained your model to work with RGB order, you need to reconvert your model using ``mo`` with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
@ -122,8 +122,8 @@ See Also
- :doc:`Integrate the OpenVINO™ into Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`C API Reference <api/api_reference>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- `C API Reference <https://docs.openvino.ai/latest/api/api_reference.html>`__
@endsphinxdirective

4
samples/cpp/benchmark/sync_benchmark/README.md
View File

@ -67,7 +67,7 @@ To run the sample, you need to specify a model:
.. note::
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
@ -127,6 +127,6 @@ See Also
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

4
samples/cpp/benchmark/throughput_benchmark/README.md
View File

@ -70,7 +70,7 @@ To run the sample, you need to specify a model:
.. note::
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
@ -130,6 +130,6 @@ See Also
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

4
samples/cpp/benchmark_app/README.md
View File

@ -125,7 +125,7 @@ Advanced Usage
.. note::
By default, OpenVINO samples, tools and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channel order in the sample or demo application or reconvert your model using the Model Optimizer tool with --reverse_input_channels argument specified. For more information about the argument, refer to When to Reverse Input Channels section of Converting a Model to Intermediate Representation (IR).
By default, OpenVINO samples, tools and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channel order in the sample or demo application or reconvert your model using ``mo`` with ``reverse_input_channels`` argument specified. For more information about the argument, refer to When to Reverse Input Channels section of Converting a Model to Intermediate Representation (IR).
Per-layer performance and logging
+++++++++++++++++++++++++++++++++
@ -405,7 +405,7 @@ See Also
####################
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Model Downloader <omz_tools_downloader>`
@endsphinxdirective

6
samples/cpp/classification_sample_async/README.md
View File

@ -88,9 +88,9 @@ To run the sample, you need to specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using ``mo`` with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
@ -196,7 +196,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/cpp/hello_classification/README.md
View File

@ -74,8 +74,8 @@ To run the sample, you need to specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using ``mo`` with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
@ -149,7 +149,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/cpp/hello_nv12_input_classification/README.md
View File

@ -70,8 +70,8 @@ The sample accepts an uncompressed image in the NV12 color format. To run the sa
.. note::
- Because the sample reads raw image files, you should provide a correct image size along with the image path. The sample expects the logical size of the image, not the buffer size. For example, for 640x480 BGR/RGB image the corresponding NV12 logical image size is also 640x480, whereas the buffer size is 640x720.
- By default, this sample expects that model input has BGR channels order. If you trained your model to work with RGB order, you need to reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, this sample expects that model input has BGR channels order. If you trained your model to work with RGB order, you need to reconvert your model using ``mo`` with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
@ -147,7 +147,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/cpp/hello_reshape_ssd/README.md
View File

@ -65,8 +65,8 @@ To run the sample, you need to specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, OpenVINO™ Toolkit Samples and Demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using ``mo`` with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (\*.onnx) that do not require preprocessing.
Example
@ -137,7 +137,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

4
samples/cpp/model_creation_sample/README.md
View File

@ -83,7 +83,7 @@ Running
.. note::
- you can use LeNet model weights in the sample folder: ``lenet.bin`` with FP32 weights file
- The ``lenet.bin`` with FP32 weights file was generated by the :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` tool from the public LeNet model with the ``--input_shape [64,1,28,28]`` parameter specified.
- The ``lenet.bin`` with FP32 weights file was generated by :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` from the public LeNet model with the ``input_shape [64,1,28,28]`` parameter specified.
The original model is available in the `Caffe* repository <https://github.com/BVLC/caffe/tree/master/examples/mnist>`__ on GitHub\*.
@ -214,7 +214,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/cpp/speech_sample/README.md
View File

@ -154,7 +154,7 @@ Usage message:
Model Preparation
+++++++++++++++++
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
You can use the following model conversion command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
.. code-block:: sh
@ -181,7 +181,7 @@ Here, the floating point Kaldi-generated reference neural network scores (``dev9
.. note::
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample supports input and output in numpy file format (.npz)
@ -275,7 +275,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

2
samples/python/benchmark/bert_benchmark/README.md
View File

@ -64,6 +64,6 @@ See Also
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

4
samples/python/benchmark/sync_benchmark/README.md
View File

@ -63,7 +63,7 @@ To run the sample, you need to specify a model:
.. note::
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
@ -124,6 +124,6 @@ See Also
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

4
samples/python/benchmark/throughput_benchmark/README.md
View File

@ -68,7 +68,7 @@ To run the sample, you need to specify a model:
.. note::
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
@ -129,6 +129,6 @@ See Also
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/python/classification_sample_async/README.md
View File

@ -76,9 +76,9 @@ To run the sample, you need specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using model conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
@ -162,7 +162,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/python/hello_classification/README.md
View File

@ -63,8 +63,8 @@ To run the sample, you need to specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using model conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
@ -128,7 +128,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/python/hello_reshape_ssd/README.md
View File

@ -56,8 +56,8 @@ To run the sample, you need to specify a model and image:
.. note::
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using model conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
@ -109,7 +109,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

4
samples/python/model_creation_sample/README.md
View File

@ -64,7 +64,7 @@ To run the sample, you need to specify model weights and device.
- This sample supports models with FP32 weights only.
- The ``lenet.bin`` weights file was generated by the :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` tool from the public LeNet model with the ``--input_shape [64,1,28,28]`` parameter specified.
- The ``lenet.bin`` weights file was generated by :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` from the public LeNet model with the ``input_shape [64,1,28,28]`` parameter specified.
- The original model is available in the `Caffe* repository <https://github.com/BVLC/caffe/tree/master/examples/mnist>`__ on GitHub\*.
@ -154,7 +154,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
samples/python/speech_sample/README.md
View File

@ -180,7 +180,7 @@ Usage message:
Model Preparation
#################
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
You can use the following model conversion command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
.. code-block:: sh
@ -206,7 +206,7 @@ You can do inference on Intel® Processors with the GNA co-processor (or emulati
.. note::
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`Model Optimizer tool <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample supports input and output in numpy file format (.npz)
- Stating flags that take only single option like `-m` multiple times, for example `python classification_sample_async.py -m model.xml -m model2.xml`, results in only the last value being used.
@ -366,7 +366,7 @@ See Also
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective

6
src/frontends/tensorflow/README.md
View File

@ -2,7 +2,7 @@
The TensorFlow Frontend (TF FE) is a C++ based OpenVINO Frontend component that is responsible for reading and converting a TensorFlow model to an `ov::Model` object
that further can be serialized into the Intermediate Representation (IR) format.
This is an internal API for OpenVINO that is used to implement user-facing API such as MO tool, MO Python API, and OpenVINO Runtime `read_model` function
This is an internal API for OpenVINO that is used to implement user-facing API such as MO tool, Model Conversion API, and OpenVINO Runtime `read_model` function
for reading TensorFlow models of the original format in run-time. Also, OpenVINO Model Server uses the frontend for serving models.
Regular users should not use the frontend directly.
@ -30,8 +30,8 @@ flowchart BT
click ovms "https://github.com/openvinotoolkit/model_server"
```
The MO tool and MO Python API now use the TensorFlow Frontend as the default path for conversion to IR.
Known limitations of TF FE are described [here](https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_TensorFlow_Frontend.html).
The MO tool and model conversion API now use the TensorFlow Frontend as the default path for conversion to IR.
Known limitations of TF FE are described [here](https://docs.openvino.ai/nightly/openvino_docs_MO_DG_TensorFlow_Frontend.html).
## Key contacts

4
tools/benchmark_tool/README.md
View File

@ -122,7 +122,7 @@ Advanced Usage
.. note::
By default, OpenVINO samples, tools and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channel order in the sample or demo application or reconvert your model using the Model Optimizer tool with ``--reverse_input_channels`` argument specified. For more information about the argument, refer to When to Reverse Input Channels section of Converting a Model to Intermediate Representation (IR).
By default, OpenVINO samples, tools and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channel order in the sample or demo application or reconvert your model using Model Conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to When to Reverse Input Channels section of Converting a Model to Intermediate Representation (IR).
Per-layer performance and logging
@ -488,7 +488,7 @@ See Also
####################
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
* :doc:`Model Downloader <omz_tools_downloader>`
@endsphinxdirective

6
tools/mo/README.md
View File

@ -7,7 +7,7 @@ virtualenv -p /usr/bin/python3.7 .env3
source .env3/bin/activate
```
2. Install openvino-dev package, it contains Model Optimizer:
2. Install openvino-dev package, it contains model conversion API:
```
pip install openvino-dev
```
@ -35,7 +35,7 @@ By default, if no frameworks are specified, dependencies to support ONNX\* and T
* [Converting Model](../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md)
## Setup development environment
If you want to contribute to Model Optimizer you will need to deploy developer environment.
If you want to contribute to model conversion API you will need to deploy developer environment.
You can do that by following the steps below:
1. Create virtual environment and activate it, e.g.:
@ -59,7 +59,7 @@ or run `setup.py develop`, result will be the same:
python setup.py develop
```
This will download all requirements and deploy Model Optimizer for development in your virtual environment:
This will download all requirements and deploy model conversion API for development in your virtual environment:
specifically will create *.egg-link into the current directory in your site-packages.
As previously noted, you can also manually specify to support only selected frameworks :
```

8
tools/pot/README_dev.md
View File

@ -30,10 +30,10 @@ Post-Training Optimization Tool includes standalone command-line tool and Python
git submodule init
git submodule update
```
3) Setup Model Optimizer.
You can setup Model Optimizer that needs for POT purposed with the two ways:
1. Install Model Optimizer with pip using "python setup.py install" at the mo folder (`<openvino_path>/tools/mo/setup.py`)
2. Setup Model Optimizer for Python using PYTHONPATH environment variable. Add the following `<openvino_path>/tools/mo` into PYTHONPATH.
3) Setup model conversion API.
You can setup model conversion API that needs for POT purposed with the two ways:
1. Install model conversion API with pip using "python setup.py install" at the mo folder (`<openvino_path>/tools/mo/setup.py`)
2. Setup model conversion API for Python using PYTHONPATH environment variable. Add the following `<openvino_path>/tools/mo` into PYTHONPATH.
4) Install requirements for accuracy checker:
- From POT root: `cd ./thirdparty/open_model_zoo/tools/accuracy_checker`
- Call setup script: `python3 setup.py install`

2
tools/pot/docs/E2eExample.md
View File

@ -43,7 +43,7 @@ Model Preparation
After that, the full-precision model in the IR format is located in ``<EXAMPLE_DIR>/public/mobilenet-v2-pytorch/FP32/``.
For more information about the Model Optimizer, refer to its :doc:`documentation <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
For more information about Model Conversion API, refer to its :doc:`documentation <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
Performance Benchmarking of Full-Precision Models
#################################################

4
tools/pot/docs/FrequentlyAskedQuestions.md
View File

@ -40,8 +40,8 @@ If your dataset is not annotated, you can use :doc:`Default Quantization <pot_de
Can a model in any framework be quantized by the POT?
+++++++++++++++++++++++++++++++++++++++++++++++++++++
The POT accepts models in the OpenVINO&trade; Intermediate Representation (IR) format only. For that, you need to convert your model to the IR format using
:doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The POT accepts models in the OpenVINO&trade; Intermediate Representation (IR) format only. For that you need to convert your model to the IR format using
:doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
.. _noac-pot-faq:

4
tools/pot/docs/LowPrecisionOptimizationGuide.md
View File

@ -40,7 +40,7 @@ The diagram below shows the optimization flow for the new model with OpenVINO an
![](images/low_precision_flow.png)
- **Step 0: Model enabling**. In this step we should ensure that the model trained on the target dataset can be successfully inferred with [OpenVINO™ Runtime](@ref openvino_docs_OV_UG_OV_Runtime_User_Guide) in floating-point precision.
This process involves use of [Model Optimizer](@ref openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide) tool to convert the model from the source framework
This process involves use of [model conversion API](@ref openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide) tool to convert the model from the source framework
to the OpenVINO Intermediate Representation (IR) and run it on CPU with Inference Engine.
> **NOTE**: This step presumes that the model has the same accuracy as in the original training framework and enabled in the [Accuracy Checker](@ref omz_tools_accuracy_checker) tool or through the custom validation sample.
- **Step 1: Post-training quantization**. As the first step for optimization, we suggest using INT8 quantization from POT where in most cases it is possible to get an accurate quantized model. At this step you do not need model re-training. The only thing required is a representative dataset which is usually several hundreds of images and it is used to collect statistics during the quantization process.
@ -48,4 +48,4 @@ Post-training quantization is also really fast and usually takes several minutes
For more information on best practices of post-training optimization please refer to the [Post-training Optimization Best practices](BestPractices.md).
- **Step2: Quantization-Aware Training**: If the accuracy of the quantized model does not satisfy accuracy criteria, there is step two which implies QAT using [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf) for [PyTorch*](https://pytorch.org/) and [TensorFlow*](https://www.tensorflow.org/) models.
At this step, we assume the user has an original training pipeline of the model written on TensorFlow or PyTorch and NNCF is integrated into it.
After this step, you can get an accurate optimized model that can be converted to OpenVINO Intermediate Representation (IR) using Model Optimizer component and inferred with OpenVINO Inference Engine.
After this step, you can get an accurate optimized model that can be converted to OpenVINO Intermediate Representation (IR) using model conversion API and inferred with OpenVINO Inference Engine.

4
tools/pot/openvino/tools/pot/api/samples/speech/README.md
View File

@ -16,11 +16,11 @@ For generating data from original formats to .ark, please, follow the `Kaldi dat
How to Run the Example
######################
1. Launch :doc:`Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` with the necessary options (for details follow the :doc:`instructions for Kaldi <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi>` to generate Intermediate Representation (IR) files for the model:
1. Launch :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` with the necessary options (for details follow the :doc:`instructions for Kaldi <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi>` to generate Intermediate Representation (IR) files for the model:
.. code-block:: sh
mo --input_model <PATH_TO_KALDI_MODEL> [MODEL_OPTIMIZER_OPTIONS]
mo --input_model <PATH_TO_KALDI_MODEL> [MODEL_CONVERSION_API_PARAMETERS]
2. Launch the example script: