[DOCS] Tab reorder for 23.0 (#18389)
* tabs-1 * Update configure_devices.md * tab-2 * tab-order * Update installing-openvino-from-archive-linux.md * Update installing-openvino-from-archive-linux.md * win-linux-fix * Update GPU_Extensibility.md
This commit is contained in:
Maciej Smyk
GitHub
parent
68b8748c9f
commit
247eb8a9b9
|
|
@ -18,18 +18,20 @@ There are two options for using the custom operation configuration file:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.py
|
||||
:language: python
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
|
||||
All OpenVINO samples, except the trivial ``hello_classification``, and most Open Model Zoo demos
|
||||
feature a dedicated command-line option ``-c`` to load custom kernels. For example, to load custom operations for the classification sample, run the command below:
|
||||
|
|
|
|||
|
|
@ -95,6 +95,13 @@ Extensions can be loaded from a code with the ``:ref:`ov::Core::add_extension <
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: [add_extension]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -102,18 +109,18 @@ Extensions can be loaded from a code with the ``:ref:`ov::Core::add_extension <
|
|||
:language: cpp
|
||||
:fragment: [add_extension]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: [add_extension]
|
||||
|
||||
|
||||
The ``Identity`` is a custom operation class defined in :doc:`Custom Operation Guide <openvino_docs_Extensibility_UG_add_openvino_ops>`. This is sufficient to enable reading OpenVINO IR which uses the ``Identity`` extension operation emitted by Model Optimizer. In order to load original model directly to the runtime, add a mapping extension:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: [add_frontend_extension]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -121,13 +128,6 @@ The ``Identity`` is a custom operation class defined in :doc:`Custom Operation G
|
|||
:language: cpp
|
||||
:fragment: [add_frontend_extension]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: [add_frontend_extension]
|
||||
|
||||
When Python API is used, there is no way to implement a custom OpenVINO operation. Even if custom OpenVINO operation is implemented in C++ and loaded into the runtime by a shared library, there is still no way to add a frontend mapping extension that refers to this custom operation. In this case, use C++ shared library approach to implement both operations semantics and framework mapping.
|
||||
|
||||
Python can still be used to map and decompose operations when only operations from the standard OpenVINO operation set are used.
|
||||
|
|
@ -177,13 +177,6 @@ This CMake script finds OpenVINO, using the ``find_package`` CMake command.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [add_extension_lib]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -191,6 +184,13 @@ This CMake script finds OpenVINO, using the ``find_package`` CMake command.
|
|||
:language: python
|
||||
:fragment: [add_extension_lib]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [add_extension_lib]
|
||||
|
||||
|
||||
See Also
|
||||
########
|
||||
|
|
|
|||
|
|
@ -94,13 +94,6 @@ In this case, you can directly say that 'MyRelu' -> ``Relu`` mapping should be u
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_MyRelu]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -108,6 +101,13 @@ In this case, you can directly say that 'MyRelu' -> ``Relu`` mapping should be u
|
|||
:language: python
|
||||
:fragment: [py_frontend_extension_MyRelu]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_MyRelu]
|
||||
|
||||
|
||||
In the resulting converted OpenVINO model, “MyRelu” operation will be replaced by the standard operation
|
||||
``Relu`` from the latest available OpenVINO operation set. Notice that when standard operation is used,
|
||||
|
|
@ -335,13 +335,6 @@ from ONNX according to the formula: ``ThresholdedRelu(x, alpha) -> Multiply(x, C
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_ThresholdedReLU_header]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -349,14 +342,14 @@ from ONNX according to the formula: ``ThresholdedRelu(x, alpha) -> Multiply(x, C
|
|||
:language: python
|
||||
:fragment: [py_frontend_extension_ThresholdedReLU_header]
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_ThresholdedReLU]
|
||||
:fragment: [frontend_extension_ThresholdedReLU_header]
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -365,6 +358,13 @@ from ONNX according to the formula: ``ThresholdedRelu(x, alpha) -> Multiply(x, C
|
|||
:language: python
|
||||
:fragment: [py_frontend_extension_ThresholdedReLU]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_ThresholdedReLU]
|
||||
|
||||
|
||||
The next example shows how to use ``ConversionExtension`` to convert PyTorch
|
||||
`aten::hardtanh <https://pytorch.org/docs/stable/generated/torch.nn.functional.hardtanh.html>`_
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ To convert a model to OpenVINO model format (``ov.Model``), you can use the foll
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -34,7 +34,7 @@ To convert a model to OpenVINO model format (``ov.Model``), you can use the foll
|
|||
ov_model = convert_model(INPUT_MODEL)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -61,7 +61,7 @@ To get the full list of conversion parameters, run the following command:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -69,7 +69,7 @@ To get the full list of conversion parameters, run the following command:
|
|||
ov_model = convert_model(help=True)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -86,7 +86,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -94,7 +94,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
ov_model = convert_model("MobileNet.pb")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -106,7 +106,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -114,7 +114,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
ov_model = convert_model("BERT", input_shape=[[2,30],[2,30],[2,30]])
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -128,7 +128,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -136,7 +136,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
ov_model = convert_model("ocr.onnx", output="probabilities")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -154,7 +154,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -162,7 +162,7 @@ Below is a list of separate examples for different frameworks and model conversi
|
|||
ov_model = convert_model("unet.pdmodel", mean_values=[123,117,104], scale=255)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ For example, the following command specifies the ``NHWC`` layout for a Tensorflo
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -42,7 +42,7 @@ For example, the following command specifies the ``NHWC`` layout for a Tensorflo
|
|||
ov_model = convert_model("tf_nasnet_large.onnx", layout="nhwc")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -59,7 +59,7 @@ having two dimensions: batch and size of the image expressed as the ``N?`` layou
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -67,7 +67,7 @@ having two dimensions: batch and size of the image expressed as the ``N?`` layou
|
|||
ov_model = convert_model("yolov3-tiny.onnx", layout={"input_1": "nchw", "image_shape": "n?"})
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -87,7 +87,7 @@ the following commands to provide data in the ``NCHW`` layout:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -97,7 +97,7 @@ the following commands to provide data in the ``NCHW`` layout:
|
|||
ov_model = convert_model("tf_nasnet_large.onnx", layout="nhwc->nchw")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -115,7 +115,7 @@ mentioned earlier, use the following commands:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -125,7 +125,7 @@ mentioned earlier, use the following commands:
|
|||
ov_model = convert_model("yolov3-tiny.onnx", layout={"input_1": "nchw->nhwc", "image_shape": "n?"}
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -168,7 +168,7 @@ model and applies mean-scale normalization to the input data:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -176,7 +176,7 @@ model and applies mean-scale normalization to the input data:
|
|||
ov_model = convert_model("unet.pdmodel", mean_values=[123,117,104], scale=255)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -211,7 +211,7 @@ model and embeds the ``reverse_input_channel`` preprocessing block into OpenVINO
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -219,7 +219,7 @@ model and embeds the ``reverse_input_channel`` preprocessing block into OpenVINO
|
|||
ov_model = convert_model("alexnet.pb", reverse_input_channels=True)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ To compress the model, use the ``compress_to_fp16=True`` option:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -20,7 +20,7 @@ To compress the model, use the ``compress_to_fp16=True`` option:
|
|||
ov_model = convert_model(INPUT_MODEL, compress_to_fp16=False)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ and specify the input shape of ``[2,300,300,3]``:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -39,7 +39,7 @@ and specify the input shape of ``[2,300,300,3]``:
|
|||
ov_model = convert_model("MobileNet.pb", input_shape=[2,300,300,3])
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -54,7 +54,7 @@ and specify shapes ``[3,150,200,1]`` and ``[3]`` for them:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ and specify shapes ``[3,150,200,1]`` and ``[3]`` for them:
|
|||
ov_model = convert_model("ocr.onnx", input=["data","seq_len"], input_shape=[[3,150,200,1],[3]])
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -74,7 +74,7 @@ Alternatively, specify input shapes, using the ``input`` parameter as follows:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -82,7 +82,7 @@ Alternatively, specify input shapes, using the ``input`` parameter as follows:
|
|||
ov_model = convert_model("ocr.onnx", input=[("data",[3,150,200,1]),("seq_len",[3])])
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -97,7 +97,7 @@ For example, launch model conversion for the ONNX OCR model and specify dynamic
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -105,7 +105,7 @@ For example, launch model conversion for the ONNX OCR model and specify dynamic
|
|||
ov_model = convert_model("ocr.onnx", input=["data","seq_len"], input_shape=[[-1,150,200,1],[-1]]
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -119,7 +119,7 @@ For example, launch model conversion for the ONNX OCR model and specify a bounda
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -128,7 +128,7 @@ For example, launch model conversion for the ONNX OCR model and specify a bounda
|
|||
ov_model = convert_model("ocr.onnx", input=["data","seq_len"], input_shape=[[Dimension(1,3),150,200,1],[Dimension(1,3)]]
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ Convert this model to ``ov.Model``:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -61,7 +61,7 @@ Convert this model to ``ov.Model``:
|
|||
ov_model = convert_model("inception_v1.pb", batch=1)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -115,7 +115,7 @@ Due to automatic identification of inputs and outputs, providing the ``input`` a
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -125,7 +125,7 @@ Due to automatic identification of inputs and outputs, providing the ``input`` a
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, input="input", output="InceptionV1/Logits/Predictions/Reshape_1")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -154,7 +154,7 @@ If you want to cut your model at the end, you have the following options:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -162,7 +162,7 @@ If you want to cut your model at the end, you have the following options:
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -217,7 +217,7 @@ If you want to cut your model at the end, you have the following options:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -225,7 +225,7 @@ If you want to cut your model at the end, you have the following options:
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu:0")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -280,7 +280,7 @@ If you want to cut your model at the end, you have the following options:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -288,7 +288,7 @@ If you want to cut your model at the end, you have the following options:
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, output="0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -337,7 +337,7 @@ If you want to go further and cut the beginning of the model, leaving only the `
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -345,7 +345,7 @@ If you want to go further and cut the beginning of the model, leaving only the `
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu", input="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -388,7 +388,7 @@ If you want to go further and cut the beginning of the model, leaving only the `
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -396,7 +396,7 @@ If you want to go further and cut the beginning of the model, leaving only the `
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, input="0:InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu", output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -439,7 +439,7 @@ If you want to go further and cut the beginning of the model, leaving only the `
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -447,7 +447,7 @@ If you want to go further and cut the beginning of the model, leaving only the `
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, input="InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1:0", output="InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -494,7 +494,7 @@ Following this behavior, ``convert_model()`` creates an ``Input`` layer for port
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -502,7 +502,7 @@ Following this behavior, ``convert_model()`` creates an ``Input`` layer for port
|
|||
ov_model = convert_model("inception_v1.pb", batch=1, input="InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution")
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -516,7 +516,7 @@ Different behavior occurs when ``input_shape`` is also used as an attempt to ove
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -524,7 +524,7 @@ Different behavior occurs when ``input_shape`` is also used as an attempt to ove
|
|||
ov_model = convert_model("inception_v1.pb", input="InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution", input_shape=[1,224,224,3])
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -546,7 +546,7 @@ The correct command line is:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -554,7 +554,7 @@ The correct command line is:
|
|||
ov_model = convert_model("inception_v1.pb", input="0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution", input_shape=[1,224,224,3])
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -27,6 +27,13 @@ There are six levels of logs, which can be called explicitly or set via the ``OP
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto.py
|
||||
:language: python
|
||||
:fragment: [part6]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -34,15 +41,8 @@ There are six levels of logs, which can be called explicitly or set via the ``OP
|
|||
:language: cpp
|
||||
:fragment: [part6]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto.py
|
||||
:language: python
|
||||
:fragment: [part6]
|
||||
|
||||
.. tab-item:: OS environment variable
|
||||
:sync: os_env_variable
|
||||
:sync: os-environment-variable
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -43,13 +43,6 @@ To enable model caching, the application must specify a folder to store the cach
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -57,6 +50,13 @@ To enable model caching, the application must specify a folder to store the cach
|
|||
:language: py
|
||||
:fragment: [ov:caching:part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part0]
|
||||
|
||||
|
||||
With this code, if the device specified by ``device_name`` supports import/export model capability,
|
||||
a cached blob is automatically created inside the ``/path/to/cache/dir`` folder.
|
||||
|
|
@ -77,13 +77,6 @@ For these cases, there is a more convenient API to compile the model in a single
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part1]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -91,18 +84,18 @@ For these cases, there is a more convenient API to compile the model in a single
|
|||
:language: py
|
||||
:fragment: [ov:caching:part1]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part1]
|
||||
|
||||
|
||||
With model caching enabled, the total load time is even shorter, if ``read_model`` is optimized as well.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part2]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -110,6 +103,13 @@ With model caching enabled, the total load time is even shorter, if ``read_model
|
|||
:language: py
|
||||
:fragment: [ov:caching:part2]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part2]
|
||||
|
||||
|
||||
.. image:: _static/images/caching_times.svg
|
||||
|
||||
|
|
@ -121,13 +121,6 @@ To check in advance if a particular device supports model caching, your applicat
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part3]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -135,6 +128,13 @@ To check in advance if a particular device supports model caching, your applicat
|
|||
:language: py
|
||||
:fragment: [ov:caching:part3]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_caching.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:caching:part3]
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
|
|||
|
|
@ -34,13 +34,6 @@ to set a new batch size with the ``reshape`` method:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: picture_snippet
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -48,6 +41,13 @@ to set a new batch size with the ``reshape`` method:
|
|||
:language: Python
|
||||
:fragment: picture_snippet
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: picture_snippet
|
||||
|
||||
The diagram below presents the results of using the method, where the size of
|
||||
model input is changed with an image input:
|
||||
|
||||
|
|
@ -63,13 +63,6 @@ When using the ``reshape`` method, you may take one of the approaches:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: spatial_reshape
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -77,6 +70,13 @@ When using the ``reshape`` method, you may take one of the approaches:
|
|||
:language: python
|
||||
:fragment: simple_spatials_change
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: spatial_reshape
|
||||
|
||||
|
||||
To do the opposite - to resize input image to match the input shapes of the model,
|
||||
use the :doc:`pre-processing API <openvino_docs_OV_UG_Preprocessing_Overview>`.
|
||||
|
|
@ -90,15 +90,6 @@ When using the ``reshape`` method, you may take one of the approaches:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
``map<ov::Output<ov::Node>, ov::PartialShape`` specifies input by passing actual input port:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: [obj_to_shape]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -109,18 +100,18 @@ When using the ``reshape`` method, you may take one of the approaches:
|
|||
:language: python
|
||||
:fragment: [obj_to_shape]
|
||||
|
||||
.. tab-item:: Index
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
``map<size_t, ov::PartialShape>`` specifies input by its index:
|
||||
``map<ov::Output<ov::Node>, ov::PartialShape`` specifies input by passing actual input port:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: [idx_to_shape]
|
||||
:fragment: [obj_to_shape]
|
||||
|
||||
.. tab-item:: Index
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -132,18 +123,18 @@ When using the ``reshape`` method, you may take one of the approaches:
|
|||
:language: python
|
||||
:fragment: [idx_to_shape]
|
||||
|
||||
.. tab-item:: Tensor Name
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
``map<string, ov::PartialShape>`` specifies input by its name:
|
||||
``map<size_t, ov::PartialShape>`` specifies input by its index:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: [name_to_shape]
|
||||
:fragment: [idx_to_shape]
|
||||
|
||||
.. tab-item:: Tensor Name
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -155,6 +146,15 @@ When using the ``reshape`` method, you may take one of the approaches:
|
|||
:language: python
|
||||
:fragment: [name_to_shape]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
``map<string, ov::PartialShape>`` specifies input by its name:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: [name_to_shape]
|
||||
|
||||
|
||||
You can find the usage scenarios of the ``reshape`` method in
|
||||
:doc:`Hello Reshape SSD Samples <openvino_inference_engine_samples_hello_reshape_ssd_README>`.
|
||||
|
|
@ -173,13 +173,6 @@ To change the batch dimension of the model, :ref:`set the layout <declare_model_
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: set_batch
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -187,6 +180,13 @@ To change the batch dimension of the model, :ref:`set the layout <declare_model_
|
|||
:language: Python
|
||||
:fragment: set_batch
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ShapeInference.cpp
|
||||
:language: cpp
|
||||
:fragment: set_batch
|
||||
|
||||
|
||||
The ``set_batch`` method is a high-level API of the reshape functionality, so all
|
||||
information about the ``reshape`` method implications are applicable for ``set_batch``
|
||||
|
|
|
|||
|
|
@ -186,19 +186,19 @@ See the following code for using AUTO and specifying devices:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO0.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto.py
|
||||
:language: python
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO0.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
|
||||
|
||||
|
|
@ -211,15 +211,6 @@ To check what devices are present in the system, you can use Device API, as list
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::runtime::Core::get_available_devices()
|
||||
|
||||
See the Hello Query Device C++ Sample for reference.
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -229,6 +220,15 @@ To check what devices are present in the system, you can use Device API, as list
|
|||
|
||||
See the Hello Query Device Python Sample for reference.
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::runtime::Core::get_available_devices()
|
||||
|
||||
See the Hello Query Device C++ Sample for reference.
|
||||
|
||||
|
||||
Excluding Devices from Device Candidate List
|
||||
--------------------------------------------
|
||||
|
|
@ -237,13 +237,6 @@ You can also exclude hardware devices from AUTO, for example, to reserve CPU for
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::CompiledModel compiled_model = core.compile_model(model, "AUTO:-CPU");
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -251,6 +244,13 @@ You can also exclude hardware devices from AUTO, for example, to reserve CPU for
|
|||
|
||||
compiled_model = core.compile_model(model=model, device_name="AUTO:-CPU")
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::CompiledModel compiled_model = core.compile_model(model, "AUTO:-CPU");
|
||||
|
||||
|
||||
AUTO will then query all available devices and remove CPU from the candidate list.
|
||||
|
||||
|
|
@ -298,13 +298,6 @@ If device priority is specified when using CUMULATIVE_THROUGHPUT, AUTO will run
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::CompiledModel compiled_model = core.compile_model(model, "AUTO:GPU,CPU", ov::hint::performance_mode(ov::hint::PerformanceMode::CUMULATIVE_THROUGHPUT));
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -312,18 +305,17 @@ If device priority is specified when using CUMULATIVE_THROUGHPUT, AUTO will run
|
|||
|
||||
compiled_model = core.compile_model(model, "AUTO:GPU,CPU", {"PERFORMANCE_HINT" : {"CUMULATIVE_THROUGHPUT"}})
|
||||
|
||||
|
||||
If AUTO is used without specifying any device names, and if there are multiple GPUs in the system, CUMULATIVE_THROUGHPUT mode will use all of the GPUs by default. If the system has more than two GPU devices, AUTO will remove CPU from the device candidate list to keep the GPUs running at full capacity. A full list of system devices and their unique identifiers can be queried using ov::Core::get_available_devices (for more information, see :doc:`Query Device Properties <openvino_docs_OV_UG_query_api>`). To explicitly specify which GPUs to use, set their priority when compiling with AUTO:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::CompiledModel compiled_model = core.compile_model(model, "AUTO:GPU.1,GPU.0", ov::hint::performance_mode(ov::hint::PerformanceMode::CUMULATIVE_THROUGHPUT));
|
||||
ov::CompiledModel compiled_model = core.compile_model(model, "AUTO:GPU,CPU", ov::hint::performance_mode(ov::hint::PerformanceMode::CUMULATIVE_THROUGHPUT));
|
||||
|
||||
|
||||
If AUTO is used without specifying any device names, and if there are multiple GPUs in the system, CUMULATIVE_THROUGHPUT mode will use all of the GPUs by default. If the system has more than two GPU devices, AUTO will remove CPU from the device candidate list to keep the GPUs running at full capacity. A full list of system devices and their unique identifiers can be queried using ov::Core::get_available_devices (for more information, see :doc:`Query Device Properties <openvino_docs_OV_UG_query_api>`). To explicitly specify which GPUs to use, set their priority when compiling with AUTO:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -332,6 +324,14 @@ If AUTO is used without specifying any device names, and if there are multiple G
|
|||
|
||||
compiled_model = core.compile_model(model, "AUTO:GPU.1,GPU.0", {"PERFORMANCE_HINT" : {"CUMULATIVE_THROUGHPUT"})
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
ov::CompiledModel compiled_model = core.compile_model(model, "AUTO:GPU.1,GPU.0", ov::hint::performance_mode(ov::hint::PerformanceMode::CUMULATIVE_THROUGHPUT));
|
||||
|
||||
|
||||
Code Examples
|
||||
--------------------
|
||||
|
||||
|
|
@ -340,13 +340,6 @@ To enable performance hints for your application, use the following code:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO3.cpp
|
||||
:language: cpp
|
||||
:fragment: [part3]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -354,6 +347,13 @@ To enable performance hints for your application, use the following code:
|
|||
:language: python
|
||||
:fragment: [part3]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO3.cpp
|
||||
:language: cpp
|
||||
:fragment: [part3]
|
||||
|
||||
|
||||
Disabling Auto-Batching for THROUGHPUT and CUMULATIVE_THROUGHPUT
|
||||
----------------------------------------------------------------
|
||||
|
|
@ -368,13 +368,6 @@ The ``ov::hint::model_priority`` property enables you to control the priorities
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO4.cpp
|
||||
:language: cpp
|
||||
:fragment: [part4]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -382,6 +375,13 @@ The ``ov::hint::model_priority`` property enables you to control the priorities
|
|||
:language: python
|
||||
:fragment: [part4]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO4.cpp
|
||||
:language: cpp
|
||||
:fragment: [part4]
|
||||
|
||||
|
||||
Checking Target Runtime Devices
|
||||
+++++++++++++++++++++++++++++++
|
||||
|
|
@ -391,13 +391,6 @@ To query the runtime target devices on which the inferences are being executed u
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO7.cpp
|
||||
:language: cpp
|
||||
:fragment: [part7]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -405,6 +398,12 @@ To query the runtime target devices on which the inferences are being executed u
|
|||
:language: python
|
||||
:fragment: [part7]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO7.cpp
|
||||
:language: cpp
|
||||
:fragment: [part7]
|
||||
|
||||
|
||||
Configuring Individual Devices and Creating the Auto-Device plugin on Top
|
||||
|
|
@ -414,13 +413,6 @@ Although the methods described above are currently the preferred way to execute
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO5.cpp
|
||||
:language: cpp
|
||||
:fragment: [part5]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -428,6 +420,13 @@ Although the methods described above are currently the preferred way to execute
|
|||
:language: python
|
||||
:fragment: [part5]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/AUTO5.cpp
|
||||
:language: cpp
|
||||
:fragment: [part5]
|
||||
|
||||
|
||||
.. _using-auto-with-openvino-samples-and-benchmark-app:
|
||||
|
||||
|
|
|
|||
|
|
@ -30,19 +30,19 @@ How Automatic Batching Works
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.py
|
||||
:language: Python
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
To enable Auto-batching in the legacy apps not akin to the notion of performance hints, you need to use the **explicit** device notion, such as ``BATCH:GPU``.
|
||||
|
||||
|
|
@ -53,13 +53,6 @@ How Automatic Batching Works
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_no_auto_batching]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -67,6 +60,13 @@ How Automatic Batching Works
|
|||
:language: Python
|
||||
:fragment: [compile_model_no_auto_batching]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_no_auto_batching]
|
||||
|
||||
|
||||
Configuring Automatic Batching
|
||||
++++++++++++++++++++++++++++++
|
||||
|
|
@ -90,13 +90,6 @@ The support for Auto-batching is not limited to GPU. However, if a device does n
|
|||
This "automatic batch size selection" works on the presumption that the application queries ``ov::optimal_number_of_infer_requests`` to create the requests of the returned number and run them simultaneously:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [query_optimal_num_requests]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -104,6 +97,13 @@ This "automatic batch size selection" works on the presumption that the applicat
|
|||
.. doxygensnippet:: docs/snippets/ov_auto_batching.py
|
||||
:language: Python
|
||||
:fragment: [query_optimal_num_requests]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [query_optimal_num_requests]
|
||||
|
||||
|
||||
.. _limiting-batch-size:
|
||||
|
|
@ -118,13 +118,6 @@ For example, when the application processes only 4 video streams, there is no ne
|
|||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [hint_num_requests]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -132,6 +125,13 @@ For example, when the application processes only 4 video streams, there is no ne
|
|||
.. doxygensnippet:: docs/snippets/ov_auto_batching.py
|
||||
:language: Python
|
||||
:fragment: [hint_num_requests]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [hint_num_requests]
|
||||
|
||||
|
||||
For the *explicit* usage, you can limit the batch size by using ``BATCH:GPU(4)``, where 4 is the number of requests running in parallel.
|
||||
|
|
|
|||
|
|
@ -47,14 +47,6 @@ Running Deployment Manager in Interactive Mode
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
|
@ -62,6 +54,14 @@ Running Deployment Manager in Interactive Mode
|
|||
|
||||
cd <INSTALL_DIR>\tools\deployment_manager
|
||||
.\deployment_manager.py
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: macos
|
||||
|
|
@ -109,14 +109,6 @@ Running Deployment Manager in Standard CLI Mode
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
|
@ -124,6 +116,14 @@ Running Deployment Manager in Standard CLI Mode
|
|||
|
||||
cd <INSTALL_DIR>\tools\deployment_manager
|
||||
.\deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: macos
|
||||
|
|
@ -161,19 +161,19 @@ To deploy the OpenVINO Runtime components from the development machine to the ta
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
Use the archiver of your choice to unzip the file.
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: macos
|
||||
|
|
@ -202,14 +202,6 @@ To deploy the OpenVINO Runtime components from the development machine to the ta
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <destination_dir>/openvino/
|
||||
source ./setupvars.sh
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
|
@ -217,6 +209,14 @@ To deploy the OpenVINO Runtime components from the development machine to the ta
|
|||
|
||||
cd <destination_dir>\openvino\
|
||||
.\setupvars.bat
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <destination_dir>/openvino/
|
||||
source ./setupvars.sh
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: macos
|
||||
|
|
|
|||
|
|
@ -51,6 +51,7 @@ As shown in the picture above, some plugin libraries may have OS-specific depend
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
| Device | Dependency | Location |
|
||||
|
|
@ -67,7 +68,8 @@ As shown in the picture above, some plugin libraries may have OS-specific depend
|
|||
| Arm® CPU | — | — |
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
|
||||
.. tab-item:: Linux arm64
|
||||
.. tab-item:: Linux arm64
|
||||
:sync: linux-arm-64
|
||||
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
| Device | Dependency | Location |
|
||||
|
|
@ -76,6 +78,7 @@ As shown in the picture above, some plugin libraries may have OS-specific depend
|
|||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
|
||||
.. tab-item:: Linux x86_64
|
||||
:sync: linux-x86-64
|
||||
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
| Device | Dependency | Location |
|
||||
|
|
@ -88,8 +91,8 @@ As shown in the picture above, some plugin libraries may have OS-specific depend
|
|||
| GNA | libgna.so | ``./runtime/lib/intel64/libgna.so.3`` |
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
|
||||
|
||||
.. tab-item:: macOS arm64
|
||||
.. tab-item:: macOS arm64
|
||||
:sync: macos-arm-64
|
||||
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
| Device | Dependency | Location |
|
||||
|
|
@ -98,6 +101,7 @@ As shown in the picture above, some plugin libraries may have OS-specific depend
|
|||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
|
||||
.. tab-item:: macOS x86_64
|
||||
:sync: macos-x86-64
|
||||
|
||||
+--------------+-------------------------+-------------------------------------------------------+
|
||||
| Device | Dependency | Location |
|
||||
|
|
|
|||
|
|
@ -48,13 +48,6 @@ Randomly selecting operations and setting affinities may lead to decrease in mod
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [set_manual_affinities]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -62,6 +55,13 @@ Randomly selecting operations and setting affinities may lead to decrease in mod
|
|||
:language: Python
|
||||
:fragment: [set_manual_affinities]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [set_manual_affinities]
|
||||
|
||||
|
||||
The Automatic Mode
|
||||
--------------------
|
||||
|
|
@ -74,13 +74,6 @@ It does not take into account device peculiarities such as the inability to infe
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -88,6 +81,13 @@ It does not take into account device peculiarities such as the inability to infe
|
|||
:language: Python
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
|
||||
Using Manual and Automatic Modes in Combination
|
||||
-----------------------------------------------
|
||||
|
|
@ -97,13 +97,6 @@ In some cases you may need to consider manually adjusting affinities which were
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [fix_automatic_affinities]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -111,6 +104,13 @@ In some cases you may need to consider manually adjusting affinities which were
|
|||
:language: Python
|
||||
:fragment: [fix_automatic_affinities]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [fix_automatic_affinities]
|
||||
|
||||
|
||||
Importantly, the automatic mode will not work if any operation in a model has its ``"affinity"`` already initialized.
|
||||
|
||||
|
|
@ -126,13 +126,6 @@ If you want different devices in Hetero execution to have different device-speci
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [configure_fallback_devices]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -140,6 +133,13 @@ If you want different devices in Hetero execution to have different device-speci
|
|||
:language: Python
|
||||
:fragment: [configure_fallback_devices]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_hetero.cpp
|
||||
:language: cpp
|
||||
:fragment: [configure_fallback_devices]
|
||||
|
||||
|
||||
In the example above, the ``GPU`` device is configured to enable profiling data and uses the default execution precision, while ``CPU`` has the configuration property to perform inference in ``fp32``.
|
||||
|
||||
|
|
|
|||
|
|
@ -33,13 +33,6 @@ Include next files to work with OpenVINO™ Runtime:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [include]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -47,6 +40,13 @@ Include next files to work with OpenVINO™ Runtime:
|
|||
:language: python
|
||||
:fragment: [import]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [include]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -59,13 +59,6 @@ Use the following code to create OpenVINO™ Core to manage available devices an
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part1]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -73,6 +66,13 @@ Use the following code to create OpenVINO™ Core to manage available devices an
|
|||
:language: python
|
||||
:fragment: [part1]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part1]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -90,53 +90,6 @@ Compile the model for a specific device using ``ov::Core::compile_model()``:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: IR
|
||||
:sync: ir
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_1]
|
||||
|
||||
.. tab-item:: ONNX
|
||||
:sync: onnx
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_2]
|
||||
|
||||
.. tab-item:: PaddlePaddle
|
||||
:sync: pdpd
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_3]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tf
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_4]
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
:sync: tf_lite
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_5]
|
||||
|
||||
.. tab-item:: ov::Model
|
||||
:sync: ovmodel
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_6]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -157,33 +110,80 @@ Compile the model for a specific device using ``ov::Core::compile_model()``:
|
|||
:fragment: [part2_2]
|
||||
|
||||
.. tab-item:: PaddlePaddle
|
||||
:sync: pdpd
|
||||
:sync: paddlepaddle
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.py
|
||||
:language: python
|
||||
:fragment: [part2_3]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tf
|
||||
:sync: tensorflow
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.py
|
||||
:language: python
|
||||
:fragment: [part2_4]
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
:sync: tf_lite
|
||||
:sync: tensorflow_lite
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.py
|
||||
:language: python
|
||||
:fragment: [part2_5]
|
||||
|
||||
.. tab-item:: ov::Model
|
||||
:sync: ovmodel
|
||||
:sync: openvinomodel
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.py
|
||||
:language: python
|
||||
:fragment: [part2_6]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: IR
|
||||
:sync: ir
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_1]
|
||||
|
||||
.. tab-item:: ONNX
|
||||
:sync: onnx
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_2]
|
||||
|
||||
.. tab-item:: PaddlePaddle
|
||||
:sync: paddlepaddle
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_3]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tensorflow
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_4]
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
:sync: tensorflow_lite
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_5]
|
||||
|
||||
.. tab-item:: ov::Model
|
||||
:sync: openvinomodel
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2_6]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -204,28 +204,28 @@ Compile the model for a specific device using ``ov::Core::compile_model()``:
|
|||
:fragment: [part2_2]
|
||||
|
||||
.. tab-item:: PaddlePaddle
|
||||
:sync: pdpd
|
||||
:sync: paddlepaddle
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.c
|
||||
:language: cpp
|
||||
:fragment: [part2_3]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tf
|
||||
:sync: tensorflow
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.c
|
||||
:language: cpp
|
||||
:fragment: [part2_4]
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
:sync: tf_lite
|
||||
:sync: tensorflow_lite
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.c
|
||||
:language: cpp
|
||||
:fragment: [part2_5]
|
||||
|
||||
.. tab-item:: ov::Model
|
||||
:sync: ovmodel
|
||||
:sync: openvinomodel
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.c
|
||||
:language: cpp
|
||||
|
|
@ -248,13 +248,6 @@ Create an infer request using the following code (see
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part3]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -262,6 +255,13 @@ Create an infer request using the following code (see
|
|||
:language: python
|
||||
:fragment: [part3]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part3]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -277,13 +277,6 @@ You can use external memory to create ``ov::Tensor`` and use the ``ov::InferRequ
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part4]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -291,6 +284,13 @@ You can use external memory to create ``ov::Tensor`` and use the ``ov::InferRequ
|
|||
:language: python
|
||||
:fragment: [part4]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part4]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -306,13 +306,6 @@ OpenVINO™ Runtime supports inference in either synchronous or asynchronous mod
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part5]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -320,6 +313,13 @@ OpenVINO™ Runtime supports inference in either synchronous or asynchronous mod
|
|||
:language: python
|
||||
:fragment: [part5]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part5]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -338,13 +338,6 @@ Go over the output tensors and process the inference results.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part6]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -352,6 +345,13 @@ Go over the output tensors and process the inference results.
|
|||
:language: python
|
||||
:fragment: [part6]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/src/main.cpp
|
||||
:language: cpp
|
||||
:fragment: [part6]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
|
|||
|
|
@ -33,13 +33,6 @@ The easiest way is to fully specify each dimension with one alphabet letter.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:simple
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -47,6 +40,13 @@ The easiest way is to fully specify each dimension with one alphabet letter.
|
|||
:language: python
|
||||
:fragment: ov:layout:simple
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:simple
|
||||
|
||||
|
||||
This assigns ``N`` to the first dimension, ``C`` to the second, ``H`` to the third, and ``W`` to the fourth.
|
||||
|
||||
|
|
@ -58,13 +58,6 @@ The advanced syntax allows assigning a word to a dimension. To do this, wrap a l
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:complex
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -72,6 +65,13 @@ The advanced syntax allows assigning a word to a dimension. To do this, wrap a l
|
|||
:language: python
|
||||
:fragment: ov:layout:complex
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:complex
|
||||
|
||||
|
||||
Partially Defined Layout
|
||||
++++++++++++++++++++++++
|
||||
|
|
@ -81,13 +81,6 @@ If a certain dimension is not important, its name can be set to ``?``.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:partially_defined
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -95,6 +88,13 @@ If a certain dimension is not important, its name can be set to ``?``.
|
|||
:language: python
|
||||
:fragment: ov:layout:partially_defined
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:partially_defined
|
||||
|
||||
|
||||
Dynamic Layout
|
||||
++++++++++++++++++++
|
||||
|
|
@ -104,13 +104,6 @@ If several dimensions are not important, an ellipsis ``...`` can be used to spec
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:dynamic
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -118,6 +111,13 @@ If several dimensions are not important, an ellipsis ``...`` can be used to spec
|
|||
:language: python
|
||||
:fragment: ov:layout:dynamic
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:dynamic
|
||||
|
||||
|
||||
Predefined Names
|
||||
++++++++++++++++++++
|
||||
|
|
@ -135,13 +135,6 @@ These names are used in :doc:`PreProcessing API <openvino_docs_OV_UG_Preprocessi
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:predefined
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -149,6 +142,13 @@ These names are used in :doc:`PreProcessing API <openvino_docs_OV_UG_Preprocessi
|
|||
:language: python
|
||||
:fragment: ov:layout:predefined
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:predefined
|
||||
|
||||
|
||||
Equality
|
||||
++++++++++++++++++++
|
||||
|
|
@ -163,13 +163,6 @@ A layout can be converted to a string in the advanced syntax format. It can be u
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:dump
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -177,6 +170,13 @@ A layout can be converted to a string in the advanced syntax format. It can be u
|
|||
:language: python
|
||||
:fragment: ov:layout:dump
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:dump
|
||||
|
||||
|
||||
Get layout from Model Input/Output
|
||||
++++++++++++++++++++++++++++++++++
|
||||
|
|
@ -186,13 +186,6 @@ OpenVINO provides helpers which provide a simple interface to get layout from Mo
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:get_from_model
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -200,6 +193,13 @@ OpenVINO provides helpers which provide a simple interface to get layout from Mo
|
|||
:language: python
|
||||
:fragment: ov:layout:get_from_model
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_layout.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:layout:get_from_model
|
||||
|
||||
|
||||
See also
|
||||
####################
|
||||
|
|
|
|||
|
|
@ -33,13 +33,6 @@ Based on the steps, the following code demonstrates how to change the applicatio
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:create_core
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -47,6 +40,13 @@ Based on the steps, the following code demonstrates how to change the applicatio
|
|||
:language: python
|
||||
:fragment: ie:create_core
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:create_core
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -58,13 +58,6 @@ Based on the steps, the following code demonstrates how to change the applicatio
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:create_core
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -72,6 +65,13 @@ Based on the steps, the following code demonstrates how to change the applicatio
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:create_core
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:create_core
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -91,13 +91,6 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:load_old_extension
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -105,6 +98,13 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
:language: python
|
||||
:fragment: ie:load_old_extension
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:load_old_extension
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -117,13 +117,6 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:load_old_extension
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -131,6 +124,13 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:load_old_extension
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:load_old_extension
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -146,13 +146,6 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:read_model
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -160,6 +153,13 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
:language: python
|
||||
:fragment: ie:read_model
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:read_model
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -172,13 +172,6 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:read_model
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -186,6 +179,13 @@ to write extensions. However, you can also load the old extensions to the new Op
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:read_model
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:read_model
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -213,13 +213,6 @@ preprocessing may be necessary. See :doc:`preprocessing in API 2.0 <openvino_2_0
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:compile_model
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -227,6 +220,13 @@ preprocessing may be necessary. See :doc:`preprocessing in API 2.0 <openvino_2_0
|
|||
:language: python
|
||||
:fragment: ie:compile_model
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:compile_model
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -239,13 +239,6 @@ preprocessing may be necessary. See :doc:`preprocessing in API 2.0 <openvino_2_0
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:compile_model
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -253,6 +246,13 @@ preprocessing may be necessary. See :doc:`preprocessing in API 2.0 <openvino_2_0
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:compile_model
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:compile_model
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -271,13 +271,6 @@ If you need to configure devices with additional parameters for OpenVINO Runtime
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:create_infer_request
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -285,6 +278,13 @@ If you need to configure devices with additional parameters for OpenVINO Runtime
|
|||
:language: python
|
||||
:fragment: ie:create_infer_request
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:create_infer_request
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -297,13 +297,6 @@ If you need to configure devices with additional parameters for OpenVINO Runtime
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:create_infer_request
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -311,6 +304,13 @@ If you need to configure devices with additional parameters for OpenVINO Runtime
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:create_infer_request
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:create_infer_request
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -329,17 +329,10 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: IR v10
|
||||
:sync: irv10
|
||||
:sync: ir-v10
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -347,6 +340,13 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
:language: python
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -355,17 +355,10 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: IR v11
|
||||
:sync: irv11
|
||||
:sync: ir-v11
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -373,6 +366,13 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
:language: python
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -385,13 +385,6 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -399,6 +392,13 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
:language: python
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -412,13 +412,6 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -426,6 +419,13 @@ The Inference Engine API fills inputs with data of the ``I32`` precision (**not*
|
|||
:language: python
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_input_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -441,17 +441,10 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: IR v10
|
||||
:sync: irv10
|
||||
:sync: ir-v10
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_v10
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -459,6 +452,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_input_tensor_v10
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_v10
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -467,17 +467,10 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:fragment: ov_api_2_0:get_input_tensor_v10
|
||||
|
||||
.. tab-item:: IR v11
|
||||
:sync: irv11
|
||||
:sync: ir-v11
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -485,6 +478,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -497,13 +497,6 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -511,6 +504,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -520,17 +520,10 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
|
||||
|
||||
.. tab-item:: Model created in code
|
||||
:sync: model_created_in_code
|
||||
:sync: model-created-in-code
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -538,6 +531,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_input_tensor_aligned
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -558,13 +558,6 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:inference
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -572,6 +565,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ie:inference
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:inference
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -584,13 +584,6 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:start_async_and_wait
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -598,6 +591,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ie:start_async_and_wait
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:start_async_and_wait
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -615,13 +615,6 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:inference
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -629,6 +622,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:inference
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:inference
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -641,13 +641,6 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:start_async_and_wait
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -655,6 +648,13 @@ API 2.0 fills inputs with data of the ``I64`` precision (aligned with the origin
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:start_async_and_wait
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:start_async_and_wait
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -673,17 +673,10 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: IR v10
|
||||
:sync: irv10
|
||||
:sync: ir-v10
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -691,6 +684,13 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
:language: python
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -699,17 +699,10 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: IR v11
|
||||
:sync: irv11
|
||||
:sync: ir-v11
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -717,6 +710,13 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
:language: python
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -729,13 +729,6 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -743,6 +736,13 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
:language: python
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -756,13 +756,6 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -770,6 +763,13 @@ The Inference Engine API processes outputs as they are of the ``I32`` precision
|
|||
:language: python
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ie_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ie:get_output_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -788,17 +788,10 @@ API 2.0 processes outputs as they are of:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: IR v10
|
||||
:sync: irv10
|
||||
:sync: ir-v10
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_v10
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -806,6 +799,13 @@ API 2.0 processes outputs as they are of:
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_output_tensor_v10
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_v10
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -814,17 +814,10 @@ API 2.0 processes outputs as they are of:
|
|||
:fragment: ov_api_2_0:get_output_tensor_v10
|
||||
|
||||
.. tab-item:: IR v11
|
||||
:sync: irv11
|
||||
:sync: ir-v11
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -832,6 +825,13 @@ API 2.0 processes outputs as they are of:
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -844,13 +844,6 @@ API 2.0 processes outputs as they are of:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -858,6 +851,13 @@ API 2.0 processes outputs as they are of:
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -867,17 +867,10 @@ API 2.0 processes outputs as they are of:
|
|||
|
||||
|
||||
.. tab-item:: Model created in code
|
||||
:sync: model
|
||||
:sync: model-created-in-code
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -885,6 +878,13 @@ API 2.0 processes outputs as they are of:
|
|||
:language: python
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_common.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_api_2_0:get_output_tensor_aligned
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
|
|||
|
|
@ -124,32 +124,6 @@ Setting Configuration Values
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Devices
|
||||
:sync: devices
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_set_property
|
||||
|
||||
.. tab-item:: Model Loading
|
||||
:sync: model-loading
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_compile_model
|
||||
|
||||
.. tab-item:: Execution
|
||||
:sync: execution
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: compiled_model_set_property
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -176,6 +150,32 @@ Setting Configuration Values
|
|||
:language: python
|
||||
:fragment: compiled_model_set_property
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Devices
|
||||
:sync: devices
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_set_property
|
||||
|
||||
.. tab-item:: Model Loading
|
||||
:sync: model-loading
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_compile_model
|
||||
|
||||
.. tab-item:: Execution
|
||||
:sync: execution
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: compiled_model_set_property
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -211,46 +211,13 @@ Getting Information
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-config
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_config
|
||||
|
||||
.. tab-item:: Device metrics
|
||||
:sync: device-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_metric
|
||||
|
||||
.. tab-item:: Execution config
|
||||
:sync: execution-config
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: executable_network_set_config
|
||||
|
||||
.. tab-item:: Execution metrics
|
||||
:sync: execution-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: executable_network_get_metric
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-config
|
||||
:sync: device-configuration
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.py
|
||||
:language: python
|
||||
|
|
@ -277,13 +244,46 @@ Getting Information
|
|||
:language: python
|
||||
:fragment: executable_network_get_metric
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-configuration
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_config
|
||||
|
||||
.. tab-item:: Device metrics
|
||||
:sync: device-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_metric
|
||||
|
||||
.. tab-item:: Execution config
|
||||
:sync: execution-config
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: executable_network_set_config
|
||||
|
||||
.. tab-item:: Execution metrics
|
||||
:sync: execution-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: executable_network_get_metric
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-config
|
||||
:sync: device-configuration
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.c
|
||||
:language: c
|
||||
|
|
@ -316,46 +316,13 @@ Getting Information
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-config
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_rw_property
|
||||
|
||||
.. tab-item:: Device metrics
|
||||
:sync: device-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_ro_property
|
||||
|
||||
.. tab-item:: Execution config
|
||||
:sync: execution-config
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: compiled_model_get_rw_property
|
||||
|
||||
.. tab-item:: Execution metrics
|
||||
:sync: execution-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: compiled_model_get_ro_property
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-config
|
||||
:sync: device-configuration
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.py
|
||||
:language: python
|
||||
|
|
@ -382,13 +349,46 @@ Getting Information
|
|||
:language: python
|
||||
:fragment: compiled_model_get_ro_property
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-configuration
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_rw_property
|
||||
|
||||
.. tab-item:: Device metrics
|
||||
:sync: device-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: core_get_ro_property
|
||||
|
||||
.. tab-item:: Execution config
|
||||
:sync: execution-config
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: compiled_model_get_rw_property
|
||||
|
||||
.. tab-item:: Execution metrics
|
||||
:sync: execution-metrics
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: compiled_model_get_ro_property
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Device Configuration
|
||||
:sync: device-config
|
||||
:sync: device-configuration
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_migration.c
|
||||
:language: c
|
||||
|
|
|
|||
|
|
@ -111,7 +111,7 @@ It is possible to build applications without the CMake interface by using: MSVC
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Include dirs
|
||||
:sync: inc-dirs
|
||||
:sync: include-dirs
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -150,7 +150,7 @@ It is possible to build applications without the CMake interface by using: MSVC
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Include dirs
|
||||
:sync: inc-dirs
|
||||
:sync: include-dirs
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -17,13 +17,6 @@ nGraph API
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ngraph.cpp
|
||||
:language: cpp
|
||||
:fragment: ngraph:graph
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -31,6 +24,13 @@ nGraph API
|
|||
:language: Python
|
||||
:fragment: ngraph:graph
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ngraph.cpp
|
||||
:language: cpp
|
||||
:fragment: ngraph:graph
|
||||
|
||||
|
||||
API 2.0
|
||||
####################
|
||||
|
|
@ -38,13 +38,6 @@ API 2.0
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_graph.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:graph
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -52,6 +45,13 @@ API 2.0
|
|||
:language: Python
|
||||
:fragment: ov:graph
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_graph.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:graph
|
||||
|
||||
|
||||
Additional Resources
|
||||
####################
|
||||
|
|
|
|||
|
|
@ -67,13 +67,6 @@ Using Mean and Scale Values
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: mean_scale
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -81,6 +74,13 @@ Using Mean and Scale Values
|
|||
:language: python
|
||||
:fragment: mean_scale
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: mean_scale
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -94,13 +94,6 @@ Using Mean and Scale Values
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_mean_scale
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -108,6 +101,13 @@ Using Mean and Scale Values
|
|||
:language: python
|
||||
:fragment: ov_mean_scale
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_mean_scale
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -124,13 +124,6 @@ Converting Precision and Layout
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: conversions
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -138,6 +131,13 @@ Converting Precision and Layout
|
|||
:language: python
|
||||
:fragment: conversions
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: conversions
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -151,13 +151,6 @@ Converting Precision and Layout
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_conversions
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -165,6 +158,13 @@ Converting Precision and Layout
|
|||
:language: python
|
||||
:fragment: ov_conversions
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_conversions
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -181,13 +181,6 @@ Using Image Scaling
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: image_scale
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -195,6 +188,13 @@ Using Image Scaling
|
|||
:language: python
|
||||
:fragment: image_scale
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: image_scale
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -208,13 +208,6 @@ Using Image Scaling
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_image_scale
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -222,6 +215,13 @@ Using Image Scaling
|
|||
:language: python
|
||||
:fragment: ov_image_scale
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_image_scale
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -266,13 +266,6 @@ Converting Color Space
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_color_space
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -280,6 +273,13 @@ Converting Color Space
|
|||
:language: python
|
||||
:fragment: ov_color_space
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing_migration.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_color_space
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
|
|||
|
|
@ -27,13 +27,6 @@ OpenVINO™ Runtime enables you to use different approaches to work with model i
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [all_inputs_ouputs]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -41,6 +34,13 @@ OpenVINO™ Runtime enables you to use different approaches to work with model i
|
|||
:language: cpp
|
||||
:fragment: [all_inputs_ouputs]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [all_inputs_ouputs]
|
||||
|
||||
|
||||
For details on how to build a model in OpenVINO™ Runtime, see the :ref:`Build a Model in OpenVINO Runtime <ov_ug_build_model>` section.
|
||||
|
||||
|
|
@ -59,13 +59,6 @@ OpenVINO™ Runtime provides two types for shape representation:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:partial_shape]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -73,6 +66,13 @@ OpenVINO™ Runtime provides two types for shape representation:
|
|||
:language: cpp
|
||||
:fragment: [ov:partial_shape]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:partial_shape]
|
||||
|
||||
|
||||
However, in most cases, before getting static shape using the ``get_shape()`` method, you need to check if that shape is static.
|
||||
|
||||
|
|
@ -104,13 +104,6 @@ To build an ``:ref:`ov::Model <doxid-classov_1_1_model>``` instance from ``opset
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:include]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -118,17 +111,17 @@ To build an ``:ref:`ov::Model <doxid-classov_1_1_model>``` instance from ``opset
|
|||
:language: cpp
|
||||
:fragment: [import]
|
||||
|
||||
|
||||
The following code demonstrates how to create a simple model:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:create_simple_model]
|
||||
:fragment: [ov:include]
|
||||
|
||||
|
||||
The following code demonstrates how to create a simple model:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -137,18 +130,18 @@ The following code demonstrates how to create a simple model:
|
|||
:language: cpp
|
||||
:fragment: [ov:create_simple_model]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:create_simple_model]
|
||||
|
||||
|
||||
The following code creates a model with several outputs:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:create_advanced_model]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -156,6 +149,13 @@ The following code creates a model with several outputs:
|
|||
:language: cpp
|
||||
:fragment: [ov:create_advanced_model]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:create_advanced_model]
|
||||
|
||||
|
||||
Model Debugging Capabilities
|
||||
###########################################
|
||||
|
|
@ -168,13 +168,6 @@ OpenVINO™ provides several debug capabilities:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:visualize]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -182,6 +175,13 @@ OpenVINO™ provides several debug capabilities:
|
|||
:language: cpp
|
||||
:fragment: [ov:visualize]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:visualize]
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -204,13 +204,6 @@ OpenVINO™ provides several debug capabilities:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:serialize]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -218,6 +211,13 @@ OpenVINO™ provides several debug capabilities:
|
|||
:language: cpp
|
||||
:fragment: [ov:serialize]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:serialize]
|
||||
|
||||
|
||||
Additional Resources
|
||||
########################
|
||||
|
|
|
|||
|
|
@ -53,13 +53,6 @@ The following commands are accepted by the API:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/MULTI0.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -67,6 +60,13 @@ The following commands are accepted by the API:
|
|||
:language: python
|
||||
:fragment: [MULTI_0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/MULTI0.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
|
||||
To check what devices are present in the system, you can use the Device API. For information on how to do it, check :doc:`Query device properties and configuration <openvino_docs_OV_UG_query_api>`.
|
||||
|
||||
|
|
@ -78,13 +78,6 @@ As mentioned previously, executing inference with MULTI may be set up by configu
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/MULTI4.cpp
|
||||
:language: cpp
|
||||
:fragment: [part4]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -92,6 +85,13 @@ As mentioned previously, executing inference with MULTI may be set up by configu
|
|||
:language: python
|
||||
:fragment: [MULTI_4]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/MULTI4.cpp
|
||||
:language: cpp
|
||||
:fragment: [part4]
|
||||
|
||||
|
||||
Alternatively, you can combine all the individual device settings into a single config file and load it for MULTI to parse. See the code example in the next section.
|
||||
|
||||
|
|
|
|||
|
|
@ -66,13 +66,6 @@ The examples below show how to set dynamic dimensions with a model that has a st
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:reshape_undefined]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -82,6 +75,13 @@ The examples below show how to set dynamic dimensions with a model that has a st
|
|||
|
||||
With Python, you may also pass all dimensions as a string and use ``?`` for the dynamic dimensions (e.g. ``model.reshape(“1, 3, ?, ?”)``).
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_dynamic_shapes:reshape_undefined]
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -96,13 +96,6 @@ The examples above assume that the model has a single input layer. To change mod
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:reshape_multiple_inputs
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -110,6 +103,13 @@ The examples above assume that the model has a single input layer. To change mod
|
|||
:language: python
|
||||
:fragment: reshape_multiple_inputs
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:reshape_multiple_inputs
|
||||
|
||||
|
||||
For more examples of how to change multiple input layers, see :doc:`Changing Input Shapes <openvino_docs_OV_UG_ShapeInference>`.
|
||||
|
||||
|
|
@ -122,13 +122,6 @@ To check if a model already has dynamic dimensions, first load it with the ``rea
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:check_inputs
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -136,6 +129,13 @@ To check if a model already has dynamic dimensions, first load it with the ``rea
|
|||
:language: python
|
||||
:fragment: check_inputs
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:check_inputs
|
||||
|
||||
|
||||
If the input model already has dynamic dimensions, that will not change during inference. If the inputs will not be used dynamically, it is recommended to set them to static values using the ``reshape`` method to save application memory and potentially improve inference speed. The OpenVINO API supports any combination of static and dynamic dimensions.
|
||||
|
||||
|
|
@ -147,16 +147,7 @@ Dimension Bounds
|
|||
The lower and/or upper bounds of a dynamic dimension can also be specified. They define a range of allowed values for the dimension. Dimension bounds can be set by passing the lower and upper bounds into the ``reshape`` method using the options shown below.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
The dimension bounds can be coded as arguments for ``ov::Dimension``, as shown in these examples:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:reshape_bounds
|
||||
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -172,6 +163,15 @@ The lower and/or upper bounds of a dynamic dimension can also be specified. They
|
|||
:language: python
|
||||
:fragment: reshape_bounds
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
The dimension bounds can be coded as arguments for ``ov::Dimension``, as shown in these examples:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:reshape_bounds
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -203,19 +203,19 @@ The sample below shows how a model can accept different input shapes. In the fir
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:set_input_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.py
|
||||
:language: python
|
||||
:fragment: set_input_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:set_input_tensor
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
|
@ -238,13 +238,6 @@ To determine if the output has dynamic dimensions, the ``partial_shape`` propert
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:print_dynamic
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -252,6 +245,13 @@ To determine if the output has dynamic dimensions, the ``partial_shape`` propert
|
|||
:language: python
|
||||
:fragment: print_dynamic
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:print_dynamic
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
@ -267,13 +267,6 @@ Output layers can also be checked for dynamic dimensions using the ``partial_sha
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:detect_dynamic
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -281,6 +274,13 @@ Output layers can also be checked for dynamic dimensions using the ``partial_sha
|
|||
:language: python
|
||||
:fragment: detect_dynamic
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_dynamic_shapes.cpp
|
||||
:language: cpp
|
||||
:fragment: ov_dynamic_shapes:detect_dynamic
|
||||
|
||||
.. tab-item:: C
|
||||
:sync: c
|
||||
|
||||
|
|
|
|||
|
|
@ -20,13 +20,6 @@ The ``ov::InferRequest`` can be created from the ``ov::CompiledModel``:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: create_infer_request
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -34,6 +27,13 @@ The ``ov::InferRequest`` can be created from the ``ov::CompiledModel``:
|
|||
:language: python
|
||||
:fragment: create_infer_request
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: create_infer_request
|
||||
|
||||
|
||||
Run Inference
|
||||
####################
|
||||
|
|
@ -48,13 +48,6 @@ You can use ``ov::InferRequest::infer``, which blocks the application execution,
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: sync_infer
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -62,6 +55,13 @@ You can use ``ov::InferRequest::infer``, which blocks the application execution,
|
|||
:language: python
|
||||
:fragment: sync_infer
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: sync_infer
|
||||
|
||||
|
||||
Asynchronous Mode
|
||||
++++++++++++++++++++
|
||||
|
|
@ -71,13 +71,6 @@ The asynchronous mode can improve application's overall frame-rate, by making it
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: async_infer
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -85,6 +78,13 @@ The asynchronous mode can improve application's overall frame-rate, by making it
|
|||
:language: python
|
||||
:fragment: async_infer
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: async_infer
|
||||
|
||||
|
||||
Asynchronous mode supports two ways the application waits for inference results:
|
||||
|
||||
|
|
@ -92,13 +92,6 @@ Asynchronous mode supports two ways the application waits for inference results:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: wait_for
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -106,11 +99,25 @@ Asynchronous mode supports two ways the application waits for inference results:
|
|||
:language: python
|
||||
:fragment: wait_for
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: wait_for
|
||||
|
||||
|
||||
* ``ov::InferRequest::wait`` - waits until inference result becomes available
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.py
|
||||
:language: python
|
||||
:fragment: wait
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -118,12 +125,6 @@ Asynchronous mode supports two ways the application waits for inference results:
|
|||
:language: cpp
|
||||
:fragment: wait
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.py
|
||||
:language: python
|
||||
:fragment: wait
|
||||
|
||||
|
||||
Both methods are thread-safe.
|
||||
|
|
@ -133,13 +134,6 @@ When you are running several inference requests in parallel, a device can proces
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: set_callback
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -147,6 +141,13 @@ When you are running several inference requests in parallel, a device can proces
|
|||
:language: python
|
||||
:fragment: set_callback
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: set_callback
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -160,13 +161,6 @@ You can use the ``ov::InferRequest::cancel`` method if you want to abort executi
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: cancel
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -174,6 +168,13 @@ You can use the ``ov::InferRequest::cancel`` method if you want to abort executi
|
|||
:language: python
|
||||
:fragment: cancel
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: cancel
|
||||
|
||||
|
||||
.. _in_out_tensors:
|
||||
|
||||
|
|
@ -187,13 +188,6 @@ Working with Input and Output tensors
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: get_set_one_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -201,6 +195,12 @@ Working with Input and Output tensors
|
|||
:language: python
|
||||
:fragment: get_set_one_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: get_set_one_tensor
|
||||
|
||||
|
||||
* ``ov::InferRequest::get_input_tensor``, ``ov::InferRequest::set_input_tensor``, ``ov::InferRequest::get_output_tensor``, ``ov::InferRequest::set_output_tensor`` methods with argument can be used to get or set input/output tensor by input/output index:
|
||||
|
|
@ -208,13 +208,6 @@ Working with Input and Output tensors
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: get_set_index_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -222,18 +215,18 @@ Working with Input and Output tensors
|
|||
:language: python
|
||||
:fragment: get_set_index_tensor
|
||||
|
||||
|
||||
* ``ov::InferRequest::get_tensor``, ``ov::InferRequest::set_tensor`` methods can be used to get or set input/output tensor by tensor name:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: get_set_tensor
|
||||
:fragment: get_set_index_tensor
|
||||
|
||||
|
||||
* ``ov::InferRequest::get_tensor``, ``ov::InferRequest::set_tensor`` methods can be used to get or set input/output tensor by tensor name:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -242,18 +235,18 @@ Working with Input and Output tensors
|
|||
:language: python
|
||||
:fragment: get_set_tensor
|
||||
|
||||
|
||||
* ``ov::InferRequest::get_tensor``, ``ov::InferRequest::set_tensor`` methods can be used to get or set input/output tensor by port:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: get_set_tensor_by_port
|
||||
:fragment: get_set_tensor
|
||||
|
||||
|
||||
* ``ov::InferRequest::get_tensor``, ``ov::InferRequest::set_tensor`` methods can be used to get or set input/output tensor by port:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -262,6 +255,13 @@ Working with Input and Output tensors
|
|||
:language: python
|
||||
:fragment: get_set_tensor_by_port
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: get_set_tensor_by_port
|
||||
|
||||
|
||||
Examples of Infer Request Usages
|
||||
################################
|
||||
|
|
@ -277,13 +277,6 @@ In this case, you can get the output tensor from the first request, using ``ov::
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: cascade_models
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -291,6 +284,13 @@ In this case, you can get the output tensor from the first request, using ``ov::
|
|||
:language: python
|
||||
:fragment: cascade_models
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: cascade_models
|
||||
|
||||
|
||||
Using of ROI Tensors
|
||||
++++++++++++++++++++
|
||||
|
|
@ -300,13 +300,6 @@ It is possible to re-use shared input in several models. You do not need to allo
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: roi_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -315,6 +308,13 @@ It is possible to re-use shared input in several models. You do not need to allo
|
|||
:fragment: roi_tensor
|
||||
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: roi_tensor
|
||||
|
||||
Using Remote Tensors
|
||||
++++++++++++++++++++
|
||||
|
||||
|
|
@ -323,13 +323,6 @@ By using ``ov::RemoteContext`` you can create a remote tensor to work with remot
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: remote_tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -337,5 +330,12 @@ By using ``ov::RemoteContext`` you can create a remote tensor to work with remot
|
|||
:language: python
|
||||
:fragment: remote_tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_infer_request.cpp
|
||||
:language: cpp
|
||||
:fragment: remote_tensor
|
||||
|
||||
|
||||
@endsphinxdirective
|
||||
|
|
|
|||
|
|
@ -57,13 +57,6 @@ In the example code snippet below, ``ov::hint::PerformanceMode::THROUGHPUT`` is
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -71,6 +64,13 @@ In the example code snippet below, ``ov::hint::PerformanceMode::THROUGHPUT`` is
|
|||
:language: python
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
|
||||
Additional (Optional) Hints from the App
|
||||
########################################
|
||||
|
|
@ -80,13 +80,6 @@ As mentioned earlier, this will limit the batch size for the GPU and the number
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [hint_num_requests]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -94,6 +87,13 @@ As mentioned earlier, this will limit the batch size for the GPU and the number
|
|||
:language: python
|
||||
:fragment: [hint_num_requests]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [hint_num_requests]
|
||||
|
||||
|
||||
Optimal Number of Inference Requests
|
||||
####################################
|
||||
|
|
@ -102,13 +102,6 @@ The hints are used on the presumption that the application queries ``ov::optimal
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [query_optimal_num_requests]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -116,6 +109,13 @@ The hints are used on the presumption that the application queries ``ov::optimal
|
|||
:language: python
|
||||
:fragment: [query_optimal_num_requests]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [query_optimal_num_requests]
|
||||
|
||||
|
||||
While an application is free to create more requests if needed (for example to support asynchronous inputs population) **it is very important to at least run the** ``ov::optimal_number_of_infer_requests`` **of the inference requests in parallel**. It is recommended for efficiency, or device utilization, reasons.
|
||||
|
||||
|
|
@ -143,13 +143,6 @@ For example, use ``ov::hint::PerformanceMode::THROUGHPUT`` to prepare a general
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [hint_plus_low_level]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -157,6 +150,13 @@ For example, use ``ov::hint::PerformanceMode::THROUGHPUT`` to prepare a general
|
|||
:language: python
|
||||
:fragment: [hint_plus_low_level]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [hint_plus_low_level]
|
||||
|
||||
|
||||
Testing Performance of the Hints with the Benchmark_App
|
||||
#######################################################
|
||||
|
|
|
|||
|
|
@ -30,13 +30,6 @@ Code examples:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/ov_execution_mode.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:execution_mode:part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -44,6 +37,14 @@ Code examples:
|
|||
:language: python
|
||||
:fragment: [ov:execution_mode:part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/ov_execution_mode.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:execution_mode:part0]
|
||||
|
||||
|
||||
Inference Precision
|
||||
###################
|
||||
|
||||
|
|
|
|||
|
|
@ -21,13 +21,6 @@ If the model has only one input, then simple `ov::preprocess::PrePostProcessor::
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:input_1
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -35,18 +28,18 @@ If the model has only one input, then simple `ov::preprocess::PrePostProcessor::
|
|||
:language: python
|
||||
:fragment: ov:preprocess:input_1
|
||||
|
||||
|
||||
In general, when a model has multiple inputs/outputs, each one can be addressed by a tensor name.
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:input_name
|
||||
:fragment: ov:preprocess:input_1
|
||||
|
||||
|
||||
In general, when a model has multiple inputs/outputs, each one can be addressed by a tensor name.
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -55,17 +48,17 @@ In general, when a model has multiple inputs/outputs, each one can be addressed
|
|||
:language: python
|
||||
:fragment: ov:preprocess:input_name
|
||||
|
||||
|
||||
Or by it's index.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:input_index
|
||||
:fragment: ov:preprocess:input_name
|
||||
|
||||
|
||||
Or by it's index.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -74,6 +67,13 @@ Or by it's index.
|
|||
:language: python
|
||||
:fragment: ov:preprocess:input_index
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:input_index
|
||||
|
||||
|
||||
C++ references:
|
||||
|
||||
|
|
@ -96,13 +96,6 @@ Typical data normalization includes 2 operations for each data item: subtract me
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:mean_scale
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -110,18 +103,18 @@ Typical data normalization includes 2 operations for each data item: subtract me
|
|||
:language: python
|
||||
:fragment: ov:preprocess:mean_scale
|
||||
|
||||
|
||||
In Computer Vision area normalization is usually done separately for R, G, B values. To do this, :doc:`layout with 'C' dimension <openvino_docs_OV_UG_Layout_Overview>` shall be defined. Example:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:mean_scale_array
|
||||
:fragment: ov:preprocess:mean_scale
|
||||
|
||||
|
||||
In Computer Vision area normalization is usually done separately for R, G, B values. To do this, :doc:`layout with 'C' dimension <openvino_docs_OV_UG_Layout_Overview>` shall be defined. Example:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -130,6 +123,13 @@ In Computer Vision area normalization is usually done separately for R, G, B val
|
|||
:language: python
|
||||
:fragment: ov:preprocess:mean_scale_array
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:mean_scale_array
|
||||
|
||||
|
||||
C++ references:
|
||||
|
||||
|
|
@ -147,6 +147,13 @@ To integrate precision conversion into an execution graph as a pre-processing st
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.py
|
||||
:language: python
|
||||
:fragment: ov:preprocess:convert_element_type
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -154,12 +161,7 @@ To integrate precision conversion into an execution graph as a pre-processing st
|
|||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_element_type
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.py
|
||||
:language: python
|
||||
:fragment: ov:preprocess:convert_element_type
|
||||
|
||||
|
||||
C++ references:
|
||||
|
|
@ -178,13 +180,6 @@ Conversion can be done implicitly, using the :doc:`layout <openvino_docs_OV_UG_L
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_layout
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -192,18 +187,18 @@ Conversion can be done implicitly, using the :doc:`layout <openvino_docs_OV_UG_L
|
|||
:language: python
|
||||
:fragment: ov:preprocess:convert_layout
|
||||
|
||||
|
||||
For a manual transpose of axes without the use of a :doc:`layout <openvino_docs_OV_UG_Layout_Overview>` in the code:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_layout_2
|
||||
:fragment: ov:preprocess:convert_layout
|
||||
|
||||
|
||||
For a manual transpose of axes without the use of a :doc:`layout <openvino_docs_OV_UG_Layout_Overview>` in the code:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -212,6 +207,13 @@ For a manual transpose of axes without the use of a :doc:`layout <openvino_docs_
|
|||
:language: python
|
||||
:fragment: ov:preprocess:convert_layout_2
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_layout_2
|
||||
|
||||
|
||||
It performs the same transpose. However, the approach where source and destination layout are used can be easier to read and understand.
|
||||
|
||||
|
|
@ -232,13 +234,6 @@ To resize the input image, it is needed to define ``H`` and ``W`` dimensions of
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:resize_1
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -246,18 +241,18 @@ To resize the input image, it is needed to define ``H`` and ``W`` dimensions of
|
|||
:language: python
|
||||
:fragment: ov:preprocess:resize_1
|
||||
|
||||
|
||||
When original model has known spatial dimensions (``width``+``height``), target ``width``/``height`` can be omitted.
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:resize_2
|
||||
:fragment: ov:preprocess:resize_1
|
||||
|
||||
|
||||
When original model has known spatial dimensions (``width``+``height``), target ``width``/``height`` can be omitted.
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -266,6 +261,13 @@ When original model has known spatial dimensions (``width``+``height``), target
|
|||
:language: python
|
||||
:fragment: ov:preprocess:resize_2
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:resize_2
|
||||
|
||||
|
||||
C++ references:
|
||||
* `ov::preprocess::PreProcessSteps::resize()`
|
||||
|
|
@ -280,13 +282,6 @@ Typical use case is to reverse color channels from ``RGB`` to ``BGR`` and vice v
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_color_1
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -294,6 +289,13 @@ Typical use case is to reverse color channels from ``RGB`` to ``BGR`` and vice v
|
|||
:language: python
|
||||
:fragment: ov:preprocess:convert_color_1
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_color_1
|
||||
|
||||
|
||||
Color Conversion - NV12/I420
|
||||
----------------------------
|
||||
|
|
@ -304,13 +306,6 @@ In advanced cases, such YUV images can be split into separate planes, e.g., for
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_color_2
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -318,6 +313,13 @@ In advanced cases, such YUV images can be split into separate planes, e.g., for
|
|||
:language: python
|
||||
:fragment: ov:preprocess:convert_color_2
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:convert_color_2
|
||||
|
||||
|
||||
In this example, the original ``input`` is split to ``input/y`` and ``input/uv`` inputs. You can fill ``input/y`` from one source, and ``input/uv`` from another source. Color conversion to ``RGB`` will be performed, using these sources. It is more efficient as there will be no additional copies of NV12 buffers.
|
||||
|
||||
|
|
@ -342,13 +344,6 @@ If there is a need to insert additional operations to the execution graph right
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:custom
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -356,6 +351,13 @@ If there is a need to insert additional operations to the execution graph right
|
|||
:language: python
|
||||
:fragment: ov:preprocess:custom
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:custom
|
||||
|
||||
|
||||
C++ references:
|
||||
|
||||
|
|
@ -382,6 +384,13 @@ Usage of these operations is similar to pre-processing. See the following exampl
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.py
|
||||
:language: python
|
||||
:fragment: ov:preprocess:postprocess
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -389,12 +398,7 @@ Usage of these operations is similar to pre-processing. See the following exampl
|
|||
:language: cpp
|
||||
:fragment: ov:preprocess:postprocess
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.py
|
||||
:language: python
|
||||
:fragment: ov:preprocess:postprocess
|
||||
|
||||
|
||||
C++ references:
|
||||
|
|
|
|||
|
|
@ -60,13 +60,6 @@ The `ov::preprocess::PrePostProcessor <classov_1_1preprocess_1_1PrePostProcessor
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:create
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -74,6 +67,13 @@ The `ov::preprocess::PrePostProcessor <classov_1_1preprocess_1_1PrePostProcessor
|
|||
:language: python
|
||||
:fragment: ov:preprocess:create
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:create
|
||||
|
||||
|
||||
Declare User's Data Format
|
||||
++++++++++++++++++++++++++
|
||||
|
|
@ -83,13 +83,6 @@ To address particular input of a model/preprocessor, use the ``ov::preprocess::P
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:tensor
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -97,6 +90,13 @@ To address particular input of a model/preprocessor, use the ``ov::preprocess::P
|
|||
:language: python
|
||||
:fragment: ov:preprocess:tensor
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:tensor
|
||||
|
||||
|
||||
Below is all the specified input information:
|
||||
|
||||
|
|
@ -116,13 +116,6 @@ Model input already has information about precision and shape. Preprocessing API
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:model
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -130,6 +123,13 @@ Model input already has information about precision and shape. Preprocessing API
|
|||
:language: python
|
||||
:fragment: ov:preprocess:model
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:model
|
||||
|
||||
|
||||
Now, if the model input has ``{1,3,224,224}`` shape, preprocessing will be able to identify the ``height=224``, ``width=224``, and ``channels=3`` of that model. The ``height``/ ``width`` information is necessary for ``resize``, and ``channels`` is needed for mean/scale normalization.
|
||||
|
||||
|
|
@ -141,13 +141,6 @@ Now, the sequence of preprocessing steps can be defined:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:steps
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -155,6 +148,13 @@ Now, the sequence of preprocessing steps can be defined:
|
|||
:language: python
|
||||
:fragment: ov:preprocess:steps
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:steps
|
||||
|
||||
|
||||
Perform the following:
|
||||
|
||||
|
|
@ -173,13 +173,6 @@ Once the preprocessing steps have been finished the model can be finally built.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:build
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -187,6 +180,13 @@ Once the preprocessing steps have been finished the model can be finally built.
|
|||
:language: python
|
||||
:fragment: ov:preprocess:build
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:build
|
||||
|
||||
|
||||
The ``model`` will accept ``U8`` input with the shape of ``{1, 480, 640, 3}`` and the ``BGR`` channel order. All conversion steps will be integrated into the execution graph. Now, model can be loaded on the device and the image can be passed to the model without any data manipulation in the application.
|
||||
|
||||
|
|
|
|||
|
|
@ -43,13 +43,6 @@ in the model preparation script for such a case.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:save_headers
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -57,19 +50,19 @@ in the model preparation script for such a case.
|
|||
:language: Python
|
||||
:fragment: ov:preprocess:save_headers
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:save_headers
|
||||
|
||||
|
||||
* Preprocessing & Saving to the OpenVINO IR code.
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:save
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -77,6 +70,13 @@ in the model preparation script for such a case.
|
|||
:language: Python
|
||||
:fragment: ov:preprocess:save
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:save
|
||||
|
||||
|
||||
Application Code - Load Model to Target Device
|
||||
##############################################
|
||||
|
|
@ -88,13 +88,6 @@ time when the cached model is available.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:save_load
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -102,6 +95,13 @@ time when the cached model is available.
|
|||
:language: Python
|
||||
:fragment: ov:preprocess:save_load
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
|
||||
:language: cpp
|
||||
:fragment: ov:preprocess:save_load
|
||||
|
||||
|
||||
Additional Resources
|
||||
####################
|
||||
|
|
|
|||
|
|
@ -29,13 +29,6 @@ In order to use CPU for inference, the device name should be passed to the ``ov:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_default]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -43,6 +36,13 @@ In order to use CPU for inference, the device name should be passed to the ``ov:
|
|||
:language: py
|
||||
:fragment: [compile_model_default]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_default]
|
||||
|
||||
|
||||
Supported Inference Data Types
|
||||
###########################################################
|
||||
|
|
@ -109,13 +109,6 @@ to query ``ov::device::capabilities`` property, which should contain ``BF16`` in
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference0.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -123,6 +116,13 @@ to query ``ov::device::capabilities`` property, which should contain ``BF16`` in
|
|||
:language: py
|
||||
:fragment: [part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference0.cpp
|
||||
:language: cpp
|
||||
:fragment: [part0]
|
||||
|
||||
|
||||
Inference Precision Hint
|
||||
-----------------------------------------------------------
|
||||
|
|
@ -132,32 +132,25 @@ the ``ov::CompiledModel::get_property`` call. The code below demonstrates how to
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference1.cpp
|
||||
:language: cpp
|
||||
:fragment: [part1]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference.py
|
||||
:language: py
|
||||
:fragment: [part1]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference1.cpp
|
||||
:language: cpp
|
||||
:fragment: [part1]
|
||||
|
||||
To infer the model in ``f32`` precision instead of ``bf16`` on targets with native ``bf16`` support, set the ``ov::hint::inference_precision`` to ``ov::element::f32``.
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference2.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -165,6 +158,13 @@ To infer the model in ``f32`` precision instead of ``bf16`` on targets with nati
|
|||
:language: py
|
||||
:fragment: [part2]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/Bfloat16Inference2.cpp
|
||||
:language: cpp
|
||||
:fragment: [part2]
|
||||
|
||||
|
||||
The ``Bfloat16`` software simulation mode is available on CPUs with Intel® AVX-512 instruction set that do not support the
|
||||
native ``avx512_bf16`` instruction. This mode is used for development purposes and it does not guarantee good performance.
|
||||
|
|
@ -202,13 +202,6 @@ This can be achieved by specifying ``MULTI:CPU,GPU.0`` as a target device in cas
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_multi]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -216,6 +209,13 @@ This can be achieved by specifying ``MULTI:CPU,GPU.0`` as a target device in cas
|
|||
:language: py
|
||||
:fragment: [compile_model_multi]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_multi]
|
||||
|
||||
|
||||
For more details, see the :doc:`Multi-device execution <openvino_docs_OV_UG_Running_on_multiple_devices>` article.
|
||||
|
||||
|
|
@ -256,13 +256,6 @@ with the static input shape to get the best performance.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/dynamic_shape.cpp
|
||||
:language: cpp
|
||||
:fragment: [static_shape]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -270,6 +263,13 @@ with the static input shape to get the best performance.
|
|||
:language: py
|
||||
:fragment: [static_shape]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/dynamic_shape.cpp
|
||||
:language: cpp
|
||||
:fragment: [static_shape]
|
||||
|
||||
|
||||
For more details, see the :doc:`dynamic shapes guide <openvino_docs_OV_UG_DynamicShapes>`.
|
||||
|
||||
|
|
@ -388,6 +388,13 @@ User can use the following properties to limit available CPU resource for model
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/multi_threading.py
|
||||
:language: python
|
||||
:fragment: [ov:intel_cpu:multi_threading:part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -395,12 +402,6 @@ User can use the following properties to limit available CPU resource for model
|
|||
:language: cpp
|
||||
:fragment: [ov:intel_cpu:multi_threading:part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/multi_threading.py
|
||||
:language: python
|
||||
:fragment: [ov:intel_cpu:multi_threading:part0]
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -410,6 +411,13 @@ By default, OpenVINO Runtime will enable CPU threads pinning for better performa
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/multi_threading.py
|
||||
:language: python
|
||||
:fragment: [ov:intel_cpu:multi_threading:part1]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
|
@ -417,12 +425,6 @@ By default, OpenVINO Runtime will enable CPU threads pinning for better performa
|
|||
:language: cpp
|
||||
:fragment: [ov:intel_cpu:multi_threading:part1]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/multi_threading.py
|
||||
:language: python
|
||||
:fragment: [ov:intel_cpu:multi_threading:part1]
|
||||
|
||||
user can check the :doc:`optimization guide <openvino_docs_deployment_optimization_guide_tput_advanced>` for details on multi-stream execution
|
||||
|
||||
|
|
@ -460,13 +462,6 @@ To enable denormals optimization in the application, the ``denormals_optimizatio
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_denormals.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:intel_cpu:denormals_optimization:part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -474,6 +469,13 @@ To enable denormals optimization in the application, the ``denormals_optimizatio
|
|||
:language: python
|
||||
:fragment: [ov:intel_cpu:denormals_optimization:part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_denormals.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:intel_cpu:denormals_optimization:part0]
|
||||
|
||||
|
||||
Sparse weights decompression (Intel® x86-64)
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
|
@ -504,13 +506,6 @@ Code examples of how to use ``sparse_weights_decompression_rate``:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/ov_sparse_weights_decompression.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:intel_cpu:sparse_weights_decompression:part0]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -518,6 +513,13 @@ Code examples of how to use ``sparse_weights_decompression_rate``:
|
|||
:language: python
|
||||
:fragment: [ov:intel_cpu:sparse_weights_decompression:part0]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/cpu/ov_sparse_weights_decompression.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:intel_cpu:sparse_weights_decompression:part0]
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
|
|||
|
|
@ -85,17 +85,6 @@ Therefore, there is no need for explicitly switching between GNA and CPU.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/configure.cpp
|
||||
:language: cpp
|
||||
:fragment: [include]
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/configure.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_exec_mode_hw_with_sw_fback]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -107,6 +96,17 @@ Therefore, there is no need for explicitly switching between GNA and CPU.
|
|||
:language: py
|
||||
:fragment: [ov_gna_exec_mode_hw_with_sw_fback]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/configure.cpp
|
||||
:language: cpp
|
||||
:fragment: [include]
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/configure.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_exec_mode_hw_with_sw_fback]
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -163,13 +163,6 @@ To export a model for a specific version of GNA HW, use the ``ov::intel_gna::com
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/import_export.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_export]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -177,19 +170,19 @@ To export a model for a specific version of GNA HW, use the ``ov::intel_gna::com
|
|||
:language: py
|
||||
:fragment: [ov_gna_export]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/import_export.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_export]
|
||||
|
||||
|
||||
Import model:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/import_export.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_import]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -197,6 +190,13 @@ Import model:
|
|||
:language: py
|
||||
:fragment: [ov_gna_import]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/import_export.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_import]
|
||||
|
||||
|
||||
To compile a model, use either :ref:`compile Tool <openvino_ecosystem>` or
|
||||
:doc:`Speech C++ Sample <openvino_inference_engine_samples_speech_sample_README>`.
|
||||
|
|
@ -220,16 +220,16 @@ With the following methods, you can collect profiling information with various p
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
``ov::InferRequest::get_profiling_info``
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
``openvino.runtime.InferRequest.get_profiling_info``
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
``ov::InferRequest::get_profiling_info``
|
||||
|
||||
|
||||
The current GNA implementation calculates counters for the whole utterance scoring and does not provide per-layer information.
|
||||
The API enables you to retrieve counter units in cycles. You can convert cycles to seconds as follows:
|
||||
|
|
@ -368,17 +368,6 @@ To set the layout of model inputs in runtime, use the :doc:`Optimize Preprocessi
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/set_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: [include]
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/set_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_set_nc_layout]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -390,17 +379,21 @@ To set the layout of model inputs in runtime, use the :doc:`Optimize Preprocessi
|
|||
:language: py
|
||||
:fragment: [ov_gna_set_nc_layout]
|
||||
|
||||
|
||||
then set batch size:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/set_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_set_batch_size]
|
||||
:fragment: [include]
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/set_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_set_nc_layout]
|
||||
|
||||
|
||||
then set batch size:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
|
@ -409,6 +402,13 @@ then set batch size:
|
|||
:language: py
|
||||
:fragment: [ov_gna_set_batch_size]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gna/set_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov_gna_set_batch_size]
|
||||
|
||||
|
||||
Increasing batch size only improves efficiency of ``MatMul`` layers.
|
||||
|
||||
|
|
|
|||
|
|
@ -56,13 +56,6 @@ Then, the device name can be passed to the ``ov::Core::compile_model()`` method,
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_default_gpu
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -70,17 +63,17 @@ Then, the device name can be passed to the ``ov::Core::compile_model()`` method,
|
|||
:language: Python
|
||||
:fragment: compile_model_default_gpu
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_default_gpu
|
||||
|
||||
.. tab-item:: specific GPU
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_gpu_with_id
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -88,17 +81,17 @@ Then, the device name can be passed to the ``ov::Core::compile_model()`` method,
|
|||
:language: Python
|
||||
:fragment: compile_model_gpu_with_id
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_gpu_with_id
|
||||
|
||||
.. tab-item:: specific tile
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_gpu_with_id_and_tile
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -106,6 +99,13 @@ Then, the device name can be passed to the ``ov::Core::compile_model()`` method,
|
|||
:language: Python
|
||||
:fragment: compile_model_gpu_with_id_and_tile
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_gpu_with_id_and_tile
|
||||
|
||||
Supported Inference Data Types
|
||||
#######################################
|
||||
|
||||
|
|
@ -149,13 +149,6 @@ It is done by specifying ``MULTI:GPU.1,GPU.0`` as a target device.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_multi
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -163,6 +156,13 @@ It is done by specifying ``MULTI:GPU.1,GPU.0`` as a target device.
|
|||
:language: Python
|
||||
:fragment: compile_model_multi
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_multi
|
||||
|
||||
|
||||
For more details, see the :doc:`Multi-device execution <openvino_docs_OV_UG_Running_on_multiple_devices>`.
|
||||
|
||||
|
|
@ -180,13 +180,6 @@ Alternatively, it can be enabled explicitly via the device notion, for example `
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_batch_plugin
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -194,17 +187,17 @@ Alternatively, it can be enabled explicitly via the device notion, for example `
|
|||
:language: Python
|
||||
:fragment: compile_model_batch_plugin
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_batch_plugin
|
||||
|
||||
.. tab-item:: Batching via throughput hint
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_auto_batch
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -212,6 +205,13 @@ Alternatively, it can be enabled explicitly via the device notion, for example `
|
|||
:language: Python
|
||||
:fragment: compile_model_auto_batch
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/compile_model.cpp
|
||||
:language: cpp
|
||||
:fragment: compile_model_auto_batch
|
||||
|
||||
|
||||
For more details, see the :doc:`Automatic batching<openvino_docs_OV_UG_Automatic_Batching>`.
|
||||
|
||||
|
|
@ -272,13 +272,6 @@ The code snippet below demonstrates examples of a bounded dynamic batch:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/dynamic_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: dynamic_batch
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -286,6 +279,13 @@ The code snippet below demonstrates examples of a bounded dynamic batch:
|
|||
:language: Python
|
||||
:fragment: dynamic_batch
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/dynamic_batch.cpp
|
||||
:language: cpp
|
||||
:fragment: dynamic_batch
|
||||
|
||||
|
||||
Notes for performance and memory consumption in dynamic shapes
|
||||
--------------------------------------------------------------
|
||||
|
|
@ -348,13 +348,6 @@ The GPU plugin has the following additional preprocessing options:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/preprocessing_nv12_two_planes.cpp
|
||||
:language: cpp
|
||||
:fragment: init_preproc
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -362,6 +355,13 @@ The GPU plugin has the following additional preprocessing options:
|
|||
:language: Python
|
||||
:fragment: init_preproc
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/preprocessing_nv12_two_planes.cpp
|
||||
:language: cpp
|
||||
:fragment: init_preproc
|
||||
|
||||
|
||||
With such preprocessing, GPU plugin will expect ``ov::intel_gpu::ocl::ClImage2DTensor`` (or derived) to be passed for each NV12 plane via ``ov::InferRequest::set_tensor()`` or ``ov::InferRequest::set_tensors()`` methods.
|
||||
|
||||
|
|
|
|||
|
|
@ -51,32 +51,6 @@ of ``ov::RemoteContext`` derived classes.
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux/C++
|
||||
:sync: linux-cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Create from cl_context
|
||||
:sync: create-from-cl-context
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_cl_context]
|
||||
|
||||
.. tab-item:: Create from cl_queue
|
||||
:sync: create-from-cl-queue
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_cl_queue]
|
||||
|
||||
.. tab-item:: Create from VADisplay
|
||||
:sync: create-from-vadisplay
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_va_display]
|
||||
|
||||
.. tab-item:: Windows/C++
|
||||
:sync: windows-cpp
|
||||
|
||||
|
|
@ -102,6 +76,58 @@ of ``ov::RemoteContext`` derived classes.
|
|||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_d3d_device]
|
||||
|
||||
.. tab-item:: Windows/C
|
||||
:sync: windows-c
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Create from cl_context
|
||||
:sync: create-from-cl-context
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [context_from_cl_context]
|
||||
|
||||
.. tab-item:: Create from cl_queue
|
||||
:sync: create-from-cl-queue
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [context_from_cl_queue]
|
||||
|
||||
.. tab-item:: Create from ID3D11Device
|
||||
:sync: create-from-id3d11device
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [context_from_d3d_device]
|
||||
|
||||
.. tab-item:: Linux/C++
|
||||
:sync: linux-cpp
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Create from cl_context
|
||||
:sync: create-from-cl-context
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_cl_context]
|
||||
|
||||
.. tab-item:: Create from cl_queue
|
||||
:sync: create-from-cl-queue
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_cl_queue]
|
||||
|
||||
.. tab-item:: Create from VADisplay
|
||||
:sync: create-from-vadisplay
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [context_from_va_display]
|
||||
|
||||
.. tab-item:: Linux/C
|
||||
:sync: linux-c
|
||||
|
|
@ -129,32 +155,6 @@ of ``ov::RemoteContext`` derived classes.
|
|||
:language: c
|
||||
:fragment: [context_from_va_display]
|
||||
|
||||
.. tab-item:: Windows/C
|
||||
:sync: windows-c
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Create from cl_context
|
||||
:sync: create-from-cl-context
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [context_from_cl_context]
|
||||
|
||||
.. tab-item:: Create from cl_queue
|
||||
:sync: create-from-cl-queue
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [context_from_cl_queue]
|
||||
|
||||
.. tab-item:: Create from ID3D11Device
|
||||
:sync: create-from-id3d11device
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [context_from_d3d_device]
|
||||
|
||||
Getting RemoteContext from the Plugin
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
|
|
@ -173,14 +173,14 @@ To request the current default context of the plugin, use one of the following m
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Get context from Core
|
||||
:sync: context-core
|
||||
:sync: get-context-core
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
:fragment: [default_context_from_core]
|
||||
|
||||
.. tab-item:: Get context from compiled model
|
||||
:sync: context-compiled-model
|
||||
:sync: get-context-compiled-model
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
|
|
@ -192,14 +192,14 @@ To request the current default context of the plugin, use one of the following m
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Get context from Core
|
||||
:sync: context-core
|
||||
:sync: get-context-core
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
:fragment: [default_context_from_core]
|
||||
|
||||
.. tab-item:: Get context from compiled model
|
||||
:sync: context-compiled-model
|
||||
:sync: get-context-compiled-model
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
|
|
@ -255,7 +255,7 @@ For more details, see the code snippets below:
|
|||
:fragment: [wrap_cl_image]
|
||||
|
||||
.. tab-item:: biplanar NV12 surface
|
||||
:sync: biplanar
|
||||
:sync: biplanar-nv12-surface
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation.cpp
|
||||
:language: cpp
|
||||
|
|
@ -323,7 +323,7 @@ For more details, see the code snippets below:
|
|||
:fragment: [wrap_cl_image]
|
||||
|
||||
.. tab-item:: biplanar NV12 surface
|
||||
:sync: biplanar
|
||||
:sync: biplanar-nv12-surface
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/remote_objects_creation_c.cpp
|
||||
:language: c
|
||||
|
|
@ -370,7 +370,7 @@ should be added before model compilation:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: two-plane
|
||||
:sync: two_plane
|
||||
:sync: two-plane
|
||||
|
||||
.. tab-set::
|
||||
|
||||
|
|
@ -389,7 +389,7 @@ should be added before model compilation:
|
|||
:fragment: [init_preproc]
|
||||
|
||||
.. tab-item:: single-plane
|
||||
:sync: single_plane
|
||||
:sync: single-plane
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/preprocessing_nv12_single_plane.cpp
|
||||
:language: cpp
|
||||
|
|
@ -467,7 +467,7 @@ inputs need to be set via the ``ov::InferRequest::set_tensors`` method with vect
|
|||
:fragment: [batched_case]
|
||||
|
||||
.. tab-item:: NV12 to Grey
|
||||
:sync: single-plane
|
||||
:sync: nv12-grey
|
||||
|
||||
.. doxygensnippet:: docs/snippets/gpu/preprocessing_nv12_to_gray.cpp
|
||||
:language: cpp
|
||||
|
|
|
|||
|
|
@ -43,13 +43,6 @@ devices enabled by OpenVINO plugins and returns information, using the ``ov::Cor
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [get_available_devices]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -57,6 +50,13 @@ devices enabled by OpenVINO plugins and returns information, using the ``ov::Cor
|
|||
:language: py
|
||||
:fragment: [get_available_devices]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [get_available_devices]
|
||||
|
||||
|
||||
The function returns a list of available devices, for example:
|
||||
|
||||
|
|
@ -100,13 +100,6 @@ The code below demonstrates how to query ``HETERO`` device priority of devices w
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [hetero_priorities]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -114,6 +107,13 @@ The code below demonstrates how to query ``HETERO`` device priority of devices w
|
|||
:language: py
|
||||
:fragment: [hetero_priorities]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [hetero_priorities]
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -125,13 +125,6 @@ supported properties (``ov::supported_properties``), and others, use the ``ov::C
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [cpu_device_name]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -139,6 +132,13 @@ supported properties (``ov::supported_properties``), and others, use the ``ov::C
|
|||
:language: py
|
||||
:fragment: [cpu_device_name]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [cpu_device_name]
|
||||
|
||||
|
||||
A returned value appears as follows: ``Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz``.
|
||||
|
||||
|
|
@ -163,13 +163,6 @@ accept a selection of properties as last arguments. Each of the properties shoul
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_with_property]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -177,6 +170,13 @@ accept a selection of properties as last arguments. Each of the properties shoul
|
|||
:language: py
|
||||
:fragment: [compile_model_with_property]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model_with_property]
|
||||
|
||||
|
||||
The example below specifies hints that a model should be compiled to be inferred with multiple inference requests in parallel
|
||||
to achieve best throughput, while inference should be performed without accuracy loss with FP32 precision.
|
||||
|
|
@ -191,13 +191,6 @@ However, setting properties on a specific ``ov::Core::compile_model`` call appli
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [core_set_property_then_compile]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -205,6 +198,13 @@ However, setting properties on a specific ``ov::Core::compile_model`` call appli
|
|||
:language: py
|
||||
:fragment: [core_set_property_then_compile]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [core_set_property_then_compile]
|
||||
|
||||
|
||||
Properties on CompiledModel Level
|
||||
###########################################################
|
||||
|
|
@ -218,13 +218,6 @@ compiled model level property such as ``ov::optimal_number_of_infer_requests``:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [optimal_number_of_infer_requests]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -232,19 +225,19 @@ compiled model level property such as ``ov::optimal_number_of_infer_requests``:
|
|||
:language: py
|
||||
:fragment: [optimal_number_of_infer_requests]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [optimal_number_of_infer_requests]
|
||||
|
||||
|
||||
Or the number of threads that would be used for inference on ``CPU`` device:
|
||||
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [inference_num_threads]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -252,6 +245,13 @@ Or the number of threads that would be used for inference on ``CPU`` device:
|
|||
:language: py
|
||||
:fragment: [inference_num_threads]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [inference_num_threads]
|
||||
|
||||
|
||||
Setting Properties for Compiled Model
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
|
@ -261,13 +261,6 @@ The only mode that supports this method is [Multi-Device execution](../multi_dev
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [multi_device]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
|
@ -275,6 +268,13 @@ The only mode that supports this method is [Multi-Device execution](../multi_dev
|
|||
:language: py
|
||||
:fragment: [multi_device]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_properties_api.cpp
|
||||
:language: cpp
|
||||
:fragment: [multi_device]
|
||||
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
|
|
|||
|
|
@ -105,6 +105,13 @@ This guide used the following model to run the Image Classification Sample:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
omz_downloader --name googlenet-v1 --output_dir %USERPROFILE%\Documents\models
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -112,15 +119,8 @@ This guide used the following model to run the Image Classification Sample:
|
|||
|
||||
omz_downloader --name googlenet-v1 --output_dir ~/models
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
omz_downloader --name googlenet-v1 --output_dir %USERPROFILE%\Documents\models
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -131,6 +131,23 @@ This guide used the following model to run the Image Classification Sample:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
################|| Downloading models ||################
|
||||
|
||||
========== Downloading C:\Users\username\Documents\models\public\googlenet-v1\googlenet-v1.prototxt
|
||||
... 100%, 9 KB, ? KB/s, 0 seconds passed
|
||||
|
||||
========== Downloading C:\Users\username\Documents\models\public\googlenet-v1\googlenet-v1.caffemodel
|
||||
... 100%, 4834 KB, 571 KB/s, 8 seconds passed
|
||||
|
||||
################|| Post-processing ||################
|
||||
|
||||
========== Replacing text in C:\Users\username\Documents\models\public\googlenet-v1\googlenet-v1.prototxt
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -147,25 +164,8 @@ This guide used the following model to run the Image Classification Sample:
|
|||
|
||||
========= Replacing text in /home/username/models/public/googlenet-v1/googlenet-v1.prototxt =========
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
################|| Downloading models ||################
|
||||
|
||||
========== Downloading C:\Users\username\Documents\models\public\googlenet-v1\googlenet-v1.prototxt
|
||||
... 100%, 9 KB, ? KB/s, 0 seconds passed
|
||||
|
||||
========== Downloading C:\Users\username\Documents\models\public\googlenet-v1\googlenet-v1.caffemodel
|
||||
... 100%, 4834 KB, 571 KB/s, 8 seconds passed
|
||||
|
||||
################|| Post-processing ||################
|
||||
|
||||
========== Replacing text in C:\Users\username\Documents\models\public\googlenet-v1\googlenet-v1.prototxt
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -202,6 +202,13 @@ Create an ``<ir_dir>`` directory to contain the model's Intermediate Representat
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
mkdir %USERPROFILE%\Documents\ir
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -209,15 +216,8 @@ Create an ``<ir_dir>`` directory to contain the model's Intermediate Representat
|
|||
|
||||
mkdir ~/ir
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
mkdir %USERPROFILE%\Documents\ir
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -238,6 +238,13 @@ The command with most placeholders filled in and FP16 precision:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
mo --input_model %USERPROFILE%\Documents\models\public\googlenet-v1\googlenet-v1.caffemodel --compress_to_fp16 --output_dir %USERPROFILE%\Documents\ir
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -245,15 +252,8 @@ The command with most placeholders filled in and FP16 precision:
|
|||
|
||||
mo --input_model ~/models/public/googlenet-v1/googlenet-v1.caffemodel --compress_to_fp16 --output_dir ~/ir
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
mo --input_model %USERPROFILE%\Documents\models\public\googlenet-v1\googlenet-v1.caffemodel --compress_to_fp16 --output_dir %USERPROFILE%\Documents\ir
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -282,6 +282,13 @@ To run the **Image Classification** code sample with an input image using the IR
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
<INSTALL_DIR>\setupvars.bat
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -289,15 +296,8 @@ To run the **Image Classification** code sample with an input image using the IR
|
|||
|
||||
source <INSTALL_DIR>/setupvars.sh
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
<INSTALL_DIR>\setupvars.bat
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -307,6 +307,13 @@ To run the **Image Classification** code sample with an input image using the IR
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
cd %USERPROFILE%\Documents\Intel\OpenVINO\openvino_samples_build\intel64\Release
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -314,15 +321,8 @@ To run the **Image Classification** code sample with an input image using the IR
|
|||
|
||||
cd ~/openvino_cpp_samples_build/intel64/Release
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
cd %USERPROFILE%\Documents\Intel\OpenVINO\openvino_samples_build\intel64\Release
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -332,6 +332,13 @@ To run the **Image Classification** code sample with an input image using the IR
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
classification_sample_async.exe -i <path_to_media> -m <path_to_model> -d <target_device>
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -339,15 +346,8 @@ To run the **Image Classification** code sample with an input image using the IR
|
|||
|
||||
classification_sample_async -i <path_to_media> -m <path_to_model> -d <target_device>
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
classification_sample_async.exe -i <path_to_media> -m <path_to_model> -d <target_device>
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -363,6 +363,13 @@ The following command shows how to run the Image Classification Code Sample usin
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
.\classification_sample_async.exe -i %USERPROFILE%\Downloads\dog.bmp -m %USERPROFILE%\Documents\ir\googlenet-v1.xml -d CPU
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -370,15 +377,8 @@ The following command shows how to run the Image Classification Code Sample usin
|
|||
|
||||
./classification_sample_async -i ~/Downloads/dog.bmp -m ~/ir/googlenet-v1.xml -d CPU
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
.\classification_sample_async.exe -i %USERPROFILE%\Downloads\dog.bmp -m %USERPROFILE%\Documents\ir\googlenet-v1.xml -d CPU
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -418,13 +418,6 @@ Running Inference on GPU
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
./classification_sample_async -i ~/Downloads/dog.bmp -m ~/ir/googlenet-v1.xml -d GPU
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
|
@ -432,6 +425,13 @@ Running Inference on GPU
|
|||
|
||||
.\classification_sample_async.exe -i %USERPROFILE%\Downloads\dog.bmp -m %USERPROFILE%\Documents\ir\googlenet-v1.xml -d GPU
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
./classification_sample_async -i ~/Downloads/dog.bmp -m ~/ir/googlenet-v1.xml -d GPU
|
||||
|
||||
|
||||
Other Demos and Samples
|
||||
#######################
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ Depending on your operating system, there may be different methods to install th
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Ubuntu 22.04 LTS
|
||||
:sync: ubuntu22
|
||||
:sync: ubuntu-22
|
||||
|
||||
Download and install the `deb` packages published `here <https://github.com/intel/compute-runtime/releases/latest>`__ and install the apt package `ocl-icd-libopencl1` with the OpenCl ICD loader.
|
||||
|
||||
|
|
@ -42,7 +42,7 @@ Depending on your operating system, there may be different methods to install th
|
|||
apt-get install -y ocl-icd-libopencl1 intel-opencl-icd intel-level-zero-gpu level-zero
|
||||
|
||||
.. tab-item:: Ubuntu 20.04 LTS
|
||||
:sync: ubuntu20
|
||||
:sync: ubuntu-20
|
||||
|
||||
Ubuntu 20.04 LTS is not updated with the latest driver versions. You can install the updated versions up to the version 22.43 from apt:
|
||||
|
||||
|
|
@ -57,7 +57,7 @@ Depending on your operating system, there may be different methods to install th
|
|||
Alternatively, download older `deb` version from `here <https://github.com/intel/compute-runtime/releases>`__. Note that older driver version might not include some of the bug fixes and might be not supported on some latest platforms. Check the supported hardware for the versions you are installing.
|
||||
|
||||
.. tab-item:: RedHat UBI 8
|
||||
:sync: redhat8
|
||||
:sync: redhat-8
|
||||
|
||||
Follow the `guide <https://dgpu-docs.intel.com/driver/installation.html#rhel-install-steps>`__ to add Yum repository.
|
||||
|
||||
|
|
|
|||
|
|
@ -70,19 +70,19 @@ Create a virtual Python environment to avoid dependency conflicts. To create a v
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: lin-mac
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python3 -m venv openvino_env
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python -m venv openvino_env
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: linux-and-macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python3 -m venv openvino_env
|
||||
|
||||
|
||||
|
||||
|
|
@ -93,13 +93,6 @@ Activate the newly created Python virtual environment by issuing this command:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: lin-mac
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
source openvino_env/bin/activate
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
|
@ -107,6 +100,13 @@ Activate the newly created Python virtual environment by issuing this command:
|
|||
|
||||
openvino_env\Scripts\activate
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: linux-and-macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
source openvino_env/bin/activate
|
||||
|
||||
.. important::
|
||||
|
||||
The above command must be re-run every time a new command terminal window is opened.
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ Prerequisites
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`__
|
||||
|
|
@ -47,7 +47,7 @@ Prerequisites
|
|||
See `Product Specifications <https://ark.intel.com/>`__ for information about your processor.
|
||||
|
||||
.. tab-item:: Software Requirements
|
||||
:sync: soft-req
|
||||
:sync: software-requirements
|
||||
|
||||
* `CMake 3.13 or higher, 64-bit <https://cmake.org/download/>`__
|
||||
* GCC 7.5.0 (for Ubuntu 18.04), GCC 9.3.0 (for Ubuntu 20.04) or GCC 11.3.0 (for Ubuntu 22.04)
|
||||
|
|
@ -91,21 +91,21 @@ Step 1: Set Up the OpenVINO Toolkit APT Repository
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Ubuntu 22
|
||||
:sync: ubuntu22
|
||||
:sync: ubuntu-22
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
echo "deb https://apt.repos.intel.com/openvino/2023 ubuntu22 main" | sudo tee /etc/apt/sources.list.d/intel-openvino-2023.list
|
||||
|
||||
.. tab-item:: Ubuntu 20
|
||||
:sync: ubuntu20
|
||||
:sync: ubuntu-20
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
echo "deb https://apt.repos.intel.com/openvino/2023 ubuntu20 main" | sudo tee /etc/apt/sources.list.d/intel-openvino-2023.list
|
||||
|
||||
.. tab-item:: Ubuntu 18
|
||||
:sync: ubuntu18
|
||||
:sync: ubuntu-18
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -136,7 +136,7 @@ Install OpenVINO Runtime
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: The Latest Version
|
||||
:sync: latest
|
||||
:sync: latest-version
|
||||
|
||||
Run the following command:
|
||||
|
||||
|
|
|
|||
|
|
@ -27,25 +27,16 @@ See `Installing Additional Components <#optional-installing-additional-component
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`__
|
||||
|
||||
.. tab-item:: Software Requirements
|
||||
:sync: soft-req
|
||||
:sync: software-requirements
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
|
||||
* `Homebrew <https://brew.sh/>`_
|
||||
* `CMake 3.13 or higher <https://cmake.org/download/>`__ (choose "macOS 10.13 or later"). Add ``/Applications/CMake.app/Contents/bin`` to path (for default installation).
|
||||
* `Python 3.7 - 3.11 <https://www.python.org/downloads/mac-osx/>`__ . Install and add it to path.
|
||||
* Apple Xcode Command Line Tools. In the terminal, run ``xcode-select --install`` from any directory to install it.
|
||||
* (Optional) Apple Xcode IDE (not required for OpenVINO™, but useful for development)
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -53,7 +44,16 @@ See `Installing Additional Components <#optional-installing-additional-component
|
|||
* `CMake 3.13 or higher, 64-bit <https://cmake.org/download/>`__
|
||||
* GCC 7.5.0 (for Ubuntu 18.04), GCC 9.3.0 (for Ubuntu 20.04) or GCC 11.3.0 (for Ubuntu 22.04)
|
||||
* `Python 3.7 - 3.10, 64-bit <https://www.python.org/downloads/>`__
|
||||
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: macos
|
||||
|
||||
* `Homebrew <https://brew.sh/>`_
|
||||
* `CMake 3.13 or higher <https://cmake.org/download/>`__ (choose "macOS 10.13 or later"). Add ``/Applications/CMake.app/Contents/bin`` to path (for default installation).
|
||||
* `Python 3.7 - 3.11 <https://www.python.org/downloads/mac-osx/>`__ . Install and add it to path.
|
||||
* Apple Xcode Command Line Tools. In the terminal, run ``xcode-select --install`` from any directory to install it.
|
||||
* (Optional) Apple Xcode IDE (not required for OpenVINO™, but useful for development)
|
||||
|
||||
|
||||
|
||||
Installing OpenVINO Runtime
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`__
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ See the `Release Notes <https://www.intel.com/content/www/us/en/developer/articl
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
Full requirement listing is available in `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`__
|
||||
|
||||
|
|
@ -35,7 +35,7 @@ See the `Release Notes <https://www.intel.com/content/www/us/en/developer/articl
|
|||
Processor graphics are not included in all processors. See `Product Specifications <https://ark.intel.com/>`__ for information about your processor.
|
||||
|
||||
.. tab-item:: Software
|
||||
:sync: soft
|
||||
:sync: software
|
||||
|
||||
* `CMake 3.13 or higher, 64-bit <https://cmake.org/download/>`__
|
||||
* `Python 3.7 - 3.11, 64-bit <https://www.python.org/downloads/>`__
|
||||
|
|
@ -43,23 +43,23 @@ See the `Release Notes <https://www.intel.com/content/www/us/en/developer/articl
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Ubuntu 20.04
|
||||
:sync: ubuntu-20
|
||||
|
||||
* GCC 9.3.0
|
||||
|
||||
.. tab-item:: Ubuntu 18.04
|
||||
:sync: ubuntu18
|
||||
:sync: ubuntu-18
|
||||
|
||||
* GCC 7.5.0
|
||||
|
||||
.. tab-item:: Ubuntu 20.04
|
||||
:sync: ubuntu20
|
||||
|
||||
* GCC 9.3.0
|
||||
|
||||
.. tab-item:: RHEL 8
|
||||
:sync: rhel8
|
||||
:sync: rhel-8
|
||||
|
||||
* GCC 8.4.1
|
||||
|
||||
.. tab-item:: CentOS 7
|
||||
:sync: centos7
|
||||
:sync: centos-7
|
||||
|
||||
* GCC 8.3.1
|
||||
Use the following instructions to install it:
|
||||
|
|
@ -114,7 +114,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Ubuntu 22.04
|
||||
:sync: ubuntu22
|
||||
:sync: ubuntu-22
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -123,7 +123,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
sudo mv l_openvino_toolkit_ubuntu22_2023.0.1.11005.fa1c41994f3_x86_64 /opt/intel/openvino_2023.0.1
|
||||
|
||||
.. tab-item:: Ubuntu 20.04
|
||||
:sync: ubuntu20
|
||||
:sync: ubuntu-20
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -132,7 +132,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
sudo mv l_openvino_toolkit_ubuntu20_2023.0.1.11005.fa1c41994f3_x86_64 /opt/intel/openvino_2023.0.1
|
||||
|
||||
.. tab-item:: Ubuntu 18.04
|
||||
:sync: ubuntu18
|
||||
:sync: ubuntu-18
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -141,7 +141,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
sudo mv l_openvino_toolkit_ubuntu18_2023.0.1.11005.fa1c41994f3_x86_64 /opt/intel/openvino_2023.0.1
|
||||
|
||||
.. tab-item:: RHEL 8
|
||||
:sync: rhel8
|
||||
:sync: rhel-8
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -150,7 +150,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
sudo mv l_openvino_toolkit_rhel8_2023.0.1.11005.fa1c41994f3_x86_64 /opt/intel/openvino_2023.0.1
|
||||
|
||||
.. tab-item:: CentOS 7
|
||||
:sync: centos7
|
||||
:sync: centos-7
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -20,13 +20,13 @@ See the `Release Notes <https://www.intel.com/content/www/us/en/developer/articl
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`__
|
||||
|
||||
.. tab-item:: Software Requirements
|
||||
:sync: soft-req
|
||||
:sync: software-requirements
|
||||
|
||||
* `CMake 3.13 or higher <https://cmake.org/download/>`__ (choose "macOS 10.13 or later"). Add ``/Applications/CMake.app/Contents/bin`` to path (for default install).
|
||||
* `Python 3.7 - 3.11 <https://www.python.org/downloads/mac-osx/>`__ (choose 3.7 - 3.11). Install and add to path.
|
||||
|
|
@ -66,7 +66,7 @@ Step 1: Install OpenVINO Core Components
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: x86, 64-bit
|
||||
:sync: x86-64bit
|
||||
:sync: x86-64
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -75,7 +75,7 @@ Step 1: Install OpenVINO Core Components
|
|||
sudo mv m_openvino_toolkit_macos_10_15_2023.0.1.11005.fa1c41994f3_x86_64 /opt/intel/openvino_2023.0.1
|
||||
|
||||
.. tab-item:: ARM, 64-bit
|
||||
:sync: arm-64bit
|
||||
:sync: arm-64
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ System Requirements
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys_req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`_
|
||||
|
|
|
|||
|
|
@ -30,20 +30,20 @@ To create a virtual environment, use the following command:
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: lin-mac
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python3 -m venv openvino_env
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: win
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python -m venv openvino_env
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: linux-and-macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python3 -m venv openvino_env
|
||||
|
||||
|
||||
Step 2. Activate Virtual Environment
|
||||
++++++++++++++++++++++++++++++++++++
|
||||
|
|
@ -51,20 +51,20 @@ Step 2. Activate Virtual Environment
|
|||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: lin-mac
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
source openvino_env/bin/activate
|
||||
|
||||
.. tab-item:: Windows
|
||||
:sync: win
|
||||
:sync: windows
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
openvino_env\Scripts\activate
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: linux-and-macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
source openvino_env/bin/activate
|
||||
|
||||
|
||||
.. important::
|
||||
|
||||
|
|
|
|||
|
|
@ -14,13 +14,13 @@ Development and Target Systems
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`_
|
||||
|
||||
.. tab-item:: Software Requirements
|
||||
:sync: soft-req
|
||||
:sync: software-requirements
|
||||
|
||||
* CMake 3.10 or higher
|
||||
* Python 3.7 - 3.11
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ Prerequisites
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: System Requirements
|
||||
:sync: sys-req
|
||||
:sync: system-requirements
|
||||
|
||||
| Full requirement listing is available in:
|
||||
| `System Requirements Page <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html>`__
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ Check the versions of OpenVINO Runtime and Development Tools
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: mo-python-api
|
||||
:sync: py
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
@ -26,7 +26,7 @@ Check the versions of OpenVINO Runtime and Development Tools
|
|||
ov_model = convert_model(version=True)
|
||||
|
||||
.. tab-item:: CLI
|
||||
:sync: cli-tool
|
||||
:sync: cli
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
|
|||
|
|
@ -32,10 +32,10 @@ If you have installed OpenVINO Runtime from archive files, you can uninstall it
|
|||
|
||||
rmdir /s <extracted_folder>
|
||||
del <path_to_archive>
|
||||
|
||||
|
||||
|
||||
|
||||
.. tab-item:: Linux & macOS
|
||||
:sync: lin-mac
|
||||
:sync: linux-and-macos
|
||||
|
||||
If you have created the symbolic link, remove the link first:
|
||||
|
||||
|
|
@ -49,6 +49,5 @@ If you have installed OpenVINO Runtime from archive files, you can uninstall it
|
|||
|
||||
rm -r <extracted_folder> && rm <path_to_archive>
|
||||
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
|
|
|||
|
|
@ -126,7 +126,7 @@ operating system or environment.
|
|||
sudo yum install python36-devel mesa-libGL
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
Alternatively, you may skip steps 1-3 if you prefer to manually install `Python 3 <https://www.python.org/>`__ and `Git <https://git-scm.com/>`__.
|
||||
|
||||
|
|
@ -158,7 +158,7 @@ operating system or environment.
|
|||
If OpenVINO is installed globally, do not run any of these commands in a terminal where ``setupvars.sh`` is sourced.
|
||||
|
||||
.. tab-item:: Azure ML
|
||||
:sync: azure
|
||||
:sync: azure-ml
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -302,7 +302,6 @@ operating system or environment.
|
|||
|
||||
pip install -r requirements.txt
|
||||
|
||||
|
||||
.. tab-item:: Linux
|
||||
:sync: linux
|
||||
|
||||
|
|
@ -341,10 +340,9 @@ operating system or environment.
|
|||
.. code-block:: sh
|
||||
|
||||
pip install -r requirements.txt
|
||||
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
1. **Create a Virtual Environment**
|
||||
|
||||
|
|
@ -383,7 +381,7 @@ operating system or environment.
|
|||
|
||||
|
||||
.. tab-item:: Azure ML
|
||||
:sync: azure
|
||||
:sync: azure-ml
|
||||
|
||||
1. Create a Conda environment
|
||||
|
||||
|
|
@ -564,7 +562,7 @@ To reactivate your environment, run:
|
|||
source openvino_env/bin/activate
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code:: bash
|
||||
|
||||
|
|
@ -595,7 +593,7 @@ This operation is optional. However, if you want to remove your virtual environm
|
|||
rm -rf openvino_env
|
||||
|
||||
.. tab-item:: macOS
|
||||
:sync: mac
|
||||
:sync: macos
|
||||
|
||||
.. code:: bash
|
||||
|
||||
|
|
|
|||
|
|
@ -37,6 +37,13 @@ In general, most throughput-oriented inference applications should:
|
|||
* Setup the configuration for the *device* (for example, as parameters of the ``ov::Core::compile_model``) via either previously introduced :doc:`low-level explicit options <openvino_docs_deployment_optimization_guide_tput_advanced>` or :doc:`OpenVINO performance hints <openvino_docs_OV_UG_Performance_Hints>` (**preferable**):
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.py
|
||||
:language: python
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
|
@ -44,14 +51,6 @@ In general, most throughput-oriented inference applications should:
|
|||
.. doxygensnippet:: docs/snippets/ov_auto_batching.cpp
|
||||
:language: cpp
|
||||
:fragment: [compile_model]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_auto_batching.py
|
||||
:language: python
|
||||
:fragment: [compile_model]
|
||||
|
||||
|
||||
* Query the ``ov::optimal_number_of_infer_requests`` from the ``ov::CompiledModel`` (resulted from a compilation of the model for the device) to create the number of the requests required to saturate the device.
|
||||
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ In this step, NNCF-related imports are added in the beginning of the training sc
|
|||
:fragment: [imports]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -69,7 +69,7 @@ of optimization methods (`"compression"` section).
|
|||
:fragment: [nncf_congig]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -104,7 +104,7 @@ optimization.
|
|||
:fragment: [wrap_model]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -127,7 +127,7 @@ of the original model.
|
|||
:fragment: [tune_model]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -150,7 +150,7 @@ fine-tuning that will inform optimization methods to do some adjustments to func
|
|||
:fragment: [distributed]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -172,7 +172,7 @@ the case of PyTorch and frozen graph - for TensorFlow 2.
|
|||
:fragment: [export]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -198,7 +198,7 @@ To save model checkpoint use the following API:
|
|||
:fragment: [save_checkpoint]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
@ -220,7 +220,7 @@ To restore the model from checkpoint you should use the following API:
|
|||
:fragment: [load_checkpoint]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/pruning_tf.py
|
||||
:language: python
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ The transformation function is a function that takes a sample from the dataset a
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_openvino.py
|
||||
:language: python
|
||||
|
|
@ -57,7 +57,7 @@ The transformation function is a function that takes a sample from the dataset a
|
|||
:fragment: [dataset]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tf
|
||||
:sync: tensorflow
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_tensorflow.py
|
||||
:language: python
|
||||
|
|
@ -74,7 +74,7 @@ Once the dataset is ready and the model object is instantiated, you can apply 8-
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_openvino.py
|
||||
:language: python
|
||||
|
|
@ -95,7 +95,7 @@ Once the dataset is ready and the model object is instantiated, you can apply 8-
|
|||
:fragment: [quantization]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tf
|
||||
:sync: tensorflow
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_tensorflow.py
|
||||
:language: python
|
||||
|
|
@ -107,7 +107,7 @@ After that the model can be converted into the OpenVINO Intermediate Representat
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_openvino.py
|
||||
:language: python
|
||||
|
|
@ -128,7 +128,7 @@ After that the model can be converted into the OpenVINO Intermediate Representat
|
|||
:fragment: [inference]
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
:sync: tf
|
||||
:sync: tensorflow
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_tensorflow.py
|
||||
:language: python
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ This step is similar to the :doc:`Basic 8-bit quantization <basic_quantization_f
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py
|
||||
:language: python
|
||||
|
|
@ -38,7 +38,7 @@ Validation funtion receives ``openvino.runtime.CompiledModel`` object and valida
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py
|
||||
:language: python
|
||||
|
|
@ -52,7 +52,7 @@ Run quantization with accuracy control
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py
|
||||
:language: python
|
||||
|
|
@ -67,7 +67,7 @@ After that the model can be compiled and run with OpenVINO:
|
|||
.. tab-set::
|
||||
|
||||
.. tab-item:: OpenVINO
|
||||
:sync: ov
|
||||
:sync: openvino
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py
|
||||
:language: python
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ In this step, you add NNCF-related imports in the beginning of the training scri
|
|||
:fragment: [imports]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -59,7 +59,7 @@ specific to the DL framework. It will be used at the model creation step to init
|
|||
:fragment: [nncf_congig]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -86,7 +86,7 @@ will be shown below.
|
|||
:fragment: [wrap_model]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -110,7 +110,7 @@ you can skip this step which means that the post-training optimization will be a
|
|||
:fragment: [tune_model]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -134,7 +134,7 @@ the fine-tuning that will inform optimization methods to do some adjustments to
|
|||
:fragment: [distributed]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -156,7 +156,7 @@ the case of PyTorch and frozen graph - for TensorFlow 2.
|
|||
:fragment: [export]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -186,7 +186,7 @@ To save model checkpoint use the following API:
|
|||
:fragment: [save_checkpoint]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
@ -208,7 +208,7 @@ To restore the model from checkpoint you should use the following API:
|
|||
:fragment: [load_checkpoint]
|
||||
|
||||
.. tab-item:: TensorFlow 2
|
||||
:sync: tf2
|
||||
:sync: tensorflow-2
|
||||
|
||||
.. doxygensnippet:: docs/optimization_guide/nncf/code/qat_tf.py
|
||||
:language: python
|
||||
|
|
|
|||
|
|
@ -254,400 +254,6 @@ This page lists operations supported by OpenVINO Framework Frontend.
|
|||
torchvision::roi_align
|
||||
========================================== ==========================================================================================
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
|
||||
========================================== ==========================================================================================
|
||||
TensorFlow Supported Operations Limitations
|
||||
========================================== ==========================================================================================
|
||||
Abs
|
||||
Acosh
|
||||
Add
|
||||
AddV2
|
||||
AddN
|
||||
All
|
||||
Any
|
||||
ArgMax
|
||||
ArgMin
|
||||
Asinh
|
||||
Assert Not needed for inference.
|
||||
Assign Not needed for inference.
|
||||
AssignSub Not needed for inference.
|
||||
Atanh
|
||||
AvgPool
|
||||
AvgPoolV2 Supported only for constant-foldable ``kernel_size`` and strides inputs.
|
||||
AvgPool3D
|
||||
BatchMatMul
|
||||
BatchMatMulV2
|
||||
BatchToSpaceND
|
||||
BiasAdd
|
||||
BlockLSTM
|
||||
Bucketize CPU only.
|
||||
BroadcastTo
|
||||
Cast
|
||||
Ceil
|
||||
ClipByValue
|
||||
Concat
|
||||
ConcatV2
|
||||
Const
|
||||
Conv2D
|
||||
Conv2DBackpropInput
|
||||
Conv3D
|
||||
Conv3DBackpropInputV2
|
||||
Cos
|
||||
Cosh
|
||||
CropAndResize ``method`` = ``bilinear`` only.
|
||||
CTCGreedyDecoder Supported only with decoded indices output in a dense format.
|
||||
CTCLoss Supported only with decoded indices input in a dense format.
|
||||
CumSum
|
||||
DepthToSpace
|
||||
DepthwiseConv2dNative
|
||||
Einsum Supported only with equation that does not contain repeated labels within a subscript.
|
||||
Elu
|
||||
EmptyTensorList Supported only when it is part of a sub-graph of the special form.
|
||||
Enter Supported only when it is fused to the TensorIterator layer.
|
||||
Equal
|
||||
Erf
|
||||
Exit Supported only when it is fused to the TensorIterator layer.
|
||||
Exp
|
||||
ExpandDims
|
||||
ExperimentalSparseWeightedSum CPU only.
|
||||
ExtractImagePatches
|
||||
EuclideanNorm
|
||||
FakeQuantWithMinMaxVars
|
||||
FakeQuantWithMinMaxVarsPerChannel
|
||||
FFT Supported only when it is part of a sub-graph of the special form.
|
||||
FFT2D Supported only when it is part of a sub-graph of the special form.
|
||||
FFT3D Supported only when it is part of a sub-graph of the special form.
|
||||
FIFOQueueV2 Supported only when it is part of a sub-graph of the special form.
|
||||
Fill
|
||||
Floor
|
||||
FloorDiv
|
||||
FloorMod
|
||||
FusedBatchNorm
|
||||
FusedBatchNormV2
|
||||
FusedBatchNormV3
|
||||
Gather
|
||||
GatherNd
|
||||
GatherTree
|
||||
GatherV2
|
||||
Greater
|
||||
GreaterEqual
|
||||
Identity Not needed for shape inference.
|
||||
IdentityN
|
||||
IFFT Supported only when it is part of a sub-graph of the special form.
|
||||
IFFT2D Supported only when it is part of a sub-graph of the special form.
|
||||
IFFT3D Supported only when it is part of a sub-graph of the special form.
|
||||
IteratorGetNext Supported only when it is part of a sub-graph of the special form.
|
||||
LRN
|
||||
LeakyRelu
|
||||
Less
|
||||
LessEqual
|
||||
Log
|
||||
Log1p
|
||||
LogicalAnd
|
||||
LogicalOr
|
||||
LogicalNot
|
||||
LogSoftmax
|
||||
LookupTableInsertV2 Supported only when it is part of a sub-graph of the special form.
|
||||
LoopCond Supported only when it is fused to the TensorIterator layer.
|
||||
MatMul
|
||||
Max
|
||||
MaxPool
|
||||
MaxPoolV2 Supported only for constant-foldable ``kernel_size`` and strides inputs.
|
||||
MaxPool3D
|
||||
Maximum
|
||||
Mean
|
||||
Merge Supported only when it is fused to the TensorIterator layer.
|
||||
Min
|
||||
Minimum
|
||||
MirrorPad
|
||||
Mod
|
||||
Mul
|
||||
Neg
|
||||
NextIteration Supported only when it is fused to the TensorIterator layer.
|
||||
NonMaxSuppressionV2
|
||||
NonMaxSuppressionV3
|
||||
NonMaxSuppressionV4
|
||||
NonMaxSuppressionV5
|
||||
NotEqual
|
||||
NoOp
|
||||
OneHot
|
||||
Pack
|
||||
Pad
|
||||
PadV2
|
||||
Placeholder
|
||||
PlaceholderWithDefault
|
||||
Prod
|
||||
QueueDequeue Supported only when it is part of a sub-graph of the special form.
|
||||
QueueDequeueUpToV2 Supported only when it is part of a sub-graph of the special form.
|
||||
QueueDequeueV2 Supported only when it is part of a sub-graph of the special form.
|
||||
RandomUniform
|
||||
RandomUniformInt
|
||||
Range
|
||||
Rank
|
||||
RealDiv
|
||||
Reciprocal
|
||||
Relu
|
||||
Relu6
|
||||
Reshape
|
||||
ResizeBilinear
|
||||
ResizeNearestNeighbor
|
||||
ResourceGather
|
||||
ReverseSequence
|
||||
ReverseV2 Supported only when it can be converted to the ReverseSequence operation.
|
||||
Roll
|
||||
Round
|
||||
Pow
|
||||
Rsqrt
|
||||
ScatterNd
|
||||
Select
|
||||
SelectV2
|
||||
Shape
|
||||
Sigmoid
|
||||
Sin
|
||||
Sinh
|
||||
Size
|
||||
Slice
|
||||
Softmax
|
||||
Softplus
|
||||
Softsign
|
||||
SpaceToBatchND
|
||||
SpaceToDepth
|
||||
SparseFillEmptyRows Supported only when it is part of a sub-graph of the special form.
|
||||
SparseReshape Supported only when it is part of a sub-graph of the special form.
|
||||
SparseSegmentSum Supported only when it is part of a sub-graph of the special form.
|
||||
SparseSegmentMean Supported only when it is part of a sub-graph of the special form.
|
||||
SparseToDense CPU only
|
||||
Split
|
||||
SplitV
|
||||
Sqrt
|
||||
Square
|
||||
SquaredDifference
|
||||
Square
|
||||
Squeeze Cases in which squeeze axis is not specified are not supported.
|
||||
StatelessWhile
|
||||
StopGradient Not needed for shape inference.
|
||||
StridedSlice Supported only for constant-foldable ``begin``, ``end``, and ``strides`` inputs.
|
||||
Sub
|
||||
Sum
|
||||
Swish
|
||||
swish_f32
|
||||
Switch Control flow propagation.
|
||||
Tan
|
||||
Tanh
|
||||
TensorArrayGatherV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayReadV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayScatterV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArraySizeV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayWriteV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorListPushBack Supported only when it is part of a sub-graph of the special form.
|
||||
Tile
|
||||
TopkV2
|
||||
Transpose
|
||||
Unpack
|
||||
Variable
|
||||
VariableV2
|
||||
Where Supported only when it is part of a sub-graph of the special form.
|
||||
ZerosLike
|
||||
========================================== ==========================================================================================
|
||||
|
||||
.. tab-item:: TensorFlow2 Keras
|
||||
|
||||
========================================== ==========================================================================================
|
||||
TensorFlow 2 Keras Supported Operations Limitations
|
||||
========================================== ==========================================================================================
|
||||
ActivityRegularization
|
||||
Add
|
||||
AdditiveAttention
|
||||
AlphaDropout
|
||||
Attention
|
||||
Average
|
||||
AveragePooling1D
|
||||
AveragePooling2D
|
||||
AveragePooling3D
|
||||
BatchNormalization
|
||||
Bidirectional
|
||||
Concatenate
|
||||
Conv1D
|
||||
Conv1DTranspose Not supported if ``dilation`` is not equal to 1.
|
||||
Conv2D
|
||||
Conv2DTranspose
|
||||
Conv3D
|
||||
Conv3DTranspose
|
||||
Cropping1D
|
||||
Cropping2D
|
||||
Cropping3D
|
||||
Dense
|
||||
DenseFeatures Not supported for categorical and crossed features.
|
||||
DepthwiseConv2D
|
||||
Dot
|
||||
Dropout
|
||||
ELU
|
||||
Embedding
|
||||
Flatten
|
||||
GRU
|
||||
GRUCell
|
||||
GaussianDropout
|
||||
GaussianNoise
|
||||
GlobalAveragePooling1D
|
||||
GlobalAveragePooling2D
|
||||
GlobalAveragePooling3D
|
||||
GlobalMaxPool1D
|
||||
GlobalMaxPool2D
|
||||
GlobalMaxPool3D
|
||||
LSTM
|
||||
LSTMCell
|
||||
Lambda
|
||||
LayerNormalization
|
||||
LeakyReLU
|
||||
LocallyConnected1D
|
||||
LocallyConnected2D
|
||||
MaxPool1D
|
||||
MaxPool2D
|
||||
MaxPool3D
|
||||
Maximum
|
||||
Minimum
|
||||
Multiply
|
||||
PReLU
|
||||
Permute
|
||||
RNN Not supported for some custom cells.
|
||||
ReLU
|
||||
RepeatVector
|
||||
Reshape
|
||||
Roll
|
||||
SeparableConv1D
|
||||
SeparableConv2D
|
||||
SimpleRNN
|
||||
SimpleRNNCell
|
||||
Softmax
|
||||
SpatialDropout1D
|
||||
SpatialDropout2D
|
||||
SpatialDropout3D
|
||||
StackedRNNCells
|
||||
Subtract
|
||||
ThresholdedReLU
|
||||
TimeDistributed
|
||||
UpSampling1D
|
||||
UpSampling2D
|
||||
UpSampling3D
|
||||
ZeroPadding1D
|
||||
ZeroPadding2D
|
||||
ZeroPadding3D
|
||||
========================================== ==========================================================================================
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
|
||||
========================================== ===============================================================================
|
||||
TensorFlow Lite Supported Operations Limitations
|
||||
========================================== ===============================================================================
|
||||
ABS
|
||||
ADD
|
||||
ADD_N
|
||||
ARG_MAX
|
||||
ARG_MIN
|
||||
AVERAGE_POOL_2D
|
||||
BATCH_MATMUL
|
||||
BATCH_TO_SPACE_ND
|
||||
BROADCAST_ARGS
|
||||
BROADCAST_TO
|
||||
CAST
|
||||
CEIL
|
||||
COMPLEX_ABS Supported in a specific pattern with RFFT2D
|
||||
CONCATENATION
|
||||
CONV_2D
|
||||
COS
|
||||
DEPTH_TO_SPACE
|
||||
DEPTHWISE_CONV_2D
|
||||
DEQUANTIZE
|
||||
DIV
|
||||
ELU
|
||||
EQUAL
|
||||
EXP
|
||||
EXPAND_DIMS
|
||||
FILL
|
||||
FLOOR
|
||||
FLOOR_DIV
|
||||
FLOOR_MOD
|
||||
FULLY_CONNECTED
|
||||
GATHER
|
||||
GATHER_ND
|
||||
GREATER
|
||||
GREATER_EQUAL
|
||||
HARD_SWISH
|
||||
L2_NORMALIZATION
|
||||
LEAKY_RELU
|
||||
LESS
|
||||
LESS_EQUAL
|
||||
LOG
|
||||
LOG_SOFTMAX
|
||||
LOGICAL_AND
|
||||
LOGICAL_NOT
|
||||
LOGICAL_OR
|
||||
LOGISTIC
|
||||
MATRIX_DIAG
|
||||
MAX_POOL_2D
|
||||
MAXIMUM
|
||||
MEAN
|
||||
MINIMUM
|
||||
MIRROR_PAD
|
||||
MUL
|
||||
NEG
|
||||
NOT_EQUAL
|
||||
ONE_HOT
|
||||
PACK
|
||||
PAD
|
||||
PADV2
|
||||
POW
|
||||
PRELU
|
||||
QUANTIZE
|
||||
RANGE
|
||||
RANK
|
||||
REDUCE_ALL
|
||||
REDUCE_ANY
|
||||
REDUCE_MAX
|
||||
REDUCE_MIN
|
||||
REDUCE_PROD
|
||||
RELU
|
||||
RELU6
|
||||
RESHAPE
|
||||
RESIZE_BILINEAR
|
||||
RESIZE_NEAREST_NEIGHBOR
|
||||
REVERSE_V2
|
||||
RFFT2D Supported in a specific pattern with COMPLEX_ABS
|
||||
ROUND
|
||||
RSQRT
|
||||
SCATTER_ND
|
||||
SEGMENT_SUM
|
||||
SELECT
|
||||
SELECT_V2
|
||||
SHAPE
|
||||
SIGN
|
||||
SIN
|
||||
SLICE
|
||||
SOFTMAX
|
||||
SPACE_TO_BATCH_ND
|
||||
SPACE_TO_DEPTH
|
||||
SPLIT
|
||||
SPLIT_V
|
||||
SQRT
|
||||
SQUARE
|
||||
SQUARED_DIFFERENCE
|
||||
SQUEEZE
|
||||
STRIDED_SLICE
|
||||
SUB
|
||||
SUM
|
||||
TANH
|
||||
TILE
|
||||
TOPK_V2
|
||||
TRANSPOSE
|
||||
TRANSPOSE_CONV
|
||||
UNIQUE
|
||||
UNPACK
|
||||
WHERE
|
||||
ZEROS_LIKE
|
||||
========================================== ===============================================================================
|
||||
|
||||
.. tab-item:: ONNX
|
||||
|
||||
========================================== ==========================================================================================
|
||||
|
|
@ -1139,6 +745,399 @@ This page lists operations supported by OpenVINO Framework Frontend.
|
|||
zeros_like
|
||||
========================================== ==========================================================================================
|
||||
|
||||
.. tab-item:: TensorFlow
|
||||
|
||||
========================================== ==========================================================================================
|
||||
TensorFlow Supported Operations Limitations
|
||||
========================================== ==========================================================================================
|
||||
Abs
|
||||
Acosh
|
||||
Add
|
||||
AddV2
|
||||
AddN
|
||||
All
|
||||
Any
|
||||
ArgMax
|
||||
ArgMin
|
||||
Asinh
|
||||
Assert Not needed for inference.
|
||||
Assign Not needed for inference.
|
||||
AssignSub Not needed for inference.
|
||||
Atanh
|
||||
AvgPool
|
||||
AvgPoolV2 Supported only for constant-foldable ``kernel_size`` and strides inputs.
|
||||
AvgPool3D
|
||||
BatchMatMul
|
||||
BatchMatMulV2
|
||||
BatchToSpaceND
|
||||
BiasAdd
|
||||
BlockLSTM
|
||||
Bucketize CPU only.
|
||||
BroadcastTo
|
||||
Cast
|
||||
Ceil
|
||||
ClipByValue
|
||||
Concat
|
||||
ConcatV2
|
||||
Const
|
||||
Conv2D
|
||||
Conv2DBackpropInput
|
||||
Conv3D
|
||||
Conv3DBackpropInputV2
|
||||
Cos
|
||||
Cosh
|
||||
CropAndResize ``method`` = ``bilinear`` only.
|
||||
CTCGreedyDecoder Supported only with decoded indices output in a dense format.
|
||||
CTCLoss Supported only with decoded indices input in a dense format.
|
||||
CumSum
|
||||
DepthToSpace
|
||||
DepthwiseConv2dNative
|
||||
Einsum Supported only with equation that does not contain repeated labels within a subscript.
|
||||
Elu
|
||||
EmptyTensorList Supported only when it is part of a sub-graph of the special form.
|
||||
Enter Supported only when it is fused to the TensorIterator layer.
|
||||
Equal
|
||||
Erf
|
||||
Exit Supported only when it is fused to the TensorIterator layer.
|
||||
Exp
|
||||
ExpandDims
|
||||
ExperimentalSparseWeightedSum CPU only.
|
||||
ExtractImagePatches
|
||||
EuclideanNorm
|
||||
FakeQuantWithMinMaxVars
|
||||
FakeQuantWithMinMaxVarsPerChannel
|
||||
FFT Supported only when it is part of a sub-graph of the special form.
|
||||
FFT2D Supported only when it is part of a sub-graph of the special form.
|
||||
FFT3D Supported only when it is part of a sub-graph of the special form.
|
||||
FIFOQueueV2 Supported only when it is part of a sub-graph of the special form.
|
||||
Fill
|
||||
Floor
|
||||
FloorDiv
|
||||
FloorMod
|
||||
FusedBatchNorm
|
||||
FusedBatchNormV2
|
||||
FusedBatchNormV3
|
||||
Gather
|
||||
GatherNd
|
||||
GatherTree
|
||||
GatherV2
|
||||
Greater
|
||||
GreaterEqual
|
||||
Identity Not needed for shape inference.
|
||||
IdentityN
|
||||
IFFT Supported only when it is part of a sub-graph of the special form.
|
||||
IFFT2D Supported only when it is part of a sub-graph of the special form.
|
||||
IFFT3D Supported only when it is part of a sub-graph of the special form.
|
||||
IteratorGetNext Supported only when it is part of a sub-graph of the special form.
|
||||
LRN
|
||||
LeakyRelu
|
||||
Less
|
||||
LessEqual
|
||||
Log
|
||||
Log1p
|
||||
LogicalAnd
|
||||
LogicalOr
|
||||
LogicalNot
|
||||
LogSoftmax
|
||||
LookupTableInsertV2 Supported only when it is part of a sub-graph of the special form.
|
||||
LoopCond Supported only when it is fused to the TensorIterator layer.
|
||||
MatMul
|
||||
Max
|
||||
MaxPool
|
||||
MaxPoolV2 Supported only for constant-foldable ``kernel_size`` and strides inputs.
|
||||
MaxPool3D
|
||||
Maximum
|
||||
Mean
|
||||
Merge Supported only when it is fused to the TensorIterator layer.
|
||||
Min
|
||||
Minimum
|
||||
MirrorPad
|
||||
Mod
|
||||
Mul
|
||||
Neg
|
||||
NextIteration Supported only when it is fused to the TensorIterator layer.
|
||||
NonMaxSuppressionV2
|
||||
NonMaxSuppressionV3
|
||||
NonMaxSuppressionV4
|
||||
NonMaxSuppressionV5
|
||||
NotEqual
|
||||
NoOp
|
||||
OneHot
|
||||
Pack
|
||||
Pad
|
||||
PadV2
|
||||
Placeholder
|
||||
PlaceholderWithDefault
|
||||
Prod
|
||||
QueueDequeue Supported only when it is part of a sub-graph of the special form.
|
||||
QueueDequeueUpToV2 Supported only when it is part of a sub-graph of the special form.
|
||||
QueueDequeueV2 Supported only when it is part of a sub-graph of the special form.
|
||||
RandomUniform
|
||||
RandomUniformInt
|
||||
Range
|
||||
Rank
|
||||
RealDiv
|
||||
Reciprocal
|
||||
Relu
|
||||
Relu6
|
||||
Reshape
|
||||
ResizeBilinear
|
||||
ResizeNearestNeighbor
|
||||
ResourceGather
|
||||
ReverseSequence
|
||||
ReverseV2 Supported only when it can be converted to the ReverseSequence operation.
|
||||
Roll
|
||||
Round
|
||||
Pow
|
||||
Rsqrt
|
||||
ScatterNd
|
||||
Select
|
||||
SelectV2
|
||||
Shape
|
||||
Sigmoid
|
||||
Sin
|
||||
Sinh
|
||||
Size
|
||||
Slice
|
||||
Softmax
|
||||
Softplus
|
||||
Softsign
|
||||
SpaceToBatchND
|
||||
SpaceToDepth
|
||||
SparseFillEmptyRows Supported only when it is part of a sub-graph of the special form.
|
||||
SparseReshape Supported only when it is part of a sub-graph of the special form.
|
||||
SparseSegmentSum Supported only when it is part of a sub-graph of the special form.
|
||||
SparseSegmentMean Supported only when it is part of a sub-graph of the special form.
|
||||
SparseToDense CPU only
|
||||
Split
|
||||
SplitV
|
||||
Sqrt
|
||||
Square
|
||||
SquaredDifference
|
||||
Square
|
||||
Squeeze Cases in which squeeze axis is not specified are not supported.
|
||||
StatelessWhile
|
||||
StopGradient Not needed for shape inference.
|
||||
StridedSlice Supported only for constant-foldable ``begin``, ``end``, and ``strides`` inputs.
|
||||
Sub
|
||||
Sum
|
||||
Swish
|
||||
swish_f32
|
||||
Switch Control flow propagation.
|
||||
Tan
|
||||
Tanh
|
||||
TensorArrayGatherV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayReadV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayScatterV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArraySizeV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorArrayWriteV3 Supported only when it is fused to the TensorIterator layer.
|
||||
TensorListPushBack Supported only when it is part of a sub-graph of the special form.
|
||||
Tile
|
||||
TopkV2
|
||||
Transpose
|
||||
Unpack
|
||||
Variable
|
||||
VariableV2
|
||||
Where Supported only when it is part of a sub-graph of the special form.
|
||||
ZerosLike
|
||||
========================================== ==========================================================================================
|
||||
|
||||
.. tab-item:: TensorFlow Lite
|
||||
|
||||
========================================== ===============================================================================
|
||||
TensorFlow Lite Supported Operations Limitations
|
||||
========================================== ===============================================================================
|
||||
ABS
|
||||
ADD
|
||||
ADD_N
|
||||
ARG_MAX
|
||||
ARG_MIN
|
||||
AVERAGE_POOL_2D
|
||||
BATCH_MATMUL
|
||||
BATCH_TO_SPACE_ND
|
||||
BROADCAST_ARGS
|
||||
BROADCAST_TO
|
||||
CAST
|
||||
CEIL
|
||||
COMPLEX_ABS Supported in a specific pattern with RFFT2D
|
||||
CONCATENATION
|
||||
CONV_2D
|
||||
COS
|
||||
DEPTH_TO_SPACE
|
||||
DEPTHWISE_CONV_2D
|
||||
DEQUANTIZE
|
||||
DIV
|
||||
ELU
|
||||
EQUAL
|
||||
EXP
|
||||
EXPAND_DIMS
|
||||
FILL
|
||||
FLOOR
|
||||
FLOOR_DIV
|
||||
FLOOR_MOD
|
||||
FULLY_CONNECTED
|
||||
GATHER
|
||||
GATHER_ND
|
||||
GREATER
|
||||
GREATER_EQUAL
|
||||
HARD_SWISH
|
||||
L2_NORMALIZATION
|
||||
LEAKY_RELU
|
||||
LESS
|
||||
LESS_EQUAL
|
||||
LOG
|
||||
LOG_SOFTMAX
|
||||
LOGICAL_AND
|
||||
LOGICAL_NOT
|
||||
LOGICAL_OR
|
||||
LOGISTIC
|
||||
MATRIX_DIAG
|
||||
MAX_POOL_2D
|
||||
MAXIMUM
|
||||
MEAN
|
||||
MINIMUM
|
||||
MIRROR_PAD
|
||||
MUL
|
||||
NEG
|
||||
NOT_EQUAL
|
||||
ONE_HOT
|
||||
PACK
|
||||
PAD
|
||||
PADV2
|
||||
POW
|
||||
PRELU
|
||||
QUANTIZE
|
||||
RANGE
|
||||
RANK
|
||||
REDUCE_ALL
|
||||
REDUCE_ANY
|
||||
REDUCE_MAX
|
||||
REDUCE_MIN
|
||||
REDUCE_PROD
|
||||
RELU
|
||||
RELU6
|
||||
RESHAPE
|
||||
RESIZE_BILINEAR
|
||||
RESIZE_NEAREST_NEIGHBOR
|
||||
REVERSE_V2
|
||||
RFFT2D Supported in a specific pattern with COMPLEX_ABS
|
||||
ROUND
|
||||
RSQRT
|
||||
SCATTER_ND
|
||||
SEGMENT_SUM
|
||||
SELECT
|
||||
SELECT_V2
|
||||
SHAPE
|
||||
SIGN
|
||||
SIN
|
||||
SLICE
|
||||
SOFTMAX
|
||||
SPACE_TO_BATCH_ND
|
||||
SPACE_TO_DEPTH
|
||||
SPLIT
|
||||
SPLIT_V
|
||||
SQRT
|
||||
SQUARE
|
||||
SQUARED_DIFFERENCE
|
||||
SQUEEZE
|
||||
STRIDED_SLICE
|
||||
SUB
|
||||
SUM
|
||||
TANH
|
||||
TILE
|
||||
TOPK_V2
|
||||
TRANSPOSE
|
||||
TRANSPOSE_CONV
|
||||
UNIQUE
|
||||
UNPACK
|
||||
WHERE
|
||||
ZEROS_LIKE
|
||||
========================================== ===============================================================================
|
||||
|
||||
.. tab-item:: TensorFlow2 Keras
|
||||
|
||||
========================================== ==========================================================================================
|
||||
TensorFlow 2 Keras Supported Operations Limitations
|
||||
========================================== ==========================================================================================
|
||||
ActivityRegularization
|
||||
Add
|
||||
AdditiveAttention
|
||||
AlphaDropout
|
||||
Attention
|
||||
Average
|
||||
AveragePooling1D
|
||||
AveragePooling2D
|
||||
AveragePooling3D
|
||||
BatchNormalization
|
||||
Bidirectional
|
||||
Concatenate
|
||||
Conv1D
|
||||
Conv1DTranspose Not supported if ``dilation`` is not equal to 1.
|
||||
Conv2D
|
||||
Conv2DTranspose
|
||||
Conv3D
|
||||
Conv3DTranspose
|
||||
Cropping1D
|
||||
Cropping2D
|
||||
Cropping3D
|
||||
Dense
|
||||
DenseFeatures Not supported for categorical and crossed features.
|
||||
DepthwiseConv2D
|
||||
Dot
|
||||
Dropout
|
||||
ELU
|
||||
Embedding
|
||||
Flatten
|
||||
GRU
|
||||
GRUCell
|
||||
GaussianDropout
|
||||
GaussianNoise
|
||||
GlobalAveragePooling1D
|
||||
GlobalAveragePooling2D
|
||||
GlobalAveragePooling3D
|
||||
GlobalMaxPool1D
|
||||
GlobalMaxPool2D
|
||||
GlobalMaxPool3D
|
||||
LSTM
|
||||
LSTMCell
|
||||
Lambda
|
||||
LayerNormalization
|
||||
LeakyReLU
|
||||
LocallyConnected1D
|
||||
LocallyConnected2D
|
||||
MaxPool1D
|
||||
MaxPool2D
|
||||
MaxPool3D
|
||||
Maximum
|
||||
Minimum
|
||||
Multiply
|
||||
PReLU
|
||||
Permute
|
||||
RNN Not supported for some custom cells.
|
||||
ReLU
|
||||
RepeatVector
|
||||
Reshape
|
||||
Roll
|
||||
SeparableConv1D
|
||||
SeparableConv2D
|
||||
SimpleRNN
|
||||
SimpleRNNCell
|
||||
Softmax
|
||||
SpatialDropout1D
|
||||
SpatialDropout2D
|
||||
SpatialDropout3D
|
||||
StackedRNNCells
|
||||
Subtract
|
||||
ThresholdedReLU
|
||||
TimeDistributed
|
||||
UpSampling1D
|
||||
UpSampling2D
|
||||
UpSampling3D
|
||||
ZeroPadding1D
|
||||
ZeroPadding2D
|
||||
ZeroPadding3D
|
||||
========================================== ==========================================================================================
|
||||
|
||||
|
||||
@endsphinxdirective
|
||||
Loading…
Reference in New Issue