Tatiana Savina
GitHub
parent
d9d4c9dacd
commit
1207ec4277
|
|
@ -5,7 +5,7 @@ Install OpenVINO™ Development Tools
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to install OpenVINO™ Development Tools on Windows,
|
||||
:description: Learn how to install OpenVINO™ Development Tools on Windows,
|
||||
Linux, and macOS operating systems, using a PyPi package.
|
||||
|
||||
OpenVINO Development Tools is a set of utilities that make it easy to develop and optimize models and applications for OpenVINO. It provides the following tools:
|
||||
|
|
@ -20,9 +20,9 @@ The instructions on this page show how to install OpenVINO Development Tools. If
|
|||
|
||||
In both cases, Python 3.8 - 3.11 needs to be installed on your machine before starting.
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
From the 2022.1 release, the OpenVINO™ Development Tools can only be installed via PyPI.
|
||||
From the 2022.1 release, the OpenVINO™ Development Tools can only be installed via PyPI.
|
||||
|
||||
.. _python_developers:
|
||||
|
||||
|
|
@ -30,7 +30,7 @@ For Python Developers
|
|||
#####################
|
||||
|
||||
If you are a Python developer, follow the steps in the :ref:`Installing OpenVINO Development Tools <install_dev_tools>` section on this page to install it. Installing OpenVINO Development Tools will also install OpenVINO Runtime as a dependency, so you don’t need to install OpenVINO Runtime separately. This option is recommended for new users.
|
||||
|
||||
|
||||
.. _cpp_developers:
|
||||
|
||||
For C/C++ Developers
|
||||
|
|
@ -64,7 +64,7 @@ Installation in a New Environment
|
|||
+++++++++++++++++++++++++++++++++
|
||||
|
||||
If you do not have an environment with a deep learning framework for the input model or you encounter any compatibility issues between OpenVINO
|
||||
and your version of deep learning framework, you may install OpenVINO Development Tools with validated versions of frameworks into a new environment.
|
||||
and your version of deep learning framework, you may install OpenVINO Development Tools with validated versions of frameworks into a new environment.
|
||||
|
||||
Step 1. Set Up Python Virtual Environment
|
||||
-----------------------------------------
|
||||
|
|
@ -75,19 +75,19 @@ Create a virtual Python environment to avoid dependency conflicts. To create a v
|
|||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
python -m venv openvino_env
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: linux-and-macos
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
python3 -m venv openvino_env
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Step 2. Activate Virtual Environment
|
||||
------------------------------------
|
||||
|
|
@ -98,16 +98,16 @@ Activate the newly created Python virtual environment by issuing this command:
|
|||
|
||||
.. tab-item:: Windows
|
||||
:sync: windows
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
openvino_env\Scripts\activate
|
||||
|
||||
.. tab-item:: Linux and macOS
|
||||
:sync: linux-and-macos
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
source openvino_env/bin/activate
|
||||
|
||||
.. important::
|
||||
|
|
@ -138,7 +138,7 @@ To install and configure the components of the development package together with
|
|||
|
||||
|
||||
where the ``extras`` parameter specifies the source deep learning framework for the input model
|
||||
and is one or more of the following values separated with "," : ``caffe``, ``kaldi``, ``mxnet``, ``onnx``, ``pytorch``, ``tensorflow``, ``tensorflow2``.
|
||||
and is one or more of the following values separated with "," : ``caffe``, ``kaldi``, ``mxnet``, ``onnx``, ``pytorch``, ``tensorflow``, ``tensorflow2``.
|
||||
|
||||
For example, to install and configure dependencies required for working with TensorFlow 2.x and ONNX models, use the following command:
|
||||
|
||||
|
|
@ -147,7 +147,7 @@ For example, to install and configure dependencies required for working with Ten
|
|||
pip install openvino-dev[tensorflow2,onnx]
|
||||
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
Model conversion API support for TensorFlow 1.x environment has been deprecated. Use the ``tensorflow2`` parameter to install a TensorFlow 2.x environment that can convert both TensorFlow 1.x and 2.x models. If your model isn't compatible with the TensorFlow 2.x environment, use the `tensorflow` parameter to install the TensorFlow 1.x environment. The TF 1.x environment is provided only for legacy compatibility reasons.
|
||||
|
||||
|
|
@ -197,7 +197,7 @@ Try the :doc:`C++ Quick Start Example <openvino_docs_get_started_get_started_dem
|
|||
Visit the :doc:`Samples <openvino_docs_OV_UG_Samples_Overview>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* :doc:`Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd>`
|
||||
* :doc:`Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition>`
|
||||
* :doc:`Object classification sample <openvino_sample_hello_classification>`
|
||||
|
||||
Learn OpenVINO Development Tools
|
||||
++++++++++++++++++++++++++++++++
|
||||
|
|
|
|||
|
|
@ -148,6 +148,7 @@ Step 2: Install OpenVINO Runtime Using the APT Package Manager
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
sudo apt install openvino-2023.3.0
|
||||
|
||||
.. note::
|
||||
|
|
@ -239,7 +240,7 @@ Learn more about how to integrate a model in OpenVINO applications by trying out
|
|||
* Visit the :ref:`Samples <code samples>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* `Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd.html>`_
|
||||
* `Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition.html>`_
|
||||
* `Object classification sample <openvino_sample_hello_classification.html>`_
|
||||
|
||||
You can also try the following:
|
||||
|
||||
|
|
|
|||
|
|
@ -135,6 +135,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_ubuntu22_2023.3.0.13775.ceeafaf64f3_x86_64.tgz --output openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv l_openvino_toolkit_ubuntu22_2023.3.0.13775.ceeafaf64f3_x86_64 /opt/intel/openvino_2023.3.0
|
||||
|
|
@ -144,15 +145,18 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_ubuntu20_2023.3.0.13775.ceeafaf64f3_x86_64.tgz --output openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv l_openvino_toolkit_ubuntu20_2023.3.0.13775.ceeafaf64f3_x86_64 /opt/intel/openvino_2023.3.0
|
||||
|
||||
|
||||
.. tab-item:: Ubuntu 18.04
|
||||
:sync: ubuntu-18
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_ubuntu18_2023.3.0.13775.ceeafaf64f3_x86_64.tgz --output openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv l_openvino_toolkit_ubuntu18_2023.3.0.13775.ceeafaf64f3_x86_64 /opt/intel/openvino_2023.3.0
|
||||
|
|
@ -162,6 +166,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/linux/l_openvino_toolkit_rhel8_2023.3.0.13775.ceeafaf64f3_x86_64.tgz --output openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv l_openvino_toolkit_rhel8_2023.3.0.13775.ceeafaf64f3_x86_64 /opt/intel/openvino_2023.3.0
|
||||
|
|
@ -175,11 +180,13 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv l_openvino_toolkit_centos7_2023.3.0.13775.ceeafaf64f3_x86_64 /opt/intel/openvino_2023.3.0
|
||||
|
||||
|
||||
.. tab-item:: ARM 64-bit
|
||||
:sync: arm-64
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/nightly/2024.0.0-13770-9b52171d290/l_openvino_toolkit_ubuntu18_2023.3.0.13775.ceeafaf64f3_arm64.tgz -O openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv l_openvino_toolkit_ubuntu18_2023.3.0.13775.ceeafaf64f3_arm64 /opt/intel/openvino_2023.3.0
|
||||
|
|
@ -219,6 +226,7 @@ Step 1: Download and Install the OpenVINO Core Components
|
|||
.. code-block:: sh
|
||||
|
||||
cd /opt/intel
|
||||
|
||||
sudo ln -s openvino_2023.3.0 openvino_2023
|
||||
|
||||
.. note::
|
||||
|
|
@ -300,7 +308,7 @@ Learn more about how to integrate a model in OpenVINO applications by trying out
|
|||
Visit the :doc:`Samples <openvino_docs_OV_UG_Samples_Overview>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* `Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd.html>`__
|
||||
* `Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition.html>`__
|
||||
* `Object classification sample <openvino_sample_hello_classification.html>`__
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -127,6 +127,7 @@ Install OpenVINO Runtime
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
sudo yum install openvino-2023.3.0
|
||||
|
||||
|
||||
|
|
@ -202,6 +203,7 @@ To uninstall OpenVINO Runtime via YUM, run the following command based on your n
|
|||
|
||||
|
||||
|
||||
|
||||
What's Next?
|
||||
#############
|
||||
|
||||
|
|
@ -217,7 +219,7 @@ Learn more about how to integrate a model in OpenVINO applications by trying out
|
|||
* Visit the :ref:`Samples <code samples>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* `Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd.html>`_
|
||||
* `Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition.html>`_
|
||||
* `Object classification sample <openvino_sample_hello_classification.html>`_
|
||||
|
||||
You can also try the following things:
|
||||
|
||||
|
|
|
|||
|
|
@ -72,6 +72,7 @@ Step 1: Install OpenVINO Core Components
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/macos/m_openvino_toolkit_macos_10_15_2023.3.0.13775.ceeafaf64f3_x86_64.tgz --output openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv m_openvino_toolkit_macos_10_15_2023.3.0.13775.ceeafaf64f3_x86_64 /opt/intel/openvino_2023.3.0
|
||||
|
|
@ -81,10 +82,12 @@ Step 1: Install OpenVINO Core Components
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
curl -L https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/macos/m_openvino_toolkit_macos_11_0_2023.3.0.13775.ceeafaf64f3_arm64.tgz --output openvino_2023.3.0.tgz
|
||||
tar -xf openvino_2023.3.0.tgz
|
||||
sudo mv m_openvino_toolkit_macos_11_0_2023.3.0.13775.ceeafaf64f3_arm64 /opt/intel/openvino_2023.3.0
|
||||
|
||||
|
||||
5. (Optional) Install *numpy* Python Library:
|
||||
|
||||
.. note::
|
||||
|
|
@ -102,6 +105,7 @@ Step 1: Install OpenVINO Core Components
|
|||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
sudo ln -s /opt/intel/openvino_2023.3.0 /opt/intel/openvino_2023
|
||||
|
||||
|
||||
|
|
@ -174,7 +178,7 @@ Now that you've installed OpenVINO Runtime, you're ready to run your own machine
|
|||
Visit the :ref:`Samples <code samples>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* `Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd.html>`_
|
||||
* `Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition.html>`_
|
||||
* `Object classification sample <openvino_sample_hello_classification.html>`_
|
||||
|
||||
Uninstalling Intel® Distribution of OpenVINO™ Toolkit
|
||||
#####################################################
|
||||
|
|
|
|||
|
|
@ -105,6 +105,7 @@ with the proper OpenVINO version number:
|
|||
conda remove openvino=2023.3.0
|
||||
|
||||
|
||||
|
||||
What's Next?
|
||||
############################################################
|
||||
|
||||
|
|
@ -120,7 +121,7 @@ on building and running a basic image classification C++ application.
|
|||
Visit the :doc:`Samples <openvino_docs_OV_UG_Samples_Overview>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* `Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd.html>`__
|
||||
* `Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition.html>`__
|
||||
* `Object classification sample <openvino_sample_hello_classification.html>`__
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,12 +5,12 @@ Install Intel® Distribution of OpenVINO™ toolkit from a Docker Image
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to use a prebuilt Docker image or create an image
|
||||
:description: Learn how to use a prebuilt Docker image or create an image
|
||||
manually to install OpenVINO™ Runtime on Linux and Windows operating systems.
|
||||
|
||||
This guide presents information on how to use a pre-built Docker image/create an image manually to install OpenVINO™ Runtime.
|
||||
|
||||
Supported host operating systems for the Docker Base image:
|
||||
Supported host operating systems for the Docker Base image:
|
||||
|
||||
- Linux
|
||||
- Windows (WSL2)
|
||||
|
|
@ -22,14 +22,13 @@ You can get started easily with pre-built and published docker images, which are
|
|||
* `Red Hat Quay.io <https://quay.io/organization/openvino>`__
|
||||
* `Red Hat Ecosystem Catalog (runtime image) <https://catalog.redhat.com/software/containers/intel/openvino-runtime/606ff4d7ecb5241699188fb3>`__
|
||||
* `Red Hat Ecosystem Catalog (development image) <https://catalog.redhat.com/software/containers/intel/openvino-dev/613a450dc9bc35f21dc4a1f7>`__
|
||||
* `Azure Marketplace <https://azuremarketplace.microsoft.com/en-us/marketplace/apps/intel_corporation.openvino>`__
|
||||
|
||||
You can use the `available Dockerfiles on GitHub <https://github.com/openvinotoolkit/docker_ci/tree/master/dockerfiles>`__
|
||||
or generate a Dockerfile with your settings via `DockerHub CI framework <https://github.com/openvinotoolkit/docker_ci/>`__,
|
||||
which can generate a Dockerfile, build, test, and deploy an image using the Intel® Distribution of OpenVINO™ toolkit.
|
||||
You can reuse available Dockerfiles, add your layer and customize the OpenVINO™ image to your needs.
|
||||
The Docker CI repository includes guides on how to
|
||||
`get started with docker images <https://github.com/openvinotoolkit/docker_ci/blob/master/get-started.md>`__ and how to use
|
||||
which can generate a Dockerfile, build, test, and deploy an image using the Intel® Distribution of OpenVINO™ toolkit.
|
||||
You can reuse available Dockerfiles, add your layer and customize the OpenVINO™ image to your needs.
|
||||
The Docker CI repository includes guides on how to
|
||||
`get started with docker images <https://github.com/openvinotoolkit/docker_ci/blob/master/get-started.md>`__ and how to use
|
||||
`OpenVINO™ Toolkit containers with GPU accelerators. <https://github.com/openvinotoolkit/docker_ci/blob/master/docs/accelerators.md>`__
|
||||
|
||||
To start using Dockerfiles the following conditions must be met:
|
||||
|
|
@ -40,16 +39,16 @@ To start using Dockerfiles the following conditions must be met:
|
|||
|
||||
.. note::
|
||||
|
||||
OpenVINO's `Docker <https://docs.docker.com/>`__ and :doc:`Bare Metal <openvino_docs_install_guides_overview>`
|
||||
distributions are identical, so the documentation applies to both.
|
||||
OpenVINO's `Docker <https://docs.docker.com/>`__ and :doc:`Bare Metal <openvino_docs_install_guides_overview>`
|
||||
distributions are identical, so the documentation applies to both.
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
OpenVINO development environment in a docker container is also available in the
|
||||
OpenVINO development environment in a docker container is also available in the
|
||||
`notebook repository <https://github.com/openvinotoolkit/openvino_notebooks>`__. It can be implemented in
|
||||
`OpenShift RedHat OpenData Science (RHODS) <https://github.com/openvinotoolkit/operator/blob/main/docs/notebook_in_rhods.md>`__.
|
||||
|
||||
More information about Docker CI for Intel® Distribution of OpenVINO™ toolset can be found
|
||||
More information about Docker CI for Intel® Distribution of OpenVINO™ toolset can be found
|
||||
`here <https://github.com/openvinotoolkit/docker_ci/blob/master/README.md>`__
|
||||
|
||||
* `Docker CI framework for Intel® Distribution of OpenVINO™ toolkit <https://github.com/openvinotoolkit/docker_ci/blob/master/README.md>`__
|
||||
|
|
|
|||
|
|
@ -5,16 +5,16 @@ Install Intel® Distribution of OpenVINO™ Toolkit from PyPI Repository
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to install OpenVINO™ Runtime on Windows, Linux, and
|
||||
:description: Learn how to install OpenVINO™ Runtime on Windows, Linux, and
|
||||
macOS operating systems, using a PyPi package.
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
|
||||
Note that the PyPi distribution:
|
||||
|
||||
|
||||
* offers the Python API only
|
||||
* is dedicated to users of all major OSes: Windows, Linux, and macOS
|
||||
* is dedicated to users of all major OSes: Windows, Linux, and macOS
|
||||
(all x86_64 / arm64 architectures)
|
||||
* Windows and Linux do not offer support for GNA and NPU inference
|
||||
* macOS offers support only for CPU inference
|
||||
|
|
@ -26,12 +26,12 @@ Install Intel® Distribution of OpenVINO™ Toolkit from PyPI Repository
|
|||
|
||||
| Full requirement listing is available in:
|
||||
| :doc:`System Requirements Page <system_requirements>`
|
||||
| `PyPi OpenVINO page <https://pypi.org/project/openvino/>`__
|
||||
|
||||
|
||||
| `PyPI OpenVINO page <https://pypi.org/project/openvino/>`__
|
||||
|
||||
|
||||
.. tab-item:: Processor Notes
|
||||
:sync: processor-notes
|
||||
|
||||
|
||||
| To see if your processor includes the integrated graphics technology and supports iGPU inference, refer to:
|
||||
| `Product Specifications <https://ark.intel.com/>`__
|
||||
|
||||
|
|
@ -120,8 +120,8 @@ Run the command below:
|
|||
If installation was successful, you will see the list of available devices.
|
||||
|
||||
|
||||
Congratulations! You've just Installed OpenVINO! For some use cases you may still
|
||||
need to install additional components. Check the
|
||||
Congratulations! You've just Installed OpenVINO! For some use cases you may still
|
||||
need to install additional components. Check the
|
||||
:doc:`list of additional configurations <openvino_docs_install_guides_configurations_header>`
|
||||
to see if your case needs any of them.
|
||||
|
||||
|
|
|
|||
|
|
@ -200,7 +200,7 @@ Now that you've installed OpenVINO Runtime, you're ready to run your own machine
|
|||
Visit the :ref:`Samples <code samples>` page for other C++ example applications to get you started with OpenVINO, such as:
|
||||
|
||||
* `Basic object detection with the Hello Reshape SSD C++ sample <openvino_sample_hello_reshape_ssd.html>`_
|
||||
* `Automatic speech recognition C++ sample <openvino_sample_automatic_speech_recognition.html>`_
|
||||
* `Object classification sample <openvino_sample_hello_classification.html>`_
|
||||
|
||||
|
||||
.. _uninstall-from-windows:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ Create a Yocto Image with Intel® Distribution of OpenVINO™ toolkit
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to create a Yocto image with Intel® Distribution of
|
||||
:description: Learn how to create a Yocto image with Intel® Distribution of
|
||||
OpenVINO™ toolkit on your host machine.
|
||||
|
||||
This document provides instructions for creating a Yocto image with Intel® Distribution of OpenVINO™ toolkit.
|
||||
|
|
@ -16,68 +16,68 @@ Step 1: Set Up Environment
|
|||
##########################
|
||||
|
||||
1. Clone the repositories.
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
git clone https://git.yoctoproject.org/git/poky
|
||||
git clone https://git.yoctoproject.org/meta-intel
|
||||
git clone https://git.openembedded.org/meta-openembedded
|
||||
git clone https://github.com/kraj/meta-clang.git
|
||||
|
||||
|
||||
|
||||
2. Set up the OpenEmbedded build environment.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
source poky/oe-init-build-env
|
||||
|
||||
|
||||
|
||||
3. Add BitBake layers.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
bitbake-layers add-layer ../meta-intel
|
||||
bitbake-layers add-layer ../meta-openembedded/meta-oe
|
||||
bitbake-layers add-layer ../meta-openembedded/meta-python
|
||||
bitbake-layers add-layer ../meta-clang
|
||||
|
||||
|
||||
|
||||
4. Verify if layers were added (optional step).
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
bitbake-layers show-layers
|
||||
|
||||
|
||||
|
||||
5. Set up BitBake configurations.
|
||||
Include extra configuration in the `conf/local.conf` file in your build directory as required.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
# Build with SSE4.2, AVX2 etc. extensions
|
||||
MACHINE = "intel-skylake-64"
|
||||
|
||||
|
||||
# Enable clDNN GPU plugin when needed.
|
||||
# This requires meta-clang and meta-oe layers to be included in bblayers.conf
|
||||
# and is not enabled by default.
|
||||
PACKAGECONFIG:append:pn-openvino-inference-engine = " opencl"
|
||||
|
||||
|
||||
# Enable building OpenVINO Python API.
|
||||
# This requires meta-python layer to be included in bblayers.conf.
|
||||
PACKAGECONFIG:append:pn-openvino-inference-engine = " python3"
|
||||
|
||||
|
||||
# This adds OpenVINO related libraries in the target image.
|
||||
CORE_IMAGE_EXTRA_INSTALL:append = " openvino-inference-engine"
|
||||
|
||||
|
||||
# This adds OpenVINO samples in the target image.
|
||||
CORE_IMAGE_EXTRA_INSTALL:append = " openvino-inference-engine-samples"
|
||||
|
||||
|
||||
# Include OpenVINO Python API package in the target image.
|
||||
CORE_IMAGE_EXTRA_INSTALL:append = " openvino-inference-engine-python3"
|
||||
|
||||
|
||||
# Include model conversion API in the target image.
|
||||
CORE_IMAGE_EXTRA_INSTALL:append = " openvino-model-optimizer"
|
||||
|
||||
|
||||
|
||||
Step 2: Build a Yocto Image with OpenVINO Packages
|
||||
##################################################
|
||||
|
|
@ -85,11 +85,11 @@ Step 2: Build a Yocto Image with OpenVINO Packages
|
|||
Run BitBake to build your image with OpenVINO packages. For example, to build the minimal image, run the following command:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
bitbake core-image-minimal
|
||||
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
For validation/testing/reviewing purposes, you may consider using the ``nohup`` command and ensure that your vpn/ssh connection remains uninterrupted.
|
||||
|
||||
Step 3: Verify the Yocto Image
|
||||
|
|
@ -98,14 +98,14 @@ Step 3: Verify the Yocto Image
|
|||
Verify that OpenVINO packages were built successfully. Run the following command:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
oe-pkgdata-util list-pkgs | grep openvino
|
||||
|
||||
|
||||
If the image build is successful, it will return the list of packages as below:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
openvino-inference-engine
|
||||
openvino-inference-engine-dbg
|
||||
openvino-inference-engine-dev
|
||||
|
|
@ -124,7 +124,7 @@ Additional Resources
|
|||
- `BitBake Tool <https://docs.yoctoproject.org/bitbake/>`__
|
||||
- `Poky <https://git.yoctoproject.org/poky>`__
|
||||
- `Meta-intel <https://git.yoctoproject.org/meta-intel/tree/README>`__
|
||||
- `Meta-openembedded <http://cgit.openembedded.org/meta-openembedded/tree/README>`__
|
||||
- `Meta-openembedded <http://cgit.openembedded.org/meta-openembedded/tree/README.md>`__
|
||||
- `Meta-clang <https://github.com/kraj/meta-clang/tree/master/#readme>`__
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,15 +5,15 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: A collection of troubleshooting steps and solutions to possible
|
||||
problems that may occur during the installation and configuration
|
||||
:description: A collection of troubleshooting steps and solutions to possible
|
||||
problems that may occur during the installation and configuration
|
||||
of OpenVINO™ on your system.
|
||||
|
||||
|
||||
.. _troubleshooting guide for install:
|
||||
|
||||
| This guide provides general troubleshooting steps and solutions to possible issues that
|
||||
may be encountered while installing and configuring OpenVINO™. For a comprehensive
|
||||
may be encountered while installing and configuring OpenVINO™. For a comprehensive
|
||||
database of support topics on OpenVINO, go to:
|
||||
| `Support for OpenVINO™ toolkit <https://www.intel.com/content/www/us/en/support/products/96066/software/development-software/openvino-toolkit.html>`__
|
||||
|
||||
|
|
@ -24,30 +24,29 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
.. dropdown:: Errors with Installing via PIP for Users in China
|
||||
|
||||
Users in China might encounter errors while downloading sources via PIP during OpenVINO™ installation. To resolve the issues, try one of the following options:
|
||||
|
||||
* Add the download source using the ``-i`` parameter with the Python ``pip`` command. For example:
|
||||
|
||||
* Add the download source using the ``-i`` parameter with the Python ``pip`` command. For example:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
pip install openvino-dev -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
|
||||
Use the ``--trusted-host`` parameter if the URL above is ``http`` instead of ``https``.
|
||||
You can also run the following command to install specific framework. For example:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
pip install openvino-dev[tensorflow2] -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
|
||||
* For C++ developers, if you have installed OpenVINO Runtime via APT, YUM, or the archive file, and then installed OpenVINO Development Tools via PyPI, you may run into issues. To resolve that, install the components in ``requirements.txt`` by using the following command:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
pip install openvino-dev[tensorflow2] -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
|
||||
* For C++ developers, if you have installed OpenVINO Runtime via APT, YUM, or the archive file, and then installed OpenVINO Development Tools via PyPI, you may run into issues. To resolve that, install the components in ``requirements.txt`` by using the following command:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
pip install -r <INSTALL_DIR>/tools/requirements.txt
|
||||
|
||||
|
||||
For APT and YUM users, replace the ``INSTALL_DIR`` with ``/usr/share/openvino``.
|
||||
|
||||
<!-- this part was from Docker installation -->
|
||||
|
||||
.. dropdown:: Issues with Installing OpenVINO on Linux from Docker
|
||||
|
||||
|
|
@ -68,7 +67,7 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
When using the ``bitbake-layers add-layer meta-intel`` command, the following error might occur:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
NOTE: Starting bitbake server...
|
||||
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: chrpath diffstat pzstd zstd
|
||||
|
||||
|
|
@ -76,10 +75,10 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
To resolve the issue, install the ``chrpath diffstat zstd`` tools:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
sudo apt-get install chrpath diffstat zstd
|
||||
|
||||
3. If you run into issues while installing or configuring OpenVINO™, you can try the following methods to do some quick checks first.
|
||||
3. If you run into issues while installing or configuring OpenVINO™, you can try the following methods to do some quick checks first.
|
||||
|
||||
.. dropdown:: Check the versions of OpenVINO Runtime and Development Tools
|
||||
|
||||
|
|
@ -106,13 +105,13 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
|
||||
|
||||
* To check the version of OpenVINO Runtime, use the following code:
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
from openvino.runtime import get_version
|
||||
|
||||
from openvino.runtime import get_version
|
||||
get_version()
|
||||
|
||||
|
||||
|
||||
.. dropdown:: Check the versions of Python and PIP
|
||||
|
||||
To check your Python version, run ``python -VV`` or ``python --version``. The supported Python versions should be 64-bit and between 3.8 and 3.11. If you are using Python 3.7, you are recommended to upgrade the version to 3.8 or higher.
|
||||
|
|
@ -125,22 +124,19 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
For PIP, make sure that you have installed the latest version. To check and upgrade your PIP version, run the following command:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
python -m pip install --upgrade pip
|
||||
|
||||
<!--## Check the special tips for Anaconda installation-->
|
||||
|
||||
<!--add this part in future-->
|
||||
|
||||
.. dropdown:: Check if environment variables are set correctly
|
||||
|
||||
- For Python developers, if you previously installed OpenVINO using the archive file, and are now installing OpenVINO using PIP, remove all the PATH settings and the lines with ``setupvars`` from ``.bashrc``. Note that if you installed OpenVINO with PIP in a virtual environment, you don't need to set any environment variables.
|
||||
- If you have installed OpenVINO before, you probably have added ``setupvars`` to your ``PATH /.bashrc`` or Windows environment variables. After restarting your environment, you should see similar information as below:
|
||||
- If you have installed OpenVINO before, you probably have added ``setupvars`` to your ``PATH /.bashrc`` or Windows environment variables. After restarting your environment, you should see similar information as below:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
[setupvars.sh] OpenVINO™ environment initialized
|
||||
|
||||
|
||||
|
||||
- If you don't see the information above, your PATH variables may be configured incorrectly. Check if you have typed the correct <INSTALL_DIR> or you are trying to activate in the correct directory.
|
||||
- If you added it to a ``.bashrc`` file, make sure that the command is correctly written and the file is found in the ``~/.bashrc`` folder.
|
||||
|
|
@ -152,7 +148,7 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
.. code-block:: sh
|
||||
|
||||
python -c "from openvino import Core; print(Core().available_devices)"
|
||||
|
||||
|
||||
If OpenVINO was successfully installed, you will see a list of available devices.
|
||||
|
||||
* If you installed OpenVINO Runtime using the archive file, you can search "openvino" in Apps & Features on a Windows system, or check your installation directory on Linux to see if OpenVINO is there.
|
||||
|
|
@ -169,12 +165,12 @@ Troubleshooting Guide for OpenVINO™ Installation & Configuration
|
|||
|
||||
.. dropdown:: Check firewall and network settings
|
||||
|
||||
Make sure that your firewall and network settings are configured correctly. For example, consider configuring system-wide proxy settings and specifying options for using PIP behind the proxy:
|
||||
Make sure that your firewall and network settings are configured correctly. For example, consider configuring system-wide proxy settings and specifying options for using PIP behind the proxy:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
pip install --proxy http://address:port --trusted-host pypi.org openvino
|
||||
pip install --proxy http://address:port --trusted-host pypi.org openvino
|
||||
|
||||
For specific issues, see :ref:`Errors with Installing via PIP for Users in China <install_for_prc>` and :ref:`proxy issues with installing OpenVINO on Linux from Docker <proxy-issues>`.
|
||||
For specific issues, see Errors with Installing via PIP for Users in China and Proxy issues with installing OpenVINO on Linux from Docker questions above.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ Learn OpenVINO
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Explore learning materials, including interactive
|
||||
Python tutorials and sample console applications that explain
|
||||
:description: Explore learning materials, including interactive
|
||||
Python tutorials and sample console applications that explain
|
||||
how to use OpenVINO features.
|
||||
|
||||
|
||||
|
|
@ -17,22 +17,18 @@ Learn OpenVINO
|
|||
Interactive Tutorials (Python) <tutorials>
|
||||
Sample Applications (Python & C++) <openvino_docs_OV_UG_Samples_Overview>
|
||||
Generative AI Optimization and Deployment <gen_ai_guide>
|
||||
Import TensorFlow and PyTorch Models <openvino_docs_model_processing_introduction_draft>
|
||||
|
||||
|
||||
This section will help you get a hands-on experience with OpenVINO even if you are just starting
|
||||
to learn what OpenVINO is and how it works. It includes various types of learning materials
|
||||
accommodating different learning needs, which means you should find it useful if you are a beginning,
|
||||
This section will help you get a hands-on experience with OpenVINO even if you are just starting
|
||||
to learn what OpenVINO is and how it works. It includes various types of learning materials
|
||||
accommodating different learning needs, which means you should find it useful if you are a beginning,
|
||||
as well as an experienced user.
|
||||
|
||||
| :doc:`Tutorials <tutorials>`
|
||||
| :doc:`Tutorials <tutorials>`
|
||||
| A collection of interactive Python tutorials. It introduces you to the OpenVINO™ toolkit explaining how to use the Python API and tools for optimized deep learning inference. The tutorials are available in Jupyter notebooks and can be run in your browser. No installation required.
|
||||
|
||||
| :doc:`OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
|
||||
| :doc:`OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
|
||||
| The OpenVINO samples (Python and C++) are simple console applications that show how to use specific OpenVINO API features. They can assist you in executing tasks such as loading a model, running inference, querying particular device capabilities, etc.
|
||||
|
||||
| :doc:`Optimize and Deploy Generative AI Models <gen_ai_guide>`
|
||||
| Detailed information on how OpenVINO accelerates Generative AI use cases and what models it supports. This tutorial provides instructions for running Generative AI models using Hugging Face Optimum Intel and Native OpenVINO APIs.
|
||||
|
||||
| :doc:`Import TensorFlow and PyTorch Models <openvino_docs_model_processing_introduction_draft>`
|
||||
| Learn about different import methods for TensorFlow and PyTorch models.
|
||||
|
|
@ -17,37 +17,44 @@ Model Preparation
|
|||
Setting Input Shapes <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Converting_Model>
|
||||
|
||||
|
||||
You can obtain a model in one of supported formats, **PyTorch, TensorFlow, TensorFlow Lite, ONNX, and PaddlePaddle**,
|
||||
in many ways. The easiest one is to download it from an online database,
|
||||
such as `TensorFlow Hub <https://tfhub.dev/>`__, `Hugging Face <https://huggingface.co/>`__,
|
||||
OpenVINO supports the following model formats:
|
||||
|
||||
* PyTorch,
|
||||
* TensorFlow,
|
||||
* TensorFlow Lite,
|
||||
* ONNX,
|
||||
* PaddlePaddle,
|
||||
* OpenVINO IR.
|
||||
|
||||
The easiest way to obtain a model is to download it from an online database, such as `TensorFlow Hub <https://tfhub.dev/>`__, `Hugging Face <https://huggingface.co/>`__,
|
||||
and `Torchvision models <https://pytorch.org/hub/>`__. Now you have two options:
|
||||
|
||||
* Skip model conversion and `run inference <openvino_docs_OV_UG_Integrate_OV_with_your_application>`__ directly from the **TensorFlow, TensorFlow Lite, ONNX, and PaddlePaddle** source format. Conversion
|
||||
* Skip model conversion and :doc:`run inference <openvino_docs_OV_UG_Integrate_OV_with_your_application>` directly from the **TensorFlow, TensorFlow Lite, ONNX, and PaddlePaddle** source format. Conversion
|
||||
will still be performed but it will happen automatically and "under the hood".
|
||||
This option, while convenient, offers lower performance and stability, as well as
|
||||
fewer optimization options.
|
||||
|
||||
* Explicitly :doc:`convert the model to OpenVINO IR <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_IR>`.
|
||||
This approach offers the best possible results and is the recommended one,
|
||||
especially for production-ready solutions. Consider storing your model in this format to minimize first-inference latency,
|
||||
especially for production-ready solutions. Consider storing your model in this format to minimize first-inference latency,
|
||||
perform model optimizations, and save space on your drive, in some cases. Explicit conversion can be done in two ways:
|
||||
|
||||
* the `Python API functions <#convert-a-model-with-python-convert-model>`__ (``openvino.convert_model`` and ``openvino.save_model``)
|
||||
* the `ovc <#convert-a-model-in-cli-ovc>`__ command line tool.
|
||||
|
||||
Once saved as :doc:`OpenVINO IR <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_IR>` (a set of ``.xml`` and ``.bin`` files),
|
||||
* the `Python API functions <#convert-a-model-with-python-convert-model>`__ (``openvino.convert_model`` and ``openvino.save_model``)
|
||||
* the `ovc <#convert-a-model-in-cli-ovc>`__ command line tool.
|
||||
|
||||
Once saved as :doc:`OpenVINO IR <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_IR>` (a set of ``.xml`` and ``.bin`` files),
|
||||
the model may be deployed with maximum performance. Because it is already optimized
|
||||
for `OpenVINO inference <openvino_docs_OV_UG_Integrate_OV_with_your_application>`__, it can be read, compiled, and inferred with no additional delay.
|
||||
for :doc:`OpenVINO inference <openvino_docs_OV_UG_Integrate_OV_with_your_application>`, it can be read, compiled, and inferred with no additional delay.
|
||||
|
||||
.. note::
|
||||
|
||||
Model conversion API prior to OpenVINO 2023.1 is considered deprecated.
|
||||
|
||||
Model conversion API prior to OpenVINO 2023.1 is considered deprecated.
|
||||
Existing and new projects are recommended to transition to the new
|
||||
solutions, keeping in mind that they are not fully backwards compatible
|
||||
solutions, keeping in mind that they are not fully backwards compatible
|
||||
with ``openvino.tools.mo.convert_model`` or the ``mo`` CLI tool.
|
||||
For more details, see the :doc:`Model Conversion API Transition Guide <openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition>`.
|
||||
|
||||
For PyTorch models, `Python API <#convert-a-model-with-python-convert-model>`__ is the only conversion option.
|
||||
For PyTorch models, `Python API <#convert-a-model-with-python-convert-model>`__ is the only conversion option.
|
||||
|
||||
TensorFlow may present additional considerations :doc:`TensorFlow Frontend Capabilities and Limitations <openvino_docs_MO_DG_TensorFlow_Frontend>`.
|
||||
|
||||
|
|
@ -236,15 +243,15 @@ public repositories:
|
|||
result = compiled_model(input_data)
|
||||
|
||||
|
||||
* Saving the model, **Option 1**, is used as a separate step, outside of deployment.
|
||||
* Saving the model, **Option 1**, is used as a separate step, outside of deployment.
|
||||
The file it provides is then used in the final software solution, resulting in
|
||||
maximum performance due to fewer dependencies and faster model loading.
|
||||
maximum performance due to fewer dependencies and faster model loading.
|
||||
|
||||
* Compiling the model, **Option 2**, provides a convenient way to quickly switch from
|
||||
framework-based code to OpenVINO-based code in your existing Python inference application.
|
||||
* Compiling the model, **Option 2**, provides a convenient way to quickly switch from
|
||||
framework-based code to OpenVINO-based code in your existing Python inference application.
|
||||
The converted model is not saved to IR but compiled and used for inference within the same application.
|
||||
|
||||
Before saving the model to OpenVINO IR, consider :doc:`Post-training Optimization <ptq_introduction>`
|
||||
Before saving the model to OpenVINO IR, consider :doc:`Post-training Optimization <ptq_introduction>`
|
||||
to achieve more efficient inference and smaller model size.
|
||||
|
||||
|
||||
|
|
@ -252,7 +259,7 @@ Convert a Model in CLI: ``ovc``
|
|||
###############################
|
||||
|
||||
``ovc`` is a command-line model converter, combining the ``openvino.convert_model``
|
||||
and ``openvino.save_model`` functionalities, providing the exact same results, if the same set of
|
||||
and ``openvino.save_model`` functionalities, providing the exact same results, if the same set of
|
||||
parameters is used for saving into OpenVINO IR. It converts files from one of the to :doc:`OpenVINO IR <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_IR>`, which can then be read, compiled,
|
||||
and run by the final inference application.
|
||||
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ Convert to OpenVINO IR
|
|||
Convert from PyTorch <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_PyTorch>
|
||||
Convert from TensorFlow <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_TensorFlow>
|
||||
Convert from ONNX <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_ONNX>
|
||||
Convert from TensorFlow_Lite <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_TensorFlow_Lite>
|
||||
Convert from TensorFlow Lite <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_TensorFlow_Lite>
|
||||
Convert from PaddlePaddle <openvino_docs_OV_Converter_UG_prepare_model_convert_model_Convert_Model_From_Paddle>
|
||||
|
||||
|
||||
|
|
@ -552,8 +552,8 @@ IR Conversion Benefits
|
|||
| Save storage space, even more so if FP16 is used as it may cut the size by about 50%, especially useful for large models, like Llama2-7B.
|
||||
|
||||
| **Saving to IR to avoid large dependencies in inference code**
|
||||
| Frameworks such as TensorFlow and PyTorch tend to be large dependencies (multiple gigabytes), and not all inference environments have enough space to hold them.
|
||||
| Converting models to OpenVINO IR allows them to be used in an environment where OpenVINO is the only dependency, so much less disk space is needed.
|
||||
| Frameworks such as TensorFlow and PyTorch tend to be large dependencies (multiple gigabytes), and not all inference environments have enough space to hold them.
|
||||
| Converting models to OpenVINO IR allows them to be used in an environment where OpenVINO is the only dependency, so much less disk space is needed.
|
||||
| Loading and compiling with OpenVINO directly usually takes less runtime memory than loading the model in the source framework and then converting and compiling it.
|
||||
|
||||
An example showing how to take advantage of OpenVINO IR, saving a model in OpenVINO IR once, using it many times, is shown below:
|
||||
|
|
|
|||
|
|
@ -1,228 +0,0 @@
|
|||
.. {#openvino_docs_model_processing_introduction_draft}
|
||||
|
||||
Import TensorFlow and PyTorch Models
|
||||
==============================================
|
||||
|
||||
In OpenVINO you can load a model in different formats.
|
||||
The examples below show how TensorFlow and PyTorch models. The models are loaded, converted to OpenVINO format, and compiled for inferencing in just several lines of code.
|
||||
To learn more about how models can be imported in OpenVINO, refer to the :doc:`Model Preparation <openvino_docs_model_processing_introduction>` page.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Import TensorFlow model
|
||||
|
||||
.. code-block:: py
|
||||
:force:
|
||||
|
||||
import openvino as ov
|
||||
|
||||
# 1. Compile model from file
|
||||
core = ov.Core()
|
||||
compiled_model = core.compile_model("model.pb")
|
||||
|
||||
.. tab-item:: Import PyTorch model
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino as ov
|
||||
import torch
|
||||
|
||||
# 1. Convert model loaded from PyTorch file
|
||||
model = torch.load("model.pt")
|
||||
model.eval()
|
||||
ov_model = ov.convert_model(model)
|
||||
|
||||
# 2. Compile model from memory
|
||||
core = ov.Core()
|
||||
compiled_model = core.compile_model(ov_model)
|
||||
|
||||
While the above examples provide a simple and straightforward option to import models into OpenVINO, there are other options to provide more customization and flexibility.
|
||||
|
||||
|
||||
TensorFlow Import Options
|
||||
##############################################
|
||||
|
||||
OpenVINO direct support of TensorFlow allows developers to use their models in an OpenVINO inference pipeline without changes. However, as multiple ways of doing this exist, it may not be clear which is the best approach for a given situation. The following diagram aims to simplify this decision given a certain context, although some additional considerations should be taken into account depending on the use case.
|
||||
|
||||
.. image:: _static/images/import_tensorflow.svg
|
||||
|
||||
|
||||
Method 1. Convert using ov.convert_model function (Python only)
|
||||
---------------------------------------------------------------------
|
||||
|
||||
As seen above, if your starting point is a Python object in memory, for example a ``tf.keras.Model`` or ``tf.Module``, a direct way to get the model in OpenVINO is to use ``ov.convert_model``. This method produces an ``ov.Model`` (one of the three states) that can later be reshaped, saved to OpenVINO IR or compiled to do inference. In code it may look as follows:
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino as ov
|
||||
import tensorflow as tf
|
||||
|
||||
# 1a. Convert model created with TF code
|
||||
model = tf.keras.applications.resnet50.ResNet50(weights="imagenet")
|
||||
ov_model = ov.convert_model(model)
|
||||
|
||||
# 1b. Convert model from file
|
||||
ov_model = ov.convert_model("model.pb")
|
||||
|
||||
|
||||
# 2. Compile model from memory
|
||||
core = ov.Core()
|
||||
compiled_model = core.compile_model(ov_model)
|
||||
|
||||
Method 2. Convert from file using ov.compile_model function
|
||||
---------------------------------------------------------------------
|
||||
|
||||
In case you are starting with a file, you will need to see if the needs to be customized, such as applying quantization or reshaping its inputs.
|
||||
|
||||
If the model does not need to be customized, ``ov.Core.compile_model`` should be used, which reads, converts (if needed) and compiles the model, leaving it ready for inference all in one go. The code should look like this:
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino as ov
|
||||
|
||||
# 1. Compile model from file
|
||||
core = ov.Core()
|
||||
compiled_model = core.compile_model("model.pb")
|
||||
|
||||
Method 3. Convert from file using ov.read_model function
|
||||
---------------------------------------------------------------------
|
||||
|
||||
If the model does need to be customized, ``ov.read_model`` can be used as it just returns an ``ov.Model`` ready to be quantized or have its inputs reshaped. (Note: This method also works with the OpenVINO C++ API, so it is useful for developers working in a C++ environment.)
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino as ov
|
||||
|
||||
# 1. Convert model from file
|
||||
core = ov.Core()
|
||||
ov_model = ov.read_model("model.pb")
|
||||
|
||||
# 2. Compile model from memory
|
||||
compiled_model = core.compile_model(ov_model)
|
||||
|
||||
Method 4. Convert from file using OpenVINO Model Converter (ovc CLI)
|
||||
---------------------------------------------------------------------
|
||||
|
||||
However, if the input reshaping is known in advance and/or the model has multiple outputs but only some of them are required, OpenVINO provides two equivalent ways of doing these while converting the model. One of them is the CLI command ``ovc`` while the other is the previously mentioned ``ov.convert_model`` (Method 1).
|
||||
|
||||
The ``ovc`` tool is similar to ``ov.convert_model``, except it works using the command line rather than a Python environment. It will convert the model to OpenVINO IR format, apply any configurations you specify, and save the converted model to disk. It is useful if you are not working with your model in Python (e.g., if you are developing in a C++ environment) or if you prefer using the command line rather than a Python script.
|
||||
The code below shows how to convert a model with ovc and then load it for inference:
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
# 1. Convert model from file
|
||||
ovc model.pb
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino as ov
|
||||
|
||||
# 2. Load model from file
|
||||
core = ov.Core()
|
||||
ov_model = core.read_model("model.xml")
|
||||
|
||||
# 3. Compile model from memory
|
||||
compiled_model = core.compile_model(ov_model)
|
||||
|
||||
PyTorch Import Options
|
||||
##############################################
|
||||
|
||||
OpenVINO direct support of PyTorch allows developers to use their models in an OpenVINO inference pipeline without changes. OpenVINO provides multiple ways of using PyTorch. The following diagram aims to simplify this decision given a certain context, although some additional considerations should be taken into account depending on the use case.
|
||||
|
||||
.. image:: _static/images/import_pytorch.svg
|
||||
|
||||
PyTorch models can be imported into OpenVINO directly from a Python object. Saved PyTorch files can be used as well. To use a saved PyTorch file, it needs to be loaded in PyTorch first to convert it to a Python object.
|
||||
Once the model is loaded as a PyTorch Python object, you can decide whether to start using the OpenVINO framework and its features directly or to remain within the PyTorch framework while leveraging optimizations.
|
||||
|
||||
Method 1. Convert using ov.convert_model function
|
||||
---------------------------------------------------------------------
|
||||
|
||||
If OpenVINO is preferred, ov.convert_model is the method to use. It produces an ``ov.Model`` that can later be reshaped, saved to OpenVINO IR or compiled to do inference. In code it may look as follows:
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino as ov
|
||||
import torch
|
||||
from torchvision.models import resnet50
|
||||
|
||||
# 1a. Convert model created with PyTorch code
|
||||
model = resnet50(weights="DEFAULT")
|
||||
model.eval()
|
||||
|
||||
ov_model = ov.convert_model(model, example_input=torch.rand(1, 3, 224, 224))
|
||||
|
||||
# 1b. Convert model loaded from PyTorch file
|
||||
model = torch.load("model.pt")
|
||||
model.eval()
|
||||
ov_model = ov.convert_model(model)
|
||||
|
||||
# 2. Compile model from memory
|
||||
core = ov.Core()
|
||||
compiled_model = core.compile_model(ov_model)
|
||||
|
||||
Note that the need to set ``example_input`` depends on the model used. However, it is recommended to always set it if available as it usually leads to a better quality model. For more details, check out the docs.
|
||||
|
||||
Method 2. Use OpenVINO backend in PyTorch
|
||||
---------------------------------------------------------------------
|
||||
|
||||
In case PyTorch syntax is preferred, since PyTorch 2.0 and OpenVINO 2023.1, a PyTorch model can be optimized with OpenVINO by specifying it as a backend in ``torch.compile``.
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import openvino.torch
|
||||
import torch
|
||||
from torchvision.models import resnet50
|
||||
|
||||
# 1a. Compile model created with PyTorch code
|
||||
model = resnet50(weights="DEFAULT")
|
||||
model.eval()
|
||||
compiled_model = torch.compile(model, backend="openvino")
|
||||
|
||||
# 1b. Compile model loaded from PyTorch file
|
||||
model = torch.load("model.pt")
|
||||
model.eval()
|
||||
compiled_model = torch.compile(model, backend="openvino")
|
||||
|
||||
Method 3. Export model to ONNX and use one of OpenVINO methods
|
||||
---------------------------------------------------------------------
|
||||
|
||||
If none of these two methods convert the model successfully, there is a third method that once was the main way of using PyTorch in OpenVINO, but now is mainly considered a backup plan.
|
||||
This method consists of exporting a PyTorch model to ONNX and then loading it with the different methods available in OpenVINO. See ONNX, PaddlePaddle and TensorFlow Lite Import Options for more details.
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
import torch
|
||||
import openvino as ov
|
||||
from torchvision.models import resnet50
|
||||
|
||||
# 1. Export PyTorch model to ONNX
|
||||
model = resnet50(weights="DEFAULT")
|
||||
model.eval()
|
||||
|
||||
dummy_input = torch.randn(1,3,224,224)
|
||||
torch.onnx.export(model, dummy_input, "model.onnx")
|
||||
|
||||
# 2. Use an OpenVINO method to read and compile it, for example compile_model
|
||||
core = ov.Core()
|
||||
compiled_model = core.compile_model("model.onnx")
|
||||
|
||||
Supported Model Formats
|
||||
---------------------------------------------------------------------
|
||||
|
||||
|
||||
As PyTorch does not have a save format that contains everything needed to reproduce the model without using torch, OpenVINO only supports loading Python objects directly. The support is as follows:
|
||||
|
||||
* Python objects
|
||||
|
||||
* torch.nn.Module
|
||||
* torch.jit.ScriptModule
|
||||
* torch.jit.ScriptFunction
|
||||
|
||||
|
||||
Jupyter Notebook Tutorials
|
||||
################################################
|
||||
|
||||
OpenVINO also provides example notebooks for both frameworks showing how to load a model and make inference:
|
||||
|
||||
* `Convert TensorFlow Models to OpenVINO <notebooks/101-tensorflow-classification-to-openvino-with-output.html>`__
|
||||
* `Convert PyTorch Models to OpenVINO <notebooks/102-pytorch-onnx-to-openvino-with-output.html>`__
|
||||
|
|
@ -27,8 +27,8 @@ of arbitrary length, including an empty string, and can be set independently of
|
|||
Depending on the API used (C++ or Python), the underlying data type that represents the string when accessing the tensor elements is
|
||||
different:
|
||||
|
||||
- in C++, std::string is used
|
||||
- in Python, `numpy.str_`/`numpy.bytes_` populated Numpy arrays are used, as a read-only copy of the underlying C++ content
|
||||
* in C++, std::string is used
|
||||
* in Python, `numpy.str_`/`numpy.bytes_` populated Numpy arrays are used, as a read-only copy of the underlying C++ content
|
||||
|
||||
String tensor implementation doesn't imply any limitations on string encoding, as underlying `std::string` doesn't have such limitations.
|
||||
It is capable of representing all valid UTF-8 characters but also any other byte sequence outside of the UTF-8 encoding standard.
|
||||
|
|
|
|||
Loading…
Reference in New Issue