diff --git a/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md b/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md index d5b7ba8ff52..0eb86abd370 100644 --- a/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md +++ b/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md @@ -1,49 +1,62 @@ -# Install & Deployment {#openvino_2_0_deployment} +# Installation & Deployment {#openvino_2_0_deployment} -## Introduction +"Easy to use" is one of the main concepts for OpenVINO™ API 2.0. It includes not only simplifying the migration from frameworks to OpenVINO, but also how OpenVINO is organized, how the development tools are used, and how to develop and deploy OpenVINO-based applications. -"Easy to use" is one of the main points for OpenVINO 2.0 concept. This includes not only simplification of migration from frameworks to OpenVINO Toolkit, but also how OpenVINO is organized, how the development tools are used, how to develop and deploy OpenVINO-based applications. +To accomplish that, we have made some changes on the installation and deployment of OpenVINO in the 2022.1 release. This guide will walk you through them. -Starting from OpenVINO 2.0, Model Optimizer, Post Training Optimization tool, Open Model Zoo tools and other Python-based Development tools are distributed via [PyPi](https://pypi.org/project/openvino-dev/) only. This simplifies user experience, because in older versions of OpenVINO development tools should be installed and used differently dependning on distribution type (OpenVINO Installer versus PyPi), but also ensures that dependencies are handled properly via `pip` package manager as well as supports virtual environments for development tools. +## Installer Package Contains OpenVINO™ Runtime Only -Regarding the OpenVINO Installer, we have tried further to reorganize package structure to have it in a good shape: +Starting from OpenVINO 2022.1, Model Optimizer, Post-Training Optimization tool and Python-based Development tools such as Open Model Zoo tools are distributed via [PyPI](https://pypi.org/project/openvino-dev/) only, and are no longer included in the OpenVINO installer package. This change has several benefits as it: -- `runtime` folder with OpenVINO Runtime include headers, libraries and CMake interfaces. -- `tools` folder with [compile_tool](../../../tools/compile_tool/README.md), [deployment manager](../../install_guides/deployment-manager-tool.md), `requirement.txt` files with link to corresponding version of `openvino-dev` package. -- `python` folder with OpenVINO Python Runtime. +* Simplifies the user experience. In previous versions, the installation and usage of OpenVINO Development Tools differ according to the distribution type (via an OpenVINO installer or PyPI). +* Ensures that dependencies are handled properly via the PIP package manager and support virtual environments of development tools. -## Installing development tools +The structure of OpenVINO 2022.1 installer package has been organized as below: -In older versions of OpenVINO, Development tools were a part of main package. Once the package is installed, users need to install additional tools dependencies (e.g. `requirements_tf.txt` to convert TensorFlow models via Model Optimizer, install POT and AC tools via `setup.py` scripts), then use `setupvars` scripts to make the tools available in command line: +- The `runtime` folder includes headers, libraries and CMake interfaces. +- The `tools` folder contains [the compile tool](../../../tools/compile_tool/README.md), [deployment manager](../../install_guides/deployment-manager-tool.md) and a set of `requirements.txt` files with links to the corresponding versions of the `openvino-dev` package. +- The `python` folder contains the Python version for OpenVINO Runtime. + +## Installing OpenVINO Development Tools via PyPI + +Since OpenVINO Development Tools is no longer in the installer package, the installation process has changed too. This section describes it through a comparison with previous versions. + +### For Versions Prior to 2022.1 + +In previous versions, OpenVINO Development Tools is a part of main package. After the package is installed, to convert models (for example, TensorFlow), you need to install additional dependencies by using the requirements files such as `requirements_tf.txt`, install Post-Training Optimization tool and Accuracy Checker tool via the `setup.py` scripts, and then use the `setupvars` scripts to make the tools available to the following command: ```sh $ mo.py -h ``` -Starting with OpenVINO 2.0 users can install development tools only from [PyPi](https://pypi.org/project/openvino-dev/) repository: +### For 2022.1 and After -``` -$ python3 -m pip install -r /tools/requirements_tf.txt +Starting from OpenVINO 2022.1, you can install the development tools from [PyPI](https://pypi.org/project/openvino-dev/) repository only, using the following command (taking TensorFlow as an example): + +```sh +$ python3 -m pip install -r /tools/requirements_tf.txt ``` -Which installs all OpenVINO development tools via the `openvino-dev` package and TensorFlow as an extra (see [Step 4. Install the Package](https://pypi.org/project/openvino-dev/) for details). +This will install all the development tools and additional necessary components to work with TensorFlow via the `openvino-dev` package (see **Step 4. Install the Package** on the [PyPI page](https://pypi.org/project/openvino-dev/) for parameters of other frameworks). -Then, tools can be used as: +Then, the tools can be used by commands like: ```sh $ mo -h $ pot -h ``` -Without installations of other dependencies. See [Install OpenVINO™ Development Tools](../../install_guides/installing-model-dev-tools.md) for more details. +You don't have to install any other dependencies. For more details on the installation steps, see [Install OpenVINO Development Tools](../../install_guides/installing-model-dev-tools.md). -## Build C / C++ applications +## Interface Changes for Building C/C++ Applications -### CMake interface +The new OpenVINO Runtime with API 2.0 has also brought some changes for builiding your C/C++ applications. -Since OpenVINO 2.0 introduced new OpenVINO 2.0 Runtime, the CMake interface is changed as well. +### CMake Interface -Inference Engine: +The CMake interface has been changed as below: + +**With Inference Engine of previous versions**: ```cmake find_package(InferenceEngine REQUIRED) @@ -52,7 +65,7 @@ add_executable(ie_ngraph_app main.cpp) target_link_libraries(ie_ngraph_app PRIVATE ${InferenceEngine_LIBRARIES} ${NGRAPH_LIBRARIES}) ``` -OpenVINO Runtime 2.0: +**With OpenVINO Runtime 2022.1 (API 2.0)**: ```cmake find_package(OpenVINO REQUIRED) @@ -63,48 +76,48 @@ add_executable(ov_c_app main.c) target_link_libraries(ov_c_app PRIVATE openvino::runtime::c) ``` -### Native interfaces +### Native Interfaces -In order to build application without CMake interface, users could use MSVC IDE, Unix makefiles and any other interfaces. +To build applications without CMake interface, you can also use MSVC IDE, UNIX makefiles and any other interfaces, which have been changed as below: -Older OpenVINO packages: +**With Inference Engine of previous versions**: @sphinxdirective .. tab:: Include dirs .. code-block:: sh - - /deployment_tools/inference_engine/include - /deployment_tools/ngraph/include + + /deployment_tools/inference_engine/include + /deployment_tools/ngraph/include .. tab:: Path to libs .. code-block:: sh - /deployment_tools/inference_engine/lib/intel64/Release - /deployment_tools/ngraph/lib/ + /deployment_tools/inference_engine/lib/intel64/Release + /deployment_tools/ngraph/lib/ .. tab:: Shared libs .. code-block:: sh - # Unix systems + // UNIX systems inference_engine.so ngraph.so - # Windows OS + // Windows inference_engine.dll ngraph.dll -.. tab:: (Win) .lib files +.. tab:: (Windows) .lib files .. code-block:: sh - + ngraph.lib inference_engine.lib @endsphinxdirective -OpenVINO 2.0 package: +**With OpenVINO Runtime 2022.1 (API 2.0)**: @sphinxdirective @@ -112,25 +125,25 @@ OpenVINO 2.0 package: .. code-block:: sh - /runtime/include + /runtime/include .. tab:: Path to libs .. code-block:: sh - /runtime/lib/intel64/Release + /runtime/lib/intel64/Release .. tab:: Shared libs .. code-block:: sh - # Unix systems + // UNIX systems openvino.so - # Windows OS + // Windows openvino.dll -.. tab:: (Win) .lib files +.. tab:: (Windows) .lib files .. code-block:: sh @@ -138,10 +151,24 @@ OpenVINO 2.0 package: @endsphinxdirective -## Deployment +## Clearer Library Structure for Deployment -### Libraries reorganization +OpenVINO 2022.1 has reorganized the libraries to make it easier for deployment. In previous versions, to perform deployment steps, you have to use several libraries. Now you can just use `openvino` or `openvino_c` based on your developing language plus necessary plugins to complete your task. For example, `openvino_intel_cpu_plugin` and `openvino_ir_frontend` plugins will enable you to load OpenVINO IRs and perform inference on CPU device. +Here you can find some detailed comparisons on library structure between OpenVINO 2022.1 and previous versions: + +* A single core library with all the functionalities (`openvino` for C++ Runtime, `openvino_c` for Inference Engine API C interface) is used in 2022.1, instead of the previous core libraries which contain `inference_engine`, `ngraph`, `inference_engine_transformations` and `inference_engine_lp_transformations`. +* The optional `inference_engine_preproc` preprocessing library (if `InferenceEngine::PreProcessInfo::setColorFormat` or `InferenceEngine::PreProcessInfo::setResizeAlgorithm` is used) is renamed as `openvino_gapi_preproc` and deprecated in 2022.1. See more details on [Preprocessing capabilities of OpenVINO API 2.0](preprocessing.md). +* The libraries of plugins are renamed as below: + * `openvino_intel_cpu_plugin` is used for [CPU](../supported_plugins/CPU.md) device instead of `MKLDNNPlugin` in previous versions. + * `openvino_intel_gpu_plugin` is used for [GPU](../supported_plugins/GPU.md) device instead of `clDNNPlugin` in previous versions. + * `openvino_auto_plugin` is used for [Auto-Device Plugin](../auto_device_selection.md) in 2022.1. +* The plugins for reading and converting models have been changed as below: + * `openvino_ir_frontend` is used to read IRs instead of `inference_engine_ir_reader` in previous versions. + * `openvino_onnx_frontend` is used to read ONNX models instead of `inference_engine_onnx_reader` (with its dependencies) in previous versions. + * `openvino_paddle_frontend` is added in 2022.1 to read PaddlePaddle models. + + \ No newline at end of file diff --git a/docs/OV_Runtime_UG/migration_ov_2_0/intro.md b/docs/OV_Runtime_UG/migration_ov_2_0/intro.md index b8f886087d6..a225be5e485 100644 --- a/docs/OV_Runtime_UG/migration_ov_2_0/intro.md +++ b/docs/OV_Runtime_UG/migration_ov_2_0/intro.md @@ -16,7 +16,7 @@ ### Introduction -Older versions of OpenVINO (prior to 2022.1) required to change the logic of applications when an user migrates from the frameworks like TensorFlow, ONNX Runtime, PyTorch, PaddlePaddle, etc. The change of application's logic is connected with: +Older versions of OpenVINO™ (prior to 2022.1) required to change the logic of applications when an user migrates from the frameworks like TensorFlow, ONNX Runtime, PyTorch, PaddlePaddle, etc. The change of application's logic is connected with: - Model Optimizer changed input precisions for some inputs. For example, neural langauge processing models with `I64` input are becoming to have `I32` input element type. - Model Optimizer changed layouts for TensorFlow models (see [Layouts in OpenVINO](../layout_overview.md)). It leads to unexpected user behavior that a user needs to use a different layout for its input data with compare to the framework: @@ -24,59 +24,57 @@ Older versions of OpenVINO (prior to 2022.1) required to change the logic of app - Inference Engine API (`InferenceEngine::CNNNetwork`) also applied some conversion rules for input and output precisions because of device plugins limitations. - Users need to specify input shapes during model conversions in Model Optimizer and work with static shapes in the application. -OpenVINO Runtime API 2.0 is introduced to align logic of working with model as it is done in the frameworks - no layout and precision changes, operates with tensor names and indeces to address inputs and outputs. OpenVINO Runtime is composed of Inference Engine API used for inference and ngraph API targeted to work with models, operations. The OpenVINO API 2.0 has common structure, naming convention styles, namespaces, removes duplicated structures. See [How to migrate to OpenVINO 2.0 API](./common_inference_pipeline.md) for details. +OpenVINO API 2.0 is introduced to align logic of working with model as it is done in the frameworks - no layout and precision changes, operates with tensor names and indeces to address inputs and outputs. OpenVINO Runtime is composed of Inference Engine API used for inference and ngraph API targeted to work with models, operations. The API 2.0 has common structure, naming convention styles, namespaces, removes duplicated structures. See [How to migrate to API 2.0](./common_inference_pipeline.md) for details. -> **NOTE**: Most important is that your existing application can continue working with OpenVINO Runtime 2.0 as it used to be, but we recommend migration to new API to unlock additional features like [Preprocessing](../preprocessing_overview.md) and [Dynamic shapes support](../ov_dynamic_shapes.md). +> **NOTE**: Most important is that your existing application can continue working with OpenVINO Runtime 2022.1 as it used to be, but we recommend migration to API 2.0 to unlock additional features like [Preprocessing](../preprocessing_overview.md) and [Dynamic shapes support](../ov_dynamic_shapes.md). ### Introduce IR v11 -To support these features, OpenVINO introduced IR v11 which is generated by Model Optimizer by default since 2022.1. The model represented in IR v11 fully matches the original model in a original framework format in terms of inputs and outputs. Also, a user does not have to specify input shapes during the conversion, so the resulting IR v11 contains `-1` to denote undefined dimensions (see [Working with dynamic shapes](../ov_dynamic_shapes.md) to fully utilize this feature; or [Changning input shapes](../ShapeInference.md) to reshape to static shapes in the application). +To support these features, OpenVINO introduced IR v11 which is generated by Model Optimizer by default since 2022.1. The model represented in IR v11 fully matches the original model in a original framework format in terms of inputs and outputs. Also, a user does not have to specify input shapes during the conversion, so the resulting IR v11 contains `-1` to denote undefined dimensions (see [Working with dynamic shapes](../ov_dynamic_shapes.md) to fully utilize this feature; or [Changing input shapes](../ShapeInference.md) to reshape to static shapes in the application). What is also important to mention - the IR v11 is fully compatible with old applications written with Inference Engine API from older versions of OpenVINO. This is achieved by adding additional runtime information to the IR v11 which is responsible for backwark compatible behavior. So, once the IR v11 is read by the old Inference Engine based application, it's internally converted to IR v10 to provide backward-compatible behavior. -The IR v11 is supported by all OpenVINO Development tools including Post Training Optimization tool, Benchmark app, etc. +The IR v11 is supported by all OpenVINO Development tools including Post-Training Optimization tool, Benchmark app, etc. -### IR v10 compatibility +### IR v10 Compatibility -OpenVINO Runtime API 2.0 also supports model in IR v10 for backward compatibility. So, if a user has an IR v10, such IR v10 can be fed to OpenVINO Runtime as well (see [migration steps](./common_inference_pipeline.md)). +OpenVINO API 2.0 also supports model in IR v10 for backward compatibility. So, if a user has an IR v10, such IR v10 can be fed to OpenVINO Runtime as well (see [migration steps](./common_inference_pipeline.md)). Some OpenVINO Development Tools also support both IR v10 and IR v11 as an input: - Accuracy checker also supports IR v10, but requires an additional option to denote which API is used underneath. -- [Compile tool](../../../tools/compile_tool/README.md) compiles the model to be used in OpenVINO 2.0 API by default. If a user wants to use the resulting compiled blob in Inference Engine API, the additional `ov_api_1_0` option should be passed. +- [Compile tool](../../../tools/compile_tool/README.md) compiles the model to be used in API 2.0 by default. If a user wants to use the resulting compiled blob in Inference Engine API, the additional `ov_api_1_0` option should be passed. But the following OpenVINO tools don't support IR v10 as an input, they require to regenerate an IR v11 from the original model with latest Model Optimizer: - Post Training Optimization tool - Deep Learning WorkBench -> **NOTE**: If IR v10 models need to be quantized, the recommendation is to download and use Post Training Optimization tool from OpenVINO 2021.4 release. +### Differences between Inference Engine and OpenVINO Runtime 2022.1 -### Differences between Inference Engine and OpenVINO Runtime 2.0 - -Inference Engine and ngraph APIs are not deprecated, they are fully functional and can be used in applications. But OpenVINO recommends users to migrate to new OpenVINO Runtime API 2.0, because it already has additional features and this list will be extended later. The following list of additional features is supported by new API: +Inference Engine and ngraph APIs are not deprecated, they are fully functional and can be used in applications. But we recommend you to migrate to API 2.0, because it already has additional features and this list will be extended later. The following list of additional features is supported by API 2.0: - [Working with dynamic shapes](../ov_dynamic_shapes.md). The feature is quite usefull for best performance for NLP (Neural Language Processing) models, super resolution models and other which accepts dynamic input shapes. - [Preprocessing of the model](../preprocessing_overview.md) to add preprocessing operations to the inference models and fully ocupay the accelerator and free CPU resources. -To define a difference on the API level between Inference Engine and OpenVINO RUntime API 2.0, let's define two types of behaviors: +To define a difference on the API level between Inference Engine and API 2.0, let's define two types of behaviors: - **Old behavior** of OpenVINO supposes: - Model Optimizer can change input element types, order of dimensions (layouts) with compare to the model from the original framework. - Inference Engine can override input and output element types. - Inference Engine API operates with operation names to address inputs and outputs (e.g. InferenceEngine::InferRequest::GetBlob). - Does not support compiling of models with dynamic input shapes. -- **New behavior** assumes full model aligment with the framework and is implemented in OpenVINO 2.0: +- **New behavior** assumes full model aligment with the framework and is implemented in OpenVINO 2022.1: - Model Optimizer preserves the input element types, order of dimensions (layouts) and stores tensor names from the original models. - - OpenVINO Runtime 2.0 reads models in any formats (IR v10, IR v11, ONNX, PaddlePaddle, etc) as is. - - OpenVINO Runtime API 2.0 operates with tensor names. Note, the difference between tensor names and operations names is that in case if a single operation has several output tensors, such tensors cannot identified in a unique manner, so tensor names are used for addressing as it's usually done in the frameworks. - - OpenVINO Runtime API 2.0 can address input and outputs tensors also by its index. Some model formats like ONNX are sensitive to order of inputs, outputs and its preserved by OpenVINO Runtime 2.0. + - OpenVINO Runtime 2022.1 reads models in any formats (IR v10, IR v11, ONNX, PaddlePaddle, etc) as is. + - API 2.0 operates with tensor names. Note, the difference between tensor names and operations names is that in case if a single operation has several output tensors, such tensors cannot identified in a unique manner, so tensor names are used for addressing as it's usually done in the frameworks. + - API 2.0 can address input and outputs tensors also by its index. Some model formats like ONNX are sensitive to order of inputs, outputs and its preserved by OpenVINO Runtime 2022.1. The table below demonstrates which behavior **old** or **new** is used depending on a model source, used APIs. | API | IR v10 | IR v11 | ONNX file | Model created in code | |-------------------------------|---------|---------|-----------|-----------------------| |Inference Engine / ngraph APIs | Old | Old | Old | Old | -|OpenVINO Runtime API 2.0 | Old | New | New | New | +|API 2.0 | Old | New | New | New | -Please look at next transition guides to understand how migrate Inference Engine-based application to OpenVINO™ Runtime API 2.0: - - [Install & deployment](deployment_migration.md) +Please look at next transition guides to understand how migrate Inference Engine-based application to API 2.0: + - [Installation & Deployment](deployment_migration.md) - [OpenVINO™ Common Inference pipeline](common_inference_pipeline.md) - [Preprocess your model](./preprocessing.md) - [Configure device](./configure_devices.md) diff --git a/docs/get_started.md b/docs/get_started.md index c55b1055421..9b6c49224a9 100644 --- a/docs/get_started.md +++ b/docs/get_started.md @@ -69,11 +69,7 @@

Choose how you want to progress:

-