natas_hw
/
openvino
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[DOCS] Compile tool docs port (#17753) 

* [DOCS] Compile tool docs change (#17460)

* add compile tool description

* change refs

* remove page to build docs

* doc reference fix

* review comments

* fix comment

* snippet comment

* Update docs/snippets/compile_model.cpp

Co-authored-by: Ilya Churaev <ilyachur@gmail.com>

* change snippet name

* create ov object

* code block fix

* cpp code block

* include change

* code test

* change snippet

* Update docs/snippets/export_compiled_model.cpp

Co-authored-by: Ilya Churaev <ilyachur@gmail.com>

---------

Co-authored-by: Ilya Churaev <ilyachur@gmail.com>

* Fixed compile_tool install (#17666)

---------

Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
This commit is contained in:
Tatiana Savina 2023-05-31 11:27:01 +02:00 committed by GitHub
parent c18a24c05b
commit 4fb2cebf28
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 32 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
docs/Documentation/openvino_ecosystem.md
View File

@ -8,13 +8,13 @@
ote_documentation
ovsa_get_started
openvino_inference_engine_tools_compile_tool_README
openvino_docs_tuning_utilities
OpenVINO™ is not just one tool. It is an expansive ecosystem of utilities, providing a comprehensive workflow for deep learning solution development. Learn more about each of them to reach the full potential of OpenVINO™ Toolkit.
**Neural Network Compression Framework (NNCF)**
A suite of advanced algorithms for Neural Network inference optimization with minimal accuracy drop. NNCF applies quantization, filter pruning, binarization and sparsity algorithms to PyTorch and TensorFlow models during training.
@ -36,6 +36,7 @@ More resources:
* `GitHub <https://github.com/openvinotoolkit/training_extensions>`__
* `Documentation <https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/introduction.html>`__
**OpenVINO™ Security Add-on**
A solution for Model Developers and Independent Software Vendors to use secure packaging and secure model execution.

3
docs/OV_Runtime_UG/Model_caching_overview.md
View File

@ -26,9 +26,6 @@ automatically and reuses it to significantly reduce the model compilation time.
.. important::
The :doc:`Compile Tool <openvino_inference_engine_tools_compile_tool_README>` may serve the same purpose
for C++ applications, but is considered a legacy solution and you should use Model Caching instead.
Not all devices support the network import/export feature. They will perform normally but will not
enable the compilation stage speed-up.

2
docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md
View File

@ -23,7 +23,7 @@ Since OpenVINO 2022.1, development tools have been distributed only via `PyPI <h
The structure of the OpenVINO 2022.1 installer package has been organized as follows:
* The ``runtime`` folder includes headers, libraries and CMake interfaces.
* The ``tools`` folder contains :doc:`the compile tool <openvino_inference_engine_tools_compile_tool_README>`, :doc:`deployment manager <openvino_docs_install_guides_deployment_manager_tool>`, and a set of ``requirements.txt`` files with links to the corresponding versions of the ``openvino-dev`` package.
* The ``tools`` folder contains :doc:`the compile tool <openvino_ecosystem>`, :doc:`deployment manager <openvino_docs_install_guides_deployment_manager_tool>`, and a set of ``requirements.txt`` files with links to the corresponding versions of the ``openvino-dev`` package.
* The ``python`` folder contains the Python version for OpenVINO Runtime.
Installing OpenVINO Development Tools via PyPI

2
docs/OV_Runtime_UG/migration_ov_2_0/intro.md
View File

@ -53,7 +53,7 @@ API 2.0 also supports backward compatibility for models of OpenVINO IR v10. If y
Some of the OpenVINO Development Tools also support both OpenVINO IR v10 and v11 as an input:
- Accuracy checker uses API 2.0 for model accuracy measurement by default. It also supports switching to the old API by using the ``--use_new_api False`` command-line parameter. Both launchers accept OpenVINO IR v10 and v11, but in some cases configuration files should be updated. For more details, see the `Accuracy Checker documentation <https://github.com/openvinotoolkit/open_model_zoo/blob/master/tools/accuracy_checker/openvino/tools/accuracy_checker/launcher/openvino_launcher_readme.md>`__.
- :doc:`Compile tool <openvino_inference_engine_tools_compile_tool_README>` compiles the model to be used in API 2.0 by default. To use the resulting compiled blob under the Inference Engine API, the additional ``ov_api_1_0`` option should be passed.
- :doc:`Compile tool <openvino_ecosystem>` compiles the model to be used in API 2.0 by default. To use the resulting compiled blob under the Inference Engine API, the additional ``ov_api_1_0`` option should be passed.
However, Post-Training Optimization Tool of OpenVINO 2022.1 does not support OpenVINO IR v10. They require the latest version of Model Optimizer to generate OpenVINO IR v11 files.

2
docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md
View File

@ -45,7 +45,7 @@ The table below demonstrates support of key features by OpenVINO device plugins.
:doc:`Multi-stream execution <openvino_docs_deployment_optimization_guide_tput>` Yes Yes No Yes
:doc:`Models caching <openvino_docs_OV_UG_Model_caching_overview>` Yes Partial Yes No
:doc:`Dynamic shapes <openvino_docs_OV_UG_DynamicShapes>` Yes Partial No No
:doc:`Import/Export <openvino_inference_engine_tools_compile_tool_README>` Yes No Yes No
:doc:`Import/Export <openvino_ecosystem>` Yes No Yes No
:doc:`Preprocessing acceleration <openvino_docs_OV_UG_Preprocessing_Overview>` Yes Yes No Partial
:doc:`Stateful models <openvino_docs_OV_UG_model_state_intro>` Yes No Yes No
:doc:`Extensibility <openvino_docs_Extensibility_UG_Intro>` Yes Yes No No

2
docs/OV_Runtime_UG/supported_plugins/GNA.md
View File

@ -203,7 +203,7 @@ Import model:
:fragment: [ov_gna_import]
To compile a model, use either :doc:`compile Tool <openvino_inference_engine_tools_compile_tool_README>` or
To compile a model, use either :ref:`compile Tool <openvino_ecosystem>` or
:doc:`Speech C++ Sample <openvino_inference_engine_samples_speech_sample_README>`.
Stateful Models

19
docs/snippets/export_compiled_model.cpp Normal file
View File

@ -0,0 +1,19 @@
#include <openvino/runtime/core.hpp>
int main() {
//! [export_compiled_model]
ov::Core core;
std::stringstream stream;
ov::CompiledModel model = core.compile_model("modelPath", "deviceName");
model.export_model(stream);
//! [export_compiled_model]
return 0;
}

7
docs/snippets/export_compiled_model.py Normal file
View File

@ -0,0 +1,7 @@
#! [export_compiled_model]
from openvino.runtime import Core
ov.Core().compile_model(device, modelPath, properties).export_model(compiled_blob)
#! [export_compiled_model]

3
tools/compile_tool/CMakeLists.txt
View File

@ -47,7 +47,4 @@ else()
install(TARGETS compile_tool
RUNTIME DESTINATION ${OV_CPACK_TOOLSDIR}/compile_tool
COMPONENT ${OV_CPACK_COMP_CORE_TOOLS})
install(FILES README.md
DESTINATION ${OV_CPACK_TOOLSDIR}/compile_tool
COMPONENT ${OV_CPACK_COMP_CORE_TOOLS})
endif()

89
tools/compile_tool/README.md
View File

@ -1,89 +0,0 @@
# Compile Tool {#openvino_inference_engine_tools_compile_tool_README}
@sphinxdirective
Compile tool is a C++ application that enables you to compile a model for inference on a specific device and export the compiled representation to a binary file.
With this tool, you can compile a model using supported OpenVINO Runtime devices on a machine that does not have the physical device connected, and then transfer a generated file to any machine with the target inference device available. To learn which device supports the import / export functionality, see the :doc:`feature support matrix <openvino_docs_OV_UG_Working_with_devices>`.
The tool is delivered as an executable file that can be run on both Linux and Windows. It is located in the ``<INSTALLROOT>/tools/compile_tool`` directory.
Workflow of the Compile tool
############################
First, the application reads command-line parameters and loads a model to the OpenVINO Runtime device. After that, the application exports a blob with the compiled model and writes it to the output file.
Also, the Compile tool supports the following capabilities:
- Embedding :doc:`layout <openvino_docs_OV_UG_Layout_Overview>` and precision conversions (for more details, see the :doc:`Optimize Preprocessing <openvino_docs_OV_UG_Preprocessing_Overview>`). To compile the model with advanced preprocessing capabilities, refer to the :doc:`Use Case - Integrate and Save Preprocessing Steps Into OpenVINO IR <openvino_docs_OV_UG_Preprocess_Usecase_save>`, which shows how to have all the preprocessing in the compiled blob.
- Compiling blobs for OpenVINO Runtime API 2.0 by default or for Inference Engine API with explicit option ``-ov_api_1_0``.
- Accepting device specific options for customizing the compilation process.
Running the Compile Tool
########################
Running the application with the ``-h`` option yields the following usage message:
.. code-block:: bash
./compile_tool -h
OpenVINO Runtime version ......... 2022.1.0
Build ........... custom_changed_compile_tool_183a1adfcd7a001974fe1c5cfa21ec859b70ca2c
compile_tool [OPTIONS]
Common options:
-h Optional. Print the usage message.
-m <value> Required. Path to the XML model.
-d <value> Required. Specify a target device for which executable network will be compiled.
Use "-d HETERO:<comma-separated_devices_list>" format to specify HETERO plugin.
Use "-d MULTI:<comma-separated_devices_list>" format to specify MULTI plugin.
The application looks for a suitable plugin for the specified device.
-o <value> Optional. Path to the output file. Default value: "<model_xml_file>.blob".
-c <value> Optional. Path to the configuration file.
-ip <value> Optional. Specifies precision for all input layers of the network.
-op <value> Optional. Specifies precision for all output layers of the network.
-iop "<value>" Optional. Specifies precision for input and output layers by name.
Example: -iop "input:FP16, output:FP16".
Notice that quotes are required.
Overwrites precision from ip and op options for specified layers.
-il <value> Optional. Specifies layout for all input layers of the network.
-ol <value> Optional. Specifies layout for all output layers of the network.
-iol "<value>" Optional. Specifies layout for input and output layers by name.
Example: -iol "input:NCHW, output:NHWC".
Notice that quotes are required.
Overwrites layout from il and ol options for specified layers.
-iml <value> Optional. Specifies model layout for all input layers of the network.
-oml <value> Optional. Specifies model layout for all output layers of the network.
-ioml "<value>" Optional. Specifies model layout for input and output tensors by name.
Example: -ionl "input:NCHW, output:NHWC".
Notice that quotes are required.
Overwrites layout from il and ol options for specified layers.
-ov_api_1_0 Optional. Compile model to legacy format for usage in Inference Engine API,
by default compiles to OV 2.0 API
Running the application with the empty list of options yields an error message.
For example, to compile a blob for inference on an Intel® Neural Compute Stick 2 from a trained network, run the command below:
.. code-block:: bash
./compile_tool -m <path_to_model>/model_name.xml -d CPU
Stating flags that take only single option like `-m` multiple times, for example `./compile_tool -m model.xml -m model2.xml`, results in only the first value being used.
Import a Compiled Blob File to Your Application
+++++++++++++++++++++++++++++++++++++++++++++++
To import a blob with the network from a generated file into your application, use the
``ov::Core::import_model`` method:
.. code-block:: cpp
ov::Core ie;
std::ifstream file{"model_name.blob"};
ov::CompiledModel compiled_model = ie.import_model(file, "CPU");
@endsphinxdirective