From 4fb2cebf28b14d65ab425c613b217d3add9ad755 Mon Sep 17 00:00:00 2001 From: Tatiana Savina Date: Wed, 31 May 2023 11:27:01 +0200 Subject: [PATCH] [DOCS] Compile tool docs port (#17753) * [DOCS] Compile tool docs change (#17460) * add compile tool description * change refs * remove page to build docs * doc reference fix * review comments * fix comment * snippet comment * Update docs/snippets/compile_model.cpp Co-authored-by: Ilya Churaev * change snippet name * create ov object * code block fix * cpp code block * include change * code test * change snippet * Update docs/snippets/export_compiled_model.cpp Co-authored-by: Ilya Churaev --------- Co-authored-by: Ilya Churaev * Fixed compile_tool install (#17666) --------- Co-authored-by: Ilya Churaev Co-authored-by: Ilya Churaev --- docs/Documentation/openvino_ecosystem.md | 3 +- docs/OV_Runtime_UG/Model_caching_overview.md | 3 - .../migration_ov_2_0/deployment_migration.md | 2 +- docs/OV_Runtime_UG/migration_ov_2_0/intro.md | 2 +- .../supported_plugins/Device_Plugins.md | 2 +- docs/OV_Runtime_UG/supported_plugins/GNA.md | 2 +- docs/snippets/export_compiled_model.cpp | 19 ++++ docs/snippets/export_compiled_model.py | 7 ++ tools/compile_tool/CMakeLists.txt | 3 - tools/compile_tool/README.md | 89 ------------------- 10 files changed, 32 insertions(+), 100 deletions(-) create mode 100644 docs/snippets/export_compiled_model.cpp create mode 100644 docs/snippets/export_compiled_model.py delete mode 100644 tools/compile_tool/README.md diff --git a/docs/Documentation/openvino_ecosystem.md b/docs/Documentation/openvino_ecosystem.md index eaac4c561c1..da5c78020fd 100644 --- a/docs/Documentation/openvino_ecosystem.md +++ b/docs/Documentation/openvino_ecosystem.md @@ -8,13 +8,13 @@ ote_documentation ovsa_get_started - openvino_inference_engine_tools_compile_tool_README openvino_docs_tuning_utilities OpenVINO™ is not just one tool. It is an expansive ecosystem of utilities, providing a comprehensive workflow for deep learning solution development. Learn more about each of them to reach the full potential of OpenVINO™ Toolkit. + **Neural Network Compression Framework (NNCF)** A suite of advanced algorithms for Neural Network inference optimization with minimal accuracy drop. NNCF applies quantization, filter pruning, binarization and sparsity algorithms to PyTorch and TensorFlow models during training. @@ -36,6 +36,7 @@ More resources: * `GitHub `__ * `Documentation `__ + **OpenVINO™ Security Add-on** A solution for Model Developers and Independent Software Vendors to use secure packaging and secure model execution. diff --git a/docs/OV_Runtime_UG/Model_caching_overview.md b/docs/OV_Runtime_UG/Model_caching_overview.md index 3199ce2c153..dd80f193ea6 100644 --- a/docs/OV_Runtime_UG/Model_caching_overview.md +++ b/docs/OV_Runtime_UG/Model_caching_overview.md @@ -26,9 +26,6 @@ automatically and reuses it to significantly reduce the model compilation time. .. important:: - The :doc:`Compile Tool ` may serve the same purpose - for C++ applications, but is considered a legacy solution and you should use Model Caching instead. - Not all devices support the network import/export feature. They will perform normally but will not enable the compilation stage speed-up. diff --git a/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md b/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md index 46d8a693094..41e55e7fe79 100644 --- a/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md +++ b/docs/OV_Runtime_UG/migration_ov_2_0/deployment_migration.md @@ -23,7 +23,7 @@ Since OpenVINO 2022.1, development tools have been distributed only via `PyPI `, :doc:`deployment manager `, and a set of ``requirements.txt`` files with links to the corresponding versions of the ``openvino-dev`` package. +* The ``tools`` folder contains :doc:`the compile tool `, :doc:`deployment manager `, and a set of ``requirements.txt`` files with links to the corresponding versions of the ``openvino-dev`` package. * The ``python`` folder contains the Python version for OpenVINO Runtime. Installing OpenVINO Development Tools via PyPI diff --git a/docs/OV_Runtime_UG/migration_ov_2_0/intro.md b/docs/OV_Runtime_UG/migration_ov_2_0/intro.md index 80438604a56..e0e579ce91c 100644 --- a/docs/OV_Runtime_UG/migration_ov_2_0/intro.md +++ b/docs/OV_Runtime_UG/migration_ov_2_0/intro.md @@ -53,7 +53,7 @@ API 2.0 also supports backward compatibility for models of OpenVINO IR v10. If y Some of the OpenVINO Development Tools also support both OpenVINO IR v10 and v11 as an input: - Accuracy checker uses API 2.0 for model accuracy measurement by default. It also supports switching to the old API by using the ``--use_new_api False`` command-line parameter. Both launchers accept OpenVINO IR v10 and v11, but in some cases configuration files should be updated. For more details, see the `Accuracy Checker documentation `__. -- :doc:`Compile tool ` compiles the model to be used in API 2.0 by default. To use the resulting compiled blob under the Inference Engine API, the additional ``ov_api_1_0`` option should be passed. +- :doc:`Compile tool ` compiles the model to be used in API 2.0 by default. To use the resulting compiled blob under the Inference Engine API, the additional ``ov_api_1_0`` option should be passed. However, Post-Training Optimization Tool of OpenVINO 2022.1 does not support OpenVINO IR v10. They require the latest version of Model Optimizer to generate OpenVINO IR v11 files. diff --git a/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md b/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md index 7b401097c81..0f2eeca5d8d 100644 --- a/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md +++ b/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md @@ -45,7 +45,7 @@ The table below demonstrates support of key features by OpenVINO device plugins. :doc:`Multi-stream execution ` Yes Yes No Yes :doc:`Models caching ` Yes Partial Yes No :doc:`Dynamic shapes ` Yes Partial No No - :doc:`Import/Export ` Yes No Yes No + :doc:`Import/Export ` Yes No Yes No :doc:`Preprocessing acceleration ` Yes Yes No Partial :doc:`Stateful models ` Yes No Yes No :doc:`Extensibility ` Yes Yes No No diff --git a/docs/OV_Runtime_UG/supported_plugins/GNA.md b/docs/OV_Runtime_UG/supported_plugins/GNA.md index 1b07d5de1bf..65b432306d5 100644 --- a/docs/OV_Runtime_UG/supported_plugins/GNA.md +++ b/docs/OV_Runtime_UG/supported_plugins/GNA.md @@ -203,7 +203,7 @@ Import model: :fragment: [ov_gna_import] -To compile a model, use either :doc:`compile Tool ` or +To compile a model, use either :ref:`compile Tool ` or :doc:`Speech C++ Sample `. Stateful Models diff --git a/docs/snippets/export_compiled_model.cpp b/docs/snippets/export_compiled_model.cpp new file mode 100644 index 00000000000..912e752478d --- /dev/null +++ b/docs/snippets/export_compiled_model.cpp @@ -0,0 +1,19 @@ + +#include + +int main() { + +//! [export_compiled_model] +ov::Core core; + +std::stringstream stream; + +ov::CompiledModel model = core.compile_model("modelPath", "deviceName"); + +model.export_model(stream); + +//! [export_compiled_model] + +return 0; +} + diff --git a/docs/snippets/export_compiled_model.py b/docs/snippets/export_compiled_model.py new file mode 100644 index 00000000000..39bebffe1b8 --- /dev/null +++ b/docs/snippets/export_compiled_model.py @@ -0,0 +1,7 @@ +#! [export_compiled_model] + +from openvino.runtime import Core + +ov.Core().compile_model(device, modelPath, properties).export_model(compiled_blob) + +#! [export_compiled_model] diff --git a/tools/compile_tool/CMakeLists.txt b/tools/compile_tool/CMakeLists.txt index 1cb39b77030..8ef771f4ba5 100644 --- a/tools/compile_tool/CMakeLists.txt +++ b/tools/compile_tool/CMakeLists.txt @@ -47,7 +47,4 @@ else() install(TARGETS compile_tool RUNTIME DESTINATION ${OV_CPACK_TOOLSDIR}/compile_tool COMPONENT ${OV_CPACK_COMP_CORE_TOOLS}) - install(FILES README.md - DESTINATION ${OV_CPACK_TOOLSDIR}/compile_tool - COMPONENT ${OV_CPACK_COMP_CORE_TOOLS}) endif() diff --git a/tools/compile_tool/README.md b/tools/compile_tool/README.md deleted file mode 100644 index 097d5ebf229..00000000000 --- a/tools/compile_tool/README.md +++ /dev/null @@ -1,89 +0,0 @@ -# Compile Tool {#openvino_inference_engine_tools_compile_tool_README} - -@sphinxdirective - -Compile tool is a C++ application that enables you to compile a model for inference on a specific device and export the compiled representation to a binary file. - -With this tool, you can compile a model using supported OpenVINO Runtime devices on a machine that does not have the physical device connected, and then transfer a generated file to any machine with the target inference device available. To learn which device supports the import / export functionality, see the :doc:`feature support matrix `. - -The tool is delivered as an executable file that can be run on both Linux and Windows. It is located in the ``/tools/compile_tool`` directory. - -Workflow of the Compile tool -############################ - -First, the application reads command-line parameters and loads a model to the OpenVINO Runtime device. After that, the application exports a blob with the compiled model and writes it to the output file. - -Also, the Compile tool supports the following capabilities: - -- Embedding :doc:`layout ` and precision conversions (for more details, see the :doc:`Optimize Preprocessing `). To compile the model with advanced preprocessing capabilities, refer to the :doc:`Use Case - Integrate and Save Preprocessing Steps Into OpenVINO IR `, which shows how to have all the preprocessing in the compiled blob. -- Compiling blobs for OpenVINO Runtime API 2.0 by default or for Inference Engine API with explicit option ``-ov_api_1_0``. -- Accepting device specific options for customizing the compilation process. - -Running the Compile Tool -######################## - -Running the application with the ``-h`` option yields the following usage message: - -.. code-block:: bash - - ./compile_tool -h - OpenVINO Runtime version ......... 2022.1.0 - Build ........... custom_changed_compile_tool_183a1adfcd7a001974fe1c5cfa21ec859b70ca2c - - compile_tool [OPTIONS] - - Common options: - -h Optional. Print the usage message. - -m Required. Path to the XML model. - -d Required. Specify a target device for which executable network will be compiled. - Use "-d HETERO:" format to specify HETERO plugin. - Use "-d MULTI:" format to specify MULTI plugin. - The application looks for a suitable plugin for the specified device. - -o Optional. Path to the output file. Default value: ".blob". - -c Optional. Path to the configuration file. - -ip Optional. Specifies precision for all input layers of the network. - -op Optional. Specifies precision for all output layers of the network. - -iop "" Optional. Specifies precision for input and output layers by name. - Example: -iop "input:FP16, output:FP16". - Notice that quotes are required. - Overwrites precision from ip and op options for specified layers. - -il Optional. Specifies layout for all input layers of the network. - -ol Optional. Specifies layout for all output layers of the network. - -iol "" Optional. Specifies layout for input and output layers by name. - Example: -iol "input:NCHW, output:NHWC". - Notice that quotes are required. - Overwrites layout from il and ol options for specified layers. - -iml Optional. Specifies model layout for all input layers of the network. - -oml Optional. Specifies model layout for all output layers of the network. - -ioml "" Optional. Specifies model layout for input and output tensors by name. - Example: -ionl "input:NCHW, output:NHWC". - Notice that quotes are required. - Overwrites layout from il and ol options for specified layers. - -ov_api_1_0 Optional. Compile model to legacy format for usage in Inference Engine API, - by default compiles to OV 2.0 API - -Running the application with the empty list of options yields an error message. - -For example, to compile a blob for inference on an Intel® Neural Compute Stick 2 from a trained network, run the command below: - -.. code-block:: bash - - ./compile_tool -m /model_name.xml -d CPU - - -Stating flags that take only single option like `-m` multiple times, for example `./compile_tool -m model.xml -m model2.xml`, results in only the first value being used. - -Import a Compiled Blob File to Your Application -+++++++++++++++++++++++++++++++++++++++++++++++ - -To import a blob with the network from a generated file into your application, use the -``ov::Core::import_model`` method: - -.. code-block:: cpp - - ov::Core ie; - std::ifstream file{"model_name.blob"}; - ov::CompiledModel compiled_model = ie.import_model(file, "CPU"); - -@endsphinxdirective -