natas_hw
/
openvino
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[DOCS] Removing the note about GPU support for static models only (#22803) 

Removing the note about GPU support for static models only, as GPU now
supports both type of shapes. Porting:
https://github.com/openvinotoolkit/openvino/pull/22802
This commit is contained in:
Sebastian Golebiewski 2024-02-13 08:20:45 +01:00 committed by GitHub
parent bed27aa64d
commit 4bf8a469dc
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 15 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
docs/articles_en/openvino_workflow/running_inference_with_openvino/dldt_deployment_optimization_guide/dldt_deployment_optimization_latency/Model_caching_overview.rst
View File

@ -5,34 +5,34 @@ Model Caching Overview
.. meta::
:description: Enabling model caching to export compiled model
automatically and reusing it can significantly
:description: Enabling model caching to export compiled model
automatically and reusing it can significantly
reduce duration of model compilation on application startup.
As described in :doc:`Integrate OpenVINO™ with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`,
As described in :doc:`Integrate OpenVINO™ with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`,
a common application flow consists of the following steps:
1. | **Create a Core object**:
1. | **Create a Core object**:
| First step to manage available devices and read model objects
2. | **Read the Intermediate Representation**:
2. | **Read the Intermediate Representation**:
| Read an Intermediate Representation file into an object of the `ov::Model <classov_1_1Model.html#doxid-classov-1-1-model>`__
3. | **Prepare inputs and outputs**:
3. | **Prepare inputs and outputs**:
| If needed, manipulate precision, memory layout, size or color format
4. | **Set configuration**:
4. | **Set configuration**:
| Pass device-specific loading configurations to the device
5. | **Compile and Load Network to device**:
5. | **Compile and Load Network to device**:
| Use the `ov::Core::compile_model() <classov_1_1Core.html#doxid-classov-1-1-core-1a46555f0803e8c29524626be08e7f5c5a>`__ method with a specific device
6. | **Set input data**:
6. | **Set input data**:
| Specify input tensor
7. | **Execute**:
7. | **Execute**:
| Carry out inference and process results
Step 5 can potentially perform several time-consuming device-specific optimizations and network compilations.
To reduce the resulting delays at application startup, you can use Model Caching. It exports the compiled model
Step 5 can potentially perform several time-consuming device-specific optimizations and network compilations.
To reduce the resulting delays at application startup, you can use Model Caching. It exports the compiled model
automatically and reuses it to significantly reduce the model compilation time.
.. important::
.. important::
Not all devices support the network import/export feature. They will perform normally but will not
enable the compilation stage speed-up.
@ -60,11 +60,11 @@ To enable model caching, the application must specify a folder to store the cach
:fragment: [ov:caching:part0]
With this code, if the device specified by ``device_name`` supports import/export model capability,
With this code, if the device specified by ``device_name`` supports import/export model capability,
a cached blob is automatically created inside the ``/path/to/cache/dir`` folder.
If the device does not support the import/export capability, cache is not created and no error is thrown.
Note that the first ``compile_model`` operation takes slightly longer, as the cache needs to be created -
Note that the first ``compile_model`` operation takes slightly longer, as the cache needs to be created -
the compiled blob is saved into a cache file:
.. image:: _static/images/caching_enabled.svg
@ -137,10 +137,3 @@ To check in advance if a particular device supports model caching, your applicat
:language: cpp
:fragment: [ov:caching:part3]
.. note::
For GPU, model caching is currently supported fully for static models only. For dynamic models,
kernel caching is used and multiple ‘.cl_cache’ files are generated along with the ‘.blob’ file.
See the :doc:`GPU plugin documentation <openvino_docs_OV_UG_supported_plugins_GPU>`.