diff --git a/docs/articles_en/openvino_workflow/running_inference_with_openvino/dldt_deployment_optimization_guide/dldt_deployment_optimization_latency/Model_caching_overview.rst b/docs/articles_en/openvino_workflow/running_inference_with_openvino/dldt_deployment_optimization_guide/dldt_deployment_optimization_latency/Model_caching_overview.rst index b38c93767a8..133e351e873 100644 --- a/docs/articles_en/openvino_workflow/running_inference_with_openvino/dldt_deployment_optimization_guide/dldt_deployment_optimization_latency/Model_caching_overview.rst +++ b/docs/articles_en/openvino_workflow/running_inference_with_openvino/dldt_deployment_optimization_guide/dldt_deployment_optimization_latency/Model_caching_overview.rst @@ -5,34 +5,34 @@ Model Caching Overview .. meta:: - :description: Enabling model caching to export compiled model - automatically and reusing it can significantly + :description: Enabling model caching to export compiled model + automatically and reusing it can significantly reduce duration of model compilation on application startup. -As described in :doc:`Integrate OpenVINO™ with Your Application `, +As described in :doc:`Integrate OpenVINO™ with Your Application `, a common application flow consists of the following steps: -1. | **Create a Core object**: +1. | **Create a Core object**: | First step to manage available devices and read model objects -2. | **Read the Intermediate Representation**: +2. | **Read the Intermediate Representation**: | Read an Intermediate Representation file into an object of the `ov::Model `__ -3. | **Prepare inputs and outputs**: +3. | **Prepare inputs and outputs**: | If needed, manipulate precision, memory layout, size or color format -4. | **Set configuration**: +4. | **Set configuration**: | Pass device-specific loading configurations to the device -5. | **Compile and Load Network to device**: +5. | **Compile and Load Network to device**: | Use the `ov::Core::compile_model() `__ method with a specific device -6. | **Set input data**: +6. | **Set input data**: | Specify input tensor -7. | **Execute**: +7. | **Execute**: | Carry out inference and process results -Step 5 can potentially perform several time-consuming device-specific optimizations and network compilations. -To reduce the resulting delays at application startup, you can use Model Caching. It exports the compiled model +Step 5 can potentially perform several time-consuming device-specific optimizations and network compilations. +To reduce the resulting delays at application startup, you can use Model Caching. It exports the compiled model automatically and reuses it to significantly reduce the model compilation time. -.. important:: +.. important:: Not all devices support the network import/export feature. They will perform normally but will not enable the compilation stage speed-up. @@ -60,11 +60,11 @@ To enable model caching, the application must specify a folder to store the cach :fragment: [ov:caching:part0] -With this code, if the device specified by ``device_name`` supports import/export model capability, +With this code, if the device specified by ``device_name`` supports import/export model capability, a cached blob is automatically created inside the ``/path/to/cache/dir`` folder. If the device does not support the import/export capability, cache is not created and no error is thrown. -Note that the first ``compile_model`` operation takes slightly longer, as the cache needs to be created - +Note that the first ``compile_model`` operation takes slightly longer, as the cache needs to be created - the compiled blob is saved into a cache file: .. image:: _static/images/caching_enabled.svg @@ -137,10 +137,3 @@ To check in advance if a particular device supports model caching, your applicat :language: cpp :fragment: [ov:caching:part3] - -.. note:: - - For GPU, model caching is currently supported fully for static models only. For dynamic models, - kernel caching is used and multiple ‘.cl_cache’ files are generated along with the ‘.blob’ file. - See the :doc:`GPU plugin documentation `. -