[DOCS] Fixes for OpenVINO Workflow for 2024 (#23278)
* Fixed some lines in OpenVINO Workflow section * Fixed a PaddlePaddle link
This commit is contained in:
parent
da0497bd60
commit
51769e45ed
|
|
@ -99,7 +99,7 @@ To learn more about these methods, visit the `NNCF repository on GitHub <https:/
|
|||
Recommended Workflow
|
||||
++++++++++++++++++++
|
||||
|
||||
Using compression-aware training requires a training pipeline, an annotated dataset, and compute ../../about-openvino/additional-resources (such as CPUs or GPUs). If you don't already have these set up and available, it can be easier to start post-training quantization to quickly see quantized results. Then you can use compression-aware training if the model isn't accurate enough. We recommend the following workflow for compressing models with NNCF:
|
||||
Using compression-aware training requires a training pipeline, an annotated dataset, and compute resources (such as CPUs or GPUs). If you don't already have these set up and available, it can be easier to start post-training quantization to quickly see quantized results. Then you can use compression-aware training if the model isn't accurate enough. We recommend the following workflow for compressing models with NNCF:
|
||||
|
||||
1. :doc:`Perform post-training quantization <quantizing-models-post-training>` on your model and then compare performance to the original model.
|
||||
2. If the accuracy is too degraded, use :doc:`Quantization-aware Training <compressing-models-during-training/quantization-aware-training>` to increase accuracy while still achieving faster inference time.
|
||||
|
|
|
|||
|
|
@ -235,7 +235,7 @@ Deploying pruned model
|
|||
######################
|
||||
|
||||
The pruned model requres an extra step that should be done to get performance improvement. This step involves removal of the
|
||||
zero filters from the model. This is done at the model conversion step using :doc:`model conversion API <../../model-preparation>`>` tool when model is converted from the framework representation (ONNX, TensorFlow, etc.) to OpenVINO Intermediate Representation.
|
||||
zero filters from the model. This is done at the model conversion step using :doc:`model conversion API <../../model-preparation>` tool when model is converted from the framework representation (ONNX, TensorFlow, etc.) to OpenVINO Intermediate Representation.
|
||||
|
||||
* To remove zero filters from the pruned model add the following parameter to the model conversion command: ``transform=Pruning``
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ This page provides general instructions on how to convert a model from the Paddl
|
|||
Converting PaddlePaddle Model Files
|
||||
###################################
|
||||
|
||||
PaddlePaddle inference model includes ``.pdmodel`` (storing model structure) and ``.pdiparams`` (storing model weight). For details on how to export a PaddlePaddle inference model, refer to the `Exporting PaddlePaddle Inference Model <https://www.paddlepaddle.org.cn/../../documentation/docs/zh/develop/guides/beginner/model_save_load_cn.html>`__ Chinese guide.
|
||||
PaddlePaddle inference model includes ``.pdmodel`` (storing model structure) and ``.pdiparams`` (storing model weight). For details on how to export a PaddlePaddle inference model, refer to the `Exporting PaddlePaddle Inference Model <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/beginner/model_save_load_cn.html>`__ Chinese guide.
|
||||
|
||||
To convert a PaddlePaddle model, use the ``ovc`` or ``openvino.convert_model`` and specify the path to the input ``.pdmodel`` model file:
|
||||
|
||||
|
|
|
|||
|
|
@ -142,7 +142,7 @@ For best performance when using the MULTI execution mode you should consider a f
|
|||
- Just like with any throughput-oriented execution mode, it is highly recommended to query the optimal number of inference requests directly from the instance of the ``ov:compiled_model``. Refer to the code of the previously mentioned ``benchmark_app`` for more details.
|
||||
- Execution on certain device combinations, for example CPU+GPU, performs better with certain knobs. Refer to the ``benchmark_app`` code for details. One specific example is disabling GPU driver polling, which in turn requires multiple GPU streams to balance out slower communication of inference completion from the device to the host.
|
||||
- The MULTI logic always attempts to save on copying data between device-agnostic and user-facing inference requests, and device-specific 'worker' requests that are being actually scheduled behind the scene. To facilitate the copy savings, it is recommended to run the requests in the order in which they were created.
|
||||
- While performance of accelerators combines well with MULTI, the CPU+GPU execution may introduce certain performance issues. It is due to the devices sharing some ../../../about-openvino/additional-resources, like power or bandwidth. Enabling the GPU throttling hint, which saves a CPU thread for CPU inference, is an example of a recommended solution addressing this issue.
|
||||
- While performance of accelerators combines well with MULTI, the CPU+GPU execution may introduce certain performance issues. It is due to the devices sharing some resources, like power or bandwidth. Enabling the GPU throttling hint, which saves a CPU thread for CPU inference, is an example of a recommended solution addressing this issue.
|
||||
|
||||
|
||||
Additional Resources
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ Typically, human expertise is required to get more "throughput" out of the devic
|
|||
:doc:`OpenVINO performance hints <high-level-performance-hints>` is a recommended way for performance configuration, which is both device-agnostic and future-proof.
|
||||
|
||||
|
||||
**When multiple models are to be used simultaneously**, consider running inference on separate devices for each of them. Finally, when multiple models are executed in parallel on a device, using additional ``ov::hint::model_priority`` may help to define relative priorities of the models. Refer to the ../../../documentation on the :ref:`OpenVINO feature support for devices <../../../about-openvino/compatibility-and-support/supported-devices>` to check if your device supports the feature.
|
||||
**When multiple models are to be used simultaneously**, consider running inference on separate devices for each of them. Finally, when multiple models are executed in parallel on a device, using additional ``ov::hint::model_priority`` may help to define relative priorities of the models. Refer to the documentation on the :ref:`OpenVINO feature support for devices <../../../about-openvino/compatibility-and-support/supported-devices>` to check if your device supports the feature.
|
||||
|
||||
**First-Inference Latency and Model Load/Compile Time**
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue