diff --git a/docs/_static/css/custom.css b/docs/_static/css/custom.css index 9ae36d232b3..d4bc812f50f 100644 --- a/docs/_static/css/custom.css +++ b/docs/_static/css/custom.css @@ -37,6 +37,7 @@ ul#navbar-main-elements li:first-of-type { ul#navbar-main-elements > li:hover { text-decoration: underline; + color: #fff; } diff --git a/docs/_static/css/homepage_style.css b/docs/_static/css/homepage_style.css index 6dc1aac1560..ac268cbdcb0 100644 --- a/docs/_static/css/homepage_style.css +++ b/docs/_static/css/homepage_style.css @@ -5,15 +5,14 @@ display: none; } -img { - cursor: default; +main img { + cursor: pointer; } + /* === OPENVINO INTRO ================================================= */ .openvino-intro-text { - font-size: 1em; - } /* === OPENVINO DIAGRAM ================================================= */ diff --git a/docs/index.rst b/docs/index.rst index da662dc3125..f002146d31e 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,5 +7,5 @@ OpenVINO™ Documentation Install Blog Forum - Training + Support GitHub diff --git a/tools/pot/docs/FrequentlyAskedQuestions.md b/tools/pot/docs/FrequentlyAskedQuestions.md index cb2cfbb0a22..5bba4c20041 100644 --- a/tools/pot/docs/FrequentlyAskedQuestions.md +++ b/tools/pot/docs/FrequentlyAskedQuestions.md @@ -3,101 +3,108 @@ If your question is not covered below, use the [OpenVINO™ Community Forum page](https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit), where you can participate freely. -- Is the Post-training Optimization Tool opensourced? -- Can I quantize my model without a dataset? -- Can a model in any framework be quantized by the POT? -- What is a tradeoff when you go to low precision? -- I'd like to quantize a model and I've converted it to IR but I don't have the Accuracy Checker config. What can I do? -- I tried all recommendations from "Post-Training Optimization Best Practices" but either have a high accuracy drop or bad performance after quantization. -What else can I do? -- I get “RuntimeError: Cannot get memory” and “RuntimeError: Output data was not allocated” when I quantize my model by the POT. -- I have successfully quantized my model with a low accuracy drop and improved performance but the output video generated from the low precision model is much worse than from the full precision model. What could be the root cause? -- The quantization process of my model takes a lot of time. Can it be decreased somehow? -- I get "Import Error:... No such file or directory". How can I avoid it? -- When I execute POT CLI, I get "File "/workspace/venv/lib/python3.7/site-packages/nevergrad/optimization/base.py", line 35... SyntaxError: invalid syntax". What is wrong? -- What does a message "ModuleNotFoundError: No module named 'some\_module\_name'" mean? -- Is there a way to collect an intermidiate IR when the AccuracyAware mechanism fails? -- What do the messages "Output name: result_operation_name not found" or "Output node with result_operation_name is not found in graph" mean? + +.. dropdown:: Is the Post-training Optimization Tool (POT) open-sourced? + + Yes, POT is developed on GitHub as a part of `openvinotoolkit/openvino `__ under Apache-2.0 License. + +.. dropdown:: Can I quantize my model without a dataset? + + In general, you should have a dataset. The dataset should be annotated if you want to validate the accuracy. + If your dataset is not annotated, you can use :doc:`Default Quantization ` + to quantize the model or command-line interface with :doc:`Simplified mode `. + +.. dropdown:: Can a model in any framework be quantized by the POT? + + The POT accepts models in the OpenVINO™ Intermediate Representation (IR) format only. For that you need to convert your model to the IR format using + :doc:`model conversion API `. -### Is the Post-training Optimization Tool (POT) opensourced? +.. dropdown:: I'd like to quantize a model and I've converted it to IR but I don't have the Accuracy Checker config. What can I do? -Yes, POT is developed on GitHub as a part of [openvinotoolkit/openvino](https://github.com/openvinotoolkit/openvino) under Apache-2.0 License. + 1. Try quantization using Python API of the Post-training Optimization Tool. For more details see :doc:`Default Quantization `. + 2. If you consider command-line usage only refer to :doc:`Accuracy Checker documentation ` to create the Accuracy Checker configuration file, + and try to find the configuration file for your model among the ones available in the Accuracy Checker examples. + 3. An alternative way is to quantize the model in the :doc:`Simplified mode ` but you will not be able to measure the accuracy. -### Can I quantize my model without a dataset? +.. dropdown:: What is a tradeoff when you go to low precision? -In general, you should have a dataset. The dataset should be annotated if you want to validate the accuracy. -If your dataset is not annotated, you can use [Default Quantization](@ref pot_default_quantization_usage) to quantize the model or command-line interface with [Simplified mode](@ref pot_docs_simplified_mode). + The tradeoff is between the accuracy drop and performance. When a model is in low precision, it is usually performed + compared to the same model in full precision but the accuracy might be worse. You can find some benchmarking results in + :doc:`INT8 vs FP32 Comparison on Select Networks and Platforms `. + The other benefit of having a model in low precision is its smaller size. -### Can a model in any framework be quantized by the POT? +.. dropdown:: I tried all recommendations from "Post-Training Optimization Best Practices" but either have a high accuracy drop or bad performance after quantization. What else can I do? -The POT accepts models in the OpenVINO™ Intermediate Representation (IR) format only. For that you need to convert your model to the IR format using -[Model Optimizer](@ref openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide). + First of all, you should validate the POT compression pipeline you are running, which can be done with the following steps: -### I'd like to quantize a model and I've converted it to IR but I don't have the Accuracy Checker config. What can I do? + 1. Make sure the accuracy of the original uncompressed model has the value you expect. Run your POT pipeline with an empty compression config and evaluate the resulting model metric. + Compare this uncompressed model accuracy metric value with your reference. + 2. Run your compression pipeline with a single compression algorithm (:doc:`Default Quantization ` or :doc:`Accuracy-aware Quantization `) + without any parameter values specified in the config (except for ``preset`` and ``stat_subset_size``). Make sure you get the desirable accuracy drop/performance gain in this case. -1. Try quantization using Python API of the Post-training Optimization Tool. For more details see [Default Quantization](@ref pot_default_quantization_usage). -2. If you consider command-line usage only refer to [Accuracy Checker documentation](@ref omz_tools_accuracy_checker) to create the Accuracy Checker configuration file, and try to find the configuration file for your model among the ones available in the Accuracy Checker examples. -3. An alternative way is to quantize the model in the [Simplified mode](@ref pot_docs_simplified_mode) but you will not be able to measure the accuracy. + Finally, if you have done the steps above and the problem persists, you could try to compress your model using the + `Neural Network Compression Framework (NNCF) `__. Note that NNCF usage requires you to have a + PyTorch or TensorFlow 2 based training pipeline of your model to perform Quantization-aware Training. + See :doc:`Model Optimization Guide ` for more details. -### What is a tradeoff when you go to low precision? +.. dropdown:: I get “RuntimeError: Cannot get memory” and “RuntimeError: Output data was not allocated” when I quantize my model by the POT. -The tradeoff is between the accuracy drop and performance. When a model is in low precision, it is usually performed -compared to the same model in full precision but the accuracy might be worse. You can find some benchmarking results in -[INT8 vs FP32 Comparison on Select Networks and Platforms](@ref openvino_docs_performance_int8_vs_fp32). -The other benefit of having a model in low precision is its smaller size. + These issues happen due to insufficient available amount of memory for statistics collection during the quantization process of a huge model or + due to a very high resolution of input images in the quantization dataset. If you do not have a possibility to increase your RAM size, one of the following options can help: -### I tried all recommendations from "Post-Training Optimization Best Practices" but either have a high accuracy drop or bad performance after quantization. What else can I do? + - Set ``inplace_statistics`` parameters to ``True``. In that case, the POT will change the method to collect statistics and use less memory. + Note that such change might increase the time required for quantization. + - Set ``eval_requests_number`` and ``stat_requests_number`` parameters to 1. In that case, the POT will limit the number of infer requests by 1 and use less memory. + Note that such change might increase the time required for quantization. + - Set ``use_fast_bias`` parameter to ``false``. In that case, the POT will switch from the FastBiasCorrection algorithm to the full BiasCorrection algorithm + which is usually more accurate and takes more time but requires less memory. See :doc:`Post-Training Optimization Best Practices ` for more details. + - Reshape your model to a lower resolution and resize the size of images in the dataset. Note that such change might impact the accuracy. -First of all, you should validate the POT compression pipeline you are running, which can be done with the following steps: -1. Make sure the accuracy of the original uncompressed model has the value you expect. Run your POT pipeline with an empty compression config and evaluate the resulting model metric. Compare this uncompressed model accuracy metric value with your reference. -2. Run your compression pipeline with a single compression algorithm ([Default Quantization](@ref pot_default_quantization_usage) or [Accuracy-aware Quantization](@ref pot_accuracyaware_usage)) without any parameter values specified in the config (except for `preset` and `stat_subset_size`). Make sure you get the desirable accuracy drop/performance gain in this case. +.. dropdown:: I have successfully quantized my model with a low accuracy drop and improved performance but the output video generated from the low precision model is much worse than from the full precision model. What could be the root cause? -Finally, if you have done the steps above and the problem persists, you could try to compress your model using the [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf_pytorch). -Note that NNCF usage requires you to have a PyTorch or TensorFlow 2 based training pipeline of your model to perform Quantization-aware Training. See [Model Optimization Guide](@ref openvino_docs_model_optimization_guide) for more details. + It can happen due to the following reasons: + + - A wrong or not representative dataset was used during the quantization and accuracy validation. + Please make sure that your data and labels are correct and they sufficiently reflect the use case. + - If the command-line interface was used for quantization, a wrong Accuracy Checker configuration file could lead to this problem. + Refer to :doc:`Accuracy Checker documentation ` for more information. + - If :doc:`Default Quantization ` was used for quantization you can also try + :doc:`Accuracy-aware Quantization ` method that allows controlling maximum accuracy deviation. -### I get “RuntimeError: Cannot get memory” and “RuntimeError: Output data was not allocated” when I quantize my model by the POT. +.. dropdown:: The quantization process of my model takes a lot of time. Can it be decreased somehow? -These issues happen due to insufficient available amount of memory for statistics collection during the quantization process of a huge model or -due to a very high resolution of input images in the quantization dataset. If you do not have a possibility to increase your RAM size, one of the following options can help: -- Set `inplace_statistics` parameters to "True". In that case the POT will change method collect statistics and use less memory. Note that such change might increase time required for quantization. -- Set `eval_requests_number` and `stat_requests_number` parameters to 1. In that case the POT will limit the number of infer requests by 1 and use less memory. -Note that such change might increase time required for quantization. -- Set `use_fast_bias` parameter to `false`. In that case the POT will switch from the FastBiasCorrection algorithm to the full BiasCorrection algorithm -which is usually more accurate and takes more time but requires less memory. See [Post-Training Optimization Best Practices](BestPractices.md) for more details. -- Reshape your model to a lower resolution and resize the size of images in the dataset. Note that such change might impact the accuracy. + Quantization time depends on multiple factors such as the size of the model and the dataset. It also depends on the algorithm: + the :doc:`Default Quantization ` algorithm takes less time than the :doc:`Accuracy-aware Quantization ` algorithm. + The following configuration parameters also impact the quantization time duration + (see details in :doc:`Post-Training Optimization Best Practices `): + + - ``use_fast_bias``: when set to ``false``, it increases the quantization time + - ``stat_subset_size``: the higher the value of this parameter, the more time will be required for the quantization + - ``tune_hyperparams``: if set to ``true`` when the AccuracyAwareQuantization algorithm is used, it increases the quantization time + - ``stat_requests_number``: the lower number, the more time might be required for the quantization + - ``eval_requests_number``: the lower number, the more time might be required for the quantization -### I have successfully quantized my model with a low accuracy drop and improved performance but the output video generated from the low precision model is much worse than from the full precision model. What could be the root cause? + Note that higher values of ``stat_requests_number`` and ``eval_requests_number`` increase memory consumption by POT. -It can happen due to the following reasons: -- A wrong or not representative dataset was used during the quantization and accuracy validation. Please make sure that your data and labels are correct and they sufficiently reflect the use case. -- If the command-line interface was used for quantization, a wrong Accuracy Checker configuration file could lead to this problem. Refer to [Accuracy Checker documentation](@ref omz_tools_accuracy_checker) for more information. -- If [Default Quantization](@ref pot_default_quantization_usage) was used for quantization you can also try [Accuracy-aware Quantization](@ref pot_accuracyaware_usage) method that allows controlling maximum accuracy deviation. +.. dropdown:: When I execute POT CLI, I get "File "/workspace/venv/lib/python3.7/site-packages/nevergrad/optimization/base.py", line 35... SyntaxError: invalid syntax". What is wrong? -### The quantization process of my model takes a lot of time. Can it be decreased somehow? + This error is reported when you have a Python version older than 3.7 in your environment. Upgrade your Python version. -Quantization time depends on multiple factors such as the size of the model and the dataset. It also depends on the algorithm: -the [Default Quantization](@ref pot_default_quantization_usage) algorithm takes less time than the [Accuracy-aware Quantization](@ref pot_accuracyaware_usage) algorithm. -The following configuration parameters also impact the quantization time duration -(see details in [Post-Training Optimization Best Practices](BestPractices.md)): -- `use_fast_bias`: when set to `false`, it increases the quantization time -- `stat_subset_size`: the higher the value of this parameter, the more time will be required for the quantization -- `tune_hyperparams`: if set to `true` when the AccuracyAwareQuantization algorithm is used, it increases the quantization time -- `stat_requests_number`: the lower number, the more time might be required for the quantization -- `eval_requests_number`: the lower number, the more time might be required for the quantization -Note that higher values of `stat_requests_number` and `eval_requests_number` increase memory consumption by POT. +.. dropdown:: What does the message "ModuleNotFoundError: No module named 'some\_module\_name'" mean? -### When I execute POT CLI, I get "File "/workspace/venv/lib/python3.7/site-packages/nevergrad/optimization/base.py", line 35... SyntaxError: invalid syntax". What is wrong? + It means that some required python module is not installed in your environment. To install it, run ``pip install some_module_name``. -This error is reported when you have a Python version older than 3.7 in your environment. Upgrade your Python version. +.. dropdown:: Is there a way to collect an intermediate IR when the AccuracyAware mechanism fails? -### What does a message "ModuleNotFoundError: No module named 'some\_module\_name'" mean? + You can add ``"dump_intermediate_model": true`` to the POT configuration file and it will drop an intermediate IR to ``accuracy_aware_intermediate`` folder. -It means that some required python module is not installed in your environment. To install it, run `pip install some_module_name`. +.. dropdown:: What do the messages "Output name: result_operation_name not found" or "Output node with result_operation_name is not found in graph" mean? -### Is there a way to collect an intermidiate IR when the AccuracyAware mechanism fails? + Errors are caused by missing output nodes names in a graph when using the POT tool for model quantization. + It might appear for some models only for IRs converted from ONNX models using the new frontend (which is the default + conversion path starting from 2022.1 release). To avoid such errors, use the legacy MO frontend to convert a model + to IR by passing the ``--use_legacy_frontend`` option. Then, use the produced IR for quantization. -You can add `"dump_intermediate_model": true` to the POT configuration file and it will drop an intermidiate IR to `accuracy_aware_intermediate` folder. +@endsphinxdirective -### What do the messages "Output name: result_operation_name not found" or "Output node with result_operation_name is not found in graph" mean? -Errors are caused by missing output nodes names in a graph when using the POT tool for model quantization. It might appear for some models only for IRs converted from ONNX models using new frontend (which is the default conversion path starting from 2022.1 release). To avoid such errors, use legacy MO frontend to convert a model to IR by passing the --use_legacy_frontend option. Then, use the produced IR for quantization.