Docs benchmarks page update port 22.3 (#16187)
changes to benchmarks page to align with theme
This commit is contained in:
Karol Blaszczak
GitHub
parent
74272ed05e
commit
ab4f31c2f1
Binary file not shown.
|
|
@ -1,3 +0,0 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:64e64059e7416353cfd2ad836a36c12071804addf4fb165f0cf5150aa7658fa4
|
||||
size 123996
|
||||
|
|
@ -310,7 +310,7 @@ class Graph {
|
|||
|
||||
$(document).ready(function () {
|
||||
|
||||
$('#build-graphs-btn').on('click', showModal);
|
||||
$('.ov-toolkit-benchmark-results').on('click', showModal);
|
||||
|
||||
function clickBuildGraphs(graph, networkModels, ietype, platforms, kpis, precisions) {
|
||||
renderData(graph, networkModels, ietype, platforms, kpis, precisions);
|
||||
|
|
|
|||
|
|
@ -21,7 +21,6 @@ Benchmarks are available for:
|
|||
* [Intel® Distribution of OpenVINO™ toolkit](performance_benchmarks_openvino.md).
|
||||
|
||||
|
||||
|
||||
You can also test performance for your system yourself, following the guide on [getting performance numbers](../MO_DG/prepare_model/Getting_performance_numbers.md).
|
||||
Performance of a particular application can also be evaluated virtually using [Intel® DevCloud for the Edge](https://devcloud.intel.com/edge/). It is a remote development environment with access to Intel® hardware and the latest versions of the Intel® Distribution of the OpenVINO™ Toolkit. To learn more about it, visit [the website](https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/overview.html) or [create an account](https://www.intel.com/content/www/us/en/forms/idz/devcloud-registration.html?tgt=https://www.intel.com/content/www/us/en/secure/forms/devcloud-enrollment/account-provisioning.html).
|
||||
|
||||
|
|
|
|||
|
|
@ -9,89 +9,81 @@
|
|||
openvino_docs_performance_int8_vs_fp32
|
||||
Performance Data Spreadsheet (download xlsx) <https://docs.openvino.ai/2022.3/_static/benchmarks_files/OV-2022.3-Performance-Data.xlsx>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
Click the "Benchmark Graphs" button to see the OpenVINO™ benchmark graphs. Select the models, the hardware platforms (CPU SKUs),
|
||||
precision and performance index from the lists and click the “Build Graphs” button.
|
||||
|
||||
@sphinxdirective
|
||||
.. button-link:: #
|
||||
:class: ov-toolkit-benchmark-results
|
||||
:color: primary
|
||||
:outline:
|
||||
|
||||
:material-regular:`bar_chart;1.4em` Benchmark Graphs
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<section class="build-benchmark-section">
|
||||
<div class="title">
|
||||
<h3>Build benchmark graphs to your specifications</h3>
|
||||
</div>
|
||||
<div class="btn-container">
|
||||
<button id="build-graphs-btn" class="configure-graphs-btn">Configure Graphs</button>
|
||||
</div>
|
||||
<img src="_static/images/sample-graph-image.png" class="sample-graph-image">
|
||||
</section>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
Measuring inference performance involves many variables and is extremely use-case and application dependent.
|
||||
Below are four parameters for measurements, which are key elements to consider for a successful deep learning inference application:
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. raw:: html
|
||||
.. tab:: :material-regular:`keyboard_double_arrow_right;1.4em` Throughput
|
||||
|
||||
<div class="picker-options">
|
||||
<span class="selectable option throughput selected" data-option="throughput">
|
||||
Throughput
|
||||
</span>
|
||||
<span class="selectable option value" data-option="value">
|
||||
Value
|
||||
</span>
|
||||
<span class="selectable option efficiency" data-option="efficiency">
|
||||
Efficiency
|
||||
</span>
|
||||
<span class="selectable option latency" data-option="latency">
|
||||
Latency
|
||||
</span>
|
||||
<p class="selectable throughput selected">
|
||||
Measures the number of inferences delivered within a latency threshold. (for example, number of Frames Per Second - FPS). When deploying a system with deep learning inference, select the throughput that delivers the best trade-off between latency and power for the price and performance that meets your requirements.
|
||||
</p>
|
||||
<p class="selectable value">
|
||||
While throughput is important, what is more critical in edge AI deployments is the performance efficiency or performance-per-cost. Application performance in throughput per dollar of system cost is the best measure of value. The value KPI is calculated as “Throughput measured as inferences per second / price of inference engine”. This means for a 2 socket system 2x the price of a CPU is used. Prices are as per date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below.
|
||||
<p class="selectable efficiency">
|
||||
System power is a key consideration from the edge to the data center. When selecting deep learning solutions, power efficiency (throughput/watt) is a critical factor to consider. Intel designs provide excellent power efficiency for running deep learning workloads. The efficiency KPI is calculated as “Throughput measured as inferences per second / TDP of inference engine”. This means for a 2 socket system 2x the power dissipation (TDP) of a CPU is used. TDP-values are as per date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below.
|
||||
<p class="selectable latency">
|
||||
This measures the synchronous execution of inference requests and is reported in milliseconds. Each inference request (for example: preprocess, infer, postprocess) is allowed to complete before the next is started. This performance metric is relevant in usage scenarios where a single image input needs to be acted upon as soon as possible. An example would be the healthcare sector where medical personnel only request analysis of a single ultra sound scanning image or in real-time or near real-time applications for example an industrial robot's response to actions in its environment or obstacle avoidance for autonomous vehicles.
|
||||
</p>
|
||||
</div>
|
||||
Measures the number of inferences delivered within a latency threshold (for example, number of Frames Per Second - FPS). When deploying a system with deep learning inference, select the throughput that delivers the best trade-off between latency and power for the price and performance that meets your requirements.
|
||||
|
||||
<h3>Platform & Configurations </h3>
|
||||
<p>For a listing of all platforms and configurations used for testing, refer to the following:</p>
|
||||
<container class="platform-configurations">
|
||||
<div>
|
||||
<a href="https://docs.openvino.ai/2022.3/_static/benchmarks_files/platform_list_22.3.pdf" target="_blank" class="pdf"><img src="_static/css/media/pdf-icon.svg"/>Hardware Platforms (PDF)</a>
|
||||
</div>
|
||||
<div>
|
||||
<a href="https://docs.openvino.ai/2022.3/_static/benchmarks_files/OV-2022.3-system-info-detailed.xlsx" class="xls"><img src="_static/css/media/xls-icon.svg"/>Configuration Details (XLSX)</a>
|
||||
</div>
|
||||
</container>
|
||||
.. tab:: :material-regular:`attach_money;1.4em` Value
|
||||
|
||||
While throughput is important, what is more critical in edge AI deployments is the performance efficiency or performance-per-cost. Application performance in throughput per dollar of system cost is the best measure of value. The value KPI is calculated as “Throughput measured as inferences per second / price of inference engine”. This means for a 2 socket system 2x the price of a CPU is used. Prices are as per date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below.
|
||||
|
||||
.. tab:: :material-regular:`flash_on;1.4em` Efficiency
|
||||
|
||||
System power is a key consideration from the edge to the data center. When selecting deep learning solutions, power efficiency (throughput/watt) is a critical factor to consider. Intel designs provide excellent power efficiency for running deep learning workloads. The efficiency KPI is calculated as “Throughput measured as inferences per second / TDP of inference engine”. This means for a 2 socket system 2x the power dissipation (TDP) of a CPU is used. TDP-values are as per date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below.
|
||||
|
||||
.. tab:: :material-regular:`hourglass_empty;1.4em` Latency
|
||||
|
||||
This measures the synchronous execution of inference requests and is reported in milliseconds. Each inference request (for example: preprocess, infer, postprocess) is allowed to complete before the next is started. This performance metric is relevant in usage scenarios where a single image input needs to be acted upon as soon as possible. An example would be the healthcare sector where medical personnel only request analysis of a single ultra sound scanning image or in real-time or near real-time applications for example an industrial robot's response to actions in its environment or obstacle avoidance for autonomous vehicles.
|
||||
|
||||
|
||||
|
||||
Platform & Configurations
|
||||
####################################
|
||||
|
||||
For a listing of all platforms and configurations used for testing, refer to the following:
|
||||
|
||||
.. button-link:: _static/benchmarks_files/platform_list_22.3.pdf
|
||||
:color: primary
|
||||
:outline:
|
||||
|
||||
:material-regular:`download;1.5em` Click for Hardware Platforms [PDF]
|
||||
|
||||
.. button-link:: _static/benchmarks_files/OV-2022.3-system-info-detailed.xlsx
|
||||
:color: primary
|
||||
:outline:
|
||||
|
||||
:material-regular:`download;1.5em` Click for Configuration Details [XLSX]
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
This benchmark setup includes a single machine on which both the benchmark application and the OpenVINO™ installation reside. The presented performance benchmark numbers are based on the release 2022.3 of the Intel® Distribution of OpenVINO™ toolkit.
|
||||
The benchmark application loads the OpenVINO™ Runtime and executes inferences on the specified hardware (CPU, GPU or VPU).
|
||||
It measures the time spent on actual inferencing (excluding any pre or post processing) and then reports on the inferences per second (or Frames Per Second).
|
||||
The benchmark application loads the OpenVINO™ Runtime and executes inferences on the specified hardware (CPU, GPU or GNA).
|
||||
It measures the time spent on actual inference (excluding any pre or post processing) and then reports on the inferences per second (or Frames Per Second).
|
||||
|
||||
|
||||
## Disclaimers
|
||||
Disclaimers
|
||||
####################################
|
||||
|
||||
Intel® Distribution of OpenVINO™ toolkit performance benchmark numbers are based on release 2022.3.
|
||||
|
||||
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. Performance results are based on testing as of December 13, 2022 and may not reflect all publicly available updates. See configuration disclosure for details. No product can be absolutely secure.
|
||||
|
||||
Performance varies by use, configuration and other factors. Learn more at [www.intel.com/PerformanceIndex](https://www.intel.com/PerformanceIndex).
|
||||
Performance varies by use, configuration and other factors. Learn more at :ref:`www.intel.com/PerformanceIndex<https://www.intel.com/PerformanceIndex>`.
|
||||
|
||||
Your costs and results may vary.
|
||||
|
||||
Intel optimizations, for Intel compilers or other products, may not optimize to the same degree for non-Intel products.
|
||||
|
||||
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
|
||||
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
|
||||
|
||||
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Model Accuracy and Performance for INT8 and FP32 {#openvino_docs_performance_int8_vs_fp32}
|
||||
# Model Accuracy {#openvino_docs_performance_int8_vs_fp32}
|
||||
|
||||
The following table presents the absolute accuracy drop calculated as the accuracy difference between FP32 and INT8 representations of a model on two platforms
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +1,12 @@
|
|||
# Supported Models {#openvino_supported_models}
|
||||
|
||||
|
||||
@sphinxdirective
|
||||
The OpenVINO team continues the effort to support as many models out-of-the-box as possible.
|
||||
Based on our research and user feedback, we prioritize the most common models and test them
|
||||
before every release. These models are considered officially supported.
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. button-link:: _static/download/OV_2022_models_supported.pdf
|
||||
.. button-link:: _static/download/OV_2023_models_supported.pdf
|
||||
:color: primary
|
||||
:outline:
|
||||
|
||||
|
|
@ -18,36 +17,33 @@ before every release. These models are considered officially supported.
|
|||
| If your model is not included but is similar to those that are, it is still very likely to work.
|
||||
If your model fails to execute properly there are a few options available:
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
* If the model originates from a framework like TensorFlow or PyTorch, OpenVINO™ offers a hybrid solution. The original model can be run without explicit conversion into the OpenVINO format. For more information, see [OpenVINO TensorFlow Integration](https://docs.openvino.ai/latest/ovtf_integration.html).
|
||||
* If the model originates from a framework like TensorFlow or PyTorch, OpenVINO™ offers a hybrid solution. The original model can be run without explicit conversion into the OpenVINO format. For more information, see :ref:`OpenVINO TensorFlow Integration <https://docs.openvino.ai/latest/ovtf_integration.html>`.
|
||||
* You can create a GitHub request for the operation(s) that are missing. These requests are reviewed regularly. You will be informed if and how the request will be accommodated. Additionally, your request may trigger a reply from someone in the community who can help.
|
||||
* As OpenVINO™ is open source you can enhance it with your own contribution to the GitHub repository. To learn more, see the articles on [OpenVINO Extensibility](https://docs.openvino.ai/latest/openvino_docs_Extensibility_UG_Intro.html).
|
||||
* As OpenVINO™ is open source you can enhance it with your own contribution to the GitHub repository. To learn more, see the articles on :ref:`OpenVINO Extensibility<https://docs.openvino.ai/latest/openvino_docs_Extensibility_UG_Intro.html>`.
|
||||
|
||||
|
||||
The following table summarizes the number of models supported by OpenVINO™ in different categories:
|
||||
|
||||
@sphinxdirective
|
||||
+--------------------------------------------+-------------------+
|
||||
| Model Categories: | Number of Models: |
|
||||
+============================================+===================+
|
||||
| Object Detection | 149 |
|
||||
| Object Detection | 149 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Instance Segmentation | 3 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Semantic Segmentation | 19 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Image Processing, Enhancement | 16 |
|
||||
| Image Processing, Enhancement | 16 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Monodepth | 2 |
|
||||
| Monodepth | 2 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Colorization | 2 |
|
||||
| Colorization | 2 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Behavior / Decision Prediction | 1 |
|
||||
| Behavior / Decision Prediction | 1 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Action Recognition | 2 |
|
||||
| Action Recognition | 2 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Time Series Forecasting | 1 |
|
||||
| Time Series Forecasting | 1 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Image Classification | 68 |
|
||||
+--------------------------------------------+-------------------+
|
||||
|
|
@ -55,14 +51,15 @@ The following table summarizes the number of models supported by OpenVINO™ in
|
|||
+--------------------------------------------+-------------------+
|
||||
| Image Classification, Emotion | 1 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Image Translation | 1 |
|
||||
| Image Translation | 1 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Natural language Processing | 35 |
|
||||
| Natural language Processing | 35 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Text Detection | 18 |
|
||||
| Text Detection | 18 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Audio Enhancement | 3 |
|
||||
| Audio Enhancement | 3 |
|
||||
+--------------------------------------------+-------------------+
|
||||
| Sound Classification | 2 |
|
||||
| Sound Classification | 2 |
|
||||
+--------------------------------------------+-------------------+
|
||||
|
||||
@endsphinxdirective
|
||||
Loading…
Reference in New Issue