[DOCS] final benchmark data + conf version (#22375)

* [DOCS] final benchmark data
* conf.py update 23.3
This commit is contained in:
Karol Blaszczak 2024-01-24 11:09:17 +01:00 committed by GitHub
parent 900d7f8048
commit 489cc9304d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 58 additions and 40 deletions

View File

@ -48,7 +48,7 @@ Click the buttons below to see the chosen benchmark data.
:material-regular:`bar_chart;1.4em` OVMS Benchmark Graphs
For a successful deep learning inference application, the following four key metrics need to be considered:
Please visit the tabs below for more information on key performance indicators and workload parameters.
.. tab-set::
@ -91,6 +91,24 @@ For a successful deep learning inference application, the following four key met
near real-time applications for example an industrial robot's response to actions in its environment
or obstacle avoidance for autonomous vehicles.
.. tab-item:: Workload Parameters
:sync: workloadparameters
The workload parameters affect the performance results of the different models we use for benchmarking.
Image processing models have different image size definitions and the Natural Language Processing models
have different max token list lengths. All these can be found in detail in the :doc:`FAQ section <openvino_docs_performance_benchmarks_faq>`.
All models are executed using a batch size of 1. Below are the parameters for the GenAI models we display.
* Input tokens: 1024,
* Output tokens: 128,
* number of beams: 1
For text to image:
* iteration steps: 20,
* image size (HxW): 256 x 256,
* input token length: 1024 (the tokens for GenAI models are in English).
Platforms, Configurations, Methodology
###########################################################
@ -168,9 +186,9 @@ or `create an account <https://www.intel.com/content/www/us/en/secure/forms/devc
Disclaimers
####################################
* Intel® Distribution of OpenVINO™ toolkit performance results are based on release 2023.2, as of November 15, 2023.
* Intel® Distribution of OpenVINO™ toolkit performance results are based on release 2023.3, as of January 16, 2024.
* OpenVINO Model Server performance results are based on release 2023.0, as of June 01, 2023.
* OpenVINO Model Server performance results are based on release 2023.0, as of June 06, 2023.
The results may not reflect all publicly available updates. Intel technologies features and benefits depend on system configuration
and may require enabled hardware, software, or service activation. Learn more at intel.com, or from the OEM or retailer.

View File

@ -5,9 +5,9 @@ Model Accuracy
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
the table for more information.
* A - Intel® Core™ i9-9000K (AVX2), INT8 and FP32
@ -105,30 +105,30 @@ the table for more information.
* - chatGLM2-6b
- lambada openai
- ppl
-
-
-
-
-
- 17.37 (ref. 17.48)
- 17.41 (ref. 17.48)
- 17.16 (ref. 17.48)
* - Llama-2-7b-chat
- Wiki, StackExch, Crawl
- ppl
-
-
-
-
-
- 3.24 (ref. 3.26)
- 3.24 (ref. 3.26)
- 3.25 (ref. 3.26)
* - Stable-Diffusion-V2-1
- LIAON-5B
- ppl
-
-
-
-
-
-
-
* - Mistral-7b
- proprietary Mistral.ai
- ppl
-
-
-
-
- 3.29 (ref. 3.19)
- 3.28 (ref. 3.19)
-
.. list-table:: Model Accuracy for BF16, FP32 and FP16 (FP16: Flex-170 only. BF16: Xeon(R) 8490H only)
:header-rows: 1
@ -232,37 +232,37 @@ the table for more information.
* - chatGLM2-6b
- lambada openai
- ppl
-
- 17.48 (ref. 17.48)
- 17.56 (ref. 17.48)
-
-
-
-
-
- 17.49 (ref. 17.48)
* - Llama-2-7b-chat
- Wiki, StackExch, Crawl
- ppl
-
- 3.26 (ref. 3.26)
- 3.26 (ref. 3.26)
-
-
-
-
-
-
* - Stable-Diffusion-V2-1
- LIAON-5B
- ppl
-
-
-
-
-
-
-
-
-
* - Mistral-7b
- proprietary Mistral.ai
- ppl
-
- 3.18 (ref. 3.19)
- 3.18 (ref. 3.19)
-
-
-
-
-
-
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
accuracy for the model.

View File

@ -22,11 +22,11 @@ from sphinx.ext.autodoc import ClassDocumenter
# -- Project information -----------------------------------------------------
project = 'OpenVINO™'
copyright = '2023, Intel®'
copyright = '2024, Intel®'
author = 'Intel®'
language = 'en'
version_name = 'nightly'
version_name = '2023.3'
# -- General configuration ---------------------------------------------------