[DOCS] final benchmark data + conf version (#22375)
* [DOCS] final benchmark data * conf.py update 23.3
This commit is contained in:
parent
900d7f8048
commit
489cc9304d
|
|
@ -48,7 +48,7 @@ Click the buttons below to see the chosen benchmark data.
|
|||
:material-regular:`bar_chart;1.4em` OVMS Benchmark Graphs
|
||||
|
||||
|
||||
For a successful deep learning inference application, the following four key metrics need to be considered:
|
||||
Please visit the tabs below for more information on key performance indicators and workload parameters.
|
||||
|
||||
.. tab-set::
|
||||
|
||||
|
|
@ -91,6 +91,24 @@ For a successful deep learning inference application, the following four key met
|
|||
near real-time applications for example an industrial robot's response to actions in its environment
|
||||
or obstacle avoidance for autonomous vehicles.
|
||||
|
||||
.. tab-item:: Workload Parameters
|
||||
:sync: workloadparameters
|
||||
|
||||
The workload parameters affect the performance results of the different models we use for benchmarking.
|
||||
Image processing models have different image size definitions and the Natural Language Processing models
|
||||
have different max token list lengths. All these can be found in detail in the :doc:`FAQ section <openvino_docs_performance_benchmarks_faq>`.
|
||||
All models are executed using a batch size of 1. Below are the parameters for the GenAI models we display.
|
||||
|
||||
* Input tokens: 1024,
|
||||
* Output tokens: 128,
|
||||
* number of beams: 1
|
||||
|
||||
For text to image:
|
||||
|
||||
* iteration steps: 20,
|
||||
* image size (HxW): 256 x 256,
|
||||
* input token length: 1024 (the tokens for GenAI models are in English).
|
||||
|
||||
|
||||
Platforms, Configurations, Methodology
|
||||
###########################################################
|
||||
|
|
@ -168,9 +186,9 @@ or `create an account <https://www.intel.com/content/www/us/en/secure/forms/devc
|
|||
Disclaimers
|
||||
####################################
|
||||
|
||||
* Intel® Distribution of OpenVINO™ toolkit performance results are based on release 2023.2, as of November 15, 2023.
|
||||
* Intel® Distribution of OpenVINO™ toolkit performance results are based on release 2023.3, as of January 16, 2024.
|
||||
|
||||
* OpenVINO Model Server performance results are based on release 2023.0, as of June 01, 2023.
|
||||
* OpenVINO Model Server performance results are based on release 2023.0, as of June 06, 2023.
|
||||
|
||||
The results may not reflect all publicly available updates. Intel technologies’ features and benefits depend on system configuration
|
||||
and may require enabled hardware, software, or service activation. Learn more at intel.com, or from the OEM or retailer.
|
||||
|
|
|
|||
|
|
@ -5,9 +5,9 @@ Model Accuracy
|
|||
|
||||
|
||||
|
||||
The following two tables present the absolute accuracy drop calculated as the accuracy difference
|
||||
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
|
||||
FP16 representations of a model on three platform architectures. Please also refer to notes below
|
||||
The following two tables present the absolute accuracy drop calculated as the accuracy difference
|
||||
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
|
||||
FP16 representations of a model on three platform architectures. Please also refer to notes below
|
||||
the table for more information.
|
||||
|
||||
* A - Intel® Core™ i9-9000K (AVX2), INT8 and FP32
|
||||
|
|
@ -105,30 +105,30 @@ the table for more information.
|
|||
* - chatGLM2-6b
|
||||
- lambada openai
|
||||
- ppl
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
- 17.37 (ref. 17.48)
|
||||
- 17.41 (ref. 17.48)
|
||||
- 17.16 (ref. 17.48)
|
||||
* - Llama-2-7b-chat
|
||||
- Wiki, StackExch, Crawl
|
||||
- ppl
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
- 3.24 (ref. 3.26)
|
||||
- 3.24 (ref. 3.26)
|
||||
- 3.25 (ref. 3.26)
|
||||
* - Stable-Diffusion-V2-1
|
||||
- LIAON-5B
|
||||
- ppl
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
* - Mistral-7b
|
||||
- proprietary Mistral.ai
|
||||
- ppl
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
- 3.29 (ref. 3.19)
|
||||
- 3.28 (ref. 3.19)
|
||||
-
|
||||
.. list-table:: Model Accuracy for BF16, FP32 and FP16 (FP16: Flex-170 only. BF16: Xeon(R) 8490H only)
|
||||
:header-rows: 1
|
||||
|
|
@ -232,37 +232,37 @@ the table for more information.
|
|||
* - chatGLM2-6b
|
||||
- lambada openai
|
||||
- ppl
|
||||
-
|
||||
- 17.48 (ref. 17.48)
|
||||
- 17.56 (ref. 17.48)
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
- 17.49 (ref. 17.48)
|
||||
* - Llama-2-7b-chat
|
||||
- Wiki, StackExch, Crawl
|
||||
- ppl
|
||||
-
|
||||
- 3.26 (ref. 3.26)
|
||||
- 3.26 (ref. 3.26)
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
* - Stable-Diffusion-V2-1
|
||||
- LIAON-5B
|
||||
- ppl
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
* - Mistral-7b
|
||||
- proprietary Mistral.ai
|
||||
- ppl
|
||||
-
|
||||
- 3.18 (ref. 3.19)
|
||||
- 3.18 (ref. 3.19)
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
-
|
||||
|
||||
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
|
||||
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
|
||||
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
|
||||
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
|
||||
accuracy for the model.
|
||||
|
||||
|
|
|
|||
|
|
@ -22,11 +22,11 @@ from sphinx.ext.autodoc import ClassDocumenter
|
|||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'OpenVINO™'
|
||||
copyright = '2023, Intel®'
|
||||
copyright = '2024, Intel®'
|
||||
author = 'Intel®'
|
||||
|
||||
language = 'en'
|
||||
version_name = 'nightly'
|
||||
version_name = '2023.3'
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue