From 0245affbb46889bd45e8fde97e91fe98dabd67ad Mon Sep 17 00:00:00 2001 From: Karol Blaszczak Date: Wed, 24 Jan 2024 08:34:59 +0100 Subject: [PATCH] [DOCS] releasenotes 23.3 update (#22362) --- .../releasenotes_for_openvino.rst | 553 ++++++++++-------- .../about_openvino/system_requirements.rst | 124 ++-- 2 files changed, 360 insertions(+), 317 deletions(-) diff --git a/docs/articles_en/about_openvino/releasenotes_for_openvino.rst b/docs/articles_en/about_openvino/releasenotes_for_openvino.rst index 49e4608a0e0..5def73179ca 100644 --- a/docs/articles_en/about_openvino/releasenotes_for_openvino.rst +++ b/docs/articles_en/about_openvino/releasenotes_for_openvino.rst @@ -1,314 +1,347 @@ .. {#openvino_release_notes} OpenVINO Release Notes -====================== +======================================== -The Intel® Distribution of OpenVINO™ toolkit is an open-source solution for optimizing -and deploying AI inference in domains such as computer vision,automatic speech -recognition, natural language processing, recommendation systems, and generative AI. -With its plug-in architecture, OpenVINO enables developers to write once and deploy -anywhere. We are proud to announce the release of OpenVINO 2023.2 introducing a range -of new features, improvements, and deprecations aimed at enhancing the developer -experience. - -New and changed in 2023.2 +2023.3 (LTS) - 24.01.2024 ########################### Summary of major features and improvements ++++++++++++++++++++++++++++++++++++++++++++ -* More Generative AI coverage and framework integrations to minimize code changes. +* More Generative AI coverage and framework integrations to minimize code changes. - * **Expanded model support for direct PyTorch model conversion** - automatically convert - additional models directly from PyTorch or execute via ``torch.compile`` with OpenVINO - as the backend. - * **New and noteworthy models supported** - we have enabled models used for chatbots, - instruction following, code generation, and many more, including prominent models - like Llava, chatGLM, Bark (text to audio) and LCM (Latent Consistency Models, an - optimized version of Stable Diffusion). - * **Easier optimization and conversion of Hugging Face models** - compress LLM models - to Int8 with the Hugging Face Optimum command line interface and export models to - the OpenVINO IR format. - * **OpenVINO is now available on Conan** - a package manager which allows more seamless - package management for large scale projects for C and C++ developers. + * Introducing `OpenVINO Gen AI repository `__ + on GitHub that demonstrates native C and C++ pipeline samples for Large Language Models + (LLMs). String tensors are now supported as inputs and tokenizers natively to reduce + overhead and ease production. + * New and noteworthy models validated; Mistral, Zephyr, Qwen, ChatGLM3, and Baichuan + * New Jupyter Notebooks for + `Latent Consistency Models (LCM) `__ + and `Distil-Whisper `__. + Updated `LLM Chatbot notebook `__ + to include LangChain, Neural Chat, TinyLlama, ChatGLM3, Qwen, Notus and Youri models. + * Torch.compile is now fully integrated with OpenVINO, which now includes a hardware + 'options' parameter allowing for seamless inference hardware selection by leveraging + the plugin architecture in OpenVINO. -* Broader Large Language Model (LLM) support and more model compression techniques. +* Broader Large Language Model (LLM) support and more model compression techniques. - * Accelerate inference for LLM models on Intel® CoreTM CPU and iGPU with the - use of Int8 model weight compression. - * Expanded model support for dynamic shapes for improved performance on GPU. - * Preview support for Int4 model format is now included. Int4 optimized model - weights are now available to try on Intel® Core™ CPU and iGPU, to accelerate - models like Llama 2 and chatGLM2. - * The following Int4 model compression formats are supported for inference - in runtime: - - * Generative Pre-training Transformer Quantization (GPTQ); with GPTQ-compressed - models, you can access them through the Hugging Face repositories. - * Native Int4 compression through Neural Network Compression Framework (NNCF). + * As part of the Neural Network Compression Framework (NNCF), INT4 weight compression model + formats are now fully supported on Intel® Xeon® CPUs in addition to Intel® Core™ and iGPU, + adding more performance, lower memory usage, and accuracy opportunity when using LLMs. + * Improved performance of transformer based LLM on CPU and GPU using stateful model technique + to increase memory efficiency where internal states are shared among multiple iterations of + inference. + * Easier optimization and conversion of Hugging Face models - compress LLM models to INT8 + and INT4 with Hugging Face Optimum command line interface and export models to OpenVINO + format. Note this is part of `Optimum-Intel `__ + which needs to be installed separately. + * Tokenizer and TorchVision transform support is now available in the OpenVINO runtime + (via new API) requiring less preprocessing code and enhancing performance by automatically + handling this model setup. More details on Tokenizers support in Ecosystem section. * More portability and performance to run AI at the edge, in the cloud, or locally. - - * **In 2023.1 we announced full support for ARM** architecture, now we have improved - performance by enabling FP16 model formats for LLMs and integrating additional - acceleration libraries to improve latency. - + + * Full support for 5th Gen Intel® Xeon® Scalable processors (codename Emerald Rapids). + * Further optimized performance on Intel® Core™ Ultra (codename Meteor Lake) CPU with + latency hint, by leveraging both P-core and E-cores. + * Improved performance on ARM platforms using throughput hint, which increases efficiency + in utilization of CPU cores and memory bandwidth. + * Preview JavaScript API to enable node JS development to access JavaScript binding via + source code. See details below. + * Improved `model serving of LLMs `__ + through OpenVINO Model Server. This not only enables LLM serving over KServe v2 gRPC + and REST APIs for more flexibility, but also improves throughput by running processing + like tokenization on the server side. More details in the Ecosystem section. + + Support Change and Deprecation Notices ++++++++++++++++++++++++++++++++++++++++++ -* The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated - and will be removed from installation options and distribution channels with - 2025.0. To learn more, refer to the - :doc:`OpenVINO Legacy Features and Components page `. - To ensure optimal performance, install the OpenVINO package (pip install openvino), - which includes essential components such as OpenVINO Runtime, OpenVINO Converter, - and Benchmark Tool. +* The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated and will be + removed from installation options and distribution channels beginning with 2025.0. + For more details, refer to the :doc:`OpenVINO Legacy Features and Components ` + page. +* Ubuntu 18.04 support will be discontinued in the 2023.3 LTS release. The recommended version + of Ubuntu is 22.04. +* Starting in release 2023.3 OpenVINO will no longer support Python 3.7 due to the Python + community discontinuing support. Update to a newer version (currently 3.8-3.11) to avoid + interruptions. +* All ONNX Frontend legacy API (known as ONNX_IMPORTER_API) will no longer be available in + 2024.0 release. +* ``PerfomanceMode.UNDEFINED`` property as part of the OpenVINO Python API will be + discontinued in the 2024.0 release. * Tools: * :doc:`Deployment Manager ` - is deprecated and will be removed in the 2024.0 release. + is deprecated and will be supported for two years according to our LTS policy. + Visit our :doc:`selector tool ` to see + package distribution options or our :doc:`deployment guide ` + documentation. * Accuracy Checker is deprecated and will be discontinued with 2024.0. - * Post-Training Optimization Tool (POT) is deprecated and will be - discontinued with 2024.0. - * Model Optimizer is deprecated and will be fully supported up until the 2025.0 - release. Model conversion to the OpenVINO format should be performed through - OpenVINO Model Converter, which is part of the PyPI package. Follow the - :doc:`Model Optimizer to OpenVINO Model Converter transition ` - guide for smoother transition. Known limitations are TensorFlow model with - TF1 Control flow and object detection models. These limitations relate to - the gap in TensorFlow direct conversion capabilities which will be addressed - in upcoming releases. - * PyTorch 1.13 support is deprecated in Neural Network Compression Framework (NNCF) + * Post-Training Optimization Tool (POT) has been deprecated and the 2023.3 LTS will be + the last release that will support the tool. Developers are encouraged to use the Neural + Network Compression Framework (NNCF) for this feature. + * Model Optimizer is deprecated and will be fully supported until the 2025.0 release. + We encourage developers to perform model conversion through OpenVINO Model Converter + (API call: OVC). Follow the + :doc:`model conversion transition guide ` + for more details. + * Deprecated support for a `git patch `__ + for NNCF integration with `huggingface/transformers `__. + The recommended approach is to use `huggingface/optimum-intel `__ + for applying NNCF optimization on top of models from Hugging Face. + * Support for Apache MXNet, Caffe, and Kaldi model formats is deprecated and will be + discontinued with the 2024.0 release. * Runtime: - * Intel® Gaussian & Neural Accelerator (Intel® GNA) will be deprecated in a future - release. We encourage developers to use the Neural Processing Unit (NPU) for - low powered systems like Intel® Core™ Ultra or 14th generation and beyond. - * OpenVINO C++/C/Python 1.0 APIs will be discontinued with 2024.0. - * Python 3.7 support has been discontinued. + * Intel® Gaussian & Neural Accelerator (Intel® GNA) will be deprecated in a future release. + We encourage developers to use the Neural Processing Unit (NPU) for low-powered systems + like Intel® CoreTM Ultra or 14th generation and beyond. + * OpenVINO C++/C/Python 1.0 APIs are deprecated and will be discontinued in the 2024.0 release. + Please use API 2.0 in your applications going forward to avoid disruption. + * OpenVINO property Affinity API will be deprecated from 2024.0 and will be discontinued in 2025.0. + It will be replaced with CPU binding configurations (``ov::hint::enable_cpu_pinning``). + OpenVINO™ Development Tools ++++++++++++++++++++++++++++++++++++++++++ -List of components and their changes: ------------------------------------------- - -* :doc:`OpenVINO Model Converter tool ` - now supports the original framework shape format. * `Neural Network Compression Framework (NNCF) `__ - - * Added data-free Int4 weight compression support for LLMs in OpenVINO IR with - ``nncf.compress_weights()``. - * Improved quantization time of LLMs with NNCF PTQ API for ``nncf.quantize()`` - and ``nncf.quantize_with_accuracy_control()``. - * Added support for SmoothQuant and ChannelAlighnment algorithms in NNCF HyperParameter - Tuner for automatic optimization of their hyperparameters during quantization. - * Added quantization support for the ``IF`` operation of models in OpenVINO format - to speed up such models. - * NNCF Post-training Quantization for PyTorch backend is now supported with - ``nncf.quantize()`` and the common implementation of quantization algorithms. - * Added support for PyTorch 2.1. PyTorch 1.13 support has been deprecated. -OpenVINO™ Runtime (previously known as Inference Engine) ---------------------------------------------------------- + * Weight compression API, ``nncf.compress_weights()``, has been extended by: -* OpenVINO Common + * When using the 'all_layers' parameter, it compresses the model, including embeddings + and final layers, to the 4-bit format. This helps make the model footprint smaller + and improves performance, but it might impact the model accuracy. By default, this + parameter is disabled, and the backup precision (INT8) is assigned for the embeddings + and last layers. + * When using INT8_SYM compression mode for better performance of the compressed model + in case of 8-bit weight compression you might experience an impact on model accuracy + therefore by default, we use INT8_ASYM mode to better balance performance and accuracy. + * We implemented a 4-bit data-aware weight compression feature, introducing the 'dataset' + optional parameter in ``nncf.compress_weights()``. This parameter can be utilized to + mitigate accuracy loss in compressed models. It's important to note that enabling + this option will extend the compression time. + * Post-training Quantization with Accuracy Control, ``nncf.quantize_with_accuracy_control()``, + has been extended by the 'restore_mode' optional parameter to revert weights to INT8 + instead of the original precision. This parameter helps to reduce the size of the + quantized model and improves its performance. By default, it is disabled and model + weights are reverted to the original precision in ``nncf.quantize_with_accuracy_control()``. - * Operations for reference implementations updated from legacy API to API 2.0. - * Symbolic transformation introduced the ability to remove Reshape operations - surrounding MatMul operations. - -* OpenVINO Python API - - * Better support for the ``openvino.properties`` submodule, which now allows the use - of properties directly, without additional parenthesis. Example use-case: - ``{openvino.properties.cache_dir: “./some_path/”}``. - * Added missing properties: ``execution_devices`` and ``loaded_from_cache``. - * Improved error propagation on imports from OpenVINO package. - -* AUTO device plug-in (AUTO) - - * o Provided additional option to improve performance of cumulative throughput - (or MULTI), where part of CPU resources can be reserved for GPU inference - when GPU and CPU are both used for inference (using ``ov::hint::enable_cpu_pinning(true)``). - This avoids the performance issue of CPU resource contention where there - is not enough CPU resources to schedule tasks for GPU - (`PR #19214 `__). - -* CPU - - * Introduced support of GPTQ quantized Int4 models, with improved performance - compared to Int8 weight-compressed or FP16 models. In the CPU plugin, - the gain in performance is achieved by FullyConnected acceleration with - 4bit weight decompression - (`PR #20607 `__). - * Improved performance of Int8 weight-compressed large language models on - some platforms, such as 13th Gen Intel Core - (`PR #20607 `__). - * Further reduced memory consumption of select large language models on - CPU platforms with AMX and AVX512 ISA, by eliminating extra memory copy - with a unified weight layout - (`PR #19575 `__). - - * Fixed performance issue observed in 2023.1 release on select Xeon CPU - platform with improved thread workload partitioning matching L2 cache - utilization - (`PR #20436 `__). - * Extended support of configuration (enable_cpu_pinning) on Windows - platforms to allow fine-grain control on CPU resource used for inference - workload, by binding inference thread to CPU cores - (`PR #19418 `__). - * Optimized YoloV8n and YoloV8s model performance for BF16/FP32 precision. - * Optimized Falcon model on 4th Gen Intel® Xeon® Scalable Processors. - * Enabled support for FP16 inference precision on ARM. - -* GPU - - * Enhanced inference performance for Large Language Models. - * Introduced int8 weight compression to boost LLM performance. - (`PR #19548 `__). - * Implemented Int4 GPTQ weight compression for improved LLM performance. - * Optimized constant weights for LLMs, resulting in better memory usage - and faster model loading. - * Optimized gemm (general matrix multiply) and fc (fully connected) for - enhanced performance on iGPU. - (`PR #19780 `__). - * Completed GPU plugin migration to API 2.0. - * Added support for oneDNN 3.3 version. +OpenVINO™ Runtime +++++++++++++++++++++++++ * Model Import Updates - * TensorFlow Framework Support + * TensorFlow Framework Support - * Supported conversion of models from memory in keras.Model and tf.function formats. - `PR #19903 `__ - * Supported TF 2.14. - `PR #20385 `__ + * Supported TF1 While Control flow construction w/o TensorArray operations + (`PR #20800 `__). + * Support for complex tensors has been added + (`PR #20860 `__), + (`PR #21477 `__). + * Provided fixes for the following: - * PyTorch Framework Support + * Accept any model file extension for frozen protobuf format + (`PR #21508 `__). + * Correct ArgMin/ArgMax translators for repeating elements case + (`PR #21364 `__). + * Correct PartitionedCall translator when numbers of external and internal + body inputs mismatch + (`PR #20825 `__). - * Supported Int4 GPTQ models. - * New operations supported. + * PyTorch Framework Support - * ONNX Framework Support + * Added support of nested dictionaries and lists as example input. + * Disabled ``torch.jit.freeze`` in default model tracing scenario and + improved support for models without freezing, extending model + coverage and improving accuracy for some models. - * Added support for ONNX version 1.14.1 - (`PR #18359 `__) + * ONNX Framework Support + + * Switched to ONNX 1.15.0 as a supported version of original framework + (`PR #20929 `__). + +* CPU + + * Full support for 5th Gen Intel® Xeon® Scalable processors (codename Emerald Rapids) + with sub-numa (SNC) and efficient core resource scheduling to improve performance. + * Further optimized performance on Intel® Core™ Ultra (codename Meteor Lake) CPU with + latency hint, by leveraging both P-core and E-cores. + * Further improved performance of LLMs in INT4 weight compression, especially on 1st + token latency and on 4th and 5th Gen of Intel Xeon platforms (codename Sapphire + Rapids and Emerald Rapids) with AMX capabilities. + * Improved performance of transformer-based LLM using stateful model technique to + increase memory efficiency where internal states (KV cache) are shared among multiple + iterations of inference. The stateful model implementation supports both greedy search + and beam search (preview) for LLMs. This technique also reduces the memory footprint + of LLMs, where Intel Core and Ultra platforms like Raptor Lake and Meteor Lake can + run INT4 models, such as Llama v2 7B. + * Improved performance on ARM platforms with throughput hint, by increasing + efficiency in usage of the CPU cores and memory bandwidth. + +* GPU + + * Full support for Intel® Core™ Ultra (codename Meteor Lake) integrated graphics. + * For LLMs, the first inference latency for INT8 and INT4 weight-compressed models has + been improved on iGPU thanks to more efficient context processing. Overall average + token latency for INT8 and INT4 has also been enhanced on iGPU with graph compilation + optimization, various host overhead optimization, and dynamic padding support for GEMM. + * Stateful model is functionally supported for LLMs. + * Model caching for dynamically shaped models is now supported. Model loading time is + improved for these models, including LLMs. + * API for switching between size mode (model caching) and speed mode (kernel caching) + is introduced. + * The model cache file name is changed to be independent of GPU driver versions. + The GPU will not generate separate model cache files when the driver is updated. + * Compilation time for Stable Diffusion models has been improved. + +* NPU + + * NPU plugin is available as part of OpenVINO. With the Intel(R) Core Ultra NPU driver + installed, inference can run on the NPU device. + +* AUTO device plug-in (AUTO) + + * Introduced the round-robin policy to AUTO cumulative throughput hint, which dispatches + inference requests to multiple devices (such as multiple GPU devices) in the round-robin + sequence, instead of in the device priority sequence. The device priority sequence + remains as the default configuration. + * AUTO loads stateful models to GPU or CPU per device priority, since GPU now supports + stateful model inference. + +* OpenVINO Common + + * Enhanced support of String tensors has been implemented, enabling the use of operators + and models that rely on string tensors. This update also enhances the capability in + the torchvision preprocessing (`PR #21244 `__). + * A new feature has been added that enables the selection of P-Cores for model compilation + on CPU device(s) with hybrid architecture (i.e. Intel® Core™ 12th Gen and beyond). + This will reduce compilation time compared to previous implementation where P-cores + and E-cores are used randomly by OS scheduling. + +* OpenVINO JavaScript API (preview feature) + + * We've introduced a preview version of + `JS API `__ + for OpenVINO runtime in this release. We hope that you will try this feature and + provide your feedback through GitHub `issues `__. + * Known limitations: + + * Only supported in manylinux and x86 (Windows, ARM, ARM64, and macOS have not been tested) + * Node.js version >= 18.16 + * CMake version < 3.14 is not supported + * gcc compiler version < 7 is not supported + +* OpenVINO Python API + + * Introducing string tensor support for Python API. + * Added support for the following: + * Create ov.Tensor from Python lists + * Create ov.Tensor from empty numpy arrays. + * Constants from empty numpy arrays. + * Autogenerated get/set methods for Node attributes. + * Inference functions (InferRequest.infer/start_async, CompiledModel.__call__ etc.) + support OVDict as the input. + * PILLOW interpolation modes bindings. + (`PR #21188 `__ + external contribution: @meetpatel0963) + * Torchvision to :doc:`OpenVINO preprocessing ` + converter documentation has been added to OpenVINO docs. OpenVINO Ecosystem +++++++++++++++++++++++++++++++++++++++++++++ -OpenVINO Model Server --------------------------- +* OpenVINO Tokenizer (Preview feature) -Introduced an extension of the KServe gRPC API, enabling streaming input and -output for servables with Mediapipe graphs. This extension ensures the persistence -of Mediapipe graphs within a user session, improving processing performance. -This enhancement supports stateful graphs, such as tracking algorithms, and -enables the use of source calculators. -(`see additional documentation `__) + * OpenVINO Tokenizer adds text processing operations to OpenVINO: -* Mediapipe framework has been updated to the version 0.10.3. -* model_api used in the openvino inference Mediapipe calculator has been updated - and included with all its features. -* Added a demo showcasing gRPC streaming with Mediapipe graph. - (`see here `__) -* Added parameters for gRPC quota configuration and changed default gRPC channel - arguments to add rate limits. It will minimize the risks of impact of the service - from uncontrolled flow of requests. -* Updated python clients requirements to match wide range of python versions from 3.6 to 3.11 + * Text PrePostprocessing without third-party dependencies + * Convert a HuggingFace tokenizer into the OpenVINO model tokenizer and the + detokenizer using a CLI tool or Python API + * Connect a tokenizer and a model to get a single model with text input -Learn more about the changes in https://github.com/openvinotoolkit/model_server/releases - -Jupyter Notebook Tutorials ------------------------------ - -* The following notebooks have been updated or newly added: - - * `LaBSE `__ - Cross-lingual Books Alignment With Transformers - * `LLM chatbot `__ - Create LLM-powered Chatbot - - * Updated to include Int4 weight compression and Zephyr 7B model - - * `Bark Text-to-Speech `__ - Text-to-Speech generation using Bark - * `LLaVA Multimodal Chatbot `__ - Visual-language assistant with LLaVA - * `BLIP-Diffusion - Subject-Driven Generation `__ - Subject-driven image generation and editing using BLIP Diffusion - * `DeciDiffusion `__ - Image generation with DeciDiffusion - * `Fast Segment Anything `__ - Object segmentations with FastSAM - * `SoftVC VITS Singing Voice Conversion `__ - * `QR Code Monster `__ - Generate creative QR codes with ControlNet QR Code Monster - * `Würstchen `__ - Text-to-image generation with Würstchen - * `Distil-Whisper `__ - Automatic speech recognition using Distil-Whisper and OpenVINO™ + * OpenVINO Tokenizer models work only on the CPU device + * Supported platforms: Linux (x86 and ARM), Windows and Mac (x86 and ARM) -* Added optimization support (8-bit quantization, weight compression) - by NNCF for the following notebooks: +* OpenVINO Model Server + + * Added support for serving pipelines with custom nodes implemented as a + `python code `__ + This greatly simplifies exposing GenAI algorithms based on Hugging Face + and Optimum libraries. It can be also applied for arbitrary pre and + post-processing in model serving pipelines. + * Included a new set of model serving demos that use custom nodes with python + code. These include LLM `text generation `__, + `stable diffusion `__, + and `seq2seq translation `__. + * Improved video stream analysis `demo `__. + A simple client example can now process the + video stream from a local camera, video file or RTSP stream. + * Learn more about these changes on + `GitHub `__. + + +* Jupyter Notebook Tutorials + + * The following notebooks have been updated or newly added: + + * `Sound generation with AudioLDM2 and OpenVINO `__. + * `Single-step image generation using SDXL-turbo and OpenVINO `__. + * `Paint by Example using Diffusion models and OpenVINO™ `__. + * `LLM-powered chatbot using Stable-Zephyr-3b and OpenVINO `__. + * `Object segmentations with EfficientSAM and OpenVINO `__. + * `Create an LLM-powered RAG system using OpenVINO `__ + - Demonstrates an integration with LangChain. + * `High-resolution image generation with Segmind-VegaRT and OpenVINO `__. + * `Text-to-Image Generation with LCM LoRA and ControlNet Conditioning `__. + * `LLM Instruction-following pipeline with OpenVINO `__ - + Demonstrates how to run an instruction-following text generation pipeline using + tiny-llama-1b-chat, phi-2, dolly-v2-3b, red-pajama-3b-instruct and mistral-7b models. + * `LLM chabot notebook `__ + updated with support for new LLMs and INT4/INT8 Weight Compression: TinyLlama-1b-chat, + Mistral-7B, neural-chat-7b, notus-7b, ChatGLM3, youri-7b-chat (for Japanese language). + + * Added optimization support (8-bit quantization, weight compression) by NNCF for the following notebooks: + + * `Image generation with Würstchen and OpenVINO `__ + * `QR-code monster `__ + * `INT4-compression support for LLaVA multimodal chatbot `__ + * `Distil-whisper quantization `__ + - * `Image generation with DeepFloyd IF `__ - * `Instruction following using Databricks Dolly 2.0 `__ - * `Visual Question Answering and Image Captioning using BLIP `__ - * `Grammatical Error Correction `__ - * `Universal segmentation with OneFormer `__ - * `Visual-language assistant with LLaVA and OpenVINO `__ - * `Image editing with InstructPix2Pix `__ - * `MMS: Scaling Speech Technology to 1000+ languages `__ - * `Image generation with Latent Consistency Model `__ - * `Object segmentations with FastSAM `__ - * `Automatic speech recognition using Distil-Whisper `__ Known issues ++++++++++++++++++++++++++++++++++++++++++++ -| **ID - 118179** -| *Component* - Python API, Plugins +| **ID - 127202** +| *Component* - CPU Plugin | *Description:* -| When input byte sizes are matching, inference methods accept incorrect inputs - in copy mode (share_inputs=False). Example: [1, 4, 512, 512] is allowed when - [1, 512, 512, 4] is required by the model. +| Deeplabv3 model from TF framework shows lower performance than previous + release. This is because the TopK layer in the model is now correctly + conducting the stable sort as specified by the model, slower than the + previous unstable sort. | *Workaround:* -| Pass inputs which shape and layout match model ones. - -| **ID - 124181** -| *Component* - CPU plugin -| *Description:* -| On CPU platform with L2 cache size less than 256KB, such as i3 series of 8th - Gen Intel CORE platforms, some models may hang during model loading. -| *Workaround:* -| Rebuild the software from OpenVINO master or use the next OpenVINO release. - -| **ID - 121959** -| *Component* - CPU plugin -| *Description:* -| During inference using latency hint on selected hybrid CPU platforms - (such as 12th or 13th Gen Intel CORE), there is a sporadic occurrence of - increased latency caused by the operating system scheduling of P-cores or - E-cores during OpenVINO initialization. -| *Workaround:* -| This will be fixed in the next OpenVINO release. +| This release has the correct behavior. If performance is critical, + please use the previous version of OpenVINO, or tune the model. | **ID - 123101** -| *Component* - GPU plugin +| *Component* - GPU plugin | *Description:* -| Hung up of GPU plugin on A770 Graphics (dGPU) in case of - large batch size (1750). +| Hung up of GPU plugin on A770 Graphics (dGPU) in case of large + batch size (1750). | *Workaround:* -| Decrease the batch size, wait for fixed driver released. +| Decrease the batch size, and wait for the fixed driver released. + Included in This Release +++++++++++++++++++++++++++++++++++++++++++++ @@ -320,19 +353,19 @@ three types of operating systems: Windows, Linux, and macOS. || Component || License | Location | +================================+===================================+=================+=================+=======================+=================================================+ || OpenVINO (Inference Engine) C++ Runtime || Dual licensing: || /runtime/* | -|| Unified API to integrate the inference with application logic || Intel® OpenVINO™ Distribution License (Version May 2021) || /runtime/include/* | -|| OpenVINO (Inference Engine) Headers || Apache 2.0 || | +|| Unified API to integrate the inference with application logic || Intel® OpenVINO™ Distribution License (Version May 2021) || /runtime/include/* | +|| OpenVINO (Inference Engine) Headers || Apache 2.0 || | +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ -|| OpenVINO (Inference Engine) Pythion API || Apache 2.0 || /python/* | +|| OpenVINO (Inference Engine) Python API || Apache 2.0 || /python/* | +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ || OpenVINO (Inference Engine) Samples || Apache 2.0 || /samples/* | || Samples that illustrate OpenVINO C++/ Python API usage || || | +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ -|| [Deprecated] Deployment manager || Apache 2.0 || /tools/deployment_manager/* | -|| The Deployment Manager is a Python* command-line tool that || || | -|| creates a deployment package by assembling the model, IR files, || || | -|| your application, and associated dependencies into a runtime || || | -|| package for your target device. || || | +|| [Deprecated] Deployment manager || Apache 2.0 || /tools/deployment_manager/* | +|| The Deployment Manager is a Python command-line tool that || || | +|| creates a deployment package by assembling the model, IR files, || || | +|| your application, and associated dependencies into a runtime || || | +|| package for your target device. || || | +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ @@ -360,7 +393,7 @@ enabled hardware, software or service activation. Learn more at `http://www.intel.com/ `__ or from the OEM or retailer. -No computer system can be absolutely secure. +No computer system can be absolutely secure. Intel, Atom, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. @@ -371,18 +404,18 @@ Other names and brands may be claimed as the property of others. Copyright © 2023, Intel Corporation. All rights reserved. -For more complete information about compiler optimizations, see our Optimization Notice. - -Performance varies by use, configuration and other factors. Learn more at +For more complete information about compiler optimizations, see our Optimization Notice. + +Performance varies by use, configuration and other factors. Learn more at `www.Intel.com/PerformanceIndex `__. Download +++++++++++++++++++++++++++++++++++++++++++++ `The OpenVINO product selector tool `__ -provides easy access to the right packages that match your desired OS, version, +provides easy access to the right packages that match your desired OS, version, and distribution options. - + diff --git a/docs/articles_en/about_openvino/system_requirements.rst b/docs/articles_en/about_openvino/system_requirements.rst index 12395af9239..2448214d3ba 100644 --- a/docs/articles_en/about_openvino/system_requirements.rst +++ b/docs/articles_en/about_openvino/system_requirements.rst @@ -2,13 +2,16 @@ System Requirements =================== - -Certain hardware requires specific drivers to work properly with OpenVINO. -These drivers, including Linux* kernels, might require updates to your operating system, -which is not part of OpenVINO installation. Refer to your hardware's documentation -for updating instructions. +.. note:: + + Certain hardware (including but not limited to GPU, GNA, and latest CPUs) requires manual + installation of specific drivers and/or other software components to work correctly and/or + to utilize hardware capabilities at their best. This might require updates to operating + system, including but not limited to Linux kernel, please refer to their documentation + for details. These modifications should be handled by user and are not part of OpenVINO + installation. CPU @@ -20,26 +23,24 @@ CPU * Intel Atom® processor with Intel® SSE4.2 support * Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics - * 6th - 13th generation Intel® Core™ processors - * Intel® Core™ Ultra (codename Meteor Lake) - * Intel® Xeon® Scalable Processors (code name Skylake) - * 2nd Generation Intel® Xeon® Scalable Processors (code name Cascade Lake) - * 3rd Generation Intel® Xeon® Scalable Processors (code name Cooper Lake and Ice Lake) - * 4th Generation Intel® Xeon® Scalable Processors (code name Sapphire Rapids) - * ARM* and ARM64 CPUs; Apple M1, M2 and Raspberry Pi + * 6th - 14th generation Intel® Core™ processors + * Intel® Core™ Ultra (codename Meteor Lake) + * 1st - 5th generation Intel® Xeon® Scalable Processors + * ARM and ARM64 CPUs; Apple M1, M2, and Raspberry Pi .. tab-item:: Supported Operating Systems * Ubuntu 22.04 long-term support (LTS), 64-bit (Kernel 5.15+) * Ubuntu 20.04 long-term support (LTS), 64-bit (Kernel 5.15+) - * Ubuntu 18.04 long-term support (LTS) with limitations, 64-bit (Kernel 5.4+), ARM64 - * Windows* 10 - * Windows* 11 - * macOS* 10.15 and above, 64-bit + * Ubuntu 18.04 long-term support (LTS) with limitations, 64-bit (Kernel 5.4+) + * Windows 10 + * Windows 11 + * macOS 10.15 and above, 64-bit * macOS 11 and above, ARM64 - * Red Hat Enterprise Linux* 8, 64-bit + * CentOS 7 + * Red Hat Enterprise Linux 8, 64-bit + * Ubuntu 18 ARM64 * Debian 9 ARM - * CentOS 7 64-bit GPU ########## @@ -53,7 +54,7 @@ GPU * Intel® Iris® Pro Graphics * Intel® Iris® Xe Graphics * Intel® Iris® Xe Max Graphics - * Intel® Arc ™ GPU Series + * Intel® Arc™ GPU Series * Intel® Data Center GPU Flex Series * Intel® Data Center GPU Max Series @@ -63,7 +64,7 @@ GPU * Ubuntu 20.04 long-term support (LTS), 64-bit * Windows 10, 64-bit * Windows 11, 64-bit - * Centos 7 + * CentOS 7 * Red Hat Enterprise Linux 8, 64-bit .. tab-item:: Additional considerations @@ -72,17 +73,17 @@ GPU Distribution of OpenVINO™ toolkit package. * A chipset that supports processor graphics is required for Intel® Xeon® processors. Processor graphics are not included in all processors. See - `Product Specifications `__ - for information about your processor. - * Although this release works with Ubuntu 20.04 for discrete graphic cards, - Ubuntu 20.04 is not POR for discrete graphics drivers, so OpenVINO support - is limited. + `Product Specifications `__ + for information about your processor. + * While this release of OpenVINO supports Ubuntu 20.04, the driver stack + for Intel discrete graphic cards does not fully support Ubuntu 20.04. + We recommend using Ubuntu 22.04 when executing on discrete graphics. * The following minimum (i.e., used for old hardware) OpenCL™ driver's versions were used during OpenVINO internal validation: 22.43 for Ubuntu 22.04, 21.48 - for Ubuntu 20.04 and 21.49 for Red Hat Enterprise Linux 8. + for Ubuntu 20.04 and 21.49 for Red Hat Enterprise Linux 8. -NPU and GNA -############################# +Intel® Neural Processing Unit +################################ .. tab-set:: @@ -100,10 +101,23 @@ NPU and GNA .. tab-item:: Additional considerations - * These Accelerators require drivers that are not included in the - Intel® Distribution of OpenVINO™ toolkit package. + * These Accelerators require :doc:`drivers ` + that are not included in the Intel® Distribution of OpenVINO™ toolkit package. * Users can access the NPU plugin through the OpenVINO archives on - the download page. + the :doc:`download page `. + + +Intel® Gaussian & Neural Accelerator +########################################## + +.. tab-set:: + + .. tab-item:: Operating Systems for GNA + + * Ubuntu 22.04 long-term support (LTS), 64-bit + * Ubuntu 20.04 long-term support (LTS), 64-bit + * Windows 10, 64-bit + * Windows 11, 64-bit Operating systems and developer environment @@ -111,7 +125,7 @@ Operating systems and developer environment .. tab-set:: - .. tab-item:: Linux + .. tab-item:: Linux OS * Ubuntu 22.04 with Linux kernel 5.15+ * Ubuntu 20.04 with Linux kernel 5.15+ @@ -127,16 +141,13 @@ Operating systems and developer environment * `GNU Compiler Collection (GCC) `__ 7.5 and above * `CMake `__ 3.10 or higher - Higher versions of kernel might be required for 10th Gen Intel® Core™ Processors, - 11th Gen Intel® Core™ Processors, 11th Gen Intel® Core™ Processors S-Series Processors, - 12th Gen Intel® Core™ Processors, 13th Gen Intel® Core™ Processors, Intel® Core™ Ultra - Processors, or 4th Gen Intel® Xeon® Scalable Processors to support CPU, GPU, GNA or - hybrid-cores CPU capabilities. + Higher versions of kernel might be required for 10th Gen Intel® Core™ Processors, 11th Gen + Intel® Core™ Processors, 11th Gen Intel® Core™ Processors S-Series Processors, 12th Gen + Intel® Core™ Processors, 13th Gen Intel® Core™ Processors, 14th Gen + Intel® Core™ Processors, Intel® Core™ Ultra Processors, 4th Gen Intel® Xeon® Scalable Processors + or 5th Gen Intel® Xeon® Scalable Processors to support CPU, GPU, GNA or hybrid-cores CPU capabilities. - .. tab-item:: Windows - - * Windows 10 - * Windows 11 + .. tab-item:: Windows 10 and 11 Build environment components: @@ -158,26 +169,25 @@ Operating systems and developer environment .. tab-item:: DL frameworks versions: - * TensorFlow* 1.15, 2.12 - * MxNet* 1.9.0 - * ONNX* 1.14.1 - * PaddlePaddle* 2.4 + * TensorFlow 1.15, 2.12 + * MxNet 1.9.0 + * ONNX 1.14.1 + * PaddlePaddle 2.4 - This package can be installed on other versions of DL Framework - but only the version specified here is fully validated. + This package can be installed on other versions of DL Frameworks + but only the versions specified here are fully validated. .. note:: - OpenVINO Python binaries and binaries on Windows/CentOS7/MACOS(x86) are built - with oneTBB libraries. Other binaries on Ubuntu and Redhat OSes are built with - legacy TBB which is released by OS distribution. OpenVINO can be built with - either oneTBB or legacy TBB by the user on all OS systems listed. System - compatibility and performance are improved on Hybrid CPUs, + OpenVINO Python binaries and binaries on Windows, CentOS 7, and macOS (x86) are built + with oneTBB libraries, and others on Ubuntu and RedHat systems are built with + legacy TBB which is released by OS distribution. OpenVINO can be built from source + with either oneTBB or legacy TBB on all the systems listed here. System + compatibility and performance are improved on Hybrid CPUs such as 12th Gen Intel Core and above. - Legal Information +++++++++++++++++++++++++++++++++++++++++++++ @@ -202,7 +212,7 @@ enabled hardware, software or service activation. Learn more at `http://www.intel.com/ `__ or from the OEM or retailer. -No computer system can be absolutely secure. +No computer system can be absolutely secure. Intel, Atom, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. @@ -213,9 +223,9 @@ Other names and brands may be claimed as the property of others. Copyright © 2023, Intel Corporation. All rights reserved. -For more complete information about compiler optimizations, see our Optimization Notice. - -Performance varies by use, configuration and other factors. Learn more at +For more complete information about compiler optimizations, see our Optimization Notice. + +Performance varies by use, configuration and other factors. Learn more at `www.Intel.com/PerformanceIndex `__.