55c11c6324
### Details: Ported SDPA to PagedAttention transformation from python to C++ code. the related PRs: https://github.com/openvinotoolkit/openvino/pull/24127 https://github.com/openvinotoolkit/openvino/pull/24177 Tested model scope: - [x] "hf-internal-testing/tiny-random-BloomForCausalLM", - [x] "hf-internal-testing/tiny-random-FalconForCausalLM", - [x] "hf-internal-testing/tiny-random-Starcoder2ForCausalLM", - [x] "hf-internal-testing/tiny-random-GPTJForCausalLM", - [x] "hf-internal-testing/tiny-random-StableLmForCausalLM", - [x] "hf-internal-testing/tiny-random-LlamaForCausalLM", - [x] "hf-internal-testing/tiny-random-MistralForCausalLM", - [x] "hf-internal-testing/tiny-random-OPTForCausalLM", - [x] "hf-internal-testing/tiny-random-PhiForCausalLM", - [x] "hf-internal-testing/tiny-random-StableLmForCausalLM", - [x] "facebook/opt-125m", - [x] "llama2", - [x] "bigcode/starcoder2-7b" - [ ] "mosaicml/mpt-7b-chat" (FAILED both py/c++) - acceptable for this PR Issue: RuntimeError: Check '(axis_range_min <= axis) && (axis <= axis_range_max)' failed at src/core/src/validation_util.cpp:386: Concat Parameter axis 2 out of the tensor rank range [0, 0]. - [x] _means, that the response to the dedicated prompt is the same for the py and c++ transformations._ ### Tickets: - *CVS-138664* --------- Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> Co-authored-by: Andrii Staikov <andrii.staikov@intel.com> |
||
|---|---|---|
| .. | ||
| dev_api/openvino | ||
| docs | ||
| include/openvino | ||
| reference | ||
| shape_inference | ||
| src | ||
| template_extension | ||
| tests | ||
| CMakeLists.txt | ||
| README.md | ||
README.md
OpenVINO™ Core
OpenVINO Core is a part of OpenVINO Runtime library. The component is responsible for:
- Model representation - component provides classes for manipulation with models inside the OpenVINO Runtime. For more information please read Model representation in OpenVINO Runtime User Guide
- Operation representation - contains all from the box supported OpenVINO operations and opsets. For more information read Operations enabling flow guide.
- Model modification - component provides base classes which allow to develop transformation passes for model modification. For more information read Transformation enabling flow guide.
OpenVINO Core supports conditional compilation feature and uses the common coding style rules.
Key person
People from the openvino-ngraph-maintainers allows to approve and merge PRs to the core component. These guys can help in case of any questions about core component.
Components
OpenVINO Core has the next structure:
- dev_api contains developer API. In order to use this API, you need to link your component against
openvino::runtime::dev. - docs contains developer documentation pages for the component.
- include contains public API. Detailed information about provided API can be found here.
- reference is a library which provides reference implementations for all supported operations. Operations with evaluate method use these implementations inside.
- shape_inference library contains implementation of shape inference for OpenVINO operations.
- src folder contains sources of the core component.
- tests contains tests for OpenVINO Core components. More information about OpenVINO Core tests can be found here.
Tutorials
- How to add new operations.
- How to add OpenVINO Extension. This document is based on the template_extension.
- How to debug the component.