openvino/src/bindings/python/src/pyopenvino
Ivan Tikhonov 55c11c6324
Port SDPA to PagedAttention transformation (#24336)
### Details:
Ported SDPA to PagedAttention transformation from python to C++ code.

the related PRs:
https://github.com/openvinotoolkit/openvino/pull/24127
https://github.com/openvinotoolkit/openvino/pull/24177

Tested model scope:
- [x]          "hf-internal-testing/tiny-random-BloomForCausalLM",
- [x]          "hf-internal-testing/tiny-random-FalconForCausalLM",
- [x]          "hf-internal-testing/tiny-random-Starcoder2ForCausalLM",
- [x]          "hf-internal-testing/tiny-random-GPTJForCausalLM",
- [x]          "hf-internal-testing/tiny-random-StableLmForCausalLM",
- [x]          "hf-internal-testing/tiny-random-LlamaForCausalLM",
- [x]          "hf-internal-testing/tiny-random-MistralForCausalLM",
- [x]          "hf-internal-testing/tiny-random-OPTForCausalLM",
- [x]          "hf-internal-testing/tiny-random-PhiForCausalLM",
- [x]          "hf-internal-testing/tiny-random-StableLmForCausalLM",
- [x]          "facebook/opt-125m",
- [x]          "llama2",
- [x]          "bigcode/starcoder2-7b"
- [ ] "mosaicml/mpt-7b-chat" (FAILED both py/c++) - acceptable for this
PR
Issue: RuntimeError: Check '(axis_range_min <= axis) && (axis <=
axis_range_max)' failed at src/core/src/validation_util.cpp:386:
Concat Parameter axis 2 out of the tensor rank range [0, 0].
- [x] _means, that the response to the dedicated prompt is the same for
the py and c++ transformations._




### Tickets:
 - *CVS-138664*

---------

Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
Co-authored-by: Andrii Staikov <andrii.staikov@intel.com>
2024-05-02 16:02:04 +04:00
..
core Port SDPA to PagedAttention transformation (#24336) 2024-05-02 16:02:04 +04:00
frontend [PyOV] Fix deprecation warnings for gcc (#24239) 2024-04-29 12:26:41 +00:00
graph Port SDPA to PagedAttention transformation (#24336) 2024-05-02 16:02:04 +04:00
test_utils Update year in src/bindings/python folder (#22186) 2024-01-16 18:58:06 +01:00
utils [intel-npu] Adding property to query PCI information of npu device (#24111) 2024-05-01 11:16:50 +00:00
CMakeLists.txt [PyOV] Fix deprecation warnings for gcc (#24239) 2024-04-29 12:26:41 +00:00
pyopenvino.cpp PagedAttention experimental operation (#23837) 2024-04-04 07:54:08 +00:00