natas_hw
/
openvino
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
openvino/src/plugins
History
Vladimir Paramuzov d247233fb1
[GPU] Added indirect KV Cache and Gemm (#22726) 
### Details:
- Allows to avoid kv cache rearrange in cases when beam search is used.
Instead of that beam table is maintained to track the history of
required cache modifications which is passed to subsequent gemm
primitive
- Beam table is a part of the model state, so GPU `VariableState` was
extended to multi-tensor case
- Improves 2nd+ token latency and reduces memory consumption with beam
size > 1

### Tickets:
 - *124119*

---------

Co-authored-by: Sergey Shlyapnikov <sergey.shlyapnikov@intel.com>
 		2024-02-15 07:30:36 +00:00
..
auto Rename IE to OV (#22733) 2024-02-09 13:36:41 +00:00
auto_batch Remove `ie_*` includes (#22657) 2024-02-05 20:12:50 +00:00
hetero Remove API 1.0 macros (#22330) 2024-02-09 07:53:36 +00:00
intel_cpu [CPU] Dynamic quantization for compressed FullyConnected (#22738) 2024-02-14 17:31:59 +00:00
intel_gpu [GPU] Added indirect KV Cache and Gemm (#22726) 2024-02-15 07:30:36 +00:00
proxy Remove API 1.0 macros (#22330) 2024-02-09 07:53:36 +00:00
template [Ref][Core][CPU][Opset14] Inverse Operation (#22649) 2024-02-14 11:58:38 +00:00
CMakeLists.txt Mark some legacy API as deprecated (#17469) 2023-05-17 11:31:01 +04:00
README.md [DOCS] Move of developer documentation from wiki to md documents - master (#15372) 2023-02-16 11:03:11 +04:00

README.md

OpenVINO Plugins

OpenVINO Plugins provide support for hardware devices.

The list of supported plugins:

See also