04a7ecfadf
### Details: - *Merge first token and second token inference into one parallel loop* - *~~Additional optimization: pre-transpose k-cache, pre-pack v-cache if needed~~* - *Additional optimization for first token: save q * k' upper triangle matrix computation and (q * k') * v lower triangle matrix computation* - *C++ pipeline can enable it: https://github.com/ilya-lavrenov/openvino.genai/pull/9* - *TODO(in another PR):* - alibi support - performance tuning - testcase ### Tickets: - *[138673](https://jira.devtools.intel.com/browse/CVS-138673)* |
||
|---|---|---|
| .. | ||
| cross_compiled_disp_gen.cmake | ||
| cross_compiled_disp_gen_options.in | ||
| cross_compiled_func.cmake | ||
| find_commands.cmake | ||
| native_compile.cmake | ||
| python_helpers.cmake | ||