### Details:
- Added dev pass to log operations statistics when VERBOSE=1
- Added debug knob to collect network::execute() time to measure time
required to schedule all primitives
- Updated common results comparison routine to print coordinate and to
show first mismatched element info in release build
### Details:
- Currently, the unit test for HSVToRGB is creating a tensor of all ones
in the Grayscale special case as mentioned in
[#24875](https://github.com/openvinotoolkit/openvino/pull/24875). Since
the images are expected to be in HSV and not RGB format, this will
create images that are green, and not gray. Grayscale in HSV can be
achieved by creating a tensor with HSV entries of [0, 0, .5].
### Tickets:
- N/A
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
### Details:
- Relax SDPA head size limitations for LLMs from 128 only to a range of
64 to 256
- Fix accuracy issue in SDPA first token processing for
`TARGET_SEQ_LEN_BLOCK_SIZE % SUBGROUPS_PER_WG != 0` case
### Tickets:
- *ticket-id*
### Details:
- New instance of Core creating on execution of readModel method. This
PR is fixing this behavior.
### Tickets:
- 138160
---------
Co-authored-by: Alicja Miloszewska <alicja.miloszewska@intel.com>
### Details:
- *When scripting upsample creates `prim::If` depending from input
shape, to determine which version of upsample to use: 1d, 2d or 3d. This
is not supported by openvino, but this PR fixes segfault that happen in
this case.*
### Tickets:
- *#24270*
Hello maintainers,
I've implemented the IsInf JIT emitter for the ARM64 SIMD platform, as
outlined in the OpenVINO CPU plugin JIT emitters documentation.
This PR addresses
[#24419](https://github.com/openvinotoolkit/openvino/issues/24419).
Please let me know if any further adjustments are needed.
Thank you!
We only check whether shape of subgraph is changed or not in dynamic
node. But static node also need to be skipped. The node after shape_of
can be static.
### Tickets:
- *142892*
---------
Signed-off-by: hyunback <hyunback.kim@intel.com>
**Details:** Implement LSTMCell fusion for Keras LSTM layers. Such
sub-graph with split weights appears for bi-directional LSTM layer.
For custom model, we managed to fuse 6 LSTMCell sub-graphs and obtained
19% performance gain on CPU.
**Ticket:** 142924
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
### Details:
- *Renamed `Buffer` attribute `id` to `regGroup`*
- *Replaced `allocation_shape` of `Buffer` with `allocation_size`*
- *Added the separate pass `ComputeBufferAllocationSize` to calculate
allocation size for Buffers. By default `allocation_size` is undefined
after `InsertBuffers`*
- *Added new attribute `cluster_id` to `Buffer`*
- *Added util expressions to LinearIR: `m_buffer_expressions` for the
quick access in configurator*
- *Added functionality to prohibit passes to update `work_amount` in
`LoopInfo`: it's needed for inner splitted loops (since their
`work_amount` is `increment` of outer splitted loop*
- *Split `SolveMemoryBuffers` logic into dynamic and static buffers: now
for all defined buffers the allocation size will be efficiently
calculated and update `offsets` that can be compiled in jit code.
Additional allocation size for dynamic buffers will be calculated in
`RuntimeConfigurator`.*
### Tickets:
- *123274*
### Prerequisites:
- https://github.com/openvinotoolkit/openvino/pull/24525
### TODO:
- [x] Add CPU support of dynamic offsets in memory access emitters
### Details:
- Using #24033, implemented and registered loader for HSVToRGB using
already existing logic.
- Created unit test test_tf_HSVToRGB.py
### Tickets:
- #24791
Currently, my pytest is saying that for the unit test and for
test_tf_AdjustHue.py and test_tf_AdjustSaturation.py the loader is not
found.
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
There is so many developers are try to install openvino in opencloudos
and tencentos.
This pr add support for opencloudos / tencentos.
OpenCloudOS is an open source operating system community jointly
initiated by Tencent and its partners.
for more news:
https://www.opencloudos.org/https://docs.opencloudos.org/en/
Signed-off-by: abushwang <abushwangs@gmail.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Bumps
[reviewdog/action-shellcheck](https://github.com/reviewdog/action-shellcheck)
from 1.20.0 to 1.21.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/reviewdog/action-shellcheck/releases">reviewdog/action-shellcheck's
releases</a>.</em></p>
<blockquote>
<h2>Release v1.21.0</h2>
<h2>What's Changed</h2>
<ul>
<li>chore(deps): update reviewdog/reviewdog to 0.17.5 by <a
href="https://github.com/github-actions"><code>@github-actions</code></a>
in <a
href="https://redirect.github.com/reviewdog/action-shellcheck/pull/59">reviewdog/action-shellcheck#59</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/reviewdog/action-shellcheck/compare/v1.20.0...v1.21.0">https://github.com/reviewdog/action-shellcheck/compare/v1.20.0...v1.21.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="6e3a862f23"><code>6e3a862</code></a>
Merge pull request <a
href="https://redirect.github.com/reviewdog/action-shellcheck/issues/59">#59</a>
from reviewdog/depup/reviewdog/reviewdog</li>
<li><a
href="c82af54388"><code>c82af54</code></a>
chore(deps): update reviewdog/reviewdog to 0.17.5</li>
<li>See full diff in <a
href="72365a51bf...6e3a862f23">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps
[reviewdog/action-suggester](https://github.com/reviewdog/action-suggester)
from 1.12.0 to 1.13.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/reviewdog/action-suggester/releases">reviewdog/action-suggester's
releases</a>.</em></p>
<blockquote>
<h2>Release v1.13.0</h2>
<h2>What's Changed</h2>
<ul>
<li>chore(deps): update reviewdog/reviewdog to 0.17.5 by <a
href="https://github.com/github-actions"><code>@github-actions</code></a>
in <a
href="https://redirect.github.com/reviewdog/action-suggester/pull/55">reviewdog/action-suggester#55</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/reviewdog/action-suggester/compare/v1.12.0...v1.13.0">https://github.com/reviewdog/action-suggester/compare/v1.12.0...v1.13.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9e1cd88b79"><code>9e1cd88</code></a>
Merge pull request <a
href="https://redirect.github.com/reviewdog/action-suggester/issues/55">#55</a>
from reviewdog/depup/reviewdog/reviewdog</li>
<li><a
href="2ebb5a11b1"><code>2ebb5a1</code></a>
chore(deps): update reviewdog/reviewdog to 0.17.5</li>
<li>See full diff in <a
href="185c9c06d0...9e1cd88b79">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Details:
The transformation works in 2 cases:
* 1. all in/out dims are static, or we can match them via the symbols.
2. only one out dim doesn't have the corresponding input static dim,
and we can't match it using symbols. Besides that the output shape must
not contain zero dims,
because then value -1 in 2nd input to Reshape op can't guarantee an
unambiguous determination of the remaining dim value.
### Tickets:
- *CVS-143026*
### Details:
- Add new element into OpenVINO `f4e2m1`
- Add new type `float4_e2m1`
### Tickets:
- [CVS-141565](https://jira.devtools.intel.com/browse/CVS-141565)
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
## Details:
in throughput oriented use case like vLLM, the computation bound of
Fullyconnect layers has a chance to become similar or even bigger than
memory-bound when batch-size is big, in such case, some optimization
opportunities was exploited in this PR:
### fusing multiple FC to get more computations per memory-read
- when split the problem `C=matmul(A, B)` among multi-cores, each core
has sub-problem of `subC=matmul(A, subB)`, with shapes:
- A `[BM, K]`
- subB `[K, BN]`
- subC `[BM, BN]`
if we ignore subC matrix (since it's being loaded/stored only once
during `brgemm` procedure), the ratio of computations per memory-reads
is ` BM*K*BN / (BM*K + BN*K) = 1/(1/BM + 1/BN)`, and to get more
computations per memory-read, we prefer to increase this ratio, so we'd
like BN to be bigger (that's why we merge gate_FC with up_FC, and q_proj
with k_proj and v_proj).
in `down_proj`, there is no way to increase output channels (BN), we
split along K dimension once to reduce the number of splits of N, to
keep BN big too.
### prefetch sub-weight matrix to hide memory latency by computation
for execution of sub-problem `subC=matmul(A, subB)` on single core, we
further decompose it into sub-problems as following, to mix
SW-prefetching with AMX computations:
```python
for k0 in range(0, K, K_step):
k1 = k0 + K_step
blkB = subB[k0:k1, :]
blkA = A[:, k0:k1]
next_blkB = subB[k1:k1+K_step, :]
# blkB's shape is [K_step, BN], which is L2 cache-able
# inside following AMX based matmul jit kernel
# we evenly scatter SW-prefetching instructions for next_blkB
subC+=matmul_kernel(blkA, blkB, sw_prefetch = next_blkB) .
```
### flexible post-ops
jit-based post-Ops allows us to:
- interleaving fusion of weights of gate & up along N dimension in unit
of 16 fp32 elements, so post-op jit-kernel can combine them using
activation(like SiLU) while both results are hot in cache.
- QKV fusion requires final outputs was stored into 3 independent output
memories, special post-op jit kernel can do that easily.
### Tickets:
- *ticket-id*
---------
Co-authored-by: Luo Cheng <cheng.luo@intel.com>
### Details:
- Initial problem: `test_custom_op` hanged on destruction because it was
waiting for a thread which tried to acquire GIL.
- The second problem is that pybind11 doesn't allow to work with GIL
besides of current scope and it's impossible to release GIL for
destructors. https://github.com/pybind/pybind11/issues/1446
- Current solution allows to release GIL for InferRequest and all called
by chain destructors.
### Tickets:
- CVS-141744
[TRANSFORMATIONS] Add a check to SDPAToPagedAttention to fail fast
Add a check to SDPAToPagedAttention to fail fast if no
ScaledDotProductAttention operation present in a graph hence no
transformation will be executed.
### Tickets:
- CVS-143067
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
---------
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Remove the limitation that oneDNN convolution doesn't support dynamic
input and output.
With this limitation, clDNN convolution will run forever in dpas
platform.
Because oneDNN Opt convolutions require blocked-format and current shape
agnostic convolution doesn't support block-format, convolution do
nothing in compile_graph stage and actual kernel compilation will happen
at first inference.
### Tickets:
- *141040*
Signed-off-by: hyunback <hyunback.kim@intel.com>
### Details:
- *Fixed Invalid Read in FuseLoops pass. Before we save loop ports by
reference from `current_loop_info`. However, this `current_loop_Info`
might be changed (another shared pointer) and the previous saved loop
ports by reference are invalid (the original `current_loop_info` is
expired)*
- *`Valgrind` doesn't throw exception anymore with this fix*
### Tickets:
- *N/A*
Taylor Yeonbok Lee