**Details:** Previously, we were able to convert models requiring
openvino-tokenizers extensions (like with string operations, ragged
tensors) only in Python by `convert_model`. Now we add option to convert
using `ovc` tool. The tool internally imports extensions if they are
installed.
**Ticket:** TBD
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Enabled model: bigcode/starcoderbase-3b, bigcode/gpt_bigcode-santacoder.
There is a moderate refactoring in the code: move common code for
ReadValue-based pattern matching K, V and merged KV state management
into a single piece of code.
Extended UBR pattern for case when MQA doesn't have Reshape in the end
because KV cache is 3D instead of 4D and corresponding changes in the
detection of the number of KV heads.
In sync with https://github.com/ilya-lavrenov/openvino.genai/pull/20
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Updates the requirements on [h5py](https://github.com/h5py/h5py) to
permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2c80981022"><code>2c80981</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2405">#2405</a> from
takluyver/rever-3.11</li>
<li><a
href="76afe10048"><code>76afe10</code></a>
Add HDF5 1.14.4 fixes to release notes</li>
<li><a
href="009047936d"><code>0090479</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2406">#2406</a> from
ajelenak/libhdf5-1.14.4</li>
<li><a
href="0f071a4d64"><code>0f071a4</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2407">#2407</a> from
h5py/dependabot/github_actions/actions-45223d3826</li>
<li><a
href="944a4a6967"><code>944a4a6</code></a>
Bump peaceiris/actions-gh-pages from 3 to 4 in the actions group</li>
<li><a
href="8f513ced5e"><code>8f513ce</code></a>
Clean up release notes</li>
<li><a
href="62bd813202"><code>62bd813</code></a>
Rename new whatsnew file, add to index</li>
<li><a
href="9c305e23ce"><code>9c305e2</code></a>
Updated CHANGELOG for 3.11.0</li>
<li><a
href="9e107c4a78"><code>9e107c4</code></a>
bumped version to 3.11.0</li>
<li><a
href="71af64dec8"><code>71af64d</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2360">#2360</a> from
loichuder/visit-links</li>
<li>Additional commits viewable in <a
href="https://github.com/h5py/h5py/compare/3.1.0...3.11.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
### Details:
- To reduce Infer overhead
- The new flow is:
1) Define sync points from all the nodes perspective.
2) Use sync points in scope of Allocate
3) Define executable sync points from overall sync points but
considering only executable nodes.
4) Keep executable sync points ordered to avoid additional logic in
scope of Infer call.
### Tickets:
- *ticket-id*
### Details:
- *Extended LIR interface to make LIR creation from scratch possible*
- *Introduced `LoweredPassTestsF` base class for LIR comparison*
- *Added some helpers for easier LIR creation from scratch (not from
`ov::Model`)*
- *Introduced `LIRComparator` class. For LIR related things comparison,
a custom comparators were written. For `ov::Node` comparison, the
existing comparator from `GraphComparator` class is used.*
- *Added a test case for `InsertLoadStore` pass: as an example of a
simple LIR comparison (only expressions + `ov::Node` comparison is
necessary to fully cover the pass code)*. The pass was not fully
covered.
- *Added test cases for `BrgemmBlocking` cpu lowered pass: as an example
of complicated LIR comparison, where `LoopManager` and descriptors
comparison are also necessary. This pass was fully covered.*
### Tickets:
- *CVS-139926*
Details:
Due to the issue in the e2e test in
https://github.com/openvinotoolkit/openvino/pull/24597 we decided to
exclude Concat to Tile conversion from the transformation. It will be
covered in another ticket.
- Add a ConcatToBroadcast transformation to replace Concat having inputs
from the same output with a Broadcast
- Add a test for the ConcatToBroadcast transformation
Significantly reduce model compile time and performance time.
Tickets:
[CVS-138829](https://jira.devtools.intel.com/browse/CVS-138829),
[CVS-138077](https://jira.devtools.intel.com/browse/CVS-138077)
---------
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
Co-authored-by: Andrii Staikov <andrii.staikov@intel.com>
Convolution which has u8 AvgPool layer as input0 is handled as quantized
in transformation and it has zp constants in transformed graph. But gpu
plugin has changed AvgPool output data type to fp32 due to accuracy
issue. The change causes convolution ocl impl doesn't recognize
quantized convolution.
### Details:
- Add Workaround in convolution ocl impl for fp32 output hack of avgpool
when u8/i8 input
### Tickets:
- 141396
This reverts commit 03aad66230.
### Details:
Revert the commit because there are some models with stateful OPS
(assign/readvalue) not in same attention node, hetero pipeline
parallelism does not work.
### Tickets:
- *CVS-141771*
### Details:
- *I lost code `t_stream_count_map[(void*)this] = item.first;` in
#19832*
- *the thread safe issue happen in below workflow*
- create thread A
call CustomThreadLocal:local() in thread A -> create stream A (the count
of stream A is 2)
destory thread A (the count of stream A is 1)
create thread B (same thread id with thread A)
call CustomThreadLocal:local() in thread B -> use stream A(the count of
stream A is 1, so it's broken)
- *add testcase, also fix
https://github.com/openvinotoolkit/openvino/pull/19986/files#r1332774754*
### Tickets:
- Closes https://github.com/openvinotoolkit/openvino/issues/24509
---------
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
### Details:
- update logic of only loading cached model to GPU with AUTO if GPU
cached blob exists and `ov::intel_auto::enable_startup_fallback` is
enable
### Tickets:
- CVS-138574
### Details:
- Apply dynamic padding for onednn gemm
- Update memory descriptor to handle the padded dims and strides
### Tickets:
- 140516
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
### Details:
- Removed 'actual comparison size' parameter from base template test
class `CommonReferenceTest`.
- Moved partial comparison logic to
`ReferenceExperimentalPGGLayerTest::Validate()` - it's needed only in
this test.
### Tickets:
- CVS-137159
### Details:
- Reference implementation and ref tests for
[internal::RMS](70142121c1/src/common/transformations/include/ov_ops/rms.hpp)
(moved from gpu custom ops) available for common transformations
- Enable Accuracy tests for RMSFusion transformation
The changes moving RMS from gpu to internal are from PR:
- #24539 (to be merged first)
Internal RMS op docs:
- https://github.com/openvinotoolkit/openvino/pull/24564
### Tickets:
- Related to 136262
---------
Co-authored-by: michal-miotk <michal.miotk@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
### Details:
QAT documentation for NNCF Torch and NNCF Tensorflow is spitted on two
pages
* quantization-aware-training-pytorch.rst is QAT by `nncf.quantize`
method documentation
* quantization-aware-training-tensorflow.rst file is a copy of original
quantization-aware-training.rst without the PyTorch part
Link to generated doc:
http://openvino-doc.iotg.sclab.intel.com/nncf-docs/openvino-workflow/model-optimization-guide/compressing-models-during-training.html
### Tickets:
- 129586
---------
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
dependabot[bot]