Creating `Limitation of the bf16 inference precision` section to
describe an issue of higher accuracy loss than the pre-determined
threshold of 0.5% and present possible solutions.
This PR addresses JIRA ticket: 138832.
### Details:
- Fixed bug with inlined inputs for fx decoder which were being added as
parameters and are not needed.
### Tickets:
- (https://jira.devtools.intel.com/browse/CVS-116702)
---------
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* Use block read in 4byte aligned left-over case. Static was already
used (https://github.com/openvinotoolkit/openvino/pull/23400) This PR
will apply in dynamic.
### Tickets:
- *141032*
---------
Signed-off-by: hyunback <hyunback.kim@intel.com>
### Details:
- FC with symmetrically quantized/compressed weight may have i8 (instead
of u8) as weight data type (it saves the zero-point subtraction cost),
this change added support to such weight dt.
- oneDNN fork PR: https://github.com/openvinotoolkit/oneDNN/pull/249
### Tickets:
- *CVS-131042*
### Details:
- Fix Electron compatibility on Windows by specifying flag:
`CMAKE_SHARED_LINKER_FLAGS="/DELAYLOAD:NODE.EXE"`
### Tickets:
- 141536
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Updates the requirements on [h5py](https://github.com/h5py/h5py) to
permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2c80981022"><code>2c80981</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2405">#2405</a> from
takluyver/rever-3.11</li>
<li><a
href="76afe10048"><code>76afe10</code></a>
Add HDF5 1.14.4 fixes to release notes</li>
<li><a
href="009047936d"><code>0090479</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2406">#2406</a> from
ajelenak/libhdf5-1.14.4</li>
<li><a
href="0f071a4d64"><code>0f071a4</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2407">#2407</a> from
h5py/dependabot/github_actions/actions-45223d3826</li>
<li><a
href="944a4a6967"><code>944a4a6</code></a>
Bump peaceiris/actions-gh-pages from 3 to 4 in the actions group</li>
<li><a
href="8f513ced5e"><code>8f513ce</code></a>
Clean up release notes</li>
<li><a
href="62bd813202"><code>62bd813</code></a>
Rename new whatsnew file, add to index</li>
<li><a
href="9c305e23ce"><code>9c305e2</code></a>
Updated CHANGELOG for 3.11.0</li>
<li><a
href="9e107c4a78"><code>9e107c4</code></a>
bumped version to 3.11.0</li>
<li><a
href="71af64dec8"><code>71af64d</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2360">#2360</a> from
loichuder/visit-links</li>
<li>Additional commits viewable in <a
href="https://github.com/h5py/h5py/compare/3.1.0...3.11.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Extended model support in PagedAttention transformation:
- SDPA with `scale` input
- Case when KV inputs into SDPA are implicitly broadcasted by 2nd
dimension.
- More accurate application of PrevSequenceLengthPattern that now can
track batch dimension as well.
Enables tiiuae/falcon-7b and BAAI/AquilaChat2-7B.
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
### Details:
- Dev documentation for the existing
[internal::RMS](70142121c1/src/common/transformations/include/ov_ops/rms.hpp)
op
(specified as is without any additional features, to be developed and
updated if needed)
- RMS was created initially as a [custom gpu
RMS](9fadb5ac64/src/plugins/intel_gpu/include/intel_gpu/op/rms.hpp)
operation, and it's going to be moved to op::internal::RMS to be
available for common transformations
- Similar approach as it was done for AUGRUCell/AUGRUSeqence, so the
place for such documents has been already agreed and it's not a part of
the official web docs tree
### Tickets:
- 134914, dicsussion 129027
### Details:
- Fix the issue where `conv_params.quantization` was not correctly
identified as a quantization case when using U8 weights data (which is
allowed by ConvertConvolutionToInternal transformation pass)
### Tickets:
- 139740
Sometimes the approximate attribute is not specified in the .tflite
file, we need to check its availability before retriving it, otherwise
we might hit the 'Check 'opts != nullptr' failed' failure with calling
get_attribute() directly.
Signed-off-by: Keyon Jie <yang.jie@linux.intel.com>
### Details:
This patch contains all the necessary changes to support internal states
in the composite nodes (i.e. If, Loop, TensorIterator operations). Main
changes:
1. Move the internal states registry to the graph context, so that all
the nested internal states become accessible on the infer request level.
2. Move the state assing/commit actions to the node execute level, so
that Loop and TensorIterator ops are automatically supported.
### Tickets:
- CVS-134470
### Details:
- *Create PoC implementation for EmbeddingBag Offsets and Packed that
introduce mode 'mean'*
- *Add Core, Reference and CPU implementation for both ops*
Specification will be added in separate PR
### Tickets:
- *138229*
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
### Details:
- *Extract BrgemmCopyB from M blocking if there are no KN blocking
loops*
- *Tests will be added after #24152 merge*
### Tickets:
- *CVS-140751*
### Details:
- This PR extends existing weights type conversion + repack optimization
for bf16 precision. This change is needed to minimize memory footprint
on model compilation stage for models with BF16 compressed weights.
Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/psf/requests/releases">requests's
releases</a>.</em></p>
<blockquote>
<h2>v2.32.0</h2>
<h2>2.32.0 (2024-05-20)</h2>
<h2>🐍 PYCON US 2024 EDITION 🐍</h2>
<p><strong>Security</strong></p>
<ul>
<li>Fixed an issue where setting <code>verify=False</code> on the first
request from a
Session will cause subsequent requests to the <em>same origin</em> to
also ignore
cert verification, regardless of the value of <code>verify</code>.
(<a
href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li>
</ul>
<p><strong>Improvements</strong></p>
<ul>
<li><code>verify=True</code> now reuses a global SSLContext which should
improve
request time variance between first and subsequent requests. It should
also minimize certificate load time on Windows systems when using a
Python
version built with OpenSSL 3.x. (<a
href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li>
<li>Requests now supports optional use of character detection
(<code>chardet</code> or <code>charset_normalizer</code>) when
repackaged or vendored.
This enables <code>pip</code> and other projects to minimize their
vendoring
surface area. The <code>Response.text()</code> and
<code>apparent_encoding</code> APIs
will default to <code>utf-8</code> if neither library is present. (<a
href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li>
</ul>
<p><strong>Bugfixes</strong></p>
<ul>
<li>Fixed bug in length detection where emoji length was incorrectly
calculated in the request content-length. (<a
href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li>
<li>Fixed deserialization bug in JSONDecodeError. (<a
href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li>
<li>Fixed bug where an extra leading <code>/</code> (path separator)
could lead
urllib3 to unnecessarily reparse the request URI. (<a
href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li>
</ul>
<p><strong>Deprecations</strong></p>
<ul>
<li>Requests has officially added support for CPython 3.12 (<a
href="https://redirect.github.com/psf/requests/issues/6503">#6503</a>)</li>
<li>Requests has officially added support for PyPy 3.9 and 3.10 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
<li>Requests has officially dropped support for CPython 3.7 (<a
href="https://redirect.github.com/psf/requests/issues/6642">#6642</a>)</li>
<li>Requests has officially dropped support for PyPy 3.7 and 3.8 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
</ul>
<p><strong>Documentation</strong></p>
<ul>
<li>Various typo fixes and doc improvements.</li>
</ul>
<p><strong>Packaging</strong></p>
<ul>
<li>Requests has started adopting some modern packaging practices.
The source files for the projects (formerly <code>requests</code>) is
now located
in <code>src/requests</code> in the Requests sdist. (<a
href="https://redirect.github.com/psf/requests/issues/6506">#6506</a>)</li>
<li>Starting in Requests 2.33.0, Requests will migrate to a PEP 517
build system
using <code>hatchling</code>. This should not impact the average user,
but extremely old
versions of packaging utilities may have issues with the new packaging
format.</li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matthewarmand"><code>@matthewarmand</code></a>
made their first contribution in <a
href="https://redirect.github.com/psf/requests/pull/6258">psf/requests#6258</a></li>
<li><a href="https://github.com/cpzt"><code>@cpzt</code></a> made their
first contribution in <a
href="https://redirect.github.com/psf/requests/pull/6456">psf/requests#6456</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's
changelog</a>.</em></p>
<blockquote>
<h2>2.32.0 (2024-05-20)</h2>
<p><strong>Security</strong></p>
<ul>
<li>Fixed an issue where setting <code>verify=False</code> on the first
request from a
Session will cause subsequent requests to the <em>same origin</em> to
also ignore
cert verification, regardless of the value of <code>verify</code>.
(<a
href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li>
</ul>
<p><strong>Improvements</strong></p>
<ul>
<li><code>verify=True</code> now reuses a global SSLContext which should
improve
request time variance between first and subsequent requests. It should
also minimize certificate load time on Windows systems when using a
Python
version built with OpenSSL 3.x. (<a
href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li>
<li>Requests now supports optional use of character detection
(<code>chardet</code> or <code>charset_normalizer</code>) when
repackaged or vendored.
This enables <code>pip</code> and other projects to minimize their
vendoring
surface area. The <code>Response.text()</code> and
<code>apparent_encoding</code> APIs
will default to <code>utf-8</code> if neither library is present. (<a
href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li>
</ul>
<p><strong>Bugfixes</strong></p>
<ul>
<li>Fixed bug in length detection where emoji length was incorrectly
calculated in the request content-length. (<a
href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li>
<li>Fixed deserialization bug in JSONDecodeError. (<a
href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li>
<li>Fixed bug where an extra leading <code>/</code> (path separator)
could lead
urllib3 to unnecessarily reparse the request URI. (<a
href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li>
</ul>
<p><strong>Deprecations</strong></p>
<ul>
<li>Requests has officially added support for CPython 3.12 (<a
href="https://redirect.github.com/psf/requests/issues/6503">#6503</a>)</li>
<li>Requests has officially added support for PyPy 3.9 and 3.10 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
<li>Requests has officially dropped support for CPython 3.7 (<a
href="https://redirect.github.com/psf/requests/issues/6642">#6642</a>)</li>
<li>Requests has officially dropped support for PyPy 3.7 and 3.8 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
</ul>
<p><strong>Documentation</strong></p>
<ul>
<li>Various typo fixes and doc improvements.</li>
</ul>
<p><strong>Packaging</strong></p>
<ul>
<li>Requests has started adopting some modern packaging practices.
The source files for the projects (formerly <code>requests</code>) is
now located
in <code>src/requests</code> in the Requests sdist. (<a
href="https://redirect.github.com/psf/requests/issues/6506">#6506</a>)</li>
<li>Starting in Requests 2.33.0, Requests will migrate to a PEP 517
build system
using <code>hatchling</code>. This should not impact the average user,
but extremely old
versions of packaging utilities may have issues with the new packaging
format.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="d6ebc4a2f1"><code>d6ebc4a</code></a>
v2.32.0</li>
<li><a
href="9a40d12778"><code>9a40d12</code></a>
Avoid reloading root certificates to improve concurrent performance (<a
href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li>
<li><a
href="0c030f78d2"><code>0c030f7</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>
from nateprewitt/no_char_detection</li>
<li><a
href="555b870eb1"><code>555b870</code></a>
Allow character detection dependencies to be optional in post-packaging
steps</li>
<li><a
href="d6dded3f00"><code>d6dded3</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6700">#6700</a>
from franekmagiera/update-redirect-to-invalid-uri-test</li>
<li><a
href="bf24b7d8d1"><code>bf24b7d</code></a>
Use an invalid URI that will not cause httpbin to throw 500</li>
<li><a
href="2d5f54779a"><code>2d5f547</code></a>
Pin 3.8 and 3.9 runners back to macos-13 (<a
href="https://redirect.github.com/psf/requests/issues/6688">#6688</a>)</li>
<li><a
href="f1bb07d39b"><code>f1bb07d</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6687">#6687</a>
from psf/dependabot/github_actions/github/codeql-act...</li>
<li><a
href="60047ade64"><code>60047ad</code></a>
Bump github/codeql-action from 3.24.0 to 3.25.0</li>
<li><a
href="31ebb8102c"><code>31ebb81</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6682">#6682</a>
from frenzymadness/pytest8</li>
<li>Additional commits viewable in <a
href="https://github.com/psf/requests/compare/v2.31.0...v2.32.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/openvinotoolkit/openvino/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Details:
- *This PR https://github.com/openvinotoolkit/openvino/pull/23384
triggers the existing bugs in `GatherCompressed` and `Fullyconnected`*
- *After converting all possible constants to FP16, `GatherCompresssed`
needs to output `FP16` when scale is changed to FP16*
- *After converting all possible constants to FP16, `FullyConnected`
needs to explicitly refuse `FP16` bias since `oneDNN` doesn't support
`FP16` bias.*
### Tickets:
- *CVS-140154*
### Details:
- *set correct numa id for different tbb version*
- *numa id has different mapping method in TBBBind since oneTBB
2021.4.0*
### Tickets:
- *CVS-141067*
### Details:
- enable AUTO to retrieve the hash id for cached model
- load cached model to GPU only with AUTO if cached blob exists and not
set `ov::intel_auto::enable_startup_fallback` explicitly to AUTO
### Tickets:
- CVS-140059
### Details:
- This PR updates the `kernels_cache` to build the selected kernels in
separate batches.
- This is a temporary WA to resolve performance degradation when some
kernels are built with other kernels in the same batch
- Currently, the selected kernel includes `gemm_tiled_opt`.
- The impacted scenario : Qwen INT4 first token latency for > 1K input
in MTL
### Tickets:
- GSD-8910
### Details:
- In set_state of gpu plugin, the src Tensor can have padding, which has
not handled by set_state
- Fixed set_state to handle src Tensor's padding properly
### Tickets:
- 140418
### Details:
- *After support for i64 was added in frontend `aten::adaptive_avg_pool`
started to fail if shape input is list.*
### Tickets:
- *CVS-141335*
**Details:** Remove test-generator dependency. It is removed since this
module seems to be no longer developed and maintained. So it is better
to remove this dependency. `pytest` can substitute `test-generator`.
**Ticket:** TBD
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
### Details:
PyTorch semantic of Expand operation requires Abs operation for
translation which is usually a no-op.
1. partial value and symbol propagation for Abs operation
2. optimize out the Abs with nonnegative input
3. for this case Abs(Concat([Gather(ShapeOf), -1, 1, -1])) pulls Abs and
tries to const-fold it. Resulting in Concat([Abs(Gather(ShapeOf)), 1, 1,
1]). Applies (2) to the graph and gets Concat([Gather(ShapeOf), 1, 1,
1]) because ShapeOf's lower bound is 0.
### Tickets:
- *CVS-141274*
### Details:
As ov RMSNorm has been not released yet as a public opset op, it was
proposed to keep it as internal.
RMSNorm is not currently a part of any frontend framework and the
requirements for RMSNorm may change.
There is a corresponding
[`internal::RMS`](70142121c1/src/common/transformations/include/ov_ops/rms.hpp)
op availavble for common transformations (moved from custom GPU).
The goal is to unify this work and tests into single RMS implementation.
---------------------------
The reference implementation is going to be reused for the
internal::RMS:
- https://github.com/openvinotoolkit/openvino/pull/24556
### Tickets:
- Related to: 129027
### Details:
- *Change the context_lens to past_lens*
- *past lens corresponds to first token within subgroup of scheduled
tokens but not includes itself*
- *...*
### Tickets:
- *ticket-id*
Sergey Shlyapnikov