### Details:
- Added dev pass to log operations statistics when VERBOSE=1
- Added debug knob to collect network::execute() time to measure time
required to schedule all primitives
- Updated common results comparison routine to print coordinate and to
show first mismatched element info in release build
### Details:
- Currently, the unit test for HSVToRGB is creating a tensor of all ones
in the Grayscale special case as mentioned in
[#24875](https://github.com/openvinotoolkit/openvino/pull/24875). Since
the images are expected to be in HSV and not RGB format, this will
create images that are green, and not gray. Grayscale in HSV can be
achieved by creating a tensor with HSV entries of [0, 0, .5].
### Tickets:
- N/A
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
### Details:
- Relax SDPA head size limitations for LLMs from 128 only to a range of
64 to 256
- Fix accuracy issue in SDPA first token processing for
`TARGET_SEQ_LEN_BLOCK_SIZE % SUBGROUPS_PER_WG != 0` case
### Tickets:
- *ticket-id*
### Details:
- New instance of Core creating on execution of readModel method. This
PR is fixing this behavior.
### Tickets:
- 138160
---------
Co-authored-by: Alicja Miloszewska <alicja.miloszewska@intel.com>
### Details:
- *When scripting upsample creates `prim::If` depending from input
shape, to determine which version of upsample to use: 1d, 2d or 3d. This
is not supported by openvino, but this PR fixes segfault that happen in
this case.*
### Tickets:
- *#24270*
Hello maintainers,
I've implemented the IsInf JIT emitter for the ARM64 SIMD platform, as
outlined in the OpenVINO CPU plugin JIT emitters documentation.
This PR addresses
[#24419](https://github.com/openvinotoolkit/openvino/issues/24419).
Please let me know if any further adjustments are needed.
Thank you!
We only check whether shape of subgraph is changed or not in dynamic
node. But static node also need to be skipped. The node after shape_of
can be static.
### Tickets:
- *142892*
---------
Signed-off-by: hyunback <hyunback.kim@intel.com>
**Details:** Implement LSTMCell fusion for Keras LSTM layers. Such
sub-graph with split weights appears for bi-directional LSTM layer.
For custom model, we managed to fuse 6 LSTMCell sub-graphs and obtained
19% performance gain on CPU.
**Ticket:** 142924
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
### Details:
- *Renamed `Buffer` attribute `id` to `regGroup`*
- *Replaced `allocation_shape` of `Buffer` with `allocation_size`*
- *Added the separate pass `ComputeBufferAllocationSize` to calculate
allocation size for Buffers. By default `allocation_size` is undefined
after `InsertBuffers`*
- *Added new attribute `cluster_id` to `Buffer`*
- *Added util expressions to LinearIR: `m_buffer_expressions` for the
quick access in configurator*
- *Added functionality to prohibit passes to update `work_amount` in
`LoopInfo`: it's needed for inner splitted loops (since their
`work_amount` is `increment` of outer splitted loop*
- *Split `SolveMemoryBuffers` logic into dynamic and static buffers: now
for all defined buffers the allocation size will be efficiently
calculated and update `offsets` that can be compiled in jit code.
Additional allocation size for dynamic buffers will be calculated in
`RuntimeConfigurator`.*
### Tickets:
- *123274*
### Prerequisites:
- https://github.com/openvinotoolkit/openvino/pull/24525
### TODO:
- [x] Add CPU support of dynamic offsets in memory access emitters
### Details:
- Using #24033, implemented and registered loader for HSVToRGB using
already existing logic.
- Created unit test test_tf_HSVToRGB.py
### Tickets:
- #24791
Currently, my pytest is saying that for the unit test and for
test_tf_AdjustHue.py and test_tf_AdjustSaturation.py the loader is not
found.
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
There is so many developers are try to install openvino in opencloudos
and tencentos.
This pr add support for opencloudos / tencentos.
OpenCloudOS is an open source operating system community jointly
initiated by Tencent and its partners.
for more news:
https://www.opencloudos.org/https://docs.opencloudos.org/en/
Signed-off-by: abushwang <abushwangs@gmail.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Bumps
[reviewdog/action-shellcheck](https://github.com/reviewdog/action-shellcheck)
from 1.20.0 to 1.21.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/reviewdog/action-shellcheck/releases">reviewdog/action-shellcheck's
releases</a>.</em></p>
<blockquote>
<h2>Release v1.21.0</h2>
<h2>What's Changed</h2>
<ul>
<li>chore(deps): update reviewdog/reviewdog to 0.17.5 by <a
href="https://github.com/github-actions"><code>@github-actions</code></a>
in <a
href="https://redirect.github.com/reviewdog/action-shellcheck/pull/59">reviewdog/action-shellcheck#59</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/reviewdog/action-shellcheck/compare/v1.20.0...v1.21.0">https://github.com/reviewdog/action-shellcheck/compare/v1.20.0...v1.21.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="6e3a862f23"><code>6e3a862</code></a>
Merge pull request <a
href="https://redirect.github.com/reviewdog/action-shellcheck/issues/59">#59</a>
from reviewdog/depup/reviewdog/reviewdog</li>
<li><a
href="c82af54388"><code>c82af54</code></a>
chore(deps): update reviewdog/reviewdog to 0.17.5</li>
<li>See full diff in <a
href="72365a51bf...6e3a862f23">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps
[reviewdog/action-suggester](https://github.com/reviewdog/action-suggester)
from 1.12.0 to 1.13.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/reviewdog/action-suggester/releases">reviewdog/action-suggester's
releases</a>.</em></p>
<blockquote>
<h2>Release v1.13.0</h2>
<h2>What's Changed</h2>
<ul>
<li>chore(deps): update reviewdog/reviewdog to 0.17.5 by <a
href="https://github.com/github-actions"><code>@github-actions</code></a>
in <a
href="https://redirect.github.com/reviewdog/action-suggester/pull/55">reviewdog/action-suggester#55</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/reviewdog/action-suggester/compare/v1.12.0...v1.13.0">https://github.com/reviewdog/action-suggester/compare/v1.12.0...v1.13.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9e1cd88b79"><code>9e1cd88</code></a>
Merge pull request <a
href="https://redirect.github.com/reviewdog/action-suggester/issues/55">#55</a>
from reviewdog/depup/reviewdog/reviewdog</li>
<li><a
href="2ebb5a11b1"><code>2ebb5a1</code></a>
chore(deps): update reviewdog/reviewdog to 0.17.5</li>
<li>See full diff in <a
href="185c9c06d0...9e1cd88b79">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Details:
The transformation works in 2 cases:
* 1. all in/out dims are static, or we can match them via the symbols.
2. only one out dim doesn't have the corresponding input static dim,
and we can't match it using symbols. Besides that the output shape must
not contain zero dims,
because then value -1 in 2nd input to Reshape op can't guarantee an
unambiguous determination of the remaining dim value.
### Tickets:
- *CVS-143026*
### Details:
- Add new element into OpenVINO `f4e2m1`
- Add new type `float4_e2m1`
### Tickets:
- [CVS-141565](https://jira.devtools.intel.com/browse/CVS-141565)
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
## Details:
in throughput oriented use case like vLLM, the computation bound of
Fullyconnect layers has a chance to become similar or even bigger than
memory-bound when batch-size is big, in such case, some optimization
opportunities was exploited in this PR:
### fusing multiple FC to get more computations per memory-read
- when split the problem `C=matmul(A, B)` among multi-cores, each core
has sub-problem of `subC=matmul(A, subB)`, with shapes:
- A `[BM, K]`
- subB `[K, BN]`
- subC `[BM, BN]`
if we ignore subC matrix (since it's being loaded/stored only once
during `brgemm` procedure), the ratio of computations per memory-reads
is ` BM*K*BN / (BM*K + BN*K) = 1/(1/BM + 1/BN)`, and to get more
computations per memory-read, we prefer to increase this ratio, so we'd
like BN to be bigger (that's why we merge gate_FC with up_FC, and q_proj
with k_proj and v_proj).
in `down_proj`, there is no way to increase output channels (BN), we
split along K dimension once to reduce the number of splits of N, to
keep BN big too.
### prefetch sub-weight matrix to hide memory latency by computation
for execution of sub-problem `subC=matmul(A, subB)` on single core, we
further decompose it into sub-problems as following, to mix
SW-prefetching with AMX computations:
```python
for k0 in range(0, K, K_step):
k1 = k0 + K_step
blkB = subB[k0:k1, :]
blkA = A[:, k0:k1]
next_blkB = subB[k1:k1+K_step, :]
# blkB's shape is [K_step, BN], which is L2 cache-able
# inside following AMX based matmul jit kernel
# we evenly scatter SW-prefetching instructions for next_blkB
subC+=matmul_kernel(blkA, blkB, sw_prefetch = next_blkB) .
```
### flexible post-ops
jit-based post-Ops allows us to:
- interleaving fusion of weights of gate & up along N dimension in unit
of 16 fp32 elements, so post-op jit-kernel can combine them using
activation(like SiLU) while both results are hot in cache.
- QKV fusion requires final outputs was stored into 3 independent output
memories, special post-op jit kernel can do that easily.
### Tickets:
- *ticket-id*
---------
Co-authored-by: Luo Cheng <cheng.luo@intel.com>
### Details:
- Initial problem: `test_custom_op` hanged on destruction because it was
waiting for a thread which tried to acquire GIL.
- The second problem is that pybind11 doesn't allow to work with GIL
besides of current scope and it's impossible to release GIL for
destructors. https://github.com/pybind/pybind11/issues/1446
- Current solution allows to release GIL for InferRequest and all called
by chain destructors.
### Tickets:
- CVS-141744
[TRANSFORMATIONS] Add a check to SDPAToPagedAttention to fail fast
Add a check to SDPAToPagedAttention to fail fast if no
ScaledDotProductAttention operation present in a graph hence no
transformation will be executed.
### Tickets:
- CVS-143067
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
---------
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Remove the limitation that oneDNN convolution doesn't support dynamic
input and output.
With this limitation, clDNN convolution will run forever in dpas
platform.
Because oneDNN Opt convolutions require blocked-format and current shape
agnostic convolution doesn't support block-format, convolution do
nothing in compile_graph stage and actual kernel compilation will happen
at first inference.
### Tickets:
- *141040*
Signed-off-by: hyunback <hyunback.kim@intel.com>
### Details:
- *Fixed Invalid Read in FuseLoops pass. Before we save loop ports by
reference from `current_loop_info`. However, this `current_loop_Info`
might be changed (another shared pointer) and the previous saved loop
ports by reference are invalid (the original `current_loop_info` is
expired)*
- *`Valgrind` doesn't throw exception anymore with this fix*
### Tickets:
- *N/A*
### Details:
- Currently we insert crop primitive for each out port of
(Variadic)Split ops, and if some of them are not used we just waste some
execution time. This patch modifies converter for Split to process ports
with real users only.
### Details:
- Removed potential tests of operator which won't be added to opset 1
(as it's frozen),
- enabled those which are present.
- Renamed function.
### Tickets:
- n/a
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
### Details:
- *Complete the remaining work after the
[PR](https://github.com/openvinotoolkit/openvino/pull/24297) is merged.*
- *Add test case.*
### Tickets:
- *141542*
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
### Details:
- Aligned build jobs with runner configuration to speed up the build
- added temporary variable data_type to resolve internal compilation
error with the new msvc (19.40)
https://jira.devtools.intel.com/browse/CVS-142993
### Tickets:
- *ticket-id*
### Details:
- *This PR fixes geomean level performance issue caused by
https://github.com/openvinotoolkit/openvino/pull/23180 on platforms with
sufficient cores.*
- *In the PR mensioned above, `parallel_nt` is used for executing
subgraph instead of `parallel_for5d`, becuase the former has performance
advantage for the case where #tasks is more than #threads*. That is
becuase using `parallel_nt`, we can initilize parameters for each
thread, while parameters are initilized for each task in using
`parallel_for5d`. So `parallel_nt` has less overheads.
- *However, for the case where #tasks is less than #threads,
`parallel_nt` brings performance regression. Because compared with
`parallel_for5d`, `parallel_nt` doesn't apply the argument
`tbb::static_partitioner{}` when calling `tbb::parallel_for`. Here we
use `parallel_nt_static` instead to fix the performance regression.*
### Details:
- In case when scale tensor was used by multiple compressed FCs we had
an exception on attempt to insert reorder between them. This patch
changes reorder name so now multiple reorders are inserted for each
Scale+FC pair, and then on later stages we merge them if they do same
tensor transform.
**Details:** EliminateLoopInputsOutputs transformation helps to avoid an
isolated sub-graphs `Parameter->Result` in a body graph of `Loop` and
`TensorIterator`. Such sub-graphs are useless and can by-pass `Loop`
operation.
**Ticket:** 142440
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Evgeny Kotov <evgeny.kotov@intel.com>
Co-authored-by: Tikhonov Ivan <ivan.tikhonov@intel.com>
### Details:
- Optimize RMS kernel by subgroup block IO
- Compute maximum possible LWS and the number of data in subgroups
- Add test cases which covers leftover handling and unaligned data shape
### Tickets:
- 141845
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
### Details:
- `ov::intel_npu::create_executor` is declared a supported property only
for the `intel_npu::CompiledModel` class. The current PR adds an entry
inside `intel_npu::Plugin` as well.
### Tickets:
- *CVS-143060*
### Details:
- Change the default softmax accumulator type from FP16 to FP32 (this
doesn't affect performance, as well as accuracy at first glance;
however, let's stick to higher precision here)
- Applied review comments from PR
https://github.com/openvinotoolkit/openvino/pull/24466
- Minor internal operations refactoring (SDPA and IndirectSDPA)
- Add new tests for GQA optimization and for indirect inputs
### Details:
- Changes needed to enable Squeeze (and models) with axes pointing to
non-squeezable dims (currently ov throws an error for such case) -
considered as backward compatible change
- If agreed to update functionality within the same version of the op,
actual changes are needed only in shape infer related functions to cover
**Template, CPU and GPU plugins**:
- src/core/shape_inference/include/squeeze_shape_inference.hpp
- src/plugins/intel_cpu/src/shape_inference/custom/reshape.cpp
- The rest of the changes are test related.
- Pytorch FE Squeeze test enabled - passed
### Tickets:
- 142387
**Details:** In pre-commit we observe sporadic failure of Rint test on
arm
**Tickets:** TBD
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
### Details:
- *introduce performance hint and threads scheduling in CPU inference*
### Tickets:
- *CVS-138834*
---------
Co-authored-by: Sun Xiaoxia <xiaoxia.sun@intel.com>
### Details:
- WA of out-of-bound access for fc kernel.
- Use OneDNN FC kernel for fp16 FC when possible. It hides the issue
because the target platform can use OneDNN.
- For shape-agnostic kernel, TILE_B=8 is chosen. It causes out-of-bound
memory access for LLM 2nd token.
### Tickets:
- 142277
### Details:
- *United `LoopEndStatic` and `LoopEndDynamic` into one node `LoopEnd`
to avoid extra conditions in the code and improve performance since some
pointer data shifts might be known and compiled in JIT code*
- *Removed dynamic aarch64 loop emitters since they don't work anyway
CVS-141550*
- *Added support dynamism to `IdentifyBuffers` and
`DefineBufferClusters`. It's not efficient algorithm since we don't know
exact values of data pointer shifts and cannot be sure that they will be
proportionally in runtime. It should be implemented as the separate
feature based on some judgments, for example*
### Tickets:
- *141268*
### Prerequisites:
- *https://github.com/openvinotoolkit/openvino/pull/21922*
### Details:
- SliceToStridedSlice was moved from MOC to Common transformations.
- Updated RoPE fusion patterns
- 2 new transformations: EliminateSlice, SliceSequenceToSingleSlice
- Updated EliminateStridedSlice transformation to support int32_max
case, deleted transformation duplicate from CPU
- Added new unit tests
Tested locally with these models:
- [x] Means that Model compilation passed, and all RoPE related
transformations were applied in the same order
- [x] hf-internal-testing/tiny-random-StableLmForCausalLM
- [x] hf-internal-testing/tiny-random-FalconForCausalLM
- [x] hf-internal-testing/tiny-random-Starcoder2ForCausalLM
- [x] hf-internal-testing/tiny-random-LlamaForCausalLM
- [x] hf-internal-testing/tiny-random-GPTNeoXForCausalLM
- [x] hf-internal-testing/tiny-random-GPTJForCausalLM
- [x] hf-internal-testing/tiny-random-CodeGenForCausalLM
- [x] hf-internal-testing/tiny-random-MistralForCausalLM
- [x] hf-internal-testing/tiny-random-PhiForCausalLM
- [x] Qwen/Qwen1.5-7B
- [x] THUDM/chatglm3-6b
- [x] EleutherAI/gpt-neox-20b
- [x] google/gemma-2b-it
- [x] EleutherAI/gpt-j-6b
- [x] meta-llama/Meta-Llama-3-8B
- [x] mistralai/Mistral-7B-v0.1
### Tickets:
- *CVS-126971*
### Details:
As a result, you can use this code for most of your needs:
- in a Jupiter notebook
- as a cmd tool
- with model weights being far away
```
def configure_visualizer(enable=True, max_const_elements=7):
# parameter max_const_elements manipulates the number of first elements of a constant that will be serialized
# to svg. Put 0 if the model is on a share
import os
envs_to_set = [
'OV_VISUALIZE_TREE_IO',
'OV_VISUALIZE_TREE_OUTPUT_SHAPES',
'OV_VISUALIZE_TREE_OUTPUT_TYPES',
'OV_VISUALIZE_TREE_EDGE_LABELS',
# symbolic information and partial values
'OV_VISUALIZE_APPLY_SYMBOLIC_PROPAGATION',
'OV_VISUALIZE_PARTIAL_VALUES_AND_LABELS'
]
for e in envs_to_set:
os.environ[e] = '1' if enable else '0'
os.environ["OV_VISUALIZE_TREE_CONST_MAX_ELEMENTS"] = str(max_const_elements) if enable else "7"
def serialize_model_to_svg(model, output_name='serialized_model.svg'):
from openvino.runtime.passes import VisualizeTree
configure_visualizer(True)
VisualizeTree(output_name).run_on_model(model)
configure_visualizer(False)
def convert_model_to_svg(input_name, output_name=None):
import openvino as ov
core = ov.Core()
model = core.read_model(input_name)
if output_name is None:
output_name = os.path.splitext(os.path.basename(input_name))[0] + '.svg'
serialize_model_to_svg(model, output_name)
if __name__ == "__main__":
import sys, os
arguments = sys.argv[1:]
if len(arguments) not in [1, 2]:
print(f"Usage {sys.executable} {__file__} /path/to/model [output_file_name.svg]")
sys.exit(1)
output_file = os.path.splitext(os.path.basename(arguments[0]))[0] + '.svg' if len(arguments) == 1 else arguments[1]
convert_model_to_svg(arguments[0], output_file)
```
SharedPtr mutual reference fix
### Details:
- use lambda function replacing "Return" function of Gmock in ON_CALL
for *mockIExeNet.get()
- use lambda function replacing "Return" function of Gmock in ON_CALL
for *mockIExeNetActual.get()
"Retrun" function would cause increment of use account of
inferReqInternal and inferReqInternalActual, thereby causing mutual
reference of SharedPtr between mockIExeNet and inferReqInternal or
mockIExeNetActual and inferReqInternalActual.
### Ticket:
- [CVS-134423](https://jira.devtools.intel.com/browse/CVS-134423)
### Details:
Add tests for the SPDAToPagedAttention transformation to Precommit
1) Check if PagedAttentionExtension node appeared in the model after the
transformation
2) Check 2 last dimensions of 'key_cache.' and 'value_cache.' inputs to
be static.
### Tickets:
- [CVS-138943](https://jira.devtools.intel.com/browse/CVS-138943)
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
### Details:
- Windows std::wstring convert to std::string some unicode will be ???,
such as L"unicode_Яㅎあ" -> "unicode_???"
### Tickets:
- CVS-141444
- CVS-141729
---------
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
### Details:
- Reduce the condition for not-using usm-host input buffer
- This is to reduce the overhead in next-gen GPU performance where
usm_host is the input buffer
### Tickets:
- 141493
### Details:
This PR is a copy of
https://github.com/openvinotoolkit/openvino/pull/24757 from release
branch, it includes:
- Enabled SDPA by default
- Updated SDPA decomposition rule to cover only well-checked cases
- Updated functional tests accordingly
ze_device_properties_t has pNext pointer and if it's not nullptr then
it's assumed that it points to a different structure. pNext being
uninitialized can lead to a crash during zeDeviceGetProperties.
### Details:
- Fixed Densify reading implementation
- Added test for Densify
- Added test for Dequantize
- Removed unnecessary model test
### Tickets:
- 127943
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
### Details:
- Port of #24703
- The issue that was present on the release branches does not affect the
master branch yet but it could if, for example, a new folder is added to
the cache path.
### Tickets:
- *142372*
### Details:
- *Exception handling: exception message logging*
### Tickets:
- *NotSupported exception can have message, let's display it. For
example this ticket will be clear in this case: CVS-139934*
- *Part of CVS-142409*
During the elimination of dependencies from `beam_idx` input and
`ReadValue`(s), we are replacing them by the new PA-related inputs and
sub-expressions dependent on other remaining inputs. In such
replacements we need to guarantee matching shape and element type of old
and new nodes. Before this PR it was not guaranteed for shape and
sometimes a scalar was replaced by a shape of rank 1 that led to errors
like `'start' input is not a scalar`. Now the shape is aligned.
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
This is a follow up to #23955
### Details:
- Added functional test for ROI Align Rotated
- Fixed a "bug" with wrong batch index inside cl kernel revealed by
functional test for ROI Align Rotated.
### Tickets:
- *[141877](https://jira.devtools.intel.com/browse/CVS-141877)*
### Details:
- Coverity Fix as a line of the code is not reachable
- Remove the line of dead code and add different log message for 3
different possible return
### Tickets:
- E-125476
LNL does not support 8 as subgroup size. It need to check before
compiling fc_imad shape agnostic kernel.
### Details:
- *Set subgroup size to 16 if simd8 is not supported in the target
device.*
Fix for TEST_DEVICE=CPU comparison of float16 infinite values
### Details:
- Introduced specialization for conversion float16->float32 in the CPU
plugin
- xfail in layer tests is disabled for CPU device
### Tickets:
- *24245*
### Details:
- Added indirect inputs support for SDPA kernel
- Added setter for causal flag for ScaledDotProductAttention operation
- Added `ov::intel_gpu::hint::enable_sdpa_optimization` to
`ov::supported_properties` list
- Removed unused `TARGET_SEQ_LEN_BLOCK_SIZE > 1` check from kernel for
single token processing
- Minor refactoring
- Added `OV_GPU_EnableSDPA` debug option (which allows to force SDPA
kernel for any ScaledDotProductAttention operation _(=1)_ / or
completely disable SDPA kernel _(=0)_, ignoring
`ov::intel_gpu::hint::enable_sdpa_optimization` property)
### Tickets:
- *CVS-141213*
**Details:** Previously, we were able to convert models requiring
openvino-tokenizers extensions (like with string operations, ragged
tensors) only in Python by `convert_model`. Now we add option to convert
using `ovc` tool. The tool internally imports extensions if they are
installed.
**Ticket:** TBD
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Enabled model: bigcode/starcoderbase-3b, bigcode/gpt_bigcode-santacoder.
There is a moderate refactoring in the code: move common code for
ReadValue-based pattern matching K, V and merged KV state management
into a single piece of code.
Extended UBR pattern for case when MQA doesn't have Reshape in the end
because KV cache is 3D instead of 4D and corresponding changes in the
detection of the number of KV heads.
In sync with https://github.com/ilya-lavrenov/openvino.genai/pull/20
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Updates the requirements on [h5py](https://github.com/h5py/h5py) to
permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2c80981022"><code>2c80981</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2405">#2405</a> from
takluyver/rever-3.11</li>
<li><a
href="76afe10048"><code>76afe10</code></a>
Add HDF5 1.14.4 fixes to release notes</li>
<li><a
href="009047936d"><code>0090479</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2406">#2406</a> from
ajelenak/libhdf5-1.14.4</li>
<li><a
href="0f071a4d64"><code>0f071a4</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2407">#2407</a> from
h5py/dependabot/github_actions/actions-45223d3826</li>
<li><a
href="944a4a6967"><code>944a4a6</code></a>
Bump peaceiris/actions-gh-pages from 3 to 4 in the actions group</li>
<li><a
href="8f513ced5e"><code>8f513ce</code></a>
Clean up release notes</li>
<li><a
href="62bd813202"><code>62bd813</code></a>
Rename new whatsnew file, add to index</li>
<li><a
href="9c305e23ce"><code>9c305e2</code></a>
Updated CHANGELOG for 3.11.0</li>
<li><a
href="9e107c4a78"><code>9e107c4</code></a>
bumped version to 3.11.0</li>
<li><a
href="71af64dec8"><code>71af64d</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2360">#2360</a> from
loichuder/visit-links</li>
<li>Additional commits viewable in <a
href="https://github.com/h5py/h5py/compare/3.1.0...3.11.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
### Details:
- To reduce Infer overhead
- The new flow is:
1) Define sync points from all the nodes perspective.
2) Use sync points in scope of Allocate
3) Define executable sync points from overall sync points but
considering only executable nodes.
4) Keep executable sync points ordered to avoid additional logic in
scope of Infer call.
### Tickets:
- *ticket-id*
### Details:
- *Extended LIR interface to make LIR creation from scratch possible*
- *Introduced `LoweredPassTestsF` base class for LIR comparison*
- *Added some helpers for easier LIR creation from scratch (not from
`ov::Model`)*
- *Introduced `LIRComparator` class. For LIR related things comparison,
a custom comparators were written. For `ov::Node` comparison, the
existing comparator from `GraphComparator` class is used.*
- *Added a test case for `InsertLoadStore` pass: as an example of a
simple LIR comparison (only expressions + `ov::Node` comparison is
necessary to fully cover the pass code)*. The pass was not fully
covered.
- *Added test cases for `BrgemmBlocking` cpu lowered pass: as an example
of complicated LIR comparison, where `LoopManager` and descriptors
comparison are also necessary. This pass was fully covered.*
### Tickets:
- *CVS-139926*
Details:
Due to the issue in the e2e test in
https://github.com/openvinotoolkit/openvino/pull/24597 we decided to
exclude Concat to Tile conversion from the transformation. It will be
covered in another ticket.
- Add a ConcatToBroadcast transformation to replace Concat having inputs
from the same output with a Broadcast
- Add a test for the ConcatToBroadcast transformation
Significantly reduce model compile time and performance time.
Tickets:
[CVS-138829](https://jira.devtools.intel.com/browse/CVS-138829),
[CVS-138077](https://jira.devtools.intel.com/browse/CVS-138077)
---------
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
Co-authored-by: Andrii Staikov <andrii.staikov@intel.com>
Convolution which has u8 AvgPool layer as input0 is handled as quantized
in transformation and it has zp constants in transformed graph. But gpu
plugin has changed AvgPool output data type to fp32 due to accuracy
issue. The change causes convolution ocl impl doesn't recognize
quantized convolution.
### Details:
- Add Workaround in convolution ocl impl for fp32 output hack of avgpool
when u8/i8 input
### Tickets:
- 141396
This reverts commit 03aad66230.
### Details:
Revert the commit because there are some models with stateful OPS
(assign/readvalue) not in same attention node, hetero pipeline
parallelism does not work.
### Tickets:
- *CVS-141771*
### Details:
- *I lost code `t_stream_count_map[(void*)this] = item.first;` in
#19832*
- *the thread safe issue happen in below workflow*
- create thread A
call CustomThreadLocal:local() in thread A -> create stream A (the count
of stream A is 2)
destory thread A (the count of stream A is 1)
create thread B (same thread id with thread A)
call CustomThreadLocal:local() in thread B -> use stream A(the count of
stream A is 1, so it's broken)
- *add testcase, also fix
https://github.com/openvinotoolkit/openvino/pull/19986/files#r1332774754*
### Tickets:
- Closes https://github.com/openvinotoolkit/openvino/issues/24509
---------
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
### Details:
- update logic of only loading cached model to GPU with AUTO if GPU
cached blob exists and `ov::intel_auto::enable_startup_fallback` is
enable
### Tickets:
- CVS-138574
### Details:
- Apply dynamic padding for onednn gemm
- Update memory descriptor to handle the padded dims and strides
### Tickets:
- 140516
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
### Details:
- Removed 'actual comparison size' parameter from base template test
class `CommonReferenceTest`.
- Moved partial comparison logic to
`ReferenceExperimentalPGGLayerTest::Validate()` - it's needed only in
this test.
### Tickets:
- CVS-137159
### Details:
- Reference implementation and ref tests for
[internal::RMS](70142121c1/src/common/transformations/include/ov_ops/rms.hpp)
(moved from gpu custom ops) available for common transformations
- Enable Accuracy tests for RMSFusion transformation
The changes moving RMS from gpu to internal are from PR:
- #24539 (to be merged first)
Internal RMS op docs:
- https://github.com/openvinotoolkit/openvino/pull/24564
### Tickets:
- Related to 136262
---------
Co-authored-by: michal-miotk <michal.miotk@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
### Details:
QAT documentation for NNCF Torch and NNCF Tensorflow is spitted on two
pages
* quantization-aware-training-pytorch.rst is QAT by `nncf.quantize`
method documentation
* quantization-aware-training-tensorflow.rst file is a copy of original
quantization-aware-training.rst without the PyTorch part
Link to generated doc:
http://openvino-doc.iotg.sclab.intel.com/nncf-docs/openvino-workflow/model-optimization-guide/compressing-models-during-training.html
### Tickets:
- 129586
---------
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Remove stringop overflow warning as a temporal workaround for oneDNN
warning issue, which caused compilation issues on Ubuntu 24.04 (GCC 13 +
LTO)
### Details:
- Remove warning from intel-gpu link flag properties
### Tickets:
- CVS-141874
### Details:
- Continuation of https://github.com/openvinotoolkit/openvino/pull/24588
- Initialized "int-to-type" map with more clear "string-to-type" map
first, using `num()` value from constructed dtypes.
### Tickets:
- *...*
Creating `Limitation of the bf16 inference precision` section to
describe an issue of higher accuracy loss than the pre-determined
threshold of 0.5% and present possible solutions.
This PR addresses JIRA ticket: 138832.
### Details:
- Fixed bug with inlined inputs for fx decoder which were being added as
parameters and are not needed.
### Tickets:
- (https://jira.devtools.intel.com/browse/CVS-116702)
---------
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* Use block read in 4byte aligned left-over case. Static was already
used (https://github.com/openvinotoolkit/openvino/pull/23400) This PR
will apply in dynamic.
### Tickets:
- *141032*
---------
Signed-off-by: hyunback <hyunback.kim@intel.com>
### Details:
- FC with symmetrically quantized/compressed weight may have i8 (instead
of u8) as weight data type (it saves the zero-point subtraction cost),
this change added support to such weight dt.
- oneDNN fork PR: https://github.com/openvinotoolkit/oneDNN/pull/249
### Tickets:
- *CVS-131042*
### Details:
- Fix Electron compatibility on Windows by specifying flag:
`CMAKE_SHARED_LINKER_FLAGS="/DELAYLOAD:NODE.EXE"`
### Tickets:
- 141536
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Updates the requirements on [h5py](https://github.com/h5py/h5py) to
permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2c80981022"><code>2c80981</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2405">#2405</a> from
takluyver/rever-3.11</li>
<li><a
href="76afe10048"><code>76afe10</code></a>
Add HDF5 1.14.4 fixes to release notes</li>
<li><a
href="009047936d"><code>0090479</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2406">#2406</a> from
ajelenak/libhdf5-1.14.4</li>
<li><a
href="0f071a4d64"><code>0f071a4</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2407">#2407</a> from
h5py/dependabot/github_actions/actions-45223d3826</li>
<li><a
href="944a4a6967"><code>944a4a6</code></a>
Bump peaceiris/actions-gh-pages from 3 to 4 in the actions group</li>
<li><a
href="8f513ced5e"><code>8f513ce</code></a>
Clean up release notes</li>
<li><a
href="62bd813202"><code>62bd813</code></a>
Rename new whatsnew file, add to index</li>
<li><a
href="9c305e23ce"><code>9c305e2</code></a>
Updated CHANGELOG for 3.11.0</li>
<li><a
href="9e107c4a78"><code>9e107c4</code></a>
bumped version to 3.11.0</li>
<li><a
href="71af64dec8"><code>71af64d</code></a>
Merge pull request <a
href="https://redirect.github.com/h5py/h5py/issues/2360">#2360</a> from
loichuder/visit-links</li>
<li>Additional commits viewable in <a
href="https://github.com/h5py/h5py/compare/3.1.0...3.11.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Extended model support in PagedAttention transformation:
- SDPA with `scale` input
- Case when KV inputs into SDPA are implicitly broadcasted by 2nd
dimension.
- More accurate application of PrevSequenceLengthPattern that now can
track batch dimension as well.
Enables tiiuae/falcon-7b and BAAI/AquilaChat2-7B.
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
### Details:
- Dev documentation for the existing
[internal::RMS](70142121c1/src/common/transformations/include/ov_ops/rms.hpp)
op
(specified as is without any additional features, to be developed and
updated if needed)
- RMS was created initially as a [custom gpu
RMS](9fadb5ac64/src/plugins/intel_gpu/include/intel_gpu/op/rms.hpp)
operation, and it's going to be moved to op::internal::RMS to be
available for common transformations
- Similar approach as it was done for AUGRUCell/AUGRUSeqence, so the
place for such documents has been already agreed and it's not a part of
the official web docs tree
### Tickets:
- 134914, dicsussion 129027
### Details:
- Fix the issue where `conv_params.quantization` was not correctly
identified as a quantization case when using U8 weights data (which is
allowed by ConvertConvolutionToInternal transformation pass)
### Tickets:
- 139740
Sometimes the approximate attribute is not specified in the .tflite
file, we need to check its availability before retriving it, otherwise
we might hit the 'Check 'opts != nullptr' failed' failure with calling
get_attribute() directly.
Signed-off-by: Keyon Jie <yang.jie@linux.intel.com>
### Details:
This patch contains all the necessary changes to support internal states
in the composite nodes (i.e. If, Loop, TensorIterator operations). Main
changes:
1. Move the internal states registry to the graph context, so that all
the nested internal states become accessible on the infer request level.
2. Move the state assing/commit actions to the node execute level, so
that Loop and TensorIterator ops are automatically supported.
### Tickets:
- CVS-134470
### Details:
- *Create PoC implementation for EmbeddingBag Offsets and Packed that
introduce mode 'mean'*
- *Add Core, Reference and CPU implementation for both ops*
Specification will be added in separate PR
### Tickets:
- *138229*
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
### Details:
- *Extract BrgemmCopyB from M blocking if there are no KN blocking
loops*
- *Tests will be added after #24152 merge*
### Tickets:
- *CVS-140751*
### Details:
- This PR extends existing weights type conversion + repack optimization
for bf16 precision. This change is needed to minimize memory footprint
on model compilation stage for models with BF16 compressed weights.
Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/psf/requests/releases">requests's
releases</a>.</em></p>
<blockquote>
<h2>v2.32.0</h2>
<h2>2.32.0 (2024-05-20)</h2>
<h2>🐍 PYCON US 2024 EDITION 🐍</h2>
<p><strong>Security</strong></p>
<ul>
<li>Fixed an issue where setting <code>verify=False</code> on the first
request from a
Session will cause subsequent requests to the <em>same origin</em> to
also ignore
cert verification, regardless of the value of <code>verify</code>.
(<a
href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li>
</ul>
<p><strong>Improvements</strong></p>
<ul>
<li><code>verify=True</code> now reuses a global SSLContext which should
improve
request time variance between first and subsequent requests. It should
also minimize certificate load time on Windows systems when using a
Python
version built with OpenSSL 3.x. (<a
href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li>
<li>Requests now supports optional use of character detection
(<code>chardet</code> or <code>charset_normalizer</code>) when
repackaged or vendored.
This enables <code>pip</code> and other projects to minimize their
vendoring
surface area. The <code>Response.text()</code> and
<code>apparent_encoding</code> APIs
will default to <code>utf-8</code> if neither library is present. (<a
href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li>
</ul>
<p><strong>Bugfixes</strong></p>
<ul>
<li>Fixed bug in length detection where emoji length was incorrectly
calculated in the request content-length. (<a
href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li>
<li>Fixed deserialization bug in JSONDecodeError. (<a
href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li>
<li>Fixed bug where an extra leading <code>/</code> (path separator)
could lead
urllib3 to unnecessarily reparse the request URI. (<a
href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li>
</ul>
<p><strong>Deprecations</strong></p>
<ul>
<li>Requests has officially added support for CPython 3.12 (<a
href="https://redirect.github.com/psf/requests/issues/6503">#6503</a>)</li>
<li>Requests has officially added support for PyPy 3.9 and 3.10 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
<li>Requests has officially dropped support for CPython 3.7 (<a
href="https://redirect.github.com/psf/requests/issues/6642">#6642</a>)</li>
<li>Requests has officially dropped support for PyPy 3.7 and 3.8 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
</ul>
<p><strong>Documentation</strong></p>
<ul>
<li>Various typo fixes and doc improvements.</li>
</ul>
<p><strong>Packaging</strong></p>
<ul>
<li>Requests has started adopting some modern packaging practices.
The source files for the projects (formerly <code>requests</code>) is
now located
in <code>src/requests</code> in the Requests sdist. (<a
href="https://redirect.github.com/psf/requests/issues/6506">#6506</a>)</li>
<li>Starting in Requests 2.33.0, Requests will migrate to a PEP 517
build system
using <code>hatchling</code>. This should not impact the average user,
but extremely old
versions of packaging utilities may have issues with the new packaging
format.</li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matthewarmand"><code>@matthewarmand</code></a>
made their first contribution in <a
href="https://redirect.github.com/psf/requests/pull/6258">psf/requests#6258</a></li>
<li><a href="https://github.com/cpzt"><code>@cpzt</code></a> made their
first contribution in <a
href="https://redirect.github.com/psf/requests/pull/6456">psf/requests#6456</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's
changelog</a>.</em></p>
<blockquote>
<h2>2.32.0 (2024-05-20)</h2>
<p><strong>Security</strong></p>
<ul>
<li>Fixed an issue where setting <code>verify=False</code> on the first
request from a
Session will cause subsequent requests to the <em>same origin</em> to
also ignore
cert verification, regardless of the value of <code>verify</code>.
(<a
href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li>
</ul>
<p><strong>Improvements</strong></p>
<ul>
<li><code>verify=True</code> now reuses a global SSLContext which should
improve
request time variance between first and subsequent requests. It should
also minimize certificate load time on Windows systems when using a
Python
version built with OpenSSL 3.x. (<a
href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li>
<li>Requests now supports optional use of character detection
(<code>chardet</code> or <code>charset_normalizer</code>) when
repackaged or vendored.
This enables <code>pip</code> and other projects to minimize their
vendoring
surface area. The <code>Response.text()</code> and
<code>apparent_encoding</code> APIs
will default to <code>utf-8</code> if neither library is present. (<a
href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li>
</ul>
<p><strong>Bugfixes</strong></p>
<ul>
<li>Fixed bug in length detection where emoji length was incorrectly
calculated in the request content-length. (<a
href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li>
<li>Fixed deserialization bug in JSONDecodeError. (<a
href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li>
<li>Fixed bug where an extra leading <code>/</code> (path separator)
could lead
urllib3 to unnecessarily reparse the request URI. (<a
href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li>
</ul>
<p><strong>Deprecations</strong></p>
<ul>
<li>Requests has officially added support for CPython 3.12 (<a
href="https://redirect.github.com/psf/requests/issues/6503">#6503</a>)</li>
<li>Requests has officially added support for PyPy 3.9 and 3.10 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
<li>Requests has officially dropped support for CPython 3.7 (<a
href="https://redirect.github.com/psf/requests/issues/6642">#6642</a>)</li>
<li>Requests has officially dropped support for PyPy 3.7 and 3.8 (<a
href="https://redirect.github.com/psf/requests/issues/6641">#6641</a>)</li>
</ul>
<p><strong>Documentation</strong></p>
<ul>
<li>Various typo fixes and doc improvements.</li>
</ul>
<p><strong>Packaging</strong></p>
<ul>
<li>Requests has started adopting some modern packaging practices.
The source files for the projects (formerly <code>requests</code>) is
now located
in <code>src/requests</code> in the Requests sdist. (<a
href="https://redirect.github.com/psf/requests/issues/6506">#6506</a>)</li>
<li>Starting in Requests 2.33.0, Requests will migrate to a PEP 517
build system
using <code>hatchling</code>. This should not impact the average user,
but extremely old
versions of packaging utilities may have issues with the new packaging
format.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="d6ebc4a2f1"><code>d6ebc4a</code></a>
v2.32.0</li>
<li><a
href="9a40d12778"><code>9a40d12</code></a>
Avoid reloading root certificates to improve concurrent performance (<a
href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li>
<li><a
href="0c030f78d2"><code>0c030f7</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>
from nateprewitt/no_char_detection</li>
<li><a
href="555b870eb1"><code>555b870</code></a>
Allow character detection dependencies to be optional in post-packaging
steps</li>
<li><a
href="d6dded3f00"><code>d6dded3</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6700">#6700</a>
from franekmagiera/update-redirect-to-invalid-uri-test</li>
<li><a
href="bf24b7d8d1"><code>bf24b7d</code></a>
Use an invalid URI that will not cause httpbin to throw 500</li>
<li><a
href="2d5f54779a"><code>2d5f547</code></a>
Pin 3.8 and 3.9 runners back to macos-13 (<a
href="https://redirect.github.com/psf/requests/issues/6688">#6688</a>)</li>
<li><a
href="f1bb07d39b"><code>f1bb07d</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6687">#6687</a>
from psf/dependabot/github_actions/github/codeql-act...</li>
<li><a
href="60047ade64"><code>60047ad</code></a>
Bump github/codeql-action from 3.24.0 to 3.25.0</li>
<li><a
href="31ebb8102c"><code>31ebb81</code></a>
Merge pull request <a
href="https://redirect.github.com/psf/requests/issues/6682">#6682</a>
from frenzymadness/pytest8</li>
<li>Additional commits viewable in <a
href="https://github.com/psf/requests/compare/v2.31.0...v2.32.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/openvinotoolkit/openvino/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Details:
- *This PR https://github.com/openvinotoolkit/openvino/pull/23384
triggers the existing bugs in `GatherCompressed` and `Fullyconnected`*
- *After converting all possible constants to FP16, `GatherCompresssed`
needs to output `FP16` when scale is changed to FP16*
- *After converting all possible constants to FP16, `FullyConnected`
needs to explicitly refuse `FP16` bias since `oneDNN` doesn't support
`FP16` bias.*
### Tickets:
- *CVS-140154*
### Details:
- *set correct numa id for different tbb version*
- *numa id has different mapping method in TBBBind since oneTBB
2021.4.0*
### Tickets:
- *CVS-141067*
### Details:
- enable AUTO to retrieve the hash id for cached model
- load cached model to GPU only with AUTO if cached blob exists and not
set `ov::intel_auto::enable_startup_fallback` explicitly to AUTO
### Tickets:
- CVS-140059
### Details:
- This PR updates the `kernels_cache` to build the selected kernels in
separate batches.
- This is a temporary WA to resolve performance degradation when some
kernels are built with other kernels in the same batch
- Currently, the selected kernel includes `gemm_tiled_opt`.
- The impacted scenario : Qwen INT4 first token latency for > 1K input
in MTL
### Tickets:
- GSD-8910
### Details:
- In set_state of gpu plugin, the src Tensor can have padding, which has
not handled by set_state
- Fixed set_state to handle src Tensor's padding properly
### Tickets:
- 140418
### Details:
- *After support for i64 was added in frontend `aten::adaptive_avg_pool`
started to fail if shape input is list.*
### Tickets:
- *CVS-141335*
**Details:** Remove test-generator dependency. It is removed since this
module seems to be no longer developed and maintained. So it is better
to remove this dependency. `pytest` can substitute `test-generator`.
**Ticket:** TBD
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
### Details:
PyTorch semantic of Expand operation requires Abs operation for
translation which is usually a no-op.
1. partial value and symbol propagation for Abs operation
2. optimize out the Abs with nonnegative input
3. for this case Abs(Concat([Gather(ShapeOf), -1, 1, -1])) pulls Abs and
tries to const-fold it. Resulting in Concat([Abs(Gather(ShapeOf)), 1, 1,
1]). Applies (2) to the graph and gets Concat([Gather(ShapeOf), 1, 1,
1]) because ShapeOf's lower bound is 0.
### Tickets:
- *CVS-141274*
### Details:
As ov RMSNorm has been not released yet as a public opset op, it was
proposed to keep it as internal.
RMSNorm is not currently a part of any frontend framework and the
requirements for RMSNorm may change.
There is a corresponding
[`internal::RMS`](70142121c1/src/common/transformations/include/ov_ops/rms.hpp)
op availavble for common transformations (moved from custom GPU).
The goal is to unify this work and tests into single RMS implementation.
---------------------------
The reference implementation is going to be reused for the
internal::RMS:
- https://github.com/openvinotoolkit/openvino/pull/24556
### Tickets:
- Related to: 129027
### Details:
- *Change the context_lens to past_lens*
- *past lens corresponds to first token within subgroup of scheduled
tokens but not includes itself*
- *...*
### Tickets:
- *ticket-id*
### Details:
- Enable permute + onednn gemm fusion for dynamic shape
- Fix unittests to test various permute orders fused with gemm
- This pr has dependency on onednn change on memory tag. Do not merge
until the memory tag change is ready
### Tickets:
- 138961
### Details:
- *Shared decompression between 2 Matmuls is supported for 8bits and
4bits weights*
- *Shared decompression between Matmul and Gather is supported for 8bits
and 4bits weights*
- *Added CPU subgraph test*
### Tickets:
- *CVS-140442* <= Validation results can be found there
- *CVS-137591*
- *CVS-136594*
### Details:
- Use float for intermediate calculation of grid sample op
- Adjust data ranges for fc unit test to avoid huge numbers
### Tickets:
- *CVS-134015*
### Details:
- Changed from the adjugate matrix to transpose of the matrix
### Tickets:
- 138715
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
### Details:
- *Add patch function to make 16bit models traceable. This reduce memory
consumption and time spent during conversion of models that are natively
saved in 16bit precision*
### Tickets:
- *CVS-136828*
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/jinja/releases">jinja2's
releases</a>.</em></p>
<blockquote>
<h2>3.1.4</h2>
<p>This is the Jinja 3.1.4 security release, which fixes security issues
and bugs but does not otherwise change behavior and should not result in
breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Jinja2/3.1.4/">https://pypi.org/project/Jinja2/3.1.4/</a>
Changes: <a
href="https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-4">https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-4</a></p>
<ul>
<li>The <code>xmlattr</code> filter does not allow keys with
<code>/</code> solidus, <code>></code> greater-than sign, or
<code>=</code> equals sign, in addition to disallowing spaces.
Regardless of any validation done by Jinja, user input should never be
used as keys to this filter, or must be separately validated first.
GHSA-h75v-3vvj-5mfj</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/jinja/blob/main/CHANGES.rst">jinja2's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.1.4</h2>
<p>Released 2024-05-05</p>
<ul>
<li>The <code>xmlattr</code> filter does not allow keys with
<code>/</code> solidus, <code>></code>
greater-than sign, or <code>=</code> equals sign, in addition to
disallowing spaces.
Regardless of any validation done by Jinja, user input should never be
used
as keys to this filter, or must be separately validated first.
:ghsa:<code>h75v-3vvj-5mfj</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="dd4a8b5466"><code>dd4a8b5</code></a>
release version 3.1.4</li>
<li><a
href="0668239dc6"><code>0668239</code></a>
Merge pull request from GHSA-h75v-3vvj-5mfj</li>
<li><a
href="d655030770"><code>d655030</code></a>
disallow invalid characters in keys to xmlattr filter</li>
<li><a
href="a7863ba9d3"><code>a7863ba</code></a>
add ghsa links</li>
<li><a
href="b5c98e78c2"><code>b5c98e7</code></a>
start version 3.1.4</li>
<li><a
href="da3a9f0b80"><code>da3a9f0</code></a>
update project files (<a
href="https://redirect.github.com/pallets/jinja/issues/1968">#1968</a>)</li>
<li><a
href="0ee5eb41d1"><code>0ee5eb4</code></a>
satisfy formatter, linter, and strict mypy</li>
<li><a
href="20477c6357"><code>20477c6</code></a>
update project files (<a
href="https://redirect.github.com/pallets/jinja/issues/5457">#5457</a>)</li>
<li><a
href="e491223739"><code>e491223</code></a>
update pyyaml dev dependency</li>
<li><a
href="36f98854c7"><code>36f9885</code></a>
fix pr link</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/jinja/compare/3.1.3...3.1.4">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/openvinotoolkit/openvino/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
### Details:
A method `get_output_shape()` to the Model class in the Node.js API of
OpenVINO framework.This method would allow users to retrieve the output
shape of a model, which is useful for understanding the model's output
data.
### Implemented
- [x] add get_output_shape() method on C++ side
(src/bindings/js/node/src/model.cpp)
- [x] update TypeScript definition (src/bindings/js/node/lib/addon.ts)
- [x] create unit test for added functionality using Node.js Test Runner
Fixes#23568
---------
Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>
### Details:
- Check the value of setlocale for export/import, if different with "C"
will set to "C" and record the original value, after export/import done
will reset to the original.
- *Fix the error caused by pugixml library with The setlocale function
installs the specified system locale or its portion as the new C locale.
different C may return unexpected results with setlocal()*
### Tickets:
- https://github.com/openvinotoolkit/openvino/issues/24370
---------
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: River Li <river.li@intel.com>
### Details:
- *Remove specific plugin names in shared test case*
- *Throw exception if device list in both ov::device::priorities() and
device name (For example AUTO:GPU,CPU)*
### Tickets:
- *CVS-130214*
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
### Details:
- Extended stateful API to CompiledModel class to allow working on
underlaying InferRequest.
- Added test cases and refactored them.
### Tickets:
- CVS-139972
### Details:
- Added `calc_output_layouts()` impl for remaining operations, so since
that patch all ops have new shape infer method, so we can start
migration to the new path and removal of the legacy shape infer
- Added RotateNMS handling in
`non_max_suppression_inst::calc_output_layouts`
- Pass max iterations count in loop shape infer
- Added second out handling in pooling primitive
- Added PSROIPooling and DeformablePSROIPooling handling in
`roi_pooling_inst::calc_output_layouts`
- Updated gpu tensor accessor to return empty tensor for non-existing
dependency instead of exception to simplify usages
### Details:
- *Support torch.any and torch.all on both FX and TS side*
- *Support aten::isfinite, aten::isinf and aten::isnan*
### Tickets:
- *ticket-id*
### Details:
- In dynamic models, only the `convolution_gpu_bfyx_os_iyx_osv16` kernel
has been used, which was one of the reasons for the performance gap
between the static and dynamic models.
- This PR adds a new `convolution_gpu_bfyx_os_iyx_osv32` kernel for
better performance of dynamic models.
### Tickets:
- 134700
### Details:
- *Fix regression for model orca.*
- *Add test to conver orca for CPU/GPU test.*
### Tickets:
- *138406*
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
### Details:
- When there are some submodels need to be merged into one model, if
there is a independent submodel in the submodels, such as `chatglm,
qwen-7b-chat`:
submodel A (independent )
submodel B
submodel C
B and C will be the same because they have connection(input/output), so
when merge C into A after B merged, it will throw `parameters already
added` errior in add_parameters().
So add a distinct_submodels_index set to store the distinct submodels
will be merged.
- When split `baichuan-13b` into 2 parts, it will has error: Model
references undeclared parameters: opset1::Parameter input_ids () ->
(i64[?,?]), because there are some ops not removed correctly.
### Tickets:
- *CVS-140527*
### Details:
- Update OneDNN to v3.5
- Fix OOB memory access with fake alignment
- Do not fake-align if input-memory is not allocated properly (it
impacts test cases where FC gets input from data node)
- This fix is applied only to some hardware because of performance
concern
### Tickets:
- 141215
- 138012
### Details:
- *Re-enable conv with brgemm implement on avx2 platform with fp32 ,
also heuristic cause by wavernn-upsampler regression*
- *Switch OV cpu deconv implement from ONEDNN convbackward_data
primitive to deconv primitive. Support const and non-const weight;
Always expose planar weight layout to cpu graph; weight cache to save
blocked const weight ; runtime reorder for none-const weight*
- *enable bias fusing for fp32 to replace legacy depthwise fusing*
- *enable deconv brgconv for fp32 and INT8 deconv*
- *update test case*
### About test case/WA update:
- *OV has feature to set the output shape of conv_backward_data. Which
would cause pad_r < 0 && pad_r + dil < 0, so applied the WA in onednn to
create deconv OP*
- *Deconv only has one ref entry(CPU_INSTANCE(ref_deconvolution_fwd_t)
for JIT,gemm and c++ reference implement. So one layout can't use RTinfo
to filter implement. Update test cases with 1x1 related or SSE related.
For example: backward data has 2 entries when input is nChw16C :
jit_avx512_common_1x1_convolution_bwd_data_f32_t
jit_avx512_common_convolution_bwd_data_t<f32> to support
So OV filter can work to get desired implement type. However, deconv
only has only, it means that nChw16C would only get one implement type.
So filter would not work.*
- *Change stride to 1 unit because of brgemm limitation*
- *subgraph concat deconv test can't hit ref implement because the gemm
and ref share one entry implement.*
- *Conformance test conv related failed, but can pass when merging 3.3
with brg. Failed caused by
.
Seems threshold changes. Also AMX and avv512 core would fail on these
conformance test but not enabled on Azure platforms. Guess the expected
result is based on avx2 jit. So current just remove these test*
### Tickets:
- *CVS-133120, CVS-122688*
### Details:
- * Set needs_completion_event when current node has runtime skippable
users in primitive_inst ctor*
- * Add marking runtime skippable for reorder in build time*
### Tickets:
- *CVS-135257*
@tadamczx and @ilya-lavrenov It took a while but I delivered what was
promised, openVINO at the official openSUSE factory. It took more than 6
months of work to get approval from the community...
---------
Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
### Details:
- *If there are several MHA-patterns with the same Brgemms inside (the
same configs), first Brgemm calls `compile_kernel` using executor:
initializes `palette` and compile `kernel`. The brgemm from the second
MHA finds in the cache this `kernel` by the config that is equal to
first Brgemm has. Hovewer, previously `palette` was stored in `config` -
this `config` is not cached and not shared between emitters. Thus,
second Brgemm had cached `kernel` and config with empty `palette` since
it's inited `compile_kernel` that is not called for this `Brgemm`. The
PR introduces new structure that contains `compiled_kernel` and
`palette` - these values are cached together and can be shared between
nodes and emitters*
### Tickets:
- *140771*
### Details:
- Added basic implementation of ROIAlignRotated
- It uses implementation from core::reference - the goal was just to add
support for the op, not the optimized implementation.
- No unit tests added, since impl is already tested by core::reference
and functional tests.
### Tickets:
- [CVS-135847](https://jira.devtools.intel.com/browse/CVS-135847)
---------
Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>
### Details:
- *Merge first token and second token inference into one parallel loop*
- *~~Additional optimization: pre-transpose k-cache, pre-pack v-cache if
needed~~*
- *Additional optimization for first token: save q * k' upper triangle
matrix computation and (q * k') * v lower triangle matrix computation*
- *C++ pipeline can enable it:
https://github.com/ilya-lavrenov/openvino.genai/pull/9*
- *TODO(in another PR):*
- alibi support
- performance tuning
- testcase
### Tickets:
- *[138673](https://jira.devtools.intel.com/browse/CVS-138673)*
Fix PagedAttention PrevSequenceLengthPattern pattern
Fix pattern of PrevSequenceLengthPattern that makes it match twice than
needed resulting in the incorrect inference.
Provide the correct version of Gather (v8::Gather) and amend
PrevSequenceLengthPattern pattern for the correct matching.
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
### Details:
- *item1*
- *...*
### Tickets:
- *ticket-id*
Signed-off-by: Andrii Staikov <andrii.staikov@intel.com>
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
### Details:
To complete task-I added PERF_COUNT to the plugin.cpp
And To complete the task-2 I have added get_property_params()
# Tickets:
#23075, CVS-130236
### Details:
- *meta-plugins are tested with dedicated test binaries*
- *Remove CPU/GPU/TEMPLATE from meta-plugin revalidate list*
- *Keep GPU in AUTO-BATCH revalidate list since AUTO-BATCH is enabled by
default for GPU throughput mode*
- ***revalidate**: Includes the test cases which can be impacted by the
changes in this component*
- ***build**: Includes the dependencies for the test cases of this
component*
### Tickets:
- *CVS-129316, CVS-129317, CVS-129319*
---------
Signed-off-by: Chen Peter <peter.chen@intel.com>
### Details:
- *In FuseLoops pass, update the state of current_loop_info after loop
fusion, so that the state of port connections will be updated and some
other loops may fuse to the current loop.*
### Tickets:
- *[CVS-141023](https://jira.devtools.intel.com/browse/CVS-141023)*
2024-05-14 04:01:30 +00:00
1884 changed files with 71708 additions and 20239 deletions
message:Thank you for looking into this issue! Please let us know if you have any questions or require any help.
issueCurrentlyAssignedMessage:Thanks for being interested in this issue. It looks like this ticket is already assigned to a contributor. Please communicate with the assigned contributor to confirm the status of the issue.
@ -46,4 +46,12 @@ Intel Global Human Right Principles
Intel is committed to respecting human rights and avoiding causing or contributing to adverse
impacts on human rights. See `Intel's Global Human Rights Principles <https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf>`__.
Intel's products and software are intended only to be used in applications that do not cause or