### Details:
- By PR(https://github.com/openvinotoolkit/openvino/pull/22245),
`clamp_fp16_output` opt pass was moved to ngraph
- Because nodes such as eltwise(`Add`, `Subtract`, `Multiply`, `Divide`)
that were fused into target node `gemm` are not supported in pattern,
corresponding pattern was extended for this purpose
### Tickets:
- 135060
### Details:
- Fuse `broadcast` and `reshape` layers into `gemm` layer for LLM's 2nd
latency optimization
- before : [`broadcast`] --> [`reshape`] --> `gemm`
- after : `gemm`
- `gemm` is extended to have `input0_target_shape`,
`input1_target_shape`, `input0_output_pattern` and
`input1_output_pattern` from `broadcast` and `reshape` layers
### Tickets:
- 128343
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
### Details:
- *Comparation function was changed to compare tensors based on element
comparation*
- *`std::abs(ref_value - plugin_value) <= abs_threshold + rel_threshold
* ref_value`*
- *`abs_threshold ` =
std::max(std::numeric_limits::eps<plugin_element_type>(),
std::numeric_limits::eps<ref_element_type>())*
- *`ref_threshold = eps_by_expected_type()`, which is based on half `bit
length of mantissa`*
### Tickets:
- [CVS-133173](https://jira.devtools.intel.com/browse/CVS-133173)
- [CVS-135540](https://jira.devtools.intel.com/browse/CVS-135540)
---------
Co-authored-by: sbalandi <sofya.balandina@intel.com>
### Details:
Limit the Concat layer to have maximum 3 children. The third one is
allowed to be a ShapeOf op only (to support Mixtral).
### Tickets:
- 135375
### Details:
- Adds buffer reset to 0 in `count_nonzero` impl in case of empty input
tensor as currently we may try to allocate random amount of memory in
subsequent `gather_nonzero` call
### Details:
- *Enable new property model_distribution_policy for CPU inference*
-- *Add C++ interface and test cases*
-- *Add Python interface and test cases*
### Tickets:
- *CVS-127844*
### Details:
- Since #22726 gemm is derived from multi-stage impl which had memory
reuse flag enforced to false for all sub-classes.
- This patch enables memory reuse back for gemm kernel to reduce memory
consumption.
### Tickets:
- *135361*
### Details:
- *enable concat nspc layout inplace for channel only cases, with these
concat node use inplace impl, urlnet model gain performance benefits,
and this(intermediate concat node is nspc layout but actually is one
dimension) could be common case especially for models with 1D input*
### Tickets:
- *130282*
### Details:
- *Element by element read is the bottle-neck in gemm_tiled kernel.
Enable block-read when N and K size are aligned 4byte with N and K are
leftover*.
- *Increasing tile_n_size has performance improvement when m_size and
n_size are not shallow and n_size is aligned at 32.*
- *Add GEMM_TILE_M/N/K/SIMD environment variables for convenience.*
### Tickets:
- *134279*
---------
Signed-off-by: hyunback <hyunback.kim@intel.com>
### Details:
- *Fix the issue which second infer with updated shape in dynamic loop
doesn't update sliced layout.*
- *Fix the issue that the optimized reshape doesn't reinterpret output
memory in update_output_layout()*
### Tickets:
- *122739*
- *131544*
### Details:
- Introduce new low precision types `u2`, `u3`, `u6`.
- Introduce `ov::element::Iterator` for low precision types like `u1,
u2, u3, u4, i4, u6`:
- Gives pointer like access to low precision values in Tensor,
containers etc.
- Can be used by STL algorithms to access data in unified algorithms for
data manipulation.
- Can be used in Constant, Convert operators to replace duplicate
implementations for accessing low precision data (bin-size reduction).
- Can be used for operator reference implementation or plugin if there
is no hardware specific solution.
### Tickets:
- [CVS-126998](https://jira.devtools.intel.com/browse/CVS-126998)
- Part of
[CVS-128024](https://jira.devtools.intel.com/browse/CVS-128024)
### Details:
NopStridedSlice in MOC caused accuracy issues on GPU because of
incorrect dynamic paddings handling in Unsqueeze operation.
As a temporary solution, we decided to move NopStridedSlice to Common
transformations and disable it for GPU:
### Tickets:
- *CVS-135242*
Co-authored-by: Pavel Durandin <pavel.durandin@intel.com>
### Details:
- *MergeTransposeReorder: reverse operations order support*
- *MergeTransposeReorder: reshape between Transpose and Reorder is
supported with the following limitations:*
- Direct order (Transpose -> Reshape -> Reorder): reshape which
separates one of the dimension on 2 consecutive ones
- reverse order: (Reorder -> Reshape -> Transpose): reshape which fuses
2 consecutive dimensions into one
- Common code, which performs the fusion, is moved to a separate
function.
### Tickets:
- *CVS-113363*
### Details:
- *change the default case resolution policy so that it resolves cyclic
inplace to downstream instead of upstream for outputs.*
- *this is a more optimal decision as more cases are going to reuse
infer request's memory in PullOutputData.*
### Tickets:
- *116823*
- *120566*
This reopens legacy PR
https://github.com/openvinotoolkit/openvino/pull/19684 as more models
requires this feature.
---------
Co-authored-by: Maksim Kutakov <maxim.kutakov@gmail.com>
### Details:
- fixed `GatherCompressed` not to reshape to 2d if there is no reshape
layer
- fixed a bug in `gemm_inst::transform_output_layout`
- updated to propagate dependent events to its users if it is in
`shape-of_subgraph`
### Details:
- *Fixed problem with precision in RNN and TensorIterator layers*
- *Corrected shape calculation of current layers in ACL Scheduler*
### Tickets:
- CVS-123900
- CVS-134520
New PR due to merge
conflicts(https://github.com/openvinotoolkit/openvino/pull/22917)
Updated the copyright year from 2018-2023 to 2018-2024 in all openvino
files
Ref. to script: CVS-101144
Command used:
```bash
git grep -lz '2018-2023 Intel Corporation' | xargs -0 sed -i '' -e 's/2018-2023 Intel Corporation/2018-2024 Intel Corporation/g'
```
### Details:
- *The pass `isSuitableChildForFusingMatMul` first checked that the
`node` has constant inputs, without checking what the `node` type is and
what parent is. It leaded to infinity checks for constant path for the
node which cannot be even fused to `MatMul`. Thus, no make sense to
check that parent `MatMul` has `constant` path.*
- *The PR refactored check `isSuitableChildForFusingMatMul`*:
- *Firstly, we check for possible fusion `MatMul (FC)` with `Bias` and
`DQScales` based on node type;*
- *Secondly, we add specific checks from `MatMul::CanFuse` for binary
`Eltwise` and `FQ`. Moreover, this code has been updated for the first
time since Snippets support!*
- *Thirdly, we check that the node is supported for fusion with `MatMul`
(is `Eltwise` op at least) and only after that (!) we check for constant
input paths for inputs that are not `MatMul` path to avoid endless
searches.*
- *Added additional `NodeFusingType` for `FullyConnected` to make the
code clearer: now we can init `channelAxis` correctly and separate
additional checks in `MatMul::CanFuse`*
### Tickets:
- *CVS-134292*
### TODO:
- [x] *Performance Validation (Passed - the report is attached to the
ticket)*
### Details:
- Currently, `amx_tile_configure` in brgemm emitter is called for each
inference whereas reconfiguration can be called only when input shapes
are differ from previous inference. This PR introduces `amx_tile_config`
structure with the latest used dimensions. The structure is passed as a
runtime argument in compiled kernel. Brgemm emitter gets the structure
and checks if the current shapes are the same as previous. If they are
the same, `amx_tile_configure` is skipped. Otherwise, the
reconfiguration is called, and `amx_tile_config` is updated with new
shapes.
### Tickets:
- *CVS-125676*
Oleg Pipikin