* Handle Reshape's special zero in SimplifySecondInputOfReshape
SimplifySecondInputOfReshape detects ShapeOf->Gather->Concat
subgraphs on Reshape's second input and replaces ShapeOf->Gather
with a Constant with zero(s). Currently it works only with Reshapes
that have special_zero set to true, but it can work for Reshapes
with special_zero == false if non-Gather inputs to Concat are Constants
and don't contain any zero.
Ticket: CVS-123434
* fix no default output
Co-authored-by: Mateusz Tabaka <mateusz.tabaka@intel.com>
* Add static shape adapter
- Adapters holds CPU dimension which can be reference to it or vector
- Add ov::optional for holding optional result from shape inference
- Add new `infer` function in `IStaticShapeInfer`
* Temporary support of StaticShape
* Minor corrections in ShapeInferenceTA
* Migrate shape_infer to new interface version
* Replace StaticShape by adapter implementation
* Replace IShapeInferCommon by IStaticShapeInfer
* Correct code formatting
* Fix build issues
* NodeValidationFailure::create for StaticShapeRef
* Review shape inference for reshape operator
- review shape_infer implementation
- add more unit test for static and dynamic shapes
* Fix build issues
* Correct minus one dim calculation
* Fix build issues on windows
* Improve resolving special minus one
* Use NODE_SHAPE_INFER_CHECK
* Update product in/out calculations
* Temporary add ngraph header to solve build issue
* Correct minus one dim calc when static part same
* Add check for scalar input
* Remove debug message
* Fix `minus one` dynamic dimension calculation
* Fix `minus one` dynamic dimension calculation
* Fix merge issues in reshape
Minor refactor reshape evaluate
* Don't pass input label on minus one pattern
when input dimension will be modified.
* Add Rotation support to primitive and kernel
* Add unit tests
* Add transformation for NMSRotated
* add single-layer tests
* Fix: angle value for the same box may have its sign changed several times passing through iterations of batch and class loops.
* fix review comments
Transformation fuses Transpose on first or second MatMul's input
and sets MatMul's transpose_a/transpose_b accordingly.
TransposeMatMul is already part of SmartReshape, but it can be added
to MOCTransformations as well so native models that are don't use reshape
can benefit from that.
Ticket: CVS-118908
Current implementation tries to leverage branchless approach, but it's not correct
if scale is 0. In that case - zero point can can become inf or nan and multiplication
by 0 doesn't change its value. That causes another issue - infinite or NaN zero point
cannot be optimized out later.
Ticket: CVS-122931
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
* [TF FE] Provide full support of TF1 Control flow and TensorArray ops
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add missed header for TensorArrayV3 op
* Temporarily disable GRU cell fusion
* Update src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp
* Fix a case when element_shape for TensorArrayV3
* Fix translator for TensorArrayCloseV3
* Update summarize graph with TensorArrayCloseV3
* Add layer tests for TensorArrayScatterV3, Close, Size, Array
* Fix output shape for Merge node
* Remove unused variable
* Fix translator for TensorArrayConcatV3
* Fix translator for TensorArrayConcatV3
* Add layer tests for TensorArrayWriteV3, Gather, and Concat
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add translator for GatherTree
* Fix TF FE unit-test for GatherTree
* Fix GatherTree translator
* Fix GatherTree translator to handle 1d end_token
* Fix undeclared parameter issue
* Fix GatherTree unit-test
* Add TensorArrayV3Replacer transformation
* Temporarily disable dangling transformation
* Recover RemoveMultiSubGraphOpDanglingParamsResults transformation
* Recover GRUCellFusion transformation
* Simplify check for GRUCellFusion transformation
* Use proper name for unit-tests
* Simplify translator for TensorArrayWriteV3
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix RemoveMultiSubgraphOpDanglingParamsResults transformation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Additional fix for remove_multi_subgraph_op_dangling_params
* Make static TI run a dynamic subgraph
* Dedicated SL test
* Change condition to respect stat shapes
* Adjust test to cover the code path properly
* Recover fallback for still failing case GNMT
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
* Reference implementation for u4 constant compression from pytorch model based on bitwise ops pattern
* Fixed order of 4-bit halfs in byte
* Switched PyTorch FE to dev mode: in case if model cannot be fully converted, give partially converted model with PTFrameworkNode's with a printed warning (normally would raise an exception in case).
* Moved u4 compression to utils_quantize. Implemented not-interleaved version of u4 compression
* Removed debug output
* Added aten::matmul to the list of exceptions in may_produce_alias as a workaround for gptq models
* Added patching for gptq models applied automatically in convert_model
* WA for an inssue with u4 with earlier convert to fp16
* U4 blocked repacking for gptq patched model layout
* Deleted obsolete u4 re-packing based on aten::cat. Fixed the resulting u4 constant shape. Removed debug output.
* Revert "Switched PyTorch FE to dev mode: in case if model cannot be fully converted, give partially converted model with PTFrameworkNode's with a printed warning (normally would raise an exception in case)."
This reverts commit 0ef1455e70.
* Update src/frontends/pytorch/src/op/cat.cpp
* Check mask and shift values in u4 pattern. deque -> OutputVector for u4_compression_stack
* Convert to a given floating type instead of half in gptq patching. Better structured code.
* Code style fix
* Removed deque include
* Code style fixes
* Trailing space removed
* Fixed patched_forward and ts_decoder after unvalidated commits.
* Swap nibbles in u4/i4
* Better exception handling around jit.trace and gptq.patch_model
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
* Code style
* Revers int4 byte order
* Fixed core tests
* Fixed unguarded dynamic_cast result
Co-authored-by: Evgenya Nugmanova <eva.my.link@gmail.com>
* Fixed transformation tests
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* Prevent patching of non-gptq models
* Removed extra calling of quantized weights decompression patterns
* Better detection of supported AutoGPTQ models + more diagnostics
* Accurate diagnostics in case when aten::stack has multiple axes
---------
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: Evgenya Nugmanova <eva.my.link@gmail.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* add new operations as unary
* get unary as input(0) instead of iterating pattern map
* add CumSum + unit tests
* add Tile + unit tests
* add tile
* fix ts_tile
* code review fix: use ADD_MATCHER
* fix bug CI tests
* Optimize CompressQuantizeWeights transformation
- remove CoordinateTransform usage from FakeQuantize reference implementation
- move ZeroPointOptimizer functionality inside CompressQuantizeWeights
- compute scale and zero point in the same loop
Ticket: CVS-119273
* review comments
* clang format
* fix comments
* Do not normalize negative indices for Gather v8
* code style fix
* added transformation test with accuracy check for Gather-v8
* removed GatherNegativeConstIndicesNormalize transformation at all
* ConvertGather8ToGather7 conversion: added more checks
* Introduced shared Gather8withIndicesDataLayerTest: added CPU, GPU instances
* code style fix
* small fix
* review fixes
* do negative indices normalization if possible
* code style fix
* refactor cpu test instances
* code style fix
* [transformations] WeightsDequantizeToFakeQuantize: Extend pattern matching with the case when both Subtract inputs are Convert
* [transformations] WeightsDequantizeToFakeQuantize: Added new tests to cover the extention added to pattern match
* Fix review comments
* Add EnableShapeOfConstantFolding transformation
Transpose sinking (that is used in TF frontend) disables ShapeOf constant folding
which prevents some optimizations further in the pipeline.
This patch introduces EnableShapeOfConstantFolding that removes DisableConstantFolding
from ShapeOf nodes.
Ticket: CVS-118890
* add description
* review comments
* headers
* Symbolic shape inference and graph optimizations
- Prepares a place in CommonOptimizations pipeline for symbolic optimizations
- Introduces symbolic propagation and symbolic optimizations for ChainedMaximum, NopBroadcast and shape sub-graph optimization
- Introduces utility runtime info for TableOfEquivalence passing and disabling of value invalidation during shape inference
* Executes NgramFusion in a symbolic environment. Relaxes Ngram fusion pattern utilizing symbolic knowledge
* Remove debug model visualization
* rt_info copying to new Add operation
* Fix visualization and place validation in nicer place in symbolic transformation
* Fix Slice operation not to propagate labels if input and output dimension is fully dynamic
* Covering Vladislav comments
* Replace value invalidation followed by validation to revalidation since it does the same thing
* Adding back invalidation of cached values to Symbolic Propagation pass
* Fix StridedSlice label propagation. Code style
* Update src/common/transformations/tests/symbolic_transformations/nop_broadcast.cpp
* Handle negative values in GroupedSliceToVSplitOptimization
CVS-118897
* change the way of getting slice inputs
* clamp value
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
* Move BroadcastTransition to MOC
Broadcast that could be eliminated by BroadcastElementwiseFusion are moved down the graph
(by BroadcastTransition registered in the plugins). That prevents BroadcastElementwiseFusion
to eliminate them.
Ticket: CVS-118899
* dont count const layers
* remove virtual inheritance
* Restored opset1::Reshape label peropagation for -1 special value
* Lets opset1::Reshape keep same shape infer. Makes FindBatch transformation keep labels in output shapes of Result node
* uses Parameter from correct namespace
Ivan Tikhonov