Commit Graph

2782 Commits

Author SHA1 Message Date
Maksim Kutakov 5203d3483b
[CPU] Dynamic shapes support for memory states with zero copy (#20368) 2023-11-20 09:58:38 +04:00
Tingqian Li a61a9be495
[CPU] Add general pattern serialization & matcher & RoPE (#20729) 2023-11-20 09:50:50 +04:00
Min, Byungil 081f5aad8a
[GPU] Fix perf regression from dynamic tensor iterator (#21065)
+ Not to use reference::split. It is very slow.

Signed-off-by: Min, Byungil <byungil.min@intel.com>
2023-11-20 10:03:39 +09:00
Min, Byungil 8181f30a67
[GPU] Fix accuracy issue of blocked format batching on iGPU (#20890)
* [GPU] Fix accuracy issue for blocked format batching on iGPU

+ Bugfix GET_INDEX for batching of b_fs_zyx_fsv16 of target model
+ Updated fetch_data.cl
+ Minor fix of error message

Signed-off-by: Min, Byungil <byungil.min@intel.com>
2023-11-20 09:46:37 +09:00
Oleg Pipikin a9dffa8d70
Refactor ngraph builders (#20998)
* Remove makeInputLayer builders

* Refactor makeBroadcast builder

* Deprecated makeCTC builders

* Fix for broadcast builder

* Remove and deprecate makeSplit builder

* fix

* Fix
2023-11-17 16:10:19 +01:00
Oleg Pipikin 71af0eef38
Refactor ngraph builders 4 (#21100)
* Remove and deprecate some ngraph builders
2023-11-17 14:51:56 +00:00
Luo Cheng e4311aae46
[CPU] Fix ScaledDotProductAttention build failure on ubuntu18 (#21123) 2023-11-17 10:14:01 +00:00
Paul Youngsoo Ahn 8b5b7a627b
[GPU] Fix loop issues (#21091)
* [GPU] Fix loop issues
* if loop does not run inner body, return zero dimension layout for dynamic dimension layout

* Support dynamic sliced input in inner body in creaet concate mapping

* Modify update_shape to call reset_shape_changed() when inputs of loop are not changed
* Add unit test

* Follow-up codereview
2023-11-17 00:16:58 -08:00
Aleksandr Voron 60730f61a4
[CPU][ARM] Call to_string() method of ov::element::Type in ACL Convert executor (#21136) 2023-11-17 07:02:13 +00:00
Taylor Yeonbok Lee cc463a7757
[GPU] Skip empty subgraph of If operator (#21087)
* Skip empty subgraph of If operator

* Added test
Applied review comment

* Fix for the following case:
1) body network output has only constant input
2) body network output itself is the constant
2023-11-17 09:39:03 +04:00
Oleg Pipikin 819feeb56d
Refactor NMS shared tests (#20897)
* Refactor MatrixNmsLayerTest

* Refactor MulticlassNmsLayerTest

* Refactor NmsLayerTest

* Apply comments

* Apply comments
2023-11-16 13:57:56 +00:00
Maksim Kutakov 315c916ced
[CPU] Postpone throwing error for dnnl memory until it's requested (#20082)
* Postpone throwing excp when dnnl memory has error

* Change test name

* Enable tests

* Fix the exception macro

* Fix after merge
2023-11-16 09:38:04 +01:00
Wang Wangwang 1693ef17f0
[HETERO] Fix split subgraphs with parameter of different affinity (#21026) 2023-11-16 12:37:00 +04:00
Oleg Pipikin 76459a316f
Refactor ngraph builders 5 (#21101)
* Remove makeGatherElements builder

* Remove makeTile builder

* Remove makeNormalizeL2 builder

* Deprecate makeNms buider

* Remove makeOneHot builder

* Remove makeEinsum
2023-11-16 10:29:48 +04:00
Xuejun Zhai 83ef7d7ff9
[Func Test] Upgrade func test of QuantMatMulTest related to 2.0 (#21103)
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2023-11-16 06:29:10 +00:00
Sergey Shlyapnikov af34891750
[GPU] Add more flexible check if user's buffer is USM Host and can be reused (for iGPUs only) (#21018) 2023-11-16 09:46:10 +04:00
Luo Cheng 0061d0b286
[CPU] ScaledDotProductAttention op support (#20506) 2023-11-16 05:30:30 +00:00
Yaroslav Torziuk f45dccad15
[GPU] Multinomial-13 (#20616)
* Update random_uniform in multinomial reference implementation to generate values of the same type as the first input

* Add kernel selector for multinomial

* Add multinomial primitive and its instantiation

* Add calc_outut_layout implementation for slice_inst

* Multinomial builder

* Add multinomial kernel

* Add multinomial single layer test class

* Add GPU multinomial tests instantiation
2023-11-15 16:20:43 -08:00
Xuejun Zhai 8eb529ceac
[Func Test] Upgrade func test of QuantGroupConvBackpropDataLayerTest related to 2.0 (#21094)
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2023-11-15 18:14:25 +00:00
Xuejun Zhai 6755a7e3a8
[Func Test] Upgrade func test of QuantGroupConvLayerTest related to 2.0 (#21092)
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2023-11-15 17:28:07 +00:00
Xuejun Zhai 00d0c0ad0d
[Func Test] Upgrade func test of QuantConvBackpropDataLayerTest related to 2.0 (#21082)
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2023-11-15 14:12:33 +01:00
Alexandra Sidorova aef989f585
[CPU][Tests] Fixed linking of specific tests (#21084) 2023-11-15 13:35:16 +01:00
Haiqi Pan 79c9f4ad44
migrate executable_network test case to compiled_model (#19541)
* add get property test case

* add CanCreateTwoExeNetworksAndCheckFunction and pluginDoesNotChangeOriginalNetwork

* tmp

* add canSetInputPrecisionForNetwork and canSetOutputPrecisionForNetwork

* fix code style

* tmp

* Revert "tmp"

This reverts commit ff3f8d56d5.

* tmp

* add CanCompileModelWithEmptyProperties

* remove comment

* add infer request

* add CanLoadNetworkWithCustomLocale

* add CanLoadNetworkWithCustomLocale

* fix error

* remove case in io_tensor.cpp

* fix Unsupported metric key: OPTIMAL_BATCH_SIZE

* Update src/tests/functional/plugin/conformance/test_runner/api_conformance_runner/include/api_conformance_helpers.hpp

Co-authored-by: Chen Peter <peter.chen@intel.com>

* fix batch size

* remove useless code

* remove failed case CanCompileModelWithEmptyProperties, CanLoadNetworkWithCustomLocale and LoadNetworkWithBigDeviceIDThrows to test

* Revert "remove failed case CanCompileModelWithEmptyProperties, CanLoadNetworkWithCustomLocale and LoadNetworkWithBigDeviceIDThrows to test"

This reverts commit 1317d0773c.

* Revert "remove useless code"

This reverts commit b3dd0ffaab.

* Revert "fix batch size"

This reverts commit 2afd673cff.

* Revert "Update src/tests/functional/plugin/conformance/test_runner/api_conformance_runner/include/api_conformance_helpers.hpp"

This reverts commit 9d6030952f.

* Revert "fix Unsupported metric key: OPTIMAL_BATCH_SIZE"

This reverts commit 2de26547ea.

* try to add optimal_batch_size in cpu plugin

* return model when optimal_batch_size not found in apply_auto_batching

* revert cpu plugin

* gna cannot support some model

* update test case name

* skip CanCreateTwoCompiledModelsAndCheckRuntimeModel in gna

* Update src/inference/src/dev/core_impl.cpp

Co-authored-by: Chen Peter <peter.chen@intel.com>

* Update src/inference/src/dev/core_impl.cpp

Co-authored-by: Chen Peter <peter.chen@intel.com>

* replace deviceName.substr(pos + 1) with deviceNameWithoutBatch

* fix bug

* Update src/inference/src/dev/core_impl.cpp

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* Update src/tests/functional/plugin/shared/include/behavior/compiled_model/compiled_model_base.hpp

Co-authored-by: River Li <river.li@intel.com>

* Update src/tests/functional/plugin/shared/include/behavior/compiled_model/compiled_model_base.hpp

Co-authored-by: River Li <river.li@intel.com>

* Update src/plugins/intel_gna/tests/functional/shared_tests_instances/skip_tests_config.cpp

Co-authored-by: River Li <river.li@intel.com>

* Update src/plugins/intel_gna/tests/functional/shared_tests_instances/skip_tests_config.cpp

Co-authored-by: River Li <river.li@intel.com>

---------

Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: River Li <river.li@intel.com>
2023-11-15 13:27:42 +01:00
Oleg Pipikin c7765c91f5
Refactor ExperimentalDetectron tests (#20846)
* Refactor ExperimentalDetectronDetectionOutputLayerTest

* Refactor ExperimentalDetectronGenerateProposalsSingleImageLayerTest

* Refactor ExperimentalDetectronPriorGridGeneratorLayerTest

* ExperimentalDetectronROIFeatureExtractorLayerTest

* Refactor ExperimentalDetectronTopKROIsLayerTest

* Apply comments

* Aplly comments
2023-11-15 13:16:37 +01:00
Xuejun Zhai 66a8600564
[CPU Plugin 2.0] replace ie::precision with ov::element (#21041)
* [CPU Plugin 2.0] replace ie::precision with ov::element

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

* [CPU Plugin 2.0] fix MatmulWeightsDecompression.CompareWithRefs test error

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

* [CPU Plugin 2.0] fix compiler error

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

* [CPU Plugin 2.0] fix compiler error

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

* [CPU Plugin 2.0] fix review comment

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

---------

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2023-11-15 12:32:43 +01:00
Wilson Seok 0180078c70
[GPU] Fix permute reorder fusing bfzyx (#20896)
* fix accuracy issue of permute reorder fusing

* add func test

* remove debug code

* use unit test

* simplify condition

* add additionall condition
2023-11-15 09:33:12 +00:00
Sungeun Kim 5365bfe0a7
[GPU] remove setting threshold manually for dynamic 1d conv (#21028) 2023-11-15 16:54:22 +09:00
Oleg Pipikin 534ec2e9ab
Refactor ngraph builders 3 (#21077)
* Remove and deprecate makeSqueezeUnsqueeze

* Remove and deprecate  makeMinMax makeProposal

* Remove makeCumSum makeSelect builders

* Remove makeSpaceToDepth builder

* Remove makeShuffleChannels builder

* Remove makeMatMul builder
2023-11-15 10:41:45 +04:00
Sergey Shlyapnikov c360de0f07
[GPU] Optimize Condition operation by better integrating its subnetwork primitives with the main network (#21062)
* [GPU] Optimize Condition operation by better integrating its subnetwork primitives with the main network

* Reuse main network's stream in sub-networks
2023-11-15 08:47:36 +04:00
Wang, Yang 859319e2a5
[AUTO] Fix segment fault appeared in the compiled model.get_property() of AUTO (#20777)
* disable CPU_HELP context if it released.

* Enable test case.

* Updated.

* Updated.

* Updated.
2023-11-15 10:20:35 +08:00
Oleksii Khovan 7c027fe19a
[GPU] Fix ShapeOf primitive (#20983)
* Determine rank via input layout instead of primitive's member

* Remove unused primitive's member

* fix shape_of() usage - remove unused parameter
2023-11-14 11:09:52 -08:00
Paul Youngsoo Ahn da2a886477
[GPU] Fix acc issue for LSTMSequence w/ -1 seq_length (#21054)
* [GPU] Fix acc issue for LSTMSequence w/ -1 seq_length
* add output port for multiple outputs of node
* add functional test for lstm_sequence

* Fix CI test failures
2023-11-14 10:47:03 -08:00
River Li a720b43041
[CPU] CPU plugin API 2.0 migration for IE_THROW and Exception (#21005)
* Replace IE_THROW with OPENVINO_THROW

   1. IE_THROW() is replaced by OPENVINO_THROW
   2. IE_THROW(NOTIMPLEMENT) is replaced by OPENVINO_THROW_NOT_IMPLEMENT
   3. IE_ASSERT is replaced by OPENVINO_ASSERT

* Minor update
2023-11-14 21:50:31 +04:00
Vladislav Golubev b43b9f9d50
[CPU] Weights decompression support for hybrid models (#20973) 2023-11-14 13:27:42 +04:00
Zhang Yi 7cb3bf5610
[CPU] Enable bf16 acdb layout for transpose (#21030) 2023-11-14 10:24:54 +04:00
River Li 89494ab412
[CPU] CPU plugin API 2.0 migration for ngraph namespace (#21013)
* [CPU] cleanup misc IE and ngraph

1. InferenceEngine::SizeVector
2. InferenceEngine::parallel_for
3. All ngraph namespace except ngraph::op

* [CPU] clean ngraph opset namespace and header files

* Remove unused header files

1. ngraph/ngraph.hpp
2. ngraph/ops.hpp
3. blob_factory.hpp
3. using namespace ngraph

* Cleanup ie_parallel

* Fix compile error
2023-11-14 00:37:27 +04:00
Maksim Kutakov 75acdac5f9
Extend the existing IO binding subgraph tests (#20987) 2023-11-13 15:22:41 +01:00
Xuejun Zhai 4563f71ffc
[Func Test] remove ngraph::helpers::QuantizationGranularity, replaced with ov::test::utils::QuantizationGranularity (#20985)
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
2023-11-13 13:29:36 +00:00
River Li 5dd317c733
[CPU] cleanup misc IE and ngraph (#21007)
1. InferenceEngine::SizeVector
2. InferenceEngine::parallel_for
3. All ngraph namespace except ngraph::op
2023-11-13 17:06:59 +04:00
hyunback kim 03e48f3c17
[GPU] Fix LCM notebook failure issue for int8 model. (#20944)
Fix to use partial_shape in dynamic case. Not get_dims().

Signed-off-by: hyunback <hyunback.kim@intel.com>
2023-11-13 20:39:02 +09:00
Maciej Smyk 5a04359200
[DOCS] Updating links for master 2023-11-13 12:23:23 +01:00
Maksim Kutakov fb3751717f
[Inference] Return state ptr by value (#21011)
* Return state ptr by value

* Fix mock class
2023-11-13 14:24:54 +04:00
Vitaliy Urusovskij 1a5b0b70f9
Remove `castOps2Nodes` and `convert2OutputVector` (#21015) 2023-11-13 13:29:53 +04:00
Pawel Raasz c451a94572
Fix compile warning regarding ABI changes (#20990)
and using std::true_type/false_type
2023-11-13 07:44:54 +01:00
River Li fef0df6ee1
[CPU] Add deprecation message to extension API (#20970) 2023-11-13 10:37:35 +04:00
Sergey Shlyapnikov 306137f86b
[GPU] Make ShapePredictor instance unique for each InferRequest instead of the cldnn::network (#21019) 2023-11-13 08:52:07 +04:00
Paul Youngsoo Ahn 51da30b48d
Support LSTMSequence with -1 sequence length (#20935)
* [GPU] Support LSTMSequence w/ -1 seq_length

Co-authored-by:Taylor Yeonbok Lee <taylor.lee@intel.com>
Co-authored-by:Andrew Park <andrew.park@intel.com>

* Fix GetInputInfo to retrieve input pid from LSTMCell

* LSTMCell use ov::PartialShape instead of cldnn::tensor
* implement lstm_elt_inst::calc_output_layouts
* implement lstm_elt_impl::static_canonicalize_shapes

* Add functional tests

* Fix unit test failure

---------

Co-authored-by: Andrew Park <andrew.park@intel.com>
2023-11-11 00:06:08 -08:00
Sungeun Kim b1705e8bd3
[GPU] clean up for extend pad/stride/dilation (#20828)
* clean up for extend pad/stride/dilation
2023-11-09 20:48:34 +09:00
Sungeun Kim 9cc4c25e48
[GPU] print datashape of input for benchmark_app (#20943)
* print datashape of input for benchmark_app
2023-11-09 20:46:50 +09:00
Zhang Yi 09a45bceae
[CPU][MLAS]Apply lower bound protection for K stride (#20873) 2023-11-09 15:14:27 +04:00