Commit Graph

136 Commits

Author SHA1 Message Date
Gladilov, Gleb e61a594199
[IE][VPU]: Configuration options in VPU plugins refactoring (#3211)
* [IE]: Enables Abstract class -> Parameter conversion support

Parameter has templated constructor allowing to write code

```
Parameter p = i; // i of type int for example
```

This constructor uses SFINAE to resolve ambiguity with
move-constructor, so checks that argument is not of the same type.
In case it's not the same type it calls std::tuple constructors that
constructs an instance of argument type. In the following case:

```
Parameter p = static_cast<Parameter>(abstractRef);
// abstractRef is a reference to abstract class
```

We have a reference to some abstract class that defines explicit
cast operator to Parameter. In contrast with expectations,
instead of cast operator, Parameter constructor is instantiated,
since template type deduction for Parameter constructor didn't fail
(abstract class has not the same type as Parameter). Instantiation
of tuple constructor used inside failed: it's impossible to create an
instance of abstract class what lead to compile-time error. To resolve
the issue additional condition introduced to check if argument type is
abstract.

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE]: Enables PrintTo method for Parameter and tests on it

Inference Engine API for configuration options uses Parameter
type as a return type of GetConfig method. Parameter is intended
to store object associated with configuration option.
To support objects of different types its constructor is templated.
Parameter overloads cast operators which are templated
as well. Both constructor and cast operators are implicit, which
makes it possible to implicitly convert any type to Parameter
and vice versa.

Since Parameter is a part of Inference Engine configuration API it's
essential google tests on API contain Parameter as tests parameter.
For each test parameter Google Test framework tries to print it to
an output stream. For that purpose, Google Test checks if test
parameter has output stream operator or PrintTo method. If not, it
checks if it could be implicitly converted to integral type and,
in this case, prints it as a long integer.

InferenceEngine::Parameter does not define output stream operator,
but could be implicitly converted to an integer, according cast
operators mentioned above, so Google Test tries to convert to
integer. Since Parameter not necessarily contains integer, this
conversion throws an exception of type mismatch, which makes it
impossible to use Parameter in Google Test framework as is.

In order to resolve that issue Parameter should define either
output stream operator or PrintTo method. If Parameter will
define output stream operator it will make it possible to compile
streaming almost any object to an output stream. The reason for it
is C++ checks if object could be implicitly converted to other type
which defines output stream operator, if objects itself doesn't do it
(e.g. `stream << "text";` calls std::string::operator<<, since
char const* is implicitly convertible to std::string).

Taking this into consideration the only way to support Parameter in
Google Test without breaking backward compatibility is define PrintTo
method.

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE]: Fixes ill-formed extending std names

According to the standard:

The behavior of a C++ program is undefined if
it adds declarations or definitions to namespace
std or to a namespace within namespace std unless
otherwise specified. A program may add a template
specialization for any standard library template
to namespace std only if the declaration depends
on a user-defined type and the specialization meets
the standard library requirements for the original
template and is not explicitly prohibited.

As as an unexpected result, InferenceEngine::Parameter
that contains std::vector<std::string> can be printed
via PrintTo. In that case operator<< version from
Inference Engine is picked up.

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Moves CompilationConfig out of GT header

Keeping config in a separate header simplifies migration
to new interface.

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Removes Platform enum

Since there is enum from MVNC for the same purpose
there is no need in Platform anyway

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Introduces containers utility header

Contains some helpers to work with C++ maps

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Introduces new configuration API

The main ideas are separate option-specific logic
from common container, automate logic processing
public vs private, deprecated, compile-time vs
runtime-time options and remove code duplication.

Since IE defines configuration API using std::string
and Parameter, options have to provide ways to be
represented as Parameter (ex.: GetConfig is called)
and be defined using std::string (ex.: SetConfig is
called). Keeping information about actual key value
is useful for error reporting.

New API fallbacks to previous version in case of
unsupported options are requested. This way migration
becomes iterative and looks simpler.

Options containers are related to corresponding components:
CompilationConfig (name to be changed) - GraphTransformer,
PluginConfiguration - base class for plugins configurations,
MyriadConfiguration - Myriad plugin configuration,
HDDLConfiguration - HDDL plugin configuration (to be
introduced in a separate request)

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Replaces CompilationConfig with PluginConfiguration

Some of options to be refactored are stored inside CompilationConfig.
CompilationConfig is passed to graph transformer as a compiler to be
processed. Since it's separate data structure and migration process
is iterative we need a mechanism to provide some of compilation
options from new interface and some from old. It cannot be done via
plugin specific class (MyriadConfiguration), since there are others
plugins as graph transformer users. Plugin specific class
(MyriadConfiguration) already inherits from old version (MyriadConfig),
which in turn inherits from ParsedConfig containing CompilationConfig.

To resolve the issue MyriadConfig inheritance from ParsedConfig is made
virtual to make it possible for PluginConfiguration to virtually inherit
from ParsedConfig as well an so make PluginConfiguration data structure
for configuration options for graph transformer. Since
PluginConfiguration is base class of MyriadConfiguration as well as
MyriadConfig and inheritance is virtual plugin just casts its specific
configuration to base one passing to graph transformer.

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Enables new tests on configuration API

* Enables following new shared tests on configuration API
  * Can load network with empty configuration
  * Check default value for configuration option
  * Can load network with correct configuration
  * Check custom value for configuration option (set and compare)
  * Check public configuration options are visible through API
  * Check private configuration options are invisible through API
  * Check GetConfig throws an exception on incorrect key
* Refactors myriad plugin instantiations for shared tests

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Extracts LogLevel enum to a separate header

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Refactors LOG_LEVEL configuration option

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Refactors COPY_OPTIMIZATION configuration option

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Fixes behavior tests build

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Updates tests on new exception class

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Removes unused variable from mvnc test

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>

* [IE][VPU]: Removes SizeVector streaming call

New assertion macro IE_ASSERT implementation uses
output streaming operator with r-value reference
argument as a stream. This prevents the compiler
from picking up overload from InferenceEngine::details,
since our version takes stream by non-const l-value
reference.

Since there is no simple solution to provide output
streaming operator overload for r-value references as
well and this call is just a message for assert in
test utilities, it was decided just to remove call
for now.

Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com>
2021-06-17 18:54:39 +03:00
Alexander Zhogov 17a899ed0c
Change version to 2022.1 2021-06-10 10:33:47 +03:00
Ilya Lavrenov 6e2d13937a
ImportNetwork with explicit device name only (#5689)
* Import with explicit name

* Fixed LoadHetero_MultiArchs tests

* Fixed MYRIAD tests on Windows

* Fixed compilation in tests

* Updated tesets

* Fixed test

* Removed useless lines

* Removed custom VPU tests, replaced with common ones

* Fixed Windows

* Reverted SKIP_IF_NOT_IMPLEMENTED macro
2021-06-09 10:09:25 +03:00
Vladimir Paramuzov aa4a18dda1
[IE CLDNN] Updated GPU device config (#6040) 2021-06-09 09:02:25 +03:00
Ivan Tikhonov c1608628d4
LowLatency v2 ngraph transformation (#5160)
* LowLatency 2.0: transformation and unit tests

* low latency 2.0: unit tests

* documentation and ngraph codestyle

* update CNN Interface of LowLatency transformation

* fix build on Windows

* fix build on Windows

* investigation of a failed build on Win OS

* ngraph codestyle

* fix build (werrors)

* New unit tests, refactoring

* update functional tests for Memory

* update LowLatency functional tests

* extend Memory tests to cover LowLatency v2 transformation

* clean up, code style

* fix unit tests

* update and fix unit tests, add feature to apply LLTv2 after LLTv1

* update docs, refactoring

* add several gna tests to skip config

* fix python api tests

* update python api, rename LowLatency_v2 to LowLatency2

* deprecate LowLatency v1

* Deprecate LowLatency v1 in IE

* fix wrong merge, codestyle

* resolve review comments

* fix python test

* update skip config

* apply online review notes, fix unit tests

* clean up, code style

* fix docs

* Use debug_messages instead of exceptions in llt v2

* fix unit tests

* Resolve review remarks
2021-06-07 15:13:41 +03:00
Ilya Lavrenov eff9f00320
Refactored ie_plugin_config.hpp (#5899) 2021-06-01 16:31:29 +03:00
Ilya Lavrenov a0ce93639d
Fixed klockwork issue (#5898) 2021-05-31 13:21:20 +03:00
Eunsoo Sheen d05c3c63b5
[IE CLDNN] Add an initial loop primitive for fixed length sequence (#4954) 2021-05-28 11:55:48 +03:00
Anton Pankratv d899606493
Merged plugin and exec network internal implementation (#4923)
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2021-05-27 13:33:09 +03:00
Maksim Shabunin 84392f33d1
ie_precision: added FloatTrait to fix VS2015 client compilation (#5810) 2021-05-26 01:04:46 +03:00
Anton Pankratv 4db8285a4b
Simplified throwing macro (#5753) 2021-05-26 00:06:38 +03:00
Ilya Lavrenov b7c5edc944
Deprecated all ICNNNetwork methods (#5488)
* Deprecated ICNNNetwork

* ICNNNetwork deprecation

* Fixed comments

* More suppressions for ICNNNetwork

* Fixed C API compilation

* Deprecated ICNNNetwork methods only

* [IE CLDNN] Fixed cpplint for clDNN with Ninja generator

* Fixed compilation for ApiVersion with clang

Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
2021-05-25 07:11:55 +03:00
Ilya Lavrenov bb3ea830cc
Returned C++ wrappers creation from interface API (#5729) 2021-05-21 20:50:16 +03:00
Shoujiang Ma 370617d909
Auto plugin async infer request implementation (#5707)
* Async auto-request, now with revamped SetCallback (after https://github.com/openvinotoolkit/openvino/pull/5645 merged) it is safe to set.
Also test modification to verify that the callback is called on the same (user's) request and e.g. not on the actual device's request

* Override CreateInferRequestImpl() instead of CreateInferRequest()

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

Co-authored-by: myshevts <maxim.y.shevtsov@intel.com>
2021-05-20 15:05:37 +03:00
Ilya Lavrenov a7d95bbfa3
Added SetExeNetworkInfo to Plugin API level (#5715) 2021-05-20 14:28:01 +03:00
Ilya Lavrenov 2bdd955d6e
Inference engine versioning (#5562)
* Exclude xbyak from install

* Added automatically generated InferenceEngineConfig.cmake

* Reverted a version back

* Fixed issues with target aliases

* Make TBB dependency private

* Made ie_parallel.cmake self-sufficient

* Don't expose ie_paralle.cmake to end users

* Fixed compilation with TBB

* Fixes for TBB

* Fixed vpu_graph_transformer compilation

* Fixed tests compilation

* Added install of ie_parallel.cmake

* Switched ENABLE_ALTERNATIVE_TEMP to OFF. Fixed COMPONENTS for TBB

* Fixed file name in install rules

* Added find_dependency for TBB in ie_parallel.cmake

* WA for cmake bug with PACKAGE_PREFIX_DIR

* Fixed no-deprecation to fix speech-library build

* Reverted version from 2.1.0 to 2.1

* Revert "Reverted version from 2.1.0 to 2.1"

This reverts commit 7cb5d1563c.

* Added versions to cmake

* Added versions to ie_version.hpp

* Returned custom version file back

* Added InferenceEngineConfig-version.cmake to share as well

* Disabled one more GPU test

* Added one more WA for CI

* WA for CI issue for C API

* Added InferenceEngineConfig-version.cmake to share as well

* Added verison parsing from ie_version.hpp

* Revert "[CPU] Add Roll support (#5112)"

This reverts commit 5d8f209df6.

* Revert "[CPU] windows_Interpolate_fused-FQ_nearest-mode_nspc-layout_fix (#5317)"

This reverts commit 0808975a37.

* Revert "[INT8][BF16] INT8 + BF16 feature was enabled (#5059)"

This reverts commit 7d2ec02d65.

* Support for components

* No version for IEDevScripts package

* Removed IE_VS_VER_HAS_VERSION from vs_version.rc.in

* Added compatibility for 2.x old versioning
2021-05-20 10:21:44 +03:00
Shoujiang Ma 90a18d9cef
[AUTO] Implement auto-plugin limited devices feature (#5545)
* Update SelectDevice policy in auto plugin

Signed-off-by: Zhengtian Xie <zhengtian.xie@intel.com>

* Implement limit device list for AUTO plugin

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Add tests for AUTO limit device feature

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Add gpu tests for auto-plugin

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Fix CI cpuFuncTests issue due to BATCHED_BLOB

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Override LoadNetwork(modelPath, config) in AUTO plugin

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Update SelectDevice() logic for LoadNetwork(model, config)

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Update GetNetworkPrecision logic for auto-plugin

Signed-off-by: Zhengtian Xie <zhengtian.xie@intel.com>

* Address reviewers' comments

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Add tests for AUTO:GPU,CPU case

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Update logic in GetNetworkPrecision for auto-plugin

Signed-off-by: Zhengtian Xie <zhengtian.xie@intel.com>

* Address reviewer's comment: clean and simplify code

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Fix wrong usage of convolution weight index

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Address reviewer comment: fix get network precision logic

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Fix rebase issue

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

* Fix ie_core.cpp header change

Signed-off-by: Shoujiang Ma <shoujiang.ma@intel.com>

Co-authored-by: zhengtian.xie <zhengtian.xie@intel.com>
2021-05-19 23:47:54 +03:00
Vladimir Paramuzov d52c4d433a
[IE CLDNN] QueryAPI extension with gpu device info (#5440) 2021-05-19 16:44:40 +03:00
Ilya Lavrenov 181ad06668
Auto (#5645)
* Added LoadNetwork(filename) to AUTO

* Added more files

* So pointer can be used without loading

* Changed InferencePlugin, ICore to return internal interfaces

* Added SoPointers for InferRequest, ExecutableNetwork

* Fixed Windows

* Fixed KMB

* Fixes for KMB

* Removed dereference operator

* Play with include files

* Fixed compilation with older compilers

* Fixed comments

* Fixed win build

* Try  to fix Windows

* Try  to fix Windows 2

* Fixed windows

* Fixed windows

* Removed SOPointer as a base class

* Reverted back SOPointer split

* Code review

Co-authored-by: apankratovantonp <anton.pankratov@intel.com>
2021-05-19 13:18:58 +03:00
Aleksandr Pertovsky 76131da7ca
Add BF16 IE Precision specialization to is_float (#5654)
* Add bf16 specialization to is_float IE Precision

* Add IE precision tests
2021-05-19 07:29:03 +03:00
Dmitry Kurtaev 3dbf4b340b
Use extern template instantiation only for Clang (RTTI mess) (#5647)
* Remove extern template from headers for RTTI classes

* Moove instantiation out of the namespace

* Use __ANDROID__ conditional compilation for TBlob

* One more attempt
2021-05-18 11:51:26 +03:00
Ilya Lavrenov abd663463d
Mak SO objct to be mandatory argument for VariablState (#5572) 2021-05-12 14:17:43 +03:00
Anton Pankratv 9db7f849df
Fixed return warnings (#5410) 2021-05-11 20:02:00 +03:00
Ilya Lavrenov a6d8391c4c
Removed details::no_copy (#5553) 2021-05-11 13:05:18 +03:00
Krzysztof Bruniecki 7c07c6154b
[GNA] New flags to select GNA target generation (#5429)
* New flags for GNA trget

* Add new flags to configuration

* Update utest

* Update speech_Sample

* Add unit tests

* Guard for wrong values

* Use gna execution target as consistent device

* Apply review
2021-05-07 13:02:49 +03:00
Ilya Lavrenov b47d11e31e
Removed useless iostream include (#5357)
* Removed useless iostream include

* Fixed samples compilation

* Klockwork fixes for template plugin
2021-05-03 17:12:00 +03:00
Ilya Lavrenov c350f61a42
Move all base wrapper classes from Plugin API to source folder (#5419)
* Small refactoring in TEMPLATE plugin

* Fixed compilation on Windows

* Fixed code style

* Hide CALL_STATUS_FNC helpers to private API

* Moved some base classes to private place from plugin_api

* Updates for VariableState creation

* Take Jane's changes for Demension names

* Revert "Take Jane's changes for Demension names"

This reverts commit 9f6c8fa5a6.

* Removed ICNNNetwork include

* removed more icnnnetwork includes

* Added missed include with ie_input_info.hpp

* Fixed GNA plugin to provide names w/o \0
2021-04-29 19:50:46 +03:00
Maxim Shevtsov 80f5fe953b
Openvino hybrid awareness (#5261)
* change the deprecated method to the recent

* first ver of the hybrid cores aware CPU streams (+debug info)

* more debug and fixed sum threads

* disabled NUMA pinning to experiment with affinity via OS

* further brushing of stream to core type logic

* hybrid CPU-aware getNumberOfCPUCores

* adding check on the efficiency

* experimental TBB package (that cmake should pull from the internal server)

* iterating over core types in the reversed order (so the big cores are populated first in case user specified less than all #threads)

* adding back the NUMA affinity code-path for the full validation (incl 2 sockets Windows Server)

* cpplint fix and tabbing the #if clauses for the readbility

* pre-production TBB from internal server

* wrapping over #cores/types

* wrapping over #cores/types, ver 2

* wrapping over #streams instead

* disabling warnings as errors for a while (to unlock testing)

* accomodating new TBB layout for dependencies.bat

* next tbb ver (with debug binaries that probably can unlock the commodity builds, without playing product_configs)

* minor brushing for experiments (so that pinning can be disabled)

* minor brushing from code review

* Updating the SHA hash which appeared when rebasing to the master

* WIP refactoring

* Completed refactoring of the "config" phase of the cpu stream executor and on-the-fly streams to core types mapping

* making the benchmark_app aware about new pinning mode

* Brushing a bit (in preparation for the "soft" affinity)

* map to vector to simplify the things

* updated executors comparison

* more fine-grained pinning scheme for the HYBRID (required to allow all cores on 2+8 1+4, and other LITTLE-skewed scenarios)

TODO: seprate little to big ratio for the fp322 and int8 (and pass the fp32Only flag to the MakeDefaultMultiTHreaded)

* separating fp32 and int8 intensive cases for hybrid execution, also leveraging the HT if the #big_cores is small, refactored. also switched to the 2021.2 oneTBB RC package

* code style

* stripped tbb archives from unused folders and files, also has to rename the LICENSE.txt to the LICENSE to match existing OV packaging tools

* assigning nodeId regradless of pinning mode

* tests OpenCV builds with same 2021.2 oneTBB, ubuntu 18/20

* cmake install paths for oneTBB, alos a ie_parallel.cmake warning on older ver of TBB

* Updated latency case desc to cover multi-socket machines

* adding centos8 OCV with oneTBB build

updating TBB drops with hwloc shared libs added.

* enabled internal OCV from THIRD_PARTY_SERVER to test thru CI..
Added Centos7 notbb OCV build (until g-api get ready for onetbb) to unlock the Centos7 CI build

* separate rpath log to respect one-tbb specific paths

* fixed SEQ code-path

* fixed doc misprint

* allowing all cores in 2+8 for int8 as well

* cleaned from debug printfs

* HYBRID_AWARE pinning option for the Python benchmark_app

* OpenVINO Hybrid CPUs support

* Remove custom::task_arena abstraction layout

* Get back to the custom::task_arena interface

* Add windows.h inclusion

* Fix typo in macro name

* Separate TBB and TBBbind packages

* Fix compile-time conditions

* Fix preprocessors conditions

* Fix typo

* Fix linking

* make linking private

* Fix typo

* Fix target_compile_definitions syntax

* Implement CMake install logic, update sha hash for the tbbbind_2_4 package

* Add tbbbind_2_4 required paths to setup_vars

* Update CI paths

* Include ie_parallel.hpp to ie_system_conf.cpp

* Try to update dependencies scripts

* Try to fix dependencies.bat

* Modify dependencies script

* Use static tbbbind_2_4 library

* Remove redundant paths from CI

* Revert "cleaned from debug printfs"

This reverts commit 82c9bd90c5.

# Conflicts:
#	inference-engine/src/inference_engine/os/win/win_system_conf.cpp
#	inference-engine/src/inference_engine/threading/ie_cpu_streams_executor.cpp
#	inference-engine/src/mkldnn_plugin/config.cpp

* Update tbbbind package version

* fixed compilation

* removing the direct tbb::info calls from CPU plugin, to aggregate everything in the single module (that exposes the higher level APIs)

* Update tbbbind package version

(cherry picked from commit f66b8f6aa6)

* compilation fix

* brushing the headers a bit

* Make custom::task_arena inherited from tbb::task_arena

* change to the latest TBB API, and more debug printfs

* code-style

* ARM compilation

* aligned "failed system config" between OV and TBB (by using '-1')

* macos compilation fix

* default arena creation (to make sure all code-path have that fallback)

* Incapsulate all TBB versions related logic inside the custom namespace

* Move custom layer header to internal scope + minor improvements

* with all NUMA/Hybrid checks now consolidated in the custom_arena, cleaning the ugly ifdefs thta we had

* Introduce new ThreadBindingType + fix compilation

* fixing OMP compilation

* OpenVINO Hybrid CPUs support

* Remove custom::task_arena abstraction layout

* Get back to the custom::task_arena interface

* Add windows.h inclusion

* Fix typo in macro name

* Separate TBB and TBBbind packages

* Fix compile-time conditions

* Fix preprocessors conditions

* Fix typo

* Fix linking

* make linking private

* Fix typo

* Fix target_compile_definitions syntax

* Implement CMake install logic, update sha hash for the tbbbind_2_4 package

* Add tbbbind_2_4 required paths to setup_vars

* Update CI paths

* Include ie_parallel.hpp to ie_system_conf.cpp

* Try to update dependencies scripts

* Try to fix dependencies.bat

* Modify dependencies script

* Use static tbbbind_2_4 library

* Remove redundant paths from CI

* Update tbbbind package version

* Make custom::task_arena inherited from tbb::task_arena

* Incapsulate all TBB versions related logic inside the custom namespace

* Move custom layer header to internal scope + minor improvements

* Introduce new ThreadBindingType + fix compilation

* Fix compilation

* Use public tbbbind_2_4 package

* fixed macos build, corrected comments/desc

* reverted to the default binding selection logic ( to preserve the legacy beh)

* OpenVINO Hybrid CPUs support

* Remove custom::task_arena abstraction layout

* Get back to the custom::task_arena interface

* Add windows.h inclusion

* Fix typo in macro name

* Separate TBB and TBBbind packages

* Fix compile-time conditions

* Fix preprocessors conditions

* Fix typo

* Fix linking

* make linking private

* Fix typo

* Fix target_compile_definitions syntax

* Implement CMake install logic, update sha hash for the tbbbind_2_4 package

* Add tbbbind_2_4 required paths to setup_vars

* Update CI paths

* Include ie_parallel.hpp to ie_system_conf.cpp

* Try to update dependencies scripts

* Try to fix dependencies.bat

* Modify dependencies script

* Use static tbbbind_2_4 library

* Remove redundant paths from CI

* Update tbbbind package version

* Make custom::task_arena inherited from tbb::task_arena

* Incapsulate all TBB versions related logic inside the custom namespace

* Move custom layer header to internal scope + minor improvements

* Introduce new ThreadBindingType + fix compilation

* Fix compilation

* Use public tbbbind_2_4 package

* Apply review comments

* Fix compilation without tbbbind_2_4

* Fix compilation with different TBB versions

* code review remarks

* fix for the NONE pinning code-path under HYBRID_AWAR

* whitespace and cleaning the debug printfs (per review)

* code-review comments

* fixed code-style

Co-authored-by: Kochin, Ivan <ivan.kochin@intel.com>
Co-authored-by: Kochin Ivan <kochin.ivan@intel.com>
2021-04-28 17:42:58 +03:00
Ilya Lavrenov 6c83e0f8a5
Small refactoring in TEMPLATE plugin (#5398)
* Small refactoring in TEMPLATE plugin

* Fixed compilation on Windows

* Fixed code style
2021-04-27 18:52:45 +03:00
Ilya Lavrenov 0337b055db
Added GetAvailableDevices for AUTO device (#5377)
* Added GetAvailableDevices for AUTO device

* Fixed tests compilation
2021-04-27 09:12:55 +03:00
Ilya Lavrenov 4073e16a9e
Fixed compilation for deprecated API (#5356)
* Fixed compilation of offline API

* Fixed compilation with gcc 5.4

* Fixed compilation of ONNX importer dependencies
2021-04-23 09:27:39 +03:00
Ivan Kochin 636736825c
OpenVINO Hybrid CPUs support internal interface (#4602)
* OpenVINO Hybrid CPUs support

* Remove custom::task_arena abstraction layout

* Get back to the custom::task_arena interface

* Add windows.h inclusion

* Fix typo in macro name

* Separate TBB and TBBbind packages

* Fix compile-time conditions

* Fix preprocessors conditions

* Fix typo

* Fix linking

* make linking private

* Fix typo

* Fix target_compile_definitions syntax

* Implement CMake install logic, update sha hash for the tbbbind_2_4 package

* Add tbbbind_2_4 required paths to setup_vars

* Update CI paths

* Include ie_parallel.hpp to ie_system_conf.cpp

* Try to update dependencies scripts

* Try to fix dependencies.bat

* Modify dependencies script

* Use static tbbbind_2_4 library

* Remove redundant paths from CI

* Update tbbbind package version

* Make custom::task_arena inherited from tbb::task_arena

* Incapsulate all TBB versions related logic inside the custom namespace

* Move custom layer header to internal scope + minor improvements

* Introduce new ThreadBindingType + fix compilation

* Fix compilation

* Use public tbbbind_2_4 package

* Apply review comments

* Fix compilation without tbbbind_2_4

* Fix compilation with different TBB versions

Co-authored-by: Kochin, Ivan <ivan.kochin@intel.com>
2021-04-22 18:11:13 +03:00
Ilya Lavrenov 02bc98a03f
Removed suppressions for IInferRequest deprecation (#5310)
* Removed suppressions for IInferRequest deprecation

* Fixed Windows

* More fixes for Windows

* Fixed compilation on Windows

* Fixed comment in documentatipn

* Fixes for Andorid

* Fixes for old gcc 4.8

* WA for cross-compilations

* Fixed compilation

* Fixed HETERO plugin compilation for old compilers

* Flags

Co-authored-by: lab_ddpqa <lab_ddpqa@intel.com>
2021-04-21 16:05:30 +03:00
Anton Pankratv 46987def54
Merged internal Infer Request implementation (#5125) 2021-04-19 15:16:47 +03:00
Ilya Lavrenov 3bbc9d2837
Added deprecation messages to docs for ExecutableNetwork (#5201) 2021-04-14 06:05:41 +03:00
Anton Pankratv 9a46851a33
Removed throwing destructor (#5023) 2021-04-12 07:20:59 +03:00
Anton Pankratv b20d9b725c
Used internal exec network impl (#4922) 2021-04-08 12:29:41 +03:00
Szymon Irzabek 2486c5b90a
GNA low precision (#4109)
* [GNA] Add low precision support for some layers (affine, eltwise, ReLU, sigmoid)

* [GNA] Fix convolution and scaleshift regression fail

* [GNA] Fix const scale factor calc

* [GNA] Fix extra whitespace

* [GNA] Eltwise padding and low precision primitive creation support

* [GNA] Changes after review

* [GNA] Remove INPUT_PRECISION GNA flag

Remove INPUT_PRECISION flag, all references to it
and tests using it. Other minor stylistic changes and cleanup.
2021-04-07 14:28:24 +03:00
Taylor Yeonbok Lee b58c648d2d
[IE CLDNN] Build time optimization for OCL kernels (#4725)
Build OCL kernel batches of different buckets in parallel

Co-authored-by: Donghyeon Jeong <donghyeon.jeong@intel.com>
2021-03-31 10:22:19 +03:00
Dmitry Kurtaev 64874fa69c
Remove __clang__ macros from headers (#4575)
* Fix headers for Clang compilation

* Parameter::Any destructor

* Force RTTI for any compiler

* Parameter::RealData RTTI

* Revert Parameter::Any
2021-03-25 12:26:43 +03:00
Alexey Suhov a748c26fee
Align copyright notice in cpp and cmake source files (CVS-51320) (#4950) 2021-03-25 02:40:09 +03:00
Ilya Churaev f0e574903a
Int4 support for nGraph (#4712)
* Start int4 type

* Changed int4 type

* Fixed code style

* Added u4 type

* Added tests

* Add U4 I4 precisions to the IE

* Removed redundant tests

* Added convert precision

* Fixed tests

* Fixed C tests

* Fixed logic for signed values

* Fixed nGraph tests

* Added more tests

* Fixed code style

* Update inference-engine/src/transformations/src/transformations/convert_precision.cpp

Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>

Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
2021-03-24 16:17:43 +03:00
Anton Pankratv 3b3d9a0989
Used IE_THROW macro (#4869)
* Added ie throw macro

* Used IE_THROW macro
2021-03-23 18:57:12 +03:00
Anton Pankratv 35bff0d8a1
Moved executable network implementation from header (#4049) 2021-03-19 11:58:16 +03:00
Anton Pankratv a2b8b974b8
Simplified IE Exceptions Implementation (#4258) 2021-03-18 16:30:16 +03:00
Mikhail Nosov 7f9daadc08
Model caching support - Core part (#4661)
* Model caching support - Core part

Introducing model caching support
Use core.SetConfig({{CONFIG_KEY(CACHE_DIR), <dir>}}); to enable caching of models

OpenVINO will try to create caching folder if it doesn't exist, but it is recommended for client to create caching folder with necessary permissions before enabling cache in config

For caching, plugins shall support import/export functionality
Plugin requirements:
- Add METRIC_KEY(IMPORT_EXPORT_SUPPORT) in SUPPORTED_METRICS to support caching
If plugin has different device architectures with different caches, i.e.
For "GNA.0" - one cache, for "GNA.10" - another cache
In this case plugin shall support DEVICE_ARCHITECTURE metric and return different strings for different DEVICE_ID's

Added functional tests

* Fix CentOS build issues

* Few updates according to code review

* Revert unnecessary changes for Import/Export core implementation

These changes affect old behavior and may be undesired
For caching support these is no need to change anything in this area
If needed, such removal of 'Magic' usage can be done under separate task in future

* More tests:
1) Verify that Imported data from stream is the same as was exported
2) Verify that cache is not loaded when config in LoadNetwork is changed
3) Verify that if CNN Network is changed between ReadNetwork and LoadNetwork - cache is not loaded

* Update of NetworkCompilationContext

Put back functionality of calculating hash based on runtime information, weights
Implemented OstreamHashWrapper to avoid serialization to buffer

* Correction of CACHE_DIR key description

* Unit tests for compilation_context

Changes:
1) Improved handling of OstreamHashAdapter
2) Improved runtime info serialization (not just PrimitivesPriority and affinity)
3) Removed redundant weights hash calculation

* Fix GCC 4.8 build issues

* Compilation context updates
1) Use hash of sum of serialized data to get hash of network. It is more efficient comparing to weights sum calculation
2) CalculateFileInfo - convert path to absolute ("./test.blob" and "test.blob" shall give same hash)

* Hash - added more rt_info attributes + tests

- PrimitivesPriority
- FusedNames
- Dequantization

* Moved "get_absolute_path" macro to file_utils.h

* Make 'absoluteFilePath' a library API, not macro

* One more unit test for fileName hashing

* Fix compilation error after merge with latest master

* Allow tests to be executed in parallel (stress mode)

* More minor updates for stress testing

Now it allows to execute tests with '--repeat=100' option where one test is executed in multiple processes simultaneously

Example:
./gtest-parallel <openvino_dir>/bin/intel64/Debug/ieFuncTests --gtest_filter=CachingTest* --repeat=10

* Use absolute model file path for calculating blob name

* Added 'createDirectoryRecursive' API to plugin_api/file_utils
2021-03-16 14:13:45 +03:00
Ilya Lavrenov d77a07bd6a
Deprecated IE::Parameter <-> ngraph::Variant (#4734) 2021-03-12 06:36:28 +03:00
Ilya Lavrenov 95e3a1bf5d
Ported #4657 (#4698)
* Updated API changes document

* Comment for CVS-49440
2021-03-10 14:57:37 +03:00
Anton Pankratv 2fcf92be42
Removed IRelease Interface (#4032) 2021-03-05 12:08:01 +03:00
Bartosz Sochacki 3daacb5e09
[GNA] POT enabling (#4358)
* POT enabling for GNA

* POT enabling for GNA

* POT enabling for GNA - accuracy improvements
2021-03-04 16:10:01 +03:00