* fix benchmark filed to pass on MULTI device when enable num_streams.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove part of test cases because MULTI plguin will not check if the unspported property is valid now.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove some incorrect config for MULTI test case since MULTI will pass through those unrecognized config without any exception.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support MULTI to set nstreams to multi target devices by using ov::device:properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support AUTO to set nstreams with multi target devices bu using ov:device::properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* When set batch_size, disable auto batching
* auto-batch only supprt auto plugin
* Set property allow_auto_batching for devices
* cancel static
* Clean Code
* Add test case for testing allow_auto_batching
* Fix the issue after merging latest master branch code
* Add allow_auto_batching in python benchmark_app
* Fix the quotes issue
* When load_network is set to allow-auto-batching, there is no need to read the value of global
* Fix the clang-format
* Change the location
Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* WA for Benchmark App: Remote tensor use for internaly dynamic original models (NMS) that turn static due to transformation to legacy op and lack of dynamism support
* Style
* Updated ia32 toolchain with system / processor
* Allow pkg-config during cross-compilation
* added error message
* Search TBB using tbb.pc as well
* WA
* Fix for yocto
* Added print
* Use tbb.pc only when system libs are enabled
* Improved OpenVINOConfig.cmake
* Don't find pkg-config as required
* [SAMPLES] Add GNA library version log to speech_sample
* [GNA] Add OV version to TLV export file
Co-authored-by: Szymon Irzabek <szymon.jakub.irzabek@intel.com>
* update logic of setting TPUT by default for MULTI and AUTO.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set config to MULTI instaed of target device when using -d MULTI:target_device in benchmark app.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update logit to check if perfomance hint is set.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set the correct _so from passthrough executable network when batch plugin is disabel to fix lifecycle coredump.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Same update for AUTO plugin
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* [C API 2.0]Redifine partial shape and property wrapper
1. Use dimension object to initialize partial_shape rather than string
2. Use void* to unify property value rather than union
3. rename some C api name to strict align with C++ method.
Change-Id: I64b5c521461264dba2d23543808584632fbd6d4b
* [C API 2.0]Memory check and implement all reshape interface
1. Memory safe create and free
2. Implement all reshape interface align with C++ interface
3. rename some api to align with C++ interface
Change-Id: Ib5e4192bdbd8a11cdd7e30b1dc84881ba3f2d505
* Rename prepostprocess to strict align with C++ name
Change-Id: I7a4d0a6e835b2d6ed01cd218ac81b1b621f600bf
* [C API 2.0]redefine ov_node and ov_model interface
1. redefine ov_node and ov_model interface
2. rename some api to aligne with C++ interface
3. remove some redundant code
4. align CMakeLists.txt with OpenVINO 2.0 convention
Change-Id: I4d5e92157e7891319c9754da8e70b9c6150ae2e3
* Redefine ov_layout to support more than one char
Change-Id: I39e5389246cf3edcc2f4734d13157457773d89b8
* Add interface to get partial_shape from node
Change-Id: I8cef77db581b43d2f0a9ac48cfdc09a86e39b694
* Use unique_ptr prevent memory leaks in case of exception
Change-Id: I150b375108a3eded400bdde087ab5c858958c25f
* Put legacy C API and 2.0 C API into a library
Change-Id: I067a55a00e78b80cdede5ae7adad316ee98cabd1
* Only keep OV 2.0 C sample and move legacy C sample to legacy directory
1. Move legacy C samples to tools/legacy/c directory
2. Keep OV 2.0 C samples in samples/c directory
Change-Id: I05880d17ee7cb7eafc6853ebb5394f3969258592
* Fix format and log issues
Change-Id: I05d909b3d7046d41b807e35808a993bb09672e68
* Restore documents update
Change-Id: I82dd92081c0aa1a2d7dca7f114cf6a35131d6f92
* Change config map data be const
Change-Id: I9043859e8308c01d80794dc8280ae236947f3bbb
* Update api document
Change-Id: I35bc149bad0de17424d95f48c3027030b708e147
* Add clang enable
Change-Id: I335639c05fb5fb38e682dbb72bfaf78380c0adaf
* Fix clang issue after enable clang for ie_c_api.c
Change-Id: Idcb4dda9d66e47a169eb79a9c4fe7a7d4df838db
* split header file and c file into multiple files
Change-Id: I7c3398966809ef70d7fcb799f2d612a33b471e31
* Fix clang format issue
Change-Id: Ibd18b45537c8f3bcbb5b995c90ae28999161d54d
* Add single ov_dimension_create method
Change-Id: Icd06b50e4f4df8f7897c7c4327edb67178162544
* Remove all legacy c samples completely
Change-Id: I098360a0d9002340e8769074181f7997b43bce8f
* Update ov_property_value to replace only ptr
Change-Id: I9f5a11b4cf07e759c1998e78e2624f0a1266d9b0
* Split more header files, add static dimension api
Change-Id: I14e4fb8585fc629480c06b86bd8219e75a9682f7
* Change ov_dimensions_create to be ov_dimensions_create_dynamic
Change-Id: I50c02749cea96f12bcea702b53a89c65b289550e
* rename status and get_out_tensor
Change-Id: I762c1d0c5a069454506fe3c04283c63ddbfacf31
* Split ov_c_api_test.cpp
* Split OV2.0 CAPI tests
* move var into Setup
* Merge legacy and 2.0 C API test
* Merge InferenceEngineCAPITests into openvino_capi_test
1. put InferenceEngineCAPITests into openvino_capi_test
2. resolve some format issues
Change-Id: I47bbba6bd70a871ee063becbd80eb57919fa9fb0
* legacy api test skips clang format
Change-Id: Id54ecdba827cf98c99b92295c0a0772123098b63
* Fix clang format issue
Change-Id: I7ed510d8178971fe04a895e812c261db99d8b9f2
* Restore InferenceEngineCAPITests
Change-Id: I4d641ffb1de9ce4d20ebecf35fc036fa7bd73e55
* rename openvino_capi_test to ov_capi_test
Change-Id: I6b6fe0cdb89aab7210abb17f32dbfdcdce72ba25
* unify list size name and refine ov_core_version_t
Change-Id: I137fc6d990c7b07f597ee94fa3b98d07ae843cb6
* align header file path to be openvino/c/openvino.h
Change-Id: I1a4552e1d558098af704942fe45488b0d6d53a90
* Fix path issue
Change-Id: I84d425d25e3b08c1516cbcc842fb9cb75574bf17
* move ov_color_format and remove opencv depenency
Change-Id: I486145f9e92e8bbf2e937d3572334aa9f0e68841
* Resolve some memory allocation error handling issues and read model with empty weight issue
Change-Id: Icd8e3b6de9741147993fa215a0c7cfd7debd5500
* Add GPU test cases
Change-Id: I13324ef019b5b1af79259ca932a36a0cec792c27
* Fix clang issue
Change-Id: I9bb4c47de301d142b5e2a77a39f667689ad9fe38
* Resolve CI test failure
Change-Id: Ia327d5edab19d8dd44ac369670f190d5c57aca79
* Redefine ov_shape and add default ov_core_create
Change-Id: I3e47d607f8aad65cb99cdddacaecf7bf34b1361b
* Remove some unnecessary api of node
Remove the unnecessary node api:
ov_node_get_any_name(ov_output_const_node_t* node, char** tensor_name)
ov_node_get_element_type(ov_output_const_node_t* node, ov_element_type_e* tensor_type)
Change-Id: I80a3243676800263a9e56afa3cfffce7b4bd2ae7
* Rename reshape api
ov_model_reshape should be common case which allow to reshape any models with different number of input.
Change-Id: I26bafeeb8a3dda7cd5164cda15fdb338db8668cb
* Rename ov_node api
Change-Id: I03114ecb6de5c46b6d02c909b6f6fb6c8bfd5cba
* Remove subfolder out of source code
Change-Id: Ib033ae7712cc0460d6fc21a0f89818381ae503c0
* apply absolute path for all header files
Change-Id: I8024c897d424b407025e21460ed4b62829b853d2
* Fix CI issue ov_capi_test failed to find libgna
Change-Id: I166e79a818498c6721fe956f43873f36d9ae1e07
* Resolve build issue to align with PR12214
Change-Id: I9e6094db213b431ee1b46e0d64199131db33bb36
Co-authored-by: ruiqi <ruiqi.yang@intel.com>
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* First version of Windows installer
* WIndows NSIS installer
* Improved version of debian packages
* Improvements
* Improvements
* Debian packages now look good
* Library versioning
* Fixed tests to run against debian packages
* Fixed frontend tests
* Fixed code style
* FIxed Windows
* Fixed python tests
* Fixed paths in tests
* fdvfdv
* Fixes
* USe versioning only for debian packages
* Relocatable tests
* Fixed
* Fixed all tests
* Fixed clang-format
* Fixed more tests
* Fixed some tests
* Absolute paths in .ci
* Fixes
* Added support of OpenCV 3.4
* Trying to fix gnaUnitTests
* Add some tip description about enable OpenCL for benchmark_app.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Export doesn't work, we need to add -Dopencl_root_hints=[PATH]/OpenCL-CLHPP/include to cmake command.
Signed-off-by: xipingya <xiping.yan@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* revert test code
* revert test code
* enable brgconv f32
* use config to enable brgconv f32
* when brg disabled not init bin-postops
* change prop name for extensive
* use more general field
* fix review comments.
* Initial files & cmakefiles for ov 2.0 c api development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add all ov 2.0 C APIs define
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix review comments
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Disable test of OV 2.0 C APIs test for tmp
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add related property key for ov 2.0 C-API
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add description for ov_property_key_e
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add EXECEPTION handling
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* compiledModel add interface
* add inferrequest interface
* solve cpplint problem
* Finished OV 2.0 C-APIs PPP related development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix code review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add ov::tensor API
* add compiled model func
* Finished C-API funs about core, model, node development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C-API] add const to ov_output_node
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C-API] Using define GET_OV_ELEMENT_TYPE & GET_CAPI_ELEMENT_TYPE in tensor APIs
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C-API] add string initialize
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* add inferrequest func
* add move construction to runtime_model
* supplement two infer request interface functions
* [OV 2.0 C-API] Add the common framwork of unit test
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* modify ov_infer_request_get_profiling_info
* add tests dir
* restore CMakeLists.txt
* Fix the bug of COPY in Tensor
* [OV 2.0 C API] Finished core related function unite test
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add ov:Tensor API test
* [OV 2.0 C API] fix some review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* add some infer request test
* add compiled model test
* [OV 2.0 C API] Finished preprocess related function unite test
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] Fix review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] Modify to use default model
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* transfer device_name from fix value to parameter
* add some infer request test
* remove compiled model get_property test
* add infer request tests
* Add ov::model Test and modify Tensor Test name
* Determine whether partial shape meets the standard
* Add get tensor name function and Modify reshape test case
* modify fixed tensor name,remove unnecessary comparison
* add ov_model_get_nodes_info, modify according to comments
* Update reshape test
* extract common function, modify interface about get tensor name,shape and type
* modify according comments
* [OV 2.0 C API] Finished hello classification with ov 2.0 c-api development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] Fixed hello classification with ov 2.0 c-api review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] delete inactive code hello classification with ov 2.0 c-api
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix clang format issue
* [OV 2.0 C API] rename
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix windows build erre
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Apply qsort for sorting data
Apply qsort for sarting data
Fix issues of "potentially uninitialized local pointer variable"
* Not use deprecated INSTANTIATE_TEST_CASE_P for c api gtest
INSTANTIATE_TEST_CASE_P is deprecated, should use INSTANTIATE_TEST_SUITE_P.
* Fix some review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [Ov 2.0 C API] Add error info
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix some review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* polish error message for ov c api
* Redefined ov_shape_t, ov_partial_shape_t and ov_layout_t. Modified functions and test cases involving these variables
* Added the conversion between char* and partial_shape
* Add partial_shape_to_shape
* prune code
* modify split
* Use regex to split and search pattern
* Modify str_to_char_array delete
* Add the judgment of rank
* Fix compiling error
Fix issue: address of array 'shape.dims' will always evaluate to 'true' if -Wpointer-bool-conversion
Co-authored-by: xuejun <Xuejun.Zhai@intel.com>
Co-authored-by: sunxiaoxia2022 <xiaoxia.sun@intel.com>
Co-authored-by: ruiqi <ruiqi.yang@intel.com>
* selectdevice returns MULTI:device in cumulative_throughput
* load multi with throughput and disable cpu helper in cumulative
* disable cpu helper in cumulative_throughput
* add cumulative to bechmark_app help message
* modify benchmark_app.hpp clang-format
* Try to improve gflags
* Try to improve gflags: part 2
* Tried to use dependencies on system
* Use nlohmann_jsonConfig from system
* Enabled nlohmann_json from system
* Improvements
* handle system gflags in developer package
* Simplifications
* Simplify dependency management
* Corrected package names
* Fixed subgraphsDumper configure stage
* Try to fix rhel8
* Try to fix macosx
* Fixed VPUX build
* Fixed aliasing issues
* Suppress some wanrings
* export gflags when build it
* Fixed some LTO
* Try to fix Mac
* revert
* use gflags as private dependency
* Aligned targets in developer package
* Fixed frontends tests build on U20 with LTO
* PAssed
* Don't use pkg_search_module(zlib ..) during cross-compilation
* Removed unused variables
* Fixed finding of zlib during cross-compilation
* CVS-83529
* Use nothreads_static
* Fixed python
* [IE Samples] Activating new parameter is compact mode(memory_reuse) in speech sample
* changed format
* renamed the option to memory_reuse
* renamed the option
* Try to improve gflags
* Try to improve gflags: part 2
* Tried to use dependencies on system
* Use nlohmann_jsonConfig from system
* Enabled nlohmann_json from system
* Improvements
* handle system gflags in developer package
* Simplifications
* Simplify dependency management
* Corrected package names
* Fixed subgraphsDumper configure stage
* Try to fix rhel8
* Try to fix macosx
* Fixed VPUX build
* Fixed aliasing issues
* Suppress some wanrings
* export gflags when build it
* Fixed some LTO
* Try to fix Mac
* revert
* use gflags as private dependency
* Aligned targets in developer package
* Fixed frontends tests build on U20 with LTO
* PAssed
* Don't use pkg_search_module(zlib ..) during cross-compilation
* Removed unused variables
* Fixed finding of zlib during cross-compilation
* Build with system TBB
* Fixes
* Check whether system TBB is available
* Try to fix ONNX Runtime build with system TBB
* Test
* Fixed compilation of threading.cpp
* Fixed unset of cache dirs
* Limit dearch paths of TBB
* Try to enable pip packages with custom TBB
* Fix for TBB 2021.2
* Install only needed TBB libraries
* Install TBB from system to pip package
* Reverted usage of TBBROOT
* Fixed oneTBB case
* Try to fix Android
* Escape some paths
* Added samples path
* Fixed TBBBind usage for case of system TBB
* New command line parameters format for speech sample
* fixed notes
* changed format for scale factor
* changed format for scale factor in tests
* added more variants, when name is directy specified for i/o/r like it is done for sf
* removed nthreads flag
* fixed notes
* changed output params
* updated tests with new format
Co-authored-by: Alexander Zhogov <alexander.zhogov@intel.com>
* Mark `get_type_info_static()` as hidden
Each plugin linked with openvino library contains `type_info_static` symbols. In case when one of the libraries is unloaded and app tries to get opset, it leads to segfault. So mark `get_type_info_static()` as hidden to use only one implementation exactly from openvino lib
* Fix "'visibility' attribute ignored" issue by moving `TestPass` out of test scope
* Fix clang format
* Small update of `If` op
* Revert "fix 79520 (#10449)" to correctly compare DiscreteTypeInfo via `==`
This reverts commit 29883a152a.
* InputTensorInfo::from implementation
If user's application already has `ov::runtime::Tensor` object created,
it will be possible to reuse basic characteristics for input (shape, precision) from tensor using InputTensorInfo::from method
* Rename 'from' to 'set_from' as in Python 'from' keyword is used for import modules
Python bindings: from ov.Tensor and from numpy array
* Style fix (quotes)
* Apply suggestions from code review
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* Fix code style
* Use set_from in hello_classification CPP sample
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* fix .ncc_style target names
it was breaking configure on system with libclang-12-dev, clang-12,
ninja and cmake 3.17+(ninja complains about duplicate
target). with lower cmake version configure succeeds, but build exits
immediately with error. by replacing ninja with make error becomes
warning(it's still significant, make just skips duplicate rules, i.e.
doesn't check style of some source files, rule duplication is genuine
bug). without libclang-12-dev and clang-12 ENABLE_NCC_STYLE is OFF and
bug is not triggered
* silence uninitialized warning in core_integration
probably it was always initialized before use, but compiler wasn't made
aware of it
* fix function spelling to unbreak code style checks in benchmark_app
* include <thread> for std::this_thread
existing code was relying on namespace pollution by old libstdc++
* replace is_pod with is_standard_layout && is_trivial
is_pod is deprecated, it breaks build on current gcc
Co-authored-by: Serhii Pavlovskyi <spavlovskyi@lohika.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* Fix result saving when batch size is not 1
* Remove useless if statement
* improved processing scores for model with more than one outputs
* added checking on count of model outputs
* improve if statements
* divide fix for model with several outputs to other PR
Co-authored-by: Maxim Gordeev <maxim.gordeev@intel.com>
* Improve `-o` and `-oname` flags
* Apply clang-format tool
* fix saving output files
* Apply clang-format
* Fix error when `-oname` not specified
* apply clang format
* Fix error `-oname`
* Use output name with port to find model output
* fix comment line breaking
* fix comparison with reference for multiple outputs
* Fix output name printing error
* try to fix clang format
* fix problem with bs > 1
* minimal change to rerun test pipeline
* clang format
* Revert "Fix error `-oname`"
This reverts commit c33d5f16e8.
* Used new config for streams and threads
* Fixed review coments in ba
* format fix
* fixed hello_query_device
* Added STL string io
* fixed tests
* Fixed test
* Fixed build
* fixed format
* Fixed build
* try fix win
* other any io specialization
* Fixed after merge
* renamed streams
* build fixed
* fixed build
* fixed format
* fix for old mac build
* Fixed type of exception
* test fix
* Depricated Any implicit cast
* Fixed test
* fixed gna build
* Fixed warnings in benchmark_app
* Fixed test build
* ncc exception for PrintTo
* Error mesage in test
* Error mesage in test
* fixed build
* Added new correct gna frequency result for Alder Lake
* Update samples/cpp/speech_sample/utils.hpp
Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
* Refactored statistics output with JSON support
* Detailed/average reports are added
* stylefix
* Update samples/cpp/benchmark_app/statistics_report.hpp
Co-authored-by: Ivan Vikhrev <ivan.vikhrev@intel.com>
* Linux Fixes
* stylefixes
* data_shape field format is changed
* stylefix
Co-authored-by: Ivan Vikhrev <ivan.vikhrev@intel.com>
* [IE SAMPLES] activated NCC tool for c++ samples
* exclude ov_ncc_naming_style for tests
* fixed NCC hit
* Added support for source files in samples
* changed style of methods for benchmark
* changed style for speech sample
* changed code style
* changed code style for shared_tensor
* benchmark changes
* changed remote_tensors_filling
* fixed notes
* rebase of branch
* Set THROUGHPUT as the default configration for all the plugin and display the config of the plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* updated format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update benchmark python API.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Replace str 'THROUGHPUT' with CONFIG_VALUE(THROUGHPUT).
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Using CONFIG_VALUE(THROUGHPUT) replace 'THROUGHPUT' string.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update code style.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Move the setting output code into the try block.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
I'll merge this. We need to talk with someone who know openvino build system better than we to "use cmake function ov_ncc_style_check to perform such check automatically". As far as I can see, currently it used only in one place, so it is not common approach for openvino components
* auto-batching POC squashed (all commits from auto-batch-2021.3 branch)
(cherry picked from commit d7742f2c747bc514a126cc9a4d5b99f0ff5cbbc7)
* applying/accomodating the API changes after rebase to the master
* replaying modified version of actual batch selection
* eearly experiments with model mem footprint
* changes from rebasing to the latest master
* experimenting with DG1 on the batch size selection, also collecting the mem footprint
* WIP:moving the auto-batching to the icore to let the MULT/AUTO support that, ALLOW_AUTO_BATCHING as a conventional config key. still fials hot device swap
* quick-n-dirty batch footpint vs device total mem
* code style
* testing which models perform badly due to kernels and NOT (batched) footprint
* stub pipeline task to comunicate the readiness rather than promise/future
* quick-n-dirty timeout impl
* explicit _completionTasks,reverting BA to use the timeout
* inputs outputs copies, works with AUTO and demo now
* accomodate the config per device-id, after rebase to the latest master
* allowing the auto-batching only with tput hint to let more conventional tests pass
* fix the pre-mature timeout restaring via waiting for batch1 requests completion
* moved the bacthed request statring ( along with input copies) to the dedicated thread
* [IE CLDNN] Disable bs_fs_yx_bsv16_fsv16 format for int8 convolution
* code style
* increasing the timeout to test the ssd_* models perf (timeout?) issues
* reducing number of output stuff in BA to avoid bloating the logs in experiments
* more aggressive batching for experiments, not limited to 32 and also 4 as a min
* more accurate timeout debugging info
* getting the reqs limitation from the plugin SetConfig as well
* refactor the reshape logic a bit to accomodate CPU for bathcing, also added remeote context
* let the benchamrk_app to consume specific batch values for the auto-batching such as BATCH:GPU(4)
* auto-batching functional test (with results check vs ref) and GPU instance for that
* fixed arithemtic on blobs ptrs
* clang
* handling possible batched network failure
* BATCH as the constants device name in test
* ENABLE_BATCH
* func tests for CPU, also DetectionOutput hetero tests (CPU and GPU)
* DetectionOutput hetero test for the CPU
* reenabling the Auto-Batching in the AUTO
* auto-batching device enabled in the test
* fixed the DO test
* improve the loading loop logic
* brushed the config keys
* allow hetero code-path for explicit device name like BATCH:GPU(4), used in the hetero code-path tests
* fix the test after refactoring
* clang
* moving ThreadSafeQueue to the ie_parallel, as it is re-used in the AUTO/MULTI and BATCH now
* auto-batching hetero test (subgraph with DetectionOutput)
* fixed minor changes that were result of experiments with impl
* code-style
* brushing, disabling CPU's HETERO tests until planned activity for 22.2
* removing home-baked MAX_BATCH_SZIE and swicthing to the official impl by GPU team
* remote blobs tests for the auto-batching (old API)
* brushed names a bit
* CreateContext and LoadNEtwork with context for the Auto-Batching plus remote-blobs tests
* fixed the ieUnitTests with adding CreateContext stub to the MockICore
* clang
* improved remote-blobs tests
* revert the back BA from exeprimenents with AB + device_use_mem
* conformance tests for BATCH, alos batch size 1 is default for BATCH:DEVICE
* remote blobs 2.0 tests, issue with context having the orig device name
* debugging DG1 perf drop (presumably due to non-fitting the device-mem)
* disbaling WA with batch/=2 for excesive mem footptint, leaving only streams 2
* remote blobs 2.0 tests for different tensor sharing types
* converting assert to throw to accomodate legacy API where the lock() was possible to be called
* revert the timeout back to avoid mixing the studies, fixed the footprint calc
* reverting to estimating the max batch by extrapolating from bacth1 size
* more conservative footptint etimation (with bacth1), graceful bacth 1 handling without duplication
* even graceful batch 1 handling without duplication
* WA for MAX_BATCH_SIZE failure, removing batch4 as a min for the auto-batching
* AutoBatchPlugin -> ov_auto_batch_plugin
* WA for gcc 4.8
* clang
* fix misprint
* fixed errors resulted from recent OV's Variant to Any transition
* skip auto-batching for already-batched networks
* AUTO_BATCH_TIMEOUT and tests
* GPU-specific L3
* switched to pure config, also improved ALLOW_AUTO_BATCHING config key handling logic
* debugging device info
* enabling the config tests for the GPU and fixing the Auto-batching tests to pass
* making the default (when not recognized the driver) cache size more aggressive, to accomodate recent HW with old drivers
* skip auto-batching for RNNs and alikes (e.g. single CHW input)
* fixed fallback to the bacth1 and moved HETERO path under condition to avoid bloating
* brushing
* Auto plugin GetMetric support gpu auto-batch
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* add test case
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* add comments on test
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* brushing the vars names, alos adding the excpetion handling
* disabling the auto-batching for the networks with non-batched outputs and faster-rcnn and alikes (CVS-74085) to minimize the of #failures
* add try catch
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* brushing the code changed in the GPU plugin
* Auto-Batch requests tests
* brushed varibles a bit (ref)
* cleaned debug output from the ie_core
* cleaned cmake for the Auto-Batch
* removed batchN estimation from batch1
* cleaned from debug printf
* comments, cleanup
* WA the mock test errors introduced with merging the https://github.com/myshevts/openvino/pull/13
* Adding back removed batchN estimation from batch1 to debug degradations on DG1 (resulted from too optimistic MAX_BATCH_SIZE?). This partially reverts commit e8f1738ac1.
* brushing ie_core.cpp
* fix 32bit compilation
* Code review: ENABLE_AUTO_BATCH
* consolidate the auot-batching logic in ie_core.cpp into single ApplyAutoBAtching
* renamed brushed the OPTIMAL_BATCH (now with_SIZE) and mimicks the MAX_BATCH_SZIE wrt MODEL_PTR
* default value for the OPTIMAL_BATCH_SIZE
* clang
* accomodate new func tests location
* fix shuffle of headers after clang + copyrights
* fixed misprint made during code refactoring
* moving the common therad-safe containers (like ThreadSafeQueue) to the dedicated dev_api header
* switch from the device name to the OPTIMAL_BATCH_SIZE metric presence as a conditin to consider Auto-Batching
* switching from the unsafe size() and minimizing time under lock
* code style
* brushed the ApplyAutoBatching
* brushed the netric/config names and descriptions
* completed the core intergration tests for the auto-batching
* ExecGraphInfo and check for incorrect cfg
* removed explicit dependencies from cmake file of the plugin
* disabling Auto-Batching thru the tput hint (to preserve current product default), only excplicit like BATCH:GPU used in the tests
Co-authored-by: Roman Lyamin <roman.lyamin@intel.com>
Co-authored-by: Hu, Yuan2 <yuan2.hu@intel.com>
* Moved Speech sample to OpenVINO 2.0
* improved samples's score
* changed code style
* added GNA configs
* renamed function due to new API
* added dynamic batch
* reordered includes
* speech_sample has more than 1 input
* added oname processing
* added multi input
* fixed notes
* removed getFullDeviceName with old api
* getFullDeviceName for benchmark
* Merged and compiling
* Fix for dynamic shape type
* review fixes
* renamed blob shape to tensor shape, small improvements
* fix code style
* added parsing of multiple shapes
* store latency per group, add isIdleRequestAvailable() to Infer Queue
* added cached random inputs
* redesign pipeline, added new metrics(avg, max, min), added metrics per groups
* fixed code style
* small improvements
* modified tensor parameters parsing
* modified -i parameter parsing: added possibility to specify input names
* implemented image cashing
* added cashed blobs creating
* added -pcseq flag, modified batch filling, changes fps formula
* improvements
* code formatting
* code formatting2
* apply suggestions from review
* replaced Buffer class with InferenceEngine Blobs
* use batch size in blobs filling
* added shared blob allocator to handle blob's data
* fixed warnings & code style
* allocate blobs
* fix for networks with image info input
* added comments & fixed codestyle
* clear data in free() in SharedBlobAllocator
* remove unnecessary check
* Delimeter is changed to ::
* stylefix
* added layout from string function, small improvements
* modified parsing to enable : in input parameters
* small fixes
* small fixes
* added missed blob allocation, fixes
* [TEST]added support for remote blobs
* fix remote blobs
* new inputs/files output format
* removed vectors resize which caused bugs
* made cl::Buffer type under ifdef, fix inputs filling
* changed batch() function to not throwing exceptions
* removed unused var
* fix code style
* replace empty name in input files with name from net input
* restored old behaviour for static models
* fix code style
* fix warning - made const iterator
* fix warning - remove reference in loop variable
* added random and image_info input types to -i, fix problem with layout
* replaced batch() with getBatchSize() in main
* fix layout, shape, tensor shape parameters parsing
* upd help messages for input, tensor shape and pcseq command
* added buffer for cl output blobs, small fixes
Signed-off-by: ivikhrev <ivan.vikhrev@intel.com>
* added legacy mode
* restore setBlob
* code style formatting
* move collecting latency for groups under flag
* removed not applicable layouts
* added hint to error message when wrong input name in -tensor_shape was specified
* added new metrics to statistics report
* Apply suggestions from code review
* fix binary blobs filling when layout is CN
* apply suggestions
* moved file in the right place after rebase
* improved -pcseq output
* updated args and readme
* removed TEMPLATE plugin registration
* fix -shape arg decsription
* enable providing several -i args as input
* renamed legacy_mode to inference_only and made it default for static models, renamed tensor_shape to data_shape
* upd readme
* use getBlob() in inference only mode
* fix old input type for static case
* fix typo
* upd readme
* move log about benchmark mode to the measuring perfomance step
* added class for latency metrics
* upd readme, fix typos, renamed funcs
* fix warning and upd parsing to avoid error with : in file paths
* fix error on centos : error: use of deleted function ‘std::basic_stringstream<char>::basic_stringstream(const std::basic_stringstream<char>&)
* added check for key in inputs
* renamed input to inputs
* adjust batch size for binary blobs
* replaced warning with exception in bench mode defining
* align measurement cycle with master
Co-authored-by: ivikhrev <ivan.vikhrev@intel.com>
* ov2.0 IE samples modification
apply code style
turn off clang style check for headers order
unify samples a bit
add yuv nv12 reader to format_reader, helloe_nv112 sample
hello_reshape_ssd ov2.0
* sync with PR 8629 preprocessing api changes
* fix for slog << vector<int>
* add operator<< for ov::Version from PR-8687
* Update samples/cpp/hello_nv12_input_classification/main.cpp
Co-authored-by: Mikhail Nosov <mikhail.nosov@intel.com>
* apply code style
* change according to review comments
* add const qualifier
* apply code style
* std::ostream for old inference engine version to make VPU plugin tests happy
* apply code style
* revert changes in print version for old api samples
* keep inference_engine.hpp for not ov2.0 yet samples
* fix merge artifacts
* fix compilation
* apply code style
* Fixed classification sample test
* Revert changes in hello_reshape_ssd sample
* rebase to master, sync with PR-9054
* fix issues found by C++ tests
* rebased and sync with PR-9051
* fix test result parsers for classification tests (except unicode one)
* fix mismatches after merge
* rebase and sync with PR-9144
Co-authored-by: Mikhail Nosov <mikhail.nosov@intel.com>
Co-authored-by: antonrom23 <anton.romanov@intel.com>
* OV2.0 API C++ hello_reashe_ssd sample
* clean header
* fix test for changed sample cmd line
* adopt to PR-8898
* sync with PR-9054, simplify code
* apply code_style.diff
* sync with PR-9051
* Renamed ov::Function to ov::Model
* Fixed all for macos
* Fixed build
* Fixed build
* Revert changes in GPU plugin
* Fixed ngraphFunctions
* Fixed all for mac
* Fixed new test
* Fixed if for Windows
* Fixed unit tests and renamed Function in python API
* Fixed code style
* Fixed import
* Fixed conflict
* Fixed merge issues
* Initial version (no tests)
* Added tests
* Fix centos
* Applied review comments
* Renamed 'ov::util::get_batch_size' to 'ov::pass::get_batch'. For set_batch_size update is the same
* Changed to ov::get_batch and ov::set_batch
* Removed 'inline' Preprocessing API
Even though this API provided a way to specify all pre/post-processing in one line - it was considered inconvinient
With 'getters' API preprocessing code looks more clear for user, so old' inline' API is removed
* Fix pyopenvino build issues
* Update after merged PR#8717
* Use ninja multu-config
* Reused OV_GENERATOR_MULTI_CONFIG
* Try to use config in api_validator post build task
* Fixes
* Revert "[GPU] Fix multi configuration build of onednn_gpu (#8743)"
This reverts commit a862331732.
* Propogate disable deprecated for _RELEASE flags as well
* Added samples smoke tests
Added implementation into azure pipeline
Update copyrights
Added python path
Update common part
modify windows.yml
* resolve conflicts
* New redesign of samples_smoke_tests including adding CMakeLists in order to install smoke tests on system, getting of smoke_tests_data archive
* fixed windows.yaml
* added checking path of samples
* changed path to INSTALL_DIR
* Fixed windows.yml file
* changed IE_APP_PATH
* Updated readme file for smoke tests
* Disable run tests on linux static build
* skipped test with related issue 71298
* change after review
Co-authored-by: Maxim Gordeev <maxim.gordeev@intel.com>
Wang, Yang