* 1. refine the logic to ov::device::properties setting.
2. the config overrides will be performed if same config setting is came from CMD line.-a
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update configuration sample file within README.md.
* Update.
* Update.
* 1. Update configuration example file within REAMDME.md for Python version.
2. implement the config DEVICE_PROPERTIES value convertation between the string type and dictionary of Python type.
3. Update the configuration file loading and dumping logic.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update.
* Update.
* Update.
* Update.
* 1. Enable configs to be interchangeable between C++ and Python.
2. Update perf_count showing logic.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert the logic of showing show performance counters.
* Update help msg for loading config option.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update ov::hint::performance_hint UNDEFINED value from empty string to "UNDEFINED".
* Update benchmark Python version.
* Update.
* Update.
* Update.
* Update the description about hint setting within benchmark APP README and help message.
* benchmark_app: factor out advanced options
* cpp/benchmark_app: add usage word
* move api to advanced, group b/shape/data_shape/layout into Input shapes, factor out Statistics dumping options
* Factor out Device-specific performance options
* Factor out Preprocessing options
* Minor regroup
* [C++/BENCHMARK_APP] Introduce Numpy array loading for C++ benchmark app
* [DOCS/BENCHMARK_APP] Update docs to reflect changes, update list of available extensions from OpenCV, align help messages
* Update inputs_filling.cpp
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/inputs_filling.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update samples/cpp/benchmark_app/inputs_filling.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [C++/Python] Implement quality-of-life improvements from PR comments
* [C++] Fix compilation errors, fix linter output
* [C++/PYTHON] Apply requested changes
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update samples/cpp/benchmark_app/utils.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON] Separate loading of numpy arrays similar to images
* [PYTHON] Remove unnecessary 'Prepare xxx file' print
* Update README again because IF OPENCV.. dissapeared for some reason
* Update second README with missing IF OPENCV..
* [C++] Remove unnecessary vector print function
* [C++ Add Numpy processing function - TODO link it to the tensor filling
* Reverse OneDnn plugin modification
* [C++] Numpy array loading for C++
* [C++] Add (almost) all missing types of data
* Reverse submodule modifications
* [C++/PYTHON] Fix compilation errors, clean code
* [C++] Modify supported extensions, add numpy checking to utils, add numpy to get_image_info method
* Update samples/cpp/benchmark_app/inputs_filling.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [C++] Fix utils header file to reflect unordered set change
* [PYTHON/C++] Fix compilation errors in C++ code, fix Python dynamic shapes numpy loading
* [C++] Fix explicit instantiation of NumpyArray reader
* [C++] Clang format, minor syntax fixes
* [PYTHON/C++] Remove unnecessary data types, introduce a new approach to cast data of different types from format_rt_reader, remove uppercase types from Python precision parameters
* [PYTHON] Update README to reflect new precision settings
* [PYTHON] Fix README, fix clang format
* [C++] Clean headers
* [C++] Fix uninitialized variable error
* [C++/PYTHON] Fixed choices in Python benchmark, fixed types in C++ benchmark
* [C++] Fixed ov::float16conversion, fixed Python types map - removed redundancies
* [C++] Add back boolean support
* [C++] Fix compilation errors
---------
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* replace benchmark_app imean and iscale parameters
Tickets 80777 and 80900
* Warn about difference with imean and iscale, note that in help
* Clang format
* Clang format
* mean->scale
* Keep mean scale empty if not set
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* [AUTO] update property setting logic.
* Update core::set_property() logic to only support primary property for AUTO and MULTI.
* Separate AUTO and MULTI supported properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. revert the changes for ie core.
2. Enable AUTO/MULTI only accepting its own properties.
3. Enable AUTO/MULTI accepting device properties passed from loadnetwork().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert "[AUTO]Update the incorrect config test for Myriad (#13271)"
This reverts commit 0552d98802.
* MULTI only accepts its own properties that is same as AUTO currently.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test cases for AUTO/MULTI property test.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update to enable MULTI supporting hw primary property setting throw the compile_model().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove the added test case for setting secondary property through set_property().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. For AUTO/MULTI, property setting will be passed via core::compile_model() instead of core::set_property().
2. update the logic to infer precision setting that will transform into secondary property setting to each hw device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* set default value for nstreams when -d AUTO/MULTI and no nstreams setting from command line.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update code format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Enable BA to support if -d AUTO:MULTI,xxx/MULTI:AUTO,xxx. while AUTO Plugin need to update the logic of generating supported config list to virtual device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* move device configuration handling outside form function main.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert "move device configuration handling outside form function main."
This reverts commit ef77bfc602.
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Setting property performance hint to 'THROUGHPUT' and passing to executable network if no setting for of AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* 1. Fix the issue that the dumped configuration file retrieves empty config list when using OpenCV to dump configuration.
2. Enable load_config() to load secondary property.
3. ov:device:properties from command line will replace the ov::device::properties loaded from configuration file.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update the performance statistics showing format.
* Update format.
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update README to add some simple configuration files loaded into benchmark app.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable dump/load function for AUTO to support hw device property setting thorugh configuration JSON.
2. Update README.md to example a simple JSON file containing the hw device properties setting for AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update.
* Update.
* Drop the changes within dump/load of OpenCV config parsing.
* Update.
* Correct property name. Update ov::num_stream to ov::inference_precision.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* [PYTHON] Pipeline transfer
* [PYTHON] Align python benchmark
* [PYTHON] Align last step
* [PYTHON] Fix innacuracies of the last step - median
* [PYTHON/CPP] Add Core::get_version method to python API, refactor Ben benchmark to print version with this func
* [PYTHON] Remove get_version_string from API
* [PYTHON/CPP] Align output for model input/output info
* [PYTHON/CPP] Step 4,6 alignment of outputs, step 8 dumps all info stored in config parameters
* [CPP] Fix a bug causing nstreams parameter to never be set to AUTO in CPP benchmark_app
* [CPP] Fix clang format errors
* [CPP] Modify print order and data output for 8th step
* [PYTHON] Add verification checks from C++, modify set_thoughtput_streams to match documentation
* [CPP] Revert changes to C++ benchmark_app
* [CPP] Remove additional spacebar
* Update submodules versions on remote
* Update module from master branch
* Redownload submodules from master and override changes from commit
* [PYTHON] Remove unneccesary parse_status from validation function
* [PYTHON] Check for HINT in map, fix circular import
* [PYTHON] Remove artifacts from commit, fix args.perf_hint set to '' instead to 'none'
* [PYTHON] Reverse changes to perf hint, add key in map check, fix validation function throwing error on set hint
* [PYTHON] Fix linter
* [PYTHON] Remove linter spacebar
* [CPP] Fix wait_all exception throw
* [CPP/PYTHON] Clean artifacts and unwanted changes from work process
* [PYTHON] Fix artifacts from merge, clean submodule update
* [C++ CPU] Fix device name string by removing padding NULL characters from the back
* [CPP] Fix ba infer_request_wrap in other throw-catch clauses
* [PYTHON/CPP] Fix missing latencies in final step for shape group, fix minor misaligned messages, add missing report parameter create infer requests time
* [CPP] Clang fix formatting
* [CPP] Reverse clang fix format on plugin.cpp
* [PYTHON/CPP] Fix C++ progressbar printing endl when disabled, fix rounding in python creating infer request message
* [CPP] Fix foramtiing error
* [PYTHON/C++] Refactor network to model based on naming conventions, provide fresh README output example
* [PYTHON/C++] Add example output to C++ README, remove unnecessary device loop
* [BENCHMARK_APP/C++] Fix artifact from refactoring, remove try-catch clause
* Update samples/cpp/benchmark_app/benchmark_app.hpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* [CPP] Fix clang errors
* [CPP/PLUGIN Reverse modification to extract to separate task
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/parameters.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON/C++/BENCHMARK_APP] Fix language inconsistencies, remove unnecessary checks
* Update pyopenvino.cpp
* [CPP/BENCHMARK_APP] Remove unnecessary try-catch, fix linter errors
* [PYTHON/CPP/BENCHMARK_APP] Revert changes to Core, align version prints usin only provided methods
* [DOCS/BENCHMARK_APP] Update README with proper model examples
* Update README.md
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Improvements in rpm / debian build
* Fixed several debian warnings
* Supported old gflags from CentOS 7
* Reverted back OpenCV version
* Fixed clang-format
* Benchmark_app python and c/c++ script add op profiling feature
* Fix compile bug about benchmark c/c++ op profiling
* Fix Compile Bug
* Fix issue in PR#12350 : 1. report_type info update; 2. priority of -pc and -pcsort; 3.align format between c++ and python; 4.check 'proportion' for 0 and print N/A; 5.detect the confusing print info
* Fix 8/17 review suggestion
* Fix 8/23 suggestion
* Fix the clang-format issue
* Fix the win-cc issue
* Fix win-cc issue about conversion from 'double' to 'float', possible loss of data
* Push PR by ODT group account
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Fiona Zhao <fiona.zhao@intel.com>
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* revert test code
* revert test code
* enable brgconv f32
* use config to enable brgconv f32
* when brg disabled not init bin-postops
* change prop name for extensive
* use more general field
* fix review comments.
* selectdevice returns MULTI:device in cumulative_throughput
* load multi with throughput and disable cpu helper in cumulative
* disable cpu helper in cumulative_throughput
* add cumulative to bechmark_app help message
* modify benchmark_app.hpp clang-format
* Used new config for streams and threads
* Fixed review coments in ba
* format fix
* fixed hello_query_device
* Added STL string io
* fixed tests
* Fixed test
* Fixed build
* fixed format
* Fixed build
* try fix win
* other any io specialization
* Fixed after merge
* renamed streams
* build fixed
* fixed build
* fixed format
* fix for old mac build
* Fixed type of exception
* test fix
* Refactored statistics output with JSON support
* Detailed/average reports are added
* stylefix
* Update samples/cpp/benchmark_app/statistics_report.hpp
Co-authored-by: Ivan Vikhrev <ivan.vikhrev@intel.com>
* Linux Fixes
* stylefixes
* data_shape field format is changed
* stylefix
Co-authored-by: Ivan Vikhrev <ivan.vikhrev@intel.com>
* [IE SAMPLES] activated NCC tool for c++ samples
* exclude ov_ncc_naming_style for tests
* fixed NCC hit
* Added support for source files in samples
* changed style of methods for benchmark
* changed style for speech sample
* changed code style
* changed code style for shared_tensor
* benchmark changes
* changed remote_tensors_filling
* fixed notes
* rebase of branch
* Merged and compiling
* Fix for dynamic shape type
* review fixes
* renamed blob shape to tensor shape, small improvements
* fix code style
* added parsing of multiple shapes
* store latency per group, add isIdleRequestAvailable() to Infer Queue
* added cached random inputs
* redesign pipeline, added new metrics(avg, max, min), added metrics per groups
* fixed code style
* small improvements
* modified tensor parameters parsing
* modified -i parameter parsing: added possibility to specify input names
* implemented image cashing
* added cashed blobs creating
* added -pcseq flag, modified batch filling, changes fps formula
* improvements
* code formatting
* code formatting2
* apply suggestions from review
* replaced Buffer class with InferenceEngine Blobs
* use batch size in blobs filling
* added shared blob allocator to handle blob's data
* fixed warnings & code style
* allocate blobs
* fix for networks with image info input
* added comments & fixed codestyle
* clear data in free() in SharedBlobAllocator
* remove unnecessary check
* Delimeter is changed to ::
* stylefix
* added layout from string function, small improvements
* modified parsing to enable : in input parameters
* small fixes
* small fixes
* added missed blob allocation, fixes
* [TEST]added support for remote blobs
* fix remote blobs
* new inputs/files output format
* removed vectors resize which caused bugs
* made cl::Buffer type under ifdef, fix inputs filling
* changed batch() function to not throwing exceptions
* removed unused var
* fix code style
* replace empty name in input files with name from net input
* restored old behaviour for static models
* fix code style
* fix warning - made const iterator
* fix warning - remove reference in loop variable
* added random and image_info input types to -i, fix problem with layout
* replaced batch() with getBatchSize() in main
* fix layout, shape, tensor shape parameters parsing
* upd help messages for input, tensor shape and pcseq command
* added buffer for cl output blobs, small fixes
Signed-off-by: ivikhrev <ivan.vikhrev@intel.com>
* added legacy mode
* restore setBlob
* code style formatting
* move collecting latency for groups under flag
* removed not applicable layouts
* added hint to error message when wrong input name in -tensor_shape was specified
* added new metrics to statistics report
* Apply suggestions from code review
* fix binary blobs filling when layout is CN
* apply suggestions
* moved file in the right place after rebase
* improved -pcseq output
* updated args and readme
* removed TEMPLATE plugin registration
* fix -shape arg decsription
* enable providing several -i args as input
* renamed legacy_mode to inference_only and made it default for static models, renamed tensor_shape to data_shape
* upd readme
* use getBlob() in inference only mode
* fix old input type for static case
* fix typo
* upd readme
* move log about benchmark mode to the measuring perfomance step
* added class for latency metrics
* upd readme, fix typos, renamed funcs
* fix warning and upd parsing to avoid error with : in file paths
* fix error on centos : error: use of deleted function ‘std::basic_stringstream<char>::basic_stringstream(const std::basic_stringstream<char>&)
* added check for key in inputs
* renamed input to inputs
* adjust batch size for binary blobs
* replaced warning with exception in bench mode defining
* align measurement cycle with master
Co-authored-by: ivikhrev <ivan.vikhrev@intel.com>
* ov2.0 IE samples modification
apply code style
turn off clang style check for headers order
unify samples a bit
add yuv nv12 reader to format_reader, helloe_nv112 sample
hello_reshape_ssd ov2.0
* sync with PR 8629 preprocessing api changes
* fix for slog << vector<int>
* add operator<< for ov::Version from PR-8687
* Update samples/cpp/hello_nv12_input_classification/main.cpp
Co-authored-by: Mikhail Nosov <mikhail.nosov@intel.com>
* apply code style
* change according to review comments
* add const qualifier
* apply code style
* std::ostream for old inference engine version to make VPU plugin tests happy
* apply code style
* revert changes in print version for old api samples
* keep inference_engine.hpp for not ov2.0 yet samples
* fix merge artifacts
* fix compilation
* apply code style
* Fixed classification sample test
* Revert changes in hello_reshape_ssd sample
* rebase to master, sync with PR-9054
* fix issues found by C++ tests
* rebased and sync with PR-9051
* fix test result parsers for classification tests (except unicode one)
* fix mismatches after merge
* rebase and sync with PR-9144
Co-authored-by: Mikhail Nosov <mikhail.nosov@intel.com>
Co-authored-by: antonrom23 <anton.romanov@intel.com>
Zlobin Vladimir