Commit Graph

35 Commits

Author SHA1 Message Date
Jacek Pawlak 239466ca5d
copyright year update (#23370)
New PR due to merge
conflicts(https://github.com/openvinotoolkit/openvino/pull/22917)

Updated the copyright year from 2018-2023 to 2018-2024 in all openvino
files

Ref. to script: CVS-101144

Command used:
```bash
git grep -lz '2018-2023 Intel Corporation' | xargs -0 sed -i '' -e 's/2018-2023 Intel Corporation/2018-2024 Intel Corporation/g'
```
2024-03-14 09:37:02 +00:00
OK f2017e8c2e
Validation: Test AUTO plugin timetests (#17508)
* Validation: Test AUTO plugin

* config

* Add debug logs against AUTO

* Iteration

* iteration

* iteration

* iteration

* iter

* iteration

* iteration

* iteration

* iteration

---------

Co-authored-by: Daria Ilina <daria.ilina@intel.com>
2023-05-31 15:38:57 +02:00
Bogdan Pereanu 73ab0dd065
Fixing run_timest python script for input and output precision (#16661)
* Fixing run_timest python script for input and output precision

* Update code according to the PR review

* Update run_timetest according to the last review

* Add input_precision and output_precision to test_timetest as well

* Set input/output precision per model
2023-04-05 12:16:27 +04:00
Bogdan Pereanu 5eea99d96c
Update timetest tool to support ip and op params config (#15916)
* User can set input and output precision for timetest tool

* Update run_timetest.py with the ip and op options as well

* Use only one getType function

* Add extra line at the end of the file

* Remove unused parameters

* Update comment accordingly

---------

Co-authored-by: Vitaliy Urusovskij <vitaliy.urusovskij@intel.com>
2023-03-30 11:27:51 +04:00
Przemyslaw Wysocki 591c3e61c5
[PyOV] Simplify requirement files (#15343)
* Partial progress

* Finish v1

* Cleanup

* Remove useless files

* Fix path to pdpd

* Fix onnx path

* Minor change

* Rework MO

* Minor change

* Remove some costraints

* Add MO constraints

* Update gitignore for MO

* Minor change

* Apply tech sync discussion

* Cleanup

* CR comment

* Debug ONNX FE

* simplify ONNX FE

* Update cmake

* Hardcode ONNX requirement

* Add dependency resolver to cmake

* Add constraints for openvino/tests

* Add missing pytest-html

* Fix -c path

* Revert debug changes to path

* Add cmake to copy constraints.txt

* Update dependabot

* Remove slash

* Remove cmake

* Debug prints

* Minor changes

* Move reqs check to separate file

* Add requirements parser to benchmark_tool

* Fix smoke tests constraints

* Minor fixes

* Minor change

* My fixes were apparently wrong

* Debug - self.executable_path

* Debug - add singledispatch to tests and tools

* Debug - print IE_APP_PATHs

* Revert "Debug - print IE_APP_PATHs"

This reverts commit 67ccb6d3f5.

* Revert "Debug - add singledispatch to tests and tools"

This reverts commit 3b945931e2.

* Revert "Debug - self.executable_path"

This reverts commit 3aa724eff6.

* update dependabot

* update dependabot

* Skip benchmark_app tests

* Use CMAKE_CURRENT_BINARY_DIR in cmake

* Remove debug prints

* minor change

---------

Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
2023-03-29 14:27:27 +04:00
Albert Angles 711e2cbab0
enable first_inference_latency (first_inference+time_to_inference) (#14829)
* add time_to_first_inference_result KPI as sum of inf latency values

* append unit time after the value

tested locally, worked

* Revert "append unit time after the value"

This reverts commit 428b8cafdc.

* remove metric

* revert 42dd271c3b

* enable { SCOPED_TIMER(first_inference_latency); }

* enable first_inference_latency

tested locally, it works

* remove trailing whitespace

Co-authored-by: Daria Ilina <daria.krupnova@intel.com>
2023-01-23 19:48:07 +03:00
Ilya Churaev 0c9abf43a9
Updated copyright headers (#15124)
* Updated copyright headers

* Revert "Fixed linker warnings in docs snippets on Windows (#15119)"

This reverts commit 372699ec49.
2023-01-16 11:02:17 +04:00
Smirnova Maria 5cf6ceaf83
Fix db push for stresstest and timetests (#13964)
* Fix db push

* Fix log info for new data

* Check if file exists

* Remove _id field

* Add error message print
2022-11-17 12:52:21 +01:00
Przemyslaw Wysocki 79d92b7d8d
[PyOV] Bump pyyaml (#14010) 2022-11-16 13:10:20 +03:00
Smirnova Maria 8f97ee55fc
Modify time tests so that data can be pushed to OVVP (#13636)
* Set db_api_handler for pushing to new database in timetests

* Change function of pushing to db.

Co-authored-by: Daria Ilina <daria.krupnova@intel.com>
2022-11-10 15:17:39 +01:00
Victor Kuznetsov 6f60a7d8f0
First time inference validation for dynamic shapes (#9671)
* add performance hint to time infer

* add reshape methods for time infer tool

* add func declarations to common_utils.h

* add check if node is static (reshape pipeline)

* changes in desc of reshape funcs

* fix logic operator

* add reshape utils.h | move parsing logic in main func

* add tests_shared_lib to CMakeLists.txt

* change fill blobs with setStaticShapesBlobs

* change delimiter
2022-01-24 16:19:03 +03:00
Alexey Suhov a79830cb55
Update year to 2022 in copyright notice (#9755) 2022-01-19 01:07:49 +03:00
Victor Kuznetsov e212a1ac83
[TimeTests] Disable VPU option, rename cache option (#9114)
* add performance hint to time infer

* Update time tests with options

* fix mistake in help

* update test conf with GPU models

* move ie configs from load plugin timer
2021-12-17 13:06:56 +03:00
Victor Kuznetsov 0f1c545a19
[Time tests] Change work with perf_hint (#9090)
* add performance hint to time infer

* upd_time_tests
2021-12-09 10:29:55 +03:00
Victor Kuznetsov 55f64a505a
Expand time tests to cover CPU caching feature (#8938)
* add performance hint to time infer

* add options for vpu, cpu cache, perf hint

* change VPUX condition

* add perf hint verify

* upd perf hint usage. set as non required
2021-12-08 15:22:44 +03:00
Victor Kuznetsov 6d9a7a108e
Stress, Memory tests update to comply with new OMZ tools layout (#7886)
* work with omz: change hardcoded paths for memory_tests, fix paths for memcheck get_testdata.py script

* change paths to omz tools

* revert mo changes

* remove mo from arguments

* remove extra var from cmd string

* remove is_db_used check

* add timeout to proc_exec

* merge master
2021-10-13 10:42:00 +03:00
Victor Kuznetsov 4d377901bf
Move all utils to common folder (#7303)
* move all utils to common folder

* add utils install in CMakeLists.txt

* revert plugin utils

* add log info: test

* add logging info: test

* test imports

* change imports: test

* fix file name
2021-09-07 23:02:10 +03:00
Michał Karzyński d748f2aa99
Remove references to prototxt from documentation and docstrings (#7346) 2021-09-03 08:32:43 +03:00
Vitaliy Urusovskij 31907e51e9
Fixed several issues in time_tests (#6481)
* Fixed return in `run_timetest`

* Fixed `parse_stats()` argument type notation

* Pass message from `timetest` exe to test error msg to be uploaded to DB
2021-07-01 16:37:23 +03:00
Victor Kuznetsov af915859e3
Add float instance to time_tests parse_stats (#5726) 2021-05-21 11:12:05 +03:00
Victor Kuznetsov c907fb9f52
Log have time measures nested (#5402) 2021-05-18 18:00:10 +03:00
Victor Kuznetsov 4790c79eb4
Add time_tests dir path to sys.path (#5498) 2021-05-06 14:57:50 +03:00
Alexander Shchepetov e864972f83
Add experiments timings to data to submit to DB(#5108) 2021-04-12 09:29:10 +03:00
Victor Kuznetsov 69a06f5977
Remove time-tests outliers using Inter Quartile Range (IQR) (#5014)
Change number of times to execute binary to aggregate statistics from 3 to 10.
2021-04-02 11:35:38 +03:00
Alexey Suhov 6478f1742a
Align copyright notice in python scripts (CVS-51320) (#4974)
* Align copyright notice in python scripts (CVS-51320)
2021-03-26 17:54:28 +03:00
Victor Kuznetsov 1da8f65b45
Add removing of statistics file in time_tests (#4275) 2021-02-11 12:02:39 +03:00
Victor Kuznetsov 73f846648c
change tempfile usage (#4245) 2021-02-10 10:23:31 +03:00
Andrey Somsikov 359c2ca018
Upgrade pyyaml to the latest (#4134) 2021-02-03 07:07:01 +03:00
Vitaliy Urusovskij 12b690952c
Time tests improvements (#2642)
* Remove extra functions from run_timetest.py

* Add `log.debug` of raw and aggregated statistics in run_timetest.py

* Implement storing of models locally for test_timetest.py
2020-10-14 20:05:52 +03:00
Vitaliy Urusovskij 474dcc277f
Prepare new test configuration for test_timetest.py (#2561)
* Add logging of DB upload to tconf with refs saving

* Prepare `.automation/tgl_test_config.yml` with models from VPUX package

* Add restriction on `exe` and `model` existence in run_timetest.py
2020-10-08 10:30:43 +03:00
Vitaliy Urusovskij c3ff2948a4
Timetests test_runner improvements (#2552)
* Remove `generate_tmp_path` as unnecessary after refactoring

* Add `check_positive_int` check for `-niter` CLI key

* Replace `TestConfDumper` with number of fixtures:
1. Save all test info in global `request` and `pytestconfig` fixtures
2. Add `test_info` fixture for retrieving test info from test
3. Add `prepare_tconf_with_refs` fixture for test conf dump
2020-10-07 17:15:02 +03:00
Vitaliy Urusovskij 6949d93e42
Add `"cl_cache"` preparation in test_timetest.py (#2490) 2020-10-06 14:16:50 +03:00
Vitaliy Urusovskij 639052d0b8
Support references in test_timetest.py (#2431) 2020-10-01 11:35:57 +03:00
Vitaliy Urusovskij d48d2109dc
Test timetest MVP (#2345) 2020-09-21 21:33:42 +03:00
Vitaliy Urusovskij 20266276ec
New layout for `time_tests` (#2321)
* Change folders' layout for `time_tests`

* Refactoring of namings in `time_tests`
2020-09-18 18:53:49 +03:00