openvino/tests/samples_tests/smoke_tests
Przemyslaw Wysocki d2871fd52c
[PyOV] Enable Python 3.10 on Azure CI (#12578)
* Fix mypy for numpy==1.21.6

* Apply CR changes

* Allow Python 3.6 for open source

* Attempt to silence mypy

* Revert to 3.6 in setup.pys

* Fix merge conflict

* Add newline

* Fix Python version assert

* Test Python 3.10 CI

* Try bumping up pybind to 2.10

* Link default python to python 3.10

* Add sudo to changing default python

* Bump python version in setupvars

* Fix exception in test

* Do the same for compatibility

* Apply same CI changes to other pipelines

* Attempt to fix arm and windows CIs

* Fix win py lib version and links?

* Try to fix CI

* Fix win python path

* Update win path

* Test disable PDPD

* Disable TF for testing CI

* Add sudo to rm

* Test linux

* Update requirements

* RM sudo

* Add debug step to CI on ARM

* build py from source on linux

* Remove debug step from ARM

* Fix err

* Fill num_proc

* Fic py installation

* Bump pdpd to 2.3.1

* Bump np vers

* Try to upgrade onnx to 1.12.0

* Revert onnx upgrade, enable frontends

* Build py shared

* Export library

* activate py env

* Bump scipy for py3.10

* Another scipy bump

* Fix Windows path, add nfs helper for mount

* Attempt to fix win_cc

* Batch of changes

* Try to fix ngraph-onnx-lin

* Install lsb-release for docker on ngraph-onnx-lin

* Link lsb_release

* Try bumping Ubuntu to 22.04

* Revert ubuntu change in dockerfile

* Refactor tests for onnx1.12

* Fix ONNX compatibility and numpy warnings

* Refactor tests for onnx1.12

* Fix ONNX compatibility and numpy warnings

* Upgrade ONNX to 1.12 in cpp

* Skip unsupported ops

* Revert "Fix ONNX compatibility and numpy warnings"

This reverts commit cd8a0e6b8e.

* Small changes

* Remove unnecessary changes

* Bump Python version in github workflows

* Fix flake error

* Another flake error

* Even more flake errors

* Xfail more tests

* Fix mypy errors

* Change onnx version in tests

* Bump ONNX in MO

* Fix onnx utils in comp

* Minor changes

* Import pathlib

* Add xfails to compatibility tests

* Skip segfaulting tests

* Fix onnx nametuple

* Fix flake

* Try to fix py3.6 onnx

* Switch to venv in windows

* Fix model importer and reqs

* Add __test__

* Debug print

* Fix flake

* Update linux.yml

* Update windows.yml

* Update linux.yml

* Update windows.yml

* Install python3.10

* Update windows.yml

* Update windows.yml

* Update windows.yml

* Update windows.yml

* Update windows.yml

* Update windows.yml

* Update windows.yml

* Update windows.yml

* Install py embedded

* Fix dir

* Fix curl

* Add complete file

* CP file

* Bump py ver

* Fix missing venv

* Try to install

* Disable venv

* Bump OMZ commit

* checkout omz

* Revert omz

* Bump omz

* Revert pybind bump

* Update .gitmodules

* Change pybind to fork

* change ssize_t to size_t

* Install bz2 for py build

* Bump omz

* Bump pandas requirement version

* Revert omz

* Install missing clang

* Bump omz

* Bump versions on fork

* Bump numpy in model zoo

* Fix clang

* Skip segfaulting pdpd test

* Skip another pdpd test

* Skip pdpd places.cpp

* Bump py ver in setupvars.bat

* Disable paddle tests

* Use int instead of str

* Fix requirements path

* Enable lin-cc

* Fixes in CI

* lin-cc and ngrpah-onnx changes

* Update num_proc

* Update ngraph-onnx

* Minor change

* Fix env var

* Change ubuntu to 20

* Get compiler on ngraph-onnx

* Add sudo to apt-get

* Add SSL

* Download ssl

* Different way of installing ssl

* Minor changes

* install zlib

* Fix env var

* Fix docker building error

* Revert changes

* Modify dockerfile

* Change pychecks python version

* Cleanup

* Add LayerNorm to expanded ONNX functions

* Revert previous commit

* Add LayerNorm to expanded ONNX functions

* Apply clang

* Move LayerNorm tests to skip_segfault

* Unskip pdpd tests

* Enable pdpd tests on linux

* Refactoring

* Remove 3.6 from reqs

* Refactoring

* Minor changes

* Minor changes

* Relax MO reqs

* Bump torch version

* relax pytest

* Specify pytest version

* Fix pytest pip conflict

* Bump pybind11 to v2.10.1 release tag

* Specify separate pytest version for py3.10

* Bump attrs

* Add py dependency

* Refactor versions

* Add venv activation to tf tests

* Fix fdupes

* Bump OMZ submodule

* Remove find and replace

* Change pybind to upstream fork

* Set pybind to proper commit

* Minor changes

* Try to fix scipy

* Updated exclude patterns for nlohmann json

* [PyOV] Fix wheel requirements for new package versions

* Specify CPU plugin versions

* Bump OMZ version

* Fix linter

* Fix linter 2

* Fix linter 3

* Disable linter

Co-authored-by: Mateusz <mateusz.mikolajczyk@intel.com>
Co-authored-by: jiwaszki <jan.iwaszkiewicz@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2022-11-08 11:38:39 +04:00
..
common Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
README.md ov2.0 cpp hello reshape ssd (#8874) 2021-12-10 15:58:23 +03:00
conftest.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
env_config.yml Fix tests after update archive with models (#9063) 2021-12-08 14:53:18 +03:00
requirements.txt [PyOV] Enable Python 3.10 on Azure CI (#12578) 2022-11-08 11:38:39 +04:00
test_benchmark_app.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
test_classification_sample_async.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
test_hello_classification.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
test_hello_nv12_input_classification.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
test_hello_query_device.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
test_hello_reshape_ssd.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00
test_speech_sample.py Conda-forge fixes (#13392) 2022-10-12 12:16:13 +04:00

README.md

These tests execute IE samples on pregenerated IR

<INSTALL_DIR> - OpenVINO install directory

You can run tests not only from the <INSTALL_DIR>, but in this case you need to remember to adjust the environment variables like as WORKSPACE and SHARE

To install smoke tests: bash - cd <working directory>/tests/samples_tests/smoke_tests - mkdir build && cd build - cmake ../.. - cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR> -P cmake_install.cmake

  1. To run tests from install directory required installing some dependencies:
    • pip3 install -r <INSTALL_DIR>\tests\smoke_tests\requirements.txt
  2. Call setupvars script and then set the environment variables: a. Required:
    • IE_APP_PATH : coomon path to C++ and C samples, e.g. '<INSTALL_DIR>/samples_bin'
    • IE_APP_PYTHON_PATH : path to python IE samples, e.g. '<INSTALL_DIR>/samples/python/'
    • IE_APP_PYTHON_TOOL_PATH : path to python IE tools for benchmark_app, e.g. '<INSTALL_DIR>/tools/' b. Optional:
    • TEST_DEVICE = CPU by default
  3. Configure env_config.yml according to your paths:
    • Set WORKSPACE : working directory, e.g. '<INSTALL_DIR>'
    • Set SHARE : path to loaded data with models, e.g. '<INSTALL_DIR>/tests/smoke_tests/samples_smoke_tests_data/'
  4. Run all test via pytest:
    • python -m pytest --env_conf env_config.yml -s
  5. Run only one sample (for example, classification_sample_async):
    • python -m pytest test_classification_sample_async.py --env_conf env_config.yml -s
  6. To run performance add pytest key: "performance n", where n is number of perf iteration. Test finds in output of sample 'fps', if it exists, then tests rerun that sample adding key 'niter n' with number of perfomance run (that you passed to pytest with '--performance n' keys) Not to add 'niter' key, please, execute pytest "--performance 0"

This test using pregenerated IRs, that located right now in shared folder mentioned above. Also data (images, videos and others) locates in that shared folder.