openvino/tests/samples_tests/smoke_tests

These tests execute OV samples on pregenerated IR. Test data is downloaded on first run into a cache dir generated by pytest.

<INSTALL_DIR> - OpenVINO install directory

You can run tests not only from the <INSTALL_DIR>, but in this case you need to remember to adjust the environment variables like as WORKSPACE (for Test_model_creation_sample) and SHARE

To install smoke tests:

cd <working directory>/tests/samples_tests/smoke_tests
mkdir build && cd build
cmake ../..
cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR> -P cmake_install.cmake

1. To run tests from install directory required installing some dependencies:

    pip3 install -r  <INSTALL_DIR>\tests\smoke_tests\requirements.txt
   

2. Call setupvars script and then set the environment variables: a. Required:
   • IE_APP_PATH : coomon path to C++ and C samples, e.g. '<INSTALL_DIR>/samples_bin'
   • IE_APP_PYTHON_PATH : path to python OV samples, e.g. '<INSTALL_DIR>/samples/python/'
   • Install openvino-*.whl to provide Python benchmark_app executable b. Optional:
   • TEST_DEVICE=CPU;MULTI:CPU;AUTO by default
3. Run all test via pytest:

    python -m pytest
   

4. Run only one sample (for example, classification_sample_async):

    python -m pytest test_classification_sample_async.py
   

5. To run performance add pytest key: "performance n", where n is number of perf iteration. Test finds in output of sample 'fps', if it exists, then tests rerun that sample adding key 'niter n' with number of perfomance run (that you passed to pytest with '--performance n' keys) Not to add 'niter' key, please, execute pytest "--performance 0"

This test using pregenerated IRs, that located right now in shared folder mentioned above. Also data (images, videos and others) locates in that shared folder.