natas_hw
/
openvino
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
openvino/tests/stress_tests
History
Yuan Hu 1a4ae5e7a3
[core] fix thread safe issue in #19832 (#24562) 
### Details:
- *I lost code `t_stream_count_map[(void*)this] = item.first;` in
#19832*
 - *the thread safe issue happen in below workflow*
 - create thread A
call CustomThreadLocal:local() in thread A -> create stream A (the count
of stream A is 2)
   destory thread A (the count of stream A is 1)
   create thread B (same thread id with thread A)
call CustomThreadLocal:local() in thread B -> use stream A(the count of
stream A is 1, so it's broken)
- *add testcase, also fix
https://github.com/openvinotoolkit/openvino/pull/19986/files#r1332774754*

### Tickets:
 - Closes https://github.com/openvinotoolkit/openvino/issues/24509

---------

Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
 		2024-05-23 11:47:57 +00:00
..
.automation CVS-133383 Update models used in nightly/stress tests (#23157) 2024-03-08 11:18:00 +00:00
common [core] fix thread safe issue in #19832 (#24562) 2024-05-23 11:47:57 +00:00
memcheck_tests copyright year update (#23370) 2024-03-14 09:37:02 +00:00
memleaks_tests [core] fix thread safe issue in #19832 (#24562) 2024-05-23 11:47:57 +00:00
scripts copyright year update (#23370) 2024-03-14 09:37:02 +00:00
unittests [core] fix thread safe issue in #19832 (#24562) 2024-05-23 11:47:57 +00:00
.gitignore publish master branch snapshot, revision 8d31237e2c3f673cbb0f0ba110fc10f5cce1d2bb 2020-05-22 02:23:12 +03:00
CMakeLists.txt copyright year update (#23370) 2024-03-14 09:37:02 +00:00
README.md Rename IE to OV (#22733) 2024-02-09 13:36:41 +00:00

README.md

Stress Tests Suite

This test suite contains tests evaluating the behavior of various OpenVINO use cases under stress conditions:

  • MemCheckTests measuring memory required for the use cases and fail when memory usage exceeds a pre-defined level.

  • StressMemLeaksTests ensure that the use cases does not increase memory levels when executing continuously.

  • StressUnitTests executing various OpenVINO use cases in parallel threads and processes.

Each test refers to configuration files located in <test dir>\.automation folder.

Getting Started

Stress tests are based on the googletest framework. You can filter tests with --gtest_filter and explore tests available with --gtest_list_tests options.

Tests measuring memory have a temporary limitation - those need to be executed one at a time to mitigate memory statistics pollution. You can use gtest-parallel for massive tests execution.

Pre-requisites

Building Tests

To build the tests, you need to have OpenVINO™ installed or build from source. Before build the tests, open a terminal, set OpenVINO™ environment, and after that run the commands below:

source <OpenVINO_install_dir>/setupvars.sh
mkdir build && cd build
cmake .. && cmake --build . -j8

Preparing Test Data

Stress tests may work with models from Open Model Zoo. To use it, download and convert models to IRs using ./scripts/get_testdata.py script. Script will update test config file with data required for OMZ models execution.

From Intel network you can use models from cache at vdp_tests file share. Refer to VDP shared folders on using file shares.

Running Tests

gtest-parallel <openvino_bin>/StressMemLeaksTests

For MemCheckTests preferable way is:

python ./scripts/run_memcheck.py --gtest_parallel <gtest_parallel_py_path> 
<openvino_bin>/MemCheckTests -- --test_conf=<test_conf_path> --refs_conf=<refs_conf_path>

MemCheckTests logs can be used to gather reference values based on current memory consumption:

mkdir -p MemCheckTests-logs && \
gtest-parallel -d ./MemCheckTests-logs ./MemCheckTests && \
grep -rh ./MemCheckTests-logs -e ".*<model " | sed -e "s/.*<model /<model /" | sort