Compare commits

..

170 Commits

Author SHA1 Message Date
Eddy Kim 8e464e992e
using calloc instead of malloc for deterministic hashing (#16326) 2023-03-20 19:50:14 +04:00
Ilya Lavrenov d1a7b0e3c0
Releases/2022/3 (#16409)
* Docs: Update the doc on default hint and execution devices property (#14836)

* Docs: Update to LATENCY as default hint
* Docs: Update the doc on execution devices property
* Update auto_device_selection.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

* 22.3: remove tbb version check for using tbbbind static library (#15700)

* update symbolic link on uninstall page (#15720)

* Update deployment_simplified.svg (#15681)

* [NormalizeL2] normalization of reduction axes (#15841) (#15879)

* Add test for negative axes, preliminary solution to solve uncorrect
results

* Normalize axes in operation NormalizeL2

* Add test for negative axes

* Add EOF

* [67541] - face-detection-0205, 0206 issues fixed (incorrect dimensions error) (#14687)

* [CVS-67541] - face-detection-0205, 0206 issues fixed (incorrect dimensions error)
* [CVS-67541] - face-detection-0205, 0206 issues fixed

* Conversion fail for ov::hint::performance_mode with UNDEFINED value (#15903)

* Update ov::hint::performance_hint UNDEFINED value from empty string to "UNDEFINED".
Update benchmark Python version.
Update the description about hint setting within benchmark APP README and help message.

* Drop the reduntant changes.

* Supported OpenSUSE 15.3 (#15897) (#15907)

* [DOCS] Structure change for 'AUTO Device Selection' article - post merge fix (#15752)

* aligning with 14750

* Fixed samples build on Debian 10 with cmake 3.13 (#15939)

* Fixed samples build on Debian 10 with cmake 3.13

* Use 2022/3 branches

* Limit setuptools version

* Fixed issues in setupvars.sh (#15884) (#15952)

* Fixed issues with setupvar.sh

* Fixes setupvars realpath error

---------

Co-authored-by: Otoka, Tomasz <tomasz.otoka@intel.com>

* Apivalidator (#15951)

* Improved API validator logic (#15942)

* Fix for apiValidator when more than 1 target needs to be checked (#15950)

* Prevent infinite recursion

* [Snippets] Added matcher_name in ConvertConstantsToScalars pass (#15977)

* Install libtbb2 instead of libtbb12 on U22.04 (#15993)

* Apply Apivalidator to extra TBB libs (#15998)

* [GNA] Changed max layer limit tests to avoid SEH exceptions (#15015) (#15460)

* splitted test model

* Changed test config

* Set SF for all inputs

* [Transformations] Enable missing runtime info check (#15796) (#15972)

* Add rt info propagation to StridesOptimization

* Enable rt info check for pruning tests

* Fixed clang-format for C API (#16025)

* Port to 2022.3 from master (#16049)

* notebooks update (#16091)

20230302220806

* Update Customize_Model_Optimizer.md (#15687)

Recreating #14062

* fix benchmark_app python to support YES and NO values for -pin parameter (#16042)

* support YES and NO for -pin

* add if property_name == 'AFFINITY'

---------

Co-authored-by: Chen Peter <peter.chen@intel.com>

* [Docs] nv12 changes port to 22.3 (#16115)

Port:
#15370
#16004

add single-plane input information
create single-plane cpp snippet
menu fix
update formatting for sphinx directives
additional snippet fixes
---------
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>

* [DOCS] Port  Frontend Extensions and OTX page (#16135)

* [DOCS] Add OTX page to Ecosystem  (#16118)

* add otx page

* change ecosystem page

* add ote img

* move ote page to rst

* fix path

* add path

* img test

* otx page

* add docs to ecosystem page

* [DOCS] Fix Frontend Extensions snippets (#16120)

* move fe to rst

* fix code snippets

* add more line breaks

* fix tabsets

* fix link

* fix anchor

* test

* fixing link

* change tab directive

* fix tabs

* align code tabs

* fix link

* fix snippets

* add dlwb to ecosystem

* change ecosystem menu

* exclude fe page

* Port to 2022.3 (#16174)

* Remove setuptools upperbound (#16054)

* Added missed licenses to openvino-dev (#16057)

* Fixed OpenMP + debian package code-path (#16058)

* [CPU] Prevent out of bounds read inside Graph::InferDynamic (#16067)

* Fixed compilation on Debian 11 with gcc 12.2 (#16096)

* Fix for OpenCL

---------

Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>

* Docs benchmarks page update port 22.3 (#16187)

changes to benchmarks page to align with theme

* Andreib/2022.3 myriad plugin obs (#16079)

* Changed to OBS firmware

* Changed dependencies settings for new FW

---------

Co-authored-by: Daria Mityagina <daria.mityagina@intel.com>

* port-16085 (#16210)

* 234 update (#16212)

Adding notebook 234-encodec-audio-compression

* [DOCS] Adding 'Scrollbox' - new sphinx directive (#15307)

port https://github.com/openvinotoolkit/openvino/pull/15305

* [DOCS] Updating 'Prerequisites' section in `Configurations for GNA` article - for 22.3 (#16237)

* issue-15090
Add command for installation of prerequisites on Linux.

* DOCS-image-fix port22.3 (#16341)

(#16324)
(#16308)

* Clearing of CustomReplacementRegistry.registry in convert_model() (#15893) (#16347)

* Clearing of CustomReplacementRegistry.registry.

* Added test.

* Fixed clearing of pipeline config params and TF session in convert_model() (#16191) (#16346)

* Fixed pipeline config params clearing.

* Added clearing of TF session. Added tests.

---------

Co-authored-by: Wang Wangwang <wangwang.wang@intel.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
Co-authored-by: Fang Xu <fang.xu@intel.com>
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
Co-authored-by: Artur Kulikowski <artur.kulikowski@intel.com>
Co-authored-by: Daria Mityagina <daria.mityagina@intel.com>
Co-authored-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
Co-authored-by: Otoka, Tomasz <tomasz.otoka@intel.com>
Co-authored-by: Alexandra Sidorova <alexandra.sidorova@intel.com>
Co-authored-by: Vitaliy Urusovskij <vitaliy.urusovskij@intel.com>
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com>
Co-authored-by: Haiqi Pan <haiqi.pan@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
Co-authored-by: Andrei-George Boji <andrei-george.boji@intel.com>
Co-authored-by: Anastasiia Pnevskaia <anastasia.popova@intel.com>
2023-03-20 19:47:11 +04:00
Xuejun Zhai b692afc764
Xuejun/port cache model api (#15637)
* Add new compile model api to support hash model memory (#14543)

* Add new compile_model api for ONNX RUNTIME OV EP

Allow compile_model() accept model/weight data.

* Update minor place

* Cache model if possible

* Compute hash based on model_xml and model_weight

* Update typo

* Change hash key computation for model's weights

* Resolve test case issue

* Use tensor replace blob for hash computation

* Fix hash computation isssue and add more test cases

* Fix a build issue caused by data format

* Add ov::loaded_from_cache checking for CompileModelLoadFromMemoryTest (#15030)

* Add ov::loaded_from_cache checking for CompileModelLoadFromMemoryTestBase

* Skip gna in skip_tests_config

* Ignore empty tensor for hash calculation (#15282)

* Ignore empty tensor for hash calculation

* Added test

* Fix conflict

* Trigger ci run test for customer_A branch

---------

Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
2023-02-13 15:29:30 +04:00
Ilya Churaev 4c81ede32e
Port #15516 (#15541)
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-02-09 15:43:43 +04:00
Ilya Lavrenov ccd9cadb14
Port to 2022.3 LTS (#15506)
* scripts/install_dependencies/install_NEO_OCL_driver.sh: code style (#15484)

* scripts/install_dependencies/install_NEO_OCL_driver.sh: code style

* scripts/install_dependencies/install_NEO_OCL_driver.sh: code style warnings

* scripts/install_dependencies/install_NEO_OCL_driver.sh: add missing !

* [PyOV] Fix UserWarning in setup.py (#15334)

* [Tools] Support NumPy 1.24 (#14728)

* [Tools] Avoid use of NumPy deprecated types

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Revert "[Tools] Avoid use of NumPy deprecated types"

This reverts commit 21ffc167d1.

* Move to 1.24 Numpy

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* [PyOV] Change default opset9->opset10 (#15238)

* Meiyang/paddle 2.4 (#15177)

* PaddlePaddle 2.4.1 support

* Update src/frontends/paddle/tests/requirements.txt

lowering PDPD version

* update paddlepaddle to 2.4.0

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* Update README.md (#15504)

Just a small typo.

* Throw an exceptions in case of incorrect pointers in Model creation (#15492)

* [C++] Fix coverity issues (#15348)

* [C++] Fix coverity issues

* Merge master with this branch

* [C++] Reapply changes that got eaten during merge

* Update samples/cpp/benchmark_app/main.cpp

Co-authored-by: Ilya Churaev <ilyachur@gmail.com>

* [C++] Add missing const keyword

---------

Co-authored-by: Ilya Churaev <ilyachur@gmail.com>

* [Tests] Fix rest of NumPy deprecated types (#15245)

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Bump paddlepaddle from 2.4.0 to 2.4.1 in /src/frontends/paddle/tests (#15190)

Bumps [paddlepaddle](https://github.com/paddlepaddle/paddle) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/paddlepaddle/paddle/releases)
- [Changelog](https://github.com/PaddlePaddle/Paddle/blob/develop/RELEASE.md)
- [Commits](https://github.com/paddlepaddle/paddle/compare/v2.4.0...v2.4.1)

---
updated-dependencies:
- dependency-name: paddlepaddle
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [nGraph]: fix fused names for ShuffleChannelsFusion transformation (#15150)

* [nGraph]: fix fused names for ShuffleChannelsFusion transformation

* Review comments

* [MO] Activate pylint checker in CI (#15012)

* Fixed mo workflow

* Update pylint config with import os

* Update .github/workflows/mo.yml

* Install extras for pylint checker

* Fix pylint reported errors

* Add torch to the environment

* Add torch into whitelist

* Ignore torch

* Localize torch code

* Fix syntax error

* Remove MO unit-test from Pylint flow

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* [PyOV] Allow `numpy` patch releases (#15265)

* Update reqs

* test, minor change

* [LPT] clang merge warning fix (#15502)

* Trying to fix debian packages (#15508)

* Trying to fix debian packages

* Added conflicts for samples for RPM

* updated mo/requirements.txt

* updated python/requirements.txt

* updated compatibility/requirements.txt

* Ipdated more places

* Revert "scripts/install_dependencies/install_NEO_OCL_driver.sh: code style (#15484)"

This reverts commit 36106d9c45.

* Apply suggestions from code review

* Added policy

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Anastasia Kuporosova <anastasia.kuporosova@intel.com>
Co-authored-by: mei, yang <yang.mei@intel.com>
Co-authored-by: kyo <113977534+kyo-takano@users.noreply.github.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
Co-authored-by: Piotr Krzemiński <piotr.krzeminski@intel.com>
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nadezhda Ageeva <nadezhda.ageeva@intel.com>
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
Co-authored-by: Edward Shogulin <edward.shogulin@intel.com>
2023-02-09 15:34:18 +04:00
Mingyu Kim 494d688d7c
[GPU] Show num_ccs for RANGE_FOR_STREAMS (#15593) 2023-02-09 11:55:24 +01:00
Ilya Lavrenov 833c2865c3
Generalized OpenCL handling (#15253) (#15507)
* Generalized OpenCL handling (#15253)

* Squashed commit of the following:

commit 62c992f6a0bc3a2f559faac6912be9c5632a359f
Author: Ilya Lavrenov <ilya.lavrenov@intel.com>
Date:   Sun Jan 22 11:38:18 2023 +0400

    Generalized OpenCL handling

* Updates

* Fixes

* Update thirdparty/CMakeLists.txt

test

* Fixed build with CL/cl2.hpp

* Fixes

* Fixes

* Fixed compilation flags

* Fixed build with target OpenCL 120

* Don't use cache

* Added policy

* Fixed with build_interface
2023-02-09 14:52:25 +04:00
Artur Kulikowski 7b2273362e
Port relative path 2022.3 (#15523)
* Add test to verify add_extension with relative path (#15212)

* Add test to verify add_extension with relative path

* Fix code style

* Use std::string::find instead of std::regex

* Remove unnecessary include

* Add comments about generating relative path

* Don't add empty tokens when splitting path

* Resolve relative path to the extension

* Method available independent of build_type
2023-02-08 18:46:23 +01:00
Nikolay Shchegolev 8967b4de50
[CPU] MEMC loading failed RuntimeError: There should be only one inst… (#15380) (#15464)
* [CPU] MEMC loading failed RuntimeError: There should be only one instance of RegistersPool per thread.

* Fixes as per comments.
2023-02-08 15:26:58 +04:00
Sebastian Golebiewski f0fa2184e6
[DOCS] Supported Layers update - for 22.3 (#15360)
* LessEqual Not Supported

* port #13997

Update support "Sign" layer to not supported.
Update support "Floor" layer to limited support.
Tickets: 88871 and 88693

* port #13995

Add "Floor" in Known Layers Limitations section.
Add "ConvTranspose" in Known Layers Limitations section.
Add "GatherElements" in Known Layers Limitations section.

Tickets: 88871 and 75208 and 88690
2023-02-08 10:44:16 +01:00
Iosif Dirlea dddec34f35
Fix void function returns value kw issue (#15520)
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-02-08 12:40:21 +04:00
Xiake Sun 46843777fa
[Docs] Port fix convert tf crnn model docs for release 22.3 (#15469)
* Port fix convert tf crnn model for release 22.3
2023-02-08 08:48:17 +01:00
Adrian Varlan 7bb043287a
Fix Myriad plugin wait for inference on multiple devices (#14072)
Co-authored-by: Chen Peter <peter.chen@intel.com>
2023-02-07 19:44:13 +01:00
Artyom Anokhov bb02a94d4a
install_NEO_OCL_driver: Fixed ShellCheck errors (#15533)
* install_NEO_OCL_driver: Fixed ShellCheck errors

* install_NEO_OCL_driver: Fixed after remarks

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-02-07 21:46:07 +04:00
Daria Mityagina 3e0474ac2f
[HDDL] - Update path (#15536)
* [HDDL] - update submodule path

* [HDDL] - update submodule path - do not match hddl2

* [HDDL] - update submodule path - edit REGEX
2023-02-07 10:08:39 +04:00
Sebastian Golebiewski cd28f2e4f8
[DOCS] Fixing links in 'Install Openvino on Windows from Archive' article - for 22.3 (#15357)
* fix links

* Update docs/install_guides/installing-openvino-from-archive-windows.md

* Update docs/install_guides/installing-openvino-from-archive-windows.md
2023-02-06 15:26:51 +08:00
Zlobin Vladimir 550b127b87
Parallelize build_samples_msvc.bat (#14828) (#15480)
Close https://github.com/openvinotoolkit/openvino/issues/14825
2023-02-04 12:05:18 +04:00
Karol Blaszczak 1c562982df
Docs formatting for models article port (#15486)
* [DOCS] add to additional resources and change button formatting
2023-02-03 12:19:31 +01:00
Dalibor Segan eff35123de
Fix OpenVINO 2022.3.0 Chrome OS compilation (#15475)
This patch fixes two issues:
 * Typo in clang pragma directive
 * Reverse the order of the arguments in OPENVINO_ASSERT() in
   constant.hpp so this assert works for ov::bfloat16 and
   ov::float16 types which have impelemented binary comparison
   operators as the memeber functions
2023-02-03 13:29:26 +04:00
Artyom Anokhov fd33cc3b49
properties.hpp: Undef DEBUG macro for log level property (#15473) 2023-02-02 23:47:21 +04:00
Artyom Anokhov 358efd2d06
Updated copyrights with 2023 year (#15148)
* Updated copyrights with 2023 year.

* Updated more files
2023-02-02 16:48:28 +01:00
Ilya Lavrenov a11ae5b690
Setup minimum supported version for pybind11 (#15405) 2023-02-02 16:04:10 +04:00
Yuan Xu 580eebbc33
Add a disclaimer about CentOS for YUM install (#15453)
* update step 3

* fix formatting

* fix formatting

* fix formatting

* test formatting

* change code to use sphinx directives

* fix formatting

* test formatting

* test formatting

* rewording

* test formatting

* test formatting

* test formatting

* move content to troubleshooting guide

* fix formatting

* fixes

* remove troubleshooting item and add a disclaimer

* add centos 7
2023-02-02 18:40:58 +08:00
Karol Blaszczak b0277c865b
Docs ov 2022 3 Added results for Sapphire Rapids 6448Y. OV bench results and system config description (#15418) (#15443)
fixes the FP32 and FP16 swap of results.

* SPR-6448Y updated results

Updated all files with data for SPR 6448Y. New results in benchmark-data.csv, benchmark-performance-data.xls, benchmark-info-detailed.xls and platform-list_22.3.pdf

authored-by: Michael Frank Hansen <michael.f.hansen@intel.com>
2023-02-01 20:56:55 +01:00
Artyom Anokhov eb8b47002d
install_NEO_OCL_driver: Unlock possibility to install GPU driver on ubu22 (#15440) 2023-02-01 23:24:50 +04:00
Sebastian Golebiewski 9e1329fd61
tocsection css burger svg (#15414) 2023-02-01 19:48:01 +01:00
Iosif Dirlea b3f2187a17
Fixed possible overflows to clean KW scan (#15431)
Co-authored-by: Anastasiya Pronina <anastasiya.pronina@intel.com>
2023-02-01 14:50:32 +04:00
Artyom Anokhov e52e6e6547
Updated instructions for Linux* with setup udev usb rules for NCS2 (#15115)
* configurations-for-ncs2: Updated instructions for Linux* with setup 97-myriad-usbboot.rules

* removed install_NCS_udev_rules script.

configurations-for-ncs2: Updated doc after review

* Removed installation on NCS2 script

* install_NCS_udev_rules: return script back

* install_NCS_udev_rules: Added executive permission

* install_NCS_udev_rules: Added newline to the end of file

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-01-31 14:39:51 +01:00
Artyom Anokhov c28f91d031
Return install OCL driver script for LTS (#15395)
* Return back install_NEO_OCL_driver for LTS release

* install_NEO_OCL_driver: Added executive permission
2023-01-30 16:57:37 +04:00
Sebastian Golebiewski 1b1c9eff70
sphinx_design (#15306)
Adding new sphinx extension for designing responsive web components.
2023-01-30 10:06:20 +01:00
Anastasia Kuporosova 3e70387903
[PyOV] Update futureWarning in offline_transformations (#15319) (#15342) 2023-01-28 13:01:28 +04:00
Sebastian Golebiewski 8c85a3b8d3
architecture (#15337) 2023-01-28 15:07:36 +08:00
Artyom Anokhov 96f79b64cd
Updated instruction for installing GPU driver on Linux*. (#15160)
* configurations-for-intel-gpu: Updated instruction for GPU on Linux*.

removed install_NEO_OCL_driver script

* Update docs/install_guides/configurations-for-intel-gpu.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

* Update configurations-for-intel-gpu.md

Proposed to remove extra lines for Windows

* Updated with remarks

* Updated wording and some links

* Update wording for the note

* scripts/CMakeLists: removed install_NEO_OCL_driver from installing

removed duplication of MD-file gpu-setup-guide
Updated DM config

* Update docs/install_guides/configurations-for-intel-gpu.md

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>

* Update docs/install_guides/configurations-for-intel-gpu.md

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>

* Update docs/install_guides/configurations-for-intel-gpu.md

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>

* Update docs/install_guides/configurations-for-intel-gpu.md

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>

* Updated instructions with prerequisites for RH8

---------

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
Co-authored-by: Pavel Durandin <pavel.durandin@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2023-01-27 18:54:46 +01:00
Tatiana Savina aa330736cc
[DOCS] Delete DL WB note 22.3 (#15283)
* delete DL WB tips

* remove space

* fixed a link
2023-01-27 16:12:39 +01:00
Ilya Churaev 4211cc38b8
Port 15303 (#15354)
* Try to fix fuzzer tests

* Try to fix test
2023-01-27 12:58:47 +04:00
Gorokhov Dmitriy 5770daf303
[CORE] Fixed VEX/NonVEX performance penalty inside Convert evalute impl (#15355) 2023-01-27 12:54:48 +04:00
Ilya Lavrenov 3c8ec29423
Migrate SVG files under LFS (#15322) 2023-01-26 16:22:51 +04:00
Sergey Shlyapnikov ae52cb3e48
[GPU] Fix eltwise fusions in case of LT_ALIGNED_READ and vec_size>1 (#15261) 2023-01-25 13:27:58 +04:00
Sebastian Golebiewski da5eb6843c
DOCS: Update doxygen version - port to 22.3 (#15278)
* fixing references

* fix refs

* fix branch.hpp

* Update build_doc.yml
2023-01-24 19:39:05 +04:00
Anastasia Kuporosova 567eac4d8f
[PyOV] skip dynamic test for arm (#15257) (#15271)
* [PyOV] skip dynamic test for arm

* Update src/bindings/python/tests/test_runtime/test_infer_request.py

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-01-23 23:33:28 +04:00
Daria Mityagina c28bf68e30
[VPU] - Firmware update (#15260) 2023-01-23 14:57:44 +00:00
Yuan Xu 2462e6ae89
Fix installation issues for #15090 (#15105)
* add how to get install_NEO_OCL_driver.sh for Python users
* minor edits
2023-01-23 14:28:42 +01:00
Sebastian Golebiewski 7f229478ef
fix headings (#15209) 2023-01-23 10:26:49 +01:00
Sebastian Golebiewski 8fbdd05782
DOCS: Update of 'Creating a YOCTO image' article - for 22.3 (#15133)
* update

Updating master branch in meta-intel repository for OpenVINO 22.3

* Update installing-openvino-yocto.md
2023-01-23 10:23:54 +01:00
Sebastian Golebiewski d4e1694491
fix indentation (#15236) 2023-01-23 09:54:28 +01:00
Maciej Smyk b7f4c87553
scheme3 (#14874)
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-01-20 19:25:20 +04:00
Ilya Lavrenov 2f2c7d23ad
Added SVG files to lfs (#15228) 2023-01-20 15:57:10 +04:00
Yuan Xu be74074758
Add brew installation & update APT and YUM pages (#15141)
* create draft for brew installation

* updates

* updates

* updates

* minor updates

* fix formatting

* update with Brew uninstall and APT page

* update formatting

* fix a typo

* minor update

* update what's next & samples

* update additional resources

* update a link and YUM article

* align the beginning paragraphs

* align article structures

* fix anchors
2023-01-20 15:50:17 +04:00
Sebastian Golebiewski 8f9a4faa06
heading fix (#15226) 2023-01-20 15:11:37 +04:00
Maciej Smyk 977ad1b093
default_quantization_flow (#14850) 2023-01-20 14:24:20 +04:00
Sebastian Golebiewski d47deaaca6
fix formatting (#15174)
fix formatting and links
2023-01-20 10:26:17 +01:00
Maciej Smyk 4a893dc6de
DOCS: Model Caching Overview image recreation for 22.3 (#15025)
* Model Caching Overview

* graph-background-fix
2023-01-20 08:21:36 +01:00
Maciej Smyk 46921acd37
DOCS: OpenVINO™ Security Add-on image recreation for 22.3 (#15081)
* Security Add-on

* Update ovsa_example.svg
2023-01-20 08:16:28 +01:00
Anastasia Kuporosova c939c70f0d
[PyOV] Skip tests with bool for arm (#15208) (#15214)
* [PyOV] Skip tests with bool for arm

* exec_net export test rewrite
2023-01-20 01:23:50 +04:00
Jan Iwaszkiewicz a2dc739da4
[PyOV] Add new types to Any dispatcher (#15194) 2023-01-19 15:38:46 +04:00
Wilson Seok e998013e2e
Cherrypick PR14812 to releases/2022/3 (#15172)
* fix strided_slice_optimize for 5d new axis output, typo in strided_slice cl kernal, avoid handle_reshape when user is only one

* fix unit test of resample_eltwise_fusing_through/12
2023-01-18 19:13:37 -08:00
Albert Angles 6c9c5cb70d
enable first_inference_latency (first_inference+time_to_inference) (#14805)
* add time_to_first_inference_result KPI as sum of inf latency values

* append time unit format after the value

* Revert "append time unit format after the value"

This reverts commit 4bdd82e06b.

* remove metric, to be consistent with rest ones

* added FEIL SCOPED_TIMER  (It will report the sum of all stages)

* revert add time_to_first_inference_result KPI as sum of inf latency values

* place SCOPED_REGION(first_inference_latency) inside a block { }  - tested and worked

* enable first_inference_latency for timetest_infer_API_2.0

tested and worked

* remove trailing whitespace

Co-authored-by: Daria Ilina <daria.krupnova@intel.com>
2023-01-18 18:07:46 +00:00
Sebastian Golebiewski c6b17afa81
revert tutorials (#15165) 2023-01-18 08:32:02 +01:00
Daria Mityagina 1285443993
[VPU] - Update MyriadX firmware (#15158)
* [VPU] - Update firmware

* [VPU] - Update firmware - R20.6

* [VPU] - Update firmware - g_callCommand
2023-01-17 19:44:28 +00:00
Anastasia Kuporosova cc2c03c95d
Port skipped test (#15140)
* [PyOV] Rewriting properties tests as hardware agnostic (#14684)

* [PyOV] Mark replacement of topk in test (#15095)

* undo unwished changes

Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
2023-01-17 17:24:19 +04:00
Tatiana Savina bbdedc5860
update doc footer 22.3 (#15154) 2023-01-16 19:59:35 +03:00
Eddy Kim 1552beaab2
Adding missed conversion logics between layout and string for 2022.3 (#15111)
* added missed layout to string conversion

* added an unit test for layout_from_string

* added missing spaces around =

* replaced 95 with SCALAR

* moved layout_to_string into a new file

* changed to parametrized test
2023-01-16 17:30:44 +04:00
Ilya Lavrenov ae537310dc
Don't add samples subdirectory when both samples and tests are disabled (#14817) (#15122) 2023-01-16 10:50:17 +04:00
Sebastian Golebiewski 43c8af2fce
DOCS: Updating the diagram in the home page -for 22.3 (#15108)
restyling the first paragraph and updating the diagram
2023-01-14 17:32:56 +01:00
Yuan Xu 9da0800b31
Add install dependencies info for APT & YUM (#15023)
* add install dependencies info

* update according to comments

* fix formatting

* fix formatting

* small corrections

* fix formatting & issues

* minor correction
2023-01-14 13:12:43 +00:00
Anastasia Kuporosova 196d01b952
Port changes in python tests to 2022.3 (#15101)
* [PyOV] Rewriting properties tests as hardware agnostic (#14684)

* [PyOV][Tests] Fix some tests for M1 (#14555)

* [PyOV] Make graph tests hardware agnostic - part 1 (#14500)

* Halfway done

* Prepare part 1

* Minor changes

* Minor changes

* [PyOV] Make graph tests hardware agnostic - part 3 (#14639)

* [PyOV] Make graph tests hardware agnostic - part 2 (#14519)

* [PyOV] Make graph tests hardware agnostic - part 4 (#14705)

* [PyOV] Make graph tests hardware agnostic - part 5 (#14743)

* merge conflict resolve

Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
2023-01-14 01:39:17 +04:00
Alina Kladieva d7607a2415
Disable IB in Win Azure pipeline (#15086) (#15098) 2023-01-13 17:56:40 +04:00
Eddy Kim 810b7e198e
Update calculateFileInfo for the null string case for 2022.3 (#15078)
* updated not to find absPath for null filePath

* changed to cache only std::runtime_error

* fix clang-format error
2023-01-13 17:48:01 +04:00
Julio Morero efa7cae5a0
Change device order in MULTI python snippet (#15077) 2023-01-13 01:49:14 +04:00
Yuan Xu 5f601a5f63
fix step5 (#15065) 2023-01-12 15:41:14 +04:00
Tatiana Savina bf8d662050
update link and extras description (#15046) 2023-01-12 12:38:43 +01:00
Sebastian Golebiewski 29d7cd87d0
DOCS: Fixing formatting in 'Install From Apt' article - for 22.3 (#15066)
* fixing-formatting-apt

Fixing indentation, faulty code-blocks, numbering.

* fix link

* Update docs/install_guides/installing-openvino-apt.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

* fix link

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
2023-01-12 16:30:51 +08:00
Karol Blaszczak 8f7af3299f
adding new tutorials (#15027) (#15033)
authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
2023-01-11 09:54:25 +01:00
Vladimir Paramuzov aabda64bd8
[GPU] Fix shape infer for 0d broadcast (#14744) (#14971) 2023-01-08 12:59:24 +04:00
Zlobin Vladimir f3a25e9d1b
uint32->uint64 for RedHat gflags 2.1 failing at runtime (#14724)
* uint32->uint64 for RedHat gflags 2.1 failing at runtime

* tools/legacy/benchmark_app: uint32->uint64 for RedHat gflags 2.1 failing at runtime

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2023-01-06 18:13:17 +04:00
Maciej Smyk 2257dc8a53
DOCS: Libraries for Local Distribution image recreation for 22.3 (#14959)
* deployment_full

* Update deployment_full.svg
2023-01-05 23:14:15 +04:00
Sebastian Golebiewski 38911e4cb7
DOCS: Hiding Transition to API 2.0 banner - for 22.3 (#14952)
Using cookies to keep the banner hidden once the user have closed it.
2023-01-05 13:59:18 +01:00
Sebastian Golebiewski 57c2a07406
DOCS: Updating Interactive Tutorials - for 22.3 (#14946)
Adding new tutorials:
404-style-transfer-webcam
406-3D-pose-estimation-webcam
2023-01-05 13:48:07 +01:00
Maciej Smyk 418b455883
DOCS: Fixing broken links in documentation for 22.3 (#14941)
* fixing broken links
2023-01-05 13:46:19 +01:00
Maciej Smyk 563847bb76
Fix inference pipeline C++ doc: refer to the correct input blob (#14732) 2023-01-05 01:20:42 +04:00
Aleksandr Voron e9bb5a2bb7
update pypi docs with apple m1 status (#14932) 2023-01-04 17:32:15 +04:00
Maciej Smyk b11a977efd
yolo_tiny_v1 (#14879) 2023-01-04 11:11:50 +01:00
Maciej Smyk 7c9482a3ca
quantization_picture (#14870) 2023-01-04 11:07:51 +01:00
Karol Blaszczak b260cb2f1e
[DOCS] Remove System Requirements from install guides (#14808)
System Requirements removed where OS or hardware is mentioned... Docker articles not included

* include selector tool in install guide
2023-01-04 10:38:04 +01:00
Maciej Smyk 430e532a62
nncf_workflow (#14867) 2023-01-03 17:15:28 +01:00
Maciej Smyk 935059f0dc
deployment_simplified (#14854) 2023-01-03 15:59:41 +01:00
Maciej Smyk 048667ca70
autoplugin_accelerate (#14839) 2023-01-03 14:55:28 +01:00
Tatiana Savina afc7f8f3a0
updated links to 22.3 relnotes (#14917) 2023-01-03 14:54:16 +01:00
Maciej Smyk a1245a95a4
DOCS: The LowLatency Transformation images recreation for 22.3 (#14830) 2023-01-03 14:49:23 +01:00
Sebastian Golebiewski 4b6765a50c
format pre tags (#14913)
Porting: #14889

This fix addresses word wrapping in <pre> tags in the output html files of documentation.
2023-01-03 13:15:36 +01:00
Karol Blaszczak 8b4f1d9616
Update benchmark-data.csv (#14794) (#14891)
fixes the FP32 and FP16 swap of results.
authored-by: Michael Frank Hansen <michael.f.hansen@intel.com>
2023-01-03 08:55:57 +01:00
Karol Blaszczak c6c8ac6a0d
DOCS]-workflow-article-titles-tweaks (#14893) 2023-01-03 08:02:46 +01:00
Ilya Lavrenov 56ac14f276
Updated docs for Raspbian OS (#14816)
* Updated docs for Raspbian OS

* Update docs/install_guides/installing-openvino-raspbian.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
2023-01-03 13:35:33 +08:00
Yuan Xu d2bd73a7e7
add ways to find samples for PyPI installation (#14658) 2022-12-26 12:58:05 +04:00
Daria Mityagina 827025f8ee
Fix some tests (#14434) (#14774)
Co-authored-by: Nadezhda Ageeva <nadezhda.ageeva@intel.com>
2022-12-22 20:57:29 +04:00
Daria Mityagina 349d1f6647
[CVS-98877][MX] - NonMaxSuppression opset5 -> opset9 (#14683) 2022-12-22 20:21:40 +04:00
Tatiana Savina 61c9deb284
Release doc updates (#14784)
* apply feedback
* change POT header
* old diagram
2022-12-22 15:40:42 +01:00
Yuan Xu 9a7c0caf6c
update version numbers & windows installation (#14777) 2022-12-22 10:31:59 +01:00
Yuan Xu 88166c1c61
Add CentOS back to install archive page for 22/3 (#14625) 2022-12-22 10:30:59 +01:00
Yuan Xu 0990540083
Update Raspbian OS installation for 22/3 (#14628)
* update Raspbian OS installation

* test formatting

* fix formatting

* Apply suggestions from code review

* update version number & package names
2022-12-22 12:59:03 +08:00
Yuan Xu 578e979334
Update macOS archive installation with ARM64 for 22/3 (#14656)
* update macos archive installation with arm

* update python version

* fix formatting

* fix formatting

* update

* remove note

* update package names & version number
2022-12-22 12:58:09 +08:00
Tatiana Savina 791385f140
add warnings (#14776) 2022-12-22 04:54:34 +01:00
Karol Blaszczak 0df7039fe7
DOC-final-benchmark-changes-for-release port 22.3 (#14771) 2022-12-21 18:09:49 +01:00
Tatiana Savina a18d32a639
delete note (#14769) 2022-12-21 15:42:01 +01:00
Yuan Xu 7deb3293e7
remove a space (#14756) 2022-12-21 17:13:57 +08:00
Karol Blaszczak d857067aae
[DOCS][CPU] Added sparse weights decompression feature description (#14730) (#14749)
port https://github.com/openvinotoolkit/openvino/pull/14730

authored-by: Anton Voronov <anton.voronov@intel.com>
2022-12-21 08:18:54 +01:00
Karol Blaszczak bd96363730
Docs minor release adjustments port22.3 (#14747)
Update benchmark-data.csv
update external files
update articles, faq included

* hide ovms benchmark
2022-12-20 23:08:09 +01:00
Tatiana Savina d80f3afe05
DOCS: Add text for doc headers (#14671)
* add placeholder text

* try sphinx format for headers

* update resources and learn OV

* try ref linking

* try ref linking 2

* try ref linking 3

* change links to doc

* fix new line

* move TF frontend to "deployment" TOC st. 1

* move TF frontend to "deployment" TOC st. 2

* fix links for deployment section homepage

* add disclaimer on ARM support

* trying different links to api

* change API link

* Trying to hide an article in TOC

* change top arcticles content

* remove directive for tf article

* fix build error

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2022-12-20 22:17:34 +01:00
Roman Kazantsev 9752fafe8e
[Default Install] Recover upper-bound for the default installation openvino-dev (#14734)
Use 1.23.4 to be aligned with extras installation

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2022-12-20 00:14:53 +01:00
Maksim Kutakov 3111e2394c
[CPU][DOCS] Remove recommendation to use partially defined shapes (#14675) 2022-12-19 21:49:31 +04:00
Sebastian Golebiewski b2feb56b22
DOCS: Homepage - openvino workflow section update - for 22.3 (#14654)
Removing raw html directive
Updating/improving css
adding new diagram
updating css
2022-12-16 19:49:32 +01:00
Aleksandr Voron 7da99de9ae
Compute Library license added (#14659) 2022-12-16 22:34:07 +07:00
Anastasiia Pnevskaia 756b0de9bf
Model Optimizer Python API documentaion. (#14696) 2022-12-16 15:31:03 +04:00
Aleksandr Voron 1a72966d4e
Remove layers limitation section and add a link to wiki (#14613)
Co-authored-by: Aleksandr Voron <avoron@C07J20DQQ6NWmac.imu.intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
2022-12-16 13:10:55 +03:00
Sebastian Golebiewski 4c7c9418a5
DOCS: Low Precision Transformations proofreading for 22.3 (#14511)
* Attributes
* Update lpt_attributes.md
* defines-fix
* revert principle
2022-12-16 08:19:24 +01:00
Sebastian Golebiewski 00d462b1d6
Fixing Python API links (#14498)
Porting: https://github.com/openvinotoolkit/openvino/pull/14423

Fixing the reference to Python API.
2022-12-16 08:11:13 +01:00
Sebastian Golebiewski 21ae66ad5f
Fixing links to API (#14497)
Porting:
https://github.com/openvinotoolkit/openvino/pull/14253

Addressing: 96910
2022-12-16 08:09:49 +01:00
Karol Blaszczak a5c616bc3c
[Documentation]: Added description for NNCF PTQ (#14437) (#14665)
authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
2022-12-15 15:05:47 +01:00
Maciej Smyk d33d6b7a6d
Stateful models (#14661) 2022-12-15 13:53:43 +01:00
Karol Blaszczak dffe45ebd1
DOCS-menu-recreate-structure-step5 (#14637)
Shift to separating the Workflow section, including moving Run and Optimize inference inside Deploy locally

change several article and menu titles
minor additional restructuring
minor content tweaks
remove optimization introduction (may be brought back in parts later)
* several link fixes
* additional link fixes
2022-12-14 18:33:02 +01:00
Roman Kazantsev bba7e77242
[TF FE] Add user guide about TF FE Capabilities and Limitations (#14622) (#14634)
* [TF FE] Add user guide about TF FE Capabilities and Limitations

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Update docs/resources/tensorflow_frontend.md

* Update docs/OV_Runtime_UG/protecting_model_guide.md

Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>

* Update docs/OV_Runtime_UG/deployment/local-distribution.md

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
2022-12-14 18:58:54 +04:00
Sebastian Golebiewski 5f8d05e342
DOCS: New Homepage- 22.3 (#14581)
* new homepage concept
* new diagram
2022-12-14 12:12:28 +01:00
Roman Kazantsev b84161848e
[TF FE] Add info messages in Model Optimizer (#14593)
* [TF FE] Add info messages in Model Optimizer and the preliminary user documentation

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Add test model with future operation

* Update docs/MO_DG/prepare_model/TensorFlow_Frontend.md

Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>

* Apply code-review feedback: Recover use of dict

* Fix the rest of feedback

* Remove User documentation for TF FE

* Correct tests to provide required arguments

* Handle argv without framework argument in deduce_framework function

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
2022-12-13 13:24:23 +01:00
Jade Cho 7904b34679
[GPU] Fix a bug to not apply f-axis offset to oneDNN concat (#14595) 2022-12-13 20:28:53 +09:00
Sebastian Golebiewski bda7227502
post-merge corrections (#14603)
Fixing link to python downloads
Minor grammatical error corrected
Updating software license disclaimer

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
2022-12-13 11:49:24 +01:00
Sebastian Golebiewski f3964bcea0
DOCS: Proofreading OpenVINO Extensibility for 22.3 (#14510) 2022-12-13 11:19:26 +01:00
Maciej Smyk 8f13a77a15
DOCS: Quantization doc rewrites - port to 22.3 (#14514)
* Update introduction.md
* header fix
* graph-fix
2022-12-13 09:51:12 +01:00
Karol Blaszczak 97ae79fe80
DOCS-menu_recreate_structure_step4 (#14584)
documentation section tweaks
create deployment section for further tweaks
API reference moved
compile tool moved
2022-12-13 08:12:41 +01:00
Maciej Smyk ed5e237403
Target Operating Systems with Python Version fix (#14568) 2022-12-13 12:24:34 +08:00
mei, yang 7338f4b049
update paddle supported ops (#13923) (#14490)
* update paddle supported ops

* remove paddle tensor array related ops

* remove version information in paddle supported op name

* Update docs/MO_DG/prepare_model/Supported_Frameworks_Layers.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

* Update docs/MO_DG/prepare_model/Supported_Frameworks_Layers.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

* Update docs/MO_DG/prepare_model/Supported_Frameworks_Layers.md

* Update docs/MO_DG/prepare_model/Supported_Frameworks_Layers.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
2022-12-13 11:53:29 +08:00
Wilson Seok 288b5326cb
fix PR14422 regression for using arg_max_min opt kernel (#14487) (#14533) 2022-12-12 16:10:00 -08:00
Ilya Lavrenov 6f5edfa942
Adjusted pypi supported OS matrix (#14546) 2022-12-13 01:35:16 +04:00
Maksim Kutakov ad0ef56665
[CPU] Fix for the convolution node dummy shapes generation (#14521) 2022-12-12 23:08:15 +04:00
Artyom Anokhov 7e1b9353d5
[Deployment Manager Configs] Updated with actual 2022.3.0 structure (#14585) 2022-12-12 22:23:11 +04:00
Pavel Esir ece291a246
Revert "[IE] keep constants in shape subgraphs in f32 (#14204)" (#14457)
This reverts commit e648001b

Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
2022-12-12 18:10:41 +04:00
Karol Blaszczak 669d35b3e6
DOCS-menu_recreate_structure_step3 (#14574)
Adjusting the benchmarks structure
Restructure menu and adjust content for benchmarks and getting performance numbers
2022-12-12 14:55:46 +01:00
Vladimir Paramuzov 12b019e027
[GPU] Fixed invalid offset for tensor with batch padding (#14459) 2022-12-12 17:37:38 +04:00
Zlobin Vladimir b07fea664f
build_samples.sh: detect cmake3 for CentOS (#14562)
Close https://github.com/openvinotoolkit/openvino/issues/14553
2022-12-12 17:23:56 +04:00
Karol Blaszczak 4f9a8757c8
Docs menu lvl1 expand script (#14567)
* JS for expanding menu lvl1
Script by Kasia Bojarowska
2022-12-12 13:59:26 +01:00
Sebastian Golebiewski 12afd97e14
FaceNet (#14507) 2022-12-12 13:12:22 +01:00
Sebastian Golebiewski 9183d96394
lm_1b (#14506) 2022-12-12 13:10:31 +01:00
Sebastian Golebiewski 2d0f8b4f27
NCF_start images (#14505) 2022-12-12 13:07:39 +01:00
Sebastian Golebiewski 05d8648854
image-fix (#14504) 2022-12-12 13:04:44 +01:00
Sebastian Golebiewski 3914fd8aa8
DOCS: Image fix cutting models - porting #13853 to 22.3 (#14501)
* image fix
2022-12-12 11:21:39 +01:00
Vladislav Golubev 4911f80f37
[CPU] Convolution shape inference fix (#14550) 2022-12-12 13:59:14 +04:00
Roman Kazantsev 53265f0097
[ONNX FE] Support value freezing without specifying type (#14481) (#14554)
* [ONNX FE] Support value freezing without specifying type

Implement gete_element_type method for ONNX InputModel.
Implement tests with freezing integer input and float models

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Update src/frontends/onnx/frontend/src/input_model.cpp

Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>

* Update src/frontends/onnx/frontend/src/input_model.cpp

Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>

* Update src/frontends/onnx/frontend/src/input_model.cpp

Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>

* Return only element type for model inputs and add ONNX FE tests

* Rename variable in the test

* Add syntax mark

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
2022-12-12 10:53:52 +04:00
Karol Blaszczak 3dd6b50bff
Docs menu recreate structure step2 (#14551)
model optimization
deploy locally
run inference
remove OVSA from security (it was duplicated)
2022-12-11 14:30:00 +01:00
Sebastian Golebiewski 9b8ab44500
DOCS: Reintroducing APT and YUM installation methods for 22.3 (#14492)
* reintroducing apt and yum

* fix indents and create tabs

fixing indent in apt
creating tabs in yum article

* Update installing-openvino-yum.md

* tabbed content

Inserting installation steps into tabs.
Correcting formatting.

* correcting indents

* correct indent

* indent fix

* Update docs/install_guides/installing-model-dev-tools.md

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>

* removing non-cpu dev

Removing GNA, NCS2, VPU from Step 5 in YUM article.
Correcting links to Python.

* add apt and yum for installation

* revising openvino-dev installation

* removing opencv from apt

* remove opencv from yum

* removing non-cpu devs

* Update installing-model-dev-tools.md

* update disclaimer

* openvino development tools

correcting the name

* build samples sections

* add architecture

* remove install details yum

* remove install details apt

* reflink fix

* removing ncs2

* add dependencies

* install parametere

* move note

Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
2022-12-09 22:03:29 +04:00
Mateusz Bencer 7e87923f0c
Added support of custom domain for OpExtension (C++/Python) and ConversionExtension (Python) (#14523) 2022-12-09 15:01:07 +01:00
Eddy Kim 083a17e89e
[GPU] Updated to get lockable memories for device inputs in prepare_input() (#14532)
* updated to get ockable memory for device inputs

* gpuFuncTest to set f64 scalar tensor

* aligned type names

* updated to get lockable memory for some specific precisions
2022-12-09 17:00:32 +04:00
Anastasiya(Asya) Pronina 67bbc7361c
Extended speech_sample to work on VPUX (#14508) 2022-12-09 15:27:20 +04:00
Artyom Anokhov 0121921068
Added ocl-icd for ubi8 image (#14525)
* install_openvino_dependencies: Added ocl-icd for ubi8 image

* install_openvino_dependencies.sh: Added json-devel to pkgs_dev for ubi8

* install_openvino_dependencies: Added pkgconf-pkg-config to pkgs_dev

* install_openvino_dependencies: Moved pkgconf-pkg-config to high-level pkgs_dev

* install_openvino_dependencies: Fixed with case for centos7

* Update scripts/install_dependencies/install_openvino_dependencies.sh

applied comments

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
2022-12-09 14:11:56 +04:00
Roman Kazantsev 5cfc3cbf5d
[TF FE] Break the cycle in the different way (#14480) (#14518)
* [TF FE] Break the cycle in the different way

Earlier solution was incorrect due to inproper handling of forward edges cases
(edges going from parent to grand-child) for which topological sorting
of nodes can be interrupted.

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Break the cycle by NextIteration inputs

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2022-12-09 10:28:38 +04:00
Karol Blaszczak 85506f263c
Docs menu recreate structure step1 port (#14527)
* Initial TOC config

TOC depth
prev-next btn
TOC initial links

* extensibility section

header changed to section
section populated

* media processing and CV lib

header changed to section
no content in the node article... needs to be added later

* openvino security

header turned into section
intro article used as the section parent

* ecosystem

header turned to section
section populated

* menu padding adjustment

* benchmarks and release notes

benchmarks moved to resources
release notes moved to top level
case studies link

* OpenVINO IR

IR articles put in a section
the parent article will require more content
2022-12-09 08:23:09 +08:00
Ilya Lavrenov 36c6d58fb2
Added extra dependencies for debian / rpm packages (#14528) 2022-12-09 03:49:45 +04:00
Zlobin Vladimir 690a10c386
samples/cpp/build_samples_msvc.bat: don't ignore cmake error (#14499) (#14524)
Ticket 98429
2022-12-09 03:47:52 +04:00
Artemy Skrebkov bceb7dbfe1
Fix segfault on windows with MSVC (#14517)
* MSVC compiler cannot handle copying empty AnyMap
2022-12-08 21:00:07 +04:00
Roman Kazantsev 57d8e96a30
[TF FE] Support dynamic shape Placeholder freezing and PlaceholderWithDefault (#14450) (#14482)
* [TF FE] Support dynamic shape Placeholder freezing and PlaceholderWithDefault

Also, this PR contains reorganization of python unit tests for TF FE that
covers conversion and inference of different models in pbtxt.
This mini-infrastructure will be used in the future for TF FE support.

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Revert debug info

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2022-12-08 17:43:22 +04:00
Alexandra Sidorova 2d072e8129
[CPU] Fixed segfault in Generic node creation for dynamic models (#14488) 2022-12-08 17:13:34 +04:00
Zlobin Vladimir 36ab5c2192
Update OMZ submodule (#14471)
* Update OMZ submodule

This update includes models publication for 2022.3
Tickets 95909 and 95910

* Update OMZ submodlue: remove numpy upper bound
2022-12-08 11:26:07 +00:00
Vladimir Paramuzov afa9bdde93
[GPU] allow softmax_bf for axis=Y && X==1 case (#14461) 2022-12-08 14:10:48 +04:00
Vladimir Paramuzov 970d53b420
[GPU] Use transformed gemm shapes for layout optimizer checks (#14462) 2022-12-08 14:10:38 +04:00
Anastasiia Pnevskaia 633fe397a3
Fixed type check in convert_model(). (#14477) 2022-12-08 13:54:35 +04:00
Anastasiia Pnevskaia 5c33eff533
Fixed double printing of errors in MO. (#14474) 2022-12-08 12:46:22 +04:00
Xiping Yan 8680791ffe
[CPU] Fix conditional compilation for oneDNN brgemm Convolution primitive 2022-12-08 12:06:41 +04:00
Ilya Lavrenov edb3f3e820
Expose IE_THREAD definition to end users (#14476) 2022-12-08 12:05:42 +04:00
Karol Blaszczak 0eebb124c9
DOCS-top-menu-items_port_22.3 (#14465)
create top menu navigation
tweak css to display navigation properly
2022-12-07 19:34:42 +01:00
Karol Blaszczak cc886509c6
Update benchmarks articles (#14438) (#14464)
Update performance_benchmarks_faq.md
Update performance_int8_vs_fp32.md
2022-12-07 11:20:43 +01:00
Karol Blaszczak a9ad74b690
DOCS- add telemetry info page (#14463) 2022-12-07 11:15:24 +01:00
Andrey Babushkin 6947916541
Update branch to take other repos from for Azure pipelines (#14447) 2022-12-06 22:46:21 +04:00
23147 changed files with 1241112 additions and 1554884 deletions

View File

@ -0,0 +1,53 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Analyze GTest logs
"""
import re
from argparse import ArgumentParser
def get_passed_tests(log_file_path):
"""Gets passed tests with OK status"""
ok_test_line_pattern = "[ OK ] "
ok_tests = []
with open(log_file_path) as log_file_obj:
for line in log_file_obj.readlines():
if ok_test_line_pattern in line:
ok_tests.append(line.split(ok_test_line_pattern)[1])
return ok_tests
def get_total_time(tests):
"""Gets total execution time (sec)"""
re_compile_time = re.compile(r".+ \(([0-9]+) ms\)")
total_time = 0.0
for test in tests:
re_time = re_compile_time.match(test)
if re_time:
total_time += int(re_time.group(1)) / 1000
else:
print("No time in the test line:", test)
return total_time
def main():
"""The main entry point function"""
arg_parser = ArgumentParser()
arg_parser.add_argument(
"--log-file", metavar="PATH", default="gtest.log", help="Path to GTest log file"
)
args = arg_parser.parse_args()
passed_tests = get_passed_tests(args.log_file)
print("PASSED tests count:", len(passed_tests))
print("Total execution time of passed tests (sec):", get_total_time(passed_tests))
print("\nPASSED tests:")
print("".join(sorted(passed_tests)))
if __name__ == "__main__":
main()

165
.ci/azure/android_arm64.yml Normal file
View File

@ -0,0 +1,165 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
jobs:
- job: android_arm64
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
OPENVINO_REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(OPENVINO_REPO_DIR)/../openvino_contrib
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
ANDROID_TOOLS: $(WORK_DIR)/android_tools
ANDROID_SDK_VERSION: 29
ANDROID_ABI_CONFIG: arm64-v8a
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/android_arm64
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo ls /usr/bin/python3.10
rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System information'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(ANDROID_TOOLS) ; mkdir $(ANDROID_TOOLS)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(OPENVINO_REPO_DIR)/install_build_dependencies.sh
# Move into contrib install_build_dependencies.sh
sudo apt --assume-yes install scons crossbuild-essential-arm64 libprotoc-dev protobuf-compiler default-jdk
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Install Android SDK, NDK and TOOLS
wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip
unzip commandlinetools-linux-7583922_latest.zip
yes | ./cmdline-tools/bin/sdkmanager --sdk_root=$(ANDROID_TOOLS) --licenses
./cmdline-tools/bin/sdkmanager --sdk_root=$(ANDROID_TOOLS) --install "ndk-bundle" "platform-tools" "platforms;android-$(ANDROID_SDK_VERSION)"
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_TOOLCHAIN_FILE=$(ANDROID_TOOLS)/ndk-bundle/build/cmake/android.toolchain.cmake
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DANDROID_ABI=$(ANDROID_ABI_CONFIG)
-DANDROID_STL=c++_shared
-DANDROID_PLATFORM=$(ANDROID_SDK_VERSION)
-DENABLE_TESTS=ON
-DBUILD_java_api=ON
-DBUILD_nvidia_plugin=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
$(OPENVINO_REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Android ARM64'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(OPENVINO_REPO_DIR)/bin/
displayName: 'List binary files'

View File

@ -0,0 +1,6 @@
TransposeOpTest.NHWC2NCHW
TransposeOpTest.NCHW2NHWC
TransposeOpTest.TwoDim_int16
GatherOpTest.Gather_axis1_indices2d_int16
SoftmaxOperator.ThreeDimsAxis1
SoftmaxOperator.ThreeDimsAxis0

View File

@ -0,0 +1 @@
rel-1.8.1

567
.ci/azure/linux.yml Normal file
View File

@ -0,0 +1,567 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Lin
strategy:
matrix:
# Dynamic:
# CMAKE_BUILD_SHARED_LIBS: 'ON'
# PYTHON_STATIC_ARGS:
# CMAKE_CPACK_GENERATOR:
# SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/samples
# PYTHON_SAMPLES_INSTALL_DIR: $(SAMPLES_INSTALL_DIR)/python
# RUN_PREFIX: . $(SETUPVARS) -pyver 3.8 &&
# Debian:
# CMAKE_BUILD_SHARED_LIBS: 'ON'
# PYTHON_STATIC_ARGS:
# CMAKE_CPACK_GENERATOR: 'DEB'
# SAMPLES_INSTALL_DIR: /usr/share/openvino/samples
# PYTHON_SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/share/openvino/samples/python
# RUN_PREFIX: LD_LIBRARY_PATH=$(INSTALL_TEST_DIR):$(INSTALL_DIR)/opencv/lib:$LD_LIBRARY_PATH
Static:
CMAKE_BUILD_SHARED_LIBS: 'OFF'
PYTHON_STATIC_ARGS: -m "not dynamic_library and not template_plugin"
CMAKE_CPACK_GENERATOR:
SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/samples
PYTHON_SAMPLES_INSTALL_DIR: $(SAMPLES_INSTALL_DIR)/python
RUN_PREFIX: . $(SETUPVARS) &&
maxParallel: 2
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
BUILD_LAYER_TESTS_DIR: $(WORK_DIR)/build_layer_tests
BUILD_SAMPLES_TESTS_DIR: $(WORK_DIR)/build_samples_tests
INSTALL_DIR: $(WORK_DIR)/install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)/tests
LAYER_TESTS_DIR: $(INSTALL_TEST_DIR)/layer_tests
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux
CMAKE_VERSION: 3.24.0
BUILD_PYTHON: $(WORK_DIR)/build_python
INSTALL_PYTHON: $(INSTALL_OPENVINO)/extras/python
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
- script: |
set -e
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Move jdk into contrib
# 'clang' compiler is to check that samples can be built using it
sudo apt --assume-yes install openjdk-11-jdk libbz2-dev clang
# For Python API
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/requirements.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements-dev.txt
# For running Paddle frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/paddle/tests/requirements.txt
# For running ONNX frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/onnx/tests/requirements.txt
# For running TensorFlow frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements.txt
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements_dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
# Should be after 'Install dependencies' because Git lfs is not installed
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DENABLE_PYTHON=ON
-DBUILD_SHARED_LIBS=$(CMAKE_BUILD_SHARED_LIBS)
-DENABLE_ONEDNN_FOR_GPU=$(CMAKE_BUILD_SHARED_LIBS)
-DENABLE_TESTS=ON
-DENABLE_OV_ONNX_FRONTEND=ON
-DENABLE_FASTER_BUILD=ON
-DENABLE_STRICT_DEPENDENCIES=OFF
-DENABLE_REQUIREMENTS_INSTALL=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCPACK_GENERATOR=$(CMAKE_CPACK_GENERATOR)
-DBUILD_nvidia_plugin=OFF
-S $(REPO_DIR)
-B $(BUILD_DIR)
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
ninja -C $(BUILD_DIR)
displayName: 'Build Lin'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
$(REPO_DIR)/tests/layer_tests
workingDirectory: $(BUILD_LAYER_TESTS_DIR)
- script: ninja -C $(BUILD_LAYER_TESTS_DIR)
displayName: 'Build Layer Tests'
- script: sudo apt-get remove libtbb2 -y
displayName: 'Remove debian dependencies'
condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
continueOnError: false
- script: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install wheel packages'
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_LAYER_TESTS_DIR)/cmake_install.cmake
displayName: 'Install Layer Tests'
- script: python3 -m pip install openvino-dev --find-links=$(INSTALL_DIR)/tools
displayName: 'Install python wheels'
- script: |
set -e
cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install tests'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install test files'
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
# because of static build libgna is needed for python binary
export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1910/linux/x64:$LD_LIBRARY_PATH
python3 -m pytest -s $(INSTALL_TEST_DIR)/pyngraph $(PYTHON_STATIC_ARGS) \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_backend.py
displayName: 'nGraph and IE Python Bindings Tests'
continueOnError: false
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
# because of static build libgna is needed for python binary and mock_py frontend library
export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1910/linux/x64:$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
# For python imports to import pybind_mock_frontend
export PYTHONPATH=$(INSTALL_TEST_DIR):$(INSTALL_DIR)/python/python3.8:$PYTHONPATH
# For frontend manager to load mock_py library
export OV_FRONTEND_PATH=$(INSTALL_TEST_DIR)
python3 -m pytest -sv $(INSTALL_TEST_DIR)/pyopenvino $(PYTHON_STATIC_ARGS) \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_utils/test_utils.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_backend.py
displayName: 'Python API 2.0 Tests'
continueOnError: false
- script: |
# because of static build libgna is needed for python binary
export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1910/linux/x64:$LD_LIBRARY_PATH
python3 -m pytest -s $(INSTALL_TEST_DIR)/mo/unit_tests --junitxml=$(INSTALL_TEST_DIR)/TEST-ModelOptimizer.xml
displayName: 'Model Optimizer UT'
continueOnError: false
- script: |
sudo apt-get install libtbb-dev libpugixml-dev -y
cmake --build $(BUILD_DIR) --target package -j8
condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'Build Debian packages'
continueOnError: false
- script: |
# install debian packages from previous release
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-get install --no-install-recommends gnupg -y
echo "deb https://apt.repos.intel.com/openvino/2022 focal main" | sudo tee /etc/apt/sources.list.d/intel-openvino-2022.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/intel-openvino-2022.list
# sudo apt-get install openvino -y
# install our local one and make sure the conflicts are resolved
sudo apt-get install --no-install-recommends dpkg-dev -y
rm -r _CPack_Packages
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
echo "deb [trusted=yes] file:$(BUILD_DIR) ./" | sudo tee /etc/apt/sources.list.d/openvino-local.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/openvino-local.list
sudo apt-get install openvino -y
workingDirectory: $(BUILD_DIR)
condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'Install Debian packages'
continueOnError: false
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
condition: ne(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'Install openvino'
continueOnError: false
- script: ls -alR $(INSTALL_DIR)
condition: ne(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'List install files'
- script: $(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -i $(INSTALL_DIR) -b $(BUILD_DIR)/cpp_samples
displayName: 'Build cpp samples'
continueOnError: false
- script: |
export CC=clang
export CXX=clang++
$(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -b $(BUILD_DIR)/cpp_samples_clang
displayName: 'Build cpp samples - clang'
continueOnError: false
- script: |
export VERBOSE=1
$(SAMPLES_INSTALL_DIR)/c/build_samples.sh -i $(INSTALL_DIR) -b $(BUILD_DIR)/c_samples
displayName: 'Build c samples'
continueOnError: false
- script: rm -fr $(BUILD_DIR)
displayName: 'Clean build dir'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_inference_functional_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceFunc.xml
displayName: 'Inference Func Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_inference_unit_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceUnit.xml
displayName: 'Inference Unit Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_conditional_compilation_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ConditionalCompilation.xml
displayName: 'Conditional Compilation Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_ir_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-IRFrontend.xml
displayName: 'IR Frontend Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/paddle_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Paddle.xml
displayName: 'Paddle Frontend UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_tensorflow_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Tensorflow.xml
displayName: 'TensorFlow Frontend Unit Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_lp_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-LpTransformations.xml
displayName: 'Low Precision Transformations Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Transformations.xml
displayName: 'Transformations Tests'
continueOnError: false
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --workers=16 --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_gna_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_gna_unit_tests.xml
displayName: 'GNA UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/XLinkTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-XLinkTests.xml
displayName: 'XLink Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
- script: |
# because of IR reader v7
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(RUN_PREFIX) $(INSTALL_TEST_DIR)/ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_template_func_tests --gtest_filter=*smoke* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-templateFuncTests.xml
displayName: 'TEMPLATE FuncTests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_cpu_func_tests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_func_tests.xml
displayName: 'CPU FuncTests'
continueOnError: false
condition: and(succeeded(), eq(variables['CMAKE_BUILD_SHARED_LIBS'], 'OFF'))
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(RUN_PREFIX) $(INSTALL_TEST_DIR)/InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_capi_test.xml
displayName: 'OV CAPITests'
continueOnError: false
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
$(REPO_DIR)/tests/samples_tests
workingDirectory: $(BUILD_SAMPLES_TESTS_DIR)
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_SAMPLES_TESTS_DIR)
displayName: 'Install Samples Tests'
- script: python3 -m pip install -r $(INSTALL_TEST_DIR)/smoke_tests/requirements.txt
workingDirectory: $(INSTALL_TEST_DIR)
displayName: 'Install dependencies for samples smoke tests'
continueOnError: false
- script: |
export PATH=$HOME/.local/bin:$PATH
export IE_APP_PATH=$(INSTALL_DIR)/samples_bin
export LD_LIBRARY_PATH=$IE_APP_PATH:$LD_LIBRARY_PATH
export IE_APP_PYTHON_PATH=$(PYTHON_SAMPLES_INSTALL_DIR)/
export SHARE=$(INSTALL_TEST_DIR)/smoke_tests/samples_smoke_tests_data/
export WORKSPACE=$(INSTALL_DIR)
$(RUN_PREFIX) python3 -m pytest $(INSTALL_TEST_DIR)/smoke_tests/ --env_conf $(INSTALL_TEST_DIR)/smoke_tests/env_config.yml -s --junitxml=$(INSTALL_TEST_DIR)/TEST-SamplesSmokeTests.xml
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(REPO_DIR)/tools/mo/:$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/ --use_new_frontend -m precommit_tf_fe --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_fe.xmlTEST
displayName: 'TensorFlow 1 Layer Tests - TF FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(REPO_DIR)/tools/mo/:$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow2_keras_tests/ --use_new_frontend -m precommit_tf_fe --junitxml=$(INSTALL_TEST_DIR)/TEST-tf2_fe.xmlTEST
displayName: 'TensorFlow 2 Layer Tests - TF FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/test_tf_Roll.py --ir_version=10 --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_Roll.xmlTEST
displayName: 'TensorFlow 1 Layer Tests - Legacy FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow2_keras_tests/test_tf2_keras_activation.py --ir_version=11 --junitxml=./TEST-tf2_Activation.xmlTEST -k "sigmoid"
displayName: 'TensorFlow 2 Layer Tests - Legacy FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/mo_python_api_tests/test_mo_convert_complex_params.py --ir_version=11 --junitxml=./TEST-test_mo_convert_complex_params.xmlTEST
displayName: 'MO Python API Tests - Complex Python params'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/mo_python_api_tests/test_mo_convert_tf.py --ir_version=11 --junitxml=./TEST-test_mo_convert_tf.xmlTEST
displayName: 'MO Python API Tests - Import TF model from memory'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/mo_python_api_tests/test_mo_convert_pytorch.py --ir_version=11 --junitxml=./TEST-test_mo_convert_pytorch.xmlTEST
displayName: 'MO Python API Tests - Import PyTorch model from memory'
continueOnError: false
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
#searchFolder: '$(BUILD_DIR)'
mergeTestResults: false # Optional
#failTaskOnFailedTests: false # Optional
#testRunTitle: 'Pre/Post-Commit' # Optional
buildPlatform: 'x64' # Optional
buildConfiguration: 'Linux' # Optional
#publishRunAttachments: true # Optional

277
.ci/azure/linux_arm64.yml Normal file
View File

@ -0,0 +1,277 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
jobs:
- job: linux_arm64
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
PYTHON_ARM_VERSION: "3.10.6"
PYTHON_EXEC: "python3.10"
OPENVINO_ARCH: 'aarch64'
NUM_PROC: 1
BUILD_TYPE: Release
OPENVINO_REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(OPENVINO_REPO_DIR)/../openvino_contrib
OPENCV_REPO_DIR: $(OPENVINO_REPO_DIR)/../opencv
ONETBB_REPO_DIR: $(OPENVINO_CONTRIB_REPO_DIR)/../oneTBB
BUILD_PYTHON: $(WORK_DIR)/build_python
BUILD_OPENCV: $(WORK_DIR)/build_opencv
BUILD_ONETBB: $(WORK_DIR)/build_onetbb
BUILD_OPENVINO: $(WORK_DIR)/build
BUILD_OPENVINO_PYTHON: $(WORK_DIR)/build_python
BUILD_OPEN_MODEL_ZOO: $(WORK_DIR)/build_open_model_zoo
CROSSENV_DIR: $(WORK_DIR)/cross_env
INSTALL_OPENVINO: $(WORK_DIR)/install_openvino
INSTALL_PYTHON: $(INSTALL_OPENVINO)/extras/python
INSTALL_ONETBB: $(WORK_DIR)/build/extras/oneTBB
INSTALL_ONETBB_PACKAGE: $(INSTALL_OPENVINO)/extras/oneTBB
INSTALL_OPENCV: $(INSTALL_OPENVINO)/extras/opencv
INSTALL_OPEN_MODEL_ZOO: $(INSTALL_OPENVINO)/extras/open_model_zoo
WORK_DIR: $(Pipeline.Workspace)/_w
SHARE_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
OPENVINO_CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_arm64
OPENCV_CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_arm64_opencv
ONETBB_CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_arm64_onetbb
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo ls /usr/bin/python3.10
rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3 echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo "##vso[task.setvariable variable=NUM_PROC]$(nproc --all)"
echo "NUM_PROC=$(NUM_PROC)"
displayName: 'System information'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
mkdir -p $(BUILD_ONETBB) $(BUILD_OPENCV) $(BUILD_OPENVINO) $(BUILD_OPENVINO_PYTHON) $(BUILD_PYTHON) $(BUILD_OPEN_MODEL_ZOO)
mkdir -p $(INSTALL_ONETBB) $(INSTALL_ONETBB_PACKAGE) $(INSTALL_OPENVINO) $(INSTALL_PYTHON) $(INSTALL_OPENCV) $(INSTALL_OPEN_MODEL_ZOO)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(OPENVINO_CCACHE_DIR)
mkdir -p $(OPENCV_CCACHE_DIR)
mkdir -p $(ONETBB_CCACHE_DIR)
displayName: 'Make directories'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(OPENVINO_REPO_DIR)/install_build_dependencies.sh
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
export USE_CCACHE=1
export OPENCV_CCACHE_DIR=$(OPENCV_CCACHE_DIR)
export ONETBB_CCACHE_DIR=$(ONETBB_CCACHE_DIR)
export PYTHON_ARM_VERSION=$(PYTHON_ARM_VERSION)
export NUM_PROC=$(NUM_PROC)
export BUILD_PYTHON=$(BUILD_PYTHON)
export WORK_DIR=$(WORK_DIR)
export INSTALL_PYTHON=$(INSTALL_PYTHON)
export BUILD_TYPE=$(BUILD_TYPE)
export OPENVINO_REPO_DIR=$(OPENVINO_REPO_DIR)
export BUILD_ONETBB=$(BUILD_ONETBB)
export INSTALL_ONETBB=$(INSTALL_ONETBB)
export INSTALL_OPENCV=$(INSTALL_OPENCV)
export PYTHON_EXEC=$(PYTHON_EXEC)
export ONETBB_REPO_DIR=$(ONETBB_REPO_DIR)
export OPENCV_REPO_DIR=$(OPENCV_REPO_DIR)
export BUILD_OPENCV=$(BUILD_OPENCV)
export INSTALL_OPENVINO=$(INSTALL_OPENVINO)
$(OPENVINO_CONTRIB_REPO_DIR)/modules/arm_plugin/scripts/install_build_dependencies.sh
python3 -m pip install --upgrade pip
python3 -m pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Install dependencies'
- script: |
set -e
/usr/local/bin/$(PYTHON_EXEC) -m pip install -U pip
/usr/local/bin/$(PYTHON_EXEC) -m pip install crossenv
/usr/local/bin/$(PYTHON_EXEC) -m crossenv $(INSTALL_PYTHON)/bin/$(PYTHON_EXEC) $(CROSSENV_DIR)
source $(CROSSENV_DIR)/bin/activate
build-pip3 install -U pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
cross-pip3 install -U pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Create crossenv'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_COMPILE_WARNING_AS_ERROR=OFF
-DOpenCV_DIR=$(INSTALL_OPENCV)/cmake
-DENABLE_PYTHON=OFF
-DENABLE_TESTS=ON
-DENABLE_DATA=OFF
-DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_INTEL_MYRIAD=OFF
-DTHREADING=TBB
-DTBB_DIR=$(INSTALL_ONETBB)/lib/cmake/TBB
-DCMAKE_VERBOSE_MAKEFILE=ON
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules/arm_plugin
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DARM_COMPUTE_SCONS_JOBS=$(NUM_PROC)
-DCMAKE_INSTALL_PREFIX=$(INSTALL_OPENVINO)
$(OPENVINO_REPO_DIR)
workingDirectory: $(BUILD_OPENVINO)
displayName: 'CMake OpenVINO ARM plugin'
- script: |
export CCACHE_DIR=$(OPENVINO_CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
export USE_CCACHE=1
ninja
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Build OpenVINO ARM plugin'
- script: ninja install
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Install OpenVINO ARM plugin'
- script: |
source $(CROSSENV_DIR)/bin/activate
cmake \
-GNinja \
-DENABLE_PYTHON=ON \
-DENABLE_WHEEL=ON \
-DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake \
-DOpenVINODeveloperPackage_DIR=$(BUILD_OPENVINO) \
-DCMAKE_INSTALL_PREFIX=$(INSTALL_OPENVINO) \
-S $(OPENVINO_REPO_DIR)/src/bindings/python \
-B $(BUILD_OPENVINO_PYTHON)
deactivate
displayName: 'CMake OpenVINO python binding'
- script: |
export CCACHE_DIR=$(OPENVINO_CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
export USE_CCACHE=1
ninja
workingDirectory: $(BUILD_OPENVINO_PYTHON)
displayName: 'Build OpenVINO python binding'
- script: ninja install
workingDirectory: $(BUILD_OPENVINO_PYTHON)
displayName: 'Install OpenVINO python binding'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/local/bin/$(PYTHON_EXEC)
-DPYTHON_INCLUDE_DIR=$(INSTALL_PYTHON)/include/$(PYTHON_EXEC)
-DPYTHON_LIBRARY=$(INSTALL_PYTHON)/lib
-DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake
-DOpenVINO_DIR=$(BUILD_OPENVINO)
-DOpenCV_DIR=$(INSTALL_OPENCV)/cmake
-DCMAKE_INSTALL_PREFIX=$(INSTALL_OPEN_MODEL_ZOO)
-S $(OPENVINO_REPO_DIR)/thirdparty/open_model_zoo/demos
-B $(BUILD_OPEN_MODEL_ZOO)
displayName: 'CMake Open Model Zoo demos'
- script: ninja
workingDirectory: $(BUILD_OPEN_MODEL_ZOO)
displayName: 'Build Open Model Zoo demos'
- script: ninja install
workingDirectory: $(BUILD_OPEN_MODEL_ZOO)
displayName: 'Install Open Model Zoo demos'
- script: |
cp -r $(BUILD_OPEN_MODEL_ZOO)/$(OPENVINO_ARCH)/$(BUILD_TYPE)/* $(INSTALL_OPEN_MODEL_ZOO)/
cp -r $(INSTALL_ONETBB)/* $(INSTALL_ONETBB_PACKAGE)/
zip -9 -r $(Build.ArtifactStagingDirectory)/openvino_$(OPENVINO_ARCH)_linux.zip ./*
workingDirectory: $(INSTALL_OPENVINO)
displayName: 'Create OpenVINO ARM64 linux package'
- task: PublishBuildArtifacts@1
inputs:
pathToPublish: $(Build.ArtifactStagingDirectory)
artifactName: 'openvino_aarch64_linux'
displayName: 'Publish OpenVINO AArch64 linux package'

View File

@ -0,0 +1,159 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: LinCC
# About 150% of total time
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_FASTER_BUILD=ON
-DENABLE_PROFILING_ITT=ON
-DSELECTIVE_BUILD=COLLECT
-DENABLE_INTEL_GPU=OFF
-DENABLE_MULTI=OFF
-DENABLE_AUTO=OFF
-DENABLE_AUTO_BATCH=OFF
-DENABLE_HETERO=OFF
-DENABLE_TEMPLATE=OFF
-DENABLE_INTEL_MYRIAD_COMMON=OFF
-DENABLE_OV_ONNX_FRONTEND=OFF
-DENABLE_OV_PADDLE_FRONTEND=OFF
-DENABLE_OV_TF_FRONTEND=OFF
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: |
ninja
cmake --build . --target sea_itt_lib
workingDirectory: $(BUILD_DIR)
displayName: 'Build LinCC COLLECT'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- script: |
python3 thirdparty/itt_collector/runtool/sea_runtool.py --bindir $(REPO_DIR)/bin/intel64/Release -o $(BUILD_DIR)/itt_stat ! $(REPO_DIR)/bin/intel64/Release/benchmark_app -niter 1 -nireq 1 -m $(MODELS_PATH)/models/test_model/test_model_fp32.xml -d CPU
workingDirectory: $(REPO_DIR)
displayName: 'Code usage analysis'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DSELECTIVE_BUILD=ON
-DSELECTIVE_BUILD_STAT=$(BUILD_DIR)/*.csv
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: cmake --build $(BUILD_DIR)
displayName: 'Build LinCC ON'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files ON'
# TODO: ebable after the fix on CPU side
- script: |
$(REPO_DIR)/bin/intel64/Release/benchmark_app -niter 1 -nireq 1 -m $(MODELS_PATH)/models/test_model/test_model_fp32.xml -d CPU
workingDirectory: $(REPO_DIR)
displayName: 'Use OpenVINO after CC'

View File

@ -0,0 +1,150 @@
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
jobs:
- job: Lin
# About 150% of total time
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_coverity
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo ls /usr/bin/python3.10
rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Move jdk into contrib
sudo apt --assume-yes install openjdk-11-jdk
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
# Coverity has too many PARSE_ERROR errors with ENABLE_FASTER_BUILD=ON. Disabling FASTER_BUILD.
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_FASTER_BUILD=OFF
-DENABLE_STRICT_DEPENDENCIES=OFF
-DENABLE_REQUIREMENTS_INSTALL=OFF
-DBUILD_nvidia_plugin=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
set -e
wget https://scan.coverity.com/download/linux64 --post-data "token=$(COVERITY_TOKEN)&project=openvino" -O coverity_tool.tgz
tar xvf coverity_tool.tgz
rm coverity_tool.tgz
workingDirectory: $(WORK_DIR)
displayName: 'Install coverity tool'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
$(WORK_DIR)/cov-analysis*/bin/cov-build --dir $(BUILD_DIR)/cov-int ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin with Coverity'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- script: tar -C $(BUILD_DIR) -czvf openvino.tgz cov-int
workingDirectory: $(BUILD_DIR)
displayName: 'Pack cov-int folder for submission'
- script: |
curl --form token=$(COVERITY_TOKEN) \
--form email=$(COVERITY_USER) \
--form file=@openvino.tgz \
--form version="$(Build.SourceVersion)" \
--form description="https://github.com/openvinotoolkit/openvino/runs/$(Build.BuildNumber)" \
https://scan.coverity.com/builds?project=openvino
workingDirectory: $(BUILD_DIR)
displayName: 'Submit for analysis'

149
.ci/azure/linux_cuda.yml Normal file
View File

@ -0,0 +1,149 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
resources:
repositories:
- repository: openvino
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: CUDAPlugin_Lin
timeoutInMinutes: 60
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
HOME_DIR: $(Agent.HomeDirectory)
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_REPO_DIR: $(REPO_DIR)/../openvino
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BIN_DIR: $(OPENVINO_REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
GRADLE_VER: 7.1.1
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
echo # prev line output doesn't end with eol
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Speed up build
sudo apt --assume-yes install unzip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script: >
sudo docker pull openvino.azurecr.io/openvino_ci/cuda-ubuntu2004:2022.1 &&
sudo docker run --volume $(REPO_DIR)/../:/root/repos --volume $(WORK_DIR):/root/w
openvino.azurecr.io/openvino_ci/cuda-ubuntu2004:2022.1
bash -c "
sudo -E /root/repos/openvino/install_build_dependencies.sh &&
python3 -m pip install -r /root/repos/openvino/src/bindings/python/requirements.txt &&
cd /root/w/build &&
cmake -GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DOPENVINO_EXTRA_MODULES=/root/repos/openvino_contrib/modules
-DENABLE_INTEL_CPU=OFF
-DENABLE_INTEL_GPU=OFF
-DENABLE_INTEL_GNA=OFF
-DBUILD_arm_plugin=OFF
-DBUILD_java_api=OFF
-DENABLE_PYTHON=OFF
-DENABLE_NVIDIA=ON
-DENABLE_TESTS=ON
/root/repos/openvino &&
/root/w/ninja -v CudaFuncTests CudaUnitTests"
workingDirectory: $(WORK_DIR)
displayName: 'Docker build Lin'
- script: ls -alR $(OPENVINO_REPO_DIR)/bin/
displayName: 'List bin files'

453
.ci/azure/linux_debian.yml Normal file
View File

@ -0,0 +1,453 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Lin_Debian
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
BUILD_LAYER_TESTS_DIR: $(WORK_DIR)/build_layer_tests
BUILD_SAMPLES_TESTS_DIR: $(WORK_DIR)/build_samples_tests
INSTALL_DIR: $(WORK_DIR)/install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)/tests
LAYER_TESTS_DIR: $(INSTALL_TEST_DIR)/layer_tests
SAMPLES_INSTALL_DIR: /usr/share/openvino/samples
PYTHON_SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/share/openvino/samples/python
PYTHON_WHEEL_INSTALL_DIR: $HOME/.local/lib/python3.8/site-packages
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
- script: |
set -e
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Move jdk into contrib
# 'clang' compiler is to check that samples can be built using it
sudo apt --assume-yes install openjdk-11-jdk clang
sudo apt --assume-yes install --no-install-recommends libopencv-imgproc-dev libopencv-imgcodecs-dev
# For opencv-python: python3-setuptools and pip upgrade
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/requirements.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements-dev.txt
# For running Paddle frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/paddle/tests/requirements.txt
# For running ONNX frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/onnx/tests/requirements.txt
# For running TensorFlow frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements.txt
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements_dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
# Should be after 'Install dependencies' because Git lfs is not installed
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DENABLE_PYTHON=ON
-DENABLE_OPENCV=OFF
-DPYTHON_EXECUTABLE=/usr/bin/python3.8
-DENABLE_TESTS=ON
-DENABLE_FASTER_BUILD=ON
-DENABLE_STRICT_DEPENDENCIES=OFF
-DENABLE_REQUIREMENTS_INSTALL=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCPACK_GENERATOR=DEB
-DBUILD_nvidia_plugin=OFF
-S $(REPO_DIR)
-B $(BUILD_DIR)
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
ninja -C $(BUILD_DIR)
displayName: 'Build Lin'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-S $(REPO_DIR)/tests/layer_tests
-B $(BUILD_LAYER_TESTS_DIR)
- script: ninja -C $(BUILD_LAYER_TESTS_DIR)
displayName: 'Build Layer Tests'
- script: sudo apt-get remove libtbb2 libpugixml1v5 -y
displayName: 'Remove debian dependencies'
continueOnError: false
- script: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install wheel packages'
- script: cmake -DCOMPONENT=python_samples -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install Python Samples'
- script: cmake -DCOMPONENT=gna -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install GNA plugin'
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_LAYER_TESTS_DIR)/cmake_install.cmake
displayName: 'Install Layer Tests'
- script: python3 -m pip install openvino-dev --find-links=$(INSTALL_DIR)/tools
displayName: 'Install python wheels'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install tests'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install test files'
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
python3 -m pytest -s $(INSTALL_TEST_DIR)/pyngraph \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_backend.py
displayName: 'nGraph and IE Python Bindings Tests'
continueOnError: false
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
# Required by python imports to load requires libraries
# - tests install dir for mock_py
# - OpenVINO wheel installation dir for others frontend required by mock_py (is not part of wheel pkg)
export LD_LIBRARY_PATH=$(PYTHON_WHEEL_INSTALL_DIR)/openvino/libs:$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
# For frontend manager to load frontend libraries:
# - tests install dir for mock_py
# - OpenVINO wheel installation dir for others frontend (not required if this variable is not set)
export OV_FRONTEND_PATH=$(PYTHON_WHEEL_INSTALL_DIR)/openvino/libs:$(INSTALL_TEST_DIR)
# For python imports to import pybind_mock_frontend
export PYTHONPATH=$(INSTALL_TEST_DIR):$PYTHONPATH
python3 -m pytest -s $(INSTALL_TEST_DIR)/pyopenvino \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_utils/test_utils.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_backend.py -v
displayName: 'Python API 2.0 Tests'
continueOnError: false
- script: python3 -m pytest -s $(INSTALL_TEST_DIR)/mo/unit_tests --junitxml=$(INSTALL_TEST_DIR)/TEST-ModelOptimizer.xml
displayName: 'Model Optimizer UT'
continueOnError: false
- script: |
sudo apt-get install libtbb-dev libpugixml-dev -y
cmake --build $(BUILD_DIR) --target package -j8
displayName: 'Build Debian packages'
continueOnError: false
- script: |
# install debian packages from previous release
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-get install --no-install-recommends gnupg -y
echo "deb https://apt.repos.intel.com/openvino/2022 focal main" | sudo tee /etc/apt/sources.list.d/intel-openvino-2022.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/intel-openvino-2022.list
sudo apt-get install openvino -y || exit 1
# install our local one and make sure the conflicts are resolved
sudo apt-get install --no-install-recommends dpkg-dev -y
rm -r _CPack_Packages
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
echo "deb [trusted=yes] file:$(BUILD_DIR) ./" | sudo tee /etc/apt/sources.list.d/openvino-local.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/openvino-local.list
sudo apt-get install openvino -y
workingDirectory: $(BUILD_DIR)
displayName: 'Install Debian packages'
continueOnError: false
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: rm -fr $(BUILD_DIR)
displayName: 'Clean build dir'
continueOnError: false
- script: $(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -i $(INSTALL_DIR)
displayName: 'Build cpp samples'
continueOnError: false
- script: |
export CC=clang
export CXX=clang++
$(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -i $(INSTALL_DIR)
displayName: 'Build cpp samples - clang'
continueOnError: false
- script: $(SAMPLES_INSTALL_DIR)/c/build_samples.sh -i $(INSTALL_DIR)
displayName: 'Build c samples'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
$(INSTALL_TEST_DIR)/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/paddle_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Paddle.xml
displayName: 'Paddle Frontend UT'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/ov_tensorflow_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Tensorflow.xml
displayName: 'TensorFlow Frontend Unit Tests'
continueOnError: false
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --workers=16 --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ov_gna_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_gna_unit_tests.xml
displayName: 'GNA UT'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/XLinkTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-XLinkTests.xml
displayName: 'XLink Tests'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
- script: |
# because of IR reader v7
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
$(INSTALL_TEST_DIR)/ov_template_func_tests --gtest_filter=*smoke* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-templateFuncTests.xml
displayName: 'TEMPLATE FuncTests'
continueOnError: false
# run not all smoke filter to save time in post-commit
- script: $(INSTALL_TEST_DIR)/ov_cpu_func_tests --gtest_filter=*OVCLass*:*CoreThreadingTests* --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_func_tests.xml
displayName: 'CPU FuncTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(INSTALL_TEST_DIR)/InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(INSTALL_TEST_DIR)/ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_capi_test.xml
displayName: 'OV CAPITests'
continueOnError: false
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-S $(REPO_DIR)/tests/samples_tests
-B $(BUILD_SAMPLES_TESTS_DIR)
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_SAMPLES_TESTS_DIR)/cmake_install.cmake
displayName: 'Install Samples Tests'
- script: python3 -m pip install -r $(INSTALL_TEST_DIR)/smoke_tests/requirements.txt
displayName: 'Install dependencies for samples smoke tests'
continueOnError: false
- script: |
export PATH=$HOME/.local/bin:$PATH
export IE_APP_PATH=$(INSTALL_DIR)/samples_bin
export LD_LIBRARY_PATH=$IE_APP_PATH:$LD_LIBRARY_PATH
# for gna plugin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(INSTALL_DIR)/lib/x86_64-linux-gnu:$(INSTALL_DIR)/lib/x86_64-linux-gnu/openvino-2022.3.0
export IE_APP_PYTHON_PATH=$(PYTHON_SAMPLES_INSTALL_DIR)/
export SHARE=$(INSTALL_TEST_DIR)/smoke_tests/samples_smoke_tests_data/
export WORKSPACE=$(INSTALL_DIR)
python3 -m pytest $(INSTALL_TEST_DIR)/smoke_tests/ --env_conf $(INSTALL_TEST_DIR)/smoke_tests/env_config.yml -s --junitxml=$(INSTALL_TEST_DIR)/TEST-SamplesSmokeTests.xml
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/test_tf_Roll.py --ir_version=10 --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_Roll.xmlTEST
displayName: 'TensorFlow 1 Layer Tests - Legacy FE'
continueOnError: false
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
#searchFolder: '$(BUILD_DIR)'
mergeTestResults: false # Optional
#failTaskOnFailedTests: false # Optional
#testRunTitle: 'Pre/Post-Commit' # Optional
buildPlatform: 'x64' # Optional
buildConfiguration: 'Linux' # Optional
#publishRunAttachments: true # Optional

View File

@ -0,0 +1,80 @@
#resources:
# repositories:
# - repository: testdata
# type: github
# endpoint: openvinotoolkit
# name: openvinotoolkit/testdata
# ref: releases/2022/3
jobs:
- job: Lin_lohika
# About 150% of total time
timeoutInMinutes: 90
pool:
name: LIN_LOHIKA
variables:
system.debug: true
# VSTS_HTTP_RETRY: 5
# VSTS_HTTP_TIMEOUT: 200
# BUILD_TYPE: Release
# REPO_DIR: $(Build.Repository.LocalPath)
# MODELS_PATH: $(REPO_DIR)/../testdata
# WORK_DIR: $(Pipeline.Workspace)/_w
# BUILD_DIR: $(WORK_DIR)/build
steps:
- script: git -C ~/work/openvino fetch origin $(Build.SourceBranch)
displayName: fetch
# - checkout: self
# clean: true
# lfs: false
# submodules: recursive
# path: openvino
- checkout: none
- script: git -C ~/work/openvino checkout -m $(Build.SourceVersion) && git -C ~/work/openvino submodule update --init --recursive
displayName: checkout
# Should be after 'Install dependencies' because Git lfs is not installed
# - checkout: testdata
# clean: true
# lfs: true
# path: testdata
- script: env -C ~/work ./configreleasenolto.sh
displayName: CMake
# - task: CMake@1
# inputs:
# # CMake must get Python 3.x version by default
# cmakeArgs: >
# -GNinja
# -DCMAKE_VERBOSE_MAKEFILE=ON
# -DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
# -DENABLE_PYTHON=ON
# -DPYTHON_EXECUTABLE=/usr/bin/python3.8
# -DENABLE_TESTS=ON
# -DENABLE_OV_ONNX_FRONTEND=ON
# -DENABLE_FASTER_BUILD=ON
# -DENABLE_STRICT_DEPENDENCIES=OFF
# -DENABLE_REQUIREMENTS_INSTALL=OFF
# -DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
# $(REPO_DIR)
# workingDirectory: $(BUILD_DIR)
- script: >
env -C ~/work
./buildreleasenolto.sh
libopenvino_gapi_preproc.so
openvino_intel_cpu_plugin
openvino_intel_gpu_plugin
clDNN_unit_tests64
gpuFuncTests
displayName: Build Lin
- script: ~/work/testreleasenolto.sh
displayName: cldnn tests

View File

@ -0,0 +1,125 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
jobs:
- job: OpenVINO_ONNX_CI
strategy:
matrix:
Release:
BUILD_TYPE: 'Release'
TOX_COMMAND: 'tox && tox -e zoo_models'
Debug:
BUILD_TYPE: 'Debug'
TOX_COMMAND: 'tox'
maxParallel: 2
# About 300% of total time
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_ONNX_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
REPO_DIR: $(Build.Repository.LocalPath)
WORK_DIR: $(Pipeline.Workspace)/_w
MODELS_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
ONNX_MODEL_ZOO_SHA: "d58213534f2a4d1c4b19ba62b3bb5f544353256e"
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo apt install nfs-common -y
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(MODELS_DIR)/models_data
displayName: 'Make dirs'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- script: |
set -e
apt-get update && apt-get install -y lsb-release && apt-get clean all
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script:
src/bindings/python/tests/test_onnx/model_zoo_preprocess.sh -d $(MODELS_DIR)/models_data -o -s "$(ONNX_MODEL_ZOO_SHA)"
displayName: 'Update models'
condition: ne(variables['BUILD_TYPE'], 'Debug')
- script: |
sudo docker build \
--tag=openvino-onnx-ci-image \
--file=.ci/openvino-onnx/Dockerfile \
--build-arg BUILD_TYPE=$(BUILD_TYPE) .
displayName: 'Docker build $(BUILD_TYPE)'
- script: sudo fallocate -l 64G /swapfile ; sudo mkswap /swapfile ; sudo swapon /swapfile ; df ; free -h
displayName: 'Create swap'
- script: |
sudo docker run \
--name openvino-onnx-ci-container \
--volume $(MODELS_DIR)/models_data/model_zoo/onnx_model_zoo_$(ONNX_MODEL_ZOO_SHA):/root/.onnx/model_zoo/onnx_model_zoo \
--volume $(MODELS_DIR)/msft:/root/.onnx/model_zoo/MSFT openvino-onnx-ci-image \
/bin/bash -c "$(TOX_COMMAND)"
displayName: 'Docker run $(BUILD_TYPE)'

View File

@ -0,0 +1,177 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
jobs:
- job: onnxruntime
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_ONNX_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
ONNXRUNTIME_REPO_DIR: $(REPO_DIR)/../onnxruntime
WORK_DIR: $(Pipeline.Workspace)/_w
MODELS_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
INSTALL_DIR: $(WORK_DIR)/install_pkg/openvino
BUILD_DIR: $(WORK_DIR)/build
ONNXRUNTIME_UTILS: $(REPO_DIR)/.ci/azure/ci_utils/onnxruntime
ONNXRUNTIME_BUILD_DIR: $(ONNXRUNTIME_REPO_DIR)/build
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
displayName: 'Make dirs'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- script: |
branch=`tr -s '\n ' < $(ONNXRUNTIME_UTILS)/version`
git clone --branch $branch --single-branch --recursive https://github.com/microsoft/onnxruntime.git $(ONNXRUNTIME_REPO_DIR)
displayName: 'Clone onnxruntime'
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DENABLE_INTEL_MYRIAD_COMMON=OFF
-DENABLE_INTEL_GNA=OFF
-DENABLE_INTEL_GPU=OFF
-DENABLE_CPPLINT=OFF
-DENABLE_PROFILING_ITT=OFF
-DENABLE_SAMPLES=OFF
-DENABLE_COMPILE_TOOL=OFF
-DENABLE_OV_TF_FRONTEND=OFF
-DENABLE_OV_PADDLE_FRONTEND=OFF
-DENABLE_OPENVINO_DEBUG=OFF
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin ONNX'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install OpenVINO'
- script: |
source $(INSTALL_DIR)/setupvars.sh
CXXFLAGS="-Wno-error=deprecated-declarations" ./build.sh --config RelWithDebInfo --use_openvino CPU_FP32 --build_shared_lib --parallel --skip_tests --build_dir $(ONNXRUNTIME_BUILD_DIR)
workingDirectory: $(ONNXRUNTIME_REPO_DIR)
displayName: 'Build Lin ONNX Runtime'
- script: |
source $(INSTALL_DIR)/setupvars.sh
skip_tests=`tr -s '\n ' ':' < $(ONNXRUNTIME_UTILS)/skip_tests`
./onnxruntime_test_all --gtest_filter=-$skip_tests
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_test_all'
- script: |
source $(INSTALL_DIR)/setupvars.sh
./onnxruntime_shared_lib_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_shared_lib_test'
- script: |
source $(INSTALL_DIR)/setupvars.sh
./onnxruntime_global_thread_pools_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_global_thread_pools_test'
- script: |
source $(INSTALL_DIR)/setupvars.sh
./onnxruntime_api_tests_without_env
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_api_tests_without_env'
- script: |
source $(INSTALL_DIR)/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-converted"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-converted tests'
- script: |
source $(INSTALL_DIR)/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-operator"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-operator tests'

264
.ci/azure/mac.yml Normal file
View File

@ -0,0 +1,264 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Mac
# About 250% of total time (perfomace of Mac hosts is unstable, 360 is max)
timeoutInMinutes: 360
pool:
vmImage: 'macOS-11'
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
INSTALL_DIR: $(WORK_DIR)/install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)/tests
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
TMP_DIR: /tmp
CCACHE_DIR: $(WORK_DIR)/ccache/mac
steps:
- script: |
whoami
uname -a
echo test_debug ; which python3
echo ls /usr/bin/python3.10
sudo rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
xcrun --sdk macosx --show-sdk-version
env
sysctl -a
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: UsePythonVersion@0
inputs:
versionSpec: '3.10'
- script: |
brew install cython
brew install automake
python3 -m pip install -r $(REPO_DIR)/src/frontends/onnx/tests/requirements.txt
# Speed up build
brew install ninja
brew install ccache
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script: |
export PATH="/usr/local/opt/cython/bin:$PATH"
export CC=gcc
export CXX=g++
cmake -GNinja \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DENABLE_REQUIREMENTS_INSTALL=OFF \
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) \
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON \
-DENABLE_PYTHON=ON \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_LINKER_LAUNCHER=ccache \
-DCMAKE_C_LINKER_LAUNCHER=ccache \
-DBUILD_nvidia_plugin=OFF \
-S $(REPO_DIR) \
-B $(BUILD_DIR)
displayName: 'CMake'
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- task: Cache@2
inputs:
key: 'ccache | "$(Agent.OS)"'
path: $(CCACHE_DIR)
restoreKeys: |
ccache | "$(Agent.OS)"
displayName: Cache
- script: ccache --zero-stats --max-size=10G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=10G
ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Mac'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: |
set -e
cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install tests'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_ir_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-IRFrontend.xml
displayName: 'IR Frontend Tests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU*--gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_cpu_func_tests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_func_tests.xml
displayName: 'CPU FuncTests'
continueOnError: false
enabled: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
. $(SETUPVARS) && $(INSTALL_TEST_DIR)/InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
enabled: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
. $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_capi_test.xml
displayName: 'IE CAPITests'
continueOnError: false
enabled: false
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
#searchFolder: '$(BUILD_DIR)'
mergeTestResults: false # Optional
#failTaskOnFailedTests: false # Optional
#testRunTitle: 'Pre/Post-Commit' # Optional
buildPlatform: 'x64' # Optional
buildConfiguration: 'Mac' # Optional
#publishRunAttachments: true # Optional

365
.ci/azure/windows.yml Normal file
View File

@ -0,0 +1,365 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Win
strategy:
matrix:
Static:
CMAKE_BUILD_SHARED_LIBS: 'OFF'
# Dynamic:
# CMAKE_BUILD_SHARED_LIBS: 'ON'
maxParallel: 2
# About 150% of total time
timeoutInMinutes: 270 #Temporary change
pool:
name: WIN_VMSS_VENV_D8S_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)\..\openvino_contrib
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: $(WORK_DIR)\build
BUILD_SAMPLES_DIR: $(WORK_DIR)\build_samples
BUILD_SAMPLES_TESTS_DIR: $(WORK_DIR)\build_samples_tests
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
INSTALL_DIR: $(WORK_DIR)\install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)\tests
SETUPVARS: $(INSTALL_DIR)\setupvars.bat
PYTHON_DIR: C:\hostedtoolcache\windows\Python\3.10.7\x64
CMAKE_VERSION: 3.24.0
CMAKE_CMD: $(WORK_DIR)\cmake-$(CMAKE_VERSION)-windows-x86_64\cmake-$(CMAKE_VERSION)-windows-x86_64\bin\cmake.exe
OV_CMAKE_TOOLCHAIN_FILE: $(REPO_DIR)\cmake\toolchains\mt.runtime.win32.toolchain.cmake
PYTHON_VENV_DIR: $(WORK_DIR)\.venv
steps:
- script: |
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(WORK_DIR) & mkdir C:\hostedtoolcache\windows\Python\3.10.7
rd /Q /S $(BUILD_DIR) & mkdir C:\hostedtoolcache\windows\Python\3.10.7\x64
rd /Q /S $(BUILD_SAMPLES_DIR) & mkdir $(BUILD_SAMPLES_DIR)
rd /Q /S $(BUILD_SAMPLES_TESTS_DIR) & mkdir $(BUILD_SAMPLES_TESTS_DIR)
displayName: 'Make dir'
- script: curl -O https://www.python.org/ftp/python/3.10.7/python-3.10.7-amd64.exe
displayName: 'Download Python'
workingDirectory: $(WORK_DIR)
continueOnError: false
- script: |
python-3.10.7-amd64.exe /passive InstallAllUsers=0 Include_launcher=0 TargetDir=C:\hostedtoolcache\windows\Python\3.10.7\x64
cp C:\hostedtoolcache\windows\Python\3.8.2\x64.complete C:\hostedtoolcache\windows\Python\3.10.7\x64.complete
displayName: 'Install Python'
workingDirectory: $(WORK_DIR)
continueOnError: false
- task: UsePythonVersion@0
displayName: 'Use Python'
inputs:
versionSpec: '3.10.7'
disableDownloadFromRegistry: true
- script: |
powershell -command "Invoke-RestMethod -Headers @{\"Metadata\"=\"true\"} -Method GET -Uri http://169.254.169.254/metadata/instance/compute?api-version=2019-06-01 | format-custom"
tree C:\hostedtoolcache\windows\Python
where python
python --version
where java
java -version
wmic computersystem get TotalPhysicalMemory
wmic cpu list
wmic logicaldisk get description,name
wmic VOLUME list
set
displayName: 'System info'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- checkout: testdata
clean: true
lfs: true
path: testdata
- script: |
python -m pip install --upgrade pip
rem For running Python API tests
python -m pip install -r $(REPO_DIR)\src\bindings\python\src\compatibility\openvino\requirements-dev.txt
python -m pip install -r $(REPO_DIR)\src\bindings\python\wheel\requirements-dev.txt
python -m pip install -r $(REPO_DIR)\src\bindings\python\requirements.txt
rem For running Paddle frontend unit tests
python -m pip install -r $(REPO_DIR)\src\frontends\paddle\tests\requirements.txt
rem For running ONNX frontend unit tests
python -m pip install -r $(REPO_DIR)\src\frontends\onnx\tests\requirements.txt
rem For running TensorFlow frontend unit tests
python -m pip install -r $(REPO_DIR)\src\frontends\tensorflow\tests\requirements.txt
rem For MO unit tests
python -m pip install -r $(REPO_DIR)\tools\mo\requirements.txt
python -m pip install -r $(REPO_DIR)\tools\mo\requirements_dev.txt
rem Speed up build
powershell -command "Invoke-WebRequest https://github.com/Kitware/CMake/releases/download/v$(CMAKE_VERSION)/cmake-$(CMAKE_VERSION)-windows-x86_64.zip -OutFile cmake-$(CMAKE_VERSION)-windows-x86_64.zip"
powershell -command "Expand-Archive -Force cmake-$(CMAKE_VERSION)-windows-x86_64.zip"
powershell -command "Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip -OutFile ninja-win.zip"
powershell -command "Expand-Archive -Force ninja-win.zip"
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- powershell: |
Write-Host "##vso[task.setvariable variable=CMAKE_TOOLCHAIN_FILE]$(OV_CMAKE_TOOLCHAIN_FILE)"
condition: eq(variables['CMAKE_BUILD_SHARED_LIBS'], 'ON')
displayName: "Set cmake toolchain"
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && $(CMAKE_CMD) -G "Ninja Multi-Config" ^
-DENABLE_ONEDNN_FOR_GPU=$(CMAKE_BUILD_SHARED_LIBS) ^
-DBUILD_SHARED_LIBS=$(CMAKE_BUILD_SHARED_LIBS) ^
-DENABLE_REQUIREMENTS_INSTALL=OFF ^
-DENABLE_FASTER_BUILD=ON ^
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) ^
-DENABLE_TESTS=ON ^
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON ^
-DENABLE_STRICT_DEPENDENCIES=OFF ^
-DENABLE_PYTHON=ON ^
-DBUILD_nvidia_plugin=OFF ^
-DPYTHON_EXECUTABLE="C:\hostedtoolcache\windows\Python\3.10.7\x64\python.exe" ^
-DPYTHON_INCLUDE_DIR="C:\hostedtoolcache\windows\Python\3.10.7\x64\include" ^
-DPYTHON_LIBRARY="C:\hostedtoolcache\windows\Python\3.10.7\x64\libs\python310.lib" ^
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-S $(REPO_DIR) -B $(BUILD_DIR)
displayName: 'CMake OpenVINO'
- script: dir $(REPO_DIR)\temp\ /s
displayName: 'List temp SDKs'
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && $(CMAKE_CMD) --build $(BUILD_DIR) --config Release"
displayName: 'Build Win'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
- script: |
$(CMAKE_CMD) -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: |
python -m pip install openvino-dev --find-links=$(INSTALL_DIR)\tools
displayName: 'Install Wheels'
- script: |
call "$(MSVS_VARS_PATH)" && $(CMAKE_CMD) ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-S $(REPO_DIR)\tests\samples_tests ^
-B $(BUILD_SAMPLES_TESTS_DIR)
displayName: 'CMake Samples Tests'
- script: |
$(CMAKE_CMD) -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_SAMPLES_TESTS_DIR)\cmake_install.cmake
displayName: 'Install Samples Tests'
- script: $(INSTALL_DIR)\samples\cpp\build_samples_msvc.bat -i $(INSTALL_DIR)
displayName: 'Build cpp samples'
continueOnError: false
- script: $(INSTALL_DIR)\samples\c\build_samples_msvc.bat -i $(INSTALL_DIR)
displayName: 'Build c samples'
continueOnError: false
- script: |
python -m pip install -r $(INSTALL_TEST_DIR)\smoke_tests\requirements.txt
displayName: 'Install dependencies for samples smoke tests'
continueOnError: false
- script: |
call $(SETUPVARS) && set IE_APP_PATH=$(INSTALL_DIR)\samples_bin
set IE_APP_PYTHON_PATH=$(INSTALL_DIR)\samples\python\
set SHARE=$(INSTALL_DIR)\tests\smoke_tests\samples_smoke_tests_data\
set WORKSPACE=$(INSTALL_DIR)
python -m pytest $(INSTALL_DIR)\tests\smoke_tests\ --env_conf $(INSTALL_TEST_DIR)\smoke_tests\env_config.yml -s --junitxml=$(INSTALL_TEST_DIR)/TEST-SamplesSmokeTests.xml
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: $(CMAKE_CMD) -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)\cmake_install.cmake
displayName: 'Install tests'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: rd /Q /S $(BUILD_DIR)
displayName: 'Clean build dir'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_inference_functional_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceFunc.xml
displayName: 'Inference Func Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_inference_unit_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceUnit.xml
displayName: 'Inference Unit Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_conditional_compilation_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ConditionalCompilation.xml
displayName: 'Conditional Compilation Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_ir_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-IRFrontend.xml
displayName: 'IR Frontend Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\paddle_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-Paddle.xml
displayName: 'Paddle Frontend UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_tensorflow_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-Tensorflow.xml
displayName: 'TensorFlow Frontend Unit Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_lp_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\LpTransformations.xml
displayName: 'Low Precision Transformations Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\Transformations.xml
displayName: 'Transformations Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\InferenceEngineUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_gna_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_gna_unit_tests.xml
displayName: 'GNA UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\XLinkTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-XLinkTests.xml
displayName: 'XLink Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_template_func_tests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-templateFuncTests.xml
displayName: 'TEMPLATE FuncTests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_cpu_func_tests --gtest_filter=*smoke* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_cpu_func_tests.xml
displayName: 'CPU FuncTests'
continueOnError: false
condition: and(succeeded(), eq(variables['CMAKE_BUILD_SHARED_LIBS'], 'OFF'))
- script: |
set DATA_PATH=$(MODELS_PATH)
set MODELS_PATH=$(MODELS_PATH)
call $(SETUPVARS) && $(INSTALL_TEST_DIR)\InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
- script: |
set DATA_PATH=$(MODELS_PATH)
set MODELS_PATH=$(MODELS_PATH)
call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_capi_test.xml
displayName: 'OV CAPITests'
continueOnError: false
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
#searchFolder: '$(BUILD_DIR)'
mergeTestResults: false # Optional
#failTaskOnFailedTests: false # Optional
#testRunTitle: 'Pre/Post-Commit' # Optional
buildPlatform: 'x64' # Optional
buildConfiguration: 'Windows' # Optional
#publishRunAttachments: true # Optional

View File

@ -0,0 +1,177 @@
trigger:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
- customer_A
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: WinCC
# About 150% of total time
timeoutInMinutes: 120
pool:
name: WIN_VMSS_VENV_F8S_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: $(WORK_DIR)\build
BUILD_DIR_2: $(WORK_DIR)\build_s
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
INSTALL_DIR: $(WORK_DIR)\install_pkg
SETUPVARS: $(INSTALL_DIR)\setupvars.bat
steps:
- script: |
powershell -command "Invoke-RestMethod -Headers @{\"Metadata\"=\"true\"} -Method GET -Uri http://169.254.169.254/metadata/instance/compute?api-version=2019-06-01 | format-custom"
where python
python --version
where java
java -version
wmic computersystem get TotalPhysicalMemory
wmic cpu list
wmic logicaldisk get description,name
wmic VOLUME list
set
displayName: 'System info'
- script: |
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(BUILD_DIR_2) & mkdir $(BUILD_DIR_2)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- script: |
rem Speed up build
powershell -command "Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip -OutFile ninja-win.zip"
powershell -command "Expand-Archive -Force ninja-win.zip"
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- checkout: testdata
clean: true
lfs: true
path: testdata
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && cmake -GNinja ^
-DENABLE_FASTER_BUILD=ON ^
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON ^
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) ^
-DENABLE_PROFILING_ITT=ON ^
-DSELECTIVE_BUILD=COLLECT ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DENABLE_INTEL_GPU=OFF ^
-DENABLE_MULTI=OFF ^
-DENABLE_AUTO=OFF ^
-DENABLE_AUTO_BATCH=OFF ^
-DENABLE_HETERO=OFF ^
-DENABLE_TEMPLATE=OFF ^
-DENABLE_INTEL_MYRIAD=OFF ^
-DENABLE_INTEL_MYRIAD_COMMON=OFF ^
-DENABLE_TESTS=OFF ^
-DENABLE_OV_ONNX_FRONTEND=OFF ^
-DENABLE_OV_PADDLE_FRONTEND=OFF ^
-DENABLE_OV_TF_FRONTEND=OFF ^
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: dir $(REPO_DIR)\temp\ /s
displayName: 'List temp SDKs'
- script: |
call "$(MSVS_VARS_PATH)" && cmake --build . --config $(BUILD_TYPE) && cmake --build . --config $(BUILD_TYPE) --target sea_itt_lib
workingDirectory: $(BUILD_DIR)
displayName: 'Build Win CC COLLECT'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
- script: |
call "$(MSVS_VARS_PATH)" && python thirdparty\itt_collector\runtool\sea_runtool.py --bindir $(REPO_DIR)\bin\intel64\$(BUILD_TYPE) -o $(BUILD_DIR)\itt_stat ! $(REPO_DIR)\bin\intel64\$(BUILD_TYPE)\benchmark_app.exe -niter 1 -nireq 1 -m $(MODELS_PATH)\models\test_model\test_model_fp32.xml -d CPU
workingDirectory: $(REPO_DIR)
displayName: 'Code usage analysis'
- script: |
call "$(MSVS_VARS_PATH)" && cmake -G"Visual Studio 16 2019" ^
-DVERBOSE_BUILD=ON ^
-DENABLE_FASTER_BUILD=ON ^
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) ^
-DENABLE_PROFILING_ITT=OFF ^
-DSELECTIVE_BUILD=ON ^
-DSELECTIVE_BUILD_STAT=$(BUILD_DIR)\*.csv ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DENABLE_INTEL_GPU=OFF ^
-DENABLE_MULTI=OFF ^
-DENABLE_AUTO=OFF ^
-DENABLE_AUTO_BATCH=OFF ^
-DENABLE_HETERO=OFF ^
-DENABLE_TEMPLATE=OFF ^
-DENABLE_INTEL_MYRIAD=OFF ^
-DENABLE_INTEL_MYRIAD_COMMON=OFF ^
-DENABLE_TESTS=OFF ^
-DENABLE_OV_ONNX_FRONTEND=OFF ^
-DENABLE_OV_PADDLE_FRONTEND=OFF ^
-DENABLE_OV_TF_FRONTEND=OFF ^
$(REPO_DIR)
workingDirectory: $(BUILD_DIR_2)
displayName: 'CMake2'
- script: cmake --build . --config $(BUILD_TYPE)
workingDirectory: $(BUILD_DIR_2)
displayName: 'Build WinCC ON'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files ON'
# TODO: Enable after the fix on CPU side
# - script: |
# $(REPO_DIR)\bin\intel64\$(BUILD_TYPE)\benchmark_app.exe -niter 1 -nireq 1 -m $(MODELS_PATH)\models\test_model\test_model_fp32.xml -d CPU
# workingDirectory: $(REPO_DIR)
# displayName: 'Use OpenVINO after CC'

View File

@ -0,0 +1,75 @@
FROM ubuntu:22.04
LABEL version=2021.03.30.1
# Build configuration arguments
ARG BUILD_TYPE=Release
ARG http_proxy
ARG https_proxy
ENV http_proxy ${http_proxy}
ENV https_proxy ${https_proxy}
ENV CI=true
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED 1
# Install base dependencies
RUN apt-get update && apt-get install -y locales && apt-get clean autoclean && apt-get autoremove -y
# Set the locale to en_US.UTF-8
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
RUN apt-get update && apt-get -y --no-install-recommends install \
# OpenVINO dependencies
build-essential \
ninja-build \
cmake \
curl \
git \
unzip \
libtbb-dev \
libpugixml-dev \
wget \
# Python dependencies
python3 \
python3-pip \
python3-dev \
python3-virtualenv \
cython3 \
tox && \
apt-get clean autoclean && \
apt-get autoremove -y
# Build OpenVINO
COPY . /openvino/
WORKDIR /openvino/build
RUN cmake .. \
-G Ninja \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
-DENABLE_INTEL_MYRIAD_COMMON=OFF \
-DENABLE_INTEL_GNA=OFF \
-DENABLE_INTEL_GPU=OFF \
-DENABLE_HETERO=OFF \
-DENABLE_MULTI=OFF \
-DENABLE_BATCH=OFF \
-DENABLE_GAPI_PREPROCESSING=OFF \
-DENABLE_CPPLINT=OFF \
-DENABLE_NCC_STYLE=OFF \
-DENABLE_PROFILING_ITT=OFF \
-DENABLE_SAMPLES=OFF \
-DENABLE_OV_PADDLE_FRONTEND=OFF \
-DENABLE_OV_TF_FRONTEND=OFF \
-DENABLE_OPENVINO_DEBUG=OFF \
-DCMAKE_INSTALL_PREFIX=/openvino/dist
RUN ninja install
# Run tests via tox
WORKDIR /openvino/src/bindings/python
ENV OpenVINO_DIR=/openvino/dist/runtime/cmake
ENV LD_LIBRARY_PATH=/openvino/dist/runtime/lib:/openvino/dist/runtime/3rdparty/tbb/lib
ENV PYTHONPATH=/openvino/bin/intel64/${BUILD_TYPE}/python_api/python3.10:${PYTHONPATH}
CMD tox

14
.ci/pot/Jenkinsfile vendored Normal file
View File

@ -0,0 +1,14 @@
#!groovy
properties([
parameters([
string(defaultValue: '',
description: 'Pipeline shared library version (branch/tag/commit). Determined automatically if empty',
name: 'library_version')
])
])
loadOpenVinoLibrary {
potEntrypoint(this)
}

View File

@ -1,3 +0,0 @@
*
!install_build_dependencies.sh
!scripts/install_dependencies/install_openvino_dependencies.sh

14
.gitattributes vendored
View File

@ -65,5 +65,15 @@
*.vsdx filter=lfs diff=lfs merge=lfs -text
*.bmp filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
.github/scripts/workflow_rerun/tests/data/log_archive_with_error.zip filter=lfs diff=lfs merge=lfs -text
.github/scripts/workflow_rerun/tests/data/log_archive_wo_error.zip filter=lfs diff=lfs merge=lfs -text
#POT attributes
tools/pot/tests/data/test_cases_refs/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/models/*/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/reference_models/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/video/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/reference_fake_quantize_conf/* filter=lfs diff=lfs merge=lfs -text
/tools/pot/tests/** -pot_package
/tools/pot/tools/auxilary/** -pot_package
/tools/pot/tools/run_series_experiments.py -pot_package
/tools/pot/.pylintrc -pot_package
/tools/pot/README_dev.md -pot_package

137
.github/CODEOWNERS vendored
View File

@ -1,137 +0,0 @@
# See help here: https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners
* @openvinotoolkit/openvino-maintainers
# CI
/Jenkinsfile @openvinotoolkit/openvino-ci-maintainers
/.github/ @openvinotoolkit/openvino-ci-maintainers
/.ci/ @openvinotoolkit/openvino-ci-maintainers
/.github/CODEOWNERS @openvinotoolkit/openvino-admins @openvinotoolkit/openvino-maintainers
# Licensing:
/licensing/ @openvinotoolkit/openvino-legal-maintainers
/LICENSE @openvinotoolkit/openvino-legal-maintainers
# OpenVINO Scripts:
/scripts/ @openvinotoolkit/openvino-scripts-maintainers
/scripts/install_dependencies/ @openvinotoolkit/openvino-configuration-mgmt @openvinotoolkit/openvino-scripts-maintainers
/install_build_dependencies.sh @openvinotoolkit/openvino-scripts-maintainers @openvinotoolkit/openvino-ie-maintainers
# OpenVINO Core:
/src/inference/ @openvinotoolkit/openvino-ie-maintainers
/src/common/conditional_compilation/ @openvinotoolkit/openvino-ie-maintainers
/src/common/itt/ @openvinotoolkit/openvino-ie-maintainers
/src/common/preprocessing/ @openvinotoolkit/openvino-ie-maintainers
/src/common/util/ @openvinotoolkit/openvino-ie-maintainers
/thirdparty/ @openvinotoolkit/openvino-ie-maintainers
/.gitmodules @openvinotoolkit/openvino-ie-maintainers
/src/bindings/python/ @openvinotoolkit/openvino-ie-python-api-maintainers
/src/bindings/c/ @openvinotoolkit/openvino-c-api-maintainers
/src/bindings/js/ @openvinotoolkit/openvino-js-api-maintainers
/src/core/ @openvinotoolkit/openvino-ngraph-maintainers
# OpenVINO Transformations
/src/common/*transformations/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/src/pass/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/src/pattern/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/include/openvino/pass/ @openvinotoolkit/openvino-ie-transformations-maintainers
# OpenVINO Samples:
/samples/c/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-c-api-maintainers
/samples/cpp/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-maintainers
/samples/python/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-ie-python-api-maintainers
/samples/js/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-js-api-maintainers
/thirdparty/zlib/ @openvinotoolkit/openvino-samples-maintainers
/thirdparty/json/ @openvinotoolkit/openvino-samples-maintainers
/thirdparty/gflags/ @openvinotoolkit/openvino-samples-maintainers
/thirdparty/cnpy/ @openvinotoolkit/openvino-samples-maintainers
# OpenVINO Func Tests:
/src/tests/ @openvinotoolkit/openvino-ie-tests-maintainers @openvinotoolkit/openvino-ie-test-developers
/src/frontends/tests/frontend/shared/ @openvinotoolkit/openvino-ie-tests-maintainers
/thirdparty/gtest/ @openvinotoolkit/openvino-ie-tests-maintainers
# OpenVINO CPU:
/src/plugins/intel_cpu/ @openvinotoolkit/openvino-ie-cpu-maintainers @openvinotoolkit/openvino-ie-cpu-developers
/src/common/snippets/ @openvinotoolkit/openvino-ie-cpu-maintainers
/thirdparty/xbyak/ @openvinotoolkit/openvino-ie-cpu-maintainers
# OpenVINO LPT
/src/common/low_precision_transformations/ @openvinotoolkit/openvino-ie-lpt-maintainers
# OpenVINO GPU:
/src/plugins/intel_gpu/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
/src/tests/**/gpu/ @openvinotoolkit/openvino-ie-gpu-maintainers
/thirdparty/ocl/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
# OpenVINO NPU:
/src/plugins/intel_npu/ @openvinotoolkit/openvino-npu-maintainers @openvinotoolkit/openvino-npu-developers
# OpenVINO Auto (MULTI) plugin:
/src/plugins/auto/ @openvinotoolkit/openvino-ie-auto-multi-maintainers
# OpenVINO Auto (Batch) plugin:
/src/plugins/auto_batch/ @openvinotoolkit/openvino-auto-batch-maintainers
# OpenVINO Hetero plugin:
/src/plugins/hetero/ @openvinotoolkit/openvino-hetero-maintainers
# OpenVINO Template plugin:
/src/plugins/template/ @openvinotoolkit/openvino-ie-template-maintainers
# OpenVINO Frontends:
/src/frontends/common/ @openvinotoolkit/openvino-common-frontend-maintainers
/src/frontends/ir/ @openvinotoolkit/openvino-ir-frontend-maintainers
/src/frontends/paddle/ @openvinotoolkit/openvino-ie-paddle-maintainers
/src/frontends/tensorflow/ @openvinotoolkit/openvino-tf-frontend-maintainers
/src/frontends/tensorflow_common/ @openvinotoolkit/openvino-tf-frontend-maintainers
/src/frontends/tensorflow_lite/ @openvinotoolkit/openvino-tf-frontend-maintainers
/src/frontends/pytorch/ @openvinotoolkit/openvino-pytorch-frontend-maintainers
# OpenVINO ONNX Frontend:
/src/frontends/onnx/ @openvinotoolkit/openvino-onnx-frontend-maintainers
/thirdparty/onnx/ @openvinotoolkit/openvino-onnx-frontend-maintainers @openvinotoolkit/openvino-ie-maintainers
/thirdparty/protobuf/ @openvinotoolkit/openvino-onnx-frontend-maintainers @openvinotoolkit/openvino-tf-frontend-maintainers @openvinotoolkit/openvino-ie-maintainers
# QA Tests:
/tests/ @openvinotoolkit/openvino-tests-maintainers
/tests/layer_tests/ @openvinotoolkit/openvino-tests-maintainers @openvinotoolkit/openvino-mo-maintainers
/tests/layer_tests/pytorch_tests/ @openvinotoolkit/openvino-pytorch-frontend-maintainers
/tests/layer_tests/tensorflow_tests @openvinotoolkit/openvino-tf-frontend-maintainers
/tests/layer_tests/jax_tests @openvinotoolkit/openvino-tf-frontend-maintainers
/tests/model_hub_tests @openvinotoolkit/openvino-tf-frontend-maintainers
/tests/model_hub_tests/pytorch @openvinotoolkit/openvino-pytorch-frontend-maintainers
# Tools:
/tools/ @openvinotoolkit/openvino-tools-maintainers
/tools/benchmark_tool/ @openvinotoolkit/openvino-ie-python-api-maintainers
/tools/legacy/ @openvinotoolkit/openvino-samples-maintainers
/tools/openvino_dev/ @openvinotoolkit/openvino-tools-maintainers @openvinotoolkit/openvino-ie-python-api-maintainers
/tools/mo/ @openvinotoolkit/openvino-mo-maintainers
/tools/ovc/ @openvinotoolkit/openvino-mo-maintainers
/thirdparty/open_model_zoo/ @openvinotoolkit/omz-maintainers
# Documentation
/docs/ @openvinotoolkit/openvino-docs-maintainers
/docs/CMakeLists.txt @openvinotoolkit/openvino-ie-maintainers
/**/*.md @openvinotoolkit/openvino-docs-maintainers
/**/*.svg @openvinotoolkit/openvino-docs-maintainers
/docs/MO_DG/ @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-mo-maintainers
/docs/OV_Runtime_UG/ @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-ie-maintainers
/docs/IE_PLUGIN_DG/ @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-ie-maintainers
/docs/Extensibility_UG/ @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-ie-maintainers
/docs/snippets/ @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-ie-maintainers
/docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino_contrib-arm_plugin-maintainers
/docs/OV_Runtime_UG/supported_plugins/CPU.md @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-ie-cpu-maintainers
/docs/OV_Runtime_UG/supported_plugins/GPU*.md @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-ie-gpu-maintainers
# Configuration management
/**/setup.py @openvinotoolkit/openvino-configuration-mgmt
/**/requirements*.* @openvinotoolkit/openvino-configuration-mgmt
/docs/requirements.txt @openvinotoolkit/openvino-docs-maintainers @openvinotoolkit/openvino-configuration-mgmt
# CMake scripts
/**/cmake/ @openvinotoolkit/openvino-ie-maintainers
/**/*.cmake @openvinotoolkit/openvino-ie-maintainers
/CMakeLists.txt @openvinotoolkit/openvino-ie-maintainers

58
.github/ISSUE_TEMPLATE/bug.md vendored Normal file
View File

@ -0,0 +1,58 @@
---
name: Bug
about: Create a report to help us improve
title: "[Bug]"
labels: bug, support_request
assignees: ''
---
##### System information (version)
<!-- Example
- OpenVINO => 2020.4
- Operating System / Platform => Windows 64 Bit
- Compiler => Visual Studio 2017
- Problem classification: Model Conversion
- Framework: TensorFlow (if applicable)
- Model name: ResNet50 (if applicable)
-->
- OpenVINO=> :grey_question:
- Operating System / Platform => :grey_question:
- Compiler => :grey_question:
- Problem classification => :grey_question:
##### Detailed description
<!-- your description -->
##### Steps to reproduce
<!--
Describe your problem and steps you've done before you got to this point.
to add code example fence it with triple backticks and optional file extension
```.cpp
// C++ code example
```
or attach as .txt or .zip file
-->
##### Issue submission checklist
- [ ] I report the issue, it's not a question
<!--
OpenVINO team works with support forum, Stack Overflow and other communities
to discuss problems. Tickets with question without real issue statement will be
closed.
-->
- [ ] I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
<!--
Places to check:
* OpenVINO documentation: https://docs.openvino.ai/
* OpenVINO forum: https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit
* OpenVINO issue tracker: https://github.com/openvinotoolkit/openvino/issues?q=is%3Aissue
* Stack Overflow branch: https://stackoverflow.com/questions/tagged/openvino
-->
- [ ] There is reproducer code and related data files: images, videos, models, etc.
<!--
The best reproducer -- test case for OpenVINO that we can add to the library.
-->

View File

@ -1,106 +0,0 @@
name: Bug report
description: Help us improve OpenVINO.
title: "[Bug]: "
labels: ["bug", "support_request"]
body:
- type: markdown
attributes:
value: |
Please provide all the necessary information to expedite the response.
- type: input
id: ov_version
attributes:
label: OpenVINO Version
description: OpenVINO version, branch, or tag in OpenVINO GitHub
placeholder: 2021.4.0 LTS / Master Branch / tag 2022.3.0
validations:
required: true
- type: dropdown
id: os
attributes:
label: Operating System
description: What OS are you using?
options:
- Ubuntu 18.04 (LTS)
- Ubuntu 20.04 (LTS)
- Windows System
- Red Hat Enterprise Linux 8
- Android System
- Raspbian Stretch OS
- macOS Systems for Intel CPU
- macOS Systems for Apple Silicon
- WebAssembly
- Other (Please specify in description)
validations:
required: true
- type: dropdown
id: device_use
attributes:
label: Device used for inference
description: What hardware are you using for inference?
options:
- CPU
- GPU
- NPU
- AUTO
- HETERO
- BATCH
validations:
required: true
- type: dropdown
id: framework
attributes:
label: Framework
description: Framework used for model optimization
options:
- TensorFlow 1
- Keras (TensorFlow 2)
- Caffe
- ONNX
- PyTorch
- mxnet
- PaddlePaddle
validations:
required: false
- type: input
id: model_name
attributes:
label: Model used
description: Link to the model
placeholder: ResNet50 / YOLOv4
validations:
required: false
- type: textarea
id: bug_description
attributes:
label: Issue description
description: What issue are you having, and what did you expect to happen instead?
placeholder: "Error when performing model optimization on yolov4 model."
validations:
required: true
- type: textarea
id: step_by_step
attributes:
label: Step-by-step reproduction
description: How can we reproduce your issue?
placeholder: Please provide detailed instructions on how to reproduce the issue
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
- type: checkboxes
id: terms
attributes:
label: Issue submission checklist
description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
options:
- label: I'm reporting an issue. It's not a question.
required: true
- label: I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
required: true
- label: There is reproducer code and related data files such as images, videos, models, etc.
required: true

View File

@ -1,95 +0,0 @@
name: Build Issue report
description: Report a build or installation issue.
title: "[Build]: "
labels: ["build", "support_request"]
body:
- type: markdown
attributes:
value: |
Please provide all the necessary information to expedite the response.
- type: input
id: ov_version
attributes:
label: OpenVINO Version
description: OpenVINO version, branch, or tag in OpenVINO GitHub
placeholder: 2021.4.0 LTS / Master Branch / tag 2022.3.0
validations:
required: true
- type: dropdown
id: os
attributes:
label: Operating System
description: What OS are you using?
options:
- Ubuntu 18.04 (LTS)
- Ubuntu 20.04 (LTS)
- Ubuntu 22.04 (LTS)
- Windows System
- Red Hat Enterprise Linux 8
- OpenSUSE
- Android System
- Raspbian Stretch OS
- macOS Systems for Intel CPU
- macOS Systems for Apple Silicon
- WebAssembly
- WSL2 for Windows
- Other (Please specify in description)
validations:
required: true
- type: dropdown
id: architecture
attributes:
label: Hardware Architecture
description: What is your hardware architecture used in this test?
options:
- x86 (64 bits)
- x86 (32 bits)
- ARM (64 bits)
- ARM (32 bits)
- RISC-V
- Other (please specify in the description)
validations:
required: true
- type: textarea
id: target_platform
attributes:
label: Target Platform
description: |
You can also provide us full system log with the following command
Windows cmd - "systeminfo"
Linux terminal - "lscpu" and "lscpu -e"
placeholder: Paste your full platform/system information here
validations:
required: false
- type: textarea
id: build_description
attributes:
label: Build issue description
description: What issue are you facing during the build/installation?
placeholder: Please provide a detailed description of what happened
validations:
required: true
- type: textarea
id: build_script
attributes:
label: Build script or step-by-step to reproduce
description: How can we reproduce your issue?
placeholder: Please provide detailed instructions on how to reproduce the issue
validations:
required: false
- type: textarea
id: build_logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so there is no need for backticks.
render: shell
- type: checkboxes
id: terms
attributes:
label: Issue submission checklist
description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
options:
- label: I'm reporting an issue. It's not a question.
required: true
- label: I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
required: true

View File

@ -1,32 +0,0 @@
name: Documentation issue report
description: Report an issue with Documentation.
title: "[Docs]: "
labels: ["docs", "support_request"]
body:
- type: markdown
attributes:
value: |
Please provide all the necessary information to expedite the response.
- type: input
id: doc_link
attributes:
label: Documentation link
description: Please provide the link for the documentation issue
placeholder: e.g. intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html
validations:
required: true
- type: textarea
id: description
attributes:
label: Description
description: Provide a description of the issue you noticed.
validations:
required: true
- type: checkboxes
id: terms
attributes:
label: Issue submission checklist
description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
options:
- label: I'm reporting a documentation issue. It's not a question.
required: true

View File

@ -1,31 +0,0 @@
name: Feature request
description: Suggest a feature or improvement for the OpenVINO toolkit.
title: "[Feature Request]: "
labels: ["enhancement", "feature"]
assignees:
- octocat
body:
- type: textarea
id: request_description
attributes:
label: Request Description
description: What would you like us to improve on?
placeholder: Please provide a detailed description of your request.
validations:
required: true
- type: textarea
id: feature_usecase
attributes:
label: Feature Use Case
description: What is the use case of the feature you are proposing?
placeholder: What is the new feature use case? How will it be useful?
validations:
required: false
- type: checkboxes
id: check2
attributes:
label: Issue submission checklist
description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
options:
- label: The feature request or improvement must be related to OpenVINO
required: true

View File

@ -1,67 +0,0 @@
name: Good First Issue
description: Create a Good First Issue for new contributors.
title: "[Good First Issue]: "
labels: ["good first issue", "no_stale"]
body:
- type: textarea
id: context
attributes:
label: Context
description: |
Let the contributors know what your component is responsible for,
what's the importance of the change and why it's needed.
Keep in mind the Good First Issue is for new contributors.
placeholder: What is it and why is it important?
validations:
required: true
- type: textarea
id: todo_list
attributes:
label: What needs to be done?
description: |
Be as verbose as possible, provide a TODO list if viable.
validations:
required: true
- type: textarea
id: example_prs
attributes:
label: Example Pull Requests
description: |
Provide example Pull requests, if there are any.
validations:
required: false
- type: textarea
id: resources
attributes:
label: Resources
description: |
Any materials related to the task, such as operator specifications,
discussions, guides.
value: |
- [Contribution guide - start here!](https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md)
- [Intel DevHub Discord channel](https://discord.gg/7pVRxUwdWG) - engage in discussions, ask questions and talk to OpenVINO developers
- [How to link your Pull Request to an issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#manually-linking-a-pull-request-to-an-issue-using-the-pull-request-sidebar)
validations:
required: true
- type: textarea
id: contact_points
attributes:
label: Contact points
description: |
People who can be asked questions about the task.
placeholder: GitHub users
validations:
required: true
- type: textarea
id: ticket
attributes:
label: Ticket
description: |
Provide the ticket number, if available.
validations:
required: false

View File

@ -1,146 +0,0 @@
name: Performance Issue Report
description: This report is for the performance-related issue
title: "[Performance]: "
labels: ["performance", "support_request"]
body:
- type: markdown
attributes:
value: |
Please provide all the necessary information to expedite the response.
- type: input
id: ov_version
attributes:
label: OpenVINO Version
description: OpenVINO version, branch, or tag in OpenVINO GitHub
placeholder: 2021.4.0 LTS / Master Branch / tag 2022.3.0
validations:
required: false
- type: dropdown
id: os
attributes:
label: Operating System
description: What OS are you using?
options:
- Ubuntu 18.04 (LTS)
- Ubuntu 20.04 (LTS)
- Ubuntu 22.04 (LTS)
- Windows System
- Red Hat Enterprise Linux 8
- OpenSUSE
- Android System
- Raspbian Stretch OS
- macOS Systems for Intel CPU
- macOS Systems for Apple Silicon
- WebAssembly
- WSL2 on Windows
- Other (Please specify in description)
validations:
required: true
- type: dropdown
id: device_use
attributes:
label: Device used for inference
description: What hardware are you using for inference?
options:
- CPU
- iGPU
- dGPU
- NPU
validations:
required: false
- type: dropdown
id: openvino_installation
attributes:
label: OpenVINO installation
description: How do you install OpenVINO on your system?
options:
- PyPi
- Docker
- Build from source
- Other virtual machines
validations:
required: true
- type: dropdown
id: openvino_api
attributes:
label: Programming Language
description: What is the programming language you use in your performance test?
options:
- Python
- C++
- Other
validations:
required: true
- type: dropdown
id: architecture
attributes:
label: Hardware Architecture
description: What is your hardware architecture used in this test?
options:
- x86 (64 bits)
- x86 (32 bits)
- ARM (64 bits)
- ARM (32 bits)
- RISC-V
- Other (please specify in the description)
validations:
required: true
- type: input
id: model_name
attributes:
label: Model used
description: Link to the model
placeholder: ResNet50 / YOLOv4
validations:
required: true
- type: dropdown
id: model_quantized
attributes:
label: Model quantization
description: Is your model quantized?
options:
- 'Yes'
- 'No'
validations:
required: true
- type: textarea
id: target_platform
attributes:
label: Target Platform
description: |
You can also provide us full system log with the following command
Windows cmd - "systeminfo"
Linux terminal - "lscpu" and "lscpu -e"
placeholder: Paste your full platform/system information here
validations:
required: false
- type: textarea
id: performance_description
attributes:
label: Performance issue description
description: What issue are you having, and what did you expect to happen instead?
placeholder: |
Please provide a detailed description of what happened.
Can the issue be reproduced using benchmark_app?
validations:
required: true
- type: textarea
id: step_by_step
attributes:
label: Step-by-step reproduction
description: How can we reproduce your issue?
placeholder: Please provide detailed instructions on how to reproduce the issue
validations:
required: false
- type: checkboxes
id: terms
attributes:
label: Issue submission checklist
description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
options:
- label: I'm reporting a performance issue. It's not a question.
required: true
- label: I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
required: true
- label: There is reproducer code and related data files such as images, videos, models, etc.
required: true

View File

@ -1,35 +0,0 @@
name: Pre-release Feedback
description: Feedback on Pre-release OpenVINO
title: "[Pre-Release Feedback]:"
labels: ["Pre-release", "support_request"]
body:
- type: input
id: pre_version
attributes:
label: Pre-release Version
description: What is the pre-release version you are using?
placeholder: 2023.0.0.dev20230427
validations:
required: true
- type: textarea
id: feedback
attributes:
label: Pre-release feedback
description: What is the issue or feedback on the pre-release?
placeholder: There is an inference performance drop in OpenVINO 2022.4.
validations:
required: true
- type: textarea
id: thoughts
attributes:
label: New Feature Feedback
description: Do you have any feedback on the new features released in the pre-release?
placeholder: Any thoughts on the new features are welcome.
validations:
required: false
- type: markdown
attributes:
value: |
By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
Have a new feature you would like to see in OpenVINO? File a feature request <a href="https://github.com/openvinotoolkit/openvino/issues/new/choose">HERE</a>.
You can also implement the feature by following the <a href="https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md">CONTRIBUTING.MD</a>

View File

@ -1,4 +0,0 @@
lib/
dist/
node_modules/
coverage/

View File

@ -1 +0,0 @@
20.6.0

View File

@ -1,3 +0,0 @@
dist/
node_modules/
coverage/

View File

@ -1,16 +0,0 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": true,
"singleQuote": true,
"quoteProps": "as-needed",
"jsxSingleQuote": false,
"trailingComma": "none",
"bracketSpacing": true,
"bracketSameLine": true,
"arrowParens": "avoid",
"proseWrap": "always",
"htmlWhitespaceSensitivity": "css",
"endOfLine": "lf"
}

View File

@ -1,286 +0,0 @@
/**
* Unit tests for the action's main functionality, src/cleanupImpl.js
*/
const core = require('@actions/core');
const path = require('path');
const os = require('os');
const fs = require('fs');
const cleanupImpl = require('../src/cleanupImpl');
// Mock the GitHub Actions core library
const getInputMock = jest.spyOn(core, 'getInput').mockImplementation();
const setFailedMock = jest.spyOn(core, 'setFailed').mockImplementation();
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-cleanup-'));
const cacheRemotePath = path.join(tempDir, 'cache_remote');
const cacheFiles = ['cache_1.cache', 'cache_2.cache', 'cache_3.cache'];
const minAccessTime = 7 * 24 * 60 * 60 * 1000; // 1 week
// Mock the action's main function
const runMock = jest.spyOn(cleanupImpl, 'cleanUp');
describe('cleanup', () => {
beforeEach(() => {
jest.clearAllMocks();
// Set up file system before each test
fs.mkdirSync(cacheRemotePath, { recursive: true });
const fileSizeInBytes = 1024 * 1024 * 1024; // 1 GB
const buffer = Buffer.alloc(fileSizeInBytes);
let id = 1;
for (const cache of cacheFiles) {
const cachePath = path.join(cacheRemotePath, cache);
fs.writeFileSync(cachePath, buffer);
fs.utimesSync(
cachePath,
new Date(Date.now() - minAccessTime * id++),
new Date()
);
}
});
// Clean up mock file system after each test
afterEach(() => {
fs.rmSync(tempDir, { recursive: true });
});
it('Cleanup old files using restore key', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
});
it('Cleanup old files using key', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'cache-size':
return 1;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
});
it('Skip cleanup old files by size', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 5;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
for (const cache of cacheFiles) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(true);
}
});
it('Skip cleanup old files by atime', async () => {
for (const cache of cacheFiles) {
const cachePath = path.join(cacheRemotePath, cache);
fs.utimesSync(
cachePath,
new Date(Date.now() - minAccessTime / 2),
new Date()
);
}
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
case 'max-cache-size':
return 5;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
for (const cache of cacheFiles) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(true);
}
});
it('Cleanup recently used files by max cache limit', async () => {
for (const cache of cacheFiles) {
const cachePath = path.join(cacheRemotePath, cache);
fs.utimesSync(
cachePath,
new Date(Date.now() - minAccessTime / 2),
new Date()
);
}
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
case 'max-cache-size':
return 2;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
});
it('Test unexpected behaviour', async () => {
// Set folder permissions to read-only
fs.chmodSync(cacheRemotePath, '555');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 2;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
expect(setFailedMock).not.toHaveBeenCalled();
fs.chmodSync(cacheRemotePath, '777');
});
it('Cleanup absent directory', async () => {
const cachePathAbsent = path.join(tempDir, 'cache_remote_absent');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cachePathAbsent;
case 'key':
return 'cache';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 2;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Cleanup directory with subdirectory', async () => {
const cacheSubPath = path.join(cacheRemotePath, 'cache_subdir.cache');
fs.mkdirSync(cacheSubPath, { recursive: true });
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
// check that sub directory exists
expect(fs.existsSync(cacheSubPath)).toBe(true);
});
});

View File

@ -1,266 +0,0 @@
/**
* Unit tests for the action's main functionality, src/restoreImpl.js
*/
const core = require('@actions/core');
const path = require('path');
const os = require('os');
const fs = require('fs');
const tar = require('tar');
const restoreImpl = require('../src/restoreImpl');
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-restore-'));
const cacheLocalPath = path.join(tempDir, 'cache_local');
const cacheRemotePath = path.join(tempDir, 'cache_remote');
const cacheTmpPath = path.join(tempDir, 'cache_tmp');
const cacheFiles = ['cache_1.cache', 'cache_2.cache', 'cache_3.cache'];
const testFiles = ['file1.txt', 'file2.txt', 'file3.txt'];
// Mock the GitHub Actions core library
const getInputMock = jest.spyOn(core, 'getInput').mockImplementation();
const setOutputMock = jest.spyOn(core, 'setOutput').mockImplementation();
const setFailedMock = jest.spyOn(core, 'setFailed').mockImplementation();
// Clean up mock file system after each test
afterEach(() => {
fs.rmSync(tempDir, { recursive: true });
});
// Mock the action's main function
const runMock = jest.spyOn(restoreImpl, 'restore');
describe('restore', () => {
beforeEach(() => {
jest.clearAllMocks();
// Set up mock file system before each test
fs.mkdirSync(cacheLocalPath, { recursive: true });
fs.mkdirSync(cacheRemotePath, { recursive: true });
fs.mkdirSync(cacheTmpPath, { recursive: true });
// Define test files
const file1Path = path.join(cacheTmpPath, testFiles[0]);
fs.writeFileSync(file1Path, 'File 1 contents');
tar.c(
{
gzip: true,
file: cacheFiles[0],
cwd: cacheTmpPath,
sync: true
},
['.']
);
fs.renameSync(cacheFiles[0], path.join(cacheRemotePath, cacheFiles[0]));
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[0]),
new Date(Date.now() - 1500),
new Date(Date.now() - 1500)
);
//
const file2Path = path.join(cacheTmpPath, testFiles[1]);
fs.writeFileSync(file2Path, 'File 2 contents');
tar.c(
{
gzip: true,
file: cacheFiles[1],
cwd: cacheTmpPath,
sync: true
},
['.']
);
fs.renameSync(cacheFiles[1], path.join(cacheRemotePath, cacheFiles[1]));
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[1]),
new Date(Date.now() - 1000),
new Date(Date.now() - 1000)
);
//
const file3Path = path.join(cacheTmpPath, testFiles[2]);
fs.writeFileSync(file3Path, 'File 3 contents');
tar.c(
{
gzip: true,
file: cacheFiles[2],
cwd: cacheTmpPath,
sync: true
},
['.']
);
fs.renameSync(cacheFiles[2], path.join(cacheRemotePath, cacheFiles[2]));
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[2]),
new Date(Date.now() - 500),
new Date(Date.now() - 500)
);
});
it('gets the cache file', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
cacheFiles[2]
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
let id = 1;
for (const filename of testFiles) {
const filePath = path.join(cacheLocalPath, filename);
expect(fs.existsSync(filePath)).toBe(true);
const fileContent = fs.readFileSync(filePath, 'utf8');
expect(fileContent).toBe(`File ${id++} contents`);
}
});
it('gets the updated cache file', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
// updated cache 2 access time
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[1]),
new Date(Date.now() - 100),
new Date(Date.now() - 100)
);
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
cacheFiles[1]
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
let id = 1;
for (const filename of testFiles.slice(0, 1)) {
const filePath = path.join(cacheLocalPath, filename);
expect(fs.existsSync(filePath)).toBe(true);
const fileContent = fs.readFileSync(filePath, 'utf8');
expect(fileContent).toBe(`File ${id++} contents`);
}
// check that file3 is absent
expect(fs.existsSync(path.join(cacheLocalPath, testFiles[2]))).toBe(false);
});
it('gets the cache file and extract to absent dir', async () => {
const cacheAbsentPath = path.join(tempDir, 'cache_absent');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'path':
return cacheAbsentPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
cacheFiles[2]
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
let id = 1;
for (const filename of testFiles) {
const filePath = path.join(cacheAbsentPath, filename);
expect(fs.existsSync(filePath)).toBe(true);
const fileContent = fs.readFileSync(filePath, 'utf8');
expect(fileContent).toBe(`File ${id++} contents`);
}
});
it('Test when no cache found', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(1, 'cache-file', '');
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', false);
for (const filename of testFiles) {
const filePath = path.join(cacheLocalPath, filename);
expect(fs.existsSync(filePath)).toBe(false);
}
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Unsupported cache found', async () => {
const cacheFakePath = path.join(cacheRemotePath, 'cache-fake.cache');
fs.writeFileSync(cacheFakePath, 'Fake content');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).not.toHaveBeenCalled();
});
});

View File

@ -1,190 +0,0 @@
/**
* Unit tests for the action's main functionality, src/saveImpl.js
*/
const core = require('@actions/core');
const path = require('path');
const os = require('os');
const fs = require('fs');
const saveImpl = require('../src/saveImpl');
// Mock the GitHub Actions core library
const getInputMock = jest.spyOn(core, 'getInput').mockImplementation();
const setOutputMock = jest.spyOn(core, 'setOutput').mockImplementation();
const setFailedMock = jest.spyOn(core, 'setFailed').mockImplementation();
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-'));
const cacheLocalPath = path.join(tempDir, 'cache_local');
const cacheRemotePath = path.join(tempDir, 'cache_remote');
// Clean up mock file system after each test
afterEach(() => {
fs.rmSync(tempDir, { recursive: true });
});
// Mock the action's main function
const runMock = jest.spyOn(saveImpl, 'save');
describe('save', () => {
beforeEach(() => {
jest.clearAllMocks();
// Set up mock file system before each test
// Create cache pathes
fs.mkdirSync(cacheLocalPath, { recursive: true });
fs.mkdirSync(cacheRemotePath, { recursive: true });
// Define test files
const file1Path = path.join(cacheLocalPath, 'file1.txt');
const file2Path = path.join(cacheLocalPath, 'file2.txt');
// Write content to the test files
fs.writeFileSync(file1Path, 'File 1 contents');
fs.writeFileSync(file2Path, 'File 2 contents');
});
it('Cache files pass', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
'cache.cache'
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
expect(fs.existsSync(`${cacheRemotePath}/cache.cache`)).toBe(true); // Check if the tar file was created
});
it('Cache files: absent local cache dir', async () => {
// Set the action's inputs as return values from core.getInput()
const cachePathAbsent = path.join(tempDir, 'cache_local_absent');
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cachePathAbsent;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(fs.existsSync(`${cacheRemotePath}/cache.cache`)).toBe(false); // Check that the tar file was not created
});
it('Cache files: absent remote cache dir', async () => {
// Set the action's inputs as return values from core.getInput()
const cacheRemotePathAbsent = path.join(tempDir, 'cache_remote_absent');
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePathAbsent;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
'cache.cache'
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
expect(fs.existsSync(`${cacheRemotePathAbsent}/cache.cache`)).toBe(true); // Check if the tar file was created
});
it('Cache files: key is not set', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Cache files: file already exists', async () => {
const cacheFakePath = path.join(cacheRemotePath, 'cache-fake.cache');
fs.writeFileSync(cacheFakePath, 'Fake content');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache-fake';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Test unexpected behaviour', async () => {
// Set folder permissions to read-only
fs.chmodSync(cacheRemotePath, '555');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).toHaveBeenCalled();
});
});

View File

@ -1,60 +0,0 @@
name: 'Cache'
description: 'Cache to the local file system'
author: 'Mikhail Ryzhov'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
path:
description:
'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description:
'An ordered list of keys to use for restoring stale cache if no cache hit
occurred for key.'
default: ''
required: false
save-always:
description:
'Run the post step to save the cache even if another step before fails'
default: false
required: false
cleanup-always:
description:
'Run the post cleanup step to remove old cache files even if another step
before fails'
default: false
required: false
cache-size:
description:
'Allowed cache storage size in Gb. Least recently used caches will be
automatically evicted to limit the total cache storage. The cache size can
exceed this value, but it is limited by "max-cache-size" value'
default: 10
required: false
max-cache-size:
description:
'Maximum cache storage size in Gb. All cache files will be automatically
evicted to limit the total cache storage'
default: 50
required: false
outputs:
cache-hit:
description:
'A boolean value to indicate an exact match was found for the primary key'
cache-file:
description: 'Found cache file'
runs:
using: node20
main: 'dist/restore/index.js'
post: 'dist/save/index.js'
post-if:
'success() || github.event.inputs.save-always ||
github.event.inputs.cleanup-always'

View File

@ -1,37 +0,0 @@
name: 'Cleanup Cache'
description: 'Remove old cache artifacts'
author: 'OpenVINO Developers'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description:
'An ordered list of keys to use for restoring stale cache if no cache hit
occurred for key.'
default: ''
required: false
cache-size:
description:
'Allowed cache storage size in Gb. Least recently used caches will be
automatically evicted to limit the total cache storage. The cache size can
exceed this value, but it is limited by "max-cache-size" value'
default: 10
required: false
max-cache-size:
description:
'Maximum cache storage size in Gb. All cache files will be automatically
evicted to limit the total cache storage'
default: 50
required: false
runs:
using: 'node20'
main: '../dist/cleanup-only/index.js'
branding:
icon: 'archive'
color: 'gray-dark'

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -1,50 +0,0 @@
env:
commonjs: true
es6: true
jest: true
node: true
globals:
Atomics: readonly
SharedArrayBuffer: readonly
ignorePatterns:
- '!.*'
- '**/node_modules/.*'
- '**/dist/.*'
- '**/coverage/.*'
- '*.json'
parser: '@babel/eslint-parser'
parserOptions:
ecmaVersion: 2023
sourceType: module
requireConfigFile: false
babelOptions:
babelrc: false
configFile: false
presets:
- jest
plugins:
- jest
extends:
- eslint:recommended
- plugin:github/recommended
- plugin:jest/recommended
rules:
{
'camelcase': 'warn',
'eslint-comments/no-use': 'off',
'eslint-comments/no-unused-disable': 'off',
'i18n-text/no-en': 'off',
'import/no-commonjs': 'off',
'import/no-namespace': 'off',
'no-console': 'off',
'no-unused-vars': 'warn',
'prettier/prettier': 'error',
'semi': 'error'
}

View File

@ -1,18 +0,0 @@
# Unordered list style
MD004:
style: dash
# Ordered list item prefix
MD029:
style: one
# Spaces after list markers
MD030:
ul_single: 1
ol_single: 1
ul_multi: 1
ol_multi: 1
# Code block style
MD046:
style: fenced

View File

@ -1,10 +0,0 @@
rules:
document-end: disable
document-start:
level: warning
present: false
line-length:
level: warning
max: 80
allow-non-breakable-words: true
allow-non-breakable-inline-mappings: true

10054
.github/actions/cache/package-lock.json generated vendored

File diff suppressed because it is too large Load Diff

View File

@ -1,73 +0,0 @@
{
"name": "compressed-cache-action",
"description": "action to save/restore cache as tarballs",
"version": "0.0.1",
"author": "OpenVINO Developers",
"private": true,
"keywords": [
"GitHub",
"Actions",
"JavaScript"
],
"engines": {
"node": ">=20"
},
"main": "dist/restore/index.js",
"scripts": {
"bundle": "npm run format:write && npm run package",
"ci-test": "npx jest",
"coverage": "npx jest -- --coverage",
"format:write": "npx prettier --write .",
"format:check": "npx prettier --check .",
"lint": "npx eslint . -c ./linters/.eslintrc.yml ",
"package": "npx ncc build src/restore.js -o dist/restore && npx ncc build src/save.js -o dist/save && npx ncc build src/restore.js -o dist/restore-only && npx ncc build src/save.js -o dist/save-only && npx ncc build src/cleanup.js -o dist/cleanup-only",
"package:watch": "npm run package -- --watch",
"test": "npx jest",
"all": "npm run format:write && npm run lint && npm run test && npm run coverage && npm run package"
},
"eslintConfig": {
"extends": "./linters/.eslintrc.yml"
},
"jest": {
"verbose": true,
"clearMocks": true,
"testEnvironment": "node",
"moduleFileExtensions": [
"js"
],
"testMatch": [
"**/*.test.js"
],
"testPathIgnorePatterns": [
"/node_modules/",
"/dist/"
],
"coverageReporters": [
"json-summary",
"text",
"lcov"
],
"collectCoverage": true,
"collectCoverageFrom": [
"./src/**"
]
},
"dependencies": {
"@actions/core": "^1.10.1",
"fs": "^0.0.1-security",
"tar": "^6.2.0"
},
"devDependencies": {
"@babel/core": "^7.24.0",
"@babel/eslint-parser": "^7.23.10",
"@babel/preset-env": "^7.24.0",
"@vercel/ncc": "^0.38.1",
"babel-preset-jest": "^29.6.3",
"eslint": "^8.57.0",
"eslint-plugin-github": "^4.10.2",
"eslint-plugin-jest": "^27.9.0",
"fs": "0.0.1-security",
"jest": "^29.7.0",
"prettier": "^3.2.5"
}
}

View File

@ -1,35 +0,0 @@
name: 'Restore Cache'
description: 'Restore Cache artifacts to improve workflow execution time'
author: 'OpenVINO Developers'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
path:
description:
'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description:
'An ordered list of keys to use for restoring stale cache if no cache hit
occurred for key.'
default: ''
required: false
outputs:
cache-hit:
description:
'A boolean value to indicate an exact match was found for the primary key'
cache-file:
description: 'Found cache file'
runs:
using: 'node20'
main: '../dist/restore-only/index.js'
branding:
icon: 'archive'
color: 'gray-dark'

View File

@ -1,22 +0,0 @@
name: 'Save a cache'
description: 'Cache to the local file system'
author: 'OpenVINO Developers'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
path:
description:
'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
runs:
using: 'node20'
main: '../dist/save-only/index.js'
branding:
icon: 'archive'
color: 'gray-dark'

View File

@ -1,3 +0,0 @@
const { cleanUp } = require('./cleanupImpl');
cleanUp();

View File

@ -1,85 +0,0 @@
const core = require('@actions/core');
const fs = require('fs/promises');
const path = require('path');
const {
getSortedCacheFiles,
humanReadableFileSize,
calculateTotalSize
} = require('./utils');
// Function to remove old files if their combined size exceeds the allowed size
async function cleanUp() {
try {
const cacheRemotePath = core.getInput('cache-path', { required: true });
const key = core.getInput('key', { required: true });
const keysRestore = core
.getInput('restore-keys', { required: false })
.split('\n')
.map(s => s.replace(/^!\s+/, '!').trim())
.filter(x => x !== '');
const cacheSize = core.getInput('cache-size', { required: false });
const cacheMaxSize = core.getInput('max-cache-size', { required: false });
// Minimum time peroid in milliseconds when the files was not useds
const minAccessTime = 7 * 24 * 60 * 60 * 1000; // 1 week
const currentDate = new Date();
const minAccessDateAgo = new Date(currentDate - minAccessTime);
core.debug(`cache-path: ${cacheRemotePath}`);
core.debug(`key: ${key}`);
core.debug(`restore-keys: ${keysRestore}`);
core.debug(`cache-size: ${cacheSize}`);
core.debug(`max-cache-size: ${cacheMaxSize}`);
let keyPattern = key;
if (keysRestore && keysRestore.length) {
keyPattern = keysRestore.join('|');
}
const files = await getSortedCacheFiles(cacheRemotePath, keyPattern);
const minCacheSizeInBytes = cacheSize * 1024 * 1024 * 1024;
const maxCacheSizeInBytes = cacheMaxSize * 1024 * 1024 * 1024;
let totalSize = await calculateTotalSize(cacheRemotePath, files);
if (totalSize <= minCacheSizeInBytes) {
core.info(
`The cache storage size ${humanReadableFileSize(totalSize)} less then the allowed size ${humanReadableFileSize(minCacheSizeInBytes)}`
);
return;
}
core.info(
`The cache storage size ${humanReadableFileSize(totalSize)} exceeds allowed size ${humanReadableFileSize(minCacheSizeInBytes)}`
);
for (const file of files.reverse()) {
const filePath = path.join(cacheRemotePath, file);
const fileStats = await fs.stat(filePath);
// skipping recently used files if total cache size less then maxCacheSizeInBytes
if (
!fileStats.isFile() ||
(fileStats.atime >= minAccessDateAgo && totalSize < maxCacheSizeInBytes)
) {
core.info(`Skipping: ${filePath}`);
continue;
}
core.info(`Removing file: ${filePath}`);
await fs.unlink(filePath);
core.info(`${filePath} removed successfully`);
totalSize -= fileStats.size;
// Exit if total size is within limit
if (totalSize <= minCacheSizeInBytes) {
core.info('Old cache files removed successfully');
return;
}
}
} catch (error) {
core.error(`Error removing old cache files: ${error.message}`);
}
}
module.exports = {
cleanUp
};

View File

@ -1,6 +0,0 @@
/**
* The entrypoint for the action.
*/
const { restore } = require('./restoreImpl');
restore();

View File

@ -1,84 +0,0 @@
const core = require('@actions/core');
const fs = require('fs/promises');
const path = require('path');
const tar = require('tar');
const os = require('os');
const {
getSortedCacheFiles,
humanReadableFileSize,
checkFileExists
} = require('./utils');
/**
* The main function for the action.
* @returns {Promise<void>} Resolves when the action is complete.
*/
async function restore() {
try {
const cacheRemotePath = core.getInput('cache-path', { required: true });
const cacheLocalPath = core.getInput('path', { required: true });
const key = core.getInput('key', { required: true });
const keysRestore = core
.getInput('restore-keys', { required: false })
.split('\n')
.map(s => s.replace(/^!\s+/, '!').trim())
.filter(x => x !== '');
core.debug(`cache-path: ${cacheRemotePath}`);
core.debug(`path: ${cacheLocalPath}`);
core.debug(`key: ${key}`);
core.debug(`restore-keys: ${keysRestore}`);
let keyPattern = key;
if (keysRestore && keysRestore.length) {
keyPattern = keysRestore.join('|');
}
core.info(`Looking for ${keyPattern} in ${cacheRemotePath}`);
const files = await getSortedCacheFiles(cacheRemotePath, keyPattern);
if (files.length) {
const cacheFile = files[0];
const cachePath = path.join(cacheRemotePath, cacheFile);
const cacheStat = await fs.stat(cachePath);
core.info(
`Found cache file: ${cacheFile}, size: ${humanReadableFileSize(cacheStat.size)}`
);
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'cache-'));
// copy file to temp dir
await fs.copyFile(cachePath, path.join(tempDir, cacheFile));
core.info(`${cacheFile} was copied to ${tempDir}/${cacheFile}`);
// extract
if (!(await checkFileExists(cacheLocalPath))) {
await fs.mkdir(cacheLocalPath);
}
core.info(`Extracting ${cacheFile} to ${cacheLocalPath}`);
tar.x({
file: path.join(tempDir, cacheFile),
cwd: cacheLocalPath,
sync: true
});
core.info(`Cache extracted to ${cacheLocalPath}`);
core.setOutput('cache-file', cacheFile);
core.setOutput('cache-hit', true);
} else {
core.warning(
`Could not found any suitable cache files in ${cacheRemotePath} with key ${key}`
);
core.setOutput('cache-file', '');
core.setOutput('cache-hit', false);
}
} catch (error) {
// do not fail action if cache could not be restored
core.error(error.message);
}
}
module.exports = {
restore
};

View File

@ -1,14 +0,0 @@
const core = require('@actions/core');
const { save } = require('./saveImpl');
const { cleanUp } = require('./cleanupImpl');
const saveAlways = core.getInput('save-always', { required: false });
const cleanUpAlways = core.getInput('cleanup-always', { required: false });
if (saveAlways === 'true') {
save();
}
if (cleanUpAlways === 'true') {
cleanUp();
}

View File

@ -1,71 +0,0 @@
const core = require('@actions/core');
const tar = require('tar');
const fs = require('fs/promises');
const path = require('path');
const { humanReadableFileSize, checkFileExists } = require('./utils');
/**
* The main function for the action.
* @returns {Promise<void>} Resolves when the action is complete.
*/
async function save() {
try {
const cacheRemotePath = core.getInput('cache-path', { required: true });
const toCachePath = core.getInput('path', { required: true });
const key = core.getInput('key', { required: true });
core.debug(`cache-path: ${cacheRemotePath}`);
core.debug(`path: ${toCachePath}`);
core.debug(`key: ${key}`);
if (!key) {
core.warning(`Key ${key} is not specified.`);
return;
}
const tarName = `${key}.cache`;
const tarPath = path.join(cacheRemotePath, tarName);
const tarNameTmp = `${key}.tmp`;
const tarPathTmp = path.join(cacheRemotePath, tarNameTmp);
if (await checkFileExists(tarPath)) {
core.warning(`Cache file ${tarName} already exists`);
return;
}
core.info(`Preparing cache archive ${tarName}`);
tar.c(
{
gzip: true,
file: tarName,
cwd: toCachePath,
sync: true
},
['.']
);
const tarStat = await fs.stat(tarName);
core.info(
`Created cache tarball: ${tarName}, size: ${humanReadableFileSize(tarStat.size)}`
);
// remote cache directory may not be created yet
if (!(await checkFileExists(cacheRemotePath))) {
await fs.mkdir(cacheRemotePath, { recursive: true });
}
core.info('Copying cache...');
await fs.copyFile(tarName, tarPathTmp);
// After copying is done, rename file
await fs.rename(tarPathTmp, tarPath);
core.info(`${tarName} copied to ${tarPath}`);
core.setOutput('cache-file', tarName);
core.setOutput('cache-hit', true);
} catch (error) {
core.setFailed(error.message);
}
}
module.exports = {
save
};

View File

@ -1,72 +0,0 @@
const core = require('@actions/core');
const fs = require('fs');
const path = require('path');
async function getSortedCacheFiles(cachePath, key = '') {
if (!(await checkFileExists(cachePath))) {
core.warning(`${cachePath} doesn't exist`);
return [];
}
const cachePattern = new RegExp(`^((${key}).*[.]cache)$`);
const filesSorted = (await fs.promises.readdir(cachePath))
.filter(fileName => cachePattern.test(fileName))
.map(fileName => ({
name: fileName,
time: fs.statSync(path.join(cachePath, fileName)).atimeMs
}))
.sort((a, b) => b.time - a.time)
.map(file => file.name);
core.debug(
filesSorted.map(fileName => ({
name: fileName,
atime: fs.statSync(path.join(cachePath, fileName)).atimeMs
}))
);
return filesSorted;
}
function humanReadableFileSize(sizeInBytes) {
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
let id = 0;
while (sizeInBytes >= 1024 && id < units.length - 1) {
sizeInBytes /= 1024;
id++;
}
return `${sizeInBytes.toFixed(2)} ${units[id]}`;
}
// Function to calculate the total size of files in bytes
async function calculateTotalSize(dir, files) {
let totalSize = 0;
for (const file of files) {
const filePath = path.join(dir, file);
const fileStats = await fs.promises.stat(filePath);
if (fileStats.isFile()) {
totalSize += fileStats.size;
}
}
return totalSize;
}
async function checkFileExists(filePath) {
try {
await fs.promises.access(filePath);
return true;
} catch (error) {
return false;
}
}
module.exports = {
getSortedCacheFiles,
humanReadableFileSize,
calculateTotalSize,
checkFileExists
};

View File

@ -1,72 +0,0 @@
name: 'Handle Docker images'
description: 'Builds, tags and pushes a given Docker image when needed'
inputs:
images:
description: 'Image names (registry name + namespace + base name)'
required: true
registry:
description: 'Docker registry'
required: true
dockerfiles_root_dir:
description: 'Path to dockerfiles root dir relative to repository root'
required: true
push:
description: 'Push built images to registry'
required: false
default: 'true'
changed_components:
description: 'Components changed by a pull request'
required: true
outputs:
images:
description: "Images to use in workflow"
value: ${{ steps.handle_images.outputs.images }}
runs:
using: 'composite'
steps:
- name: Checkout head
uses: actions/checkout@v4
- name: Checkout base
uses: actions/checkout@v4
with:
ref: ${{ github.base_ref || github.event.merge_group.base_ref }}
sparse-checkout: ${{ inputs.dockerfiles_root_dir }}/docker_tag
path: base
- name: Install Python dependencies
uses: py-actions/py-dependency-install@v4
with:
path: "${{ github.action_path }}/requirements.txt"
update-setuptools: "false"
update-wheel: "false"
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
- name: Handle docker images
id: handle_images
shell: bash
run: |
images=$(echo "${{ inputs.images }}" | tr '\n' ',' | sed 's/,*$//')
pr="${{ github.event.pull_request.number }}"
python3 .github/actions/handle_docker/get_images_to_build.py \
-d "${{ inputs.dockerfiles_root_dir }}" \
-r "${{ inputs.registry }}" \
--images "$images" \
--head_tag_file "${{ inputs.dockerfiles_root_dir }}/docker_tag" \
--base_tag_file "base/${{ inputs.dockerfiles_root_dir }}/docker_tag" \
--docker_env_changed "${{ fromJSON(inputs.changed_components).docker_env }}" \
--dockerfiles_changed "${{ fromJSON(inputs.changed_components).dockerfiles }}" \
--docker_builder "${{ steps.buildx.outputs.name}}" \
--repo "${{ github.repository }}" \
--ref_name "${{ github.ref_name }}" \
$([[ -n $pr ]] && echo "--pr $pr" || echo '-s ${{ github.sha }}') \
$([[ -n "${{ inputs.push }}" ]] && echo "--push" || echo '')
env:
GITHUB_TOKEN: ${{ github.token }}

View File

@ -1,129 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
import json
import re
import sys
from distutils.util import strtobool
from helpers import *
from images_api import *
def parse_args():
parser = argparse.ArgumentParser(description='Returns list of Docker images to build for a given workflow')
parser.add_argument('-i', '--images', required=True, help='Comma-separated docker images')
parser.add_argument('-d', '--dockerfiles_root', required=True, help='Path to dockerfiles')
parser.add_argument('-r', '--registry', required=True, help='Docker registry name')
parser.add_argument('-s', '--commit', required=False, help='Commit SHA. If not set, --pr is used')
parser.add_argument('-b', '--docker_builder', required=False, help='Docker buildx builder name')
parser.add_argument('--pr', type=int, required=False, help='PR number, if event is pull_request')
parser.add_argument('--head_tag_file', default='.github/dockerfiles/docker_tag', help='Head docker tag file path')
parser.add_argument('--base_tag_file', default=None, required=False, help='Base docker tag file path')
parser.add_argument('--ref_name', required=False, default='', help='GitHub ref name')
parser.add_argument('--repo', default='openvinotoolkit/openvino', help='GitHub repository')
parser.add_argument('--docker_env_changed', type=lambda x: bool(strtobool(x)), default=True,
help='Whether PR changes docker env')
parser.add_argument('--dockerfiles_changed', type=lambda x: bool(strtobool(x)), default=True,
help='Whether PR changes dockerfiles')
parser.add_argument('--action_path', default='.github/actions/handle_docker', help='Path to this GitHub action')
parser.add_argument('--push', action='store_true', required=False, help='Whether to push images to registry')
parser.add_argument('--dry_run', action='store_true', required=False, help='Dry run')
args = parser.parse_args()
return args
def main():
init_logger()
logger = logging.getLogger(__name__)
args = parse_args()
for arg, value in sorted(vars(args).items()):
logger.info(f"Argument {arg}: {value}")
head_tag = Path(args.head_tag_file).read_text().strip()
base_tag_exists = args.base_tag_file and Path(args.base_tag_file).exists()
base_tag = Path(args.base_tag_file).read_text().strip() if base_tag_exists else None
all_dockerfiles = Path(args.dockerfiles_root).rglob('**/*/Dockerfile')
images = ImagesHandler(args.dry_run)
for image in all_dockerfiles:
images.add_from_dockerfile(image, args.dockerfiles_root, args.registry, head_tag, base_tag)
requested_images = set(args.images.split(','))
skip_workflow = False
missing_only = False
merge_queue_target_branch = next(iter(re.findall(f'^gh-readonly-queue/(.*)/', args.ref_name)), None)
if args.pr:
environment_affected = args.docker_env_changed or args.dockerfiles_changed
if environment_affected:
expected_tag = f'pr-{args.pr}'
if head_tag != expected_tag:
logger.error(f"Please update docker tag in {args.head_tag_file} to {expected_tag}")
sys.exit(1)
elif merge_queue_target_branch:
environment_affected = head_tag != base_tag
if environment_affected:
logger.info(f"Environment is affected by PR(s) in merge group")
else:
environment_affected = False
if environment_affected:
changeset = get_changeset(args.repo, args.pr, merge_queue_target_branch, args.commit)
changed_dockerfiles = [p for p in changeset if p.startswith(args.dockerfiles_root) and p.endswith('Dockerfile')]
if args.docker_env_changed:
logger.info(f"Common docker environment is modified, will build all requested images")
changed_images = requested_images
else:
logger.info(f"Common docker environment is not modified, will build only changed and missing images")
changed_images = set([name_from_dockerfile(d, args.dockerfiles_root) for d in changed_dockerfiles])
unchanged_images = requested_images - changed_images
unchanged_with_no_base = images.get_missing(unchanged_images, base=True)
if unchanged_with_no_base:
logger.info("The following images were unchanged, but will be built anyway since the base for them "
f"is missing in registry: {unchanged_with_no_base}")
images_to_tag = unchanged_images.difference(unchanged_with_no_base)
images_to_build = requested_images.intersection(changed_images).union(unchanged_with_no_base)
only_dockerfiles_changed = len(changeset) == len(changed_dockerfiles)
if only_dockerfiles_changed and not images_to_build:
skip_workflow = True
else:
logger.info(f"Environment is not affected, will build only missing images, if any")
images_to_build = requested_images
images_to_tag = []
missing_only = True
if not images_to_build:
logger.info(f"No images to build, will return the list of pre-built images with a new tag")
built_images = images.build(images_to_build, missing_only, args.push, args.docker_builder)
if not built_images:
logger.info(f"No images were built, a new tag will be applied to a pre-built base image if needed")
# When a custom builder is used, it allows to push the image automatically once built. Otherwise, pushing manually
if args.push and not args.docker_builder:
images.push(images_to_build, missing_only)
if environment_affected and base_tag:
images.tag(images_to_tag)
images_output = images_to_output(images.get(requested_images))
set_github_output("images", json.dumps(images_output))
if skip_workflow:
logger.info(f"Docker image changes are irrelevant for current workflow, workflow may be skipped")
set_github_output("skip_workflow", str(skip_workflow))
main()

View File

@ -1,75 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import os
import subprocess
from ghapi.all import GhApi
from pathlib import Path
def init_logger():
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(name)-15s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
def set_github_output(name: str, value: str, github_output_var_name: str = 'GITHUB_OUTPUT'):
"""Sets output variable for a GitHub Action"""
logger = logging.getLogger(__name__)
# In an environment variable "GITHUB_OUTPUT" GHA stores path to a file to write outputs to
with open(os.environ.get(github_output_var_name), 'a+') as file:
logger.info(f"Add {name}={value} to {github_output_var_name}")
print(f'{name}={value}', file=file)
def images_to_output(images: list):
images_output = {}
for image in images:
image_name, os_name = image.name.split('/', 1)
if image_name not in images_output:
images_output[image_name] = {}
images_output[image_name][os_name] = image.ref()
return images_output
def get_changeset(repo: str, pr: str, target_branch: str, commit_sha: str):
"""Returns changeset either from PR or commit"""
owner, repository = repo.split('/')
gh_api = GhApi(owner=owner, repo=repository, token=os.getenv("GITHUB_TOKEN"))
if pr:
changed_files = gh_api.pulls.list_files(pr)
elif target_branch:
target_branch_head_commit = gh_api.repos.get_branch(target_branch).commit.sha
changed_files = gh_api.repos.compare_commits(f'{target_branch_head_commit}...{commit_sha}').get('files', [])
else:
raise ValueError(f'Either "pr" or "target_branch" parameter must be non-empty')
return set([f.filename for f in changed_files])
def run(cmd: str, dry_run: bool = False, fail_on_error: bool = True):
logger = logging.getLogger('run')
logger.info(cmd)
if dry_run:
return 0, ''
with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) as proc:
for line in proc.stdout:
logger.info(line.strip())
proc.communicate()
if proc.returncode != 0:
msg = f"Command '{cmd}' returned non-zero exit status {proc.returncode}"
if fail_on_error:
raise RuntimeError(msg)
logger.warning(msg)
return proc.returncode
def name_from_dockerfile(dockerfile: str | Path, dockerfiles_root: str | Path) -> str:
image_name = str(Path(dockerfile).relative_to(dockerfiles_root).parent.as_posix())
return image_name

View File

@ -1,137 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import os
import subprocess
from pathlib import Path
from typing import Iterable
from helpers import run, name_from_dockerfile
class Image:
def __init__(self, name: str, dockerfile: Path, registry: str):
self.logger = logging.getLogger(self.__class__.__name__)
self.name = name
self.dockerfile = dockerfile
self.registry = registry
self.tag = 'latest'
self.base_tag = None
def __str__(self):
return self.name
def __eq__(self, img):
return img.name == self.name if img else False
def with_tag(self, tag: str):
self.tag = tag
return self
def with_base_tag(self, tag: str):
self.base_tag = tag
return self
def ref(self):
return f"{self.registry}/{self.name}:{self.tag}"
def base_ref(self):
if not self.base_tag:
return None
return f"{self.registry}/{self.name}:{self.base_tag}"
def push(self, dry: bool = False):
cmd = f"docker push {self.ref()} "
run(cmd, dry)
def build(self, dry: bool = False, push: bool = True, docker_builder: str = None, import_cache: bool = True,
export_cache: bool = True):
cache_cmd = ""
if import_cache:
cache_cmd += f"--cache-from type=registry,ref={self.ref()}-cache "
if self.base_tag:
cache_cmd += f"--cache-from type=registry,ref={self.base_ref()}-cache "
if export_cache:
cache_cmd += f"--cache-to type=registry,ref={self.ref()}-cache,mode=max "
build_cmd = f"docker buildx build --builder={docker_builder}" if docker_builder else "docker build"
push_cmd = f"--push" if push else ""
cmd = f"{build_cmd} " \
f"--file {self.dockerfile} " \
f"--tag {self.ref()} " \
f"{cache_cmd} " \
f"{push_cmd} " \
"."
run(cmd, dry)
def tag_base(self, dry: bool = False):
if not self.base_tag:
raise AttributeError("Tag for base image is not specified")
cmd = f"docker buildx imagetools create -t {self.ref()} {self.base_ref()}"
run(cmd, dry)
def is_missing(self, dry: bool = False, base: bool = False) -> bool:
image = self.base_ref() if base else self.ref()
if base and not image:
self.logger.warning(f"Base ref for image {self.ref()} is missing")
return True
cmd = f"docker manifest inspect {image}"
is_missing = False
self.logger.info(cmd)
if not dry:
try:
subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True, text=True)
except subprocess.CalledProcessError:
self.logger.warning(f"{image} is missing in registry")
is_missing = True
return is_missing
# Making it a class, so it's a little easier to switch to a tree structure for building inherited images if we want
class ImagesHandler:
def __init__(self, dry_run: bool = False):
self.logger = logging.getLogger(self.__class__.__name__)
self.images = dict()
self.dry_run = dry_run
def add_from_dockerfile(self, dockerfile: str | Path, dockerfiles_root: str | Path, registry: str, tag: str,
base_tag: str = None):
image_name = name_from_dockerfile(dockerfile, dockerfiles_root)
image = Image(image_name, Path(dockerfile), registry).with_tag(tag).with_base_tag(base_tag)
self.add(image)
def add(self, image: Image):
self.images[image.name] = image
def get(self, image_names: Iterable = None) -> list:
images = [self.images[name] for name in image_names] if image_names is not None else self.images.values()
return images
def get_missing(self, image_names: Iterable = None, base: bool = False) -> list:
missing_images = [image.name for image in self.get(image_names) if image.is_missing(self.dry_run, base)]
return missing_images
def build(self, image_names: Iterable = None, missing_only: bool = False, push: bool = True, builder: str = None,
import_cache: bool = True, export_cache: bool = True):
to_build = self.get(self.get_missing(image_names)) if missing_only else self.get(image_names)
for image in to_build:
image.build(self.dry_run, push, builder, import_cache, export_cache)
return to_build
def push(self, image_names: Iterable = None, missing_only: bool = False):
to_push = self.get(self.get_missing(image_names)) if missing_only else self.get(image_names)
for image in to_push:
image.push(self.dry_run)
def tag(self, image_names: Iterable = None):
for image in self.get(image_names):
image.tag_base(self.dry_run)

View File

@ -1 +0,0 @@
ghapi~=1.0.5

View File

@ -1,82 +0,0 @@
name: 'Setup Python and pip cache'
description: 'Setups Python with the provided version and sets up the pip cache'
inputs:
version:
description: 'Python version to install'
required: true
pip-cache-path:
description: 'Path on share where the pip cache is stored'
required: false
should-setup-pip-paths:
description: 'If the action should setup `PIP_CACHE_DIR` & `PIP_INSTALL_PATH` env variables'
required: false
default: 'false'
self-hosted-runner:
description: 'If the runner is self-hosted'
required: false
default: 'true'
show-cache-info:
description: 'If the action should show the share space occupied by cache'
required: false
default: 'false'
runs:
using: 'composite'
steps:
- name: Check if Python is already installed (Linux)
if: ${{ runner.os == 'Linux' }}
shell: bash
id: check_python
run: |
PYTHON_INSTALLED=$(python${{ inputs.version }} -V) || true
if [[ $PYTHON_INSTALLED ]]; then
echo "installed=true" >> $GITHUB_OUTPUT
else
echo "installed=false" >> $GITHUB_OUTPUT
fi
- if: ${{ runner.os == 'Linux' && inputs.self-hosted-runner == 'true' && steps.check_python.outputs.installed == 'false' }}
name: Install 'actions/setup-python@v4' dependencies
shell: bash
run: apt-get update && apt-get install -y ca-certificates software-properties-common gpg-agent tzdata
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
TZ: "Europe/London" # to prevent tzdata from waiting user input
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' && steps.check_python.outputs.installed == 'false' }}
name: Setup sudo and python3
shell: bash
run: apt-get update && apt-get install -y sudo python3 # Needed for the deadsnakes action
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' && steps.check_python.outputs.installed == 'false' }}
name: Setup Python ${{ inputs.version }}
uses: akashchi/deadsnakes-action@92417281055a5878a0450f240a5b95883eb2d7e2
with:
python-version: ${{ inputs.version }}
- if: ${{ runner.os == 'macOS' || runner.os == 'Windows' || (runner.os == 'Linux' && runner.arch != 'ARM64' && steps.check_python.outputs.installed == 'false' ) }}
name: Setup Python ${{ inputs.version }}
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.version }}
env:
PIP_CACHE_DIR: ${{ inputs.self-hosted-runner == 'true' && inputs.pip-cache-path || '' }}
- if: ${{ inputs.should-setup-pip-paths == 'true' }}
name: Setup pip variables (cache and install path)
shell: bash
run: |
PIP_VER=$(python3 -c "import pip; print(pip.__version__)")
echo "Using pip version: ${PIP_VER}"
echo "PIP_CACHE_DIR=${{ inputs.pip-cache-path }}/${PIP_VER}" >> $GITHUB_ENV
echo "PIP_INSTALL_PATH=$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')" >> $GITHUB_ENV
- if: ${{ inputs.show-cache-info == 'true' }}
name: Get pip cache info
shell: bash
run: |
echo "Cache size: "
du -h -d2 ${{ env.PIP_CACHE_DIR }}
echo "Cache info: "
python3 -m pip cache info
continue-on-error: true

View File

@ -1,106 +0,0 @@
name: "Smart CI action"
description: "Returns product components affected by PR or commit"
inputs:
repository:
description: "GitHub repository"
required: true
repo_token:
description: "Token for access to GitHub repository"
required: true
pr:
description: "GitHub PR number. If not set - commit is used"
required: false
commit_sha:
description: "GitHub commit hash. Used if no PR number is set"
required: false
ref_name:
description: "GitHub ref name"
required: false
component_pattern:
description: "Pattern to extract component name from PR label. If not set, any label is considered a component name"
required: false
labeler_check_name:
description: "Name of the labeler check"
required: false
default: "triage"
components_config:
description: "Path to components configuration file"
required: false
default: ".github/components.yml"
components_config_schema:
description: "Path to the schema file for components configuration"
required: false
default: ".github/actions/smart-ci/components_schema.yml"
labeler_config:
description: "Path to labeler configuration file"
required: false
default: ".github/labeler.yml"
skip_when_only_listed_labels_set:
description: "Comma-separated list of labels. If PR has only these labels set,
return indicator that CI can be skipped"
required: false
skip_when_only_listed_files_changed:
description: "Comma-separated list of patterns (fnmatch-style). If PR has only matching files changed,
return indicator that CI can be skipped"
required: false
outputs:
all_components:
description: "All components listed in configuration"
value: ${{ steps.smart_ci.outputs.all_components }}
affected_components:
description: "Affected components to run validation for and their validation scope"
value: ${{ steps.smart_ci.outputs.affected_components }}
changed_components:
description: "Actually changed components (for push events everything is marked as changed)"
value: ${{ steps.smart_ci.outputs.changed_components }}
skip_workflow:
description: "Whether the workflow should be run with Smart CI rules applied or skipped completely"
value: ${{ steps.smart_ci.outputs.skip_workflow }}
runs:
using: "composite"
steps:
- name: Wait for labeler to finish
uses: lewagon/wait-on-check-action@v1.3.1
if: ${{ github.event_name == 'pull_request' }}
with:
ref: ${{ github.event.pull_request.head.sha }}
check-name: ${{ inputs.labeler_check_name }}
repo-token: ${{ inputs.repo_token }}
wait-interval: 10
- name: checkout components file
uses: actions/checkout@v4
with:
sparse-checkout: .github/components.yml
sparse-checkout-cone-mode: false
- name: Install Python dependencies
uses: py-actions/py-dependency-install@v4
with:
path: "${{ github.action_path }}/requirements.txt"
update-setuptools: "false"
update-wheel: "false"
- name: Test functionality
run: |
python ${{ github.action_path }}/smart_ci_test.py
shell: bash
- name: Smart CI
id: smart_ci
run: |
python ${{ github.action_path }}/smart_ci.py \
$([[ -n "${{ inputs.pr }}" ]] && echo '--pr ${{ inputs.pr }}' || echo '-s ${{ inputs.commit_sha }}') \
-r ${{ inputs.repository }} \
-f "${{ inputs.ref_name }}" \
-p "${{ inputs.component_pattern }}" \
-c "${{ inputs.components_config }}" \
-m "${{ inputs.components_config_schema }}" \
-l "${{ inputs.labeler_config }}" \
--skip-when-only-listed-labels-set "${{ inputs.skip_when_only_listed_labels_set }}" \
--skip-when-only-listed-files-changed "${{ inputs.skip_when_only_listed_files_changed }}"
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.repo_token }}

View File

@ -1,41 +0,0 @@
# YAML schema for Smart CI configuration file components.yml (see https://json-schema.org)
definitions:
component_name:
type: string
pattern: "^[a-zA-Z_][a-zA-Z0-9_]*$"
component_data:
type: object # dict
additionalProperties: false
properties:
cmake:
type: array
uniqueItems: true
items:
'$ref': '#/definitions/component_name'
revalidate:
oneOf:
- type: array
uniqueItems: true
items:
'$ref': '#/definitions/component_name'
- type: string
enum: ['all']
build:
oneOf:
- type: array
uniqueItems: true
items:
'$ref': '#/definitions/component_name'
- type: string
enum: ['all']
propertyNames: # Validates component names
'$ref': '#/definitions/component_name'
patternProperties:
".*": # Component (name validated via propertyNames)
'$ref': '#/definitions/component_data'
additionalProperties: false

View File

@ -1,3 +0,0 @@
ghapi~=1.0.4
pyyaml~=6.0.1
jsonschema~=4.19.1

View File

@ -1,258 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import os
import re
import argparse
import yaml
import json
import jsonschema
import logging
from pathlib import Path
from ghapi.all import GhApi
from fnmatch import fnmatch
class ComponentConfig:
FullScope = {'build', 'test'}
ScopeKeys = {'build', 'revalidate'}
def __init__(self, config: dict, schema: dict, all_possible_components: set):
self.config = config
self.log = logging.getLogger(self.__class__.__name__)
self.all_defined_components = set(self.config.keys()) # already defined in components.yml
# can be added to components.yml (based on labeler.yml)
self.all_possible_components = all_possible_components.union(self.all_defined_components)
self.validate(schema)
def validate(self, schema: dict) -> None:
"""Validates syntax of configuration file"""
jsonschema.validate(self.config, schema)
for component_name, data in self.config.items():
dependent_components = set()
for key in self.ScopeKeys:
scope = data.get(key)
dependent_components = dependent_components.union(set(scope) if isinstance(scope, list) else set())
invalid_dependents = dependent_components.difference(self.all_possible_components)
if invalid_dependents:
error_msg = f"dependent components of {component_name} are invalid: " \
f"{invalid_dependents} are not listed in components config: {self.all_possible_components}"
raise jsonschema.exceptions.ValidationError(error_msg)
def get_affected_components(self, changed_components_names: set) -> dict:
"""Returns changed components, their dependencies and validation scope for them"""
affected_components = dict()
# If some changed components were not defined in config or no changed components detected at all,
# run full scope for everything (just in case)
changed_not_defined_components = changed_components_names.difference(self.all_defined_components)
if not changed_components_names or changed_not_defined_components:
self.log.info(f"Changed components {changed_not_defined_components} are not defined in smart ci config, "
"run full scope")
affected_components.update({name: self.FullScope for name in self.all_possible_components})
return affected_components
# Else check changed components' dependencies and add them to affected
for name in changed_components_names:
component_scopes = {k: v for k, v in self.config.get(name, dict()).items() if k in self.ScopeKeys}
for key, dependents in component_scopes.items():
if dependents == 'all':
dependents = self.all_possible_components
for dep_name in dependents:
affected_components[dep_name] = affected_components.get(dep_name, set())
scope = self.FullScope if key == 'revalidate' else {key}
affected_components[dep_name] = affected_components[dep_name].union(scope)
if not component_scopes:
self.log.info(f"Changed component '{name}' doesn't have {self.ScopeKeys} keys in components config. "
f"Assuming that it affects everything, the whole scope will be started")
for dep_name in self.all_possible_components:
affected_components[dep_name] = self.FullScope
# If the component was explicitly changed, run full scope for it
affected_components.update({name: self.FullScope for name in changed_components_names})
self.log.info(f"Changed components with dependencies: {affected_components}")
# For non-affected components that are not defined in config - run full scope
affected_components.update({name: self.FullScope for name in self.all_possible_components
if name not in self.all_defined_components})
return affected_components
def get_static_data(self, components_names: set, data_key: str, default: str = None) -> dict:
"""Returns requested generic static data defined for each component"""
data = {name: self.config[name].get(data_key, default) for name in components_names}
return data
def component_name_from_label(label: str, component_pattern: str = None) -> str:
"""Extracts component name from label"""
component = label
if component_pattern:
matches = re.findall(component_pattern, label)
component = matches[0] if matches else None
component = component.replace(' ', '_') if component else None
return component
def get_changed_component_names(pr, all_possible_components: set, component_pattern: str = None) -> set:
"""Returns component names changed in a given PR"""
components = set()
for label in pr.labels:
component = component_name_from_label(label.name, component_pattern)
if component:
components.add(component)
elif label.name in all_possible_components:
# Allow any labels defined explicitly in labeler config as components
# (predefined labels, such as "do not merge", are still ignored)
components.add(label.name)
return components
def get_changeset(gh_api, pr, target_branch, commit_sha):
"""Returns changeset either from PR or commit"""
if pr:
return gh_api.pulls.list_files(pr)
if target_branch:
target_branch_head_commit = gh_api.repos.get_branch(target_branch).commit.sha
# In merge-queue branch all commits between head of target branch and head of current branch (commit_sha)
# contain changes added to queue earlier to be validated together. Getting all of them + changes from
# commit_sha below
changed_files = gh_api.repos.compare_commits(f'{target_branch_head_commit}...{commit_sha}').get('files', [])
return changed_files
raise ValueError(f'Either "pr" or "target_branch" parameter must be non-empty')
def parse_args():
parser = argparse.ArgumentParser(description='Returns product components changed in a given PR or commit')
parser.add_argument('--pr', type=int, required=False, help='PR number. If not set, --commit is used')
parser.add_argument('-s', '--commit-sha', required=False, help='Commit SHA. If not set, --pr is used')
parser.add_argument('-r', '--repo', help='GitHub repository')
parser.add_argument('-f', '--ref_name', required=False, help='GitHub ref name')
parser.add_argument('-p', '--pattern', default=None, help='Pattern to extract component name from PR label. '
'If not set, any label is considered a component name')
parser.add_argument('-c', '--components-config', default='.github/components.yml',
help='Path to config file with info about dependencies between components')
parser.add_argument('-m', '--components-config-schema', default='.github/actions/smart-ci/components_schema.yml',
help='Path to the schema file for components config')
parser.add_argument('-l', '--labeler-config', default='.github/labeler.yml',
help='Path to PR labeler config file')
parser.add_argument('--skip-when-only-listed-labels-set',
help="Comma-separated list of labels. If PR has only these labels set, "
"return indicator that CI can be skipped")
parser.add_argument('--skip-when-only-listed-files-changed',
help="Comma-separated list of patterns (fnmatch-style). If PR has only matching files changed, "
"return indicator that CI can be skipped")
args = parser.parse_args()
return args
def init_logger():
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(name)-15s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
def set_github_output(name: str, value: str, github_output_var_name: str = 'GITHUB_OUTPUT'):
"""Sets output variable for a GitHub Action"""
logger = logging.getLogger(__name__)
# In an environment variable "GITHUB_OUTPUT" GHA stores path to a file to write outputs to
with open(os.environ.get(github_output_var_name), 'a+') as file:
logger.info(f"Add {name}={value} to {github_output_var_name}")
print(f'{name}={value}', file=file)
def main():
init_logger()
logger = logging.getLogger(__name__)
args = parse_args()
for arg, value in sorted(vars(args).items()):
logger.info(f"Argument {arg}: {value}")
with open(Path(args.components_config), 'r') as config:
components_config = yaml.safe_load(config)
owner, repository = args.repo.split('/')
gh_api = GhApi(owner=owner, repo=repository, token=os.getenv("GITHUB_TOKEN"))
pr = gh_api.pulls.get(args.pr) if args.pr else None
with open(Path(args.components_config_schema), 'r') as schema_file:
schema = yaml.safe_load(schema_file)
with open(Path(args.labeler_config), 'r') as labeler_file:
labeler_config = yaml.safe_load(labeler_file)
all_possible_components = set()
for label in labeler_config.keys():
component_name = component_name_from_label(label, args.pattern)
all_possible_components.add(component_name if component_name else label)
run_full_scope = False
# For now, we don't want to apply smart ci rules for post-commits
is_postcommit = not pr
merge_queue_prefix = 'gh-readonly-queue/'
is_merge_queue = args.ref_name.startswith(merge_queue_prefix)
merge_queue_target_branch = re.findall(f'^{merge_queue_prefix}(.*)/', args.ref_name)[0] if is_merge_queue else None
if is_merge_queue:
logger.info(f"The run is a merge-queue run, executing full validation scope for all components, if "
f"not all queued changes match patterns in 'skip-when-only-listed-files-changed'")
run_full_scope = True
elif is_postcommit:
logger.info(f"The run is a post-commit run, executing full validation scope for all components")
run_full_scope = True
else:
no_match_files_changed = 'no-match-files' in [label.name for label in pr.labels]
if no_match_files_changed:
logger.info(f"There are changed files that don't match any pattern in labeler config, "
f"executing full validation scope for all components")
run_full_scope = True
# In post-commits - validate all components regardless of changeset
# In pre-commits - validate only changed components with their dependencies
all_defined_components = components_config.keys()
changed_by_pr = get_changed_component_names(pr, all_possible_components, args.pattern) if pr else None
changed_component_names = set(all_defined_components) if run_full_scope else changed_by_pr
logger.info(f"changed_component_names: {changed_component_names}")
cfg = ComponentConfig(components_config, schema, all_possible_components)
affected_components = cfg.get_affected_components(changed_component_names)
skip_workflow = False
if is_merge_queue or (args.pr and not run_full_scope):
if args.skip_when_only_listed_labels_set:
excepted_labels = set(args.skip_when_only_listed_labels_set.split(','))
excepted_labels_only = changed_component_names - excepted_labels == set()
skip_workflow = excepted_labels_only
if not skip_workflow and args.skip_when_only_listed_files_changed:
# To avoid spending extra API requests running step below only if necessary
changed_files = get_changeset(gh_api, args.pr, merge_queue_target_branch, args.commit_sha)
patterns = set(args.skip_when_only_listed_files_changed.split(','))
matched_files_only = all(any(fnmatch(f.filename, pattern) for pattern in patterns) for f in changed_files)
logger.debug(f"matched files only: {matched_files_only}")
skip_workflow = matched_files_only
if skip_workflow:
logger.info(f"All changes are marked for skip, workflow may be skipped")
set_github_output("skip_workflow", str(skip_workflow))
# Syntactic sugar for easier use in GHA pipeline
affected_components_output = {name: {s: True for s in scope} for name, scope in affected_components.items()}
set_github_output("affected_components", json.dumps(affected_components_output))
# Components actually changed by a pull request are marked as True (if event is PR;
# otherwise all components considered changed).
changed_components_output = {name: True if not pr or name in changed_by_pr else False
for name in all_possible_components}
set_github_output("changed_components", json.dumps(changed_components_output))
if __name__ == '__main__':
main()

View File

@ -1,217 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import sys
import unittest
from smart_ci import ComponentConfig
log = logging.getLogger()
log.level = logging.DEBUG
def log_handler(func):
def wrapper(*args, **kwargs):
stream_handler = logging.StreamHandler(sys.stdout)
log.addHandler(stream_handler)
result = func(*args, **kwargs)
log.removeHandler(stream_handler)
return result
return wrapper
class TestComponentConfig(unittest.TestCase):
def setUp(self):
self.all_possible_components = {'comp1', 'comp2', 'comp3', 'comp4'}
ComponentConfig.ScopeKeys = {'build', 'revalidate', '_scope_1', '_scope_2', '_scope_3'}
@log_handler
def validate(self, config_data: dict, changed_components: set, expected_result: dict):
log.info(f"{self._testMethodName}:")
config = ComponentConfig(config_data, {}, self.all_possible_components)
result = config.get_affected_components(changed_components)
self.assertEqual(expected_result, result)
def test_no_changed_components(self):
config_data = {
'comp1': {'build': {}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = set()
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': ComponentConfig.FullScope,
'comp3': ComponentConfig.FullScope,
'comp4': ComponentConfig.FullScope,
}
self.validate(config_data, changed_components, expected_result)
def test_all_components_changed(self):
config_data = {
'comp1': {'build': {}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1', 'comp2', 'comp3', 'comp4'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': ComponentConfig.FullScope,
'comp3': ComponentConfig.FullScope,
'comp4': ComponentConfig.FullScope,
}
self.validate(config_data, changed_components, expected_result)
def test_changed_component_not_defined(self):
config_data = {
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': ComponentConfig.FullScope,
'comp3': ComponentConfig.FullScope,
'comp4': ComponentConfig.FullScope,
}
self.validate(config_data, changed_components, expected_result)
def test_component_changed_no_scope_keys(self):
config_data = {
'comp1': {},
'comp2': {},
'comp3': {},
'comp4': {},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': ComponentConfig.FullScope,
'comp3': ComponentConfig.FullScope,
'comp4': ComponentConfig.FullScope,
}
self.validate(config_data, changed_components, expected_result)
def test_one_component_changed_dependents_empty(self):
config_data = {
'comp1': {'build': {}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
}
self.validate(config_data, changed_components, expected_result)
def test_not_changed_dependent_component(self):
config_data = {
'comp1': {'build': {'comp2'}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': {'build'}
}
self.validate(config_data, changed_components, expected_result)
def test_changed_dependent_component(self):
config_data = {
'comp1': {'build': {'comp2'}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1', 'comp2'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': ComponentConfig.FullScope
}
self.validate(config_data, changed_components, expected_result)
def test_dependent_component_multiple_parents(self):
config_data = {
'comp1': {'_scope_1': {'comp2'}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, '_scope_2': {'comp2'}, '_scope_3': {'comp2'}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1', 'comp3'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': {'_scope_1', '_scope_2', '_scope_3'},
'comp3': ComponentConfig.FullScope
}
self.validate(config_data, changed_components, expected_result)
def test_dependent_component_empty_scopes(self):
config_data = {
'comp1': {'build': {}, 'revalidate': {'comp2'}},
'comp2': {},
'comp3': {},
'comp4': {},
}
changed_components = {'comp1', 'comp3'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': ComponentConfig.FullScope,
'comp3': ComponentConfig.FullScope,
'comp4': ComponentConfig.FullScope
}
self.validate(config_data, changed_components, expected_result)
def test_changed_component_empty_dependencies(self):
config_data = {
'comp1': {'build': {}, 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
}
self.validate(config_data, changed_components, expected_result)
def test_multiple_dependents(self):
config_data = {
'comp1': {'build': {'comp2'}, 'revalidate': {'comp3'}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {'comp4'}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': {'build'},
'comp3': ComponentConfig.FullScope,
# We don't consider dependencies of dependencies affected, so comp4 is not expected here
}
self.validate(config_data, changed_components, expected_result)
def test_all_as_dependents(self):
config_data = {
'comp1': {'build': 'all', 'revalidate': {}},
'comp2': {'build': {}, 'revalidate': {}},
'comp3': {'build': {}, 'revalidate': {}},
'comp4': {'build': {}, 'revalidate': {}},
}
changed_components = {'comp1'}
expected_result = {
'comp1': ComponentConfig.FullScope,
'comp2': {'build'},
'comp3': {'build'},
'comp4': {'build'},
}
self.validate(config_data, changed_components, expected_result)
if __name__ == '__main__':
unittest.main(verbosity=2)

View File

@ -1,34 +0,0 @@
name: 'System Information'
description: 'Information about the system'
runs:
using: "composite"
steps:
- if: runner.os == 'Linux'
shell: bash
run: |
# Install pre-requisites for Fedora
if [[ -e /etc/fedora-release ]]; then
yum update -y -q && yum install -y -q procps
fi
echo "System: ${{ runner.os }}"
echo "System Architecture: ${{ runner.arch }}"
echo "CPU Info: "; lscpu
echo "RAM Info: "; free -h --si
echo "MEMORY Info: "; df -h
- if: runner.os == 'macOS'
shell: bash
run: |
echo "System: ${{ runner.os }}"
echo "System Architecture: ${{ runner.arch }}"
echo "CPU and RAM Info: "; system_profiler SPHardwareDataType
echo "MEMORY Info: "; df -h
- if: runner.os == 'Windows'
shell: pwsh
run: |
echo "System: ${{ runner.os }}"
echo "System Architecture: ${{ runner.arch }}"
echo "CPU Info: "; Get-CimInstance ClassName Win32_Processor | Select-Object -Property Name, NumberOfCores, NumberOfLogicalProcessors
echo "RAM info: $(systeminfo | Select-String 'Total Physical Memory:')"

227
.github/components.yml vendored
View File

@ -1,227 +0,0 @@
Core:
revalidate: 'all'
transformations:
revalidate: 'all'
LP_transformations:
revalidate:
- CPU
- GPU
- PyTorch_FE
- TF_FE
- TFL_FE
- ONNX_FE
- PDPD_FE
preprocessing:
revalidate:
- inference
- C_API
- Python_API
inference:
revalidate: 'all'
CPU:
revalidate:
- C_API
- Python_API
- samples
- ONNX_RT
- PyTorch_FE
- TF_FE
- ONNX_FE
build:
- AUTO
- HETERO
- AUTO_BATCH
- TEMPLATE
- IR_FE
GPU:
build:
- AUTO
- HETERO
- AUTO_BATCH
- TEMPLATE
- IR_FE
- PROXY
NPU:
revalidate: []
build: []
HETERO:
revalidate:
- Python_API
build:
- TEMPLATE
- IR_FE
AUTO_BATCH:
revalidate:
- GPU
- C_API
- Python_API
build:
- TEMPLATE
- IR_FE
TEMPLATE:
revalidate:
- CPU
- GPU
- HETERO
- AUTO_BATCH
- AUTO
- C_API
- Python_API
- NVIDIA
build:
- IR_FE
AUTO:
revalidate:
- C_API
- Python_API
build:
- TEMPLATE
- IR_FE
PROXY:
revalidate:
- inference
- GPU
build: []
IR_FE:
revalidate:
- C_API
- Python_API
- samples
build:
- CPU
ONNX_FE:
revalidate:
- MO
- ONNX_RT
build:
- CPU
- Python_API
PDPD_FE:
revalidate:
- MO
build:
- CPU
- Python_API
TF_FE:
revalidate:
- MO
build:
- CPU
- Python_API
- TOKENIZERS
TFL_FE:
revalidate:
- MO
build:
- CPU
- Python_API
PyTorch_FE:
revalidate:
- MO
build:
- CPU
- Python_API
- TOKENIZERS
C_API:
build:
- CPU
- GPU
- HETERO
- AUTO_BATCH
- AUTO
- IR_FE
Python_API:
revalidate:
- samples
- MO
- tools
build:
- CPU
- HETERO
- AUTO_BATCH
- TEMPLATE
- AUTO
- IR_FE
- ONNX_FE
- PDPD_FE
- TF_FE
- TFL_FE
- PyTorch_FE
JS_API:
build:
- CPU
- IR_FE
samples:
build:
- CPU
- AUTO_BATCH
- AUTO
- IR_FE
- C_API
- Python_API
IE_Tests:
revalidate:
- CPU
- GPU
- HETERO
- AUTO_BATCH
- TEMPLATE
- AUTO
- NVIDIA
build:
- IR_FE
MO:
revalidate:
- PyTorch_FE
- TF_FE
build:
- Python_API
tools:
build:
- CPU
- Python_API
docs:
revalidate: []
build: []
licensing:
revalidate: []
build: []
NVIDIA:
revalidate: []
build: []
ONNX_RT:
revalidate: []
build: []
TOKENIZERS:
revalidate: []
build: []

148
.github/dependabot.yml vendored
View File

@ -2,142 +2,16 @@
version: 2
updates:
#
# Python product dependencies
#
# Python API, Frontends
# Enable version updates for Python API
- package-ecosystem: pip
directory: "/src/bindings/python/"
directory: "/src/bindings/python"
schedule:
interval: "daily"
time: "09:00"
timezone: "Poland"
open-pull-requests-limit: 3
assignees:
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "rkazants"
- "ceciliapeng2011"
- "meiyang-intel"
- "mbencer"
- "tomdol"
- "jane-intel"
versioning-strategy: increase-if-necessary
# Tests
- package-ecosystem: pip
directory: "/tests"
schedule:
interval: "daily"
time: "09:00"
timezone: "Poland"
open-pull-requests-limit: 3
assignees:
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "rkazants"
versioning-strategy: increase-if-necessary
# Model Optimizer, openvino_dev and Benchmark tool
- package-ecosystem: pip
directory: "/tools"
schedule:
interval: "daily"
time: "09:00"
timezone: "Asia/Dubai"
open-pull-requests-limit: 3
assignees:
- "rkazants"
- "andrei-kochin"
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "Wovchena"
allow:
- dependency-name: "*"
dependency-type: "production"
versioning-strategy: increase-if-necessary
#
# Python Samples
#
- package-ecosystem: pip
directory: "/samples/python/hello_reshape_ssd/"
schedule:
interval: "daily"
time: "09:00"
timezone: "Asia/Dubai"
open-pull-requests-limit: 3
assignees:
- "Wovchena"
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "rkazants"
versioning-strategy: increase-if-necessary
- package-ecosystem: pip
directory: "/samples/python/classification_sample_async/"
schedule:
interval: "daily"
time: "09:00"
timezone: "Asia/Dubai"
open-pull-requests-limit: 3
assignees:
- "Wovchena"
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "rkazants"
versioning-strategy: increase-if-necessary
- package-ecosystem: pip
directory: "/samples/python/benchmark/bert_benchmark/"
schedule:
interval: "daily"
time: "09:00"
timezone: "Asia/Dubai"
open-pull-requests-limit: 3
assignees:
- "Wovchena"
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "rkazants"
versioning-strategy: increase-if-necessary
- package-ecosystem: pip
directory: "/samples/python/hello_classification/"
schedule:
interval: "daily"
time: "09:00"
timezone: "Asia/Dubai"
open-pull-requests-limit: 3
assignees:
- "Wovchena"
- "jiwaszki"
- "p-wysocki"
- "akuporos"
- "rkazants"
versioning-strategy: increase-if-necessary
#
# Github actions - CI
#
# Github actions
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: "daily"
time: "09:00"
timezone: "Asia/Dubai"
assignees:
- "akashchi"
- "mryzhov"
- "ilya-lavrenov"
open-pull-requests-limit: 3
interval: weekly
day: sunday
time: "13:00"
open-pull-requests-limit: 0
reviewers:
- jiwaszki
- akuporos
labels:
- "category: dependencies"

View File

@ -1,22 +0,0 @@
fail-on-severity: 'low'
allow-licenses:
- 'BSD-2-Clause'
- 'BSD-3-Clause'
- 'BSD-2-Clause AND BSD-3-Clause'
- 'MIT'
- 'Apache-2.0'
- 'ISC'
- 'Apache-2.0 AND MIT'
- 'BlueOak-1.0.0'
- '0BSD'
- 'Python-2.0'
- 'LGPL-3.0'
fail-on-scopes:
- 'runtime'
- 'development'
- 'unknown'
license-check: true
vulnerability-check: true
allow-dependencies-licenses:
- 'pkg:pypi/PyGithub@2.2.0'
- 'pkg:pypi/psycopg2-binary'

View File

@ -1 +0,0 @@
pr-24878

View File

@ -1,73 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
# For Java API
default-jdk \
# Compiler
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-aarch64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,72 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,89 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/nvidia/cuda:11.8.0-runtime-ubuntu20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
wget \
git \
ca-certificates \
gpg-agent \
tzdata \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Install CUDA
RUN wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin && \
mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 && \
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub && \
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
RUN apt update && apt install -y \
libcudnn8=8.9.4.*-1+cuda11.8 \
libcudnn8-dev=8.9.4.*-1+cuda11.8 \
libcudnn8-samples=8.9.4.*-1+cuda11.8 \
cuda-runtime-11-8 \
cuda-11-8 \
libcutensor1=1.6.1.5-1 \
libcutensor-dev=1.6.1.5-1 \
cuda-drivers=520.61.05-1 && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,73 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
libtbb2 \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,72 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
libtbb2 \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
clang \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set clang as a default compiler
RUN update-alternatives --install /usr/bin/cc cc /usr/bin/clang 100 && \
update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++ 100
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,51 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Python
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
&& \
rm -rf /var/lib/apt/lists/*
# Install openvino dependencies
ADD scripts/install_dependencies/install_openvino_dependencies.sh /install_openvino_dependencies.sh
RUN chmod +x /install_openvino_dependencies.sh && \
/install_openvino_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
ENV PIP_INSTALL_PATH=/venv/lib/python3.11/site-packages

View File

@ -1,52 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Python
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
ENV PIP_INSTALL_PATH=/venv/lib/python3.11/site-packages

View File

@ -1,52 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Python
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
ENV PIP_INSTALL_PATH=/venv/lib/python3.11/site-packages

View File

@ -6,6 +6,7 @@
"openvino-ci",
"openvino-pushbot",
"workbench-ci-bot",
"openvino-pot-ci",
"sysicvvpux",
"ote-ci-bot"
],
@ -18,31 +19,26 @@
},
"DRY_RUN": false,
"TEAM_TO_LABEL": {
"openvino-ci-maintainers": "category: CI",
"openvino-maintainers": "category: inference",
"openvino-admins": "category: CI",
"openvino-maintainers": "category: IE common",
"openvino-docs-maintainers": "category: docs",
"openvino-ie-maintainers": "category: inference",
"openvino-ie-maintainers": "category: IE common",
"openvino-ie-cpu-maintainers": "category: CPU",
"openvino-ie-gna-maintainers": "category: GNA",
"openvino-ie-gpu-maintainers": "category: GPU",
"openvino-ie-lpt-maintainers": "category: LP transformations",
"openvino-ie-transformations-maintainers": "category: transformations",
"openvino-ie-auto-multi-maintainers": "category: AUTO",
"openvino-auto-batch-maintainers": "category: AUTO BATCH",
"openvino-hetero-maintainers": "category: HETERO",
"openvino-ie-python-api-maintainers": "category: Python API",
"openvino-ie-auto-multi-maintainers": "category: MULTI",
"openvino-ie-python-api-maintainers": "category: python api",
"openvino-ie-template-maintainers": "category: TEMPLATE",
"openvino-ir-frontend-maintainers": "category: IR FE",
"openvino-ie-paddle-maintainers": "category: PDPD FE",
"openvino-tf-frontend-maintainers": "category: TF FE",
"openvino-onnx-frontend-maintainers": "category: ONNX FE",
"openvino-ie-tests-maintainers": "category: IE Tests",
"openvino-ie-vpu-maintainers": "category: VPU",
"openvino-mo-maintainers": "category: MO",
"openvino-ngraph-maintainers": "category: Core",
"openvino-ngraph-maintainers": "category: nGraph",
"openvino-scripts-maintainers": "category: build",
"openvino-tests-maintainers": "category: IE Tests",
"openvino-tools-maintainers": "category: tools",
"openvino-pot-maintainers": "category: POT",
"openvino-configuration-mgmt": "category: dependency_changes",
"openvino-samples-maintainers": "category: samples",
"openvino-c-api-maintainers": "category: C API"
"openvino-samples-maintainers": "category: samples"
}
}

View File

@ -14,8 +14,8 @@ import json
from pathlib import Path
if sys.version_info[:2] < (3, 8):
raise Exception("Python version must be >= 3.8")
if sys.version_info[:2] < (3, 7):
raise Exception("Python version must be >= 3.7")
class ConfigException(Exception):

187
.github/labeler.yml vendored
View File

@ -1,187 +0,0 @@
'category: AUTO BATCH':
- 'src/plugins/auto_batch/**/*'
'category: AUTO':
- 'src/plugins/auto/**/*'
'category: build':
- 'cmake/**/*'
- '**/CMakeLists.txt'
- '**/*.cmake'
'category: C API':
- 'src/bindings/c/**/*'
'category: CI':
- '.github/**/*'
- '.ci/**/*'
- 'Jenkinsfile'
'github_actions':
- '.github/workflows/*'
'category: Core':
- 'src/core/**/*'
- 'src/common/itt/**/*'
- 'src/common/util/**/*'
- 'src/frontends/common/**/*'
- 'src/common/conditional_compilation/**/*'
'category: CPP API':
- 'src/inference/include/**/*'
- 'src/core/include/**/*'
- 'src/frontends/common/include/**/*'
- 'src/frontends/onnx/frontend/include/**/*'
- 'src/frontends/tensorflow/include/**/*'
- 'src/frontends/tensorflow_lite/include/**/*'
- 'src/frontends/pytorch/include/**/*'
- 'src/frontends/paddle/include/**/*'
'category: CPU':
- 'src/plugins/intel_cpu/**/*'
- 'src/common/snippets/**/*'
- 'thirdparty/xbyak/**/*'
'category: dependency_changes':
- '**/requirement*.txt'
- '**/constraints*.txt'
- 'scripts/**/*'
- '.gitmodules'
- '**/setup.py'
- 'conan.lock'
- 'conanfile.txt'
- 'vcpkg.json'
- any: ['thirdparty/**/*',
'!thirdparty/**/CMakeLists.txt']
'category: docs':
- '**/*.md'
- any: ['docs/**/*',
'!docs/snippets/**/*']
'category: docs_snippets':
- 'docs/snippets/**/*'
'category: extensions':
- 'src/core/include/openvino/core/extension.hpp'
- 'src/frontends/common/include/openvino/frontend/extension.hpp'
- 'src/frontends/common/include/openvino/frontend/extension/**/*'
'category: GPU':
- 'src/plugins/intel_gpu/**/*'
- 'thirdparty/ocl/**/*'
'category: NPU':
- 'src/plugins/intel_npu/**/*'
'category: HETERO':
- 'src/plugins/hetero/**/*'
'category: PROXY':
- 'src/plugins/proxy/**/*'
'category: IE Tests':
- 'thirdparty/gtest/**/*'
- 'src/frontends/tests/frontend/shared/**/*'
- 'src/tests/**/*'
'category: inference':
- 'src/inference/**/*'
- 'src/cmake/**/*'
'category: IR FE':
- 'src/frontends/ir/**/*'
'category: LP transformations':
- 'src/common/low_precision_transformations/**/*'
'category: MO':
- 'tools/mo/**/*'
- 'tools/ovc/**/*'
- 'tests/layer_tests/mo_python_api_tests/**/*'
- 'tests/layer_tests/ovc_python_api_tests/**/*'
'category: ONNX FE':
- 'src/frontends/onnx/**/*'
- 'thirdparty/onnx/**/*'
- 'tests/layer_tests/onnx_tests/**/*'
'category: packaging':
- 'cmake/**/packaging/**/*'
- 'src/bindings/python/wheel/**/*'
- any: ['src/bindings/js/node/CMakeLists.txt',
'src/bindings/js/node/package.json',
'src/bindings/js/node/package-lock.json']
- 'tools/openvino_dev/**/*'
'category: PDPD FE':
- 'src/frontends/paddle/**/*'
- 'tests/layer_tests/py_frontend_tests/test_paddle_frontend.py'
'category: preprocessing':
- 'src/common/preprocessing/**/*'
'category: Python API':
- 'src/bindings/python/**/*'
'category: JS API':
- 'src/bindings/js/**/*'
'category: samples':
- 'samples/**/*'
- 'thirdparty/zlib/**/*'
- 'thirdparty/gflags/**/*'
- 'thirdparty/json/**/*'
- 'thirdparty/cnpy/**/*'
- 'tests/samples_tests/smoke_tests/**/*'
'category: TEMPLATE':
- 'src/plugins/template/**/*'
'category: TF FE':
- 'src/frontends/tensorflow/**/*'
- 'src/frontends/tensorflow_common/**/*'
- 'tests/layer_tests/tensorflow_tests/**/*'
- 'tests/layer_tests/tensorflow2_keras_tests/**/*'
- 'tests/layer_tests/jax_tests/**/*'
- any: ['tests/model_hub_tests/**',
'!tests/model_hub_tests/pytorch/**/*']
'category: TFL FE':
- 'src/frontends/tensorflow_lite/**/*'
- 'src/frontends/tensorflow_common/**/*'
- 'tests/layer_tests/tensorflow_lite_tests/**/*'
'category: PyTorch FE':
- 'src/frontends/pytorch/**/*'
- 'tests/layer_tests/pytorch_tests/**/*'
- 'src/bindings/python/src/openvino/frontend/pytorch/**/*'
- 'tests/layer_tests/py_frontend_tests/test_torch_decoder.py'
- 'tests/layer_tests/py_frontend_tests/test_torch_frontend.py'
- any: ['tests/model_hub_tests/**',
'!tests/model_hub_tests/tensorflow/**/*']
'category: tools':
- any: ['tools/**',
'!tools/mo/**/*',
'!tools/ovc/**/*']
'category: transformations':
- 'src/common/transformations/**/*'
- 'src/common/offline_transformations/**/*'
- 'src/core/pattern/**/*'
- 'src/core/pass/**/*'
- 'src/core/include/openvino/pass/**/*'
'category: licensing':
- 'licensing/**/*'
- 'LICENSE'
'category: docker_env':
- '.dockerignore'
- 'install_build_dependencies.sh'
- '**/install_openvino_dependencies.sh'
- '.github/actions/handle_docker/**/*'
'category: dockerfiles':
- '.github/dockerfiles/**/*'

View File

@ -1,225 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#!/usr/bin/env python3
from github import Github
from psycopg2 import sql
import os
import logging
import psycopg2
import dateutil
import argparse
def init_logger():
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
logging.basicConfig(level=LOGLEVEL,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
def make_parser():
parser = argparse.ArgumentParser()
parser.add_argument('-r', '--repository-name', type=str, required=True,
help='Repository name in OWNER/REPOSITORY format')
parser.add_argument('--run-id', type=str, required=True,
help='Workflow Run ID')
return parser
def create_db_tables(conn, cur):
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_runs(
id SERIAL PRIMARY KEY,
run_id BIGINT,
html_url TEXT,
name VARCHAR(255),
run_started_at TIMESTAMP,
created_at TIMESTAMP,
updated_at TIMESTAMP,
triggering_actor_login VARCHAR(255),
conclusion VARCHAR(25),
event VARCHAR(50),
run_attempt INT,
repository_full_name VARCHAR(255),
head_repository_full_name VARCHAR(255),
head_branch VARCHAR(255),
status VARCHAR(25),
display_title TEXT,
path TEXT,
total_duration_seconds INT
);
''')
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_jobs(
id SERIAL PRIMARY KEY,
job_id BIGINT,
parent_run_id BIGINT,
html_url TEXT,
name VARCHAR(255),
created_at TIMESTAMP,
started_at TIMESTAMP,
completed_at TIMESTAMP,
queued_duration_seconds INT,
duration_seconds INT,
runner_name VARCHAR(255),
status VARCHAR(25),
conclusion VARCHAR(25),
head_branch VARCHAR(255),
run_attempt INT,
workflow_name TEXT
);
''')
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_steps(
id SERIAL PRIMARY KEY,
parent_job_id BIGINT,
name VARCHAR(255),
conclusion VARCHAR(25),
number INT,
started_at TIMESTAMP,
completed_at TIMESTAMP,
duration_seconds INT
);
''')
conn.commit()
def main():
init_logger()
parser = make_parser()
args = parser.parse_args()
logger = logging.getLogger(__name__)
github_token = os.environ.get('GITHUB_TOKEN')
if not github_token:
raise ValueError('GITHUB_TOKEN environment variable is not set!')
run_id = args.run_id
repo_name = args.repository_name
# this should be specified in runner's env
db_username = os.environ.get('PGUSER')
db_password = os.environ.get('PGPASSWORD')
db_host = os.environ.get('PGHOST')
db_database = os.environ.get('PGDATABASE')
db_port = os.environ.get('PGPORT')
conn = psycopg2.connect(host=db_host,
port=db_port,
user=db_username,
password=db_password,
database=db_database)
# Create tables
cur = conn.cursor()
create_db_tables(conn, cur)
# Get the data
g = Github(github_token)
repo = g.get_repo(repo_name)
run = repo.get_workflow_run(int(run_id))
logger.info('Processing run ID %s - %s, URL: %s', run_id, run.name, run.html_url)
if run.status != 'completed':
logger.error('Run %s is not completed! Only completed runs should be in the database', run_id)
raise SystemExit(1)
# We rely on the following assumptions:
# - The workflow run is completed. When run.status != 'completed' we should not add it to the database
# theoretically the second attempt can be triggerred right after the completion of the first one
# or while the runner which executes this script is deploying
#
# - Job's queued duration equals "job.started_at - job.created_at" if started_at > created_at.
# Otherwise the job should not be added to the database
total_duration_seconds = round(run.timing().run_duration_ms / 1000)
workflow_data_query = sql.SQL('''INSERT INTO workflow_runs(
run_id, html_url, name,
run_started_at, created_at, updated_at, triggering_actor_login, conclusion,
event, run_attempt, repository_full_name, head_repository_full_name,
head_branch, status, display_title, path, total_duration_seconds)
VALUES(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);
''')
workflow_query_args = (run_id, run.html_url, run.name,
run.run_started_at, run.created_at, run.updated_at,
run.raw_data['triggering_actor']['login'],
run.conclusion, run.event,
run.run_attempt, run.raw_data['repository']['full_name'],
run.raw_data['head_repository']['full_name'],
run.head_branch, run.status,
run.display_title, run.path, total_duration_seconds)
logger.debug('Workflow run query: %s', cur.mogrify(workflow_data_query, workflow_query_args))
cur.execute(workflow_data_query, workflow_query_args)
for job in run.jobs():
job_id = job.id
logger.info('Processing job %s, URL: %s', job.name, job.html_url)
queued_duration_seconds = 0
duration_seconds = 0
job_created_at_date = dateutil.parser.parse(job.raw_data['created_at'])
if job_created_at_date > job.started_at:
logger.warning('Skipping job %s of run %s - most likely a stub \
job created after workflow restart', job.name, run_id)
continue
queued_duration_timedelta = job.started_at - job_created_at_date
queued_duration_seconds = round(queued_duration_timedelta.total_seconds())
duration_timedelta = job.completed_at - job.started_at
duration_seconds = round(duration_timedelta.total_seconds())
job_data_query = sql.SQL('''
INSERT INTO workflow_jobs(
job_id, parent_run_id, html_url, name,
created_at, started_at, completed_at,
queued_duration_seconds, duration_seconds,
runner_name, status, conclusion, head_branch,
run_attempt, workflow_name
)
VALUES(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);
''')
job_query_args = (job_id, run_id, job.html_url, job.name,
job.raw_data['created_at'], job.started_at, job.completed_at,
queued_duration_seconds, duration_seconds,
job.raw_data['runner_name'], job.status, job.conclusion,
job.raw_data['head_branch'], job.raw_data['run_attempt'], job.raw_data['workflow_name'])
logger.debug('Job query: %s', cur.mogrify(job_data_query, job_query_args))
cur.execute(job_data_query, job_query_args)
conn.commit()
for step in job.steps:
logger.info('Processing step %s', step.name)
# It seems like a GitHub bug, but
# sometimes there're steps "In progress"
# despite the workflow being completed
step_duration_seconds = 0
if step.status != 'completed':
logger.warning('Step %s isn\'t completed. Setting duration to zero.', step.name)
step_duration_seconds = 0
else:
step_duration_seconds_timedelta = step.completed_at - step.started_at
step_duration_seconds = round(step_duration_seconds_timedelta.total_seconds())
step_data_query = sql.SQL('''
INSERT INTO workflow_steps(
parent_job_id, name, conclusion,
number, started_at, completed_at,
duration_seconds)
VALUES (%s, %s, %s, %s, %s, %s, %s);
''')
step_query_args = (
job_id, step.name, step.conclusion,
step.number, step.started_at, step.completed_at,
step_duration_seconds)
logger.debug('Step query: %s', cur.mogrify(step_data_query, step_query_args))
cur.execute(step_data_query, step_query_args)
conn.commit()
cur.close()
conn.close()
g.close()
if __name__ == "__main__":
main()

View File

@ -1,75 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
import logging
import os
from enum import Enum
from github import Github, Auth
class ExternalPRLabels(str, Enum):
ExternalPR = 'ExternalPR'
ExternalIntelPR = 'ExternalIntelPR'
def get_arguments() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument(
'-r',
'--repository-name',
type=str,
required=True,
help='Repository name in the OWNER/REPOSITORY format',
)
parser.add_argument(
'--pr-number', type=int, required=True, help='PR number to label'
)
return parser.parse_args()
def init_logger():
logging.basicConfig(
level=os.environ.get('LOGLEVEL', 'INFO').upper(),
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S',
)
if __name__ == '__main__':
init_logger()
LOGGER = logging.getLogger('labeller')
args = get_arguments()
pr_number = args.pr_number
repository_name = args.repository_name
github = Github(auth=Auth.Token(token=os.environ.get('GITHUB_TOKEN')))
gh_repo = github.get_repo(full_name_or_id=repository_name)
pr = gh_repo.get_pull(number=pr_number)
LOGGER.info(f'CONTEXT: PR #{pr_number}. USER: {pr.user.login}. ALL PR LABELS: {list(pr.get_labels())}')
if not gh_repo.has_in_collaborators(pr.user.login):
LOGGER.info(f'THE {pr.user.login} IS NOT A COLLABORATOR')
for label in pr.get_labels():
if label.name in (ExternalPRLabels.ExternalPR, ExternalPRLabels.ExternalIntelPR):
LOGGER.info(f'THE PR ALREADY HAS THE "{label.name}" LABEL')
break
else:
is_intel_user = bool(pr.user.email and pr.user.email.endswith('@intel.com'))
label_to_add: str = ExternalPRLabels.ExternalIntelPR.name if is_intel_user else ExternalPRLabels.ExternalPR.name
pr.add_to_labels(label_to_add)
LOGGER.info(f'THE "{label_to_add}" LABEL WAS ADDED TO THE PR')
else:
LOGGER.info(
f'THE {pr.user.login} IS A COLLABORATOR, NO NEED TO ADD THE "External" LABEL'
)
github.close()

View File

@ -1 +0,0 @@
python-dateutil==2.9.0.post0

View File

@ -1,28 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
from pathlib import Path
def get_arguments() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument('-r', '--repository-name',
type=str,
required=True,
help='Repository name in the OWNER/REPOSITORY format')
parser.add_argument('--run-id',
type=int,
required=True,
help='Workflow Run ID')
parser.add_argument('--errors-to-look-for-file',
type=Path,
required=False,
help='.json file with the errors to look for in logs',
default=Path(__file__).resolve().parent.joinpath('errors_to_look_for.json'))
parser.add_argument('--dry-run',
required=False,
action='store_true',
help='Whether to run in dry mode and not actually retrigger the pipeline'
' and only collect and analyze logs')
return parser.parse_args()

View File

@ -1,20 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import os
GITHUB_TOKEN = os.environ.get('GITHUB_TOKEN')
def init_logger():
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
logging.basicConfig(level=LOGLEVEL,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
init_logger()
LOGGER = logging.getLogger('rerunner')

View File

@ -1,62 +0,0 @@
[
{
"error_text": "This is a problem related to network connectivity",
"ticket": 135929
},
{
"error_text": "Unable to make request",
"ticket": 135715
},
{
"error_text": "GnuTLS recv error",
"ticket": 131918
},
{
"error_text": "Connection was reset",
"ticket": 131818
},
{
"error_text": "Failed to connect to github.com",
"ticket": 131657
},
{
"error_text": "Could not resolve host: github.com",
"ticket": 131546
},
{
"error_text": "retrieving gpg key timed out",
"ticket": 131538
},
{
"error_text": "Retry limit has been reached for chunk",
"ticket": 131537
},
{
"error_text": "fatal error: downloading",
"ticket": 131424
},
{
"error_text": "Failure when receiving data from the peer",
"ticket": 137121
},
{
"error_text": "Network is unreachable",
"ticket": 130955
},
{
"error_text": "connection timed out",
"ticket": 130955
},
{
"error_text": "The requested URL returned error: 500",
"ticket": 139384
},
{
"error_text": "Unable to fetch some archives",
"ticket": 130965
},
{
"error_text": "status_string: \"Timeout was reached\"",
"ticket": 142653
}
]

View File

@ -1,137 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import json
import re
import tempfile
from pathlib import Path
from typing import TypedDict
from zipfile import ZipFile
from workflow_rerun.constants import LOGGER
class LogFile(TypedDict):
file_name: str
path: Path
class ErrorData(TypedDict):
error_text: str
ticket: int
class LogAnalyzer:
def __init__(self,
path_to_log_archive: Path,
path_to_errors_file: Path) -> None:
self._path_to_log_archive = path_to_log_archive
self._path_to_errors_file = path_to_errors_file
self._errors_to_look_for: list[ErrorData] = []
self._collect_errors_to_look_for()
self._log_dir = tempfile.TemporaryDirectory().name
self._log_files: list[LogFile] = []
self._collect_log_files()
all_txt_log_files_pretty = '\n'.join(map(lambda item: str(item['path']), self._log_files))
LOGGER.info(f'ALL .txt LOG FILES: \n{all_txt_log_files_pretty}')
self.found_matching_error = False
self.found_error_ticket = None
def _collect_errors_to_look_for(self) -> None:
with open(file=self._path_to_errors_file,
mode='r',
encoding='utf-8') as errors_file:
errors_data = json.load(errors_file)
for error_data in errors_data:
self._errors_to_look_for.append(
ErrorData(error_text=error_data['error_text'],
ticket=error_data['ticket'])
)
def _collect_log_files(self) -> None:
"""
Collects the .txt log files from the log archive
The GitHub Actions pipeline logs archive should have the following structure:
> Job_name_0
> step_name_0.txt
> step_name_1.txt
...
> Job_name_1
> step_name_0.txt
> step_name_1.txt
...
> Job_name_2
...
...
We need to only analyze the `*.txt` files
"""
with ZipFile(file=self._path_to_log_archive,
mode='r') as zip_file:
zip_file.extractall(self._log_dir)
for _file in Path(self._log_dir).iterdir():
if _file.is_dir():
for log_file in _file.iterdir():
self._log_files.append(LogFile(file_name=log_file.name,
path=log_file.resolve()))
def _is_error_in_log(self,
error_to_look_for: str,
log_file_path: Path) -> bool:
"""
Searches for the error in the provided log
"""
error_to_look_for = self._clean_up_string(error_to_look_for)
with open(file=log_file_path,
mode='r',
encoding='utf-8') as log_file:
for line in log_file:
if error_to_look_for in self._clean_up_string(line):
return True
return False
@staticmethod
def _clean_up_string(string: str) -> str:
"""
Replaces special characters with spaces in the string, strips it from leading and following spaces,
and lowers it
for "Could not resolve host: github.com" returns "could not resolve host github com"
This cleanup is applied to both errors to look for and logs themselves for matching
"""
return re.sub(r'[^A-Za-z0-9]+', ' ', string).lower().strip()
def analyze(self) -> None:
"""
Iterates over the known errors and tries to find them in the collected log files
"""
for error in self._errors_to_look_for:
LOGGER.info(f'LOOKING FOR "{error["error_text"]}" ERROR...')
for log_file in self._log_files:
if self._is_error_in_log(error_to_look_for=error['error_text'],
log_file_path=log_file['path']):
LOGGER.info(f'FOUND "{error["error_text"]}" ERROR IN {log_file["path"]}. TICKET: {error["ticket"]}')
self.found_matching_error = True
self.found_error_ticket = error['ticket']
return
if __name__ == '__main__':
# Usage example
log_analyzer = LogAnalyzer(path_to_log_archive=Path('/tmp/logs/log.zip'),
path_to_errors_file=Path('/tmp/errors_to_look_for.json'))
log_analyzer.analyze()
if log_analyzer.found_matching_error:
print('found matching error, see logs above')

View File

@ -1,24 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
from pathlib import Path
import requests
from github.WorkflowRun import WorkflowRun
from workflow_rerun.constants import GITHUB_TOKEN, LOGGER
def collect_logs_for_run(run: WorkflowRun,
log_archive_path: Path) -> Path:
"""
Collects log archive for a pipeline
"""
with open(file=log_archive_path,
mode='wb') as log_archive:
LOGGER.info(f'STARTED LOG COLLECTION FOR {run.id} IN {log_archive_path}')
# PyGitHub does not expose the "/repos/{owner}/{repo}/actions/runs/{run_id}/logs" endpoint so we have to use requests
log_archive.write(requests.get(url=run.logs_url,
headers={'Authorization': f'Bearer {GITHUB_TOKEN}'}).content)
LOGGER.info(f'COLLECTED LOGS FOR {run.id} IN {log_archive_path}')
return log_archive_path

View File

@ -1,71 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import os
import sys
import tempfile
from pathlib import Path
from github import Github, Auth
from workflow_rerun.argument_parser import get_arguments
from workflow_rerun.constants import GITHUB_TOKEN, LOGGER
from workflow_rerun.log_analyzer import LogAnalyzer
from workflow_rerun.log_collector import collect_logs_for_run
if __name__ == '__main__':
args = get_arguments()
run_id = args.run_id
repository_name = args.repository_name
errors_file = args.errors_to_look_for_file
is_dry_run = args.dry_run
if is_dry_run:
LOGGER.info('RUNNING IN DRY RUN MODE. IF ERROR WILL BE FOUND, WILL NOT RETRIGGER')
github = Github(auth=Auth.Token(token=GITHUB_TOKEN))
gh_repo = github.get_repo(full_name_or_id=repository_name)
run = gh_repo.get_workflow_run(id_=run_id)
LOGGER.info(f'CHECKING IF RERUN IS NEEDED FOR {run.html_url} RUN IN {repository_name}.')
# Check if the run has already been retriggered
# we do not want to fall into a loop with retriggers
if run.run_attempt > 1:
LOGGER.info(f'THERE ARE {run.run_attempt} ATTEMPTS ALREADY. NOT CHECKING LOGS AND NOT RETRIGGERING. EXITING')
sys.exit(0)
log_archive_path = Path(tempfile.NamedTemporaryFile(suffix='.zip').name)
collect_logs_for_run(
run=run,
log_archive_path=log_archive_path,
)
log_analyzer = LogAnalyzer(
path_to_log_archive=log_archive_path,
path_to_errors_file=errors_file
)
log_analyzer.analyze()
if log_analyzer.found_matching_error:
LOGGER.info(f'FOUND MATCHING ERROR, RETRIGGERING {run.html_url}')
if is_dry_run:
LOGGER.info(f'RUNNING IN DRY RUN MODE, NOT RETRIGGERING, EXITING')
sys.exit(0)
status = run.rerun()
if status:
LOGGER.info(f'RUN RETRIGGERED SUCCESSFULLY: {run.html_url}')
else:
LOGGER.info(f'RUN WAS NOT RETRIGGERED, SEE ABOVE')
# Needed to run a step after for statistics
with open(file=os.environ['GITHUB_ENV'],
mode='a') as fh:
fh.write('PIPELINE_RETRIGGERED=true\n')
fh.write(f'FOUND_ERROR_TICKET={log_analyzer.found_error_ticket}\n')
# "status" is True (which is 1) if everything is ok, False (which is 0) otherwise
sys.exit(not status)
else:
LOGGER.info(f'NO ERROR WAS FOUND, NOT RETRIGGERING')

View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:894d636bcf156a7f3fae09f3c1d61df6b3db89117a917a3079995805c29115b3
size 89247

View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3f094a737d7ea40dba8d3fb13493275cae243d08e5f1dabce90c316c951a6ac2
size 52047

View File

@ -1,55 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Integration tests
"""
import unittest
from pathlib import Path
from github import Github, Auth
import os
import tempfile
from workflow_rerun.log_analyzer import LogAnalyzer
from workflow_rerun.log_collector import collect_logs_for_run
class IntegrationTest(unittest.TestCase):
"""
A class for testing integration between LogAnalyzer and log_collection
"""
def setUp(self) -> None:
print(f'\nIn test: "{self._testMethodName}"', flush=True)
self._cwd = Path(__file__).parent
self.errors_to_look_for_file = self._cwd.parent.joinpath(
'errors_to_look_for.json'
)
self.github = Github(auth=Auth.Token(token=os.environ.get('GITHUB_TOKEN')))
self.gh_repo = self.github.get_repo(full_name_or_id='openvinotoolkit/openvino')
# Even if we use "failure" for status we cannot guarantee logs containing any of the known error
# So these tests use the logs of the most recent successfull pipeline
self.wf_run = self.gh_repo.get_workflow_runs(status='success')[0]
print(f'Workflow run for testing: {self.wf_run}', flush=True)
def test_log_collection_and_analysis(self) -> None:
"""
Ensure logs collected by collect_logs_for_run are analyzed by LogAnalyzer
"""
log_archive_path = Path(tempfile.NamedTemporaryFile(suffix='.zip').name)
collect_logs_for_run(run=self.wf_run,
log_archive_path=log_archive_path)
analyzer = LogAnalyzer(
path_to_log_archive=log_archive_path,
path_to_errors_file=self.errors_to_look_for_file,
)
analyzer.analyze()
self.assertFalse(analyzer.found_matching_error)
def tearDown(self) -> None:
self.github.close()

View File

@ -1,104 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
LogAnalyzer tests
"""
import unittest
from pathlib import Path
from workflow_rerun.log_analyzer import LogAnalyzer
class LogAnalyzerTest(unittest.TestCase):
"""
A class for testing LogAnalyzer
"""
def setUp(self) -> None:
print(f'\nIn test: "{self._testMethodName}"', flush=True)
self._cwd = Path(__file__).parent
self.log_archive_with_error = self._cwd.joinpath("data").joinpath(
'log_archive_with_error.zip'
)
self.log_archive_wo_error = self._cwd.joinpath("data").joinpath(
'log_archive_wo_error.zip'
)
self.errors_to_look_for_file = self._cwd.parent.joinpath(
'errors_to_look_for.json'
)
def test_log_analyzer_instantiation(self) -> None:
"""
Ensure LogAnalyzer is instantiated correctly.
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_wo_error,
path_to_errors_file=self.errors_to_look_for_file,
)
self.assertTrue(
hasattr(analyzer, '_errors_to_look_for'),
'Analyzer should have _errors_to_look_for',
)
self.assertTrue(
hasattr(analyzer, '_log_files'), 'Analyzer should have _log_files'
)
for error_data in analyzer._errors_to_look_for:
self.assertTrue(
error_data['error_text'], 'Each error_data should have text'
)
self.assertTrue(error_data['ticket'], 'Each error_data should have ticket')
for log_file in analyzer._log_files:
self.assertTrue(
log_file['file_name'], 'Each log_file should have file_name'
)
self.assertTrue(log_file['path'], 'Each log_file should have path')
def test_string_cleanup(self) -> None:
"""
Ensure log cleanup function returns correct results
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_wo_error,
path_to_errors_file=self.errors_to_look_for_file,
)
data = (
'Connection was reset',
'Failed to connect to github.com',
'Could not resolve host: github.com',
)
expected = (
'connection was reset',
'failed to connect to github com',
'could not resolve host github com',
)
for input_str, expected_str in zip(data, expected):
self.assertEqual(analyzer._clean_up_string(string=input_str), expected_str)
def test_analyzer_with_error(self) -> None:
"""
Ensure LogAnalyzer can find an error
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_with_error,
path_to_errors_file=self.errors_to_look_for_file,
)
analyzer.analyze()
self.assertTrue(analyzer.found_matching_error)
def test_analyzer_wo_error(self) -> None:
"""
Ensure LogAnalyzer does not find an error in the log files w/o errors
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_wo_error,
path_to_errors_file=self.errors_to_look_for_file,
)
analyzer.analyze()
self.assertFalse(analyzer.found_matching_error)

View File

@ -1,41 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
log collector tests
"""
import os
import unittest
import tempfile
from pathlib import Path
from github import Github, Auth
from workflow_rerun.log_collector import collect_logs_for_run
class LogCollectorTest(unittest.TestCase):
"""
A class for testing log collection
"""
def setUp(self) -> None:
print(f'\nIn test: "{self._testMethodName}"', flush=True)
self._cwd = Path(__file__).parent
self.github = Github(auth=Auth.Token(token=os.environ.get('GITHUB_TOKEN')))
self.gh_repo = self.github.get_repo(full_name_or_id='openvinotoolkit/openvino')
# Use the logs of the most recent successfull pipeline
self.wf_run = self.gh_repo.get_workflow_runs(status='success')[0]
print(f'Workflow run for testing: {self.wf_run}', flush=True)
def test_log_collection(self) -> None:
"""
Ensure log collection is working
"""
log_archive_path = Path(tempfile.NamedTemporaryFile(suffix='.zip').name)
collect_logs_for_run(run=self.wf_run, log_archive_path=log_archive_path)
self.assertTrue(Path(log_archive_path).exists())
def tearDown(self) -> None:
self.github.close()

View File

@ -1,206 +0,0 @@
name: Android ARM64 with vcpkg
on:
workflow_dispatch:
pull_request:
merge_group:
push:
branches:
# - master
- 'releases/**'
concurrency:
# github.ref is not unique in post-commit
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-android-arm64-vcpkg
cancel-in-progress: true
permissions: read-all
jobs:
Smart_CI:
runs-on: ubuntu-latest
outputs:
affected_components: "${{ steps.smart_ci.outputs.affected_components }}"
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
sparse-checkout: .github/actions/smart-ci
- name: Get affected components
id: smart_ci
uses: ./.github/actions/smart-ci
with:
repository: ${{ github.repository }}
pr: ${{ github.event.number }}
commit_sha: ${{ github.sha }}
ref_name: ${{ github.ref_name }}
component_pattern: "category: (.*)"
repo_token: ${{ secrets.GITHUB_TOKEN }}
skip_when_only_listed_labels_set: 'docs'
skip_when_only_listed_files_changed: '*.md,*.rst,*.png,*.jpg,*.svg,*/layer_tests_summary/*,*/conformance/*'
Build:
needs: Smart_CI
timeout-minutes: 150
defaults:
run:
shell: bash
runs-on: aks-linux-16-cores-32gb
container:
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount:/mount
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_GENERATOR: 'Ninja'
CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
SCCACHE_ERROR_LOG: /__w/openvino/sccache_log.txt
SCCACHE_LOG: warn
OPENVINO_REPO: '/__w/openvino/openvino/openvino'
VCPKG_ROOT: '/__w/openvino/openvino/vcpkg'
BUILD_DIR: '/__w/openvino/openvino/build'
ANDROID_TOOLS: '/__w/openvino/openvino/android_tools'
ANDROID_NDK_HOME: '/__w/openvino/openvino/android_tools/ndk-bundle'
ANDROID_SDK_VERSION: 29
ANDROID_ABI_CONFIG: arm64-v8a
VCPKG_DEFAULT_BINARY_CACHE: '/mount/caches/ccache/android_arm64/vcpkg_cache'
VCPKG_FORCE_SYSTEM_BINARIES: '1'
SCCACHE_AZURE_KEY_PREFIX: android_arm64
if: "!needs.smart_ci.outputs.skip_workflow"
steps:
- name: Install git
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
path: 'openvino'
- name: Init submodules for non vcpkg dependencies
run: |
pushd ${OPENVINO_REPO}
git submodule update --init -- ${OPENVINO_REPO}/src/plugins
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/zlib
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags
popd
- name: Clone vcpkg
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
repository: 'microsoft/vcpkg'
# Keep in sync with <root>/vcpkg.json <builtin-baseline>
ref: '7ba0ba7334c3346e7eee1e049ba85da193a8d821'
path: 'vcpkg'
fetch-depth: '0'
#
# Print system info
#
- name: System info
uses: ./openvino/.github/actions/system_info
#
# Dependencies
#
- name: Install dependencies
run: |
# generic dependencies
apt --assume-yes install ccache scons ninja-build build-essential python3-pip
# vcpkg requires cmake 3.19 or later
python3 -m pip install -U pip cmake~=3.28.0
# vcpkg's tool dependencies
apt --assume-yes install curl zip unzip tar
# vcpkg 'python3' port dependencies
apt --assume-yes install autoconf libtool autoconf-archive
# vcpkg tree of dependencies require extra packages
apt --assume-yes install pkgconf linux-libc-dev
# Install Android SDK, NDK and Tools
apt -y --no-install-recommends install unzip wget default-jdk
wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip
unzip commandlinetools-linux-7583922_latest.zip
echo "yes" | ./cmdline-tools/bin/sdkmanager --sdk_root=${ANDROID_TOOLS} --install "ndk-bundle" "platform-tools" "platforms;android-${{ env.ANDROID_SDK_VERSION }}"
- name: Install sccache
uses: mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd # v0.0.4
with:
version: "v0.7.5"
#
# Build
#
- name: Build vcpkg
run: |
mkdir -p ${VCPKG_DEFAULT_BINARY_CACHE}
${VCPKG_ROOT}/bootstrap-vcpkg.sh
# patch vcpkg default toolchain to build only Release configuration
echo "set(VCPKG_BUILD_TYPE release)" >> ${VCPKG_ROOT}/triplets/arm64-android.cmake
- name: CMake - configure
run: |
cmake \
-G '${{ env.CMAKE_GENERATOR }}' \
-DENABLE_INTEL_GPU=ON \
-DENABLE_TESTS=ON \
-DENABLE_SYSTEM_OPENCL=ON \
-DENABLE_SYSTEM_PROTOBUF=ON \
-DENABLE_SYSTEM_PUGIXML=ON \
-DENABLE_SYSTEM_SNAPPY=ON \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_SYSTEM_FLATBUFFERS=ON \
-DANDROID_ABI=${{ env.ANDROID_ABI_CONFIG }} \
-DANDROID_PLATFORM=${{ env.ANDROID_SDK_VERSION }} \
-DVCPKG_TARGET_TRIPLET=arm64-android \
-DVCPKG_HOST_TRIPLET=x64-linux-release \
-DCMAKE_TOOLCHAIN_FILE=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake \
-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=${ANDROID_NDK_HOME}/build/cmake/android.toolchain.cmake \
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON \
-DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
-DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
- name: Clean ccache stats
run: ${SCCACHE_PATH} --zero-stats
- name: Cmake - build
run: cmake --build ${BUILD_DIR} --parallel
- name: Show ccache stats
run: ${SCCACHE_PATH} --show-stats
#
# Upload build logs
#
- name: Upload build logs
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
if: always()
with:
name: build_logs
path: ${{ env.SCCACHE_ERROR_LOG }}
if-no-files-found: 'ignore'
Overall_Status:
name: ci/gha_overall_status_android
needs: [Smart_CI, Build]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Check status of all jobs
if: >-
${{
contains(needs.*.result, 'failure') ||
contains(needs.*.result, 'cancelled')
}}
run: exit 1

View File

@ -1,25 +0,0 @@
name: Take Issue
on:
issue_comment:
types:
- created
- edited
permissions: read-all
jobs:
take-issue:
name: Take issue
runs-on: ubuntu-latest
permissions:
issues: write
timeout-minutes: 10
steps:
- name: take an issue
uses: bdougie/take-action@1439165ac45a7461c2d89a59952cd7d941964b87 # v1.6.1
with:
message: Thank you for looking into this issue! Please let us know if you have any questions or require any help.
issueCurrentlyAssignedMessage: Thanks for being interested in this issue. It looks like this ticket is already assigned to a contributor. Please communicate with the assigned contributor to confirm the status of the issue.
trigger: .take
token: ${{ secrets.GITHUB_TOKEN }}

Some files were not shown because too many files have changed in this diff Show More