Compare commits

..

97 Commits

Author SHA1 Message Date
Maciej Smyk 5d852eb14c
[DOCS] release policy adjustment for 2024.0 (#24181)
Port from https://github.com/openvinotoolkit/openvino/pull/24145
2024-04-24 07:42:43 +00:00
Maciej Smyk 14549e85e4
[DOCS] Fixing Tensor Names examples in Model Representation for 2024.0 (#24172)
Port from https://github.com/openvinotoolkit/openvino/pull/23482

* Fixed examples of Tensor Names,
* Added an example of ov::element::Type for both Python and C++
2024-04-22 15:09:03 +00:00
Karol Blaszczak 914fb470cc
[DOCS] language in Quantization article 24.0 (#24077)
port: https://github.com/openvinotoolkit/openvino/pull/24075
2024-04-18 13:16:51 +00:00
Karol Blaszczak a8e245dbec
DOCS-removeGNA-link-from-dev-doc (#24062) 2024-04-18 13:16:31 +00:00
Maciej Smyk 103afb13a0
[DOCS] Release Policy + torch.compile updates for 2024.0 (#24051)
Port from https://github.com/openvinotoolkit/openvino/pull/22881

* Created new article about Release Policy,
* Moved System Requirements Article under Release Notes.
* Removed environment variables info from torch.compile article.

Jira: 132590
2024-04-16 12:55:04 +00:00
Andrzej Kopytko a1e0c687f4
Docs carousel changing mode (#24000)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-04-12 11:29:17 +00:00
Andrzej Kopytko d7d4991225
Docs new home page (#23956)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*

---------

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2024-04-10 17:27:21 +00:00
Karol Blaszczak 64b8e59283
[DOCS] old fix for yolo article 24.0 (#23937) 2024-04-10 10:13:38 +00:00
Andrzej Kopytko 8d637d65e1
[Docs] Main changes with homesite (#23916)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2024-04-08 13:20:10 +00:00
Maciej Smyk ccb1272b55
[DOCS] Typo fix in Conversion Parameters article for 2024 (#23884)
Port from https://github.com/openvinotoolkit/openvino/pull/23864

Added missing comma in Python code in the Conversion Parameters article.

Jira: 137610
2024-04-05 14:01:02 +00:00
Tatiana Savina 12217807be
[DOCS] Port Add MS Windows version for NPU (#23871)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-04-04 13:14:12 +00:00
Alexander Suvorov 204a02a1cb
Revert "Update Selector Tool version 2022.3.2 (#23694)" (#23753)
This reverts commit 8d46c625fc.
2024-04-04 06:13:07 +00:00
Karol Blaszczak 1baef807b8
[DOCS] pdf reference guide (#23687)
port: https://github.com/openvinotoolkit/openvino/pull/23680
2024-03-27 12:52:00 +00:00
Andrzej Kopytko 44c2bcd5b2
DOCS Update benchmarks for 2024.0 (#23681)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-27 12:30:33 +00:00
Alexander Suvorov 8d46c625fc
Update Selector Tool version 2022.3.2 (#23694) 2024-03-27 10:46:55 +00:00
Sebastian Golebiewski 08bad1b9a7
[DOCS] Update code examples with imports and save_model for 24.0 (#23678)
Updating code examples with imports for convert_model and adding an
example of how to save a converted model. This PR addresses the 120254
ticket.
Porting: https://github.com/openvinotoolkit/openvino/pull/23654
2024-03-26 12:41:56 +00:00
Karol Blaszczak 55ac8f34c9
[DOCS] ai legal disclaimer (#23588)
port: https://github.com/openvinotoolkit/openvino/pull/23587
2024-03-22 14:02:27 +00:00
Karol Blaszczak 4937e9b4fd
[DOCS] improve legacy section formatting (#23514)
port: https://github.com/openvinotoolkit/openvino/pull/23512
2024-03-22 13:54:53 +00:00
Maciej Smyk 1756c93610
[DOCS] Latency highlight for OV devices + update of Optimize Inference for 2024 (#23604)
Port from https://github.com/openvinotoolkit/openvino/pull/23575

Jira: 133389

* Added an indication on Latency being the default use for OV devices
* Streamlined the Optimize Inference article for better clarity.
2024-03-21 11:55:13 +00:00
Andrzej Kopytko 03551b89e4
[DOCS] OVMS update for 2024 (#23517)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-18 13:33:27 +00:00
Andrzej Kopytko 8fe7dc3f2f
[DOCS] Updated file (#23510)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-18 09:09:31 +00:00
Sebastian Golebiewski a9d6d2139c
[DOCS] Updating interactive tutorials port to 2024.0 (#23478)
An update of the `Interactive Tutorials` section.

Porting: https://github.com/openvinotoolkit/openvino/pull/23466
2024-03-15 10:40:01 +00:00
Andrzej Kopytko 1be304b910
[DOCS] Added off on click for 2024.0 (#23464)
port for <html xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"
xmlns="http://www.w3.org/TR/REC-html40">

<head>

<meta name=ProgId content=OneNote.File>
<meta name=Generator content="Microsoft OneNote 15">
</head>

<body lang=en-US style='font-family:Calibri;font-size:11.0pt'>
<!--StartFragment-->

<div style='direction:ltr'>


https://github.com/openvinotoolkit/openvino/pull/23411
--




</div>

<!--EndFragment-->
</body>

</html>
2024-03-14 13:18:49 +00:00
Karol Blaszczak 01b0930645
[DOCS] sysreq fix 2024 (#23457) 2024-03-14 08:30:46 +00:00
Karol Blaszczak d64b4b4ade
[DOCS] postrelease adjustments (#23364) 2024-03-13 12:57:01 +00:00
Tatiana Savina 3084d7331b
[DOCS] PORT Fix llms article format (#23424)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-12 17:07:50 +00:00
Sebastian Golebiewski 8431fe5539
[DOCS] Fix face-detection-retail-0004 reference (#23421)
Fixing reference to an OMZ model.

Porting: https://github.com/openvinotoolkit/openvino/pull/23417
2024-03-12 15:51:58 +00:00
Andrzej Kopytko f7c7d7814e
[DOCS] Updated configs to remove canonical 2024 (#23407)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-12 14:00:52 +00:00
Sebastian Golebiewski b4882b6370
[DOCS] Fixing formatting and reference issues for 24.0 (#23333)
Fixing formatting and reference issues in docs.
2024-03-12 08:54:40 +00:00
Karol Blaszczak b7ef796963
[DOCS] port llm articles merged into old master pr1 (#23348)
porting and adjusting: 
* https://github.com/openvinotoolkit/openvino/pull/23092
* https://github.com/openvinotoolkit/openvino/pull/23341
* https://github.com/openvinotoolkit/openvino/pull/23346
2024-03-11 12:13:41 +00:00
Maciej Smyk 965756118b
[DOCS] Link fix for Automatic Device Selection for 2024 (#23347)
Fixed Async API docs link in Automatic Device Selection article
2024-03-08 12:31:55 +00:00
Vishniakov Nikolai 80b0130a38
[OV JS] Fix package.json files (#23304) (#23305)
Port of: https://github.com/openvinotoolkit/openvino/pull/23304

### Details:
 - Remove unnecessary run
 - Fix run unit tests command
 - Update min Node.js version in samples
2024-03-08 09:59:59 +00:00
Sebastian Golebiewski b97d1fcdbd
[DOCS] Fixing references to articles (#23300)
Fixing directives for linking and cross-reference.
2024-03-08 08:22:48 +00:00
Karol Blaszczak 4a6144e1d8
[DOCS] final release polishing (#23295) 2024-03-06 10:53:48 +00:00
Andrzej Kopytko da59b70811
DOCS temporary benchmark update for 2024.0 (#23294)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-06 10:42:54 +00:00
Sebastian Golebiewski f4488a8ca9
[DOCS] References fixes for 24.0 (#23291)
Minor fixes for references and urls
2024-03-06 10:10:59 +00:00
Karol Blaszczak 79da9a1f39
[DOCS] 2024 benchmark fix (#23280)
replacing the faulty csv file
2024-03-05 19:19:48 +00:00
Karol Blaszczak 4e1d3c8956
[DOCS] 2024 release notes (#23258) 2024-03-05 17:28:33 +00:00
Maciej Smyk 51769e45ed
[DOCS] Fixes for OpenVINO Workflow for 2024 (#23278)
* Fixed some lines in OpenVINO Workflow section
* Fixed a PaddlePaddle link
2024-03-05 13:43:24 +00:00
Sebastian Golebiewski da0497bd60
[DOCS] Update distribution comparison for 24.0 (#23277)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-05 13:42:05 +00:00
Andrzej Kopytko edbdd39806
Docs benchmarks update for 2024.0 (#23231)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
Signed-off-by: Andrew Park <andrew.park@intel.com>
Signed-off-by: hyunback <hyunback.kim@intel.com>
Signed-off-by: Min, Byungil <byungil.min@intel.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: HyungKi Jeong <68764729+testdrive-profiling-master@users.noreply.github.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Georgy Krivoruchko <georgy.krivoruchko@intel.com>
Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com>
Co-authored-by: Andrei Kashchikhin <andrey.kashchikhin@intel.com>
Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Pavel Durandin <pavel.durandin@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Co-authored-by: Fang Xu <fang.xu@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Luwei Zhou <luwei.zhou@intel.com>
Co-authored-by: cecilia peng <cecilia.peng@intel.com>
Co-authored-by: Maksim Kutakov <maxim.kutakov@gmail.com>
Co-authored-by: Mateusz Mikolajczyk <mateusz.mikolajczyk@intel.com>
Co-authored-by: Mateusz Tabaka <mateusz.tabaka@intel.com>
Co-authored-by: Daniil Lyakhov <daniil.lyakhov@intel.com>
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
Co-authored-by: Anastasia Kuporosova <anastasia.kuporosova@intel.com>
Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>
Co-authored-by: Jorge Ragde <ragdejorge@gmail.com>
Co-authored-by: Vitaliy Urusovskij <vitaliy.urusovskij@intel.com>
Co-authored-by: Oleg Pipikin <oleg.pipikin@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
Co-authored-by: Egor Duplenskii <egor.duplensky@gmail.com>
Co-authored-by: Raasz, Pawel <pawel.raasz@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Xuejun Zhai <xuejun.zhai@intel.com>
Co-authored-by: Andrew Kwangwoong Park <andrew.park@intel.com>
Co-authored-by: Luo Cheng <cheng.luo@intel.com>
Co-authored-by: Zhang Yi <yi3.zhang@intel.com>
Co-authored-by: Sun Xiaoxia <xiaoxia.sun@intel.com>
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Sergey Shlyapnikov <sergey.shlyapnikov@intel.com>
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Evgeny Kotov <evgeny.kotov@intel.com>
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Co-authored-by: Xiuchuan Zhai <xiuchuan.zhai@intel.com>
Co-authored-by: Jade Cho <jade.cho@intel.com>
Co-authored-by: hyunback kim <hyunback.kim@intel.com>
Co-authored-by: Wilson Seok <wilson.seok@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
Co-authored-by: Vladislav Golubev <vladislav.golubev@intel.com>
Co-authored-by: Min, Byungil <byungil.min@intel.com>
Co-authored-by: Tingqian Li <tingqian.li@intel.com>
Co-authored-by: Evgenya Nugmanova <evgeniia.nugmanova@intel.com>
Co-authored-by: Anastasiia Pnevskaia <anastasiia.pnevskaia@intel.com>
Co-authored-by: Edward Shogulin <edward.shogulin@intel.com>
Co-authored-by: Irina Efode <irina.efode@intel.com>
Co-authored-by: Ekaterina Aidova <ekaterina.aidova@intel.com>
Co-authored-by: Eddy Kim <eddy.kim@intel.com>
Co-authored-by: Kelvin Choi <kelvin.choi@intel.com>
Co-authored-by: Steve Yoo <steve.yoo@intel.com>
Co-authored-by: Roman Lyamin <Roman.Lyamin@intel.com>
Co-authored-by: David Nam <david.nam@intel.com>
Co-authored-by: Alexandra Sidorova <alexandra.sidorova@intel.com>
Co-authored-by: Yury Gaydaychuk <yury.gaydaychuk@intel.com>
Co-authored-by: Denis Orlov <denis.orlov@intel.com>
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Mingyu Kim <mingyu.kim@intel.com>
Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nikolay Shchegolev <nikolay.shchegolev@intel.com>
Co-authored-by: Piotr Kowalczyk <piotr.kowalczyk@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Surya Siddharth Pemmaraju <surya.siddharth.pemmaraju@intel.com>
Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com>
Co-authored-by: Sofya Balandina <sofya.balandina@intel.com>
Co-authored-by: Nesterov Alexander <alexander.nesterov@intel.com>
Co-authored-by: Evgeny Talanin <evgeny.talanin@intel.com>
Co-authored-by: Piotr Krzemiński <piotr.krzeminski@intel.com>
Co-authored-by: Gorokhov Dmitriy <dmitry.gorokhov@intel.com>
Co-authored-by: Pavel Esir <pavel.esir@intel.com>
Co-authored-by: Łukasz Mielicki <lukasz.mielicki@intel.com>
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
Co-authored-by: Jacek Pawlak <jacek.pawlak@intel.com>
Co-authored-by: tadamczx <156996781+tadamczx@users.noreply.github.com>
Co-authored-by: Andrey Babushkin <andrey.babushkin@intel.com>
Co-authored-by: Alexander Suvorov <alexander.suvorov@intel.com>
2024-03-05 12:56:30 +00:00
Sebastian Golebiewski 5253f01bea
[DOCS] Update link to API for 24.0 (#23273)
Fixing reference to C++ API.
2024-03-05 11:57:41 +00:00
Sebastian Golebiewski f7b9890afd
[DOCS] Update distribution comparison table for 24.0 (#23271)
Updating distribution comparison table.
2024-03-05 10:02:20 +00:00
Alexander Suvorov db4e2f8b3f
Update Selector Tool NPM installation instructions link (#23260) 2024-03-04 17:25:51 +00:00
Tatiana Savina c5b4d9e30d
[DOCS] Change install links (#23262)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-04 17:03:14 +00:00
Maciej Smyk 1ae3a4df7b
[DOCS] Linkfix for Sample Applications (Python & C++) & Generative AI Optimization and Deployment sections for 2024 (#23259)
Fixed links in Sample Applications (Python & C++) & Generative AI
Optimization and Deployment sections of documentation.
2024-03-04 15:27:38 +00:00
Sebastian Golebiewski 90ca2e0130
[DOCS] Fixing incorrect references and phrases for 24.0 (#23257)
Fixing incorrect references and phrases.
2024-03-04 14:10:39 +00:00
Alexander Suvorov 81acc0783f
Update Selector Tool 2024 (#23246) 2024-03-04 13:01:39 +00:00
Maciej Smyk 62a138e169
[DOCS] Rename of operations-specifications section for 2024 (#23198)
Renamed "operations-specifications" file to "operation-specs".
2024-03-04 12:58:44 +00:00
Sebastian Golebiewski 3430424866
[DOCS] Linking issues fix in Legacy Features for 24.0 (#23242)
Fixing issues with references to articles and external resources in the
`Legacy Features` section of the docs.
2024-03-04 11:21:06 +00:00
Maciej Smyk 8d1bce2860
[DOCS] Add information about ONNX backend to the docs for 2024 (#23196)
Port from https://github.com/openvinotoolkit/openvino/pull/23096

### Details:
- The quantize with accuracy control supports the ONNX backend. I have
added information on how to use quantize with accuracy control for
`onnx.ModelProto`.

### Tickets:
 - *133388*

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
2024-03-04 10:48:39 +00:00
Vishniakov Nikolai 2d940ffba8
[DOCS][OV JS] Update JS API readme (#23138) (#23186)
Port of: https://github.com/openvinotoolkit/openvino/pull/23138

### Details:
 - Actualize JS API readme file for npm
 - Fix link to node directory
 - Fix .npmignore to do not include extra files into npm package

Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
2024-03-04 07:48:03 +00:00
Sebastian Golebiewski e34a899934
[DOCS] Add Node.js API reference (#23189)
Adding docs for OpenVINO Node.JS API.

---------

Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>
2024-03-01 17:02:03 +00:00
Karol Blaszczak 153259d191
[DOCS] 2024 linking and adjustments (#23188)
relates to: https://github.com/openvinotoolkit/openvino/pull/23094
2024-03-01 16:14:36 +00:00
Maciej Smyk 8ccd4b0408
[DOCS] Linkfix for Get Started section + removal of GPU drivers table in configurations for 2024 (#23187)
* Fixed links in Get Started section of documentation
* Hid the table with GPU drivers used for validation. Current list
outdated, and the new one will be provided at later date.
2024-03-01 15:42:12 +00:00
Sebastian Golebiewski efbc1fd5f7
[DOCS] Adding npm distribution to install guides for 24.0 (#23171)
Adding npm distribution of OpenVINO™ Toolkit do install guides.

Porting: https://github.com/openvinotoolkit/openvino/pull/23164

---------

Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>
2024-03-01 10:43:26 +00:00
Karol Blaszczak c600170500
[DOCS] 2024 linking issues run-inf (#23162)
related to: https://github.com/openvinotoolkit/openvino/pull/23094
2024-02-29 21:40:29 +00:00
Tatiana Savina 81d061fc70
[DOCS] Links fixes (#23172)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-02-29 16:41:04 +00:00
Tatiana Savina 79c8d38174
[DOCS] Fix links for 24.0 (#23150)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-02-29 13:48:17 +00:00
Andrzej Kopytko e03ea569de
[DOCS] Updated links to files for 2024 version (#23163)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-02-29 12:43:36 +00:00
Karol Blaszczak 2f8b7c011d
[DOCS] 2024 linkfix op-inf (#23139)
related to: https://github.com/openvinotoolkit/openvino/pull/23094
2024-02-29 11:44:46 +00:00
Sebastian Golebiewski 4e4f74b89b
[DOCS] Fix path to notebooks in toctree (#23160)
Fixing path to notebooks folder for `:glob:` parameter in toctree.
2024-02-29 11:22:23 +00:00
Maciej Smyk b921ad8d02
[DOCS] Update of structure for install guide files for 2024.0 (#23159)
Updating structure of files in install guide folder in repo, along with
update of link path to these files in docs.
2024-02-29 10:55:27 +00:00
Andrey Babushkin 2998e69bc9
[chore] Backport sccache idle timeout (#23147)
Backport change for Windows GitHub Actions Workflow
https://github.com/openvinotoolkit/openvino/pull/22928/files
2024-02-28 20:04:26 +00:00
Maciej Smyk 1f4476525b
[DOCS] Update to filenames and structure in install guide section for 2024.0 (#23145)
Port from https://github.com/openvinotoolkit/openvino/pull/23140

- Removed "-overview" and "-header" suffixes in filenames in install
guide scection.
- Moved files from install-guide-shared to install-openvino folder
2024-02-28 16:31:06 +00:00
Maciej Smyk 717875cd43
[DOCS] Update of hyperlinks to 2024 + new ov homepage diagram image for 2024 (#23132)
Port from https://github.com/openvinotoolkit/openvino/pull/23091

* Update of links in docs to 2024 in repo.
* Replaced ov homepage diagram with a new version without Kalid, MXNet
and Caffe
2024-02-28 12:27:23 +00:00
Zlobin Vladimir b8c4854949
Update open_model_zoo submodule (#23137)
Catch up https://github.com/openvinotoolkit/open_model_zoo/pull/3922
2024-02-28 11:15:19 +00:00
tadamczx cb1d570cae
[DOCS] New file structure for docs (#23094)
- Updated paths in an entire documentation,
- updated scripts used in docs building process,
- updated docs cmake to handle new scripts.
2024-02-27 17:26:59 +00:00
Zlobin Vladimir 172272bb36
Update open_model_zoo submodule (#23097)
Catch up https://github.com/openvinotoolkit/open_model_zoo/pull/3920
2024-02-27 05:18:39 +00:00
Sebastian Golebiewski 3185ad07a8
[DOCS] Update Interactive Tutorials with selector tool for 24.0 (#23090)
Updating Interactive Tutorials: redesigning section page and adding the
selector tool from
https://github.com/openvinotoolkit/openvino/pull/22965

Porting: 
https://github.com/openvinotoolkit/openvino/pull/23088
2024-02-26 15:17:00 +00:00
Sebastian Golebiewski d330bde10b
[DOCS] Add Node.js API to docs for 24.0 (#23083)
Porting:
https://github.com/openvinotoolkit/openvino/pull/23055
Adding article about Node.js API in the OpenVINO toolkit.
2024-02-26 10:40:16 +00:00
Andrzej Kopytko e1d4372ca1
[DOCS] Update model table fix for 2024 (#23079)
Port for https://github.com/openvinotoolkit/openvino/pull/23062
2024-02-26 09:22:01 +00:00
Maciej Smyk 140f3300b5
[DOCS] Updated Gen.AI and NNCF documentation for 2024 (#23044)
Port from https://github.com/openvinotoolkit/openvino/pull/22793

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2024-02-23 10:49:56 +00:00
Andrzej Kopytko 3804328e33
[DOCS] Udpate banchmark files PORT to 2024 (#23038)
Port from https://github.com/openvinotoolkit/openvino/pull/23029 to 2024
2024-02-23 08:58:35 +00:00
Karol Blaszczak 591f4fe670
[DOCS) install guide tweaks port 24.0 (#22982)
port: https://github.com/openvinotoolkit/openvino/pull/22976
2024-02-23 05:37:33 +00:00
Maciej Smyk 4f67b55523
[DOCS] macOS version update to 12.6 for 2024.0 (#22987)
Port from https://github.com/openvinotoolkit/openvino/pull/22986

Updating min supported version of macOS to 12.6 in docs.

Jira: 130339
2024-02-22 14:37:11 +00:00
Vishniakov Nikolai 34caeefd07
[OV JS] Specify release version and binaries links in package.json (#22960) (#22991)
Port of https://github.com/openvinotoolkit/openvino/pull/22960

### Details:
 - Specify release version in package.json
- Set up production links to binaries

### Tickets:
 - 131978
2024-02-22 12:30:48 +00:00
Andrzej Kopytko 203c00b55d
[DOCS] Update chart js to v4 port to 2024 (#23018)
port from https://github.com/openvinotoolkit/openvino/pull/23008
2024-02-22 12:18:59 +00:00
Gorokhov Dmitriy b54f753354
[CPU] Fixed BF16 Matmul inference precision (#22995)
CPU plugin uses EnforceInferencePrecision routine for BF16 precision
mark-up. Its logic assumes only activations precision is changed before
Matmul op, while weights precision keeps w/o any changes. Since
dnnlFCTypeMapping misses BF16 activation, FP32 weights optimized
configuration for bf16, execution always happens in FP32 precision even
user manually set infer_precision=bf16.
This bug is not visible on FP16 IRs (since BF16+FP16 config is present),
so only FP32 IRs affected. SInce save_model and ovc apply FP16
compression be default, the issue mostly applicable for pipelines which
use a model directly after convert_model call.

Cherry-picks: https://github.com/openvinotoolkit/openvino/pull/22994
2024-02-22 11:31:35 +00:00
Maksim Kutakov 43bc502ef1
[CPU] Fix selecting conv primitive for the FC node (#22933)
### Details:
Backport https://github.com/openvinotoolkit/openvino/pull/22929

### Tickets:
 - 132558
2024-02-22 09:41:47 +00:00
Vitaliy Urusovskij 78580d2a4f
[2024.0] Additional cleaning of `InferenceEngine` (#22959) (#22996) 2024-02-22 09:41:21 +00:00
Sebastian Golebiewski 6dbabb9284
[DOCS] Openvino api ref update for 24.0 (#22969)
Changing references from "OpenVINO API 2.0" to "OpenVINO API", as
mentioned in
https://github.com/openvinotoolkit/openvino/pull/22873#discussion_r1492115073
Porting: https://github.com/openvinotoolkit/openvino/pull/22884
2024-02-20 18:35:11 +00:00
Jacek Pawlak 7e7087bc3c
Updated copyright year to 2024 (#22958)
[CVS-132992](https://jira.devtools.intel.com/browse/CVS-132992)
2024-02-20 17:01:21 +00:00
Oleg Pipikin fd84ed7233
[DOCS] Remove some api 1.0 names and mentions from dev docs (#22873) (#22936)
### Details:
 - Remove some api 1.0 names and mentions from dev docs
- Cherry-picked from
https://github.com/openvinotoolkit/openvino/pull/22873
 
### Tickets:
 - CVS-132237
2024-02-20 15:47:49 +00:00
Sebastian Golebiewski 900b75b955
[DOCS] Update API reference for 24.0 (#22914)
Updating API reference with ``openvino`` namespace redesign changes.
This PR addresses the following ticket: 117329
Porting: https://github.com/openvinotoolkit/openvino/pull/22911
2024-02-20 14:11:01 +00:00
Gorokhov Dmitriy a4e9bd32d7
[CPU] Disabled zero-points folding for dynamic quantization case (#22918)
### Details:
- This PR disabled zero-points constant folding in case dynamic
quantization is enabled. Dynamic quantization kernels requires
zero-points to be in u4/u8 precision while constant folding converted
them into fp32.
2024-02-19 16:47:09 +00:00
Maxim Vafin b9b4fa13d1
[DOCS] Update list of supported ops for 2024.0 (#22924)
### Details:
 - *Update list of supported ops for 2024.0*

### Tickets:
 - *None*
2024-02-19 16:25:13 +00:00
Andrzej Kopytko 454d760f61
Docs update release version name for 2024 (#22921)
Version renaming for release 2024
2024-02-19 12:19:02 +00:00
Andrzej Kopytko 73a8c2e8ba
DOCS Update banner config for 2024 (#22920)
Port from https://github.com/openvinotoolkit/openvino/pull/22916
2024-02-19 12:14:08 +00:00
Alexandra Sidorova 7b2bfbfb80
[Snippets][CPU][2024.0] Fixed AddressSanitizer: stack-use-after-scope (#22915)
### Details:
- *Port the PR https://github.com/openvinotoolkit/openvino/pull/22889 to
the release 2024.0*

### Tickets:
 - *132849*
2024-02-19 10:32:07 +00:00
Ilya Lavrenov 72789e71ec
Removed /Zi from gtest (#22866) (#22898)
### Details:
- There is no need to keep `/Zi` at all, because OV will propagate all
its flags to gtest

### Details:
 - Ported https://github.com/openvinotoolkit/openvino/pull/22866
2024-02-17 22:05:36 +04:00
Jan Iwaszkiewicz 29bb5571e4
[PyOV] Remote API support for Python (#22883)
### Details:
 - Port of https://github.com/openvinotoolkit/openvino/pull/22629

### Tickets:
 - *N/A*
2024-02-16 16:13:44 +00:00
Maciej Smyk aa5ed90bd7
[DOCS] Removal of Inference Engine from 2024.0 (#22890)
Port from https://github.com/openvinotoolkit/openvino/pull/22852

Removing mentions of deprecated Inference Engine (API 1.0) in docs

Jira: 131850
2024-02-16 15:18:54 +00:00
Roman Kazantsev 610dbccbbe
[TF FE][DOC] Mark new operations with NEW mark (#22864) (#22870)
**Details:** Mark new operations with NEW mark. In release notes, we
refer this documentation where user can find all NEW operation in the
current release.

**Ticket:** TBD

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
2024-02-15 18:56:49 +00:00
Sebastian Golebiewski 62ffe2148a
[DOCS] Updating Interactive Tutorials (#22858)
Updating Interactive Tutorials - adding new notebooks. Porting:
https://github.com/openvinotoolkit/openvino/pull/22815
2024-02-15 15:18:03 +00:00
Alina Kladieva aeffb27137
[GHA] Switch refs to 2024.0 (#22863) 2024-02-15 14:48:04 +00:00
Tatiana Savina 37a071660a
[DOCS] Delete prerelease note (#22860)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-02-15 13:41:36 +00:00
6193 changed files with 163043 additions and 405049 deletions

View File

@ -1,3 +0,0 @@
*
!install_build_dependencies.sh
!scripts/install_dependencies/install_openvino_dependencies.sh

2
.gitattributes vendored
View File

@ -65,5 +65,3 @@
*.vsdx filter=lfs diff=lfs merge=lfs -text
*.bmp filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
.github/scripts/workflow_rerun/tests/data/log_archive_with_error.zip filter=lfs diff=lfs merge=lfs -text
.github/scripts/workflow_rerun/tests/data/log_archive_wo_error.zip filter=lfs diff=lfs merge=lfs -text

12
.github/CODEOWNERS vendored
View File

@ -29,14 +29,9 @@
/src/bindings/python/ @openvinotoolkit/openvino-ie-python-api-maintainers
/src/bindings/c/ @openvinotoolkit/openvino-c-api-maintainers
/src/bindings/js/ @openvinotoolkit/openvino-js-api-maintainers
/src/common/*transformations/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/ @openvinotoolkit/openvino-ngraph-maintainers
# OpenVINO Transformations
/src/common/*transformations/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/src/pass/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/src/pattern/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/include/openvino/pass/ @openvinotoolkit/openvino-ie-transformations-maintainers
# OpenVINO Samples:
/samples/c/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-c-api-maintainers
/samples/cpp/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-maintainers
@ -65,9 +60,6 @@
/src/tests/**/gpu/ @openvinotoolkit/openvino-ie-gpu-maintainers
/thirdparty/ocl/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
# OpenVINO NPU:
/src/plugins/intel_npu/ @openvinotoolkit/openvino-npu-maintainers @openvinotoolkit/openvino-npu-developers
# OpenVINO Auto (MULTI) plugin:
/src/plugins/auto/ @openvinotoolkit/openvino-ie-auto-multi-maintainers
@ -101,7 +93,7 @@
/tests/layer_tests/tensorflow_tests @openvinotoolkit/openvino-tf-frontend-maintainers
/tests/layer_tests/jax_tests @openvinotoolkit/openvino-tf-frontend-maintainers
/tests/model_hub_tests @openvinotoolkit/openvino-tf-frontend-maintainers
/tests/model_hub_tests/pytorch @openvinotoolkit/openvino-pytorch-frontend-maintainers
/tests/model_hub_tests/torch_tests @openvinotoolkit/openvino-pytorch-frontend-maintainers
# Tools:
/tools/ @openvinotoolkit/openvino-tools-maintainers

View File

@ -72,7 +72,7 @@ body:
- type: textarea
id: build_script
attributes:
label: Build script or step-by-step to reproduce
label: Build scrip or step-by-step to reproduce
description: How can we reproduce your issue?
placeholder: Please provide detailed instructions on how to reproduce the issue
validations:

View File

@ -43,7 +43,6 @@ body:
value: |
- [Contribution guide - start here!](https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md)
- [Intel DevHub Discord channel](https://discord.gg/7pVRxUwdWG) - engage in discussions, ask questions and talk to OpenVINO developers
- [How to link your Pull Request to an issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#manually-linking-a-pull-request-to-an-issue-using-the-pull-request-sidebar)
validations:
required: true

View File

@ -1,4 +0,0 @@
lib/
dist/
node_modules/
coverage/

View File

@ -1 +0,0 @@
20.6.0

View File

@ -1,3 +0,0 @@
dist/
node_modules/
coverage/

View File

@ -1,16 +0,0 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": true,
"singleQuote": true,
"quoteProps": "as-needed",
"jsxSingleQuote": false,
"trailingComma": "none",
"bracketSpacing": true,
"bracketSameLine": true,
"arrowParens": "avoid",
"proseWrap": "always",
"htmlWhitespaceSensitivity": "css",
"endOfLine": "lf"
}

View File

@ -1,286 +0,0 @@
/**
* Unit tests for the action's main functionality, src/cleanupImpl.js
*/
const core = require('@actions/core');
const path = require('path');
const os = require('os');
const fs = require('fs');
const cleanupImpl = require('../src/cleanupImpl');
// Mock the GitHub Actions core library
const getInputMock = jest.spyOn(core, 'getInput').mockImplementation();
const setFailedMock = jest.spyOn(core, 'setFailed').mockImplementation();
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-cleanup-'));
const cacheRemotePath = path.join(tempDir, 'cache_remote');
const cacheFiles = ['cache_1.cache', 'cache_2.cache', 'cache_3.cache'];
const minAccessTime = 7 * 24 * 60 * 60 * 1000; // 1 week
// Mock the action's main function
const runMock = jest.spyOn(cleanupImpl, 'cleanUp');
describe('cleanup', () => {
beforeEach(() => {
jest.clearAllMocks();
// Set up file system before each test
fs.mkdirSync(cacheRemotePath, { recursive: true });
const fileSizeInBytes = 1024 * 1024 * 1024; // 1 GB
const buffer = Buffer.alloc(fileSizeInBytes);
let id = 1;
for (const cache of cacheFiles) {
const cachePath = path.join(cacheRemotePath, cache);
fs.writeFileSync(cachePath, buffer);
fs.utimesSync(
cachePath,
new Date(Date.now() - minAccessTime * id++),
new Date()
);
}
});
// Clean up mock file system after each test
afterEach(() => {
fs.rmSync(tempDir, { recursive: true });
});
it('Cleanup old files using restore key', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
});
it('Cleanup old files using key', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'cache-size':
return 1;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
});
it('Skip cleanup old files by size', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 5;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
for (const cache of cacheFiles) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(true);
}
});
it('Skip cleanup old files by atime', async () => {
for (const cache of cacheFiles) {
const cachePath = path.join(cacheRemotePath, cache);
fs.utimesSync(
cachePath,
new Date(Date.now() - minAccessTime / 2),
new Date()
);
}
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
case 'max-cache-size':
return 5;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
for (const cache of cacheFiles) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(true);
}
});
it('Cleanup recently used files by max cache limit', async () => {
for (const cache of cacheFiles) {
const cachePath = path.join(cacheRemotePath, cache);
fs.utimesSync(
cachePath,
new Date(Date.now() - minAccessTime / 2),
new Date()
);
}
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
case 'max-cache-size':
return 2;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
});
it('Test unexpected behaviour', async () => {
// Set folder permissions to read-only
fs.chmodSync(cacheRemotePath, '555');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 2;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
expect(setFailedMock).not.toHaveBeenCalled();
fs.chmodSync(cacheRemotePath, '777');
});
it('Cleanup absent directory', async () => {
const cachePathAbsent = path.join(tempDir, 'cache_remote_absent');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cachePathAbsent;
case 'key':
return 'cache';
case 'restore-keys':
return 'cache';
case 'cache-size':
return 2;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Cleanup directory with subdirectory', async () => {
const cacheSubPath = path.join(cacheRemotePath, 'cache_subdir.cache');
fs.mkdirSync(cacheSubPath, { recursive: true });
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'restore-keys':
return 'cache';
case 'cache-size':
return 1;
default:
return '';
}
});
await cleanupImpl.cleanUp();
expect(runMock).toHaveReturned();
// cache2 and cache3 should be removed
for (const cache of cacheFiles.slice(1, 2)) {
expect(fs.existsSync(path.join(cacheRemotePath, cache))).toBe(false);
}
// check that file1 exists
expect(fs.existsSync(path.join(cacheRemotePath, cacheFiles[0]))).toBe(true);
// check that sub directory exists
expect(fs.existsSync(cacheSubPath)).toBe(true);
});
});

View File

@ -1,266 +0,0 @@
/**
* Unit tests for the action's main functionality, src/restoreImpl.js
*/
const core = require('@actions/core');
const path = require('path');
const os = require('os');
const fs = require('fs');
const tar = require('tar');
const restoreImpl = require('../src/restoreImpl');
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-restore-'));
const cacheLocalPath = path.join(tempDir, 'cache_local');
const cacheRemotePath = path.join(tempDir, 'cache_remote');
const cacheTmpPath = path.join(tempDir, 'cache_tmp');
const cacheFiles = ['cache_1.cache', 'cache_2.cache', 'cache_3.cache'];
const testFiles = ['file1.txt', 'file2.txt', 'file3.txt'];
// Mock the GitHub Actions core library
const getInputMock = jest.spyOn(core, 'getInput').mockImplementation();
const setOutputMock = jest.spyOn(core, 'setOutput').mockImplementation();
const setFailedMock = jest.spyOn(core, 'setFailed').mockImplementation();
// Clean up mock file system after each test
afterEach(() => {
fs.rmSync(tempDir, { recursive: true });
});
// Mock the action's main function
const runMock = jest.spyOn(restoreImpl, 'restore');
describe('restore', () => {
beforeEach(() => {
jest.clearAllMocks();
// Set up mock file system before each test
fs.mkdirSync(cacheLocalPath, { recursive: true });
fs.mkdirSync(cacheRemotePath, { recursive: true });
fs.mkdirSync(cacheTmpPath, { recursive: true });
// Define test files
const file1Path = path.join(cacheTmpPath, testFiles[0]);
fs.writeFileSync(file1Path, 'File 1 contents');
tar.c(
{
gzip: true,
file: cacheFiles[0],
cwd: cacheTmpPath,
sync: true
},
['.']
);
fs.renameSync(cacheFiles[0], path.join(cacheRemotePath, cacheFiles[0]));
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[0]),
new Date(Date.now() - 1500),
new Date(Date.now() - 1500)
);
//
const file2Path = path.join(cacheTmpPath, testFiles[1]);
fs.writeFileSync(file2Path, 'File 2 contents');
tar.c(
{
gzip: true,
file: cacheFiles[1],
cwd: cacheTmpPath,
sync: true
},
['.']
);
fs.renameSync(cacheFiles[1], path.join(cacheRemotePath, cacheFiles[1]));
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[1]),
new Date(Date.now() - 1000),
new Date(Date.now() - 1000)
);
//
const file3Path = path.join(cacheTmpPath, testFiles[2]);
fs.writeFileSync(file3Path, 'File 3 contents');
tar.c(
{
gzip: true,
file: cacheFiles[2],
cwd: cacheTmpPath,
sync: true
},
['.']
);
fs.renameSync(cacheFiles[2], path.join(cacheRemotePath, cacheFiles[2]));
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[2]),
new Date(Date.now() - 500),
new Date(Date.now() - 500)
);
});
it('gets the cache file', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
cacheFiles[2]
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
let id = 1;
for (const filename of testFiles) {
const filePath = path.join(cacheLocalPath, filename);
expect(fs.existsSync(filePath)).toBe(true);
const fileContent = fs.readFileSync(filePath, 'utf8');
expect(fileContent).toBe(`File ${id++} contents`);
}
});
it('gets the updated cache file', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
// updated cache 2 access time
fs.utimesSync(
path.join(cacheRemotePath, cacheFiles[1]),
new Date(Date.now() - 100),
new Date(Date.now() - 100)
);
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
cacheFiles[1]
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
let id = 1;
for (const filename of testFiles.slice(0, 1)) {
const filePath = path.join(cacheLocalPath, filename);
expect(fs.existsSync(filePath)).toBe(true);
const fileContent = fs.readFileSync(filePath, 'utf8');
expect(fileContent).toBe(`File ${id++} contents`);
}
// check that file3 is absent
expect(fs.existsSync(path.join(cacheLocalPath, testFiles[2]))).toBe(false);
});
it('gets the cache file and extract to absent dir', async () => {
const cacheAbsentPath = path.join(tempDir, 'cache_absent');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'restore-keys':
return 'cache';
case 'path':
return cacheAbsentPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
cacheFiles[2]
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
let id = 1;
for (const filename of testFiles) {
const filePath = path.join(cacheAbsentPath, filename);
expect(fs.existsSync(filePath)).toBe(true);
const fileContent = fs.readFileSync(filePath, 'utf8');
expect(fileContent).toBe(`File ${id++} contents`);
}
});
it('Test when no cache found', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'key';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
// Verify that all of the core library functions were called correctly
expect(setOutputMock).toHaveBeenNthCalledWith(1, 'cache-file', '');
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', false);
for (const filename of testFiles) {
const filePath = path.join(cacheLocalPath, filename);
expect(fs.existsSync(filePath)).toBe(false);
}
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Unsupported cache found', async () => {
const cacheFakePath = path.join(cacheRemotePath, 'cache-fake.cache');
fs.writeFileSync(cacheFakePath, 'Fake content');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await restoreImpl.restore();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).not.toHaveBeenCalled();
});
});

View File

@ -1,190 +0,0 @@
/**
* Unit tests for the action's main functionality, src/saveImpl.js
*/
const core = require('@actions/core');
const path = require('path');
const os = require('os');
const fs = require('fs');
const saveImpl = require('../src/saveImpl');
// Mock the GitHub Actions core library
const getInputMock = jest.spyOn(core, 'getInput').mockImplementation();
const setOutputMock = jest.spyOn(core, 'setOutput').mockImplementation();
const setFailedMock = jest.spyOn(core, 'setFailed').mockImplementation();
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-'));
const cacheLocalPath = path.join(tempDir, 'cache_local');
const cacheRemotePath = path.join(tempDir, 'cache_remote');
// Clean up mock file system after each test
afterEach(() => {
fs.rmSync(tempDir, { recursive: true });
});
// Mock the action's main function
const runMock = jest.spyOn(saveImpl, 'save');
describe('save', () => {
beforeEach(() => {
jest.clearAllMocks();
// Set up mock file system before each test
// Create cache pathes
fs.mkdirSync(cacheLocalPath, { recursive: true });
fs.mkdirSync(cacheRemotePath, { recursive: true });
// Define test files
const file1Path = path.join(cacheLocalPath, 'file1.txt');
const file2Path = path.join(cacheLocalPath, 'file2.txt');
// Write content to the test files
fs.writeFileSync(file1Path, 'File 1 contents');
fs.writeFileSync(file2Path, 'File 2 contents');
});
it('Cache files pass', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
'cache.cache'
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
expect(fs.existsSync(`${cacheRemotePath}/cache.cache`)).toBe(true); // Check if the tar file was created
});
it('Cache files: absent local cache dir', async () => {
// Set the action's inputs as return values from core.getInput()
const cachePathAbsent = path.join(tempDir, 'cache_local_absent');
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cachePathAbsent;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(fs.existsSync(`${cacheRemotePath}/cache.cache`)).toBe(false); // Check that the tar file was not created
});
it('Cache files: absent remote cache dir', async () => {
// Set the action's inputs as return values from core.getInput()
const cacheRemotePathAbsent = path.join(tempDir, 'cache_remote_absent');
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePathAbsent;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).toHaveBeenNthCalledWith(
1,
'cache-file',
'cache.cache'
);
expect(setOutputMock).toHaveBeenNthCalledWith(2, 'cache-hit', true);
expect(fs.existsSync(`${cacheRemotePathAbsent}/cache.cache`)).toBe(true); // Check if the tar file was created
});
it('Cache files: key is not set', async () => {
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Cache files: file already exists', async () => {
const cacheFakePath = path.join(cacheRemotePath, 'cache-fake.cache');
fs.writeFileSync(cacheFakePath, 'Fake content');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache-fake';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).not.toHaveBeenCalled();
});
it('Test unexpected behaviour', async () => {
// Set folder permissions to read-only
fs.chmodSync(cacheRemotePath, '555');
// Set the action's inputs as return values from core.getInput()
getInputMock.mockImplementation(name => {
switch (name) {
case 'cache-path':
return cacheRemotePath;
case 'key':
return 'cache';
case 'path':
return cacheLocalPath;
default:
return '';
}
});
await saveImpl.save();
expect(runMock).toHaveReturned();
expect(setOutputMock).not.toHaveBeenCalled();
expect(setFailedMock).toHaveBeenCalled();
});
});

View File

@ -1,60 +0,0 @@
name: 'Cache'
description: 'Cache to the local file system'
author: 'Mikhail Ryzhov'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
path:
description:
'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description:
'An ordered list of keys to use for restoring stale cache if no cache hit
occurred for key.'
default: ''
required: false
save-always:
description:
'Run the post step to save the cache even if another step before fails'
default: false
required: false
cleanup-always:
description:
'Run the post cleanup step to remove old cache files even if another step
before fails'
default: false
required: false
cache-size:
description:
'Allowed cache storage size in Gb. Least recently used caches will be
automatically evicted to limit the total cache storage. The cache size can
exceed this value, but it is limited by "max-cache-size" value'
default: 10
required: false
max-cache-size:
description:
'Maximum cache storage size in Gb. All cache files will be automatically
evicted to limit the total cache storage'
default: 50
required: false
outputs:
cache-hit:
description:
'A boolean value to indicate an exact match was found for the primary key'
cache-file:
description: 'Found cache file'
runs:
using: node20
main: 'dist/restore/index.js'
post: 'dist/save/index.js'
post-if:
'success() || github.event.inputs.save-always ||
github.event.inputs.cleanup-always'

View File

@ -1,37 +0,0 @@
name: 'Cleanup Cache'
description: 'Remove old cache artifacts'
author: 'OpenVINO Developers'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description:
'An ordered list of keys to use for restoring stale cache if no cache hit
occurred for key.'
default: ''
required: false
cache-size:
description:
'Allowed cache storage size in Gb. Least recently used caches will be
automatically evicted to limit the total cache storage. The cache size can
exceed this value, but it is limited by "max-cache-size" value'
default: 10
required: false
max-cache-size:
description:
'Maximum cache storage size in Gb. All cache files will be automatically
evicted to limit the total cache storage'
default: 50
required: false
runs:
using: 'node20'
main: '../dist/cleanup-only/index.js'
branding:
icon: 'archive'
color: 'gray-dark'

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -1,50 +0,0 @@
env:
commonjs: true
es6: true
jest: true
node: true
globals:
Atomics: readonly
SharedArrayBuffer: readonly
ignorePatterns:
- '!.*'
- '**/node_modules/.*'
- '**/dist/.*'
- '**/coverage/.*'
- '*.json'
parser: '@babel/eslint-parser'
parserOptions:
ecmaVersion: 2023
sourceType: module
requireConfigFile: false
babelOptions:
babelrc: false
configFile: false
presets:
- jest
plugins:
- jest
extends:
- eslint:recommended
- plugin:github/recommended
- plugin:jest/recommended
rules:
{
'camelcase': 'warn',
'eslint-comments/no-use': 'off',
'eslint-comments/no-unused-disable': 'off',
'i18n-text/no-en': 'off',
'import/no-commonjs': 'off',
'import/no-namespace': 'off',
'no-console': 'off',
'no-unused-vars': 'warn',
'prettier/prettier': 'error',
'semi': 'error'
}

View File

@ -1,18 +0,0 @@
# Unordered list style
MD004:
style: dash
# Ordered list item prefix
MD029:
style: one
# Spaces after list markers
MD030:
ul_single: 1
ol_single: 1
ul_multi: 1
ol_multi: 1
# Code block style
MD046:
style: fenced

View File

@ -1,10 +0,0 @@
rules:
document-end: disable
document-start:
level: warning
present: false
line-length:
level: warning
max: 80
allow-non-breakable-words: true
allow-non-breakable-inline-mappings: true

10054
.github/actions/cache/package-lock.json generated vendored

File diff suppressed because it is too large Load Diff

View File

@ -1,73 +0,0 @@
{
"name": "compressed-cache-action",
"description": "action to save/restore cache as tarballs",
"version": "0.0.1",
"author": "OpenVINO Developers",
"private": true,
"keywords": [
"GitHub",
"Actions",
"JavaScript"
],
"engines": {
"node": ">=20"
},
"main": "dist/restore/index.js",
"scripts": {
"bundle": "npm run format:write && npm run package",
"ci-test": "npx jest",
"coverage": "npx jest -- --coverage",
"format:write": "npx prettier --write .",
"format:check": "npx prettier --check .",
"lint": "npx eslint . -c ./linters/.eslintrc.yml ",
"package": "npx ncc build src/restore.js -o dist/restore && npx ncc build src/save.js -o dist/save && npx ncc build src/restore.js -o dist/restore-only && npx ncc build src/save.js -o dist/save-only && npx ncc build src/cleanup.js -o dist/cleanup-only",
"package:watch": "npm run package -- --watch",
"test": "npx jest",
"all": "npm run format:write && npm run lint && npm run test && npm run coverage && npm run package"
},
"eslintConfig": {
"extends": "./linters/.eslintrc.yml"
},
"jest": {
"verbose": true,
"clearMocks": true,
"testEnvironment": "node",
"moduleFileExtensions": [
"js"
],
"testMatch": [
"**/*.test.js"
],
"testPathIgnorePatterns": [
"/node_modules/",
"/dist/"
],
"coverageReporters": [
"json-summary",
"text",
"lcov"
],
"collectCoverage": true,
"collectCoverageFrom": [
"./src/**"
]
},
"dependencies": {
"@actions/core": "^1.10.1",
"fs": "^0.0.1-security",
"tar": "^6.2.0"
},
"devDependencies": {
"@babel/core": "^7.24.0",
"@babel/eslint-parser": "^7.23.10",
"@babel/preset-env": "^7.24.0",
"@vercel/ncc": "^0.38.1",
"babel-preset-jest": "^29.6.3",
"eslint": "^8.57.0",
"eslint-plugin-github": "^4.10.2",
"eslint-plugin-jest": "^27.9.0",
"fs": "0.0.1-security",
"jest": "^29.7.0",
"prettier": "^3.2.5"
}
}

View File

@ -1,35 +0,0 @@
name: 'Restore Cache'
description: 'Restore Cache artifacts to improve workflow execution time'
author: 'OpenVINO Developers'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
path:
description:
'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description:
'An ordered list of keys to use for restoring stale cache if no cache hit
occurred for key.'
default: ''
required: false
outputs:
cache-hit:
description:
'A boolean value to indicate an exact match was found for the primary key'
cache-file:
description: 'Found cache file'
runs:
using: 'node20'
main: '../dist/restore-only/index.js'
branding:
icon: 'archive'
color: 'gray-dark'

View File

@ -1,22 +0,0 @@
name: 'Save a cache'
description: 'Cache to the local file system'
author: 'OpenVINO Developers'
inputs:
cache-path:
description: 'A local path where to search and save caches'
required: true
path:
description:
'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
runs:
using: 'node20'
main: '../dist/save-only/index.js'
branding:
icon: 'archive'
color: 'gray-dark'

View File

@ -1,3 +0,0 @@
const { cleanUp } = require('./cleanupImpl');
cleanUp();

View File

@ -1,85 +0,0 @@
const core = require('@actions/core');
const fs = require('fs/promises');
const path = require('path');
const {
getSortedCacheFiles,
humanReadableFileSize,
calculateTotalSize
} = require('./utils');
// Function to remove old files if their combined size exceeds the allowed size
async function cleanUp() {
try {
const cacheRemotePath = core.getInput('cache-path', { required: true });
const key = core.getInput('key', { required: true });
const keysRestore = core
.getInput('restore-keys', { required: false })
.split('\n')
.map(s => s.replace(/^!\s+/, '!').trim())
.filter(x => x !== '');
const cacheSize = core.getInput('cache-size', { required: false });
const cacheMaxSize = core.getInput('max-cache-size', { required: false });
// Minimum time peroid in milliseconds when the files was not useds
const minAccessTime = 7 * 24 * 60 * 60 * 1000; // 1 week
const currentDate = new Date();
const minAccessDateAgo = new Date(currentDate - minAccessTime);
core.debug(`cache-path: ${cacheRemotePath}`);
core.debug(`key: ${key}`);
core.debug(`restore-keys: ${keysRestore}`);
core.debug(`cache-size: ${cacheSize}`);
core.debug(`max-cache-size: ${cacheMaxSize}`);
let keyPattern = key;
if (keysRestore && keysRestore.length) {
keyPattern = keysRestore.join('|');
}
const files = await getSortedCacheFiles(cacheRemotePath, keyPattern);
const minCacheSizeInBytes = cacheSize * 1024 * 1024 * 1024;
const maxCacheSizeInBytes = cacheMaxSize * 1024 * 1024 * 1024;
let totalSize = await calculateTotalSize(cacheRemotePath, files);
if (totalSize <= minCacheSizeInBytes) {
core.info(
`The cache storage size ${humanReadableFileSize(totalSize)} less then the allowed size ${humanReadableFileSize(minCacheSizeInBytes)}`
);
return;
}
core.info(
`The cache storage size ${humanReadableFileSize(totalSize)} exceeds allowed size ${humanReadableFileSize(minCacheSizeInBytes)}`
);
for (const file of files.reverse()) {
const filePath = path.join(cacheRemotePath, file);
const fileStats = await fs.stat(filePath);
// skipping recently used files if total cache size less then maxCacheSizeInBytes
if (
!fileStats.isFile() ||
(fileStats.atime >= minAccessDateAgo && totalSize < maxCacheSizeInBytes)
) {
core.info(`Skipping: ${filePath}`);
continue;
}
core.info(`Removing file: ${filePath}`);
await fs.unlink(filePath);
core.info(`${filePath} removed successfully`);
totalSize -= fileStats.size;
// Exit if total size is within limit
if (totalSize <= minCacheSizeInBytes) {
core.info('Old cache files removed successfully');
return;
}
}
} catch (error) {
core.error(`Error removing old cache files: ${error.message}`);
}
}
module.exports = {
cleanUp
};

View File

@ -1,6 +0,0 @@
/**
* The entrypoint for the action.
*/
const { restore } = require('./restoreImpl');
restore();

View File

@ -1,84 +0,0 @@
const core = require('@actions/core');
const fs = require('fs/promises');
const path = require('path');
const tar = require('tar');
const os = require('os');
const {
getSortedCacheFiles,
humanReadableFileSize,
checkFileExists
} = require('./utils');
/**
* The main function for the action.
* @returns {Promise<void>} Resolves when the action is complete.
*/
async function restore() {
try {
const cacheRemotePath = core.getInput('cache-path', { required: true });
const cacheLocalPath = core.getInput('path', { required: true });
const key = core.getInput('key', { required: true });
const keysRestore = core
.getInput('restore-keys', { required: false })
.split('\n')
.map(s => s.replace(/^!\s+/, '!').trim())
.filter(x => x !== '');
core.debug(`cache-path: ${cacheRemotePath}`);
core.debug(`path: ${cacheLocalPath}`);
core.debug(`key: ${key}`);
core.debug(`restore-keys: ${keysRestore}`);
let keyPattern = key;
if (keysRestore && keysRestore.length) {
keyPattern = keysRestore.join('|');
}
core.info(`Looking for ${keyPattern} in ${cacheRemotePath}`);
const files = await getSortedCacheFiles(cacheRemotePath, keyPattern);
if (files.length) {
const cacheFile = files[0];
const cachePath = path.join(cacheRemotePath, cacheFile);
const cacheStat = await fs.stat(cachePath);
core.info(
`Found cache file: ${cacheFile}, size: ${humanReadableFileSize(cacheStat.size)}`
);
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'cache-'));
// copy file to temp dir
await fs.copyFile(cachePath, path.join(tempDir, cacheFile));
core.info(`${cacheFile} was copied to ${tempDir}/${cacheFile}`);
// extract
if (!(await checkFileExists(cacheLocalPath))) {
await fs.mkdir(cacheLocalPath);
}
core.info(`Extracting ${cacheFile} to ${cacheLocalPath}`);
tar.x({
file: path.join(tempDir, cacheFile),
cwd: cacheLocalPath,
sync: true
});
core.info(`Cache extracted to ${cacheLocalPath}`);
core.setOutput('cache-file', cacheFile);
core.setOutput('cache-hit', true);
} else {
core.warning(
`Could not found any suitable cache files in ${cacheRemotePath} with key ${key}`
);
core.setOutput('cache-file', '');
core.setOutput('cache-hit', false);
}
} catch (error) {
// do not fail action if cache could not be restored
core.error(error.message);
}
}
module.exports = {
restore
};

View File

@ -1,14 +0,0 @@
const core = require('@actions/core');
const { save } = require('./saveImpl');
const { cleanUp } = require('./cleanupImpl');
const saveAlways = core.getInput('save-always', { required: false });
const cleanUpAlways = core.getInput('cleanup-always', { required: false });
if (saveAlways === 'true') {
save();
}
if (cleanUpAlways === 'true') {
cleanUp();
}

View File

@ -1,71 +0,0 @@
const core = require('@actions/core');
const tar = require('tar');
const fs = require('fs/promises');
const path = require('path');
const { humanReadableFileSize, checkFileExists } = require('./utils');
/**
* The main function for the action.
* @returns {Promise<void>} Resolves when the action is complete.
*/
async function save() {
try {
const cacheRemotePath = core.getInput('cache-path', { required: true });
const toCachePath = core.getInput('path', { required: true });
const key = core.getInput('key', { required: true });
core.debug(`cache-path: ${cacheRemotePath}`);
core.debug(`path: ${toCachePath}`);
core.debug(`key: ${key}`);
if (!key) {
core.warning(`Key ${key} is not specified.`);
return;
}
const tarName = `${key}.cache`;
const tarPath = path.join(cacheRemotePath, tarName);
const tarNameTmp = `${key}.tmp`;
const tarPathTmp = path.join(cacheRemotePath, tarNameTmp);
if (await checkFileExists(tarPath)) {
core.warning(`Cache file ${tarName} already exists`);
return;
}
core.info(`Preparing cache archive ${tarName}`);
tar.c(
{
gzip: true,
file: tarName,
cwd: toCachePath,
sync: true
},
['.']
);
const tarStat = await fs.stat(tarName);
core.info(
`Created cache tarball: ${tarName}, size: ${humanReadableFileSize(tarStat.size)}`
);
// remote cache directory may not be created yet
if (!(await checkFileExists(cacheRemotePath))) {
await fs.mkdir(cacheRemotePath, { recursive: true });
}
core.info('Copying cache...');
await fs.copyFile(tarName, tarPathTmp);
// After copying is done, rename file
await fs.rename(tarPathTmp, tarPath);
core.info(`${tarName} copied to ${tarPath}`);
core.setOutput('cache-file', tarName);
core.setOutput('cache-hit', true);
} catch (error) {
core.setFailed(error.message);
}
}
module.exports = {
save
};

View File

@ -1,72 +0,0 @@
const core = require('@actions/core');
const fs = require('fs');
const path = require('path');
async function getSortedCacheFiles(cachePath, key = '') {
if (!(await checkFileExists(cachePath))) {
core.warning(`${cachePath} doesn't exist`);
return [];
}
const cachePattern = new RegExp(`^((${key}).*[.]cache)$`);
const filesSorted = (await fs.promises.readdir(cachePath))
.filter(fileName => cachePattern.test(fileName))
.map(fileName => ({
name: fileName,
time: fs.statSync(path.join(cachePath, fileName)).atimeMs
}))
.sort((a, b) => b.time - a.time)
.map(file => file.name);
core.debug(
filesSorted.map(fileName => ({
name: fileName,
atime: fs.statSync(path.join(cachePath, fileName)).atimeMs
}))
);
return filesSorted;
}
function humanReadableFileSize(sizeInBytes) {
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
let id = 0;
while (sizeInBytes >= 1024 && id < units.length - 1) {
sizeInBytes /= 1024;
id++;
}
return `${sizeInBytes.toFixed(2)} ${units[id]}`;
}
// Function to calculate the total size of files in bytes
async function calculateTotalSize(dir, files) {
let totalSize = 0;
for (const file of files) {
const filePath = path.join(dir, file);
const fileStats = await fs.promises.stat(filePath);
if (fileStats.isFile()) {
totalSize += fileStats.size;
}
}
return totalSize;
}
async function checkFileExists(filePath) {
try {
await fs.promises.access(filePath);
return true;
} catch (error) {
return false;
}
}
module.exports = {
getSortedCacheFiles,
humanReadableFileSize,
calculateTotalSize,
checkFileExists
};

View File

@ -1,72 +0,0 @@
name: 'Handle Docker images'
description: 'Builds, tags and pushes a given Docker image when needed'
inputs:
images:
description: 'Image names (registry name + namespace + base name)'
required: true
registry:
description: 'Docker registry'
required: true
dockerfiles_root_dir:
description: 'Path to dockerfiles root dir relative to repository root'
required: true
push:
description: 'Push built images to registry'
required: false
default: 'true'
changed_components:
description: 'Components changed by a pull request'
required: true
outputs:
images:
description: "Images to use in workflow"
value: ${{ steps.handle_images.outputs.images }}
runs:
using: 'composite'
steps:
- name: Checkout head
uses: actions/checkout@v4
- name: Checkout base
uses: actions/checkout@v4
with:
ref: ${{ github.base_ref || github.event.merge_group.base_ref }}
sparse-checkout: ${{ inputs.dockerfiles_root_dir }}/docker_tag
path: base
- name: Install Python dependencies
uses: py-actions/py-dependency-install@v4
with:
path: "${{ github.action_path }}/requirements.txt"
update-setuptools: "false"
update-wheel: "false"
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
- name: Handle docker images
id: handle_images
shell: bash
run: |
images=$(echo "${{ inputs.images }}" | tr '\n' ',' | sed 's/,*$//')
pr="${{ github.event.pull_request.number }}"
python3 .github/actions/handle_docker/get_images_to_build.py \
-d "${{ inputs.dockerfiles_root_dir }}" \
-r "${{ inputs.registry }}" \
--images "$images" \
--head_tag_file "${{ inputs.dockerfiles_root_dir }}/docker_tag" \
--base_tag_file "base/${{ inputs.dockerfiles_root_dir }}/docker_tag" \
--docker_env_changed "${{ fromJSON(inputs.changed_components).docker_env }}" \
--dockerfiles_changed "${{ fromJSON(inputs.changed_components).dockerfiles }}" \
--docker_builder "${{ steps.buildx.outputs.name}}" \
--repo "${{ github.repository }}" \
--ref_name "${{ github.ref_name }}" \
$([[ -n $pr ]] && echo "--pr $pr" || echo '-s ${{ github.sha }}') \
$([[ -n "${{ inputs.push }}" ]] && echo "--push" || echo '')
env:
GITHUB_TOKEN: ${{ github.token }}

View File

@ -1,129 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
import json
import re
import sys
from distutils.util import strtobool
from helpers import *
from images_api import *
def parse_args():
parser = argparse.ArgumentParser(description='Returns list of Docker images to build for a given workflow')
parser.add_argument('-i', '--images', required=True, help='Comma-separated docker images')
parser.add_argument('-d', '--dockerfiles_root', required=True, help='Path to dockerfiles')
parser.add_argument('-r', '--registry', required=True, help='Docker registry name')
parser.add_argument('-s', '--commit', required=False, help='Commit SHA. If not set, --pr is used')
parser.add_argument('-b', '--docker_builder', required=False, help='Docker buildx builder name')
parser.add_argument('--pr', type=int, required=False, help='PR number, if event is pull_request')
parser.add_argument('--head_tag_file', default='.github/dockerfiles/docker_tag', help='Head docker tag file path')
parser.add_argument('--base_tag_file', default=None, required=False, help='Base docker tag file path')
parser.add_argument('--ref_name', required=False, default='', help='GitHub ref name')
parser.add_argument('--repo', default='openvinotoolkit/openvino', help='GitHub repository')
parser.add_argument('--docker_env_changed', type=lambda x: bool(strtobool(x)), default=True,
help='Whether PR changes docker env')
parser.add_argument('--dockerfiles_changed', type=lambda x: bool(strtobool(x)), default=True,
help='Whether PR changes dockerfiles')
parser.add_argument('--action_path', default='.github/actions/handle_docker', help='Path to this GitHub action')
parser.add_argument('--push', action='store_true', required=False, help='Whether to push images to registry')
parser.add_argument('--dry_run', action='store_true', required=False, help='Dry run')
args = parser.parse_args()
return args
def main():
init_logger()
logger = logging.getLogger(__name__)
args = parse_args()
for arg, value in sorted(vars(args).items()):
logger.info(f"Argument {arg}: {value}")
head_tag = Path(args.head_tag_file).read_text().strip()
base_tag_exists = args.base_tag_file and Path(args.base_tag_file).exists()
base_tag = Path(args.base_tag_file).read_text().strip() if base_tag_exists else None
all_dockerfiles = Path(args.dockerfiles_root).rglob('**/*/Dockerfile')
images = ImagesHandler(args.dry_run)
for image in all_dockerfiles:
images.add_from_dockerfile(image, args.dockerfiles_root, args.registry, head_tag, base_tag)
requested_images = set(args.images.split(','))
skip_workflow = False
missing_only = False
merge_queue_target_branch = next(iter(re.findall(f'^gh-readonly-queue/(.*)/', args.ref_name)), None)
if args.pr:
environment_affected = args.docker_env_changed or args.dockerfiles_changed
if environment_affected:
expected_tag = f'pr-{args.pr}'
if head_tag != expected_tag:
logger.error(f"Please update docker tag in {args.head_tag_file} to {expected_tag}")
sys.exit(1)
elif merge_queue_target_branch:
environment_affected = head_tag != base_tag
if environment_affected:
logger.info(f"Environment is affected by PR(s) in merge group")
else:
environment_affected = False
if environment_affected:
changeset = get_changeset(args.repo, args.pr, merge_queue_target_branch, args.commit)
changed_dockerfiles = [p for p in changeset if p.startswith(args.dockerfiles_root) and p.endswith('Dockerfile')]
if args.docker_env_changed:
logger.info(f"Common docker environment is modified, will build all requested images")
changed_images = requested_images
else:
logger.info(f"Common docker environment is not modified, will build only changed and missing images")
changed_images = set([name_from_dockerfile(d, args.dockerfiles_root) for d in changed_dockerfiles])
unchanged_images = requested_images - changed_images
unchanged_with_no_base = images.get_missing(unchanged_images, base=True)
if unchanged_with_no_base:
logger.info("The following images were unchanged, but will be built anyway since the base for them "
f"is missing in registry: {unchanged_with_no_base}")
images_to_tag = unchanged_images.difference(unchanged_with_no_base)
images_to_build = requested_images.intersection(changed_images).union(unchanged_with_no_base)
only_dockerfiles_changed = len(changeset) == len(changed_dockerfiles)
if only_dockerfiles_changed and not images_to_build:
skip_workflow = True
else:
logger.info(f"Environment is not affected, will build only missing images, if any")
images_to_build = requested_images
images_to_tag = []
missing_only = True
if not images_to_build:
logger.info(f"No images to build, will return the list of pre-built images with a new tag")
built_images = images.build(images_to_build, missing_only, args.push, args.docker_builder)
if not built_images:
logger.info(f"No images were built, a new tag will be applied to a pre-built base image if needed")
# When a custom builder is used, it allows to push the image automatically once built. Otherwise, pushing manually
if args.push and not args.docker_builder:
images.push(images_to_build, missing_only)
if environment_affected and base_tag:
images.tag(images_to_tag)
images_output = images_to_output(images.get(requested_images))
set_github_output("images", json.dumps(images_output))
if skip_workflow:
logger.info(f"Docker image changes are irrelevant for current workflow, workflow may be skipped")
set_github_output("skip_workflow", str(skip_workflow))
main()

View File

@ -1,75 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import os
import subprocess
from ghapi.all import GhApi
from pathlib import Path
def init_logger():
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(name)-15s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
def set_github_output(name: str, value: str, github_output_var_name: str = 'GITHUB_OUTPUT'):
"""Sets output variable for a GitHub Action"""
logger = logging.getLogger(__name__)
# In an environment variable "GITHUB_OUTPUT" GHA stores path to a file to write outputs to
with open(os.environ.get(github_output_var_name), 'a+') as file:
logger.info(f"Add {name}={value} to {github_output_var_name}")
print(f'{name}={value}', file=file)
def images_to_output(images: list):
images_output = {}
for image in images:
image_name, os_name = image.name.split('/', 1)
if image_name not in images_output:
images_output[image_name] = {}
images_output[image_name][os_name] = image.ref()
return images_output
def get_changeset(repo: str, pr: str, target_branch: str, commit_sha: str):
"""Returns changeset either from PR or commit"""
owner, repository = repo.split('/')
gh_api = GhApi(owner=owner, repo=repository, token=os.getenv("GITHUB_TOKEN"))
if pr:
changed_files = gh_api.pulls.list_files(pr)
elif target_branch:
target_branch_head_commit = gh_api.repos.get_branch(target_branch).commit.sha
changed_files = gh_api.repos.compare_commits(f'{target_branch_head_commit}...{commit_sha}').get('files', [])
else:
raise ValueError(f'Either "pr" or "target_branch" parameter must be non-empty')
return set([f.filename for f in changed_files])
def run(cmd: str, dry_run: bool = False, fail_on_error: bool = True):
logger = logging.getLogger('run')
logger.info(cmd)
if dry_run:
return 0, ''
with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) as proc:
for line in proc.stdout:
logger.info(line.strip())
proc.communicate()
if proc.returncode != 0:
msg = f"Command '{cmd}' returned non-zero exit status {proc.returncode}"
if fail_on_error:
raise RuntimeError(msg)
logger.warning(msg)
return proc.returncode
def name_from_dockerfile(dockerfile: str | Path, dockerfiles_root: str | Path) -> str:
image_name = str(Path(dockerfile).relative_to(dockerfiles_root).parent.as_posix())
return image_name

View File

@ -1,137 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import os
import subprocess
from pathlib import Path
from typing import Iterable
from helpers import run, name_from_dockerfile
class Image:
def __init__(self, name: str, dockerfile: Path, registry: str):
self.logger = logging.getLogger(self.__class__.__name__)
self.name = name
self.dockerfile = dockerfile
self.registry = registry
self.tag = 'latest'
self.base_tag = None
def __str__(self):
return self.name
def __eq__(self, img):
return img.name == self.name if img else False
def with_tag(self, tag: str):
self.tag = tag
return self
def with_base_tag(self, tag: str):
self.base_tag = tag
return self
def ref(self):
return f"{self.registry}/{self.name}:{self.tag}"
def base_ref(self):
if not self.base_tag:
return None
return f"{self.registry}/{self.name}:{self.base_tag}"
def push(self, dry: bool = False):
cmd = f"docker push {self.ref()} "
run(cmd, dry)
def build(self, dry: bool = False, push: bool = True, docker_builder: str = None, import_cache: bool = True,
export_cache: bool = True):
cache_cmd = ""
if import_cache:
cache_cmd += f"--cache-from type=registry,ref={self.ref()}-cache "
if self.base_tag:
cache_cmd += f"--cache-from type=registry,ref={self.base_ref()}-cache "
if export_cache:
cache_cmd += f"--cache-to type=registry,ref={self.ref()}-cache,mode=max "
build_cmd = f"docker buildx build --builder={docker_builder}" if docker_builder else "docker build"
push_cmd = f"--push" if push else ""
cmd = f"{build_cmd} " \
f"--file {self.dockerfile} " \
f"--tag {self.ref()} " \
f"{cache_cmd} " \
f"{push_cmd} " \
"."
run(cmd, dry)
def tag_base(self, dry: bool = False):
if not self.base_tag:
raise AttributeError("Tag for base image is not specified")
cmd = f"docker buildx imagetools create -t {self.ref()} {self.base_ref()}"
run(cmd, dry)
def is_missing(self, dry: bool = False, base: bool = False) -> bool:
image = self.base_ref() if base else self.ref()
if base and not image:
self.logger.warning(f"Base ref for image {self.ref()} is missing")
return True
cmd = f"docker manifest inspect {image}"
is_missing = False
self.logger.info(cmd)
if not dry:
try:
subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True, text=True)
except subprocess.CalledProcessError:
self.logger.warning(f"{image} is missing in registry")
is_missing = True
return is_missing
# Making it a class, so it's a little easier to switch to a tree structure for building inherited images if we want
class ImagesHandler:
def __init__(self, dry_run: bool = False):
self.logger = logging.getLogger(self.__class__.__name__)
self.images = dict()
self.dry_run = dry_run
def add_from_dockerfile(self, dockerfile: str | Path, dockerfiles_root: str | Path, registry: str, tag: str,
base_tag: str = None):
image_name = name_from_dockerfile(dockerfile, dockerfiles_root)
image = Image(image_name, Path(dockerfile), registry).with_tag(tag).with_base_tag(base_tag)
self.add(image)
def add(self, image: Image):
self.images[image.name] = image
def get(self, image_names: Iterable = None) -> list:
images = [self.images[name] for name in image_names] if image_names is not None else self.images.values()
return images
def get_missing(self, image_names: Iterable = None, base: bool = False) -> list:
missing_images = [image.name for image in self.get(image_names) if image.is_missing(self.dry_run, base)]
return missing_images
def build(self, image_names: Iterable = None, missing_only: bool = False, push: bool = True, builder: str = None,
import_cache: bool = True, export_cache: bool = True):
to_build = self.get(self.get_missing(image_names)) if missing_only else self.get(image_names)
for image in to_build:
image.build(self.dry_run, push, builder, import_cache, export_cache)
return to_build
def push(self, image_names: Iterable = None, missing_only: bool = False):
to_push = self.get(self.get_missing(image_names)) if missing_only else self.get(image_names)
for image in to_push:
image.push(self.dry_run)
def tag(self, image_names: Iterable = None):
for image in self.get(image_names):
image.tag_base(self.dry_run)

View File

@ -1 +0,0 @@
ghapi~=1.0.5

View File

@ -23,19 +23,7 @@ runs:
using: 'composite'
steps:
- name: Check if Python is already installed (Linux)
if: ${{ runner.os == 'Linux' }}
shell: bash
id: check_python
run: |
PYTHON_INSTALLED=$(python${{ inputs.version }} -V) || true
if [[ $PYTHON_INSTALLED ]]; then
echo "installed=true" >> $GITHUB_OUTPUT
else
echo "installed=false" >> $GITHUB_OUTPUT
fi
- if: ${{ runner.os == 'Linux' && inputs.self-hosted-runner == 'true' && steps.check_python.outputs.installed == 'false' }}
- if: ${{ runner.os == 'Linux' && inputs.self-hosted-runner == 'true' }}
name: Install 'actions/setup-python@v4' dependencies
shell: bash
run: apt-get update && apt-get install -y ca-certificates software-properties-common gpg-agent tzdata
@ -43,18 +31,18 @@ runs:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
TZ: "Europe/London" # to prevent tzdata from waiting user input
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' && steps.check_python.outputs.installed == 'false' }}
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' }}
name: Setup sudo and python3
shell: bash
run: apt-get update && apt-get install -y sudo python3 # Needed for the deadsnakes action
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' && steps.check_python.outputs.installed == 'false' }}
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' }}
name: Setup Python ${{ inputs.version }}
uses: akashchi/deadsnakes-action@92417281055a5878a0450f240a5b95883eb2d7e2
with:
python-version: ${{ inputs.version }}
- if: ${{ runner.os == 'macOS' || runner.os == 'Windows' || (runner.os == 'Linux' && runner.arch != 'ARM64' && steps.check_python.outputs.installed == 'false' ) }}
- if: ${{ runner.os == 'macOS' || runner.os == 'Windows' || (runner.os == 'Linux' && runner.arch != 'ARM64') }}
name: Setup Python ${{ inputs.version }}
uses: actions/setup-python@v5
with:

View File

@ -51,9 +51,6 @@ outputs:
affected_components:
description: "Affected components to run validation for and their validation scope"
value: ${{ steps.smart_ci.outputs.affected_components }}
changed_components:
description: "Actually changed components (for push events everything is marked as changed)"
value: ${{ steps.smart_ci.outputs.changed_components }}
skip_workflow:
description: "Whether the workflow should be run with Smart CI rules applied or skipped completely"
value: ${{ steps.smart_ci.outputs.skip_workflow }}

View File

@ -1,6 +1,3 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import os
import re
import argparse
@ -215,8 +212,8 @@ def main():
# In post-commits - validate all components regardless of changeset
# In pre-commits - validate only changed components with their dependencies
all_defined_components = components_config.keys()
changed_by_pr = get_changed_component_names(pr, all_possible_components, args.pattern) if pr else None
changed_component_names = set(all_defined_components) if run_full_scope else changed_by_pr
changed_component_names = set(all_defined_components) if run_full_scope else \
get_changed_component_names(pr, all_possible_components, args.pattern)
logger.info(f"changed_component_names: {changed_component_names}")
@ -247,12 +244,6 @@ def main():
affected_components_output = {name: {s: True for s in scope} for name, scope in affected_components.items()}
set_github_output("affected_components", json.dumps(affected_components_output))
# Components actually changed by a pull request are marked as True (if event is PR;
# otherwise all components considered changed).
changed_components_output = {name: True if not pr or name in changed_by_pr else False
for name in all_possible_components}
set_github_output("changed_components", json.dumps(changed_components_output))
if __name__ == '__main__':
main()

View File

@ -1,6 +1,3 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import sys
import unittest

View File

@ -27,13 +27,13 @@ CPU:
revalidate:
- C_API
- Python_API
- JS_API
- samples
- ONNX_RT
- PyTorch_FE
- TF_FE
- ONNX_FE
build:
- AUTO
- HETERO
- AUTO_BATCH
- TEMPLATE
@ -41,31 +41,34 @@ CPU:
GPU:
build:
- AUTO
- HETERO
- AUTO_BATCH
- TEMPLATE
- IR_FE
- PROXY
NPU:
revalidate: []
build: []
HETERO:
revalidate:
- CPU
- GPU
- HETERO
- AUTO_BATCH
- TEMPLATE
- C_API
- Python_API
build:
- TEMPLATE
- IR_FE
AUTO_BATCH:
revalidate:
- CPU
- GPU
- HETERO
- AUTO_BATCH
- TEMPLATE
- C_API
- Python_API
build:
- TEMPLATE
- IR_FE
TEMPLATE:
@ -74,6 +77,7 @@ TEMPLATE:
- GPU
- HETERO
- AUTO_BATCH
- TEMPLATE
- AUTO
- C_API
- Python_API
@ -83,10 +87,11 @@ TEMPLATE:
AUTO:
revalidate:
- TEMPLATE
- AUTO
- C_API
- Python_API
build:
- TEMPLATE
- IR_FE
PROXY:
@ -99,6 +104,7 @@ IR_FE:
revalidate:
- C_API
- Python_API
- JS_API
- samples
build:
- CPU
@ -106,7 +112,6 @@ IR_FE:
ONNX_FE:
revalidate:
- MO
- ONNX_RT
build:
- CPU
- Python_API
@ -144,7 +149,6 @@ PyTorch_FE:
C_API:
build:
- CPU
- GPU
- HETERO
- AUTO_BATCH
- AUTO
@ -169,6 +173,8 @@ Python_API:
- PyTorch_FE
JS_API:
revalidate:
- samples
build:
- CPU
- IR_FE

View File

@ -10,13 +10,9 @@ allow-licenses:
- 'BlueOak-1.0.0'
- '0BSD'
- 'Python-2.0'
- 'LGPL-3.0'
fail-on-scopes:
- 'runtime'
- 'development'
- 'unknown'
license-check: true
vulnerability-check: true
allow-dependencies-licenses:
- 'pkg:pypi/PyGithub@2.2.0'
- 'pkg:pypi/psycopg2-binary'

View File

@ -1 +0,0 @@
pr-24878

View File

@ -1,73 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
# For Java API
default-jdk \
# Compiler
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-aarch64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,72 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,89 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/nvidia/cuda:11.8.0-runtime-ubuntu20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
wget \
git \
ca-certificates \
gpg-agent \
tzdata \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Install CUDA
RUN wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin && \
mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600 && \
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub && \
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
RUN apt update && apt install -y \
libcudnn8=8.9.4.*-1+cuda11.8 \
libcudnn8-dev=8.9.4.*-1+cuda11.8 \
libcudnn8-samples=8.9.4.*-1+cuda11.8 \
cuda-runtime-11-8 \
cuda-11-8 \
libcutensor1=1.6.1.5-1 \
libcutensor-dev=1.6.1.5-1 \
cuda-drivers=520.61.05-1 && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,73 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
libtbb2 \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
gcc-10 \
g++-10 \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set gcc-10 as a default compiler
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 30 && \
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 30
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,72 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
libtbb2 \
# Pythons
python3.8-dev \
python3.8-venv \
python3.8-distutils \
python3.11-dev \
python3.11-venv \
python3.11-distutils \
# For Java API
default-jdk \
# Compiler \
clang \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Set clang as a default compiler
RUN update-alternatives --install /usr/bin/cc cc /usr/bin/clang 100 && \
update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++ 100
# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"
RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}

View File

@ -1,51 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Python
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
&& \
rm -rf /var/lib/apt/lists/*
# Install openvino dependencies
ADD scripts/install_dependencies/install_openvino_dependencies.sh /install_openvino_dependencies.sh
RUN chmod +x /install_openvino_dependencies.sh && \
/install_openvino_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
ENV PIP_INSTALL_PATH=/venv/lib/python3.11/site-packages

View File

@ -1,52 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Python
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.8 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default instead of Python 3.8
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
ENV PIP_INSTALL_PATH=/venv/lib/python3.11/site-packages

View File

@ -1,52 +0,0 @@
FROM openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
USER root
# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf
ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"
RUN apt-get update && \
apt-get install software-properties-common && \
add-apt-repository --yes --no-update ppa:git-core/ppa && \
add-apt-repository --yes --no-update ppa:deadsnakes/ppa && \
apt-get update && \
apt-get install \
curl \
git \
ca-certificates \
gpg-agent \
tzdata \
# Python
python3.11-dev \
python3.11-venv \
python3.11-distutils \
libhdf5-dev \
&& \
rm -rf /var/lib/apt/lists/*
# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*
# Setup pip
ENV PIP_VERSION="24.0"
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
python3.11 get-pip.py --no-cache-dir pip==${PIP_VERSION} && \
rm -f get-pip.py
# Use Python 3.11 as default
# Using venv here 'cause other methods to switch the default Python on Ubuntu 20 break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"
ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
ENV PIP_INSTALL_PATH=/venv/lib/python3.11/site-packages

19
.github/labeler.yml vendored
View File

@ -71,9 +71,6 @@
- 'src/plugins/intel_gpu/**/*'
- 'thirdparty/ocl/**/*'
'category: NPU':
- 'src/plugins/intel_npu/**/*'
'category: HETERO':
- 'src/plugins/hetero/**/*'
@ -145,7 +142,7 @@
- 'tests/layer_tests/tensorflow2_keras_tests/**/*'
- 'tests/layer_tests/jax_tests/**/*'
- any: ['tests/model_hub_tests/**',
'!tests/model_hub_tests/pytorch/**/*']
'!tests/model_hub_tests/torch_tests/**/*']
'category: TFL FE':
- 'src/frontends/tensorflow_lite/**/*'
@ -159,7 +156,7 @@
- 'tests/layer_tests/py_frontend_tests/test_torch_decoder.py'
- 'tests/layer_tests/py_frontend_tests/test_torch_frontend.py'
- any: ['tests/model_hub_tests/**',
'!tests/model_hub_tests/tensorflow/**/*']
'!tests/model_hub_tests/tf_hub_tests/**/*']
'category: tools':
- any: ['tools/**',
@ -169,19 +166,7 @@
'category: transformations':
- 'src/common/transformations/**/*'
- 'src/common/offline_transformations/**/*'
- 'src/core/pattern/**/*'
- 'src/core/pass/**/*'
- 'src/core/include/openvino/pass/**/*'
'category: licensing':
- 'licensing/**/*'
- 'LICENSE'
'category: docker_env':
- '.dockerignore'
- 'install_build_dependencies.sh'
- '**/install_openvino_dependencies.sh'
- '.github/actions/handle_docker/**/*'
'category: dockerfiles':
- '.github/dockerfiles/**/*'

View File

@ -1,225 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#!/usr/bin/env python3
from github import Github
from psycopg2 import sql
import os
import logging
import psycopg2
import dateutil
import argparse
def init_logger():
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
logging.basicConfig(level=LOGLEVEL,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
def make_parser():
parser = argparse.ArgumentParser()
parser.add_argument('-r', '--repository-name', type=str, required=True,
help='Repository name in OWNER/REPOSITORY format')
parser.add_argument('--run-id', type=str, required=True,
help='Workflow Run ID')
return parser
def create_db_tables(conn, cur):
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_runs(
id SERIAL PRIMARY KEY,
run_id BIGINT,
html_url TEXT,
name VARCHAR(255),
run_started_at TIMESTAMP,
created_at TIMESTAMP,
updated_at TIMESTAMP,
triggering_actor_login VARCHAR(255),
conclusion VARCHAR(25),
event VARCHAR(50),
run_attempt INT,
repository_full_name VARCHAR(255),
head_repository_full_name VARCHAR(255),
head_branch VARCHAR(255),
status VARCHAR(25),
display_title TEXT,
path TEXT,
total_duration_seconds INT
);
''')
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_jobs(
id SERIAL PRIMARY KEY,
job_id BIGINT,
parent_run_id BIGINT,
html_url TEXT,
name VARCHAR(255),
created_at TIMESTAMP,
started_at TIMESTAMP,
completed_at TIMESTAMP,
queued_duration_seconds INT,
duration_seconds INT,
runner_name VARCHAR(255),
status VARCHAR(25),
conclusion VARCHAR(25),
head_branch VARCHAR(255),
run_attempt INT,
workflow_name TEXT
);
''')
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_steps(
id SERIAL PRIMARY KEY,
parent_job_id BIGINT,
name VARCHAR(255),
conclusion VARCHAR(25),
number INT,
started_at TIMESTAMP,
completed_at TIMESTAMP,
duration_seconds INT
);
''')
conn.commit()
def main():
init_logger()
parser = make_parser()
args = parser.parse_args()
logger = logging.getLogger(__name__)
github_token = os.environ.get('GITHUB_TOKEN')
if not github_token:
raise ValueError('GITHUB_TOKEN environment variable is not set!')
run_id = args.run_id
repo_name = args.repository_name
# this should be specified in runner's env
db_username = os.environ.get('PGUSER')
db_password = os.environ.get('PGPASSWORD')
db_host = os.environ.get('PGHOST')
db_database = os.environ.get('PGDATABASE')
db_port = os.environ.get('PGPORT')
conn = psycopg2.connect(host=db_host,
port=db_port,
user=db_username,
password=db_password,
database=db_database)
# Create tables
cur = conn.cursor()
create_db_tables(conn, cur)
# Get the data
g = Github(github_token)
repo = g.get_repo(repo_name)
run = repo.get_workflow_run(int(run_id))
logger.info('Processing run ID %s - %s, URL: %s', run_id, run.name, run.html_url)
if run.status != 'completed':
logger.error('Run %s is not completed! Only completed runs should be in the database', run_id)
raise SystemExit(1)
# We rely on the following assumptions:
# - The workflow run is completed. When run.status != 'completed' we should not add it to the database
# theoretically the second attempt can be triggerred right after the completion of the first one
# or while the runner which executes this script is deploying
#
# - Job's queued duration equals "job.started_at - job.created_at" if started_at > created_at.
# Otherwise the job should not be added to the database
total_duration_seconds = round(run.timing().run_duration_ms / 1000)
workflow_data_query = sql.SQL('''INSERT INTO workflow_runs(
run_id, html_url, name,
run_started_at, created_at, updated_at, triggering_actor_login, conclusion,
event, run_attempt, repository_full_name, head_repository_full_name,
head_branch, status, display_title, path, total_duration_seconds)
VALUES(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);
''')
workflow_query_args = (run_id, run.html_url, run.name,
run.run_started_at, run.created_at, run.updated_at,
run.raw_data['triggering_actor']['login'],
run.conclusion, run.event,
run.run_attempt, run.raw_data['repository']['full_name'],
run.raw_data['head_repository']['full_name'],
run.head_branch, run.status,
run.display_title, run.path, total_duration_seconds)
logger.debug('Workflow run query: %s', cur.mogrify(workflow_data_query, workflow_query_args))
cur.execute(workflow_data_query, workflow_query_args)
for job in run.jobs():
job_id = job.id
logger.info('Processing job %s, URL: %s', job.name, job.html_url)
queued_duration_seconds = 0
duration_seconds = 0
job_created_at_date = dateutil.parser.parse(job.raw_data['created_at'])
if job_created_at_date > job.started_at:
logger.warning('Skipping job %s of run %s - most likely a stub \
job created after workflow restart', job.name, run_id)
continue
queued_duration_timedelta = job.started_at - job_created_at_date
queued_duration_seconds = round(queued_duration_timedelta.total_seconds())
duration_timedelta = job.completed_at - job.started_at
duration_seconds = round(duration_timedelta.total_seconds())
job_data_query = sql.SQL('''
INSERT INTO workflow_jobs(
job_id, parent_run_id, html_url, name,
created_at, started_at, completed_at,
queued_duration_seconds, duration_seconds,
runner_name, status, conclusion, head_branch,
run_attempt, workflow_name
)
VALUES(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);
''')
job_query_args = (job_id, run_id, job.html_url, job.name,
job.raw_data['created_at'], job.started_at, job.completed_at,
queued_duration_seconds, duration_seconds,
job.raw_data['runner_name'], job.status, job.conclusion,
job.raw_data['head_branch'], job.raw_data['run_attempt'], job.raw_data['workflow_name'])
logger.debug('Job query: %s', cur.mogrify(job_data_query, job_query_args))
cur.execute(job_data_query, job_query_args)
conn.commit()
for step in job.steps:
logger.info('Processing step %s', step.name)
# It seems like a GitHub bug, but
# sometimes there're steps "In progress"
# despite the workflow being completed
step_duration_seconds = 0
if step.status != 'completed':
logger.warning('Step %s isn\'t completed. Setting duration to zero.', step.name)
step_duration_seconds = 0
else:
step_duration_seconds_timedelta = step.completed_at - step.started_at
step_duration_seconds = round(step_duration_seconds_timedelta.total_seconds())
step_data_query = sql.SQL('''
INSERT INTO workflow_steps(
parent_job_id, name, conclusion,
number, started_at, completed_at,
duration_seconds)
VALUES (%s, %s, %s, %s, %s, %s, %s);
''')
step_query_args = (
job_id, step.name, step.conclusion,
step.number, step.started_at, step.completed_at,
step_duration_seconds)
logger.debug('Step query: %s', cur.mogrify(step_data_query, step_query_args))
cur.execute(step_data_query, step_query_args)
conn.commit()
cur.close()
conn.close()
g.close()
if __name__ == "__main__":
main()

View File

@ -1,75 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
import logging
import os
from enum import Enum
from github import Github, Auth
class ExternalPRLabels(str, Enum):
ExternalPR = 'ExternalPR'
ExternalIntelPR = 'ExternalIntelPR'
def get_arguments() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument(
'-r',
'--repository-name',
type=str,
required=True,
help='Repository name in the OWNER/REPOSITORY format',
)
parser.add_argument(
'--pr-number', type=int, required=True, help='PR number to label'
)
return parser.parse_args()
def init_logger():
logging.basicConfig(
level=os.environ.get('LOGLEVEL', 'INFO').upper(),
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S',
)
if __name__ == '__main__':
init_logger()
LOGGER = logging.getLogger('labeller')
args = get_arguments()
pr_number = args.pr_number
repository_name = args.repository_name
github = Github(auth=Auth.Token(token=os.environ.get('GITHUB_TOKEN')))
gh_repo = github.get_repo(full_name_or_id=repository_name)
pr = gh_repo.get_pull(number=pr_number)
LOGGER.info(f'CONTEXT: PR #{pr_number}. USER: {pr.user.login}. ALL PR LABELS: {list(pr.get_labels())}')
if not gh_repo.has_in_collaborators(pr.user.login):
LOGGER.info(f'THE {pr.user.login} IS NOT A COLLABORATOR')
for label in pr.get_labels():
if label.name in (ExternalPRLabels.ExternalPR, ExternalPRLabels.ExternalIntelPR):
LOGGER.info(f'THE PR ALREADY HAS THE "{label.name}" LABEL')
break
else:
is_intel_user = bool(pr.user.email and pr.user.email.endswith('@intel.com'))
label_to_add: str = ExternalPRLabels.ExternalIntelPR.name if is_intel_user else ExternalPRLabels.ExternalPR.name
pr.add_to_labels(label_to_add)
LOGGER.info(f'THE "{label_to_add}" LABEL WAS ADDED TO THE PR')
else:
LOGGER.info(
f'THE {pr.user.login} IS A COLLABORATOR, NO NEED TO ADD THE "External" LABEL'
)
github.close()

View File

@ -1 +0,0 @@
python-dateutil==2.9.0.post0

View File

@ -1,28 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
from pathlib import Path
def get_arguments() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument('-r', '--repository-name',
type=str,
required=True,
help='Repository name in the OWNER/REPOSITORY format')
parser.add_argument('--run-id',
type=int,
required=True,
help='Workflow Run ID')
parser.add_argument('--errors-to-look-for-file',
type=Path,
required=False,
help='.json file with the errors to look for in logs',
default=Path(__file__).resolve().parent.joinpath('errors_to_look_for.json'))
parser.add_argument('--dry-run',
required=False,
action='store_true',
help='Whether to run in dry mode and not actually retrigger the pipeline'
' and only collect and analyze logs')
return parser.parse_args()

View File

@ -1,20 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import os
GITHUB_TOKEN = os.environ.get('GITHUB_TOKEN')
def init_logger():
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
logging.basicConfig(level=LOGLEVEL,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d-%Y %H:%M:%S')
init_logger()
LOGGER = logging.getLogger('rerunner')

View File

@ -1,62 +0,0 @@
[
{
"error_text": "This is a problem related to network connectivity",
"ticket": 135929
},
{
"error_text": "Unable to make request",
"ticket": 135715
},
{
"error_text": "GnuTLS recv error",
"ticket": 131918
},
{
"error_text": "Connection was reset",
"ticket": 131818
},
{
"error_text": "Failed to connect to github.com",
"ticket": 131657
},
{
"error_text": "Could not resolve host: github.com",
"ticket": 131546
},
{
"error_text": "retrieving gpg key timed out",
"ticket": 131538
},
{
"error_text": "Retry limit has been reached for chunk",
"ticket": 131537
},
{
"error_text": "fatal error: downloading",
"ticket": 131424
},
{
"error_text": "Failure when receiving data from the peer",
"ticket": 137121
},
{
"error_text": "Network is unreachable",
"ticket": 130955
},
{
"error_text": "connection timed out",
"ticket": 130955
},
{
"error_text": "The requested URL returned error: 500",
"ticket": 139384
},
{
"error_text": "Unable to fetch some archives",
"ticket": 130965
},
{
"error_text": "status_string: \"Timeout was reached\"",
"ticket": 142653
}
]

View File

@ -1,137 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import json
import re
import tempfile
from pathlib import Path
from typing import TypedDict
from zipfile import ZipFile
from workflow_rerun.constants import LOGGER
class LogFile(TypedDict):
file_name: str
path: Path
class ErrorData(TypedDict):
error_text: str
ticket: int
class LogAnalyzer:
def __init__(self,
path_to_log_archive: Path,
path_to_errors_file: Path) -> None:
self._path_to_log_archive = path_to_log_archive
self._path_to_errors_file = path_to_errors_file
self._errors_to_look_for: list[ErrorData] = []
self._collect_errors_to_look_for()
self._log_dir = tempfile.TemporaryDirectory().name
self._log_files: list[LogFile] = []
self._collect_log_files()
all_txt_log_files_pretty = '\n'.join(map(lambda item: str(item['path']), self._log_files))
LOGGER.info(f'ALL .txt LOG FILES: \n{all_txt_log_files_pretty}')
self.found_matching_error = False
self.found_error_ticket = None
def _collect_errors_to_look_for(self) -> None:
with open(file=self._path_to_errors_file,
mode='r',
encoding='utf-8') as errors_file:
errors_data = json.load(errors_file)
for error_data in errors_data:
self._errors_to_look_for.append(
ErrorData(error_text=error_data['error_text'],
ticket=error_data['ticket'])
)
def _collect_log_files(self) -> None:
"""
Collects the .txt log files from the log archive
The GitHub Actions pipeline logs archive should have the following structure:
> Job_name_0
> step_name_0.txt
> step_name_1.txt
...
> Job_name_1
> step_name_0.txt
> step_name_1.txt
...
> Job_name_2
...
...
We need to only analyze the `*.txt` files
"""
with ZipFile(file=self._path_to_log_archive,
mode='r') as zip_file:
zip_file.extractall(self._log_dir)
for _file in Path(self._log_dir).iterdir():
if _file.is_dir():
for log_file in _file.iterdir():
self._log_files.append(LogFile(file_name=log_file.name,
path=log_file.resolve()))
def _is_error_in_log(self,
error_to_look_for: str,
log_file_path: Path) -> bool:
"""
Searches for the error in the provided log
"""
error_to_look_for = self._clean_up_string(error_to_look_for)
with open(file=log_file_path,
mode='r',
encoding='utf-8') as log_file:
for line in log_file:
if error_to_look_for in self._clean_up_string(line):
return True
return False
@staticmethod
def _clean_up_string(string: str) -> str:
"""
Replaces special characters with spaces in the string, strips it from leading and following spaces,
and lowers it
for "Could not resolve host: github.com" returns "could not resolve host github com"
This cleanup is applied to both errors to look for and logs themselves for matching
"""
return re.sub(r'[^A-Za-z0-9]+', ' ', string).lower().strip()
def analyze(self) -> None:
"""
Iterates over the known errors and tries to find them in the collected log files
"""
for error in self._errors_to_look_for:
LOGGER.info(f'LOOKING FOR "{error["error_text"]}" ERROR...')
for log_file in self._log_files:
if self._is_error_in_log(error_to_look_for=error['error_text'],
log_file_path=log_file['path']):
LOGGER.info(f'FOUND "{error["error_text"]}" ERROR IN {log_file["path"]}. TICKET: {error["ticket"]}')
self.found_matching_error = True
self.found_error_ticket = error['ticket']
return
if __name__ == '__main__':
# Usage example
log_analyzer = LogAnalyzer(path_to_log_archive=Path('/tmp/logs/log.zip'),
path_to_errors_file=Path('/tmp/errors_to_look_for.json'))
log_analyzer.analyze()
if log_analyzer.found_matching_error:
print('found matching error, see logs above')

View File

@ -1,24 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
from pathlib import Path
import requests
from github.WorkflowRun import WorkflowRun
from workflow_rerun.constants import GITHUB_TOKEN, LOGGER
def collect_logs_for_run(run: WorkflowRun,
log_archive_path: Path) -> Path:
"""
Collects log archive for a pipeline
"""
with open(file=log_archive_path,
mode='wb') as log_archive:
LOGGER.info(f'STARTED LOG COLLECTION FOR {run.id} IN {log_archive_path}')
# PyGitHub does not expose the "/repos/{owner}/{repo}/actions/runs/{run_id}/logs" endpoint so we have to use requests
log_archive.write(requests.get(url=run.logs_url,
headers={'Authorization': f'Bearer {GITHUB_TOKEN}'}).content)
LOGGER.info(f'COLLECTED LOGS FOR {run.id} IN {log_archive_path}')
return log_archive_path

View File

@ -1,71 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import os
import sys
import tempfile
from pathlib import Path
from github import Github, Auth
from workflow_rerun.argument_parser import get_arguments
from workflow_rerun.constants import GITHUB_TOKEN, LOGGER
from workflow_rerun.log_analyzer import LogAnalyzer
from workflow_rerun.log_collector import collect_logs_for_run
if __name__ == '__main__':
args = get_arguments()
run_id = args.run_id
repository_name = args.repository_name
errors_file = args.errors_to_look_for_file
is_dry_run = args.dry_run
if is_dry_run:
LOGGER.info('RUNNING IN DRY RUN MODE. IF ERROR WILL BE FOUND, WILL NOT RETRIGGER')
github = Github(auth=Auth.Token(token=GITHUB_TOKEN))
gh_repo = github.get_repo(full_name_or_id=repository_name)
run = gh_repo.get_workflow_run(id_=run_id)
LOGGER.info(f'CHECKING IF RERUN IS NEEDED FOR {run.html_url} RUN IN {repository_name}.')
# Check if the run has already been retriggered
# we do not want to fall into a loop with retriggers
if run.run_attempt > 1:
LOGGER.info(f'THERE ARE {run.run_attempt} ATTEMPTS ALREADY. NOT CHECKING LOGS AND NOT RETRIGGERING. EXITING')
sys.exit(0)
log_archive_path = Path(tempfile.NamedTemporaryFile(suffix='.zip').name)
collect_logs_for_run(
run=run,
log_archive_path=log_archive_path,
)
log_analyzer = LogAnalyzer(
path_to_log_archive=log_archive_path,
path_to_errors_file=errors_file
)
log_analyzer.analyze()
if log_analyzer.found_matching_error:
LOGGER.info(f'FOUND MATCHING ERROR, RETRIGGERING {run.html_url}')
if is_dry_run:
LOGGER.info(f'RUNNING IN DRY RUN MODE, NOT RETRIGGERING, EXITING')
sys.exit(0)
status = run.rerun()
if status:
LOGGER.info(f'RUN RETRIGGERED SUCCESSFULLY: {run.html_url}')
else:
LOGGER.info(f'RUN WAS NOT RETRIGGERED, SEE ABOVE')
# Needed to run a step after for statistics
with open(file=os.environ['GITHUB_ENV'],
mode='a') as fh:
fh.write('PIPELINE_RETRIGGERED=true\n')
fh.write(f'FOUND_ERROR_TICKET={log_analyzer.found_error_ticket}\n')
# "status" is True (which is 1) if everything is ok, False (which is 0) otherwise
sys.exit(not status)
else:
LOGGER.info(f'NO ERROR WAS FOUND, NOT RETRIGGERING')

View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:894d636bcf156a7f3fae09f3c1d61df6b3db89117a917a3079995805c29115b3
size 89247

View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3f094a737d7ea40dba8d3fb13493275cae243d08e5f1dabce90c316c951a6ac2
size 52047

View File

@ -1,55 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Integration tests
"""
import unittest
from pathlib import Path
from github import Github, Auth
import os
import tempfile
from workflow_rerun.log_analyzer import LogAnalyzer
from workflow_rerun.log_collector import collect_logs_for_run
class IntegrationTest(unittest.TestCase):
"""
A class for testing integration between LogAnalyzer and log_collection
"""
def setUp(self) -> None:
print(f'\nIn test: "{self._testMethodName}"', flush=True)
self._cwd = Path(__file__).parent
self.errors_to_look_for_file = self._cwd.parent.joinpath(
'errors_to_look_for.json'
)
self.github = Github(auth=Auth.Token(token=os.environ.get('GITHUB_TOKEN')))
self.gh_repo = self.github.get_repo(full_name_or_id='openvinotoolkit/openvino')
# Even if we use "failure" for status we cannot guarantee logs containing any of the known error
# So these tests use the logs of the most recent successfull pipeline
self.wf_run = self.gh_repo.get_workflow_runs(status='success')[0]
print(f'Workflow run for testing: {self.wf_run}', flush=True)
def test_log_collection_and_analysis(self) -> None:
"""
Ensure logs collected by collect_logs_for_run are analyzed by LogAnalyzer
"""
log_archive_path = Path(tempfile.NamedTemporaryFile(suffix='.zip').name)
collect_logs_for_run(run=self.wf_run,
log_archive_path=log_archive_path)
analyzer = LogAnalyzer(
path_to_log_archive=log_archive_path,
path_to_errors_file=self.errors_to_look_for_file,
)
analyzer.analyze()
self.assertFalse(analyzer.found_matching_error)
def tearDown(self) -> None:
self.github.close()

View File

@ -1,104 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
LogAnalyzer tests
"""
import unittest
from pathlib import Path
from workflow_rerun.log_analyzer import LogAnalyzer
class LogAnalyzerTest(unittest.TestCase):
"""
A class for testing LogAnalyzer
"""
def setUp(self) -> None:
print(f'\nIn test: "{self._testMethodName}"', flush=True)
self._cwd = Path(__file__).parent
self.log_archive_with_error = self._cwd.joinpath("data").joinpath(
'log_archive_with_error.zip'
)
self.log_archive_wo_error = self._cwd.joinpath("data").joinpath(
'log_archive_wo_error.zip'
)
self.errors_to_look_for_file = self._cwd.parent.joinpath(
'errors_to_look_for.json'
)
def test_log_analyzer_instantiation(self) -> None:
"""
Ensure LogAnalyzer is instantiated correctly.
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_wo_error,
path_to_errors_file=self.errors_to_look_for_file,
)
self.assertTrue(
hasattr(analyzer, '_errors_to_look_for'),
'Analyzer should have _errors_to_look_for',
)
self.assertTrue(
hasattr(analyzer, '_log_files'), 'Analyzer should have _log_files'
)
for error_data in analyzer._errors_to_look_for:
self.assertTrue(
error_data['error_text'], 'Each error_data should have text'
)
self.assertTrue(error_data['ticket'], 'Each error_data should have ticket')
for log_file in analyzer._log_files:
self.assertTrue(
log_file['file_name'], 'Each log_file should have file_name'
)
self.assertTrue(log_file['path'], 'Each log_file should have path')
def test_string_cleanup(self) -> None:
"""
Ensure log cleanup function returns correct results
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_wo_error,
path_to_errors_file=self.errors_to_look_for_file,
)
data = (
'Connection was reset',
'Failed to connect to github.com',
'Could not resolve host: github.com',
)
expected = (
'connection was reset',
'failed to connect to github com',
'could not resolve host github com',
)
for input_str, expected_str in zip(data, expected):
self.assertEqual(analyzer._clean_up_string(string=input_str), expected_str)
def test_analyzer_with_error(self) -> None:
"""
Ensure LogAnalyzer can find an error
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_with_error,
path_to_errors_file=self.errors_to_look_for_file,
)
analyzer.analyze()
self.assertTrue(analyzer.found_matching_error)
def test_analyzer_wo_error(self) -> None:
"""
Ensure LogAnalyzer does not find an error in the log files w/o errors
"""
analyzer = LogAnalyzer(
path_to_log_archive=self.log_archive_wo_error,
path_to_errors_file=self.errors_to_look_for_file,
)
analyzer.analyze()
self.assertFalse(analyzer.found_matching_error)

View File

@ -1,41 +0,0 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
log collector tests
"""
import os
import unittest
import tempfile
from pathlib import Path
from github import Github, Auth
from workflow_rerun.log_collector import collect_logs_for_run
class LogCollectorTest(unittest.TestCase):
"""
A class for testing log collection
"""
def setUp(self) -> None:
print(f'\nIn test: "{self._testMethodName}"', flush=True)
self._cwd = Path(__file__).parent
self.github = Github(auth=Auth.Token(token=os.environ.get('GITHUB_TOKEN')))
self.gh_repo = self.github.get_repo(full_name_or_id='openvinotoolkit/openvino')
# Use the logs of the most recent successfull pipeline
self.wf_run = self.gh_repo.get_workflow_runs(status='success')[0]
print(f'Workflow run for testing: {self.wf_run}', flush=True)
def test_log_collection(self) -> None:
"""
Ensure log collection is working
"""
log_archive_path = Path(tempfile.NamedTemporaryFile(suffix='.zip').name)
collect_logs_for_run(run=self.wf_run, log_archive_path=log_archive_path)
self.assertTrue(Path(log_archive_path).exists())
def tearDown(self) -> None:
self.github.close()

View File

@ -13,8 +13,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-android-arm64-vcpkg
cancel-in-progress: true
permissions: read-all
jobs:
Smart_CI:
runs-on: ubuntu-latest
@ -23,7 +21,7 @@ jobs:
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: .github/actions/smart-ci
@ -77,7 +75,7 @@ jobs:
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: 'openvino'
@ -89,10 +87,11 @@ jobs:
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/open_model_zoo
popd
- name: Clone vcpkg
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'microsoft/vcpkg'
# Keep in sync with <root>/vcpkg.json <builtin-baseline>
@ -117,13 +116,13 @@ jobs:
apt --assume-yes install ccache scons ninja-build build-essential python3-pip
# vcpkg requires cmake 3.19 or later
python3 -m pip install -U pip cmake~=3.28.0
python3 -m pip install -U pip cmake
# vcpkg's tool dependencies
apt --assume-yes install curl zip unzip tar
# vcpkg 'python3' port dependencies
apt --assume-yes install autoconf libtool autoconf-archive
# vcpkg tree of dependencies require extra packages
apt --assume-yes install pkgconf linux-libc-dev
apt --assume-yes install pkg-config linux-libc-dev
# Install Android SDK, NDK and Tools
apt -y --no-install-recommends install unzip wget default-jdk
@ -132,7 +131,7 @@ jobs:
echo "yes" | ./cmdline-tools/bin/sdkmanager --sdk_root=${ANDROID_TOOLS} --install "ndk-bundle" "platform-tools" "platforms;android-${{ env.ANDROID_SDK_VERSION }}"
- name: Install sccache
uses: mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd # v0.0.4
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
@ -184,7 +183,7 @@ jobs:
# Upload build logs
#
- name: Upload build logs
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: always()
with:
name: build_logs

View File

@ -6,8 +6,6 @@ on:
- created
- edited
permissions: read-all
jobs:
take-issue:
name: Take issue
@ -17,7 +15,7 @@ jobs:
timeout-minutes: 10
steps:
- name: take an issue
uses: bdougie/take-action@1439165ac45a7461c2d89a59952cd7d941964b87 # v1.6.1
uses: bdougie/take-action@v1.6.1
with:
message: Thank you for looking into this issue! Please let us know if you have any questions or require any help.
issueCurrentlyAssignedMessage: Thanks for being interested in this issue. It looks like this ticket is already assigned to a contributor. Please communicate with the assigned contributor to confirm the status of the issue.

View File

@ -10,25 +10,23 @@ concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
permissions: read-all
jobs:
Build_Doc:
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
submodules: 'true'
lfs: 'true'
- name: Install apt-get dependencies
uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
uses: awalsh128/cache-apt-pkgs-action@v1.4.1
with:
packages: graphviz texlive liblua5.2-0 libclang1-9 libclang-cpp9
version: 3.0
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
- uses: actions/setup-python@v5
id: cp310
with:
python-version: '3.10'
@ -41,7 +39,6 @@ jobs:
run: |
python3 -m pip install -r docs/requirements.txt
(cd docs/openvino_sphinx_theme && python3 setup.py install)
python3 -m pip install docs/openvino_custom_sphinx_sitemap
- name: Download and install doxygen
run: |
@ -53,12 +50,12 @@ jobs:
- name: Build docs
run: |
rm -rf build && mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=ON
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=ON -DENABLE_CPP_API=ON
cmake --build . --target sphinx_docs
- name: Cache documentation
id: cache_sphinx_docs
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
uses: actions/cache@v4
with:
path: build/docs/_build/.doctrees
key: sphinx-docs-cache
@ -71,31 +68,39 @@ jobs:
PR_NUMBER=$(echo $GITHUB_REF | awk 'BEGIN { FS = "/" } ; { print $3 }')
echo "PR_NUMBER=$PR_NUMBER" >> $GITHUB_ENV
- name: 'Upload doxygen.log'
uses: actions/upload-artifact@v3
with:
name: doxygen_build_log_${{ env.PR_NUMBER }}.log
path: build/docs/doxygen.log
- name: 'Upload sphinx.log'
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: sphinx_build_log_${{ env.PR_NUMBER }}.log
path: build/docs/sphinx.log
- name: 'Upload docs html'
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_docs_html_${{ env.PR_NUMBER }}.zip
path: build/docs/openvino_docs_html.zip
- name: Run Pytest
run: |
pytest --sphinx="./build/docs/sphinx.log" \
pytest --doxygen="./build/docs/doxygen.log" \
--sphinx="./build/docs/sphinx.log" \
--suppress-warnings="./docs/scripts/tests/suppress_warnings.txt" \
--confcutdir="./docs/scripts/tests/" \
--html="./build/docs/_artifacts/doc-generation.html" \
--doxygen-strip="$(pwd)" \
--sphinx-strip="$(pwd)/build/docs/sphinx_source" \
--xfail="./docs/scripts/tests/xfail.txt" \
--self-contained-html ./docs/scripts/tests/test_docs.py
- name: 'Upload test results'
if: failure()
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_docs_pytest
path: build/docs/_artifacts/

View File

@ -1,14 +1,12 @@
name: PR Commits
on: [pull_request]
permissions: read-all
jobs:
Checks:
runs-on: ubuntu-22.04
steps:
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
- name: Install dependencies
run: python3 -m pip install -r ./.github/github_org_control/requirements.txt

View File

@ -1,14 +1,12 @@
name: Cleanup caches
name: Cleanup PIP caches
on:
workflow_dispatch:
schedule:
# at 00:00 on the 1st day of every month
- cron: '0 0 1 * *'
permissions: read-all
jobs:
Cleanup_PIP:
Cleanup_PIP_Caches:
runs-on: aks-linux-2-cores-8gb
container:
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
@ -32,26 +30,3 @@ jobs:
run: |
echo "Cache info: "
du -h -d2 ${PIP_CACHE_PATH}
Cleanup_CCACHE:
runs-on: aks-linux-2-cores-8gb
container:
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount:/mount
env:
CCACHE_PATH: /mount/caches/ccache/ubuntu22_riscv64
steps:
- name: Checkout cach action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
sparse-checkout: .github/actions/cache
- name: Cleanup ccache
uses: ./.github/actions/cache/cleanup
with:
cache-size: 10
max-cache-size: 50
cache-path: ${{ env.CCACHE_PATH }}
key: '.'

View File

@ -16,8 +16,6 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: read-all
jobs:
Build:
strategy:
@ -27,23 +25,23 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
submodules: 'true'
- name: Install OpenCL
uses: awalsh128/cache-apt-pkgs-action@a6c3917cc929dd0345bfb2d3feaf9101823370ad # v1.4.2
uses: awalsh128/cache-apt-pkgs-action@v1.4.1
if: runner.os == 'Linux'
with:
packages: ocl-icd-opencl-dev opencl-headers
version: 3.0
- name: Install scons
if: runner.os == 'macOS'
run: brew install scons
- name: CMake configure
run: cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=SEQ -B build
- name: Get number of CPU cores
uses: SimenB/github-actions-cpu-cores@v2
id: cpu-cores
- name: Build snippets
run: cmake --build build --target openvino_docs_snippets --parallel
run: cmake --build build --target openvino_docs_snippets --parallel ${{ steps.cpu-cores.outputs.count }}

View File

@ -5,22 +5,25 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: read-all
jobs:
clang-format:
runs-on: ubuntu-22.04
runs-on: ubuntu-20.04
permissions:
pull-requests: write
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: Install clang-format-15
- name: Install clang-format-9
run: |
sudo apt update
sudo apt --assume-yes install clang-format-15
sudo apt --assume-yes install clang-format-9
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install -r ./src/bindings/python/requirements.txt
# Run cmake with -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT in order to enable codestyle check for ITT collector
- name: CMake configure
@ -31,7 +34,7 @@ jobs:
- name: suggester / clang-format
if: startsWith(github.event_name, 'pull_request')
uses: reviewdog/action-suggester@9e1cd88b79ba3c0023c94e44accd72344f032093 # v1.13.0
uses: reviewdog/action-suggester@v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
level: warning
@ -42,7 +45,7 @@ jobs:
permissions:
pull-requests: write
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@v4
with:
submodules: 'true'
@ -60,7 +63,7 @@ jobs:
# always provide suggestions even for skipped scripts in ov_shellcheck tagret
- name: ShellCheck action
if: always()
uses: reviewdog/action-shellcheck@6e3a862f231c6895fbd335b70adef8f9243d5762 # v1.21.0
uses: reviewdog/action-shellcheck@v1
with:
level: style
reporter: github-pr-review
@ -73,7 +76,7 @@ jobs:
NamingConventionCheck:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@v4
with:
submodules: 'true'

View File

@ -5,8 +5,6 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: read-all
jobs:
Coverage:
runs-on: ${{ matrix.config.os }}
@ -18,19 +16,19 @@ jobs:
steps:
- name: Setup python
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
uses: actions/setup-python@v5
with:
python-version: '3.10.10'
architecture: 'x64'
- name: Setup ccache
uses: hendrikmuhs/ccache-action@c92f40bee50034e84c763e33b317c77adaa81c92 # v1.2.13
uses: hendrikmuhs/ccache-action@v1.2
with:
max-size: 50G
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
submodules: 'true'
@ -42,6 +40,7 @@ jobs:
python3 -m pip install --upgrade pip
python3 -m pip install -r ${{ github.workspace }}/src/bindings/python/wheel/requirements-dev.txt
python3 -m pip install -r ${{ github.workspace }}/src/bindings/python/requirements.txt
# For running Paddle frontend unit tests
python3 -m pip install -r ${{ github.workspace }}/src/frontends/paddle/tests/requirements.txt
# For running ONNX frontend unit tests
@ -56,8 +55,12 @@ jobs:
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_tf2.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_dev.txt
- name: Get number of CPU cores
uses: SimenB/github-actions-cpu-cores@v2
id: cpu-cores
- name: Build OpenVINO with CMake
uses: ashutoshvarma/action-cmake-build@ade188313bc7eaa6f14349569a64d8bc716342ff # master
uses: ashutoshvarma/action-cmake-build@master
with:
build-dir: ${{ github.workspace }}/build
cc: ${{ matrix.config.cc }}
@ -79,6 +82,7 @@ jobs:
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DENABLE_SYSTEM_SNAPPY=ON
build-type: Release
parallel: ${{ steps.cpu-cores.outputs.count }}
- name: Install wheel packages
run: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=${{ github.workspace }}/install_pkg -P '${{ github.workspace }}/build/cmake_install.cmake'
@ -99,7 +103,7 @@ jobs:
run: ${{ github.workspace }}/bin/intel64/Release/ov_proxy_plugin_tests
- name: Run OV Hetero Func tests
run: ${{ github.workspace }}/bin/intel64/Release/ov_hetero_func_tests --gtest_filter="*smoke*"
run: ${{ github.workspace }}/bin/intel64/Release/ov_hetero_func_tests
- name: Run IR frontend tests
run: ${{ github.workspace }}/bin/intel64/Release/ov_ir_frontend_tests
@ -114,7 +118,7 @@ jobs:
run: ${{ github.workspace }}/bin/intel64/Release/ov_tensorflow_frontend_tests --gtest_filter=-*IE_GPU*
- name: Build coverage with CMake
uses: ashutoshvarma/action-cmake-build@ade188313bc7eaa6f14349569a64d8bc716342ff # master
uses: ashutoshvarma/action-cmake-build@master
with:
build-dir: ${{ github.workspace }}/coverage
cc: ${{ matrix.config.cc }}
@ -126,6 +130,7 @@ jobs:
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
parallel: ${{ steps.cpu-cores.outputs.count }}
- name: Print info
@ -137,6 +142,6 @@ jobs:
lcov --capture --directory ${{ github.workspace }}/. --output-file coverage.info
genhtml coverage.info --output-directory coverage-report
- name: Collect coverage
uses: codecov/codecov-action@125fc84a9a348dbcf27191600683ec096ec9021c # v4.4.1
uses: codecov/codecov-action@v4
with:
verbose: true

View File

@ -14,8 +14,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux-coverity
cancel-in-progress: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
@ -29,14 +27,22 @@ jobs:
runs-on: aks-linux-16-cores-32gb
container:
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount/caches:/mount/caches
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_BUILD_TYPE: 'Release'
CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib
BUILD_DIR: /__w/openvino/openvino/openvino_build
SCCACHE_AZURE_KEY_PREFIX: coverity_ubuntu20_x86_64
COVERITY_TOOL_DIR: /__w/openvino/openvino/coverity_tool
steps:
@ -46,19 +52,19 @@ jobs:
apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
ref: ${{ inputs.openvinoRef }}
- name: Clone OpenVINO Contrib
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true'
ref: 'master'
ref: 'releases/2024/0'
#
# Dependencies
@ -70,6 +76,11 @@ jobs:
# default-jdk - Java API
apt install --assume-yes --no-install-recommends default-jdk
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
@ -88,11 +99,20 @@ jobs:
-G "${{ env.CMAKE_GENERATOR }}" \
-DENABLE_CPPLINT=OFF \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_SYSTEM_OPENCL=ON \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DENABLE_FASTER_BUILD=OFF \
-DCPACK_GENERATOR=TGZ \
-DBUILD_nvidia_plugin=OFF \
-DOPENVINO_EXTRA_MODULES=${OPENVINO_CONTRIB_REPO}/modules \
-DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
-DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
- name: Clean sccache stats
run: ${SCCACHE_PATH} --zero-stats
- name: Install Coverity tool
run: |
rm -rf ${COVERITY_TOOL_DIR} && mkdir -p ${COVERITY_TOOL_DIR}
@ -102,7 +122,12 @@ jobs:
popd
- name: Cmake build - OpenVINO with Coverity
run: ${COVERITY_TOOL_DIR}/cov-analysis*/bin/cov-build --dir ${BUILD_DIR}/cov-int cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
run: |
${COVERITY_TOOL_DIR}/cov-analysis*/bin/cov-build --dir ${BUILD_DIR}/cov-int \
cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
- name: Show sccache stats
run: ${SCCACHE_PATH} --show-stats
- name: Pack Artefacts
run: |
@ -112,44 +137,24 @@ jobs:
- name: Submit artefacts
run: |
apt-get update && apt-get install -y curl jq
apt-get update && apt-get install -y curl
pushd ${BUILD_DIR}
curl -X POST -d token=${{ secrets.COVERITY_TOKEN }} \
-d email=${{ secrets.COVERITY_USER }} \
-d file_name="openvino.tgz" \
-d version="${{ github.sha }}" \
-d description="https://github.com/openvinotoolkit/openvino/runs/${{ github.run_number }}" \
https://scan.coverity.com/projects/21921/builds/init | tee response
upload_url=$(jq -r '.url' response)
build_id=$(jq -r '.build_id' response)
curl -X PUT \
--header 'Content-Type: application/json' \
--upload-file openvino.tgz \
$upload_url
curl -X PUT \
-d token=${{ secrets.COVERITY_TOKEN }} \
https://scan.coverity.com/projects/21921/builds/$build_id/enqueue
curl --form token=${{ secrets.COVERITY_TOKEN }} \
--form email=${{ secrets.COVERITY_USER }} \
--form file=@openvino.tgz \
--form version="${{ github.sha }}" \
--form description="https://github.com/openvinotoolkit/openvino/runs/${{ github.run_number }}" \
https://scan.coverity.com/builds?project=openvino
popd
- name: Show Coverity configure logs
continue-on-error: true
run: ${COVERITY_TOOL_DIR}/cov-analysis*/bin/cov-configure -c ${COVERITY_TOOL_DIR}/cov-analysis-linux64-2023.6.2/config/coverity_config.xml -lscc text
- name: Upload Coverity build log
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
- name: Upload Coverity logs
uses: actions/upload-artifact@v3
if: always()
with:
name: coverity_logs
path: ${{ env.BUILD_DIR }}/cov-int/build-log.txt
if-no-files-found: 'error'
- name: Upload Coverity build archive
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
if: always()
with:
name: coverity_archive
path: ${{ env.BUILD_DIR }}/openvino.tgz
if-no-files-found: 'error'

View File

@ -1,18 +1,17 @@
name: 'Dependency Review'
on: [pull_request, merge_group]
on: [pull_request]
permissions: read-all
permissions:
contents: read
jobs:
dependency-review:
runs-on: ubuntu-latest
steps:
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
- name: Dependency Review
uses: actions/dependency-review-action@72eb03d02c7872a771aacd928f3123ac62ad6d3a # v4.3.3
uses: actions/dependency-review-action@v4
with:
config-file: './.github/dependency_review.yml'
base-ref: ${{ github.pull_request.base.sha || github.event.merge_group.base_ref }}
head-ref: ${{ github.pull_request.head.sha || github.ref }}

View File

@ -13,8 +13,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-fedora33
cancel-in-progress: true
permissions: read-all
jobs:
Smart_CI:
runs-on: ubuntu-latest
@ -23,7 +21,7 @@ jobs:
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: .github/actions/smart-ci
@ -73,7 +71,7 @@ jobs:
run: yum update -y && yum install -y git
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
@ -93,7 +91,7 @@ jobs:
run: bash ${OPENVINO_REPO}/install_build_dependencies.sh
- name: Install sccache
uses: mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd # v0.0.4
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
@ -127,6 +125,7 @@ jobs:
-DENABLE_NCC_STYLE=OFF \
-DENABLE_TESTS=ON \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_SYSTEM_OPENCL=ON \
-DENABLE_PYTHON_PACKAGING=ON \
-DCPACK_GENERATOR=TGZ \
@ -161,9 +160,7 @@ jobs:
- name: Build RPM packages
run: |
cmake -UTBB* \
-DCPACK_GENERATOR=RPM \
-DENABLE_SYSTEM_TBB=ON \
cmake -DCPACK_GENERATOR=RPM \
-DENABLE_TESTS=OFF \
${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel --target package --verbose
@ -172,7 +169,7 @@ jobs:
# Upload build artifacts and logs
#
- name: Upload build logs
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: always()
with:
name: build_logs
@ -181,7 +178,7 @@ jobs:
- name: Upload openvino package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_package
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
@ -189,7 +186,7 @@ jobs:
- name: Upload openvino RPM packages
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_rpm_packages
path: ${{ env.BUILD_DIR }}/*.rpm
@ -197,7 +194,7 @@ jobs:
- name: Upload openvino tests package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_tests
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
@ -217,7 +214,7 @@ jobs:
steps:
- name: Download OpenVINO RPM packages
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_rpm_packages
path: ${{ env.RPM_PACKAGES_DIR }}

View File

@ -5,13 +5,11 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: read-all
jobs:
Check_Files_Size:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@v4
- name: git ls-tree
run: git ls-tree -r -t -l --full-name HEAD | sort -n -r -k 4

View File

@ -13,12 +13,10 @@ on:
required: false
default: null
permissions: read-all
jobs:
CPU_Functional_Tests:
name: CPU functional tests
timeout-minutes: 30
timeout-minutes: 25
runs-on: ${{ inputs.runner }}
container:
image: ${{ inputs.image }}
@ -32,14 +30,17 @@ jobs:
PARALLEL_TEST_SCRIPT: ${{ github.workspace }}/install/tests/functional_test_utils/layer_tests_summary/run_parallel.py
PARALLEL_TEST_CACHE: ${{ github.workspace }}/install/tests/test_cache.lst
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -62,8 +63,12 @@ jobs:
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Install OpenVINO dependencies (Linux)
if: runner.os == 'Linux'
run: $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -c=gpu -y
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
@ -81,7 +86,7 @@ jobs:
run: python3 -m pip install -r ${INSTALL_TEST_DIR}/functional_test_utils/layer_tests_summary/requirements.txt
- name: Restore tests execution time
uses: actions/cache/restore@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
uses: actions/cache/restore@v4
with:
path: ${{ env.PARALLEL_TEST_CACHE }}
key: ${{ runner.os }}-${{ runner.arch }}-tests-functional-cpu-stamp-${{ github.sha }}
@ -94,21 +99,19 @@ jobs:
if [[ -f "${INSTALL_DIR}/setupvars.sh" ]]; then
source ${INSTALL_DIR}/setupvars.sh
fi
# Needed as ze_loader.so is under INSTALL_TEST_DIR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${INSTALL_TEST_DIR}
python3 ${PARALLEL_TEST_SCRIPT} -e ${INSTALL_TEST_DIR}/ov_cpu_func_tests -c ${PARALLEL_TEST_CACHE} -w ${INSTALL_TEST_DIR} -s suite -rf 0 -- --gtest_print_time=1 --gtest_filter=*smoke*
timeout-minutes: 25
- name: Save tests execution time
uses: actions/cache/save@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
uses: actions/cache/save@v4
if: github.ref_name == 'master'
with:
path: ${{ env.PARALLEL_TEST_CACHE }}
key: ${{ runner.os }}-${{ runner.arch }}-tests-functional-cpu-stamp-${{ github.sha }}
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: always()
with:
name: test-results-functional-cpu

View File

@ -17,8 +17,6 @@ on:
type: string
required: true
permissions: read-all
jobs:
CXX_Unit_Tests:
name: C++ unit tests
@ -34,14 +32,17 @@ jobs:
INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -61,6 +62,10 @@ jobs:
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Install OpenVINO dependencies (Linux)
if: runner.os == 'Linux'
run: $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -c=gpu -y
#
# Tests
#
@ -156,7 +161,6 @@ jobs:
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-Transformations.xml
- name: Common test utils tests
if: ${{ runner.os != 'macOS' }} # Ticket: 134469
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_util_tests --gtest_print_time=1 \
@ -210,13 +214,6 @@ jobs:
--gtest_filter=*smoke* \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-TemplateFuncTests.xml
- name: OV utils unit tests
if: ${{ runner.os != 'macOS' }} # Ticket: 134469
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_util_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_util_tests.xml
- name: OpenVINO C API tests
if: fromJSON(inputs.affected-components).C_API.test
run: |
@ -234,7 +231,7 @@ jobs:
if: fromJSON(inputs.affected-components).AUTO_BATCH.test
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_auto_batch_func_tests --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_auto_batch_func_tests.xml --gtest_filter="*smoke*"
${INSTALL_TEST_DIR}/ov_auto_batch_func_tests --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_auto_batch_func_tests.xml
- name: Proxy Plugin func tests
if: fromJSON(inputs.affected-components).PROXY.test
@ -252,10 +249,10 @@ jobs:
if: fromJSON(inputs.affected-components).HETERO.test
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_hetero_func_tests --gtest_print_time=1 --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OVHeteroFuncTests.xml --gtest_filter="*smoke*"
${INSTALL_TEST_DIR}/ov_hetero_func_tests --gtest_print_time=1 --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OVHeteroFuncTests.xml
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: ${{ !cancelled() }}
with:
name: test-results-cpp

View File

@ -13,8 +13,6 @@ on:
required: false
default: null
permissions: read-all
jobs:
Debian_Packages:
name: Debian Packages
@ -29,11 +27,10 @@ jobs:
DEBIAN_PACKAGES_DIR: ${{ github.workspace }}/packages
steps:
- name: Set apt retries
if: runner.os == 'Linux'
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO debian packages
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_debian_packages
path: ${{ env.DEBIAN_PACKAGES_DIR }}
@ -45,13 +42,13 @@ jobs:
- name: Install debian packages & check conflicts
run: |
apt-get update -y
if [[ "${{ runner.arch }}" == "X64" ]]; then
# Install debian packages from previous release
apt-get install --no-install-recommends -y gnupg wget ca-certificates
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
echo "deb https://apt.repos.intel.com/openvino/2024 ubuntu20 main" | tee /etc/apt/sources.list.d/intel-openvino-2024.list
echo "deb https://apt.repos.intel.com/openvino/2023 ubuntu20 main" | tee /etc/apt/sources.list.d/intel-openvino-2023.list
apt-get update -y
apt-get install -y openvino
fi
@ -68,26 +65,21 @@ jobs:
run: |
/usr/share/openvino/samples/cpp/build_samples.sh
/usr/share/openvino/samples/c/build_samples.sh
[[ "${{ runner.arch }}" == "X64" ]] && path_by_arch="intel64" || path_by_arch="aarch64"
~/openvino_cpp_samples_build/$path_by_arch/Release/hello_query_device
# check integrity of OpenVINO Python API installation
apt-get install python3-pip -y
python3 -m pip check
python3 /usr/share/openvino/samples/python/hello_query_device/hello_query_device.py
python3 -c 'from openvino import Core; Core().get_property("CPU", "AVAILABLE_DEVICES")'
if [[ "${{ runner.arch }}" == "X64" ]]; then
python3 -c 'from openvino import Core; Core().get_property("GPU", "AVAILABLE_DEVICES")'
fi
python3 -c 'from openvino import Core; Core().get_property("AUTO", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("MULTI", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("HETERO", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("BATCH", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino.frontend import FrontEndManager; assert len(FrontEndManager().get_available_front_ends()) == 6'
benchmark_app --help
opt_in_out --help
ovc --help

View File

@ -1,134 +0,0 @@
name: GPU
on:
workflow_call:
inputs:
test_type:
description: 'Type of tests to execute'
type: string
required: true
device:
description: 'Device name (igpu or dgpu)'
type: string
required: true
runner:
description: 'Runner labels by which the runner will be chosen. Example: [ "self-hosted", "igpu" ]'
type: string
required: true
container:
description: 'JSON to be converted to the value of the "container" configuration for the job'
type: string
required: false
default: '{"image": null}'
jobs:
GPU:
timeout-minutes: 80
runs-on: ${{ fromJSON(inputs.runner) }}
container: ${{ fromJSON(inputs.container) }}
defaults:
run:
shell: bash
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
GTEST_PARALLEL_SCRIPT: ${{ github.workspace }}/gtest_parallel.py
steps:
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: 'openvino_package'
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: 'openvino_tests'
path: ${{ env.INSTALL_TEST_DIR }}
# Needed as ${{ github.workspace }} is not working correctly when using Docker
- name: Setup Variables
run: |
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "GTEST_PARALLEL_SCRIPT=$GITHUB_WORKSPACE/gtest_parallel.py" >> "$GITHUB_ENV"
- name: Extract OpenVINO packages
run: |
pushd $INSTALL_DIR
tar -xzf openvino_package.tar.gz -C $INSTALL_DIR
popd
pushd $INSTALL_TEST_DIR
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Install dependencies (Linux)
run: |
$INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -c=gpu -y
apt-get update && apt-get install -y wget software-properties-common ca-certificates gpg-agent tzdata clinfo
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
TZ: "Europe/London" # to prevent tzdata from waiting user input
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Get gtest-parallel script
run: wget https://raw.githubusercontent.com/google/gtest-parallel/master/gtest_parallel.py
- name: Install compute runtime drivers
run: |
wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.15985.7/intel-igc-core_1.0.15985.7_amd64.deb
wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.15985.7/intel-igc-opencl_1.0.15985.7_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.05.28454.6/intel-level-zero-gpu-dbgsym_1.3.28454.6_amd64.ddeb
wget https://github.com/intel/compute-runtime/releases/download/24.05.28454.6/intel-level-zero-gpu_1.3.28454.6_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.05.28454.6/intel-opencl-icd-dbgsym_24.05.28454.6_amd64.ddeb
wget https://github.com/intel/compute-runtime/releases/download/24.05.28454.6/intel-opencl-icd_24.05.28454.6_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.05.28454.6/libigdgmm12_22.3.11_amd64.deb
dpkg -i *.deb
- name: Install media & display runtimes
if: ${{ inputs.device == 'dgpu' }}
run: |
apt-get update && apt-get install -y \
libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri \
libglapi-mesa libgles2-mesa-dev libglx-mesa0 libigdgmm11 libxatracker2 mesa-va-drivers \
mesa-vdpau-drivers mesa-vulkan-drivers va-driver-all
- name: Verify devices
run: clinfo
#
# Tests
#
- name: OpenVINO GPU ${{ inputs.test_type }} Tests
id: run_tests
run: |
source ${INSTALL_DIR}/setupvars.sh
TEST_RESULTS_DIR="${{ inputs.device }}_${{ inputs.test_type }}_tests"
echo "test_results_dir=$TEST_RESULTS_DIR" >> $GITHUB_OUTPUT
rm -rf ${INSTALL_TEST_DIR}/${TEST_RESULTS_DIR} && mkdir -p ${INSTALL_TEST_DIR}/${TEST_RESULTS_DIR}
test_filter=''
if [[ "${{ inputs.test_type }}" == "unit" ]]; then
# Ticket: 138018
test_filter='-*scatter_nd_update_gpu.dynamic_padded_output*:*border_gpu.basic_zero_input*:*bicubic_zeros_no_align_data1x1*:*bicubic_border_align_batches*:*bilinear_zeros_no_align_data1x1*:*non_zero_gpu.empty_input*:*mark_shape_of_subgraphs.concat_with_empty_tensor_inputs*:*concat_cpu_impl.dynamic_4d_f*:*border_gpu.basic_zero_input_dynamic*:*network_test.model_with_empty_input_is_not_dynamic*:*bicubic_zeros_align_data1x1*'
else
test_filter='*smoke*'
fi
python3 ${GTEST_PARALLEL_SCRIPT} ${INSTALL_TEST_DIR}/ov_gpu_${{ inputs.test_type }}_tests --dump_json_test_results=${INSTALL_TEST_DIR}/${TEST_RESULTS_DIR}/ov_gpu_${{ inputs.test_type }}_tests.json -- --report_unique_name --gtest_filter=$test_filter
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
if: always()
with:
name: test-results-${{ inputs.test_type }}-${{ inputs.device }}
path: ${{ env.INSTALL_TEST_DIR }}/${{ steps.run_tests.outputs.test_results_dir }}
if-no-files-found: 'error'

View File

@ -13,8 +13,6 @@ on:
required: false
default: '{"image": null}'
permissions: read-all
jobs:
ONNX_Models_tests:
name: ONNX Models tests
@ -30,21 +28,21 @@ jobs:
INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
ONNX_MODELS_PATH: ${{ github.workspace }}/onnx_test_models
# instead of using static MODELS_SHARE_PATH
# choose one of the replicas dynamically instead
# depending on GITHUB_RUN_NUMBER variable
NUMBER_OF_REPLICAS: 2
MODELS_SHARE_PATH: "/mount/onnxtestdata"
ONNX_MODEL_ZOO_SHA: "5faef4c33eba0395177850e1e31c4a6a9e634c82"
if: ${{ github.event_name != 'merge_group' }}
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -55,8 +53,6 @@ jobs:
echo "OPENVINO_REPO=$GITHUB_WORKSPACE/openvino" >> "$GITHUB_ENV"
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "MODELS_SHARE_PATH=/mount/testdata$((GITHUB_RUN_NUMBER % NUMBER_OF_REPLICAS))" >> "$GITHUB_ENV"
echo $MODELS_SHARE_PATH
- name: Extract OpenVINO packages
run: |
@ -68,14 +64,27 @@ jobs:
tar -xzf openvino_tests.tar.gz -C ${INSTALL_DIR}
popd
- name: Fetch model_zoo_preprocess script
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- name: Fetch setup_python action and model_zoo_preprocess script
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
src/frontends/onnx/tests/tests_python/model_zoo_preprocess.sh
sparse-checkout-cone-mode: false
path: 'openvino'
- name: Install dependencies
run: |
# install git (required to build pip deps from the sources)
apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates git-lfs
- name: Setup Python 3.11
uses: ./openvino/.github/actions/setup_python
with:
version: '3.11'
should-setup-pip-paths: 'false'
self-hosted-runner: ${{ contains(inputs.runner, 'aks') }}
- name: Update Models
run: bash ${OPENVINO_REPO}/src/frontends/onnx/tests/tests_python/model_zoo_preprocess.sh -d ${MODELS_SHARE_PATH} -o -s "${{ env.ONNX_MODEL_ZOO_SHA }}"

View File

@ -17,8 +17,6 @@ on:
type: string
required: true
permissions: read-all
jobs:
ONNX_Runtime:
name: ONNX Runtime Integration
@ -42,8 +40,11 @@ jobs:
ONNX_RUNTIME_UTILS: ${{ github.workspace }}/install/onnxruntime
ONNX_RUNTIME_BUILD_DIR: ${{ github.workspace }}/onnxruntime/build
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
@ -57,30 +58,62 @@ jobs:
echo "ONNX_RUNTIME_UTILS=$GITHUB_WORKSPACE/install/onnxruntime" >> "$GITHUB_ENV"
echo "ONNX_RUNTIME_BUILD_DIR=$GITHUB_WORKSPACE/onnxruntime/build" >> "$GITHUB_ENV"
- name: Fetch install_build_dependencies.sh and setup_python action
uses: actions/checkout@v4
with:
sparse-checkout: |
install_build_dependencies.sh
.github/actions/setup_python/action.yml
sparse-checkout-cone-mode: false
path: 'openvino'
- name: Install git
run: |
apt-get update
apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
version: '3.11'
should-setup-pip-paths: 'false'
- name: Extract OpenVINO package
run: |
pushd ${INSTALL_DIR}
tar -xzf openvino_package.tar.gz -C ${INSTALL_DIR}
popd
- name: Install OpenVINO dependencies
run: |
${INSTALL_DIR}/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -y
# since we are on Ubuntu 22.04, but compiled OpenVINO on Ubuntu 20.04, we need to install `libtbb2`
apt-get install --assume-yes --no-install-recommends libtbb2
- name: Clone ONNX Runtime
run: |
hash=`tr -s '\n ' < ${ONNX_RUNTIME_UTILS}/version`
git clone --recursive https://github.com/microsoft/onnxruntime.git ${ONNX_RUNTIME_REPO}
cd ${ONNX_RUNTIME_REPO}
git checkout $hash
branch=`tr -s '\n ' < ${ONNX_RUNTIME_UTILS}/version`
git clone --branch $branch --single-branch --recursive https://github.com/microsoft/onnxruntime.git ${ONNX_RUNTIME_REPO}
#
# Tests
#
- name: Install Build Dependencies
run: bash ${OPENVINO_REPO}/install_build_dependencies.sh
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
- name: Build Lin ONNX Runtime
run: |
source ${INSTALL_DIR}/setupvars.sh
${ONNX_RUNTIME_REPO}/build.sh \
--config RelWithDebInfo \
--use_openvino CPU \
--use_openvino CPU_FP32 \
--build_shared_lib \
--parallel \
--skip_tests \
@ -96,11 +129,6 @@ jobs:
- name: Run onnxruntime_test_all
if: ${{ runner.arch != 'ARM64' }} # Ticket: 126277
run: |
# see https://github.com/microsoft/onnxruntime/issues/13197#issuecomment-1264542497
apt-get update && apt-get install --assume-yes --no-install-recommends language-pack-en
locale-gen en_US.UTF-8
update-locale LANG=en_US.UTF-8
source ${INSTALL_DIR}/setupvars.sh
skip_tests=$(tr -s '\n ' ':' < ${ONNX_RUNTIME_UTILS}/skip_tests)

View File

@ -13,8 +13,6 @@ on:
required: false
default: '{"image": null}'
permissions: read-all
jobs:
JS_API:
name: OpenVINO JS API
@ -28,10 +26,10 @@ jobs:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
OPENVINO_JS_DIR: ${{ github.workspace }}/openvino/src/bindings/js
OPENVINO_JS_LIBS_DIR: ${{ github.workspace }}/openvino/src/bindings/js/node/bin
NODE_VERSION: 20
NODE_VERSION: 18
steps:
- name: Fetch OpenVINO JS sources
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
src/bindings/js
@ -44,13 +42,13 @@ jobs:
echo "OPENVINO_JS_LIBS_DIR=$GITHUB_WORKSPACE/openvino/src/bindings/js/node/bin" >> "$GITHUB_ENV"
- name: Download OpenVINO JS package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_js_package
path: ${{ env.OPENVINO_JS_LIBS_DIR }}
- name: Setup Node ${{ env.NODE_VERSION }}
uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}

View File

@ -17,8 +17,6 @@ on:
type: string
required: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
@ -39,15 +37,17 @@ jobs:
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
LAYER_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/layer_tests
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -69,16 +69,20 @@ jobs:
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Install OpenVINO dependencies (Linux)
if: runner.os == 'Linux'
run: $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -y -c=gpu
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
sparse-checkout-cone-mode: false
path: 'action_root'
path: 'openvino'
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./action_root/.github/actions/setup_python
uses: ./openvino/.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ runner.os == 'Linux' && env.PIP_CACHE_PATH || '' }}
@ -132,15 +136,18 @@ jobs:
- name: Model Optimizer unit tests
if: fromJSON(inputs.affected-components).MO.test
run: |
if [[ "${{ runner.os }}" == "Linux" ]] && [[ "${{ runner.arch }}" != "ARM64" ]]; then
skip_filter=''
if [[ "${{ runner.os }}" != "Linux" ]] && [[ "${{ runner.arch }} != "ARM64" ]] || [[ "${{ runner.os }} != "macOS" ]]; then
# required for MxNet
apt-get install -y libgomp1 libquadmath0
else
# Skips under Ticket: 122666
skip_filter='--ignore-glob=**/mo/unit_tests/mo/front/mxnet/**'
fi
# Skips under tickets: 133405, 122666
python3 -m pytest -s ${INSTALL_TEST_DIR}/mo/unit_tests \
--junitxml=${INSTALL_TEST_DIR}/TEST-ModelOptimizer.xml \
--ignore-glob="**/mo/unit_tests/mo/front/mxnet/**"
"$skip_filter"
- name: Python ONNX operators tests
if: (fromJSON(inputs.affected-components).Python_API.test ||
@ -206,7 +213,7 @@ jobs:
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/py_frontend_tests --junitxml=${INSTALL_TEST_DIR}/TEST-test_py_fontend.xml
- name: PyTorch Layer Tests
if: ${{ fromJSON(inputs.affected-components).PyTorch_FE.test && runner.arch != 'ARM64' }} # Ticket: 126287, 142196
if: ${{ fromJSON(inputs.affected-components).PyTorch_FE.test && runner.arch != 'ARM64' }} # Ticket: 126287
run: python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/pytorch_tests -n logical -m precommit --junitxml=${INSTALL_TEST_DIR}/TEST-pytorch.xml
env:
TEST_DEVICE: CPU
@ -215,7 +222,7 @@ jobs:
- name: PyTorch torch.export Layer Tests
if: ${{ fromJSON(inputs.affected-components).PyTorch_FE.test && runner.arch != 'ARM64' }} # Ticket: 126287
run: |
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/pytorch_tests -n logical -m precommit_torch_export --junitxml=${INSTALL_TEST_DIR}/TEST-pytorch.xml
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/pytorch_tests -m precommit_torch_export --junitxml=${INSTALL_TEST_DIR}/TEST-pytorch.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP32
@ -230,6 +237,15 @@ jobs:
TEST_PRECISION: FP32
PYTORCH_TRACING_MODE: TORCHFX
- name: PyTorch torch.compile TORCHSCRIPT Layer Tests
if: ${{ fromJSON(inputs.affected-components).PyTorch_FE.test && runner.os != 'macOS' && runner.arch != 'ARM64' }} # Ticket: 126287
run: |
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/pytorch_tests -m precommit_ts_backend --junitxml=${INSTALL_TEST_DIR}/TEST-pytorch.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP32
PYTORCH_TRACING_MODE: TORCHSCRIPT
- name: ONNX Layer Tests
if: fromJSON(inputs.affected-components).ONNX_FE.test
run: |
@ -240,11 +256,41 @@ jobs:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: TensorFlow 1 Layer Tests - TF FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: |
# requires 'unit_tests' from 'mo'
export PYTHONPATH=${INSTALL_TEST_DIR}/mo
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/tensorflow_tests/ --use_new_frontend -m precommit_tf_fe -n logical --junitxml=${INSTALL_TEST_DIR}/TEST-tf_fe.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: TensorFlow 2 Layer Tests - TF FE
if: fromJSON(inputs.affected-components).TF_FE.test && runner.os != 'macOS' # Ticket: 123322
run: |
# requires 'unit_tests' from 'mo'
export PYTHONPATH=${INSTALL_TEST_DIR}/mo
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/tensorflow2_keras_tests/ --use_new_frontend -m precommit_tf_fe --junitxml=${INSTALL_TEST_DIR}/TEST-tf2_fe.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: JAX Layer Tests - TF FE
if: ${{ fromJSON(inputs.affected-components).TF_FE.test && runner.arch != 'ARM64' && runner.os != 'macOS' }}
if: ${{ fromJSON(inputs.affected-components).TF_FE.test && runner.arch != 'ARM64' }}
run: python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/jax_tests/ -m precommit --junitxml=${INSTALL_TEST_DIR}/TEST-jax.xml
env:
TEST_DEVICE: CPU
- name: TensorFlow 1 Layer Tests - Legacy FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/tensorflow_tests/test_tf_Roll.py --ir_version=10 --junitxml=${INSTALL_TEST_DIR}/TEST-tf_Roll.xml
- name: TensorFlow 2 Layer Tests - Legacy FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/tensorflow2_keras_tests/test_tf2_keras_activation.py --ir_version=11 -k "sigmoid" --junitxml=${INSTALL_TEST_DIR}/TEST-tf2_Activation.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: TensorFlow Lite Layer Tests - TFL FE
@ -254,25 +300,11 @@ jobs:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: TensorFlow 1 Layer Tests - Legacy FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/tensorflow_tests/test_tf_Roll.py --use_legacy_frontend --ir_version=10 --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-tf_Roll.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: TensorFlow 2 Layer Tests - Legacy FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/tensorflow2_keras_tests/test_tf2_keras_activation.py --use_legacy_frontend --ir_version=11 -k "sigmoid" --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-tf2_Activation.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: Clone API snippets
if: runner.os != 'macOS'
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: docs/snippets
sparse-checkout: openvino/docs/snippets
path: ${{ env.OPENVINO_REPO }}
submodules: 'false'
@ -286,7 +318,7 @@ jobs:
python3 ${OPENVINO_REPO}/docs/snippets/main.py
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: ${{ !cancelled() }}
with:
name: test-results-python

View File

@ -17,8 +17,6 @@ on:
type: string
required: true
permissions: read-all
jobs:
PyTorch_Models_Tests:
name: PyTorch Models tests
@ -40,7 +38,6 @@ jobs:
run: if [ "$(id -u)" -eq 0 ]; then apt update && apt --assume-yes install sudo; fi
- name: Set apt retries
if: runner.os == 'Linux'
run: |
if [ "$(id -u)" -eq 0 ]; then
echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
@ -49,19 +46,19 @@ jobs:
fi
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tokenizers extension
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tokenizers_wheel
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -85,7 +82,7 @@ jobs:
popd
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
@ -108,45 +105,22 @@ jobs:
- name: Install OpenVINO Python wheels
run: |
# To enable pytest parallel features
python3 -m pip install pytest-xdist[psutil]
python3 -m pip install ${INSTALL_DIR}/tools/openvino-*
python3 -m pip install ${INSTALL_DIR}/openvino_tokenizers-*
- name: Install PyTorch tests requirements
run: |
python3 -m pip install -r ${MODEL_HUB_TESTS_INSTALL_DIR}/pytorch/requirements.txt
python3 -m pip install -r ${MODEL_HUB_TESTS_INSTALL_DIR}/pytorch/requirements_secondary.txt
python3 -m pip install -r ${MODEL_HUB_TESTS_INSTALL_DIR}/torch_tests/requirements.txt
python3 -m pip install -r ${MODEL_HUB_TESTS_INSTALL_DIR}/torch_tests/requirements_secondary.txt
echo "Available storage:"
df -h
env:
CPLUS_INCLUDE_PATH: ${{ env.Python_ROOT_DIR }}/include/python${{ env.PYTHON_VERSION }}
- name: PyTorch Models Tests Timm and Torchvision
- name: PyTorch Models Tests
run: |
export PYTHONPATH=${MODEL_HUB_TESTS_INSTALL_DIR}:$PYTHONPATH
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/pytorch/ -m ${TYPE} --html=${INSTALL_TEST_DIR}/TEST-torch_model_timm_tv_tests.html --self-contained-html -v -n 4 -k "TestTimmConvertModel or TestTorchHubConvertModel"
env:
TYPE: ${{ inputs.event == 'schedule' && 'nightly' || 'precommit'}}
TEST_DEVICE: CPU
OP_REPORT_FILE: ${{ env.INSTALL_TEST_DIR }}/TEST-torch_unsupported_ops.log
- name: PyTorch Models Tests Not Timm or Torchvision
if: always()
run: |
export PYTHONPATH=${MODEL_HUB_TESTS_INSTALL_DIR}:$PYTHONPATH
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/pytorch -m ${TYPE} --html=${INSTALL_TEST_DIR}/TEST-torch_model_tests.html --self-contained-html -v -k "not (TestTimmConvertModel or TestTorchHubConvertModel)"
env:
TYPE: ${{ inputs.event == 'schedule' && 'nightly' || 'precommit'}}
TEST_DEVICE: CPU
USE_SYSTEM_CACHE: False
OP_REPORT_FILE: ${{ env.INSTALL_TEST_DIR }}/TEST-torch_unsupported_ops.log
- name: PagedAttention Test
if: always()
run: |
export PYTHONPATH=${MODEL_HUB_TESTS_INSTALL_DIR}:$PYTHONPATH
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/pytorch/test_pa_transformation.py -m ${TYPE} --html=${INSTALL_TEST_DIR}/TEST-torch_pagedattention_tests.html --self-contained-html -v --tb=short
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/torch_tests -m ${TYPE} --html=${INSTALL_TEST_DIR}/TEST-torch_model_tests.html --self-contained-html -v
env:
TYPE: ${{ inputs.event == 'schedule' && 'nightly' || 'precommit'}}
TEST_DEVICE: CPU
@ -156,15 +130,15 @@ jobs:
- name: Reformat unsupported ops file
if: '!cancelled()'
run: |
python3 ${MODEL_HUB_TESTS_INSTALL_DIR}/pytorch/scripts/process_op_report.py ${INSTALL_TEST_DIR}/TEST-torch_unsupported_ops.log
python3 ${MODEL_HUB_TESTS_INSTALL_DIR}/torch_tests/scripts/process_op_report.py ${INSTALL_TEST_DIR}/TEST-torch_unsupported_ops.log
- name: Available storage after tests
run: |
echo "Available storage:"
df -h
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: ${{ !cancelled() }}
with:
name: test-results-torch-models

View File

@ -17,8 +17,6 @@ on:
type: string
required: true
permissions: read-all
jobs:
Samples:
runs-on: ${{ inputs.runner }}
@ -33,14 +31,17 @@ jobs:
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
BUILD_DIR: ${{ github.workspace }}/build
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -61,12 +62,16 @@ jobs:
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Install OpenVINO dependencies (Linux)
if: runner.os == 'Linux'
run: $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -y
- name: Install OpenVINO dependencies (mac)
if: runner.os == 'macOS'
run: brew install coreutils
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
@ -78,6 +83,7 @@ jobs:
with:
version: '3.11'
should-setup-pip-paths: 'false'
self-hosted-runner: ${{ runner.os == 'Linux' }}
- name: Build cpp samples - GCC
run: $INSTALL_DIR/samples/cpp/build_samples.sh -i $INSTALL_DIR -b $BUILD_DIR/cpp_samples
@ -87,7 +93,7 @@ jobs:
- name: Build cpp samples - Clang
if: runner.os == 'Linux'
run: |
apt-get update && apt-get install -y clang
apt-get install -y clang
$INSTALL_DIR/samples/cpp/build_samples.sh -i $INSTALL_DIR -b $BUILD_DIR/cpp_samples_clang
env:
CMAKE_COMPILE_WARNING_AS_ERROR: 'ON'
@ -117,4 +123,13 @@ jobs:
source ${INSTALL_DIR}/setupvars.sh
PYTHONCOERCECLOCALE=warn python3 -bb -W error -X dev -m pytest $INSTALL_TEST_DIR/smoke_tests
PYTHONCOERCECLOCALE=warn python3 -bb -W error -X dev -X warn_default_encoding -m pytest $INSTALL_TEST_DIR/smoke_tests \
--junitxml=$INSTALL_TEST_DIR/TEST-SamplesSmokeTests.xml
- name: Upload Test Results
uses: actions/upload-artifact@v3
if: ${{ !cancelled() }}
with:
name: test-results-samples
path: ${{ env.INSTALL_TEST_DIR }}/TEST*.xml
if-no-files-found: 'warn'

View File

@ -1,4 +1,4 @@
name: TensorFlow Models tests
name: TensorFlow Hub Models tests
on:
workflow_call:
@ -12,17 +12,15 @@ on:
type: string
required: false
default: '{"image": null}'
model_scope:
description: 'Scope of models for testing.'
event:
description: 'Event that triggered the workflow. E.g., "schedule" for nightly runs'
type: string
required: true
permissions: read-all
jobs:
TensorFlow_Models_Tests:
name: TensorFlow Models tests
timeout-minutes: ${{ inputs.model_scope != 'precommit' && 400 || 40 }}
TensorFlow_Hub_Models_Tests:
name: TensorFlow Hub Models tests
timeout-minutes: ${{ inputs.event == 'schedule' && 400 || 25 }}
runs-on: ${{ inputs.runner }}
container: ${{ fromJSON(inputs.container) }}
defaults:
@ -34,22 +32,33 @@ jobs:
INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
MODEL_HUB_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/model_hub_tests
NUMBER_OF_REPLICAS: 2
steps:
- name: Check sudo
if: ${{ runner.os == 'Linux' }}
run: if [ "$(id -u)" -eq 0 ]; then apt update && apt --assume-yes install sudo; fi
- name: Set apt retries
run: |
if [ "$(id -u)" -eq 0 ]; then
echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
else
sudo sh -c "echo 'Acquire::Retries \"10\";' >> /etc/apt/apt.conf.d/80-retries"
fi
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tokenizers extension
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tokenizers_wheel
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -61,10 +70,6 @@ jobs:
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "MODEL_HUB_TESTS_INSTALL_DIR=$GITHUB_WORKSPACE/install/tests/model_hub_tests" >> "$GITHUB_ENV"
echo "TFHUB_CACHE_DIR=/mount/testdata$((GITHUB_RUN_NUMBER % NUMBER_OF_REPLICAS))/tfhub_models" >> "$GITHUB_ENV"
echo $TFHUB_CACHE_DIR
echo "HF_HUB_CACHE=/mount/testdata$((GITHUB_RUN_NUMBER % NUMBER_OF_REPLICAS))/hugging_face" >> "$GITHUB_ENV"
echo $HF_HUB_CACHE
- name: Extract OpenVINO packages
run: |
@ -77,13 +82,19 @@ jobs:
popd
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
sparse-checkout-cone-mode: false
path: 'openvino'
- name: Install dependencies
if: ${{ runner.os == 'Linux' }}
run: |
# install git (required to build pip deps from the sources)
sudo apt-get install --assume-yes --no-install-recommends g++ git ca-certificates
- name: Setup Python 3.11
uses: ./openvino/.github/actions/setup_python
with:
@ -96,28 +107,22 @@ jobs:
python3 -m pip install ${INSTALL_DIR}/tools/openvino-*
python3 -m pip install ${INSTALL_DIR}/openvino_tokenizers-*
- name: Install TF Models tests requirements
run: python3 -m pip install -r ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/requirements.txt
- name: Install TF Hub tests requirements
run: python3 -m pip install -r ${MODEL_HUB_TESTS_INSTALL_DIR}/tf_hub_tests/requirements.txt
- name: TensorFlow Models Tests - TF FE
- name: TensorFlow Hub Tests - TF FE
run: |
export PYTHONPATH=${MODEL_HUB_TESTS_INSTALL_DIR}:$PYTHONPATH
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/test_tf_convert_model.py -m ${{ inputs.model_scope }} \
--html=${INSTALL_TEST_DIR}/TEST-tf_fe_models_convert_model_${{ inputs.model_scope }}.html --self-contained-html -v
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/test_tf_read_model.py -m ${{ inputs.model_scope }} \
--html=${INSTALL_TEST_DIR}/TEST-tf_fe_models_read_model_${{ inputs.model_scope }}.html --self-contained-html -v
# decouple notebook tests due to GitHub issue in tensorflow_hub https://github.com/tensorflow/hub/issues/903
# and use WA to switch to (legacy) Keras 2
TF_USE_LEGACY_KERAS=1 python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/test_tf_hub_api_notebooks.py -m ${{ inputs.model_scope }} \
--html=${INSTALL_TEST_DIR}/TEST-tf_fe_models_notebooks_${{ inputs.model_scope }}.html --self-contained-html -v
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tf_hub_tests/ -m ${TYPE} --html=${INSTALL_TEST_DIR}/TEST-tf_hub_tf_fe.html --self-contained-html -v
env:
TYPE: ${{ inputs.event == 'schedule' && 'nightly' || 'precommit'}}
TEST_DEVICE: CPU
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: ${{ !cancelled() }}
with:
name: test-results-tensorflow-models-${{ inputs.model_scope }}
name: test-results-tensorflow-hub-models
path: |
${{ env.INSTALL_TEST_DIR }}/TEST*.html
if-no-files-found: 'error'

View File

@ -1,168 +0,0 @@
name: TensorFlow Layer Tests
on:
workflow_call:
inputs:
runner:
description: 'Machine on which the tests would run'
type: string
required: true
shell:
description: "shell to override the default shell settings in the runner's operating system."
type: string
required: true
container:
description: 'JSON to be converted to the value of the "container" configuration for the job'
type: string
required: false
default: '{"image": null}'
affected-components:
description: 'Components that are affected by changes in the commit defined by the Smart CI Action'
type: string
required: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
jobs:
TensorFlow_Layer_Tests:
name: TensorFlow Layer Tests
timeout-minutes: 30
runs-on: ${{ inputs.runner }}
container: ${{ fromJSON(inputs.container) }}
defaults:
run:
shell: ${{ inputs.shell }}
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
OPENVINO_REPO: ${{ github.workspace }}/openvino
INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
LAYER_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/layer_tests
steps:
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
- name: Download OpenVINO tokenizers extension
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: openvino_tokenizers_wheel
path: ${{ env.INSTALL_DIR }}
# Needed as ${{ github.workspace }} is not working correctly when using Docker
- name: Setup Variables
if: runner.os != 'Windows'
run: |
echo "OPENVINO_REPO=$GITHUB_WORKSPACE/openvino" >> "$GITHUB_ENV"
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "LAYER_TESTS_INSTALL_DIR=$GITHUB_WORKSPACE/install/tests/layer_tests" >> "$GITHUB_ENV"
- name: Extract OpenVINO packages (Linux, macOS)
if: runner.os != 'Windows'
run: |
pushd $INSTALL_DIR
tar -xzf openvino_package.tar.gz -C $INSTALL_DIR
popd
pushd $INSTALL_TEST_DIR
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Extract OpenVINO packages (Windows)
if: runner.os == 'Windows'
run: |
pushd ${{ env.INSTALL_DIR }}
Expand-Archive openvino_package.zip -DestinationPath ${{ env.INSTALL_DIR }}
popd
pushd ${{ env.INSTALL_TEST_DIR }}
Expand-Archive openvino_tests.zip -DestinationPath ${{ env.INSTALL_DIR }}
popd
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
sparse-checkout-cone-mode: false
path: 'openvino'
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ runner.os == 'Linux' && env.PIP_CACHE_PATH || '' }}
should-setup-pip-paths: ${{ runner.os == 'Linux' }}
self-hosted-runner: ${{ runner.os == 'Linux' }}
#
# Tests
#
- name: Install OpenVINO Python wheels (Linux and macOS)
if: runner.os != 'Windows'
run: |
# Install the core OV wheel
python3 -m pip install ${INSTALL_DIR}/tools/openvino-*.whl
# Install the core OV Tokenizers wheel
python3 -m pip install ${INSTALL_DIR}/openvino_tokenizers-*.whl
- name: Install OpenVINO Python wheels (Windows)
if: runner.os == 'Windows'
run: |
# Find and install the core OV wheel
$ovCoreWheelPath=Get-ChildItem -Path ${{ env.INSTALL_DIR }}\tools -Filter openvino-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"
# Find and install the core OV Tokenizers wheel
$ovCoreWheelPath=Get-ChildItem -Path ${{ env.INSTALL_DIR }} -Filter openvino_tokenizers-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"
- name: Install Python Layer tests dependencies
run: |
# To enable pytest parallel features
python3 -m pip install pytest-xdist[psutil]
# layer test requirements
python3 -m pip install -r ${{ env.LAYER_TESTS_INSTALL_DIR }}/requirements.txt
#
# Tests
#
- name: TensorFlow 1 Layer Tests - TF FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: |
python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/tensorflow_tests/ -n logical -m precommit --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-tf_fe.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: TensorFlow 2 Layer Tests - TF FE
if: fromJSON(inputs.affected-components).TF_FE.test && runner.os != 'macOS' # Ticket: 123322
run: |
python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/tensorflow2_keras_tests/ -n logical -m precommit --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-tf2_fe.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
if: ${{ !cancelled() }}
with:
name: test-results-python-tf-layers
path: |
${{ env.INSTALL_TEST_DIR }}/TEST*.html
${{ env.INSTALL_TEST_DIR }}/TEST*.xml
if-no-files-found: 'warn'

View File

@ -1,137 +0,0 @@
name: OpenVINO tokenizers extension
on:
workflow_call:
inputs:
runner:
description: 'Machine on which the tests would run'
type: string
required: true
shell:
description: "shell to override the default shell settings in the runner's operating system."
type: string
required: true
container:
description: 'JSON to be converted to the value of the "container" configuration for the job'
type: string
required: false
default: '{"image": null}'
affected-components:
description: 'Components that are affected by changes in the commit defined by the Smart CI Action'
type: string
required: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
jobs:
Openvino_tokenizers:
name: OpenVINO tokenizers extension
timeout-minutes: 25
defaults:
run:
shell: ${{ inputs.shell }}
runs-on: ${{ inputs.runner }}
container: ${{ fromJSON(inputs.container) }}
env:
INSTALL_DIR: ${{ github.workspace }}/install
OPENVINO_TOKENIZERS_REPO: ${{ github.workspace }}/openvino_tokenizers
EXTENSION_BUILD_DIR: ${{ github.workspace }}/build
steps:
# Needed as ${{ github.workspace }} is not working correctly when using Docker
- name: Setup Variables
run: |
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "OPENVINO_TOKENIZERS_REPO=$GITHUB_WORKSPACE/openvino_tokenizers" >> "$GITHUB_ENV"
echo "EXTENSION_BUILD_DIR=$GITHUB_WORKSPACE/build" >> "$GITHUB_ENV"
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
sparse-checkout: |
.github/actions/setup_python
.github/actions/cache
install_build_dependencies.sh
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ runner.os == 'Linux' && env.PIP_CACHE_PATH || '' }}
should-setup-pip-paths: ${{ runner.os == 'Linux' }}
self-hosted-runner: ${{ runner.os == 'Linux' }}
- name: Clone OpenVINO Tokenizers
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
repository: 'openvinotoolkit/openvino_tokenizers'
path: ${{ env.OPENVINO_TOKENIZERS_REPO }}
ref: 'master'
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Extract OpenVINO packages (Linux and macOS)
if: runner.os != 'Windows'
run: |
pushd ${INSTALL_DIR}
tar -xzf openvino_package.tar.gz -C ${INSTALL_DIR}
popd
- name: Extract OpenVINO packages (Windows)
if: runner.os == 'Windows'
run: |
pushd ${{ env.INSTALL_DIR }}
Expand-Archive openvino_package.zip -DestinationPath "${{ env.INSTALL_DIR }}"
popd
#
# Dependencies
#
- name: Install python dependencies
run: |
# wheel packaging
python3 -m pip install -r ${{ env.OPENVINO_TOKENIZERS_REPO }}/requirements-build.txt
#
# Build
#
- name: Build tokenizers wheel (Linux and macOS)
if: runner.os != 'Windows'
run: |
source ${INSTALL_DIR}/setupvars.sh
python -m build --wheel --outdir ${EXTENSION_BUILD_DIR} ${OPENVINO_TOKENIZERS_REPO}
env:
CMAKE_ARGS: '-DBUILD_FAST_TOKENIZERS=OFF'
CMAKE_BUILD_PARALLEL_LEVEL: '4'
CMAKE_GENERATOR: 'Unix Makefiles'
- name: Build tokenizers wheel (Windows)
if: runner.os == 'Windows'
run: |
. "${{ env.INSTALL_DIR }}/setupvars.ps1"
python3 -m build --wheel --outdir ${env:EXTENSION_BUILD_DIR} ${env:OPENVINO_TOKENIZERS_REPO}
env:
CMAKE_ARGS: '-DBUILD_FAST_TOKENIZERS=OFF'
CMAKE_BUILD_PARALLEL_LEVEL: '4'
#
# Upload build artifacts
#
- name: Upload openvino tokenizers wheel
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: openvino_tokenizers_wheel
path: ${{ env.EXTENSION_BUILD_DIR }}/*.whl
if-no-files-found: 'error'

View File

@ -2,8 +2,6 @@ name: "Pull Request Labeler"
on:
- pull_request_target
permissions: read-all
jobs:
triage:
permissions:
@ -11,34 +9,10 @@ jobs:
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: akladiev/labeler@eeac5941e7fb6f980d47e038ac0665168851c874 # v4.3.1
- uses: akladiev/labeler@v4.3.1
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
configuration-path: '.github/labeler.yml'
sync-labels: 'true'
dot: 'true'
non-matching-label: 'no-match-files'
external_pr_labeller:
name: Label External PR
runs-on: ubuntu-latest
steps:
- name: Checkout Labeller Script
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
sparse-checkout: '.github'
- name: Install deps
run: pip3 install PyGithub==2.2.0
- name: Dump GitHub context
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
run: echo "$GITHUB_CONTEXT"
- name: Label External PR
env:
GITHUB_TOKEN: "${{ secrets.EXTERNAL_LABELLER_TOKEN }}"
run: |
python3 ${{ github.workspace }}/.github/scripts/external_pr_labeller.py \
--repository-name ${GITHUB_REPOSITORY} --pr-number ${{ github.event.number }}

View File

@ -16,8 +16,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux
cancel-in-progress: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
@ -27,11 +25,10 @@ jobs:
runs-on: ubuntu-latest
outputs:
affected_components: "${{ steps.smart_ci.outputs.affected_components }}"
changed_components: "${{ steps.smart_ci.outputs.changed_components }}"
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: .github/actions/smart-ci
@ -48,44 +45,15 @@ jobs:
skip_when_only_listed_labels_set: 'docs'
skip_when_only_listed_files_changed: '*.md,*.rst,*.png,*.jpg,*.svg'
- name: Show affected components
run: |
echo "${{ toJSON(steps.smart_ci.outputs.affected_components) }}"
shell: bash
Docker:
needs: Smart_CI
runs-on: aks-linux-4-cores-16gb-docker-build
container:
image: openvinogithubactions.azurecr.io/docker_build:0.2
volumes:
- /mount:/mount
outputs:
images: "${{ steps.handle_docker.outputs.images }}"
steps:
- name: Checkout
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: ./.github/actions/handle_docker
id: handle_docker
with:
images: |
ov_build/ubuntu_20_04_x64
ov_build/ubuntu_20_04_x64_nvidia
ov_test/ubuntu_20_04_x64
registry: 'openvinogithubactions.azurecr.io'
dockerfiles_root_dir: '.github/dockerfiles'
changed_components: ${{ needs.smart_ci.outputs.changed_components }}
Build:
needs: [Docker, Smart_CI]
needs: Smart_CI
timeout-minutes: 150
defaults:
run:
shell: bash
runs-on: aks-linux-16-cores-32gb
container:
image: ${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount:/mount
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
@ -112,28 +80,27 @@ jobs:
if: "!needs.smart_ci.outputs.skip_workflow"
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git
run: |
apt-get update
apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
# Ticket: 139627
- name: Checkout the latest OneDNN for GPU in nightly
if: ${{ github.event_name == 'schedule' }}
working-directory: ${{ env.OPENVINO_REPO }}/src/plugins/intel_gpu/thirdparty/onednn_gpu
run: |
git fetch origin
git checkout main
git rev-parse HEAD
- name: Clone OpenVINO Contrib
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true'
ref: 'master'
ref: 'releases/2024/0'
#
# Print system info
@ -146,6 +113,26 @@ jobs:
# Dependencies
#
- name: Install build dependencies
run: |
bash ${OPENVINO_REPO}/install_build_dependencies.sh
# default-jdk - Java API
apt install --assume-yes --no-install-recommends default-jdk
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'
show-cache-info: 'true'
- name: Install python dependencies
run: |
# For Python API: build and wheel packaging
@ -175,6 +162,7 @@ jobs:
-DENABLE_NCC_STYLE=OFF \
-DENABLE_TESTS=ON \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_SYSTEM_OPENCL=ON \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DCPACK_GENERATOR=TGZ \
@ -225,8 +213,6 @@ jobs:
/usr/bin/python3.8 -m pip install -U pip
/usr/bin/python3.8 -m pip install -r ${OPENVINO_REPO}/src/bindings/python/wheel/requirements-dev.txt
cmake -UPYTHON* \
-UTBB* \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_PYTHON_PACKAGING=ON \
-DENABLE_TESTS=OFF \
-DPython3_EXECUTABLE=/usr/bin/python3.8 \
@ -237,8 +223,9 @@ jobs:
- name: Cmake & Build - OpenVINO Contrib
run: |
cmake \
-DBUILD_nvidia_plugin=OFF \
-DCUSTOM_OPERATIONS="calculate_grid;complex_mul;fft;grid_sample;sparse_conv;sparse_conv_transpose" \
-DOPENVINO_EXTRA_MODULES="${OPENVINO_CONTRIB_REPO}/modules/java_api;${OPENVINO_CONTRIB_REPO}/modules/custom_operations" \
-DOPENVINO_EXTRA_MODULES=${OPENVINO_CONTRIB_REPO}/modules \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
@ -256,7 +243,7 @@ jobs:
# Upload build artifacts and logs
#
- name: Upload build logs
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: always()
with:
name: build_logs
@ -265,7 +252,7 @@ jobs:
- name: Upload openvino package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_package
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
@ -273,7 +260,7 @@ jobs:
- name: Upload openvino js package
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
@ -281,7 +268,7 @@ jobs:
- name: Upload openvino developer package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_developer_package
path: ${{ env.BUILD_DIR }}/openvino_developer_package.tar.gz
@ -289,7 +276,7 @@ jobs:
- name: Upload openvino debian packages
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_debian_packages
path: ${{ env.BUILD_DIR }}/*.deb
@ -297,7 +284,7 @@ jobs:
- name: Upload openvino tests package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_tests
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
@ -312,26 +299,26 @@ jobs:
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
Samples:
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).samples
uses: ./.github/workflows/job_samples_tests.yml
with:
runner: 'aks-linux-4-cores-16gb'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_x64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
JS_API:
name: OpenVINO JS API
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: ./.github/workflows/job_openvino_js.yml
with:
runner: 'aks-linux-4-cores-16gb'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}"}'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04"}'
Conformance:
needs: [ Build, Smart_CI ]
timeout-minutes: ${{ matrix.TEST_TYPE == 'API' && 5 || 20 }}
timeout-minutes: ${{ matrix.TEST_TYPE == 'API' && 5 || 30 }}
defaults:
run:
shell: bash
@ -362,13 +349,13 @@ jobs:
#
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_tests
path: ${{ env.INSTALL_TEST_DIR }}
@ -383,7 +370,7 @@ jobs:
popd
- name: Fetch setup_python action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
@ -427,126 +414,76 @@ jobs:
- name: Upload Conformance Artifacts
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: conformance_artifacts_${{ matrix.TEST_TYPE }}-${{ env.TEST_DEVICE }}
path: ${{ env.CONFORMANCE_ARTIFACTS_DIR }}/conformance_artifacts.tar.gz
if-no-files-found: 'error'
- name: Mandatory API Conformance Tests (Template)
if: ${{ matrix.TEST_TYPE == 'API' }}
run: |
source ${INSTALL_DIR}/setupvars.sh
python3 ${CONFORMANCE_TOOLS_DIR}/run_conformance.py -ov=${INSTALL_DIR}/tests \
-d=TEMPLATE \
-t=${{ matrix.TEST_TYPE }} \
-w=${CONFORMANCE_ARTIFACTS_DIR} \
-sm=mandatory
- name: Pack Conformance Artifacts
if: ${{ matrix.TEST_TYPE == 'API' }}
run: |
pushd ${CONFORMANCE_ARTIFACTS_DIR}
tar -czvf ${CONFORMANCE_ARTIFACTS_DIR}/conformance_artifacts.tar.gz *
popd
- name: Upload Conformance Artifacts
if: ${{ matrix.TEST_TYPE == 'API' }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: conformance_artifacts_${{ matrix.TEST_TYPE }}-TEMPLATE
path: ${{ env.CONFORMANCE_ARTIFACTS_DIR }}/conformance_artifacts.tar.gz
if-no-files-found: 'error'
ONNX_Runtime:
name: ONNX Runtime Integration
if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_RT ||
fromJSON(needs.smart_ci.outputs.affected_components).ONNX_FE
needs: [ Build, Smart_CI, Docker ]
# Enable back once https://github.com/microsoft/onnxruntime/pull/19184 is merged
if: ${{ 'false' }}
# if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_RT ||
# fromJSON(needs.smart_ci.outputs.affected_components).ONNX_FE
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_onnx_runtime.yml
with:
runner: 'aks-linux-16-cores-32gb'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"], "options": "-e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING"}'
sccache-azure-key-prefix: 'ubuntu20_x86_64_onnxruntime'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04", "volumes": ["/mount:/mount"], "options": "-e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING"}'
sccache-azure-key-prefix: 'ubuntu22_x86_64_onnxruntime'
ONNX_Models:
name: ONNX Models Tests
if: fromJSON(needs.smart_ci.outputs.affected_components).Python_API.test ||
fromJSON(needs.smart_ci.outputs.affected_components).ONNX_FE.test
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_onnx_models_tests.yml
with:
runner: 'aks-linux-16-cores-64gb'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
runner: 'aks-linux-16-cores-32gb'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04", "volumes": ["/mount:/mount"]}'
CXX_Unit_Tests:
name: C++ unit tests
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_cxx_unit_tests.yml
with:
runner: 'aks-linux-4-cores-16gb'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_x64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
Python_Unit_Tests:
name: Python unit tests
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_python_unit_tests.yml
with:
runner: 'aks-linux-4-cores-16gb'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
TensorFlow_Layer_Tests:
name: TensorFlow Layer Tests
needs: [ Docker, Build, Smart_CI, Openvino_tokenizers ]
uses: ./.github/workflows/job_tensorflow_layer_tests.yml
with:
runner: 'aks-linux-4-cores-16gb'
shell: bash
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04", "volumes": ["/mount:/mount"]}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
CPU_Functional_Tests:
name: CPU functional tests
if: fromJSON(needs.smart_ci.outputs.affected_components).CPU.test
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_cpu_functional_tests.yml
with:
runner: 'aks-linux-8-cores-32gb'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_x64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
TensorFlow_Models_Tests_Precommit:
name: TensorFlow Models tests
# TODO: Switch back to self-hosted runners
# container:
# image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
# volumes:
# - /mount:/mount
TensorFlow_Hub_Models_Tests:
name: TensorFlow Hub Models tests
if: fromJSON(needs.smart_ci.outputs.affected_components).TF_FE.test ||
fromJSON(needs.smart_ci.outputs.affected_components).TFL_FE.test
needs: [ Docker, Build, Smart_CI, Openvino_tokenizers ]
uses: ./.github/workflows/job_tensorflow_models_tests.yml
needs: [ Build, Smart_CI, Openvino_tokenizers ]
uses: ./.github/workflows/job_tensorflow_hub_models_tests.yml
with:
runner: 'aks-linux-8-cores-16gb'
model_scope: 'precommit'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
TensorFlow_Models_Tests_Nightly_TF_HUB:
name: TensorFlow TF Hub Models tests
if: ${{ github.event_name == 'schedule' }}
needs: [ Docker, Build, Smart_CI, Openvino_tokenizers ]
uses: ./.github/workflows/job_tensorflow_models_tests.yml
with:
runner: 'aks-linux-8-cores-64gb'
model_scope: 'nightly_tf_hub'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
TensorFlow_Models_Tests_Nightly_HF:
name: TensorFlow Hugging Face Models tests
if: ${{ github.event_name == 'schedule' }}
needs: [ Docker, Build, Smart_CI, Openvino_tokenizers ]
uses: ./.github/workflows/job_tensorflow_models_tests.yml
with:
runner: 'aks-linux-8-cores-64gb'
model_scope: 'nightly_hf'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
runner: ${{ github.event_name == 'schedule' && 'ubuntu-20.04-16-cores' || 'ubuntu-20.04-8-cores' }}
event: ${{ github.event_name }}
# TODO: Switch back to self-hosted runners
# container:
@ -564,14 +501,14 @@ jobs:
NVIDIA_Plugin:
name: NVIDIA plugin
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
timeout-minutes: 15
defaults:
run:
shell: bash
runs-on: aks-linux-16-cores-32gb
container:
image: ${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64_nvidia }}
image: openvinogithubactions.azurecr.io/dockerhub/nvidia/cuda:11.8.0-runtime-ubuntu20.04
volumes:
- /mount:/mount
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
@ -595,14 +532,28 @@ jobs:
if: fromJSON(needs.smart_ci.outputs.affected_components).NVIDIA
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Fetch install_build_dependencies.sh
uses: actions/checkout@v4
with:
sparse-checkout: |
install_build_dependencies.sh
sparse-checkout-cone-mode: false
path: ${{ env.OPENVINO_REPO }}
- name: Install Prerequisites
run: apt update && apt install -y git ca-certificates
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO Developer package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_developer_package
path: ${{ env.INSTALL_DIR }}
@ -618,11 +569,43 @@ jobs:
popd
- name: Clone OpenVINO Contrib
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }}
ref: 'master'
ref: 'releases/2024/0'
#
# Dependencies
#
- name: Install build dependencies
run: |
${OPENVINO_REPO}/install_build_dependencies.sh
apt -y --no-install-recommends install software-properties-common curl
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
- name: Install CUDA
run: |
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
apt update
apt install -y \
libcudnn8=8.9.4.*-1+cuda11.8 \
libcudnn8-dev=8.9.4.*-1+cuda11.8 \
libcudnn8-samples=8.9.4.*-1+cuda11.8 \
cuda-runtime-11-8 \
cuda-11-8 \
libcutensor1=1.6.1.5-1 \
libcutensor-dev=1.6.1.5-1 \
cuda-drivers=520.61.05-1
#
# Build
@ -643,53 +626,110 @@ jobs:
Openvino_tokenizers:
name: OpenVINO tokenizers extension
needs: [ Build, Smart_CI, Docker ]
uses: ./.github/workflows/job_tokenizers.yml
with:
runner: 'aks-linux-4-cores-16gb'
shell: bash
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_x64 }}", "volumes": ["/mount:/mount"]}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
needs: [ Build, Smart_CI ]
timeout-minutes: 25
defaults:
run:
shell: bash
runs-on: aks-linux-4-cores-16gb
container:
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount:/mount
env:
INSTALL_DIR: /__w/openvino/openvino/install
OPENVINO_TOKENIZERS_REPO: /__w/openvino/openvino/openvino_tokenizers
EXTENSION_BUILD_DIR: /__w/openvino/openvino/build
if: fromJSON(needs.smart_ci.outputs.affected_components).TOKENIZERS
iGPU:
name: iGPU Tests
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_gpu_tests.yml
strategy:
max-parallel: 2
fail-fast: false
matrix:
TEST_TYPE: ['unit', 'func']
with:
device: 'igpu'
test_type: ${{ matrix.TEST_TYPE }}
runner: "[ 'self-hosted', 'igpu' ]"
container: '{"image": "ubuntu:20.04", "volumes": ["/dev/dri:/dev/dri"], "options": "--group-add 109 --group-add 44
--device /dev/dri:/dev/dri"}'
if: fromJSON(needs.smart_ci.outputs.affected_components).GPU
steps:
- name: checkout action
uses: actions/checkout@v4
with:
sparse-checkout: |
.github/actions/setup_python
install_build_dependencies.sh
dGPU:
name: dGPU Tests
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'
show-cache-info: 'false'
- name: Clone OpenVINO Tokenizers
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino_tokenizers'
path: ${{ env.OPENVINO_TOKENIZERS_REPO }}
ref: 'releases/2024/0'
- name: Download OpenVINO package
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}
- name: Extract OpenVINO packages
run: |
pushd ${INSTALL_DIR}
tar -xzf openvino_package.tar.gz -C ${INSTALL_DIR}
popd
#
# Dependencies
#
- name: Install build dependencies
run: ./install_build_dependencies.sh
- name: Install python dependencies
run: |
# wheel packaging
python3 -m pip install -r ${OPENVINO_TOKENIZERS_REPO}/requirements-build.txt
#
# Build
#
- name: Build tokenizers wheel
run: |
source ${INSTALL_DIR}/setupvars.sh
python -m build --wheel --outdir ${EXTENSION_BUILD_DIR} ${OPENVINO_TOKENIZERS_REPO}
env:
CMAKE_ARGS: '-DBUILD_FAST_TOKENIZERS=OFF'
CMAKE_BUILD_PARALLEL_LEVEL: '4'
CMAKE_GENERATOR: 'Unix Makefiles'
#
# Upload build artifacts
#
- name: Upload openvino tokenizers wheel
if: ${{ always() }}
uses: actions/upload-artifact@v3
with:
name: openvino_tokenizers_wheel
path: ${{ env.EXTENSION_BUILD_DIR }}/*.whl
if-no-files-found: 'error'
GPU_Stub:
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_gpu_tests.yml
strategy:
max-parallel: 2
fail-fast: false
matrix:
TEST_TYPE: ['unit', 'func']
with:
device: 'dgpu'
test_type: ${{ matrix.TEST_TYPE }}
runner: "[ 'self-hosted', 'dgpu' ]"
container: '{"image": "ubuntu:20.04", "volumes": ["/dev/dri:/dev/dri"], "options": "--group-add 109 --group-add 44
--device /dev/dri/card0:/dev/dri/card0 --device /dev/dri/renderD128:/dev/dri/renderD128"}'
if: ${{ github.event_name == 'schedule' }}
runs-on: ubuntu-latest
if: fromJSON(needs.smart_ci.outputs.affected_components).GPU
steps:
- name: GPU stub
run: |
echo "This is only a stub to collect statistics of GPU runs filtered by Smart CI.
It will help us to estimate hardware requirements"
shell: bash
Overall_Status:
name: ci/gha_overall_status
needs: [Smart_CI, Build, Debian_Packages, Samples, Conformance, ONNX_Runtime, CXX_Unit_Tests, Python_Unit_Tests, TensorFlow_Layer_Tests,
CPU_Functional_Tests, TensorFlow_Models_Tests_Precommit, PyTorch_Models_Tests, NVIDIA_Plugin, Openvino_tokenizers, iGPU]
needs: [Smart_CI, Build, Debian_Packages, Samples, Conformance, ONNX_Runtime, CXX_Unit_Tests, Python_Unit_Tests,
CPU_Functional_Tests, TensorFlow_Hub_Models_Tests, PyTorch_Models_Tests, NVIDIA_Plugin, Openvino_tokenizers]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:

View File

@ -13,8 +13,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux-arm
cancel-in-progress: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
@ -24,11 +22,10 @@ jobs:
runs-on: ubuntu-latest
outputs:
affected_components: "${{ steps.smart_ci.outputs.affected_components }}"
changed_components: "${{ steps.smart_ci.outputs.changed_components }}"
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: .github/actions/smart-ci
@ -50,42 +47,20 @@ jobs:
echo "${{ toJSON(steps.smart_ci.outputs.affected_components) }}"
shell: bash
Docker:
needs: Smart_CI
runs-on: aks-linux-16-cores-arm-docker-build
container:
image: openvinogithubactions.azurecr.io/docker_build:0.2
volumes:
- /mount:/mount
outputs:
images: "${{ steps.handle_docker.outputs.images }}"
steps:
- name: Checkout
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: ./.github/actions/handle_docker
id: handle_docker
with:
images: |
ov_build/ubuntu_20_04_arm64
ov_test/ubuntu_20_04_arm64
registry: 'openvinogithubactions.azurecr.io'
dockerfiles_root_dir: '.github/dockerfiles'
changed_components: ${{ needs.Smart_CI.outputs.changed_components }}
Build:
needs: [ Smart_CI, Docker ]
needs: Smart_CI
timeout-minutes: 150
defaults:
run:
shell: bash
runs-on: 'aks-linux-16-cores-arm'
container:
image: ${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_arm64 }}
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount:/mount
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_BUILD_TYPE: 'Release'
CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache
@ -107,19 +82,25 @@ jobs:
if: "!needs.smart_ci.outputs.skip_workflow"
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
- name: Clone OpenVINO Contrib
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true'
ref: 'master'
ref: 'releases/2024/0'
#
# Print system info
@ -132,6 +113,25 @@ jobs:
# Dependencies
#
- name: Install build dependencies
run: |
bash ${OPENVINO_REPO}/install_build_dependencies.sh
# default-jdk - Java API
apt install --assume-yes --no-install-recommends default-jdk
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
show-cache-info: 'true'
- name: Install python dependencies
run: |
# For Python API: build and wheel packaging
@ -167,7 +167,6 @@ jobs:
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON \
-DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
-DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
-DOV_CPU_AARCH64_USE_MULTI_ISA=OFF \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
@ -223,8 +222,9 @@ jobs:
- name: Cmake & Build - OpenVINO Contrib
run: |
cmake \
-DBUILD_nvidia_plugin=OFF \
-DCUSTOM_OPERATIONS="calculate_grid;complex_mul;fft;grid_sample;sparse_conv;sparse_conv_transpose" \
-DOPENVINO_EXTRA_MODULES="${OPENVINO_CONTRIB_REPO}/modules/java_api;${OPENVINO_CONTRIB_REPO}/modules/custom_operations" \
-DOPENVINO_EXTRA_MODULES=${OPENVINO_CONTRIB_REPO}/modules \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
@ -247,7 +247,7 @@ jobs:
# Upload build artifacts and logs
#
- name: Upload build logs
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: always()
with:
name: build_logs
@ -256,7 +256,7 @@ jobs:
- name: Upload openvino package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_package
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
@ -264,7 +264,7 @@ jobs:
- name: Upload openvino developer package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_developer_package
path: ${{ env.BUILD_DIR }}/openvino_developer_package.tar.gz
@ -272,7 +272,7 @@ jobs:
- name: Upload openvino js package
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
@ -280,7 +280,7 @@ jobs:
- name: Upload openvino debian packages
if: ${{ 'false' }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_debian_packages
path: ${{ env.BUILD_DIR }}/*.deb
@ -288,7 +288,7 @@ jobs:
- name: Upload openvino tests package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_tests
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
@ -304,109 +304,90 @@ jobs:
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
Samples:
needs: [ Build, Docker, Smart_CI ]
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).samples
uses: ./.github/workflows/job_samples_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
JS_API:
name: OpenVINO JS API
needs: [ Build, Docker, Smart_CI ]
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: ./.github/workflows/job_openvino_js.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_arm64 }}"}'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04"}'
ONNX_Runtime:
name: ONNX Runtime Integration
if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_RT ||
fromJSON(needs.smart_ci.outputs.affected_components).ONNX_FE
needs: [ Build, Smart_CI, Docker ]
# Enable back once https://github.com/microsoft/onnxruntime/pull/19184 is merged
if: ${{ 'false' }}
# if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_RT ||
# fromJSON(needs.smart_ci.outputs.affected_components).ONNX_FE
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_onnx_runtime.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_arm64 }}", "volumes": ["/mount:/mount"], "options": "-e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING"}'
sccache-azure-key-prefix: 'ubuntu20_aarch64_onnxruntime'
Openvino_tokenizers:
name: OpenVINO tokenizers extension
needs: [ Build, Docker, Smart_CI ]
uses: ./.github/workflows/job_tokenizers.yml
with:
runner: 'aks-linux-16-cores-arm'
shell: bash
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_20_04_arm64 }}", "volumes": ["/mount:/mount"]}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
if: fromJSON(needs.smart_ci.outputs.affected_components).TOKENIZERS
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04", "volumes": ["/mount:/mount"], "options": "-e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING"}'
sccache-azure-key-prefix: 'ubuntu22_aarch64_onnxruntime'
CXX_Unit_Tests:
name: C++ unit tests
needs: [ Build, Docker, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_cxx_unit_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
Python_Unit_Tests:
name: Python unit tests
needs: [ Build, Docker, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_python_unit_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}", "volumes": ["/mount:/mount"]}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
TensorFlow_Layer_Tests:
name: TensorFlow Layer Tests
needs: [ Build, Docker, Smart_CI, Openvino_tokenizers ]
uses: ./.github/workflows/job_tensorflow_layer_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
shell: bash
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}", "volumes": ["/mount:/mount"]}'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04", "volumes": ["/mount:/mount"]}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
CPU_Functional_Tests:
name: CPU functional tests
if: fromJSON(needs.smart_ci.outputs.affected_components).CPU.test
needs: [ Build, Docker, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_cpu_functional_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
TensorFlow_Models_Tests:
name: TensorFlow Models tests
TensorFlow_Hub_Models_Tests:
name: TensorFlow Hub Models tests
if: ${{ 'false' }} # TODO: Enable once the dependencies are ready for arm (no tensorflow-text available for arm from PyPI)
# if: fromJSON(needs.smart_ci.outputs.affected_components).TF_FE.test ||
# fromJSON(needs.smart_ci.outputs.affected_components).TFL_FE.test
needs: [ Build, Docker, Smart_CI, Openvino_tokenizers]
uses: ./.github/workflows/job_tensorflow_models_tests.yml
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_tensorflow_hub_models_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}"}'
model_scope: 'precommit'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04"}'
event: ${{ github.event_name }}
PyTorch_Models_Tests:
name: PyTorch Models tests
if: ${{ 'false' }} # TODO: Enable once the dependencies are ready for arm (no tensorflow-text available for arm from PyPI)
# if: fromJSON(needs.smart_ci.outputs.affected_components).PyTorch_FE.test
needs: [ Build, Docker, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_pytorch_models_tests.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_20_04_arm64 }}"}'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04"}'
event: ${{ github.event_name }}
Overall_Status:
name: ci/gha_overall_status_linux_arm64
needs: [Smart_CI, Build, Debian_Packages, Samples, ONNX_Runtime, CXX_Unit_Tests, Python_Unit_Tests, CPU_Functional_Tests,
TensorFlow_Models_Tests, PyTorch_Models_Tests, Openvino_tokenizers, TensorFlow_Layer_Tests]
TensorFlow_Hub_Models_Tests, PyTorch_Models_Tests]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:

View File

@ -13,8 +13,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux-cc
cancel-in-progress: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
@ -24,11 +22,10 @@ jobs:
runs-on: ubuntu-latest
outputs:
affected_components: "${{ steps.smart_ci.outputs.affected_components }}"
changed_components: "${{ steps.smart_ci.outputs.changed_components }}"
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: .github/actions/smart-ci
@ -45,43 +42,15 @@ jobs:
skip_when_only_listed_labels_set: 'docs'
skip_when_only_listed_files_changed: '*.md,*.rst,*.png,*.jpg,*.svg,*/layer_tests_summary/*,*/conformance/*'
- name: Show affected components
run: |
echo "${{ toJSON(steps.smart_ci.outputs.affected_components) }}"
shell: bash
Docker:
needs: Smart_CI
runs-on: aks-linux-4-cores-16gb-docker-build
container:
image: openvinogithubactions.azurecr.io/docker_build:0.2
volumes:
- /mount:/mount
outputs:
images: "${{ steps.handle_docker.outputs.images }}"
steps:
- name: Checkout
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: ./.github/actions/handle_docker
id: handle_docker
with:
images: |
ov_build/ubuntu_22_04_x64_cc
ov_test/ubuntu_22_04_x64
registry: 'openvinogithubactions.azurecr.io'
dockerfiles_root_dir: '.github/dockerfiles'
changed_components: ${{ needs.smart_ci.outputs.changed_components }}
Build:
needs: [Docker, Smart_CI]
needs: Smart_CI
timeout-minutes: 150
defaults:
run:
shell: bash
runs-on: aks-linux-16-cores-32gb
container:
image: ${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_22_04_x64_cc }}
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
volumes:
- /mount:/mount
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
@ -106,19 +75,27 @@ jobs:
if: ${{ !needs.smart_ci.outputs.skip_workflow && github.event_name != 'merge_group' }}
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git
run: |
apt-get update
apt-get install --assume-yes --no-install-recommends git ca-certificates git-lfs
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
- name: Clone test models
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/testdata'
path: ${{ env.MODELS_PATH }}
lfs: 'true'
ref: 'master'
ref: 'releases/2024/0'
#
# Print system info
@ -127,6 +104,32 @@ jobs:
- name: System info
uses: ./openvino/.github/actions/system_info
#
# Dependencies
#
- name: Install build dependencies
run: |
bash ${OPENVINO_REPO}/install_build_dependencies.sh
# use clang as a default compiler
apt --assume-yes install clang
update-alternatives --install /usr/bin/cc cc /usr/bin/clang 100
update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++ 100
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'
- name: Install python dependencies
run: |
# For running ONNX frontend unit tests
@ -149,7 +152,6 @@ jobs:
run: |
cmake \
-G "${{ env.CMAKE_GENERATOR }}" \
-DCMAKE_CXX_STANDARD=20 \
-DBUILD_SHARED_LIBS=OFF \
-DENABLE_TESTS=ON \
-DENABLE_CPPLINT=OFF \
@ -205,14 +207,10 @@ jobs:
install_dependencies/install_openvino_dependencies.sh
popd
cp -v ${OPENVINO_REPO}/temp/tbb/lib/lib* ${INSTALL_TEST_DIR}/tests
pushd ${INSTALL_TEST_DIR}
tar -czvf ${BUILD_DIR}/openvino_tests.tar.gz \
tests/ov_cpu_func_tests \
tests/libopenvino_template_extension.so \
tests/libze_loader.so* \
tests/libhwloc* \
tests/libtbb* \
tests/functional_test_utils/layer_tests_summary/*
popd
@ -220,7 +218,7 @@ jobs:
# Upload build artifacts and logs
#
- name: Upload build logs
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
if: always()
with:
name: build_logs
@ -229,7 +227,7 @@ jobs:
- name: Upload openvino package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_package
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
@ -237,7 +235,7 @@ jobs:
- name: Upload selective build statistics package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_selective_build_stat
path: ${{ env.BUILD_DIR }}/openvino_selective_build_stat.tar.gz
@ -245,7 +243,7 @@ jobs:
- name: Upload OpenVINO tests package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@v3
with:
name: openvino_tests
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
@ -253,14 +251,14 @@ jobs:
CC_Build:
name: Conditional Compilation
needs: [Build, Docker]
needs: Build
timeout-minutes: 20
defaults:
run:
shell: bash
runs-on: aks-linux-16-cores-32gb
container:
image: ${{ fromJSON(needs.docker.outputs.images).ov_build.ubuntu_22_04_x64_cc }}
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04
volumes:
- /mount:/mount
options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
@ -278,22 +276,28 @@ jobs:
SCCACHE_AZURE_KEY_PREFIX: ubuntu22_x86_64_cc_Release
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates git-lfs
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
- name: Clone test models
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/testdata'
path: ${{ env.MODELS_PATH }}
lfs: 'true'
ref: 'master'
ref: 'releases/2024/0'
- name: Download selective build statistics package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@v3
with:
name: openvino_selective_build_stat
path: ${{ env.SELECTIVE_BUILD_STAT_DIR }}
@ -302,6 +306,17 @@ jobs:
run: tar -xvzf ${SELECTIVE_BUILD_STAT_DIR}/openvino_selective_build_stat.tar.gz -C ${SELECTIVE_BUILD_STAT_DIR}
#
# Dependencies
#
- name: Install build dependencies
run: bash ${OPENVINO_REPO}/install_build_dependencies.sh
- name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.4
with:
version: "v0.7.5"
#
# Build
#
@ -336,11 +351,11 @@ jobs:
CPU_Functional_Tests:
name: CPU functional tests
if: fromJSON(needs.smart_ci.outputs.affected_components).CPU.test
needs: [ Docker, Build, Smart_CI ]
needs: [ Build, Smart_CI ]
uses: ./.github/workflows/job_cpu_functional_tests.yml
with:
runner: 'aks-linux-8-cores-32gb'
image: ${{ fromJSON(needs.docker.outputs.images).ov_test.ubuntu_22_04_x64 }}
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:22.04'
Overall_Status:
name: ci/gha_overall_status_linux_cc

View File

@ -16,8 +16,6 @@ concurrency:
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux-riscv
cancel-in-progress: true
permissions: read-all
jobs:
Smart_CI:
runs-on: ubuntu-latest
@ -26,7 +24,7 @@ jobs:
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
sparse-checkout: .github/actions/smart-ci
@ -62,12 +60,10 @@ jobs:
OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_BUILD_DIR: /__w/openvino/openvino/openvino_build
INSTALL_DIR: /__w/openvino/openvino/openvino_install
CONAN_USER_HOME: /mount/caches/ccache/ubuntu22_riscv64_master_release/.conan
CCACHE_REMOTE_DIR: /mount/caches/ccache/ubuntu22_riscv64/${{ github.base_ref || github.ref_name }}
CCACHE_DIR: /__w/openvino/openvino/ccache
CONAN_USER_HOME: /mount/caches/ccache/ubuntu22_riscv64_master/.conan
CCACHE_DIR: /mount/caches/ccache/ubuntu22_riscv64_master
CCACHE_TEMPDIR: /__w/openvino/openvino/ccache_temp
CCACHE_MAXSIZE: 2G
CCACHE_MAXSIZE: 50G
if: ${{ !needs.smart_ci.outputs.skip_workflow && github.event_name != 'merge_group' }}
steps:
- name: Set apt retries
@ -77,7 +73,7 @@ jobs:
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
uses: actions/checkout@v4
with:
path: 'openvino'
@ -88,20 +84,6 @@ jobs:
- name: System info
uses: ./openvino/.github/actions/system_info
- name: Setup ccache
id: ccache_restore
uses: ./openvino/.github/actions/cache
with:
save-always: ${{ github.event_name == 'push' && 'true' || 'false' }}
cleanup-always: ${{ github.event_name == 'push' && 'true' || 'false' }}
cache-size: 10
max-cache-size: 50
cache-path: ${{ env.CCACHE_REMOTE_DIR }}
path: ${{ env.CCACHE_DIR }}
key: ${{ runner.os }}-${{ runner.arch }}-ccache-${{ github.sha }}
restore-keys: |
${{ runner.os }}-${{ runner.arch }}-ccache
- name: Init submodules for non-Conan dependencies
run: |
pushd ${OPENVINO_REPO}
@ -109,10 +91,8 @@ jobs:
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/telemetry
git submodule update --init -- ${OPENVINO_REPO}/src/plugins/intel_cpu
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/open_model_zoo
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/flatbuffers/flatbuffers
popd
#
@ -126,7 +106,7 @@ jobs:
# install compilers to build OpenVINO for RISC-V 64
apt --assume-yes install gcc-riscv64-linux-gnu g++-riscv64-linux-gnu
apt --assume-yes install -yf gcc g++ python3-pip python3-venv python3-dev
apt --assume-yes install gcc g++ python3-pip python3-venv python3-dev
# generic dependencies
apt --assume-yes install cmake ccache ninja-build fdupes patchelf
@ -182,8 +162,6 @@ jobs:
#
# Build
#
- name: Clean ccache stats
run: ccache --zero-stats
- name: CMake - Configure
run: |
@ -195,6 +173,8 @@ jobs:
-DENABLE_INTEL_GPU=ON \
-DENABLE_PYTHON=ON \
-DENABLE_WHEEL=ON \
-DPYTHON_MODULE_EXTENSION=$(riscv64-linux-gnu-python3-config --extension-suffix) \
-DPYBIND11_PYTHON_EXECUTABLE_LAST=${OPENVINO_BUILD_DIR}/env/bin/python3.10 \
-DENABLE_TESTS=ON \
-DENABLE_PYTHON_PACKAGING=ON \
-DENABLE_SYSTEM_PROTOBUF=ON \
@ -231,9 +211,6 @@ jobs:
env:
CMAKE_TOOLCHAIN_FILE: ${{ env.OPENVINO_BUILD_DIR }}/dependencies/conan_toolchain.cmake
- name: Clean ccache
run: ccache --cleanup
Overall_Status:
name: ci/gha_overall_status_linux_riscv
needs: [Smart_CI, Build]

View File

@ -1,470 +0,0 @@
name: Linux Sanitizers (Ubuntu 20.04, Python 3.11)
on:
schedule:
# run daily at 00:00
- cron: '0 0 * * *'
workflow_dispatch:
# pull_request:
concurrency:
# github.ref is not unique in post-commit
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux-sanitizers
cancel-in-progress: true
permissions: read-all
env:
PIP_CACHE_PATH: /mount/caches/pip/linux
PYTHON_VERSION: '3.11'
jobs:
Build:
timeout-minutes: 500
defaults:
run:
shell: bash
runs-on: aks-linux-16-cores-32gb
container:
image: openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04
volumes:
- /mount:/mount
strategy:
max-parallel: 3
fail-fast: false
matrix:
include:
- SANITIZER: 'AddressAndLeak'
SANITIZER_CMAKE_OPTION: '-DENABLE_SANITIZER=ON'
- SANITIZER: 'UndefinedBehavior'
SANITIZER_CMAKE_OPTION: '-DENABLE_UB_SANITIZER=ON'
# - SANITIZER: 'Thread' # Problems with protobuf
# SANITIZER_CMAKE_OPTION: '-DENABLE_THREAD_SANITIZER=ON'
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_BUILD_TYPE: 'Release'
CMAKE_GENERATOR: 'Ninja Multi-Config'
GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib
INSTALL_DIR: /__w/openvino/openvino/openvino_install
INSTALL_TEST_DIR: /__w/openvino/openvino/tests_install
BUILD_DIR: /__w/openvino/openvino/openvino_build
LSAN_IGNORE: /__w/openvino/openvino/openvino/tests/lsan/suppressions.txt
ASAN_IGNORE: /__w/openvino/openvino/openvino/tests/asan/suppressions.supp
CXX: clang++
CC: clang
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git
run: |
apt-get update
apt-get install --assume-yes --no-install-recommends git ca-certificates
- name: Clone OpenVINO
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
- name: Clone OpenVINO Contrib
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true'
ref: 'master'
#
# Print system info
#
- name: System info
uses: ./openvino/.github/actions/system_info
#
# Dependencies
#
- name: Install build dependencies
run: |
bash ${OPENVINO_REPO}/install_build_dependencies.sh
apt --assume-yes install clang lld
- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python
with:
version: ${{ env.PYTHON_VERSION }}
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'
show-cache-info: 'true'
- name: Install python dependencies
run: |
# For Python API: build and wheel packaging
python3 -m pip install -r ${OPENVINO_REPO}/src/bindings/python/wheel/requirements-dev.txt
# For running ONNX frontend unit tests
python3 -m pip install --force-reinstall -r ${OPENVINO_REPO}/src/frontends/onnx/tests/requirements.txt
# For running TensorFlow frontend unit tests
python3 -m pip install -r ${OPENVINO_REPO}/src/frontends/tensorflow/tests/requirements.txt
# For running TensorFlow Lite frontend unit tests
python3 -m pip install -r ${OPENVINO_REPO}/src/frontends/tensorflow_lite/tests/requirements.txt
# For running Paddle frontend unit tests
python3 -m pip install -r ${OPENVINO_REPO}/src/frontends/paddle/tests/requirements.txt
#
# Build
#
- name: CMake configure - OpenVINO
run: |
export ASAN_OPTIONS=halt_on_error=0:suppressions=${ASAN_IGNORE}
export LSAN_OPTIONS=suppressions=${LSAN_IGNORE}:NEOReadDebugKeys=1:DisableDeepBind=1
export CC=clang
export CXX=clang++
cmake \
-G "${{ env.CMAKE_GENERATOR }}" \
-DENABLE_CPPLINT=OFF \
-DENABLE_NCC_STYLE=OFF \
-DENABLE_TESTS=ON \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_SYSTEM_OPENCL=ON \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DCPACK_GENERATOR=TGZ \
-DBUILD_SHARED_LIBS=ON \
-DENABLE_OV_TF_FRONTEND=ON \
-DENABLE_OV_TF_LITE_FRONTEND=ON \
-DENABLE_OV_PADDLE_FRONTEND=ON \
-DENABLE_OV_PYTORCH_FRONTEND=ON \
-DENABLE_OV_ONNX_FRONTEND=ON \
-DENABLE_ONEDNN_FOR_GPU=OFF \
-DCMAKE_COMPILE_WARNING_AS_ERROR=OFF \
${{ matrix.SANITIZER_CMAKE_OPTION }} \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
- name: Cmake build - OpenVINO
run: |
export ASAN_OPTIONS=halt_on_error=0:suppressions=${ASAN_IGNORE}
export LSAN_OPTIONS=suppressions=${LSAN_IGNORE}:NEOReadDebugKeys=1:DisableDeepBind=1
cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
- name: Cmake install - OpenVINO
run: |
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -P ${BUILD_DIR}/cmake_install.cmake
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_TEST_DIR} -DCOMPONENT=tests -P ${BUILD_DIR}/cmake_install.cmake
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -DCOMPONENT=python_wheels -P ${BUILD_DIR}/cmake_install.cmake
- name: Remove unused files to free space
run: rm -rf ${BUILD_DIR}/*
#
# Tests
#
- name: Pack Artifacts
run: |
pushd ${INSTALL_DIR}
tar -czvf ${BUILD_DIR}/openvino_package.tar.gz *
popd
pushd ${INSTALL_TEST_DIR}
tar -czvf ${BUILD_DIR}/openvino_tests.tar.gz *
popd
#
# Upload build artifacts
#
- name: Upload openvino package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: openvino_package_${{ matrix.SANITIZER }}
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
if-no-files-found: 'error'
- name: Upload openvino tests package
if: ${{ always() }}
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: openvino_tests_${{ matrix.SANITIZER }}
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
if-no-files-found: 'error'
CXX_Unit_Tests:
name: C++ unit tests
if: always()
needs: Build
timeout-minutes: 100
runs-on: 'aks-linux-16-cores-32gb'
container:
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
defaults:
run:
shell: bash
strategy:
max-parallel: 3
fail-fast: false
matrix:
include:
- SANITIZER: 'AddressAndLeak'
# - SANITIZER: 'UndefinedBehavior' # the UB library is not linked properly at the Build stage
# - SANITIZER: 'Thread' # Problems with protobuf at the Build stage
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
CC: clang
CXX: clang++
steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: ${{ format('openvino_package_{0}', matrix.SANITIZER) }}
path: ${{ env.INSTALL_DIR }}
- name: Download OpenVINO tests package
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: ${{ format('openvino_tests_{0}', matrix.SANITIZER) }}
path: ${{ env.INSTALL_TEST_DIR }}
# Needed as ${{ github.workspace }} is not working correctly when using Docker
- name: Setup Variables
continue-on-error: true
run: |
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "ASAN_OPTIONS=halt_on_error=0:suppressions=$GITHUB_WORKSPACE/openvino/tests/asan/suppressions.supp" >> "$GITHUB_ENV"
echo "LSAN_OPTIONS=suppressions=$GITHUB_WORKSPACE/openvino/tests/lsan/suppressions.txt:NEOReadDebugKeys=1:DisableDeepBind=1" >> "$GITHUB_ENV"
- name: Extract OpenVINO packages
run: |
pushd $INSTALL_DIR
tar -xzf openvino_package.tar.gz -C $INSTALL_DIR
popd
pushd $INSTALL_TEST_DIR
tar -xzf openvino_tests.tar.gz -C $INSTALL_DIR
popd
- name: Install dependencies (Linux)
run: |
$INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -c=gpu -y
apt update && apt --assume-yes install clang lld
- name: Fetch Sanitizer Suppression Lists
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
sparse-checkout: |
tests/lsan/suppressions.txt
tests/asan/suppressions.supp
sparse-checkout-cone-mode: false
path: 'openvino'
#
# Tests
#
- name: OpenVINO Core Unit Tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OVCoreUT.xml
- name: OpenVINO Inference Functional Tests
if: ${{ 'false' }} # Ticket: 134410
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_inference_functional_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-InferenceFunc.xml
- name: OpenVINO Inference Unit Tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_inference_unit_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-InferenceUnit.xml
- name: Low Precision Transformations Tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_lp_transformations_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-LpTransformations.xml
- name: OpenVINO Conditional compilation tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_conditional_compilation_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ConditionalCompilation.xml
- name: IR frontend tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_ir_frontend_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-IRFrontend.xml
- name: PaddlePaddle frontend tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/paddle_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-PaddleTests.xml
- name: ONNX frontend tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_onnx_frontend_tests --gtest_print_time=1 \
--gtest_filter=-*IE_GPU* \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ONNXFrontend.xml
- name: TensorFlow Common frontend tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_tensorflow_common_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-TensorFlowCommonFrontend.xml
- name: TensorFlow frontend tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_tensorflow_frontend_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-TensorFlowFrontend.xml
- name: TensorFlow Lite frontend tests
if: ${{ 'false' }} # Ticket: 134416
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_tensorflow_lite_frontend_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-TensorFlowLiteFrontend.xml
- name: Transformations func tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_transformations_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-Transformations.xml
- name: Common test utils tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_util_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-CommonUtilTests.xml
- name: Snippets func tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_snippets_func_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-SnippetsFuncTests.xml
- name: CPU plugin unit tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_cpu_unit_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-CPUUnitTests.xml
- name: ov_subgraphs_dumper_tests tests
if: ${{ 'false' }} # Ticket: 134419
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_subgraphs_dumper_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_subgraphs_dumper_tests.xml
- name: Template OpImpl tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_op_conformance_tests --gtest_print_time=1 --device=TEMPLATE --gtest_filter=*OpImpl*\
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OpImplTests.xml
- name: AUTO unit tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_auto_unit_tests --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_auto_unit_tests.xml
- name: AUTO func Tests
if: always()
run: |
source ${{ env.INSTALL_DIR }}/setupvars.sh
${{ env.INSTALL_TEST_DIR }}/ov_auto_func_tests --gtest_print_time=1 \
--gtest_output=xml:${{ env.INSTALL_TEST_DIR }}/TEST-ov_auto_func_tests.xml
- name: Template plugin func tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_template_func_tests --gtest_print_time=1 \
--gtest_filter=*smoke* \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-TemplateFuncTests.xml
- name: OpenVINO C API tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_capi_test --gtest_print_time=1 \
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OpenVINOCAPITests.xml
- name: AutoBatch unit tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_auto_batch_unit_tests --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_auto_batch_unit_tests.xml
- name: AutoBatch func tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_auto_batch_func_tests --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_auto_batch_func_tests.xml --gtest_filter="*smoke*"
- name: Proxy Plugin func tests
if: always()
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_proxy_plugin_tests --gtest_print_time=1 --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OVProxyTests.xml
- name: Hetero unit tests
if: always()
run: |
source ${{ env.INSTALL_DIR }}/setupvars.sh
${{ env.INSTALL_TEST_DIR }}/ov_hetero_unit_tests --gtest_print_time=1 --gtest_output=xml:${{ env.INSTALL_TEST_DIR }}/TEST-OVHeteroUnitTests.xml
- name: Hetero func tests
if: ${{ 'false' }} # Ticket: 134425
run: |
source ${INSTALL_DIR}/setupvars.sh
${INSTALL_TEST_DIR}/ov_hetero_func_tests --gtest_print_time=1 --gtest_output=xml:${INSTALL_TEST_DIR}/TEST-OVHeteroFuncTests.xml --gtest_filter="*smoke*"
- name: Upload Test Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
if: ${{ !cancelled() }}
with:
name: test-results-cpp
path: ${{ env.INSTALL_TEST_DIR }}/TEST*.xml
if-no-files-found: 'error'

Some files were not shown because too many files have changed in this diff Show More