Compare commits

...

111 Commits

Author SHA1 Message Date
Maxim Vafin fb35278525
Change opt-125m-gptq model (#24901)
# Conflicts:
#	tests/model_hub_tests/torch_tests/test_hf_transformers.py

### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-06-10 11:08:09 +02:00
Xuejun Zhai d9ef8441b1
Port 24606 to release 202303 (#24796) 2024-06-03 09:35:49 +04:00
Yuan Hu e9f8604430
port PR 24562 to release/2023/3 (#24670)
### Details:
 - *port PR #24562 to release 2023.3*
 - *add support for InferAPI1 and api_version*

### Tickets:
-
*[issue-24509](https://github.com/openvinotoolkit/openvino/issues/24509)*

---------

Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
2024-05-28 07:37:02 +08:00
Xuejun Zhai 78b55038b5
Port PR 24018 to release/2023/3 (#24371)
### Details:
 - Port PR2418 to release 2023.3
 - *...*

### Tickets:
 - *CVS-132745*

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2024-05-07 15:11:35 +04:00
Ilya Lavrenov d3cb701018
Added check for conflicting DEB / RPM versions (#24275)
### Details:
 - Added check for conflicting DEB / RPM versions
2024-04-28 01:19:34 +04:00
Ilya Lavrenov 6b3f564fe8
Port openvino-telemetry support in Debian package to 2023.3 (#24257)
### Ported PRs:
- https://github.com/openvinotoolkit/openvino/pull/23917
- https://github.com/openvinotoolkit/openvino/pull/24060
- https://github.com/openvinotoolkit/openvino/pull/23816

---------

Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
2024-04-28 00:55:35 +04:00
Xiping Yan f32c023c91
[CPU]Fix regression issue. (#23904)
### Details:
 - *Add callback fun param for selectPreferPrimitiveDescriptor*
 - *Update Snippet node selectPreferPrimitiveDescriptor algorithm.*
 - *Add test*

### Tickets:
 - *137307*

---------

Signed-off-by: xipingya <xiping.yan@intel.com>
2024-04-26 10:21:25 +02:00
Maciej Smyk 53ca00bc81
[DOCS] release policy for 2023.3 (#24185)
Added Release Policy article to docs. Updated with changes from the
https://github.com/openvinotoolkit/openvino/pull/24145
2024-04-23 18:19:37 +02:00
Maciej Smyk 0d6fadd2da
[DOCS] Fixing Tensor Names examples in Model Representation for 2023.3 (#24173)
Port from https://github.com/openvinotoolkit/openvino/pull/23482

* Fixed examples of Tensor Names,
* Added an example of ov::element::Type for both Python and C++
2024-04-22 17:20:08 +02:00
Ilya Lavrenov 28c1585d65
Port several PRs from master to 2023.3 LTS (#23461)
### Ported PRs:
- https://github.com/openvinotoolkit/openvino/pull/22928
- https://github.com/openvinotoolkit/openvino/pull/22963
- https://github.com/openvinotoolkit/openvino/pull/23345
- https://github.com/openvinotoolkit/openvino/pull/23460

---------

Co-authored-by: Andrey Babushkin <andrey.babushkin@intel.com>
2024-04-16 18:23:32 +04:00
Maksim Kutakov 15b0f99132
[CPU] fix gen_pattern::AttrAny::equal_to (#23689)
### Details:
Backport https://github.com/openvinotoolkit/openvino/pull/21890 from
master

### Tickets:
 - CVS-133621

Co-authored-by: Tingqian Li <tingqian.li@intel.com>
2024-04-16 12:35:46 +02:00
Maksim Kutakov e6fbca158a
[CPU] Fix binary PReLU spare register search (#23910)
### Details:
Fix the algorithm of assigning an auxiliary SIMD register in the binary
PReLU post op JIT injector.

OneDNN fork PR: https://github.com/openvinotoolkit/oneDNN/pull/240

### Tickets:
 - CVS-137492
2024-04-16 12:35:35 +02:00
Karol Blaszczak 89e0a839d0
[DOCS] old fix for yolo article 23.3 (#23933) 2024-04-10 12:13:49 +02:00
Tatiana Savina feaeca5191
[DOCS] Port Add MS Windows version for NPU to 23.3 (#23872)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-04-04 14:53:13 +02:00
Steve Yoo ddef44e96d
[GPU] Skip concat_input_order (#23787)
### Details:
- *Skip concat_input_order for an element which lies outside of the
buffer range*

### Tickets:
 - *135790*
2024-04-02 20:56:35 -07:00
Sungeun Kim c4ada32cc8
[GPU] add reorder between mvn and conv. (#23670) (#23765)
- add reorder to avoid inserting padding to mvn before ocl conv.
- mvn does not support padding.

### Tickets:
 - 136385
2024-04-03 10:01:34 +09:00
Sebastian Golebiewski 769fa5764e
[DOCS] Update code examples with imports and save_model for for 23.3 (#23684)
Updating code examples with imports for ``convert_model`` and adding an
example of how to save a converted model. This PR addresses the 120254
ticket.
Porting: https://github.com/openvinotoolkit/openvino/pull/23654
2024-03-26 13:42:30 +01:00
Andrzej Kopytko 687c648120
[DOCS] update banner for 2023.3 (#23656)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-25 13:44:49 +01:00
Andrzej Kopytko beff6958a9
[Docs] restore banner information (#23623)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-22 09:36:06 +01:00
Andrzej Kopytko 385782df37
[DOCS] Added off on click for 2023.3 (#23467)
Port for 
https://github.com/openvinotoolkit/openvino/pull/23411
2024-03-14 15:16:15 +01:00
tadamczx 655a72607f
[DOCS] Python API version fix (#23403)
CMakeLists.txt update, to be able to build right version of Python API.
2024-03-13 07:05:06 +01:00
Andrzej Kopytko beb158f6f0
Docs remove canonical tag from 2023.3 (#23375)
### Details:
 - *item1*
 - *...*

### Tickets:
 - *ticket-id*
2024-03-12 09:43:03 +01:00
Maciej Smyk 0c9ad9ede5
[DOCS] Fix of home tab title for 2023.3 (#23367)
Jira: 133084

* Fixed the home page tab title name.
* Fixed the missing line in banner
2024-03-11 11:51:35 +01:00
Wang, Yang 735a3ba80a
Update port name checking logic to 2023.3 (#23339)
Port from https://github.com/openvinotoolkit/openvino/pull/23081 to
2023.3
### Details:
 - *item1*
 - *...*

### Tickets:
 - 122932
2024-03-08 13:12:19 +04:00
Anastasia Kuporosova c4034018fd
[PyOV] Cherry-picked convert ExecutionMode enum to any (#23296) 2024-03-07 11:02:31 +04:00
Chen Peter 004a6e9cf2
Map configuration to API 2.0 properties (#23161)
### Details:
- *Create table to map some of the Inference Engine configurations to
API 2.0 C++/Python properties*
- *Provide configurations/properties header files for other
configurations/properties*

### Tickets:
 - *CVS-94044*

---------

Signed-off-by: Peter Chen <peter.chen@intel.com>
2024-03-01 11:25:44 +04:00
River Li 0671bf3161
Port PR-23100 for Windows unicode issue (#23120)
### Details:
 - solve windows unicode inconsistent issue
 - add more test case

### Tickets:
 - Closes https://github.com/openvinotoolkit/openvino/issues/23072
 - CVS-128116
2024-02-28 11:08:13 +04:00
Pawel Raasz d02cd0c143
[core] Restore OV exception create function (#23014)
### Details:
- Restore `create` member function in OV exception classes to be
compatible with 2023.2 API.

### Tickets:
 - [CVS-133319](https://jira.devtools.intel.com/browse/CVS-133319)
2024-02-26 13:21:44 +04:00
Karol Blaszczak 7fe8b03cba
[DOCS] benchmark fix table (#23062) 2024-02-23 20:19:17 +01:00
Andrzej Kopytko 381a550f74
DOCS Udpate banchmark files to 2023.3 (#23040)
Port from https://github.com/openvinotoolkit/openvino/pull/23029 to
2023.3
2024-02-23 10:07:37 +01:00
Andrzej Kopytko 51055ef9e1
[DOCS] upgrade chart js to v4 (#23020)
port to https://github.com/openvinotoolkit/openvino/pull/23008
2024-02-22 13:41:53 +01:00
River Li 6758735364
[Core] port PR-22525 and PR-22804 (#22554)
### Details:
 - Port: https://github.com/openvinotoolkit/openvino/pull/22525
 - Port: https://github.com/openvinotoolkit/openvino/pull/22804

### Tickets:
 - CVS-131048

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Vitaliy Urusovskij <vitaliy.urusovskij@intel.com>
2024-02-19 14:24:31 +04:00
Ilya Lavrenov 376428b59a
Ported to 2023.3 from master (#22899)
### Ported from master:
- https://github.com/openvinotoolkit/openvino/pull/22816
- https://github.com/openvinotoolkit/openvino/pull/22847
- https://github.com/openvinotoolkit/openvino/pull/22866

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-19 11:18:24 +04:00
Sebastian Golebiewski ee39aaaa84
[DOCS] Updating Interactive Tutorials for 23.3 (#22862)
Updating Interactive Tutorials - adding new notebooks. Porting:
https://github.com/openvinotoolkit/openvino/pull/22815
2024-02-15 15:35:33 +01:00
Ilya Lavrenov c17db5989c
Ported several PRs from master to 2023.3 LTS (#22843)
### Ported:
- https://github.com/openvinotoolkit/openvino/pull/22606
- https://github.com/openvinotoolkit/openvino/pull/22734
- https://github.com/openvinotoolkit/openvino/pull/22776
- https://github.com/openvinotoolkit/openvino/pull/22820
- https://github.com/openvinotoolkit/openvino/pull/22827

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrei Kashchikhin <andrey.kashchikhin@intel.com>
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-15 11:30:04 +04:00
Vishniakov Nikolai b03c36fee3
[OV JS] Port of Fix npm dependencies (#22830)
Port of:
https://github.com/openvinotoolkit/openvino/pull/22810

To fix this issue:
https://github.com/openvinotoolkit/openvino/security/dependabot/514
2024-02-14 17:39:18 +01:00
Sebastian Golebiewski 4bf8a469dc
[DOCS] Removing the note about GPU support for static models only (#22803)
Removing the note about GPU support for static models only, as GPU now
supports both type of shapes. Porting:
https://github.com/openvinotoolkit/openvino/pull/22802
2024-02-13 08:20:45 +01:00
Karol Blaszczak bed27aa64d
[DOCS] tiny fix for ovms linking (#22755) 2024-02-09 12:29:57 +01:00
Anastasia Kuporosova 493539be12
remove flake8-commas requirement (#22611)
### Details:
 - cherry-picked https://github.com/openvinotoolkit/openvino/pull/22546

### Tickets:
 - *ticket-id*
2024-02-09 12:10:03 +04:00
Karol Blaszczak 1886fc6d31
[DOCS] review for 22356 and 22410 (#22466)
relates to:
https://github.com/openvinotoolkit/openvino/pull/22410
https://github.com/openvinotoolkit/openvino/pull/22356

---------

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
2024-02-08 18:56:53 +01:00
Pavel Esir 77a0b916ba
Keep compressed constants produced by FW #22095 (#22096)
### Details:
 - keep compressed constants produced by FW
 - cherry pick of #22095

### Tickets:
 - CVS-119139
2024-02-08 07:16:44 +04:00
Maciej Smyk f33953bb0c
[DOCS] Fix for xml comments + ScatterUpdate Inputs for 23.3 (#22624)
Port from https://github.com/openvinotoolkit/openvino/pull/22623

- Fixed xml comments throughout the docs.
- Fixed ScatterUpdate Inputs paragraph.
2024-02-06 14:06:53 +01:00
River Li af87a9c5e4
[CAPI] Port PR-22579 (#22596)
https://github.com/openvinotoolkit/openvino/pull/22579
2024-02-04 06:15:46 +01:00
Andrzej Kopytko 6683952c0a
[DOCS] Benchmark Latency as default - port (#22622)
change from master
2024-02-02 12:24:59 +01:00
Alexander Suvorov 28e80c22cf
[DOCS] Update selector tool 23.3 (#22605) 2024-02-01 18:57:34 +01:00
Ilya Lavrenov 7a0f7481cc
Ported several PRs from master to 2023.3 (#22528)
* Bump actions/cache from 3 to 4 (#22224)

Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Disable proxy for GPU plugin (#22235)

* Disable proxy for GPU

* Fix

* Ported conda-forge patch to work with latest protobuf (#22277)

* Bump sccache version (#22282)

* [GHA] Set SCCACHE_IGNORE_SERVER_IO_ERROR (#22346)

* [GHA] Change default port for sccache server to avoid collisions (#22364)

* Change Windows CC workflow but based on last successful WinCC commit (#22469)

* changed linux cc build timeout (#22561)

* Bump mozilla-actions/sccache-action from 0.0.3 to 0.0.4 (#22552)

Bumps [mozilla-actions/sccache-action](https://github.com/mozilla-actions/sccache-action) from 0.0.3 to 0.0.4.
- [Release notes](https://github.com/mozilla-actions/sccache-action/releases)
- [Commits](https://github.com/mozilla-actions/sccache-action/compare/v0.0.3...v0.0.4)

---
updated-dependencies:
- dependency-name: mozilla-actions/sccache-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [CI] [GHA] Download `ninja` from GitHub instead of installing through `choco` in Windows workflows (#22570)

* download ninja from github

* add ninja to path

* add note

* add retries

* [CI] [GHA] Add retries for APT, update deadsnakes action (#22515)

* update setup_python version, add retries for apt in Linux workflows

* account for sudo

* use sudo via sh

* bump to v5 (#22505)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Oleg Pipikin <oleg.pipikin@intel.com>
Co-authored-by: Andrey Babushkin <andrey.babushkin@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Andrei Kashchikhin <andrey.kashchikhin@intel.com>
2024-02-01 17:37:27 +04:00
Sebastian Golebiewski 91cde333cb
[DOCS] Port docs patches for 23.3 (#22590)
* Porting 21679 - fixing typo

* Porting 22571 - specify number of jobs in build
2024-02-01 14:02:35 +04:00
Maciej Smyk 4cf6f57fb5
comment fix (#22557) 2024-01-31 14:02:50 +01:00
Maciej Smyk ffb5270986
[DOCS] ref fix + benchmark fix + ov:state_network fix (#22529) 2024-01-30 10:38:47 +01:00
Zlobin Vladimir ec4f36b015
Fix grammar (#22457) 2024-01-30 01:21:31 +04:00
Sebastian Golebiewski d9faf192a6
Removing NPU plugin specific properties (#22512) 2024-01-29 15:17:04 +01:00
Karol Blaszczak f52de9434b
[DOCS] conformance tables update (#22459) 2024-01-29 13:45:44 +01:00
Sebastian Golebiewski 9cd1afbe96
Updating reference to Inference Modes (#22502) 2024-01-29 13:18:03 +01:00
Sebastian Golebiewski 935533429f
Fix Ubuntu version in NPU article (#22499) 2024-01-29 13:01:17 +01:00
Sebastian Golebiewski af64de8add
[DOCS] Updating interactive tutorials for 23.3 (#22495)
* Updating Interactive Tutorials

* Updating Tutorials
2024-01-29 11:03:24 +01:00
Ivan Tikhonov 39fc555364
Stateful Models Documentation Update (#22410)
* Squashed commits: Updating Stateful Models Documentation

* Apply suggestions from code review

Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* update illustrations

* Fix links; change illustrations format to svg; apply review comments

* add missing files

* Fix missprints; apply review comments; fix links

* fix missprints

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/stateful_models_intro/ways_to_get_stateful_model.rst

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/stateful_models_intro/ways_to_get_stateful_model.rst

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/stateful_models_intro/ways_to_get_stateful_model.rst

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/stateful_models_intro/ways_to_get_stateful_model.rst

* fix illustrations

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/stateful_models_intro/ways_to_get_stateful_model.rst

* Fix the remaining links, add a correct illustration for makestateful transformation, refactoring

---------

Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2024-01-29 13:35:05 +04:00
Karol Blaszczak 48d5d4b79c
[DOCS] excell link restructure (#22463) 2024-01-26 16:55:27 +01:00
Tatiana Savina 8ea3f717cb
[DOCS] Fix nightly link (#22431)
* fix nightly link

* change link
2024-01-25 17:06:24 +01:00
Karol Blaszczak b1c8402eaa
[DOCS] fix link (#22404) 2024-01-25 14:03:19 +01:00
Sebastian Golebiewski 44d661a0ea
[DOCS] Fixing reference to PyTorch conversion for 23.3 (#22393)
* Fixing reference to PyTorch conversion
* Fixing formatting in Release Notes
* Removing Openvino Supported Models article
2024-01-24 16:19:19 +01:00
Sergey Lyalin ad379d7dd5
Updated Optimize and Deploy Generative AI Models documentation article (#22356)
* Updated Optimize and Deploy Generative AI Models article with stateful models information and new examples/resources

* Apply suggestions from code review

Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>

* Apply suggestions from code review

Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>

---------

Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
2024-01-24 18:20:11 +04:00
Maciej Smyk c4d1b11ee8
[DOCS] NPU table fix for 23.3 2024-01-24 14:44:06 +01:00
Karol Blaszczak 9645660166
[DOCS] NPU further tweaks (#22385) 2024-01-24 13:31:59 +01:00
Maciej Smyk 2d6b7e2b80
Update NPU.rst (#22377) 2024-01-24 11:21:27 +01:00
Andrzej Kopytko 0390d68ddb
[DOCS] quick fix graph.js to 23.3 2024-01-24 11:17:38 +01:00
Karol Blaszczak 489cc9304d
[DOCS] final benchmark data + conf version (#22375)
* [DOCS] final benchmark data
* conf.py update 23.3
2024-01-24 11:09:17 +01:00
Karol Blaszczak 900d7f8048
[DOCS] NPU update (#22373) 2024-01-24 09:44:14 +01:00
Karol Blaszczak 0245affbb4
[DOCS] releasenotes 23.3 update (#22362) 2024-01-24 08:34:59 +01:00
Sebastian Golebiewski 7721fa7603
[DOCS] Porting Benchmark Changes to 23.3 (#22357)
* change benchmarks to 23.3

* Updating benchmarks - porting 22353

* Porting 22353

---------

Co-authored-by: Kopytko, AndrzejX <andrzejx.kopytko@intel.com>
2024-01-23 17:52:59 +01:00
Tatiana Savina 1207ec4277
[DOCS] Docs review (#22340)
* add changes

* remove article
2024-01-23 14:32:31 +01:00
Andrzej Kopytko d9d4c9dacd
Main changes without valid data (#22065) (#22099) 2024-01-23 14:22:04 +01:00
Andrzej Kopytko a9d502b315
[Docs] benchmarks precisions port 2023 3 (#22319)
* [DOCS] update

* added rst update
2024-01-23 13:35:55 +01:00
Maciej Smyk 57a9c41a9a
[DOCS] Removing broken links from Model Representation article for 23.3 (#22344)
* Update model_representation.rst

* Update model_representation.rst
2024-01-23 13:00:12 +01:00
Tatiana Savina dd0a833d68
update ov version (#22342) 2024-01-23 12:02:15 +01:00
Vishniakov Nikolai d5c91c758c
Fix wstring join_paths (#22271) (#22305)
* Fix wstring join_paths

* Fix codestyle
2024-01-23 03:10:12 +04:00
Mikhail Ryzhov ecbb1387ab
increased timeout (#22306) 2024-01-22 22:04:40 +04:00
Andrzej Kopytko 45b501bb6f
[Docs] graph.js bf16 prec port to 2023.3 (#22317)
* Main changes without valid data

* fix
2024-01-22 16:08:27 +01:00
Alexander Kozlov aeda8aae2e
[DOCS] Updated weight compression documentation (#22063)
authored by alexander.kozlov@intel.com
2024-01-22 12:52:52 +01:00
Tatiana Savina b58d9cede7
add save model info (#22267) (#22269) 2024-01-19 14:49:17 +01:00
Ilya Lavrenov d19307ba13
Ported changes from master (#22193)
* [CI] [GHA] Introduce JS API as a part of the existing workflows (#21898)

* add js api to linux

* try inside the ov repo

* use rel path

* use a separate job for js api

* correct command formatting

* add missing var

* use spacing

* mv js building

* add node installing

* add to windows

* check pwsh and cmd running npm

* add smart CI conditions; disable for win

* use node version as env var

* extract js job into a separate workflow, add to other *nix

* fix input name

* Activate js bindings tests for arm64

* upload ov js package

* correct formatting

* add missing syntax

---------

Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>

* Cmake Python build option flags should be added to the command in step #3 not step #4. I fixed the typo (#21993)

* [CI] [GHA] [JS API] Remove explicit default values settings in Linux ARM64 `cmake` (#22019)

* rm explicit default values settings

* Activate mac arm64 js api check

* Specify test run

---------

Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>

* [OV JS] Activate validation for mac x86 (#22035)

* Extend validation for mac x86

* Remove extra params

* fixed broken doc links (#22088)

Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>

* [GHA] Update MO deps (#22130)

* [GHA] Update MO deps

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Update .github/components.yml

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>

* Avoid DOWNLOAD_EXTRACT_TIMESTAMP warning (#22135)

* Avoid DOWNLOAD_EXTRACT_TIMESTAMP warning

* Change applying policy condition

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* Fixed API validator search (#22136)

* [OV JS] Conditional enabling of JS API (#22139)

* Disable js api building for vcpkg

* Disable JS API by default

* Add disable JS API conditions in features.cmake

* Update cmake/features.cmake

* Update src/bindings/js/CMakeLists.txt

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* Fixed GHSA-h5c8-rqwp-cp95 (#22159)

* [PyOV][SAMPLES] Fix bugbear issue B038 (#22183)

* Fixed compilation on GHA CI

* Decrease number of workers for ONNX Model tests to prevent OOM kills (#22243)

* Decrease number of workers for ONNX Model tests to prevent OOM kills

* Try to use "-n auto" also

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Andrei Kashchikhin <andrey.kashchikhin@intel.com>
Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>
Co-authored-by: fredrickomondi <omondifredrick@gmail.com>
Co-authored-by: Santhosh Mamidisetti <92091342+SANTHOSH-MAMIDISETTI@users.noreply.github.com>
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: Andrey Babushkin <andrey.babushkin@intel.com>
2024-01-19 16:46:19 +04:00
Sebastian Golebiewski 187e262c92
[DOCS] Updating templates for Binder and Google Colab buttons for 23.3 (#22238)
* Update for Binder and Google Colab buttons

* change image source
2024-01-19 07:55:20 +01:00
Alexander Suvorov aeff428378
[DOCS] Update 2023.3.0 Selector Tool (#22218)
* Update selector tool
2024-01-17 19:48:54 +01:00
Sergey Lyalin 3dd41ff8c3
String Tensors Basic Documentation (#22097)
* Basic documentation of how to use string tensors

* Fix typo: str_data -> bytes_data

* More clarification about UTF-8 and represenation capabilities

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/string_tensors.rst

Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>

* Apply suggestions from code review

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply suggestions from code review

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>

* Applied code review suggestion about 'floating-point'

* Update docs/articles_en/openvino_workflow/running_inference_with_openvino/string_tensors.rst

* Fixed a snippet tab rendering and items formatting in Additional Reasources section

---------

Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
2024-01-17 19:02:16 +04:00
Sebastian Golebiewski 10cae14374
Port 21661 Merge Samples Articles Language Versions (#22187) 2024-01-17 13:27:12 +04:00
Sebastian Golebiewski af4b99e7cf
Port 22006 Updating options for torch.compile (#22188) 2024-01-16 16:51:20 +01:00
Xuejun Zhai 7b782956ab
[CAPI][TEST] fix cancel api test with infer finished before cancel call (#22175)
* [CAPI][TEST] fix cancel api test with infer finished before cancel call

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

* fix error

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

---------

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
2024-01-16 13:17:13 +00:00
Karol Blaszczak 1ef753a590
[DOCS] torchvision and optimization section (#22142)
port: https://github.com/openvinotoolkit/openvino/pull/22043
2024-01-15 08:20:18 +01:00
Karol Blaszczak e64ec3ff5a
[DOCS] fix npu and install 23.3 (#22125) 2024-01-12 17:51:51 +01:00
Sebastian Golebiewski 258dde9f52
Fix images for notebooks (#22131) 2024-01-12 14:18:51 +01:00
River Li 7c37e0916c
[CAPI] avoid random timeout issue in test cases (#22086) 2024-01-11 14:34:14 +04:00
Ilya Lavrenov b981887d6f
Compatibility with newer OpenCL versions (#22051) 2024-01-10 07:31:17 +00:00
Maciej Smyk 47c49e7e84
[DOCS] Hyperlink 23.3 update for 23.3 (#21909)
* link update

* link update

* link update

* Update dev_api_references.rst

* Update dev_api_references.rst

* Update apidoc.py

* notebooks links

* notebook link update

* filename update

* link update

* link update api
2024-01-08 14:37:37 +01:00
Luo Cheng ceeafaf64f
[CPU] Support group beam search (#21983)
* support group beam search

* support dyn batch without set_state

* apply review comments

* strides may be incorrect when batch changed

---------

Co-authored-by: Yu Xu <yu.xu@intel.com>
2024-01-08 16:42:05 +04:00
River Li 49231c0196
[CAPI] fixed ov_infer_request_wait_for return value issue (#22014) 2024-01-08 16:41:21 +04:00
Ilya Lavrenov 38ee887911
Ported changes from master to 2023.3 (#22012)
* [JS OV] Specify JS API GitHub config (#21785)

* Extend CODEOWNERS by js api responsibles

* Extend smart ci by js component

* Fix codeowners

* Fix component definition

* Update components.yml

* Update components.yml

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>

* [GHA] Fix smart ci for docs changes in linux arm64 (#21798)

Update linux_arm64.yml

---------

Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
2024-01-08 16:33:58 +04:00
Maciej Smyk 429b89f0e4
Debian9 update to Ubuntu 18.04 (#21935) 2024-01-08 15:39:37 +04:00
Vitaliy Urusovskij 1df3008870
Fix `INFER_NOT_STARTED` infer status code (#21939)
* Fix `INFER_NOT_STARTED` infer status code

* Update `returnDeviceBusyOnSetBlobAfterAsyncInfer` test
to check `INFER_NOT_STARTED` status only

* Update `wait` tests with `INFER_NOT_STARTED` check
2024-01-08 15:35:20 +04:00
Maxim Vafin 7835368489
[DOCS] Update supported ops by PT FE (#21961) 2024-01-08 11:11:16 +01:00
Maxim Vafin cc8fadf187
[DOCS] Describe dict support (#21923)
* [DOCS] Describe dict support

* Apply suggestions from code review

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>

---------

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
2024-01-08 11:11:05 +01:00
Pavel Durandin a741d5a8ba
Fix minus axis case for kv cache primitive buffer fusing (#21958)
Co-authored-by: Taylor Lee <taylor.lee@intel.com>
2024-01-05 09:22:42 +09:00
Alina Kladieva ca31765d1a
Bump OMZ submodule (#21943)
unlimit pyparsing to fix CVS-128637
2024-01-04 09:51:29 +01:00
Roman Lyamin 112a16a056
[GPU] Small fix for dynamic_shape_gather_opts pass (#21807) (#21882) 2023-12-28 14:48:38 +04:00
Roman Lyamin 7d948bf596
[GPU] Add set_arguments() call in case of dynamic dependencies of static inst (#21664) (#21884)
Co-authored-by: Sergey Shlyapnikov <sergey.shlyapnikov@intel.com>
2023-12-28 14:48:13 +04:00
Maciej Smyk 4e15cc0b7f
Port from 21815 (#21839) 2023-12-27 16:01:33 +01:00
Wilson Seok e1bd2a7b55
Revert "[GPU] Fix accuracy problems brought by permute_f_y kernel (#20241)" (#21830) (#21876)
This reverts commit 6719133af2.

Co-authored-by: Pavel Durandin <pavel.durandin@intel.com>
2023-12-27 18:31:41 +04:00
Surya Siddharth Pemmaraju 8a5e6b6738
Fixed a bug in input validation for torch.compile options (#21835)
* Fixed bug in input validation for torch.compile options

* Added default device if device is None

* Addressed PR comments
2023-12-22 14:26:08 +04:00
Sebastian Golebiewski 81b5895d24
Move Troubleshooting Reshape article to legacy section (#21837) 2023-12-22 11:15:01 +01:00
Maciej Smyk e37569ffd5
[DOCS] Removal of unused images + fixing image directives for 23.3 (#21818)
* removal of unused images

* removal of duplicate image files

* removal of unused files + fix for image directive

* removal of unused images

* removal of unused images + fix for protecting_model.rst

* overview document graph update
2023-12-21 13:14:37 +01:00
Karol Blaszczak 07fc4d0813
[DOCS] NPU update 23.3 (#21812) 2023-12-21 09:24:52 +00:00
Tatiana Savina 7ef8faecf0
fix note and img (#21797) 2023-12-21 08:47:51 +01:00
Ilya Lavrenov 53960d6018
Switched GHA branches to 2023.3 (#21793) 2023-12-20 19:22:51 +04:00
1226 changed files with 141630 additions and 41611 deletions

2
.github/CODEOWNERS vendored
View File

@ -28,6 +28,7 @@
/src/bindings/python/ @openvinotoolkit/openvino-ie-python-api-maintainers /src/bindings/python/ @openvinotoolkit/openvino-ie-python-api-maintainers
/src/bindings/c/ @openvinotoolkit/openvino-c-api-maintainers /src/bindings/c/ @openvinotoolkit/openvino-c-api-maintainers
/src/bindings/js/ @openvinotoolkit/openvino-js-api-maintainers
/src/common/*transformations/ @openvinotoolkit/openvino-ie-transformations-maintainers /src/common/*transformations/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/ @openvinotoolkit/openvino-ngraph-maintainers /src/core/ @openvinotoolkit/openvino-ngraph-maintainers
@ -35,6 +36,7 @@
/samples/c/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-c-api-maintainers /samples/c/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-c-api-maintainers
/samples/cpp/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-maintainers /samples/cpp/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-maintainers
/samples/python/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-ie-python-api-maintainers /samples/python/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-ie-python-api-maintainers
/samples/js/ @openvinotoolkit/openvino-samples-maintainers @openvinotoolkit/openvino-js-api-maintainers
/thirdparty/zlib/ @openvinotoolkit/openvino-samples-maintainers /thirdparty/zlib/ @openvinotoolkit/openvino-samples-maintainers
/thirdparty/json/ @openvinotoolkit/openvino-samples-maintainers /thirdparty/json/ @openvinotoolkit/openvino-samples-maintainers
/thirdparty/gflags/ @openvinotoolkit/openvino-samples-maintainers /thirdparty/gflags/ @openvinotoolkit/openvino-samples-maintainers

View File

@ -26,22 +26,27 @@ runs:
- if: ${{ runner.os == 'Linux' && inputs.self-hosted-runner == 'true' }} - if: ${{ runner.os == 'Linux' && inputs.self-hosted-runner == 'true' }}
name: Install 'actions/setup-python@v4' dependencies name: Install 'actions/setup-python@v4' dependencies
shell: bash shell: bash
run: apt-get update && apt-get install -y ca-certificates software-properties-common run: apt-get update && apt-get install -y ca-certificates software-properties-common gpg-agent tzdata
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
TZ: "Europe/London" # to prevent tzdata from waiting user input
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' }} - if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' }}
name: Setup sudo and python3 name: Setup sudo and python3
shell: bash shell: bash
run: apt-get update && apt-get install -y sudo python3 # Needed for the deadsnakes action run: apt-get update && apt-get install -y sudo python3 # Needed for the deadsnakes action
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
- if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' }} - if: ${{ runner.os == 'Linux' && runner.arch == 'ARM64' }}
name: Setup Python ${{ inputs.version }} name: Setup Python ${{ inputs.version }}
uses: akashchi/deadsnakes-action@f01521a69eee61eaca3a34440bea3ce838317846 uses: akashchi/deadsnakes-action@92417281055a5878a0450f240a5b95883eb2d7e2
with: with:
python-version: ${{ inputs.version }} python-version: ${{ inputs.version }}
- if: ${{ runner.os == 'macOS' || runner.os == 'Windows' || (runner.os == 'Linux' && runner.arch != 'ARM64') }} - if: ${{ runner.os == 'macOS' || runner.os == 'Windows' || (runner.os == 'Linux' && runner.arch != 'ARM64') }}
name: Setup Python ${{ inputs.version }} name: Setup Python ${{ inputs.version }}
uses: actions/setup-python@v4 uses: actions/setup-python@v5
with: with:
python-version: ${{ inputs.version }} python-version: ${{ inputs.version }}
env: env:

View File

@ -29,6 +29,7 @@ CPU:
revalidate: revalidate:
- C_API - C_API
- Python_API - Python_API
- JS_API
- samples - samples
- ONNX_RT - ONNX_RT
- PyTorch_FE - PyTorch_FE
@ -115,6 +116,7 @@ IR_FE:
revalidate: revalidate:
- C_API - C_API
- Python_API - Python_API
- JS_API
- samples - samples
build: build:
- CPU - CPU
@ -181,6 +183,13 @@ Python_API:
- TFL_FE - TFL_FE
- PyTorch_FE - PyTorch_FE
JS_API:
revalidate:
- samples
build:
- CPU
- IR_FE
samples: samples:
build: build:
- CPU - CPU
@ -205,7 +214,8 @@ IE_Tests:
MO: MO:
revalidate: revalidate:
- POT - PyTorch_FE
- TF_FE
build: build:
- Python_API - Python_API

6
.github/labeler.yml vendored
View File

@ -109,6 +109,9 @@
'category: packaging': 'category: packaging':
- 'cmake/**/packaging/**/*' - 'cmake/**/packaging/**/*'
- 'src/bindings/python/wheel/**/*' - 'src/bindings/python/wheel/**/*'
- any: ['src/bindings/js/node/CMakeLists.txt',
'src/bindings/js/node/package.json',
'src/bindings/js/node/package-lock.json']
- 'tools/openvino_dev/**/*' - 'tools/openvino_dev/**/*'
'category: PDPD FE': 'category: PDPD FE':
@ -124,6 +127,9 @@
'category: Python API': 'category: Python API':
- 'src/bindings/python/**/*' - 'src/bindings/python/**/*'
'category: JS API':
- 'src/bindings/js/**/*'
'category: samples': 'category: samples':
- 'samples/**/*' - 'samples/**/*'
- 'thirdparty/zlib/**/*' - 'thirdparty/zlib/**/*'

View File

@ -53,6 +53,8 @@ jobs:
CMAKE_GENERATOR: 'Ninja' CMAKE_GENERATOR: 'Ninja'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
OPENVINO_REPO: '/__w/openvino/openvino/openvino' OPENVINO_REPO: '/__w/openvino/openvino/openvino'
VCPKG_ROOT: '/__w/openvino/openvino/vcpkg' VCPKG_ROOT: '/__w/openvino/openvino/vcpkg'
BUILD_DIR: '/__w/openvino/openvino/build' BUILD_DIR: '/__w/openvino/openvino/build'
@ -81,13 +83,14 @@ jobs:
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/open_model_zoo
popd popd
- name: Clone vcpkg - name: Clone vcpkg
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
repository: 'microsoft/vcpkg' repository: 'microsoft/vcpkg'
# Keep in sync with <root>/vcpkg.json <builtin-baseline>
ref: '7ba0ba7334c3346e7eee1e049ba85da193a8d821'
path: 'vcpkg' path: 'vcpkg'
fetch-depth: '0' fetch-depth: '0'
@ -123,9 +126,9 @@ jobs:
echo "yes" | ./cmdline-tools/bin/sdkmanager --sdk_root=${ANDROID_TOOLS} --install "ndk-bundle" "platform-tools" "platforms;android-${{ env.ANDROID_SDK_VERSION }}" echo "yes" | ./cmdline-tools/bin/sdkmanager --sdk_root=${ANDROID_TOOLS} --install "ndk-bundle" "platform-tools" "platforms;android-${{ env.ANDROID_SDK_VERSION }}"
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
# #
# Build # Build

View File

@ -20,7 +20,7 @@ jobs:
lfs: 'true' lfs: 'true'
- name: Install apt-get dependencies - name: Install apt-get dependencies
uses: awalsh128/cache-apt-pkgs-action@v1.3.1 uses: awalsh128/cache-apt-pkgs-action@v1.4.1
with: with:
packages: graphviz texlive liblua5.2-0 libclang1-9 libclang-cpp9 packages: graphviz texlive liblua5.2-0 libclang1-9 libclang-cpp9
version: 3.0 version: 3.0
@ -54,7 +54,7 @@ jobs:
- name: Cache documentation - name: Cache documentation
id: cache_sphinx_docs id: cache_sphinx_docs
uses: actions/cache@v3 uses: actions/cache@v4
with: with:
path: build/docs/_build/.doctrees path: build/docs/_build/.doctrees
key: sphinx-docs-cache key: sphinx-docs-cache

View File

@ -30,7 +30,7 @@ jobs:
submodules: 'true' submodules: 'true'
- name: Install OpenCL - name: Install OpenCL
uses: awalsh128/cache-apt-pkgs-action@v1.3.1 uses: awalsh128/cache-apt-pkgs-action@v1.4.1
if: runner.os == 'Linux' if: runner.os == 'Linux'
with: with:
packages: ocl-icd-opencl-dev opencl-headers packages: ocl-icd-opencl-dev opencl-headers

View File

@ -36,6 +36,8 @@ jobs:
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino' GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib
@ -62,7 +64,7 @@ jobs:
repository: 'openvinotoolkit/openvino_contrib' repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }} path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true' submodules: 'true'
ref: 'master' ref: 'releases/2023/3'
# #
# Dependencies # Dependencies
@ -75,9 +77,9 @@ jobs:
apt install --assume-yes --no-install-recommends default-jdk apt install --assume-yes --no-install-recommends default-jdk
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }} - name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python uses: ./openvino/.github/actions/setup_python

View File

@ -53,6 +53,8 @@ jobs:
CMAKE_GENERATOR: 'Ninja' CMAKE_GENERATOR: 'Ninja'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino' GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
INSTALL_DIR: /__w/openvino/openvino/openvino_install INSTALL_DIR: /__w/openvino/openvino/openvino_install
@ -85,9 +87,9 @@ jobs:
run: bash ${OPENVINO_REPO}/install_build_dependencies.sh run: bash ${OPENVINO_REPO}/install_build_dependencies.sh
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Install python dependencies - name: Install python dependencies
run: | run: |

View File

@ -30,6 +30,9 @@ jobs:
PARALLEL_TEST_SCRIPT: ${{ github.workspace }}/install/tests/functional_test_utils/layer_tests_summary/run_parallel.py PARALLEL_TEST_SCRIPT: ${{ github.workspace }}/install/tests/functional_test_utils/layer_tests_summary/run_parallel.py
PARALLEL_TEST_CACHE: ${{ github.workspace }}/install/tests/test_cache.lst PARALLEL_TEST_CACHE: ${{ github.workspace }}/install/tests/test_cache.lst
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:
@ -83,7 +86,7 @@ jobs:
run: python3 -m pip install -r ${INSTALL_TEST_DIR}/functional_test_utils/layer_tests_summary/requirements.txt run: python3 -m pip install -r ${INSTALL_TEST_DIR}/functional_test_utils/layer_tests_summary/requirements.txt
- name: Restore tests execution time - name: Restore tests execution time
uses: actions/cache/restore@v3 uses: actions/cache/restore@v4
with: with:
path: ${{ env.PARALLEL_TEST_CACHE }} path: ${{ env.PARALLEL_TEST_CACHE }}
key: ${{ runner.os }}-${{ runner.arch }}-tests-functional-cpu-stamp-${{ github.sha }} key: ${{ runner.os }}-${{ runner.arch }}-tests-functional-cpu-stamp-${{ github.sha }}
@ -98,10 +101,10 @@ jobs:
fi fi
python3 ${PARALLEL_TEST_SCRIPT} -e ${INSTALL_TEST_DIR}/ov_cpu_func_tests -c ${PARALLEL_TEST_CACHE} -w ${INSTALL_TEST_DIR} -s suite -rf 0 -- --gtest_print_time=1 --gtest_filter=*smoke* python3 ${PARALLEL_TEST_SCRIPT} -e ${INSTALL_TEST_DIR}/ov_cpu_func_tests -c ${PARALLEL_TEST_CACHE} -w ${INSTALL_TEST_DIR} -s suite -rf 0 -- --gtest_print_time=1 --gtest_filter=*smoke*
timeout-minutes: 20 timeout-minutes: 25
- name: Save tests execution time - name: Save tests execution time
uses: actions/cache/save@v3 uses: actions/cache/save@v4
if: github.ref_name == 'master' if: github.ref_name == 'master'
with: with:
path: ${{ env.PARALLEL_TEST_CACHE }} path: ${{ env.PARALLEL_TEST_CACHE }}

View File

@ -1,4 +1,4 @@
name: Samples name: C++ Unit Tests
on: on:
workflow_call: workflow_call:
@ -32,6 +32,9 @@ jobs:
INSTALL_DIR: ${{ github.workspace }}/install INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:

View File

@ -26,6 +26,8 @@ jobs:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
DEBIAN_PACKAGES_DIR: ${{ github.workspace }}/packages DEBIAN_PACKAGES_DIR: ${{ github.workspace }}/packages
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO debian packages - name: Download OpenVINO debian packages
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
@ -40,7 +42,7 @@ jobs:
- name: Install debian packages & check conflicts - name: Install debian packages & check conflicts
run: | run: |
apt-get update -y apt-get update -y
if [[ "${{ runner.arch }}" == "X64" ]]; then if [[ "${{ runner.arch }}" == "X64" ]]; then
# Install debian packages from previous release # Install debian packages from previous release
apt-get install --no-install-recommends -y gnupg wget ca-certificates apt-get install --no-install-recommends -y gnupg wget ca-certificates
@ -48,7 +50,8 @@ jobs:
apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
echo "deb https://apt.repos.intel.com/openvino/2023 ubuntu20 main" | tee /etc/apt/sources.list.d/intel-openvino-2023.list echo "deb https://apt.repos.intel.com/openvino/2023 ubuntu20 main" | tee /etc/apt/sources.list.d/intel-openvino-2023.list
apt-get update -y apt-get update -y
apt-get install -y openvino apt-get install -y openvino-2023.2.0
rm /etc/apt/sources.list.d/intel-openvino-2023.list
fi fi
# install our local one and make sure the conflicts are resolved # install our local one and make sure the conflicts are resolved
@ -63,21 +66,25 @@ jobs:
run: | run: |
/usr/share/openvino/samples/cpp/build_samples.sh /usr/share/openvino/samples/cpp/build_samples.sh
/usr/share/openvino/samples/c/build_samples.sh /usr/share/openvino/samples/c/build_samples.sh
[[ "${{ runner.arch }}" == "X64" ]] && path_by_arch="intel64" || path_by_arch="aarch64" [[ "${{ runner.arch }}" == "X64" ]] && path_by_arch="intel64" || path_by_arch="aarch64"
~/openvino_cpp_samples_build/$path_by_arch/Release/hello_query_device ~/openvino_cpp_samples_build/$path_by_arch/Release/hello_query_device
# check integrity of OpenVINO Python API installation
apt-get install python3-pip -y
python3 -m pip check
python3 /usr/share/openvino/samples/python/hello_query_device/hello_query_device.py python3 /usr/share/openvino/samples/python/hello_query_device/hello_query_device.py
python3 -c 'from openvino import Core; Core().get_property("CPU", "AVAILABLE_DEVICES")' python3 -c 'from openvino import Core; Core().get_property("CPU", "AVAILABLE_DEVICES")'
if [[ "${{ runner.arch }}" == "X64" ]]; then if [[ "${{ runner.arch }}" == "X64" ]]; then
python3 -c 'from openvino import Core; Core().get_property("GPU", "AVAILABLE_DEVICES")' python3 -c 'from openvino import Core; Core().get_property("GPU", "AVAILABLE_DEVICES")'
fi fi
python3 -c 'from openvino import Core; Core().get_property("AUTO", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("AUTO", "SUPPORTED_METRICS")' python3 -c 'from openvino import Core; Core().get_property("MULTI", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("MULTI", "SUPPORTED_METRICS")' python3 -c 'from openvino import Core; Core().get_property("HETERO", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("HETERO", "SUPPORTED_METRICS")' python3 -c 'from openvino import Core; Core().get_property("BATCH", "SUPPORTED_PROPERTIES")'
python3 -c 'from openvino import Core; Core().get_property("BATCH", "SUPPORTED_METRICS")'
python3 -c 'from openvino.frontend import FrontEndManager; assert len(FrontEndManager().get_available_front_ends()) == 6' python3 -c 'from openvino.frontend import FrontEndManager; assert len(FrontEndManager().get_available_front_ends()) == 6'
benchmark_app --help benchmark_app --help
opt_in_out --help
ovc --help ovc --help

View File

@ -28,9 +28,15 @@ jobs:
INSTALL_DIR: ${{ github.workspace }}/install INSTALL_DIR: ${{ github.workspace }}/install
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
ONNX_MODELS_PATH: ${{ github.workspace }}/onnx_test_models ONNX_MODELS_PATH: ${{ github.workspace }}/onnx_test_models
MODELS_SHARE_PATH: "/mount/onnxtestdata" # instead of using static MODELS_SHARE_PATH
# choose one of the replicas dynamically instead
# depending on GITHUB_RUN_NUMBER variable
NUMBER_OF_REPLICAS: 2
ONNX_MODEL_ZOO_SHA: "d58213534f2a4d1c4b19ba62b3bb5f544353256e" ONNX_MODEL_ZOO_SHA: "d58213534f2a4d1c4b19ba62b3bb5f544353256e"
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:
@ -49,6 +55,8 @@ jobs:
echo "OPENVINO_REPO=$GITHUB_WORKSPACE/openvino" >> "$GITHUB_ENV" echo "OPENVINO_REPO=$GITHUB_WORKSPACE/openvino" >> "$GITHUB_ENV"
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV" echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV" echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "MODELS_SHARE_PATH=/mount/onnxtestdata$((GITHUB_RUN_NUMBER % NUMBER_OF_REPLICAS))" >> "$GITHUB_ENV"
echo $MODELS_SHARE_PATH
- name: Extract OpenVINO packages - name: Extract OpenVINO packages
run: | run: |
@ -103,4 +111,4 @@ jobs:
python3 -m pip install pytest-xdist[psutil] pytest-forked python3 -m pip install pytest-xdist[psutil] pytest-forked
- name: ONNX Models Tests - name: ONNX Models Tests
run: python3 -m pytest --backend="CPU" --model_zoo_dir="${MODELS_SHARE_PATH}" ${INSTALL_TEST_DIR}/onnx/tests/tests_python/test_zoo_models.py -v -n 12 --forked -k 'not _cuda' --model_zoo_xfail run: python3 -m pytest --backend="CPU" --model_zoo_dir="${MODELS_SHARE_PATH}" ${INSTALL_TEST_DIR}/onnx/tests/tests_python/test_zoo_models.py -v -n auto --forked -k 'not _cuda' --model_zoo_xfail

View File

@ -33,11 +33,16 @@ jobs:
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
SCCACHE_AZURE_KEY_PREFIX: ${{ inputs.sccache-azure-key-prefix }} SCCACHE_AZURE_KEY_PREFIX: ${{ inputs.sccache-azure-key-prefix }}
ONNX_RUNTIME_REPO: ${{ github.workspace }}/onnxruntime ONNX_RUNTIME_REPO: ${{ github.workspace }}/onnxruntime
ONNX_RUNTIME_UTILS: ${{ github.workspace }}/install/onnxruntime ONNX_RUNTIME_UTILS: ${{ github.workspace }}/install/onnxruntime
ONNX_RUNTIME_BUILD_DIR: ${{ github.workspace }}/onnxruntime/build ONNX_RUNTIME_BUILD_DIR: ${{ github.workspace }}/onnxruntime/build
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:
@ -95,9 +100,9 @@ jobs:
run: bash ${OPENVINO_REPO}/install_build_dependencies.sh run: bash ${OPENVINO_REPO}/install_build_dependencies.sh
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Build Lin ONNX Runtime - name: Build Lin ONNX Runtime
run: | run: |

61
.github/workflows/job_openvino_js.yml vendored Normal file
View File

@ -0,0 +1,61 @@
name: OpenVINO JS API
on:
workflow_call:
inputs:
runner:
description: 'Machine on which the tests would run'
type: string
required: true
container:
description: 'JSON to be converted to the value of the "container" configuration for the job'
type: string
required: false
default: '{"image": null}'
jobs:
JS_API:
name: OpenVINO JS API
timeout-minutes: 10
runs-on: ${{ inputs.runner }}
container: ${{ fromJSON(inputs.container) }}
defaults:
run:
shell: bash
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
OPENVINO_JS_DIR: ${{ github.workspace }}/openvino/src/bindings/js
OPENVINO_JS_LIBS_DIR: ${{ github.workspace }}/openvino/src/bindings/js/node/bin
NODE_VERSION: 18
steps:
- name: Fetch OpenVINO JS sources
uses: actions/checkout@v4
with:
sparse-checkout: |
src/bindings/js
path: 'openvino'
# Needed as ${{ github.workspace }} is not working correctly when using Docker
- name: Setup Variables
run: |
echo "OPENVINO_JS_DIR=$GITHUB_WORKSPACE/openvino/src/bindings/js" >> "$GITHUB_ENV"
echo "OPENVINO_JS_LIBS_DIR=$GITHUB_WORKSPACE/openvino/src/bindings/js/node/bin" >> "$GITHUB_ENV"
- name: Download OpenVINO JS package
uses: actions/download-artifact@v3
with:
name: openvino_js_package
path: ${{ env.OPENVINO_JS_LIBS_DIR }}
- name: Setup Node ${{ env.NODE_VERSION }}
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
- name: Configure OpenVINO JS API
working-directory: ${{ env.OPENVINO_JS_DIR }}/node
run: npm i
- name: Test OpenVINO JS API
working-directory: ${{ env.OPENVINO_JS_DIR }}/node
run: npm run test

View File

@ -37,6 +37,9 @@ jobs:
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
LAYER_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/layer_tests LAYER_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/layer_tests
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:
@ -68,7 +71,7 @@ jobs:
- name: Install OpenVINO dependencies (Linux) - name: Install OpenVINO dependencies (Linux)
if: runner.os == 'Linux' if: runner.os == 'Linux'
run: $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -y run: $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh -c=core -c=dev -y -c=gpu
- name: Fetch setup_python action - name: Fetch setup_python action
uses: actions/checkout@v4 uses: actions/checkout@v4

View File

@ -33,11 +33,18 @@ jobs:
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
MODEL_HUB_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/model_hub_tests MODEL_HUB_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/model_hub_tests
steps: steps:
- name: Check sudo - name: Check sudo
if: ${{ runner.os == 'Linux' }} if: ${{ runner.os == 'Linux' }}
run: if [ "$(id -u)" -eq 0 ]; then apt update && apt --assume-yes install sudo; fi run: if [ "$(id -u)" -eq 0 ]; then apt update && apt --assume-yes install sudo; fi
- name: Set apt retries
run: |
if [ "$(id -u)" -eq 0 ]; then
echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
else
sudo sh -c "echo 'Acquire::Retries \"10\";' >> /etc/apt/apt.conf.d/80-retries"
fi
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:

View File

@ -31,6 +31,9 @@ jobs:
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
BUILD_DIR: ${{ github.workspace }}/build BUILD_DIR: ${{ github.workspace }}/build
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:

View File

@ -33,11 +33,18 @@ jobs:
INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests INSTALL_TEST_DIR: ${{ github.workspace }}/install/tests
MODEL_HUB_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/model_hub_tests MODEL_HUB_TESTS_INSTALL_DIR: ${{ github.workspace }}/install/tests/model_hub_tests
steps: steps:
- name: Check sudo - name: Check sudo
if: ${{ runner.os == 'Linux' }} if: ${{ runner.os == 'Linux' }}
run: if [ "$(id -u)" -eq 0 ]; then apt update && apt --assume-yes install sudo; fi run: if [ "$(id -u)" -eq 0 ]; then apt update && apt --assume-yes install sudo; fi
- name: Set apt retries
run: |
if [ "$(id -u)" -eq 0 ]; then
echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
else
sudo sh -c "echo 'Acquire::Retries \"10\";' >> /etc/apt/apt.conf.d/80-retries"
fi
- name: Download OpenVINO package - name: Download OpenVINO package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:

View File

@ -61,10 +61,13 @@ jobs:
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino' GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib
INSTALL_DIR: /__w/openvino/openvino/openvino_install INSTALL_DIR: /__w/openvino/openvino/openvino_install
INSTALL_DIR_JS: /__w/openvino/openvino/openvino_install/js
INSTALL_TEST_DIR: /__w/openvino/openvino/tests_install INSTALL_TEST_DIR: /__w/openvino/openvino/tests_install
DEVELOPER_PACKAGE_DIR: /__w/openvino/openvino/developer_package_install DEVELOPER_PACKAGE_DIR: /__w/openvino/openvino/developer_package_install
BUILD_DIR: /__w/openvino/openvino/openvino_build BUILD_DIR: /__w/openvino/openvino/openvino_build
@ -73,6 +76,9 @@ jobs:
if: "!needs.smart_ci.outputs.skip_workflow" if: "!needs.smart_ci.outputs.skip_workflow"
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git - name: Install git
run: | run: |
apt-get update apt-get update
@ -90,7 +96,7 @@ jobs:
repository: 'openvinotoolkit/openvino_contrib' repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }} path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true' submodules: 'true'
ref: 'master' ref: 'releases/2023/3'
# #
# Print system info # Print system info
@ -110,9 +116,9 @@ jobs:
apt install --assume-yes --no-install-recommends default-jdk apt install --assume-yes --no-install-recommends default-jdk
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }} - name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python uses: ./openvino/.github/actions/setup_python
@ -222,6 +228,15 @@ jobs:
-B ${BUILD_DIR} -B ${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
- name: CMake configure, build and install - OpenVINO JS API
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
run:
cmake -DCPACK_GENERATOR=NPM -DENABLE_SYSTEM_TBB=OFF -UTBB* -S ${OPENVINO_REPO} -B ${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR_JS} -P ${BUILD_DIR}/cmake_install.cmake
# #
# Upload build artifacts # Upload build artifacts
# #
@ -234,6 +249,14 @@ jobs:
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
if-no-files-found: 'error' if-no-files-found: 'error'
- name: Upload openvino js package
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
if-no-files-found: 'error'
- name: Upload openvino developer package - name: Upload openvino developer package
if: ${{ always() }} if: ${{ always() }}
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
@ -275,6 +298,15 @@ jobs:
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04' image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
affected-components: ${{ needs.smart_ci.outputs.affected_components }} affected-components: ${{ needs.smart_ci.outputs.affected_components }}
JS_API:
name: OpenVINO JS API
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: ./.github/workflows/job_openvino_js.yml
with:
runner: 'aks-linux-4-cores-16gb'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04"}'
Conformance: Conformance:
needs: [ Build, Smart_CI ] needs: [ Build, Smart_CI ]
timeout-minutes: ${{ matrix.TEST_TYPE == 'API' && 5 || 15 }} timeout-minutes: ${{ matrix.TEST_TYPE == 'API' && 5 || 15 }}
@ -473,6 +505,8 @@ jobs:
CMAKE_CUDA_COMPILER_LAUNCHER: sccache CMAKE_CUDA_COMPILER_LAUNCHER: sccache
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
INSTALL_DIR: /__w/openvino/openvino/install INSTALL_DIR: /__w/openvino/openvino/install
OPENVINO_DEVELOPER_PACKAGE: /__w/openvino/openvino/install/developer_package OPENVINO_DEVELOPER_PACKAGE: /__w/openvino/openvino/install/developer_package
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
@ -483,6 +517,9 @@ jobs:
if: fromJSON(needs.smart_ci.outputs.affected_components).NVIDIA if: fromJSON(needs.smart_ci.outputs.affected_components).NVIDIA
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Fetch install_build_dependencies.sh - name: Fetch install_build_dependencies.sh
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
@ -521,7 +558,7 @@ jobs:
with: with:
repository: 'openvinotoolkit/openvino_contrib' repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }} path: ${{ env.OPENVINO_CONTRIB_REPO }}
ref: 'master' ref: 'releases/2023/3'
# #
# Dependencies # Dependencies
@ -533,9 +570,9 @@ jobs:
apt -y --no-install-recommends install software-properties-common curl apt -y --no-install-recommends install software-properties-common curl
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Install CUDA - name: Install CUDA
run: | run: |

View File

@ -21,6 +21,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
outputs: outputs:
affected_components: "${{ steps.smart_ci.outputs.affected_components }}" affected_components: "${{ steps.smart_ci.outputs.affected_components }}"
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps: steps:
- name: checkout action - name: checkout action
uses: actions/checkout@v4 uses: actions/checkout@v4
@ -36,6 +37,8 @@ jobs:
commit_sha: ${{ github.sha }} commit_sha: ${{ github.sha }}
component_pattern: "category: (.*)" component_pattern: "category: (.*)"
repo_token: ${{ secrets.GITHUB_TOKEN }} repo_token: ${{ secrets.GITHUB_TOKEN }}
skip_when_only_listed_labels_set: 'docs'
skip_when_only_listed_files_changed: '*.md,*.rst,*.png,*.jpg,*.svg'
- name: Show affected components - name: Show affected components
run: | run: |
@ -60,18 +63,24 @@ jobs:
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino' GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib OPENVINO_CONTRIB_REPO: /__w/openvino/openvino/openvino_contrib
INSTALL_DIR: /__w/openvino/openvino/openvino_install INSTALL_DIR: /__w/openvino/openvino/openvino_install
INSTALL_DIR_JS: /__w/openvino/openvino/openvino_install/js
INSTALL_TEST_DIR: /__w/openvino/openvino/tests_install INSTALL_TEST_DIR: /__w/openvino/openvino/tests_install
DEVELOPER_PACKAGE_DIR: /__w/openvino/openvino/developer_package_install DEVELOPER_PACKAGE_DIR: /__w/openvino/openvino/developer_package_install
BUILD_DIR: /__w/openvino/openvino/openvino_build BUILD_DIR: /__w/openvino/openvino/openvino_build
SCCACHE_AZURE_KEY_PREFIX: 'ubuntu20_aarch64_Release' SCCACHE_AZURE_KEY_PREFIX: 'ubuntu20_aarch64_Release'
ONNX_RUNTIME_UTILS: /__w/openvino/openvino/openvino/.ci/azure/ci_utils/onnxruntime ONNX_RUNTIME_UTILS: /__w/openvino/openvino/openvino/.ci/azure/ci_utils/onnxruntime
if: "!fromJSON(needs.smart_ci.outputs.affected_components).docs_only" if: "!needs.smart_ci.outputs.skip_workflow"
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git - name: Install git
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates
@ -87,7 +96,7 @@ jobs:
repository: 'openvinotoolkit/openvino_contrib' repository: 'openvinotoolkit/openvino_contrib'
path: ${{ env.OPENVINO_CONTRIB_REPO }} path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true' submodules: 'true'
ref: 'master' ref: 'releases/2023/3'
# #
# Print system info # Print system info
@ -107,9 +116,9 @@ jobs:
apt install --assume-yes --no-install-recommends default-jdk apt install --assume-yes --no-install-recommends default-jdk
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }} - name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python uses: ./openvino/.github/actions/setup_python
@ -218,6 +227,20 @@ jobs:
-B ${BUILD_DIR} -B ${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
- name: CMake configure, build and install - OpenVINO JS API
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
run: |
cmake \
-DCPACK_GENERATOR=NPM \
-DENABLE_SYSTEM_TBB=OFF -UTBB* \
-DENABLE_INTEL_GPU=OFF \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}
cmake --build ${BUILD_DIR} --parallel
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR_JS} -P ${BUILD_DIR}/cmake_install.cmake
# #
# Upload build artifacts # Upload build artifacts
# #
@ -238,6 +261,14 @@ jobs:
path: ${{ env.BUILD_DIR }}/openvino_developer_package.tar.gz path: ${{ env.BUILD_DIR }}/openvino_developer_package.tar.gz
if-no-files-found: 'error' if-no-files-found: 'error'
- name: Upload openvino js package
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
if-no-files-found: 'error'
- name: Upload openvino debian packages - name: Upload openvino debian packages
if: ${{ 'false' }} if: ${{ 'false' }}
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
@ -272,6 +303,15 @@ jobs:
image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04' image: 'openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04'
affected-components: ${{ needs.smart_ci.outputs.affected_components }} affected-components: ${{ needs.smart_ci.outputs.affected_components }}
JS_API:
name: OpenVINO JS API
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: ./.github/workflows/job_openvino_js.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "openvinogithubactions.azurecr.io/dockerhub/ubuntu:20.04"}'
ONNX_Runtime: ONNX_Runtime:
name: ONNX Runtime Integration name: ONNX Runtime Integration
if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_RT if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_RT

View File

@ -58,6 +58,8 @@ jobs:
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino' GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
INSTALL_DIR: /__w/openvino/openvino/openvino_install INSTALL_DIR: /__w/openvino/openvino/openvino_install
@ -69,6 +71,9 @@ jobs:
if: "!needs.smart_ci.outputs.skip_workflow" if: "!needs.smart_ci.outputs.skip_workflow"
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git - name: Install git
run: | run: |
apt-get update apt-get update
@ -86,7 +91,7 @@ jobs:
repository: 'openvinotoolkit/testdata' repository: 'openvinotoolkit/testdata'
path: ${{ env.MODELS_PATH }} path: ${{ env.MODELS_PATH }}
lfs: 'true' lfs: 'true'
ref: 'master' ref: 'releases/2023/3'
# #
# Print system info # Print system info
@ -109,9 +114,9 @@ jobs:
update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++ 100 update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++ 100
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Setup Python ${{ env.PYTHON_VERSION }} - name: Setup Python ${{ env.PYTHON_VERSION }}
uses: ./openvino/.github/actions/setup_python uses: ./openvino/.github/actions/setup_python
@ -232,7 +237,7 @@ jobs:
CC_Build: CC_Build:
name: Conditional Compilation name: Conditional Compilation
needs: Build needs: Build
timeout-minutes: 10 timeout-minutes: 20
defaults: defaults:
run: run:
shell: bash shell: bash
@ -246,6 +251,8 @@ jobs:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
GITHUB_WORKSPACE: '/__w/openvino/openvino' GITHUB_WORKSPACE: '/__w/openvino/openvino'
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
BUILD_DIR: /__w/openvino/openvino/openvino_build BUILD_DIR: /__w/openvino/openvino/openvino_build
@ -254,6 +261,9 @@ jobs:
SCCACHE_AZURE_KEY_PREFIX: ubuntu22_x86_64_cc_Release SCCACHE_AZURE_KEY_PREFIX: ubuntu22_x86_64_cc_Release
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git - name: Install git
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates git-lfs run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates git-lfs
@ -269,7 +279,7 @@ jobs:
repository: 'openvinotoolkit/testdata' repository: 'openvinotoolkit/testdata'
path: ${{ env.MODELS_PATH }} path: ${{ env.MODELS_PATH }}
lfs: 'true' lfs: 'true'
ref: 'master' ref: 'releases/2023/3'
- name: Download selective build statistics package - name: Download selective build statistics package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
@ -288,9 +298,9 @@ jobs:
run: bash ${OPENVINO_REPO}/install_build_dependencies.sh run: bash ${OPENVINO_REPO}/install_build_dependencies.sh
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
# #
# Build # Build
# #

View File

@ -64,6 +64,9 @@ jobs:
CCACHE_MAXSIZE: 50G CCACHE_MAXSIZE: 50G
if: "!needs.smart_ci.outputs.skip_workflow" if: "!needs.smart_ci.outputs.skip_workflow"
steps: steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries
- name: Install git - name: Install git
run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates run: apt-get update && apt-get install --assume-yes --no-install-recommends git ca-certificates
@ -86,6 +89,7 @@ jobs:
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json git submodule update --init -- ${OPENVINO_REPO}/thirdparty/json
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gtest
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags git submodule update --init -- ${OPENVINO_REPO}/thirdparty/gflags
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/telemetry
git submodule update --init -- ${OPENVINO_REPO}/src/plugins/intel_cpu git submodule update --init -- ${OPENVINO_REPO}/src/plugins/intel_cpu
git submodule update --init -- ${OPENVINO_REPO}/thirdparty/open_model_zoo git submodule update --init -- ${OPENVINO_REPO}/thirdparty/open_model_zoo
popd popd

View File

@ -73,6 +73,7 @@ jobs:
OPENVINO_REPO: ${{ github.workspace }}/openvino OPENVINO_REPO: ${{ github.workspace }}/openvino
OPENVINO_CONTRIB_REPO: ${{ github.workspace }}/openvino_contrib OPENVINO_CONTRIB_REPO: ${{ github.workspace }}/openvino_contrib
INSTALL_DIR: ${{ github.workspace }}/openvino_install INSTALL_DIR: ${{ github.workspace }}/openvino_install
INSTALL_DIR_JS: ${{ github.workspace }}/openvino_install/js
INSTALL_TEST_DIR: ${{ github.workspace }}/tests_install INSTALL_TEST_DIR: ${{ github.workspace }}/tests_install
BUILD_DIR: ${{ github.workspace }}/build BUILD_DIR: ${{ github.workspace }}/build
steps: steps:
@ -190,6 +191,17 @@ jobs:
-B ${{ env.BUILD_DIR }} -B ${{ env.BUILD_DIR }}
cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
- name: CMake configure, build and install - OpenVINO JS API
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
run: |
cmake \
-DCPACK_GENERATOR=NPM \
-S ${{ env.OPENVINO_REPO }} \
-B ${{ env.BUILD_DIR }}
cmake --build ${{ env.BUILD_DIR }} --parallel
cmake -DCMAKE_INSTALL_PREFIX=${{ env.INSTALL_DIR_JS }} -P ${{ env.BUILD_DIR }}/cmake_install.cmake
# #
# Upload build artifacts # Upload build artifacts
# #
@ -210,6 +222,14 @@ jobs:
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
if-no-files-found: 'error' if-no-files-found: 'error'
- name: Upload openvino js package
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
if-no-files-found: 'error'
Samples: Samples:
needs: [ Build, Smart_CI ] needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).samples if: fromJSON(needs.smart_ci.outputs.affected_components).samples
@ -218,6 +238,14 @@ jobs:
runner: 'macos-13' runner: 'macos-13'
affected-components: ${{ needs.smart_ci.outputs.affected_components }} affected-components: ${{ needs.smart_ci.outputs.affected_components }}
JS_API:
name: OpenVINO JS API
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: ./.github/workflows/job_openvino_js.yml
with:
runner: 'macos-13'
CXX_Unit_Tests: CXX_Unit_Tests:
name: C++ unit tests name: C++ unit tests
needs: [ Build, Smart_CI ] needs: [ Build, Smart_CI ]

View File

@ -72,6 +72,7 @@ jobs:
OPENVINO_REPO: ${{ github.workspace }}/openvino OPENVINO_REPO: ${{ github.workspace }}/openvino
OPENVINO_CONTRIB_REPO: ${{ github.workspace }}/openvino_contrib OPENVINO_CONTRIB_REPO: ${{ github.workspace }}/openvino_contrib
INSTALL_DIR: ${{ github.workspace }}/openvino_install INSTALL_DIR: ${{ github.workspace }}/openvino_install
INSTALL_DIR_JS: ${{ github.workspace }}/openvino_install/js
INSTALL_TEST_DIR: ${{ github.workspace }}/tests_install INSTALL_TEST_DIR: ${{ github.workspace }}/tests_install
BUILD_DIR: ${{ github.workspace }}/build BUILD_DIR: ${{ github.workspace }}/build
steps: steps:
@ -189,6 +190,17 @@ jobs:
-B ${{ env.BUILD_DIR }} -B ${{ env.BUILD_DIR }}
cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
- name: CMake configure, build and install - OpenVINO JS API
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
run: |
cmake \
-DCPACK_GENERATOR=NPM \
-S ${{ env.OPENVINO_REPO }} \
-B ${{ env.BUILD_DIR }}
cmake --build ${{ env.BUILD_DIR }} --parallel
cmake -DCMAKE_INSTALL_PREFIX=${{ env.INSTALL_DIR_JS }} -P ${{ env.BUILD_DIR }}/cmake_install.cmake
# #
# Upload build artifacts # Upload build artifacts
# #
@ -209,6 +221,14 @@ jobs:
path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
if-no-files-found: 'error' if-no-files-found: 'error'
- name: Upload openvino js package
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
if-no-files-found: 'error'
Samples: Samples:
needs: Build needs: Build
uses: ./.github/workflows/job_samples_tests.yml uses: ./.github/workflows/job_samples_tests.yml
@ -216,6 +236,14 @@ jobs:
runner: 'macos-13-xlarge' runner: 'macos-13-xlarge'
affected-components: ${{ needs.smart_ci.outputs.affected_components }} affected-components: ${{ needs.smart_ci.outputs.affected_components }}
JS_API:
name: OpenVINO JS API
needs: [ Build, Smart_CI ]
if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: ./.github/workflows/job_openvino_js.yml
with:
runner: 'macos-13-xlarge'
CXX_Unit_Tests: CXX_Unit_Tests:
name: C++ unit tests name: C++ unit tests
needs: [ Build, Smart_CI ] needs: [ Build, Smart_CI ]

View File

@ -29,7 +29,7 @@ jobs:
python-version: '3.10' python-version: '3.10'
- name: Cache pip - name: Cache pip
uses: actions/cache@v3 uses: actions/cache@v4
with: with:
path: ~/.cache/pip path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('tools/mo/requirements*.txt') }} key: ${{ runner.os }}-pip-${{ hashFiles('tools/mo/requirements*.txt') }}

View File

@ -51,6 +51,8 @@ jobs:
CMAKE_BUILD_TYPE: 'Release' CMAKE_BUILD_TYPE: 'Release'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
OPENVINO_REPO: /__w/openvino/openvino/openvino OPENVINO_REPO: /__w/openvino/openvino/openvino
OPENVINO_BUILD_DIR: /__w/openvino/openvino/openvino_build OPENVINO_BUILD_DIR: /__w/openvino/openvino/openvino_build
SCCACHE_AZURE_KEY_PREFIX: webassembly_Release SCCACHE_AZURE_KEY_PREFIX: webassembly_Release
@ -66,9 +68,9 @@ jobs:
submodules: 'true' submodules: 'true'
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: emcmake cmake - configure - name: emcmake cmake - configure
run: | run: |

View File

@ -51,9 +51,13 @@ jobs:
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
SCCACHE_IDLE_TIMEOUT: 0
OPENVINO_REPO: "${{ github.workspace }}\\openvino" OPENVINO_REPO: "${{ github.workspace }}\\openvino"
OPENVINO_CONTRIB_REPO: "${{ github.workspace }}\\openvino_contrib" OPENVINO_CONTRIB_REPO: "${{ github.workspace }}\\openvino_contrib"
INSTALL_DIR: "${{ github.workspace }}\\openvino_install" INSTALL_DIR: "${{ github.workspace }}\\openvino_install"
INSTALL_DIR_JS: "${{ github.workspace }}\\openvino_install\\js"
INSTALL_TEST_DIR: "${{ github.workspace }}\\tests_install" INSTALL_TEST_DIR: "${{ github.workspace }}\\tests_install"
BUILD_DIR: "${{ github.workspace }}\\openvino_build" BUILD_DIR: "${{ github.workspace }}\\openvino_build"
# TODO: specify version of compiler here # TODO: specify version of compiler here
@ -72,7 +76,7 @@ jobs:
with: with:
repository: 'openvinotoolkit/openvino_contrib' repository: 'openvinotoolkit/openvino_contrib'
path: 'openvino_contrib' path: 'openvino_contrib'
ref: 'master' ref: 'releases/2023/3'
# #
# Print system info # Print system info
@ -117,12 +121,16 @@ jobs:
python3 -m pip install certifi python3 -m pip install certifi
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Install build dependencies - name: Install build dependencies
run: choco install --no-progress ninja run: |
Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.11.1/ninja-win.zip -OutFile ninja-win.zip -MaximumRetryCount 10
Expand-Archive -Force ninja-win.zip
# Add it to the GitHub Path so it would be available in the subsequent steps
Add-Content -Path $env:GITHUB_PATH -Value "${{ github.workspace }}/ninja-win"
# #
# Build # Build
@ -194,6 +202,16 @@ jobs:
-B ${{ env.BUILD_DIR }} -B ${{ env.BUILD_DIR }}
cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} --verbose cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} --verbose
- name: CMake configure, build and install - OpenVINO JS API
if: ${{ 'false' }} # 128689
# if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
run:
cmake -DCPACK_GENERATOR=NPM -DENABLE_SYSTEM_TBB=OFF -UTBB* -S ${{ env.OPENVINO_REPO }} -B ${{ env.BUILD_DIR }}
cmake --build ${{ env.BUILD_DIR }} --parallel
cmake -DCMAKE_INSTALL_PREFIX=${{ env.INSTALL_DIR_JS }} -P ${{ env.BUILD_DIR }}/cmake_install.cmake
- name: Upload openvino package - name: Upload openvino package
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
@ -208,6 +226,15 @@ jobs:
path: ${{ env.BUILD_DIR }}/openvino_tests.zip path: ${{ env.BUILD_DIR }}/openvino_tests.zip
if-no-files-found: 'error' if-no-files-found: 'error'
- name: Upload openvino js package
if: ${{ 'false' }} # 128689
# if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
uses: actions/upload-artifact@v3
with:
name: openvino_js_package
path: ${{ env.INSTALL_DIR_JS }}
if-no-files-found: 'error'
Samples: Samples:
needs: [Build, Smart_CI] needs: [Build, Smart_CI]
if: fromJSON(needs.smart_ci.outputs.affected_components).samples if: fromJSON(needs.smart_ci.outputs.affected_components).samples
@ -289,6 +316,51 @@ jobs:
path: ${{ env.INSTALL_TEST_DIR }}/TEST*.xml path: ${{ env.INSTALL_TEST_DIR }}/TEST*.xml
if-no-files-found: 'error' if-no-files-found: 'error'
JS_API:
name: JS API
needs: [ Build, Smart_CI ]
defaults:
run:
shell: pwsh
runs-on: 'aks-win-4-cores-8gb'
env:
OPENVINO_JS_DIR: "${{ github.workspace }}\\openvino\\src\\bindings\\js"
OPENVINO_JS_LIBS_DIR: "${{ github.workspace }}\\openvino\\src\\bindings\\js\\node\\bin"
if: ${{ 'false' }} # 128689
# if: fromJSON(needs.smart_ci.outputs.affected_components).JS_API
steps:
- name: Fetch OpenVINO JS sources
uses: actions/checkout@v4
with:
sparse-checkout: |
src/bindings/js
path: 'openvino'
- name: Download OpenVINO js package
uses: actions/download-artifact@v3
with:
name: openvino_js_package
path: ${{ env.OPENVINO_JS_LIBS_DIR }}
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 18
- name: Configure OpenVINO JS
working-directory: ${{ env.OPENVINO_JS_DIR }}/node
run: npm i
- name: Test OpenVINO JS
working-directory: ${{ env.OPENVINO_JS_DIR }}/node
run: npm test
- name: Test OpenVINO JS (cmd)
shell: cmd
working-directory: ${{ env.OPENVINO_JS_DIR }}/node
run: call npm test
Python_Unit_Tests: Python_Unit_Tests:
name: Python unit tests name: Python unit tests
needs: [Build, Smart_CI] needs: [Build, Smart_CI]
@ -787,7 +859,7 @@ jobs:
run: python3 -m pip install -r ${{ github.workspace }}\install\tests\functional_test_utils\layer_tests_summary\requirements.txt run: python3 -m pip install -r ${{ github.workspace }}\install\tests\functional_test_utils\layer_tests_summary\requirements.txt
- name: Restore tests execution time - name: Restore tests execution time
uses: actions/cache/restore@v3 uses: actions/cache/restore@v4
with: with:
path: ${{ env.PARALLEL_TEST_CACHE }} path: ${{ env.PARALLEL_TEST_CACHE }}
key: ${{ runner.os }}-tests-functional-cpu-stamp-${{ github.sha }} key: ${{ runner.os }}-tests-functional-cpu-stamp-${{ github.sha }}
@ -801,7 +873,7 @@ jobs:
timeout-minutes: 60 timeout-minutes: 60
- name: Save tests execution time - name: Save tests execution time
uses: actions/cache/save@v3 uses: actions/cache/save@v4
if: github.ref_name == 'master' if: github.ref_name == 'master'
with: with:
path: ${{ env.PARALLEL_TEST_CACHE }} path: ${{ env.PARALLEL_TEST_CACHE }}

View File

@ -48,12 +48,15 @@ jobs:
defaults: defaults:
run: run:
shell: pwsh shell: pwsh
runs-on: aks-win-16-cores-32gb runs-on: aks-win-8-cores-64gb
env: env:
CMAKE_BUILD_TYPE: 'Release' CMAKE_BUILD_TYPE: 'Release'
CMAKE_GENERATOR: 'Ninja Multi-Config' CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
SCCACHE_IDLE_TIMEOUT: 0
OPENVINO_REPO: "${{ github.workspace }}\\openvino" OPENVINO_REPO: "${{ github.workspace }}\\openvino"
INSTALL_DIR: "${{ github.workspace }}\\openvino_install" INSTALL_DIR: "${{ github.workspace }}\\openvino_install"
INSTALL_TEST_DIR: "${{ github.workspace }}\\tests_install" INSTALL_TEST_DIR: "${{ github.workspace }}\\tests_install"
@ -77,7 +80,7 @@ jobs:
repository: 'openvinotoolkit/testdata' repository: 'openvinotoolkit/testdata'
path: 'testdata' path: 'testdata'
lfs: 'true' lfs: 'true'
ref: 'master' ref: 'releases/2023/3'
# #
# Print system info # Print system info
@ -98,12 +101,16 @@ jobs:
self-hosted-runner: 'false' self-hosted-runner: 'false'
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.7.5"
- name: Install build dependencies - name: Install build dependencies
run: choco install --no-progress ninja run: |
Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.11.1/ninja-win.zip -OutFile ninja-win.zip -MaximumRetryCount 10
Expand-Archive -Force ninja-win.zip
# Add it to the GitHub Path so it would be available in the subsequent steps
Add-Content -Path $env:GITHUB_PATH -Value "${{ github.workspace }}/ninja-win"
- name: Install python dependencies - name: Install python dependencies
run: | run: |
@ -250,11 +257,14 @@ jobs:
defaults: defaults:
run: run:
shell: pwsh shell: pwsh
runs-on: aks-win-16-cores-32gb runs-on: aks-win-8-cores-64gb
env: env:
CMAKE_BUILD_TYPE: 'Release' CMAKE_BUILD_TYPE: 'Release'
CMAKE_CXX_COMPILER_LAUNCHER: sccache CMAKE_CXX_COMPILER_LAUNCHER: sccache
CMAKE_C_COMPILER_LAUNCHER: sccache CMAKE_C_COMPILER_LAUNCHER: sccache
SCCACHE_IGNORE_SERVER_IO_ERROR: 1
SCCACHE_SERVER_PORT: 35555
SCCACHE_IDLE_TIMEOUT: 0
OPENVINO_REPO: "${{ github.workspace }}\\openvino" OPENVINO_REPO: "${{ github.workspace }}\\openvino"
BUILD_DIR: "${{ github.workspace }}\\openvino_build" BUILD_DIR: "${{ github.workspace }}\\openvino_build"
MODELS_PATH: "${{ github.workspace }}\\testdata" MODELS_PATH: "${{ github.workspace }}\\testdata"
@ -275,7 +285,7 @@ jobs:
repository: 'openvinotoolkit/testdata' repository: 'openvinotoolkit/testdata'
path: 'testdata' path: 'testdata'
lfs: 'true' lfs: 'true'
ref: 'master' ref: 'releases/2023/3'
- name: Download selective build statistics package - name: Download selective build statistics package
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
@ -294,7 +304,7 @@ jobs:
self-hosted-runner: 'false' self-hosted-runner: 'false'
- name: Install sccache - name: Install sccache
uses: mozilla-actions/sccache-action@v0.0.3 uses: mozilla-actions/sccache-action@v0.0.4
with: with:
version: "v0.5.4" version: "v0.5.4"

3
.gitmodules vendored
View File

@ -75,3 +75,6 @@
[submodule "src/plugins/intel_cpu/thirdparty/mlas"] [submodule "src/plugins/intel_cpu/thirdparty/mlas"]
path = src/plugins/intel_cpu/thirdparty/mlas path = src/plugins/intel_cpu/thirdparty/mlas
url = https://github.com/openvinotoolkit/mlas.git url = https://github.com/openvinotoolkit/mlas.git
[submodule "thirdparty/telemetry"]
path = thirdparty/telemetry
url = https://github.com/openvinotoolkit/telemetry.git

View File

@ -28,6 +28,11 @@ if(POLICY CMP0091)
cmake_policy(SET CMP0091 NEW) # Enables use of MSVC_RUNTIME_LIBRARY cmake_policy(SET CMP0091 NEW) # Enables use of MSVC_RUNTIME_LIBRARY
endif() endif()
# Avoid warning about DOWNLOAD_EXTRACT_TIMESTAMP in CMake 3.24:
if(POLICY CMP0135)
cmake_policy(SET CMP0135 NEW)
endif()
project(OpenVINO DESCRIPTION "OpenVINO toolkit") project(OpenVINO DESCRIPTION "OpenVINO toolkit")
find_package(OpenVINODeveloperScripts REQUIRED find_package(OpenVINODeveloperScripts REQUIRED

View File

@ -61,7 +61,7 @@ product better.
[./docs/dev](https://github.com/openvinotoolkit/openvino/tree/master/docs/dev) folder. [./docs/dev](https://github.com/openvinotoolkit/openvino/tree/master/docs/dev) folder.
* **User documentation** is built from several sources and published at * **User documentation** is built from several sources and published at
[docs.openvino.ai](docs.openvino.ai), which is the recommended place for reading [docs.openvino.ai](https://docs.openvino.ai/), which is the recommended place for reading
these documents. Use the files maintained in this repository only for editing purposes. these documents. Use the files maintained in this repository only for editing purposes.
* The easiest way to help with documentation is to review it and provide feedback on the * The easiest way to help with documentation is to review it and provide feedback on the
@ -69,7 +69,7 @@ product better.
or think more information should be added, you can reach out to any of the documentation or think more information should be added, you can reach out to any of the documentation
contributors to discuss the potential changes. contributors to discuss the potential changes.
You can also create a Pull Request directly, following the [editor's guide](./docs/CONTRIBUTING_DOCS.md). You can also create a Pull Request directly, following the [editor's guide](./CONTRIBUTING_DOCS.md).
### Promote and Support OpenVINO ### Promote and Support OpenVINO
@ -151,4 +151,4 @@ We'll make sure to review your Pull Request as soon as possible and provide you
## License ## License
By contributing to the OpenVINO project, you agree that your contributions will be By contributing to the OpenVINO project, you agree that your contributions will be
licensed under the terms stated in the [LICENSE](./LICENSE.md) file. licensed under the terms stated in the [LICENSE](./LICENSE) file.

View File

@ -56,7 +56,7 @@ Regardless of the automated tests, you should ensure the quality of your changes
## Need Additional Help? Check these Articles ## Need Additional Help? Check these Articles
* [How to create a fork](https://help.github.com/articles/fork-a-rep) * [How to create a fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo)
* [Install Git](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) * [Install Git](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup)
* If you want to add a new sample, please have a look at the Guide for contributing * If you want to add a new sample, please have a look at the Guide for contributing
to C++/C/Python IE samples and add the license statement at the top of new files for to C++/C/Python IE samples and add the license statement at the top of new files for

View File

@ -1,5 +1,5 @@
<div align="center"> <div align="center">
<img src="docs/img/openvino-logo-purple-black.png" width="400px"> <img src="docs/sphinx_setup/_static/images/img/openvino-logo-purple-black.png" width="400px">
[![PyPI Status](https://badge.fury.io/py/openvino.svg)](https://badge.fury.io/py/openvino) [![PyPI Status](https://badge.fury.io/py/openvino.svg)](https://badge.fury.io/py/openvino)
[![Anaconda Status](https://anaconda.org/conda-forge/openvino/badges/version.svg)](https://anaconda.org/conda-forge/openvino) [![Anaconda Status](https://anaconda.org/conda-forge/openvino/badges/version.svg)](https://anaconda.org/conda-forge/openvino)
@ -67,18 +67,18 @@ The OpenVINO™ Runtime can infer models on different hardware devices. This sec
<tbody> <tbody>
<tr> <tr>
<td rowspan=2>CPU</td> <td rowspan=2>CPU</td>
<td> <a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_supported_plugins_CPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-c-p-u">Intel CPU</a></tb> <td> <a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_CPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-c-p-u">Intel CPU</a></tb>
<td><b><i><a href="./src/plugins/intel_cpu">openvino_intel_cpu_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/intel_cpu">openvino_intel_cpu_plugin</a></i></b></td>
<td>Intel Xeon with Intel® Advanced Vector Extensions 2 (Intel® AVX2), Intel® Advanced Vector Extensions 512 (Intel® AVX-512), and AVX512_BF16, Intel Core Processors with Intel AVX2, Intel Atom Processors with Intel® Streaming SIMD Extensions (Intel® SSE), Intel® Advanced Matrix Extensions (Intel® AMX)</td> <td>Intel Xeon with Intel® Advanced Vector Extensions 2 (Intel® AVX2), Intel® Advanced Vector Extensions 512 (Intel® AVX-512), and AVX512_BF16, Intel Core Processors with Intel AVX2, Intel Atom Processors with Intel® Streaming SIMD Extensions (Intel® SSE), Intel® Advanced Matrix Extensions (Intel® AMX)</td>
</tr> </tr>
<tr> <tr>
<td> <a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_supported_plugins_CPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-c-p-u">ARM CPU</a></tb> <td> <a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_CPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-c-p-u">ARM CPU</a></tb>
<td><b><i><a href="./src/plugins/intel_cpu">openvino_arm_cpu_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/intel_cpu">openvino_arm_cpu_plugin</a></i></b></td>
<td>Raspberry Pi™ 4 Model B, Apple® Mac mini with Apple silicon <td>Raspberry Pi™ 4 Model B, Apple® Mac mini with Apple silicon
</tr> </tr>
<tr> <tr>
<td>GPU</td> <td>GPU</td>
<td><a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_supported_plugins_GPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-g-p-u">Intel GPU</a></td> <td><a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_GPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-g-p-u">Intel GPU</a></td>
<td><b><i><a href="./src/plugins/intel_gpu">openvino_intel_gpu_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/intel_gpu">openvino_intel_gpu_plugin</a></i></b></td>
<td>Intel Processor Graphics, including Intel HD Graphics and Intel Iris Graphics</td> <td>Intel Processor Graphics, including Intel HD Graphics and Intel Iris Graphics</td>
</tr> </tr>
@ -102,22 +102,22 @@ OpenVINO™ Toolkit also contains several plugins which simplify loading models
</thead> </thead>
<tbody> <tbody>
<tr> <tr>
<td><a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_supported_plugins_AUTO.html">Auto</a></td> <td><a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_AUTO.html">Auto</a></td>
<td><b><i><a href="./src/plugins/auto">openvino_auto_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/auto">openvino_auto_plugin</a></i></b></td>
<td>Auto plugin enables selecting Intel device for inference automatically</td> <td>Auto plugin enables selecting Intel device for inference automatically</td>
</tr> </tr>
<tr> <tr>
<td><a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_Automatic_Batching.html">Auto Batch</a></td> <td><a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_Automatic_Batching.html">Auto Batch</a></td>
<td><b><i><a href="./src/plugins/auto_batch">openvino_auto_batch_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/auto_batch">openvino_auto_batch_plugin</a></i></b></td>
<td>Auto batch plugin performs on-the-fly automatic batching (i.e. grouping inference requests together) to improve device utilization, with no programming effort from the user</td> <td>Auto batch plugin performs on-the-fly automatic batching (i.e. grouping inference requests together) to improve device utilization, with no programming effort from the user</td>
</tr> </tr>
<tr> <tr>
<td><a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_Hetero_execution.html#doxid-openvino-docs-o-v-u-g-hetero-execution">Hetero</a></td> <td><a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_Hetero_execution.html#doxid-openvino-docs-o-v-u-g-hetero-execution">Hetero</a></td>
<td><b><i><a href="./src/plugins/hetero">openvino_hetero_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/hetero">openvino_hetero_plugin</a></i></b></td>
<td>Heterogeneous execution enables automatic inference splitting between several devices</td> <td>Heterogeneous execution enables automatic inference splitting between several devices</td>
</tr> </tr>
<tr> <tr>
<td><a href="https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_Running_on_multiple_devices.html#doxid-openvino-docs-o-v-u-g-running-on-multiple-devices">Multi</a></td> <td><a href="https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_Running_on_multiple_devices.html#doxid-openvino-docs-o-v-u-g-running-on-multiple-devices">Multi</a></td>
<td><b><i><a href="./src/plugins/auto">openvino_auto_plugin</a></i></b></td> <td><b><i><a href="./src/plugins/auto">openvino_auto_plugin</a></i></b></td>
<td>Multi plugin enables simultaneous inference of the same model on several devices in parallel</td> <td>Multi plugin enables simultaneous inference of the same model on several devices in parallel</td>
</tr> </tr>
@ -164,9 +164,9 @@ The list of OpenVINO tutorials:
## System requirements ## System requirements
The system requirements vary depending on platform and are available on dedicated pages: The system requirements vary depending on platform and are available on dedicated pages:
- [Linux](https://docs.openvino.ai/2023.2/openvino_docs_install_guides_installing_openvino_linux_header.html) - [Linux](https://docs.openvino.ai/2023.3/openvino_docs_install_guides_installing_openvino_linux_header.html)
- [Windows](https://docs.openvino.ai/2023.2/openvino_docs_install_guides_installing_openvino_windows_header.html) - [Windows](https://docs.openvino.ai/2023.3/openvino_docs_install_guides_installing_openvino_windows_header.html)
- [macOS](https://docs.openvino.ai/2023.2/openvino_docs_install_guides_installing_openvino_macos_header.html) - [macOS](https://docs.openvino.ai/2023.3/openvino_docs_install_guides_installing_openvino_macos_header.html)
## How to build ## How to build
@ -206,6 +206,6 @@ Report questions, issues and suggestions, using:
\* Other names and brands may be claimed as the property of others. \* Other names and brands may be claimed as the property of others.
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo [Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
[OpenVINO™ Runtime]:https://docs.openvino.ai/2023.2/openvino_docs_OV_UG_OV_Runtime_User_Guide.html [OpenVINO™ Runtime]:https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_OV_Runtime_User_Guide.html
[OpenVINO Model Converter (OVC)]:https://docs.openvino.ai/2023.2/openvino_docs_model_processing_introduction.html#convert-a-model-in-cli-ovc [OpenVINO Model Converter (OVC)]:https://docs.openvino.ai/2023.3/openvino_docs_model_processing_introduction.html#convert-a-model-in-cli-ovc
[Samples]:https://github.com/openvinotoolkit/openvino/tree/master/samples [Samples]:https://github.com/openvinotoolkit/openvino/tree/master/samples

View File

@ -206,6 +206,8 @@ set(CMAKE_POLICY_DEFAULT_CMP0025 NEW)
set(CMAKE_POLICY_DEFAULT_CMP0026 NEW) set(CMAKE_POLICY_DEFAULT_CMP0026 NEW)
# CMake 3.0+ (2.8.12): MacOS "@rpath" in target's install name # CMake 3.0+ (2.8.12): MacOS "@rpath" in target's install name
set(CMAKE_POLICY_DEFAULT_CMP0042 NEW) set(CMAKE_POLICY_DEFAULT_CMP0042 NEW)
# CMake 3.1+: Simplify variable reference and escape sequence evaluation.
set(CMAKE_POLICY_DEFAULT_CMP0053 NEW)
# CMake 3.9+: `RPATH` settings on macOS do not affect `install_name`. # CMake 3.9+: `RPATH` settings on macOS do not affect `install_name`.
set(CMAKE_POLICY_DEFAULT_CMP0068 NEW) set(CMAKE_POLICY_DEFAULT_CMP0068 NEW)
# CMake 3.12+: find_package() uses <PackageName>_ROOT variables. # CMake 3.12+: find_package() uses <PackageName>_ROOT variables.

View File

@ -3,15 +3,15 @@
# #
if(WIN32) if(WIN32)
set(PROGRAMFILES_ENV "ProgramFiles(X86)") set(PROGRAMFILES_ENV "ProgramFiles\(X86\)")
# check that PROGRAMFILES_ENV is defined, because in case of cross-compilation for Windows # check that PROGRAMFILES_ENV is defined, because in case of cross-compilation for Windows
# we don't have such variable # we don't have such variable
if(DEFINED ENV{PROGRAMFILES_ENV}) if(DEFINED ENV{${PROGRAMFILES_ENV}})
file(TO_CMAKE_PATH $ENV{${PROGRAMFILES_ENV}} PROGRAMFILES) file(TO_CMAKE_PATH $ENV{${PROGRAMFILES_ENV}} PROGRAMFILES)
set(WDK_PATHS "${PROGRAMFILES}/Windows Kits/10/bin/${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}/x64" set(WDK_PATHS "${PROGRAMFILES}/Windows Kits/10/bin/${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}/x64"
"${PROGRAMFILES}/Windows Kits/10/bin/x64") "${PROGRAMFILES}/Windows Kits/10/bin/x64")
message(STATUS "Trying to find apivalidator in: ") message(STATUS "Trying to find apivalidator in: ")
foreach(wdk_path IN LISTS WDK_PATHS) foreach(wdk_path IN LISTS WDK_PATHS)
@ -19,9 +19,9 @@ if(WIN32)
endforeach() endforeach()
find_host_program(ONECORE_API_VALIDATOR find_host_program(ONECORE_API_VALIDATOR
NAMES apivalidator NAMES apivalidator
PATHS ${WDK_PATHS} PATHS ${WDK_PATHS}
DOC "ApiValidator for OneCore compliance") DOC "ApiValidator for OneCore compliance")
if(ONECORE_API_VALIDATOR) if(ONECORE_API_VALIDATOR)
message(STATUS "Found apivalidator: ${ONECORE_API_VALIDATOR}") message(STATUS "Found apivalidator: ${ONECORE_API_VALIDATOR}")

View File

@ -25,38 +25,10 @@ macro(ov_npm_cpack_set_dirs)
set(OV_CPACK_DEVREQDIR .) set(OV_CPACK_DEVREQDIR .)
set(OV_CPACK_PYTHONDIR .) set(OV_CPACK_PYTHONDIR .)
if(WIN32)
set(OV_CPACK_LIBRARYDIR .)
set(OV_CPACK_RUNTIMEDIR .)
set(OV_CPACK_ARCHIVEDIR .)
elseif(APPLE)
set(OV_CPACK_LIBRARYDIR .)
set(OV_CPACK_RUNTIMEDIR .)
set(OV_CPACK_ARCHIVEDIR .)
else()
set(OV_CPACK_LIBRARYDIR .)
set(OV_CPACK_RUNTIMEDIR .)
set(OV_CPACK_ARCHIVEDIR .)
endif()
set(OV_CPACK_LIBRARYDIR .) set(OV_CPACK_LIBRARYDIR .)
set(OV_CPACK_ARCHIVEDIR .) set(OV_CPACK_ARCHIVEDIR .)
set(OV_CPACK_PLUGINSDIR .) set(OV_CPACK_PLUGINSDIR .)
set(OV_CPACK_IE_CMAKEDIR .) set(OV_CPACK_RUNTIMEDIR .)
set(OV_CPACK_NGRAPH_CMAKEDIR .)
set(OV_CPACK_OPENVINO_CMAKEDIR .)
set(OV_CPACK_DOCDIR .)
set(OV_CPACK_LICENSESDIR licenses)
set(OV_CPACK_PYTHONDIR .)
# non-native stuff
set(OV_CPACK_SHAREDIR .)
set(OV_CPACK_SAMPLESDIR .)
set(OV_CPACK_DEVREQDIR .)
unset(OV_CPACK_SHAREDIR)
# skipped during debian packaging
set(OV_CPACK_WHEELSDIR .)
endmacro() endmacro()
ov_npm_cpack_set_dirs() ov_npm_cpack_set_dirs()

View File

@ -122,6 +122,32 @@ function(ov_install_with_name file component)
endif() endif()
endfunction() endfunction()
#
# checks that current OpenVINO versions has previous version in RPM / DEB conflicts
#
function(ov_check_conflicts_versions var_name)
set(ov_major ${OpenVINO_VERSION_MAJOR})
set(ov_minor ${OpenVINO_VERSION_MINOR})
set(ov_patch ${OpenVINO_VERSION_PATCH})
if(ov_patch EQUAL 0)
if(ov_minor EQUAL 0)
math(EXPR ov_major "${ov_major} - 1")
else()
math(EXPR ov_minor "${ov_minor} - 1")
endif()
else()
math(EXPR ov_patch "${ov_patch} - 1")
endif()
set(ov_prev_version "${ov_major}.${ov_minor}.${ov_patch}")
# perform check
if(NOT ov_prev_version IN_LIST ${var_name})
message(FATAL_ERROR "List ${var_name} (${${var_name}}) does not contain verison ${ov_prev_version}")
endif()
endfunction()
# #
# List of public OpenVINO components # List of public OpenVINO components
# #

View File

@ -193,6 +193,9 @@ ov_dependent_option (ENABLE_SYSTEM_SNAPPY "Enables use of system version of Snap
ov_dependent_option (ENABLE_PYTHON_PACKAGING "Enables packaging of Python API in APT / YUM" OFF ov_dependent_option (ENABLE_PYTHON_PACKAGING "Enables packaging of Python API in APT / YUM" OFF
"ENABLE_PYTHON;UNIX" OFF) "ENABLE_PYTHON;UNIX" OFF)
ov_dependent_option (ENABLE_JS "Enables JS API building" ON
"NOT WIN32" OFF)
ov_option(ENABLE_OPENVINO_DEBUG "Enable output for OPENVINO_DEBUG statements" OFF) ov_option(ENABLE_OPENVINO_DEBUG "Enable output for OPENVINO_DEBUG statements" OFF)
if(NOT BUILD_SHARED_LIBS AND ENABLE_OV_TF_FRONTEND) if(NOT BUILD_SHARED_LIBS AND ENABLE_OV_TF_FRONTEND)

View File

@ -92,6 +92,8 @@ macro(ov_cpack_settings)
2023.2.0 2023.2.0
) )
ov_check_conflicts_versions(conflicting_versions)
# #
# core: base dependency for each component # core: base dependency for each component
# #

View File

@ -78,6 +78,8 @@ macro(ov_cpack_settings)
2023.2.0 2023.2.0
) )
ov_check_conflicts_versions(conflicting_versions)
find_host_program(rpmlint_PROGRAM NAMES rpmlint DOC "Path to rpmlint") find_host_program(rpmlint_PROGRAM NAMES rpmlint DOC "Path to rpmlint")
if(rpmlint_PROGRAM) if(rpmlint_PROGRAM)
execute_process(COMMAND "${rpmlint_PROGRAM}" --version execute_process(COMMAND "${rpmlint_PROGRAM}" --version

View File

@ -59,7 +59,7 @@ function(build_docs)
if(${ENABLE_PYTHON_API}) if(${ENABLE_PYTHON_API})
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E cmake_echo_color --green "STARTED preprocessing OpenVINO Python API") list(APPEND commands COMMAND ${CMAKE_COMMAND} -E cmake_echo_color --green "STARTED preprocessing OpenVINO Python API")
list(APPEND commands COMMAND ${Python3_EXECUTABLE} -m pip install openvino) list(APPEND commands COMMAND ${Python3_EXECUTABLE} -m pip install openvino==2023.3)
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E cmake_echo_color --green "FINISHED preprocessing OpenVINO Python API") list(APPEND commands COMMAND ${CMAKE_COMMAND} -E cmake_echo_color --green "FINISHED preprocessing OpenVINO Python API")
endif() endif()

Binary file not shown.

Before

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 101 KiB

View File

@ -12,6 +12,7 @@ About OpenVINO
compatibility_and_support compatibility_and_support
system_requirements system_requirements
Release Notes <openvino_release_notes> Release Notes <openvino_release_notes>
release_policy
Additional Resources <resources> Additional Resources <resources>
OpenVINO is a toolkit for simple and efficient deployment of various deep learning models. OpenVINO is a toolkit for simple and efficient deployment of various deep learning models.
@ -24,7 +25,7 @@ OpenVINO (Open Visual Inference and Neural network Optimization) is an open-sour
Features Features
############################################################## ##############################################################
One of the main purposes of OpenVINO is to streamline the deployment of deep learning models in user applications. It optimizes and accelerates model inference, which is crucial for such domains as Generative AI, Large Language models, and use cases like object detection, classification, segmentation, and many others. One of the main purposes of OpenVINO is to streamline the deployment of deep learning models in user applications. It optimizes and accelerates model inference, which is crucial for such domains as Generative AI, Large Language models, and use cases like object detection, classification, segmentation, and many others.
* :doc:`Model Optimization <openvino_docs_model_optimization_guide>` * :doc:`Model Optimization <openvino_docs_model_optimization_guide>`
@ -32,7 +33,7 @@ OpenVINO provides multiple optimization methods for both the training and post-t
* :doc:`Model Conversion and Framework Compatibility <openvino_docs_model_processing_introduction>` * :doc:`Model Conversion and Framework Compatibility <openvino_docs_model_processing_introduction>`
Supported models can be loaded directly or converted to the OpenVINO format to achieve better performance. Supported frameworks include ONNX, PyTorch, TensorFlow, TensorFlow Lite, Keras, and PaddlePaddle. Supported models can be loaded directly or converted to the OpenVINO format to achieve better performance. Supported frameworks include ONNX, PyTorch, TensorFlow, TensorFlow Lite, Keras, and PaddlePaddle.
* :doc:`Model Inference <openvino_docs_OV_UG_OV_Runtime_User_Guide>` * :doc:`Model Inference <openvino_docs_OV_UG_OV_Runtime_User_Guide>`
@ -40,14 +41,14 @@ OpenVINO accelerates deep learning models on various hardware platforms, ensurin
* `Deployment on a server <https://github.com/openvinotoolkit/model_server>`__ * `Deployment on a server <https://github.com/openvinotoolkit/model_server>`__
A model can be deployed either locally using OpenVINO Runtime or on a model server. Runtime is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions. The model server enables quick model inference using external resources. A model can be deployed either locally using OpenVINO Runtime or on a model server. Runtime is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions. The model server enables quick model inference using external resources.
Architecture Architecture
############################################################## ##############################################################
To learn more about how OpenVINO works, read the Developer documentation on its `architecture <https://github.com/openvinotoolkit/openvino/blob/master/src/docs/architecture.md>`__ and `core components <https://github.com/openvinotoolkit/openvino/blob/master/src/README.md>`__. To learn more about how OpenVINO works, read the Developer documentation on its `architecture <https://github.com/openvinotoolkit/openvino/blob/master/src/docs/architecture.md>`__ and `core components <https://github.com/openvinotoolkit/openvino/blob/master/src/README.md>`__.
OpenVINO Ecosystem OpenVINO Ecosystem
############################################################## ##############################################################
Along with the primary components of model optimization and runtime, the toolkit also includes: Along with the primary components of model optimization and runtime, the toolkit also includes:

View File

@ -8,7 +8,6 @@ Compatibility and Support
:maxdepth: 1 :maxdepth: 1
:hidden: :hidden:
openvino_supported_models
openvino_docs_OV_UG_supported_plugins_Supported_Devices openvino_docs_OV_UG_supported_plugins_Supported_Devices
openvino_resources_supported_operations openvino_resources_supported_operations
openvino_resources_supported_operations_frontend openvino_resources_supported_operations_frontend
@ -16,9 +15,7 @@ Compatibility and Support
:doc:`Supported Devices <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` - compatibility information for supported hardware accelerators. :doc:`Supported Devices <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` - compatibility information for supported hardware accelerators.
:doc:`Supported Models <openvino_supported_models>` - a table of models officially supported by OpenVINO. :doc:`Supported Operations <openvino_resources_supported_operations>` - a listing of framework layers supported by OpenVINO.
:doc:`Supported Operations <openvino_resources_supported_operations>` - a listing of framework layers supported by OpenVINO.
:doc:`Supported Operations <openvino_resources_supported_operations_frontend>` - a listing of layers supported by OpenVINO inference devices. :doc:`Supported Operations <openvino_resources_supported_operations_frontend>` - a listing of layers supported by OpenVINO inference devices.

View File

@ -1,36 +0,0 @@
.. {#openvino_supported_models}
Supported Models
================
.. meta::
:description: Check the list of officially supported models in Intel®
Distribution of OpenVINO™ toolkit.
The OpenVINO team continues the effort to support as many models out-of-the-box as possible.
Based on our research and user feedback, we prioritize the most common models and test them
before every release. These models are considered officially supported.
.. button-link:: _static/download/OV_2023_models_supported.pdf
:color: primary
:outline:
:material-regular:`download;1.5em` Click for supported models [PDF]
The list is based on release 2023.0, as of June 01, 2023
| Note that the list provided here does not include all models supported by OpenVINO.
| If your model is not included but is similar to those that are, it is still very likely to work.
If your model fails to execute properly there are a few options available:
* You can create a GitHub request for the operation(s) that are missing. These requests are reviewed regularly. You will be informed if and how the request will be accommodated. Additionally, your request may trigger a reply from someone in the community who can help.
* As OpenVINO™ is open source you can enhance it with your own contribution to the GitHub repository. To learn more, see the articles on :doc:`OpenVINO Extensibility <openvino_docs_Extensibility_UG_Intro>`.

View File

@ -18,239 +18,372 @@ This page lists operations supported by OpenVINO Framework Frontend.
========================================== ========================================================================================== ========================================== ==========================================================================================
PyTorch Supported Operations Limitations PyTorch Supported Operations Limitations
========================================== ========================================================================================== ========================================== ==========================================================================================
aten::__and__ Only for boolean inputs aten::__and__
aten::__getitem__ Supported in limitted set of patterns aten::__derive_index
aten::__not__ aten::__getitem__
aten::__or__ aten::__not__
aten::_convolution aten::__or__
aten::_convolution_mode aten::__range_length
aten::_set_item aten::__xor__
aten::abs aten::_convolution
aten::acos aten::_convolution_mode
aten::acos_ aten::_native_multi_head_attention
aten::acosh aten::_set_item
aten::acosh_ aten::_shape_as_tensor
aten::adaptive_avg_pool2d aten::_upsample_bicubic2d_aa
aten::adaptive_avg_pool3d aten::_upsample_bilinear2d_aa
aten::adaptive_max_pool2d aten::_weight_norm
aten::add aten::abs
aten::add_ aten::acos
aten::addcmul aten::acos_
aten::addmm aten::acosh
aten::append Supported in limitted set of patterns aten::acosh_
aten::arange aten::adaptive_avg_pool1d
aten::argsort aten::adaptive_avg_pool2d
aten::as_tensor aten::adaptive_avg_pool3d
aten::asin aten::adaptive_max_pool1d
aten::asin_ aten::adaptive_max_pool2d
aten::asinh aten::adaptive_max_pool3d
aten::asinh_ aten::add
aten::atan aten::add_
aten::atan_ aten::addcmul
aten::atanh aten::addmm
aten::atanh_ aten::alias
aten::avg_pool1d aten::alias_copy
aten::avg_pool2d aten::all
aten::avg_pool3d aten::amax
aten::baddbmm aten::amin
aten::batch_norm aten::append Supported in limited set of patterns
aten::bitwise_not aten::arange
aten::bmm aten::argmax
aten::Bool aten::argmin
aten::cat aten::argsort
aten::ceil aten::as_strided
aten::ceil_ aten::as_tensor
aten::chunk Supported in limitted set of patterns aten::asin
aten::clamp aten::asin_
aten::clamp_max aten::asinh
aten::clamp_min aten::asinh_
aten::clone aten::atan
aten::contiguous aten::atan_
aten::conv_transpose1d aten::atanh
aten::conv_transpose2d aten::atanh_
aten::conv_transpose3d aten::avg_pool1d
aten::conv1d aten::avg_pool2d
aten::conv2d aten::avg_pool3d
aten::conv3d aten::baddbmm
aten::convolution aten::batch_norm
aten::copy aten::bitwise_and
aten::cos aten::bitwise_not
aten::cos_ aten::bitwise_or
aten::cosh aten::bitwise_xor
aten::cosh_ aten::bmm
aten::cumsum aten::Bool
aten::detach aten::broadcast_tensors Supported in limited set of patterns
aten::dim aten::broadcast_to
aten::div aten::cat
aten::div_ aten::cdist
aten::dropout aten::ceil
aten::dropout_ aten::ceil_
aten::einsum Supported in limitted set of patterns aten::channel_shuffle
aten::elu aten::chunk Supported in limited set of patterns
aten::embedding aten::clamp
aten::empty aten::clamp_
aten::eq aten::clamp_max
aten::exp aten::clamp_min
aten::expand Supported in limitted set of patterns aten::clip
aten::expand_as aten::clip_
aten::eye aten::clone
aten::fill_ aten::complex Supported in limited set of patterns
aten::flatten aten::concat
aten::floor aten::contiguous
aten::floor_ aten::conv1d
aten::floor_divide aten::conv2d
aten::floordiv aten::conv3d
aten::full aten::conv_transpose1d
aten::full_like aten::conv_transpose2d
aten::gather aten::conv_transpose3d
aten::ge aten::convolution
aten::gelu aten::copy
aten::glu aten::copy_
aten::grid_sampler aten::cos
aten::group_norm aten::cos_
aten::gt aten::cosh
aten::hardsigmoid aten::cosh_
aten::hardswish aten::cross
aten::hardswish_ aten::cumsum
aten::hardtanh aten::dequantize
aten::hardtanh_ aten::detach
aten::im2col aten::dim
aten::index Supported in limitted set of patterns aten::div
aten::index_put_ Supported in limitted set of patterns aten::div_
aten::index_select aten::dropout
aten::instance_norm aten::dropout_
aten::Int aten::einsum Supported in limited set of patterns
aten::IntImplicit aten::elu
aten::is_grad_enabled aten::embedding
aten::layer_norm aten::embedding_bag
aten::le aten::empty
aten::leaky_relu aten::empty_like
aten::leaky_relu_ aten::eq
aten::len aten::erf
aten::linalg_matrix_norm aten::erf_
aten::linalg_norm aten::erfc
aten::linalg_vector_norm aten::erfc_
aten::linear aten::exp
aten::log aten::exp_
aten::log_ aten::expand
aten::log2 aten::expand_as
aten::log2_ aten::eye
aten::lt aten::fake_quantize_per_channel_affine
aten::masked_fill aten::fake_quantize_per_tensor_affine
aten::masked_fill_ aten::feature_dropout
aten::matmul aten::fft_irfftn Supported in limited set of patterns
aten::max aten::fft_rfftn Supported in limited set of patterns
aten::max_pool1d aten::fill
aten::max_pool2d aten::fill_
aten::max_pool3d aten::fill_diagonal_
aten::mean aten::flatten
aten::meshgrid Supported in limitted set of patterns aten::flip
aten::min aten::floor
aten::mm aten::floor_
aten::mul aten::floor_divide
aten::mul_ aten::floordiv
aten::narrow aten::fmod
aten::ne aten::frobenius_norm
aten::neg aten::full
aten::new_empty aten::full_like
aten::new_full aten::gather
aten::new_ones aten::ge
aten::new_zeros aten::gelu
aten::nonzero aten::glu
aten::nonzero_numpy Supported in limitted set of patterns aten::grid_sampler
aten::norm aten::group_norm
aten::numel aten::gru
aten::ones aten::gt
aten::ones_like aten::hardsigmoid
aten::pad Supported in limitted set of patterns aten::hardswish
aten::permute aten::hardswish_
aten::pow aten::hardtanh
aten::reciprocal aten::hardtanh_
aten::relu aten::im2col
aten::relu_ aten::imag Supported in limited set of patterns
aten::relu6 aten::index Supported in limited set of patterns
aten::remainder aten::index_put_
aten::repeat aten::index_select
aten::repeat_interleave aten::instance_norm
aten::reshape aten::Int
aten::reshape_as aten::IntImplicit
aten::roll aten::is_grad_enabled
aten::rsqrt aten::is_nonzero
aten::rsub aten::item
aten::ScalarImplicit aten::layer_norm
aten::scaled_dot_product_attention aten::le
aten::select aten::leaky_relu
aten::selu aten::leaky_relu_
aten::selu_ aten::len
aten::sigmoid aten::lift
aten::sigmoid_ aten::lift_fresh
aten::sign aten::lift_fresh_copy
aten::silu aten::linalg_cross
aten::silu_ aten::linalg_matrix_norm
aten::sin aten::linalg_norm
aten::sin_ aten::linalg_vector_norm
aten::sinh aten::linear
aten::sinh_ aten::linspace
aten::size aten::log
aten::slice aten::log10
aten::softmax aten::log10_
aten::sort aten::log1p
aten::split Supported in limitted set of patterns aten::log1p_
aten::split_with_sizes Supported in limitted set of patterns aten::log2
aten::sqrt aten::log2_
aten::square aten::log_
aten::squeeze aten::log_softmax
aten::stack Supported in limitted set of patterns aten::logical_and
aten::sub aten::logical_not
aten::sum aten::logical_or
aten::tan aten::logical_xor
aten::tan_ aten::lstm
aten::tanh aten::lt
aten::tanh_ aten::masked_fill
aten::tensor aten::masked_fill_
aten::to aten::masked_scatter
aten::topk aten::masked_scatter_
aten::transpose aten::matmul
aten::tril aten::max
aten::triu aten::max_pool1d
aten::type_as aten::max_pool1d_with_indices
aten::unbind Supported in limitted set of patterns aten::max_pool2d
aten::unfold aten::max_pool2d_with_indices
aten::unsqueeze aten::max_pool3d
aten::unsqueeze_ aten::max_pool3d_with_indices
aten::upsample_bicubic2d aten::maximum
aten::upsample_bilinear2d aten::mean
aten::upsample_linear1d aten::meshgrid
aten::upsample_nearest1d aten::min
aten::upsample_nearest2d aten::minimum
aten::upsample_nearest3d aten::mm
aten::upsample_trilinear3d aten::mul
aten::var aten::mul_
aten::var_mean aten::multinomial
aten::view aten::multiply
aten::where Supported in limitted set of patterns aten::multiply_
aten::zeros aten::narrow
aten::zeros_like aten::ne
prim::Constant String and None constant is only supported when used by supported operation aten::neg
prim::device aten::new_empty
prim::DictConstruct Supported in limitted set of patterns aten::new_full
prim::dtype Supported in limitted set of patterns aten::new_ones
prim::GetAttr aten::new_zeros
prim::If aten::nonzero
prim::is_cuda aten::nonzero_numpy Supported in limited set of patterns
prim::ListConstruct Supported in limitted set of patterns aten::norm
prim::ListUnpack Supported in limitted set of patterns aten::normal
prim::Loop aten::normal_
prim::max Supported in limitted set of patterns aten::numel
prim::min Supported in limitted set of patterns aten::numpy_T
prim::NumToTensor aten::one_hot
prim::PythonOp Supported only in tracing aten::ones
prim::requires_grad aten::ones_like
prim::TupleConstruct Supported in limitted set of patterns aten::outer
prim::type aten::pad
torchvision::deform_conv2d aten::pairwise_distance
torchvision::nms aten::permute
torchvision::roi_align aten::pixel_shuffle
aten::pixel_unshuffle
aten::pow
aten::pow_
aten::prelu
aten::prod
aten::quantize_per_channel
aten::quantize_per_tensor
aten::rand
aten::rand_like
aten::randint
aten::randn
aten::randn_like
aten::real Supported in limited set of patterns
aten::reciprocal
aten::reflection_pad2d Supported in limited set of patterns
aten::relu
aten::relu6
aten::relu_
aten::remainder
aten::repeat
aten::repeat_interleave
aten::reshape
aten::reshape_as
aten::resolve_conj
aten::resolve_neg
aten::rnn_relu
aten::rnn_tanh
aten::roll
aten::round
aten::rsqrt
aten::rsub
aten::ScalarImplicit
aten::scaled_dot_product_attention
aten::scatter
aten::scatter_
aten::scatter_add
aten::scatter_add_
aten::scatter_reduce
aten::scatter_reduce_
aten::select
aten::selu
aten::selu_
aten::sigmoid
aten::sigmoid_
aten::sign
aten::silu
aten::silu_
aten::sin
aten::sin_
aten::sinh
aten::sinh_
aten::size
aten::slice
aten::softmax
aten::softplus
aten::sort
aten::split Supported in limited set of patterns
aten::split_with_sizes Supported in limited set of patterns
aten::sqrt
aten::square
aten::squeeze
aten::stack Supported in limited set of patterns
aten::std
aten::std_mean
aten::sub
aten::sub_
aten::sum
aten::swapaxes
aten::t
aten::t_
aten::take_along_dim
aten::tan
aten::tan_
aten::tanh
aten::tanh_
aten::tensor
aten::tensor_split Supported in limited set of patterns
aten::tile
aten::to
aten::topk
aten::transpose
aten::tril
aten::tril_
aten::triu
aten::triu_
aten::type_as
aten::unbind Supported in limited set of patterns
aten::unflatten
aten::unfold
aten::unsqueeze
aten::unsqueeze_
aten::upsample_bicubic2d
aten::upsample_bilinear2d
aten::upsample_linear1d
aten::upsample_nearest1d
aten::upsample_nearest2d
aten::upsample_nearest3d
aten::upsample_trilinear3d
aten::var
aten::var_mean
aten::view
aten::view_as
aten::where
aten::zero_
aten::zeros
aten::zeros_like
prim::Constant
prim::device
prim::DictConstruct Supported in limited set of patterns
prim::GetAttr
prim::If
prim::is_cuda
prim::ListConstruct
prim::ListUnpack
prim::Loop
prim::NumToTensor
prim::PythonOp
prim::requires_grad
prim::TupleConstruct Supported in limited set of patterns
prim::TupleIndex
prim::TupleUnpack Supported in limited set of patterns
prim::type
quantized::add
quantized::add_relu
quantized::cat
quantized::conv2d
quantized::conv2d_relu
quantized::hardswish
quantized::linear
quantized::mul
torchvision::deform_conv2d
torchvision::nms
torchvision::roi_align
========================================== ========================================================================================== ========================================== ==========================================================================================
.. tab-item:: ONNX .. tab-item:: ONNX

View File

@ -7,163 +7,171 @@ Supported Operations - by Inference Devices
This page lists operations supported by OpenVINO inference devices. The table presents general information, This page lists operations supported by OpenVINO inference devices. The table presents general information,
for a more detailed and most recent listing of operations that are implemented and tested: for a more detailed and most recent listing of operations that are implemented and tested:
.. button-link:: _static/download/operation_conformance_table_files/opset_report_omz_static.html
.. button-link:: _static/download/conformance_reports/opset_report_omz_static.html
:color: primary :color: primary
:outline: :outline:
See the full conformance report table See the full conformance report table (static)
.. button-link:: _static/download/conformance_reports/opset_report_omz_dynamic.html
:color: primary
:outline:
See the full conformance report table (dynamic)
================================= =============== ============== ================ ================== ================================= =============== ============== ================ ==================
Operations CPU (x86) GPU GNA CPU (Arm®) Operations CPU (x86) GPU GNA CPU (Arm®)
================================= =============== ============== ================ ================== ================================= =============== ============== ================ ==================
Abs Supported** Supported Not Supported Supported Abs Supported** Supported Not Supported Supported
Acos Supported** Supported Not Supported Supported**** Acos Supported** Supported Not Supported Supported****
Acosh Supported** Supported Not Supported Supported**** Acosh Supported** Supported Not Supported Supported****
Activation-Clamp Supported*** Supported Supported Supported Activation-Clamp Supported*** Supported Supported Supported
Activation-ELU Supported*** Supported Not Supported Supported Activation-ELU Supported*** Supported Not Supported Supported
Activation-Exp Supported*** Supported Supported Supported Activation-Exp Supported*** Supported Supported Supported
Activation-Leaky ReLU Supported*** Supported Supported Not Supported Activation-Leaky ReLU Supported*** Supported Supported Not Supported
Activation-Not Supported*** Supported Not Supported Not Supported Activation-Not Supported*** Supported Not Supported Not Supported
Activation-PReLU Supported*** Supported Not Supported Supported Activation-PReLU Supported*** Supported Not Supported Supported
Activation-ReLU Supported*** Supported Supported Supported Activation-ReLU Supported*** Supported Supported Supported
Activation-ReLU6 Supported*** Supported Not Supported Not Supported Activation-ReLU6 Supported*** Supported Not Supported Not Supported
Activation-Sigmoid/Logistic Supported*** Supported Supported Supported Activation-Sigmoid/Logistic Supported*** Supported Supported Supported
Activation-TanH Supported*** Supported Supported Supported Activation-TanH Supported*** Supported Supported Supported
ArgMax Supported** Supported Not Supported Not Supported ArgMax Supported** Supported Not Supported Not Supported
Asin Supported** Supported Not Supported Supported**** Asin Supported** Supported Not Supported Supported****
Asinh Supported** Supported Not Supported Supported**** Asinh Supported** Supported Not Supported Supported****
Atan Supported** Supported Not Supported Supported**** Atan Supported** Supported Not Supported Supported****
Atanh Supported** Supported Not Supported Supported**** Atanh Supported** Supported Not Supported Supported****
BatchNormalization Supported Supported Not Supported Supported BatchNormalization Supported Supported Not Supported Supported
BinaryConvolution Supported Supported Not Supported Not Supported BinaryConvolution Supported Supported Not Supported Not Supported
Broadcast Supported** Supported Not Supported Supported Broadcast Supported** Supported Not Supported Supported
Ceil Supported** Supported Not Supported Supported Ceil Supported** Supported Not Supported Supported
Concat Supported*** Supported Supported Supported Concat Supported*** Supported Supported Supported
Const Supported Supported Supported Supported Const Supported Supported Supported Supported
Convolution-Dilated Supported Supported Not Supported Supported Convolution-Dilated Supported Supported Not Supported Supported
Convolution-Dilated 3D Supported Supported Not Supported Not Supported Convolution-Dilated 3D Supported Supported Not Supported Not Supported
Convolution-Grouped Supported Supported Not Supported Supported Convolution-Grouped Supported Supported Not Supported Supported
Convolution-Grouped 3D Supported Supported Not Supported Not Supported Convolution-Grouped 3D Supported Supported Not Supported Not Supported
Convolution-Ordinary Supported Supported Supported* Supported Convolution-Ordinary Supported Supported Supported* Supported
Convolution-Ordinary 3D Supported Supported Not Supported Not Supported Convolution-Ordinary 3D Supported Supported Not Supported Not Supported
Cos Supported** Supported Not Supported Supported**** Cos Supported** Supported Not Supported Supported****
Cosh Supported** Supported Not Supported Supported**** Cosh Supported** Supported Not Supported Supported****
Crop Supported Supported Supported Not Supported Crop Supported Supported Supported Not Supported
CTCGreedyDecoder Supported** Supported** Not Supported Supported**** CTCGreedyDecoder Supported** Supported** Not Supported Supported****
Deconvolution Supported Supported Not Supported Not Supported Deconvolution Supported Supported Not Supported Not Supported
Deconvolution 3D Supported Supported Not Supported Not Supported Deconvolution 3D Supported Supported Not Supported Not Supported
DeformableConvolution Supported Supported Not Supported Not Supported DeformableConvolution Supported Supported Not Supported Not Supported
DepthToSpace Supported** Supported Not Supported Supported* DepthToSpace Supported** Supported Not Supported Supported*
DetectionOutput Supported** Supported Not Supported Supported**** DetectionOutput Supported** Supported Not Supported Supported****
Eltwise-And Supported*** Supported Not Supported Supported Eltwise-And Supported*** Supported Not Supported Supported
Eltwise-Add Supported*** Supported Not Supported Supported Eltwise-Add Supported*** Supported Not Supported Supported
Eltwise-Div Supported*** Supported Not Supported Supported Eltwise-Div Supported*** Supported Not Supported Supported
Eltwise-Equal Supported*** Supported Not Supported Supported* Eltwise-Equal Supported*** Supported Not Supported Supported*
Eltwise-FloorMod Supported*** Supported Not Supported Supported**** Eltwise-FloorMod Supported*** Supported Not Supported Supported****
Eltwise-Greater Supported*** Supported Not Supported Supported Eltwise-Greater Supported*** Supported Not Supported Supported
Eltwise-GreaterEqual Supported*** Supported Not Supported Supported Eltwise-GreaterEqual Supported*** Supported Not Supported Supported
Eltwise-Less Supported*** Supported Not Supported Supported* Eltwise-Less Supported*** Supported Not Supported Supported*
Eltwise-LessEqual Supported*** Supported Not Supported Supported* Eltwise-LessEqual Supported*** Supported Not Supported Supported*
Eltwise-LogicalAnd Supported*** Supported Not Supported Supported Eltwise-LogicalAnd Supported*** Supported Not Supported Supported
Eltwise-LogicalOr Supported*** Supported Not Supported Supported Eltwise-LogicalOr Supported*** Supported Not Supported Supported
Eltwise-LogicalXor Supported*** Supported Not Supported Supported Eltwise-LogicalXor Supported*** Supported Not Supported Supported
Eltwise-Max Supported*** Supported Not Supported Supported Eltwise-Max Supported*** Supported Not Supported Supported
Eltwise-Min Supported*** Supported Not Supported Supported Eltwise-Min Supported*** Supported Not Supported Supported
Eltwise-Mul Supported*** Supported Supported Supported Eltwise-Mul Supported*** Supported Supported Supported
Eltwise-NotEqual Supported*** Supported Not Supported Supported* Eltwise-NotEqual Supported*** Supported Not Supported Supported*
Eltwise-Pow Supported*** Supported Not Supported Supported Eltwise-Pow Supported*** Supported Not Supported Supported
Eltwise-Prod Supported*** Supported Supported Not Supported Eltwise-Prod Supported*** Supported Supported Not Supported
Eltwise-SquaredDiff Supported*** Supported Not Supported Supported Eltwise-SquaredDiff Supported*** Supported Not Supported Supported
Eltwise-Sub Supported*** Supported Supported Supported Eltwise-Sub Supported*** Supported Supported Supported
Eltwise-Sum Supported*** Supported Supported Supported**** Eltwise-Sum Supported*** Supported Supported Supported****
Erf Supported** Supported Not Supported Supported**** Erf Supported** Supported Not Supported Supported****
Exp Supported Supported Supported Supported Exp Supported Supported Supported Supported
FakeQuantize Supported Not Supported Not Supported Supported* FakeQuantize Supported Not Supported Not Supported Supported*
Fill Supported** Not Supported Not Supported Not Supported Fill Supported** Not Supported Not Supported Not Supported
Flatten Supported Supported Not Supported Not Supported Flatten Supported Supported Not Supported Not Supported
Floor Supported** Supported Not Supported Supported Floor Supported** Supported Not Supported Supported
FullyConnected (Inner Product) Supported*** Supported Supported Supported FullyConnected (Inner Product) Supported*** Supported Supported Supported
Gather Supported** Supported Not Supported Supported* Gather Supported** Supported Not Supported Supported*
GatherTree Supported** Not Supported Not Supported Supported**** GatherTree Supported** Not Supported Not Supported Supported****
Gemm Supported Supported Not Supported Not Supported Gemm Supported Supported Not Supported Not Supported
GRN Supported** Supported** Not Supported Supported GRN Supported** Supported** Not Supported Supported
HardSigmoid Supported** Supported Not Supported Supported**** HardSigmoid Supported** Supported Not Supported Supported****
Interp Supported** Supported** Not Supported Supported* Interp Supported** Supported** Not Supported Supported*
Log Supported** Supported Supported Supported Log Supported** Supported Supported Supported
LRN (Norm) Supported Supported Not Supported Supported* LRN (Norm) Supported Supported Not Supported Supported*
LSTMCell Supported Supported Supported Supported LSTMCell Supported Supported Supported Supported
GRUCell Supported Supported Supported Supported GRUCell Supported Supported Supported Supported
RNNCell Supported Supported Not Supported Supported RNNCell Supported Supported Not Supported Supported
LSTMSequence Supported Supported Supported Supported**** LSTMSequence Supported Supported Supported Supported****
GRUSequence Supported Supported Supported Supported**** GRUSequence Supported Supported Supported Supported****
RNNSequence Supported Supported Not Supported Supported**** RNNSequence Supported Supported Not Supported Supported****
LogSoftmax Supported** Supported Not Supported Supported LogSoftmax Supported** Supported Not Supported Supported
Memory Supported Not Supported Supported Not Supported Memory Supported Not Supported Supported Not Supported
MVN Supported** Supported Not Supported Supported* MVN Supported** Supported Not Supported Supported*
Neg Supported** Supported Not Supported Supported Neg Supported** Supported Not Supported Supported
NonMaxSuppression Supported** Not Supported Not Supported Supported**** NonMaxSuppression Supported** Not Supported Not Supported Supported****
Normalize Supported** Supported Not Supported Supported* Normalize Supported** Supported Not Supported Supported*
OneHot Supported** Supported Not Supported Supported**** OneHot Supported** Supported Not Supported Supported****
Pad Supported** Supported Not Supported Supported* Pad Supported** Supported Not Supported Supported*
Permute Supported Supported Supported* Not Supported Permute Supported Supported Supported* Not Supported
Pooling(AVG,MAX) Supported Supported Supported Supported Pooling(AVG,MAX) Supported Supported Supported Supported
Pooling(AVG,MAX) 3D Supported Supported Not Supported Supported* Pooling(AVG,MAX) 3D Supported Supported Not Supported Supported*
Power Supported** Supported Supported* Supported Power Supported** Supported Supported* Supported
PowerFile Supported** Not Supported Not Supported Not Supported PowerFile Supported** Not Supported Not Supported Not Supported
PriorBox Supported** Supported Not Supported Supported PriorBox Supported** Supported Not Supported Supported
PriorBoxClustered Supported** Supported** Not Supported Supported PriorBoxClustered Supported** Supported** Not Supported Supported
Proposal Supported** Supported Not Supported Supported**** Proposal Supported** Supported Not Supported Supported****
PSROIPooling Supported** Supported Not Supported Supported**** PSROIPooling Supported** Supported Not Supported Supported****
Range Supported** Not Supported Not Supported Not Supported Range Supported** Not Supported Not Supported Not Supported
Reciprocal Supported** Supported Not Supported Not Supported Reciprocal Supported** Supported Not Supported Not Supported
ReduceAnd Supported** Supported Not Supported Supported**** ReduceAnd Supported** Supported Not Supported Supported****
ReduceL1 Supported** Supported Not Supported Supported ReduceL1 Supported** Supported Not Supported Supported
ReduceL2 Supported** Supported Not Supported Supported ReduceL2 Supported** Supported Not Supported Supported
ReduceLogSum Supported** Supported Not Supported Supported ReduceLogSum Supported** Supported Not Supported Supported
ReduceLogSumExp Supported** Supported Not Supported Not Supported ReduceLogSumExp Supported** Supported Not Supported Not Supported
ReduceMax Supported** Supported Not Supported Supported ReduceMax Supported** Supported Not Supported Supported
ReduceMean Supported** Supported Not Supported Supported ReduceMean Supported** Supported Not Supported Supported
ReduceMin Supported** Supported Not Supported Supported ReduceMin Supported** Supported Not Supported Supported
ReduceOr Supported** Supported Not Supported Supported**** ReduceOr Supported** Supported Not Supported Supported****
ReduceProd Supported** Supported Not Supported Supported ReduceProd Supported** Supported Not Supported Supported
ReduceSum Supported** Supported Not Supported Supported ReduceSum Supported** Supported Not Supported Supported
ReduceSumSquare Supported** Supported Not Supported Not Supported ReduceSumSquare Supported** Supported Not Supported Not Supported
RegionYolo Supported** Supported Not Supported Supported**** RegionYolo Supported** Supported Not Supported Supported****
ReorgYolo Supported** Supported Not Supported Supported ReorgYolo Supported** Supported Not Supported Supported
Resample Supported** Supported Not Supported Not Supported Resample Supported** Supported Not Supported Not Supported
Reshape Supported*** Supported Supported Supported Reshape Supported*** Supported Supported Supported
ReverseSequence Supported** Supported Not Supported Supported**** ReverseSequence Supported** Supported Not Supported Supported****
RNN Supported Not Supported Not Supported Supported RNN Supported Not Supported Not Supported Supported
ROIPooling Supported Supported* Not Supported Supported**** ROIPooling Supported Supported* Not Supported Supported****
ScaleShift Supported*** Supported Supported Not Supported ScaleShift Supported*** Supported Supported Not Supported
ScatterUpdate Supported** Not Supported Not Supported Not Supported ScatterUpdate Supported** Not Supported Not Supported Not Supported
Select Supported Supported Not Supported Supported Select Supported Supported Not Supported Supported
Selu Supported** Supported Not Supported Supported**** Selu Supported** Supported Not Supported Supported****
ShuffleChannels Supported** Supported Not Supported Supported ShuffleChannels Supported** Supported Not Supported Supported
Sign Supported** Supported Not Supported Supported Sign Supported** Supported Not Supported Supported
Sin Supported** Supported Not Supported Supported Sin Supported** Supported Not Supported Supported
Sinh Supported** Supported Not Supported Supported**** Sinh Supported** Supported Not Supported Supported****
SimplerNMS Supported** Supported Not Supported Not Supported SimplerNMS Supported** Supported Not Supported Not Supported
Slice Supported*** Supported Supported Not Supported Slice Supported*** Supported Supported Not Supported
SoftMax Supported*** Supported Not Supported Supported SoftMax Supported*** Supported Not Supported Supported
Softplus Supported** Supported Not Supported Supported Softplus Supported** Supported Not Supported Supported
Softsign Supported** Supported Supported Not Supported Softsign Supported** Supported Supported Not Supported
SpaceToDepth Supported** Not Supported Not Supported Supported* SpaceToDepth Supported** Not Supported Not Supported Supported*
SpatialTransformer Supported** Not Supported Not Supported Not Supported SpatialTransformer Supported** Not Supported Not Supported Not Supported
Split Supported*** Supported Supported Supported Split Supported*** Supported Supported Supported
Squeeze Supported** Supported Supported Supported Squeeze Supported** Supported Supported Supported
StridedSlice Supported** Supported Not Supported Supported* StridedSlice Supported** Supported Not Supported Supported*
Tan Supported** Supported Not Supported Supported**** Tan Supported** Supported Not Supported Supported****
TensorIterator Supported Not Supported Supported Supported TensorIterator Supported Not Supported Supported Supported
Tile Supported*** Supported** Not Supported Supported Tile Supported*** Supported** Not Supported Supported
TopK Supported** Supported Not Supported Supported**** TopK Supported** Supported Not Supported Supported****
Unpooling Not Supported Supported Not Supported Not Supported Unpooling Not Supported Supported Not Supported Not Supported
Unsqueeze Supported** Supported Supported Supported Unsqueeze Supported** Supported Supported Supported
Upsampling Not Supported Supported Not Supported Not Supported Upsampling Not Supported Supported Not Supported Not Supported
================================= =============== ============== ================ ================== ================================= =============== ============== ================ ==================
| `*` - support is limited to the specific parameters. Refer to "Known Layer Limitations" section for the device :doc:`from the list of supported <openvino_docs_OV_UG_supported_plugins_Supported_Devices>`. | `*` - support is limited to the specific parameters.
| `**` - support is implemented via :doc:`Extensibility mechanism <openvino_docs_Extensibility_UG_Intro>`. | `**` - support is implemented via :doc:`Extensibility mechanism <openvino_docs_Extensibility_UG_Intro>`.
| `***` - supports NCDHW layout. | `***` - supports NCDHW layout.
| `****` - support is implemented via runtime reference. | `****` - support is implemented via runtime reference.

View File

@ -5,8 +5,8 @@ Performance Benchmarks
.. meta:: .. meta::
:description: Use the benchmark results for Intel® Distribution of OpenVINO™ :description: Use the benchmark results for Intel® Distribution of OpenVINO™
toolkit, that may help you decide what hardware to use or how toolkit, that may help you decide what hardware to use or how
to plan the workload. to plan the workload.
.. toctree:: .. toctree::
@ -15,13 +15,12 @@ Performance Benchmarks
openvino_docs_performance_benchmarks_faq openvino_docs_performance_benchmarks_faq
OpenVINO Accuracy <openvino_docs_performance_int8_vs_fp32> OpenVINO Accuracy <openvino_docs_performance_int8_vs_fp32>
Performance Data Spreadsheet (download xlsx) <https://docs.openvino.ai/2023.2/_static/benchmarks_files/OV-2023.2-Performance-Data.xlsx>
openvino_docs_MO_DG_Getting_Performance_Numbers openvino_docs_MO_DG_Getting_Performance_Numbers
This page presents benchmark results for `Intel® Distribution of OpenVINO™ toolkit <https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html>`__ This page presents benchmark results for `Intel® Distribution of OpenVINO™ toolkit <https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html>`__
and :doc:`OpenVINO Model Server <ovms_what_is_openvino_model_server>`, for a representative selection of public neural networks and Intel® devices. and :doc:`OpenVINO Model Server <ovms_what_is_openvino_model_server>`, for a representative selection of public neural networks and Intel® devices.
The results may help you decide which hardware to use in your applications or plan AI workload for the hardware you have already implemented in your solutions. The results may help you decide which hardware to use in your applications or plan AI workload for the hardware you have already implemented in your solutions.
Click the buttons below to see the chosen benchmark data. Click the buttons below to see the chosen benchmark data.
.. grid:: 1 1 2 2 .. grid:: 1 1 2 2
@ -44,53 +43,71 @@ Click the buttons below to see the chosen benchmark data.
:color: primary :color: primary
:outline: :outline:
:expand: :expand:
:material-regular:`bar_chart;1.4em` OVMS Benchmark Graphs :material-regular:`bar_chart;1.4em` OVMS Benchmark Graphs
For a successful deep learning inference application, the following four key metrics need to be considered: Please visit the tabs below for more information on key performance indicators and workload parameters.
.. tab-set:: .. tab-set::
.. tab-item:: Throughput .. tab-item:: Throughput
:sync: throughput :sync: throughput
Measures the number of inferences delivered within a latency threshold Measures the number of inferences delivered within a latency threshold
(for example, number of Frames Per Second - FPS). When deploying a system with (for example, number of Frames Per Second - FPS). When deploying a system with
deep learning inference, select the throughput that delivers the best trade-off deep learning inference, select the throughput that delivers the best trade-off
between latency and power for the price and performance that meets your requirements. between latency and power for the price and performance that meets your requirements.
.. tab-item:: Value .. tab-item:: Value
:sync: value :sync: value
While throughput is important, what is more critical in edge AI deployments is While throughput is important, what is more critical in edge AI deployments is
the performance efficiency or performance-per-cost. Application performance in the performance efficiency or performance-per-cost. Application performance in
throughput per dollar of system cost is the best measure of value. The value KPI is throughput per dollar of system cost is the best measure of value. The value KPI is
calculated as “Throughput measured as inferences per second / price of inference engine”. calculated as “Throughput measured as inferences per second / price of inference engine”.
This means for a 2 socket system 2x the price of a CPU is used. Prices are as per This means for a 2 socket system 2x the price of a CPU is used. Prices are as per
date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below. date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below.
.. tab-item:: Efficiency .. tab-item:: Efficiency
:sync: efficiency :sync: efficiency
System power is a key consideration from the edge to the data center. When selecting System power is a key consideration from the edge to the data center. When selecting
deep learning solutions, power efficiency (throughput/watt) is a critical factor to consider. deep learning solutions, power efficiency (throughput/watt) is a critical factor to consider.
Intel designs provide excellent power efficiency for running deep learning workloads. Intel designs provide excellent power efficiency for running deep learning workloads.
The efficiency KPI is calculated as “Throughput measured as inferences per second / TDP of The efficiency KPI is calculated as “Throughput measured as inferences per second / TDP of
inference engine”. This means for a 2 socket system 2x the power dissipation (TDP) of a CPU is used. inference engine”. This means for a 2 socket system 2x the power dissipation (TDP) of a CPU is used.
TDP-values are as per date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below. TDP-values are as per date of benchmarking and sources can be found as links in the Hardware Platforms (PDF) description below.
.. tab-item:: Latency .. tab-item:: Latency
:sync: latency :sync: latency
This measures the synchronous execution of inference requests and is reported in milliseconds. This measures the synchronous execution of inference requests and is reported in milliseconds.
Each inference request (for example: preprocess, infer, postprocess) is allowed to complete before Each inference request (for example: preprocess, infer, postprocess) is allowed to complete before
the next is started. This performance metric is relevant in usage scenarios where a single image the next is started. This performance metric is relevant in usage scenarios where a single image
input needs to be acted upon as soon as possible. An example would be the healthcare sector where input needs to be acted upon as soon as possible. An example would be the healthcare sector where
medical personnel only request analysis of a single ultra sound scanning image or in real-time or medical personnel only request analysis of a single ultra sound scanning image or in real-time or
near real-time applications for example an industrial robot's response to actions in its environment near real-time applications for example an industrial robot's response to actions in its environment
or obstacle avoidance for autonomous vehicles. or obstacle avoidance for autonomous vehicles.
.. tab-item:: Workload Parameters
:sync: workloadparameters
The workload parameters affect the performance results of the different models we use for benchmarking.
Image processing models have different image size definitions and the Natural Language Processing models
have different max token list lengths. All these can be found in detail in the :doc:`FAQ section <openvino_docs_performance_benchmarks_faq>`.
All models are executed using a batch size of 1. Below are the parameters for the GenAI models we display.
* Input tokens: 1024,
* Output tokens: 128,
* number of beams: 1
For text to image:
* iteration steps: 20,
* image size (HxW): 256 x 256,
* input token length: 1024 (the tokens for GenAI models are in English).
Platforms, Configurations, Methodology Platforms, Configurations, Methodology
########################################################### ###########################################################
@ -102,53 +119,60 @@ For a listing of all platforms and configurations used for testing, refer to the
.. grid-item:: .. grid-item::
.. button-link:: _static/benchmarks_files/OV-2023.2-platform_list.pdf .. button-link:: _static/benchmarks_files/OV-2023.3-platform_list.pdf
:color: primary :color: primary
:outline: :outline:
:expand: :expand:
:material-regular:`download;1.5em` Click for Hardware Platforms [PDF] :material-regular:`download;1.5em` Click for Hardware Platforms [PDF]
.. button-link:: _static/benchmarks_files/OV-2023.2-system-info-detailed.xlsx .. button-link:: _static/benchmarks_files/OV-2023.3-system-info-detailed.xlsx
:color: primary :color: primary
:outline: :outline:
:expand: :expand:
:material-regular:`download;1.5em` Click for Configuration Details [XLSX] :material-regular:`download;1.5em` Click for Configuration Details [XLSX]
.. button-link:: _static/benchmarks_files/OV-2023.3-Performance-Data.xlsx
:color: primary
:outline:
:expand:
:material-regular:`download;1.5em` Click for Performance Data [XLSX]
The OpenVINO benchmark setup includes a single system with OpenVINO™, as well as the benchmark application installed. The OpenVINO benchmark setup includes a single system with OpenVINO™, as well as the benchmark application installed.
It measures the time spent on actual inference (excluding any pre or post processing) and then reports on the inferences It measures the time spent on actual inference (excluding any pre or post processing) and then reports on the inferences
per second (or Frames Per Second). per second (or Frames Per Second).
OpenVINO™ Model Server (OVMS) employs the Intel® Distribution of OpenVINO™ toolkit runtime libraries and exposes a set of OpenVINO™ Model Server (OVMS) employs the Intel® Distribution of OpenVINO™ toolkit runtime libraries and exposes a set of
models via a convenient inference API over gRPC or HTTP/REST. Its benchmark results are measured with the configuration of models via a convenient inference API over gRPC or HTTP/REST. Its benchmark results are measured with the configuration of
multiple-clients-single-server, using two hardware platforms connected by ethernet. Network bandwidth depends on both, platforms multiple-clients-single-server, using two hardware platforms connected by ethernet. Network bandwidth depends on both, platforms
and models under investigation. It is set not to be a bottleneck for workload intensity. The connection is dedicated and models under investigation. It is set not to be a bottleneck for workload intensity. The connection is dedicated
only to measuring performance. only to measuring performance.
.. dropdown:: See more details about OVMS benchmark setup .. dropdown:: See more details about OVMS benchmark setup
The benchmark setup for OVMS consists of four main parts: The benchmark setup for OVMS consists of four main parts:
.. image:: _static/images/performance_benchmarks_ovms_02.png .. image:: _static/images/performance_benchmarks_ovms_02.png
:alt: OVMS Benchmark Setup Diagram :alt: OVMS Benchmark Setup Diagram
* **OpenVINO™ Model Server** is launched as a docker container on the server platform and it listens (and answers on) * **OpenVINO™ Model Server** is launched as a docker container on the server platform and it listens (and answers on)
requests from clients. OpenVINO™ Model Server is run on the same machine as the OpenVINO™ toolkit benchmark application requests from clients. OpenVINO™ Model Server is run on the same machine as the OpenVINO™ toolkit benchmark application
in corresponding benchmarking. Models served by OpenVINO™ Model Server are located in a local file system mounted into in corresponding benchmarking. Models served by OpenVINO™ Model Server are located in a local file system mounted into
the docker container. The OpenVINO™ Model Server instance communicates with other components via ports over a dedicated docker network. the docker container. The OpenVINO™ Model Server instance communicates with other components via ports over a dedicated docker network.
* **Clients** are run in separated physical machine referred to as client platform. Clients are implemented in Python3 * **Clients** are run in separated physical machine referred to as client platform. Clients are implemented in Python3
programming language based on TensorFlow* API and they work as parallel processes. Each client waits for a response from OpenVINO™ programming language based on TensorFlow* API and they work as parallel processes. Each client waits for a response from OpenVINO™
Model Server before it will send a new next request. The role played by the clients is also verification of responses. Model Server before it will send a new next request. The role played by the clients is also verification of responses.
* **Load balancer** works on the client platform in a docker container. HAProxy is used for this purpose. Its main role is * **Load balancer** works on the client platform in a docker container. HAProxy is used for this purpose. Its main role is
counting of requests forwarded from clients to OpenVINO™ Model Server, estimating its latency, and sharing this information by counting of requests forwarded from clients to OpenVINO™ Model Server, estimating its latency, and sharing this information by
Prometheus service. The reason of locating the load balancer on the client site is to simulate real life scenario that includes Prometheus service. The reason of locating the load balancer on the client site is to simulate real life scenario that includes
impact of physical network on reported metrics. impact of physical network on reported metrics.
* **Execution Controller** is launched on the client platform. It is responsible for synchronization of the whole measurement process, * **Execution Controller** is launched on the client platform. It is responsible for synchronization of the whole measurement process,
downloading metrics from the load balancer, and presenting the final report of the execution. downloading metrics from the load balancer, and presenting the final report of the execution.
@ -158,9 +182,9 @@ Test performance yourself
You can also test performance for your system yourself, following the guide on :doc:`getting performance numbers <openvino_docs_MO_DG_Getting_Performance_Numbers>`. You can also test performance for your system yourself, following the guide on :doc:`getting performance numbers <openvino_docs_MO_DG_Getting_Performance_Numbers>`.
Performance of a particular application can also be evaluated virtually using `Intel® DevCloud for the Edge <https://devcloud.intel.com/edge/>`__. Performance of a particular application can also be evaluated virtually using `Intel® DevCloud for the Edge <https://devcloud.intel.com/edge/>`__.
It is a remote development environment with access to Intel® hardware and the latest versions of the Intel® Distribution of the OpenVINO™ Toolkit. It is a remote development environment with access to Intel® hardware and the latest versions of the Intel® Distribution of the OpenVINO™ Toolkit.
To learn more about it, visit `the website <https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/overview.html>`__ To learn more about it, visit `the website <https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/overview.html>`__
or `create an account <https://www.intel.com/content/www/us/en/secure/forms/devcloud-enrollment/account-provisioning.html>`__. or `create an account <https://www.intel.com/content/www/us/en/secure/forms/devcloud-enrollment/account-provisioning.html>`__.
@ -168,11 +192,11 @@ or `create an account <https://www.intel.com/content/www/us/en/secure/forms/devc
Disclaimers Disclaimers
#################################### ####################################
* Intel® Distribution of OpenVINO™ toolkit performance results are based on release 2023.2, as of November 15, 2023. * Intel® Distribution of OpenVINO™ toolkit performance results are based on release 2023.3, as of February 13, 2024.
* OpenVINO Model Server performance results are based on release 2023.0, as of June 01, 2023. * OpenVINO Model Server performance results are based on release 2023.3, as of February 13, 2024.
The results may not reflect all publicly available updates. Intel technologies features and benefits depend on system configuration The results may not reflect all publicly available updates. Intel technologies features and benefits depend on system configuration
and may require enabled hardware, software, or service activation. Learn more at intel.com, or from the OEM or retailer. and may require enabled hardware, software, or service activation. Learn more at intel.com, or from the OEM or retailer.
See configuration disclosure for details. No product can be absolutely secure. See configuration disclosure for details. No product can be absolutely secure.

View File

@ -16,9 +16,7 @@ Test performance with the benchmark_app
You can run OpenVINO benchmarks in both C++ and Python APIs, yet the experience differs in each case. You can run OpenVINO benchmarks in both C++ and Python APIs, yet the experience differs in each case.
The Python one is part of OpenVINO Runtime installation, while C++ is available as a code sample. The Python one is part of OpenVINO Runtime installation, while C++ is available as a code sample.
For a detailed description, see: For a detailed description, see: :doc:`benchmark_app <openvino_sample_benchmark_tool>`.
* :doc:`benchmark_app for C++ <openvino_inference_engine_samples_benchmark_app_README>`
* :doc:`benchmark_app for Python <openvino_inference_engine_tools_benchmark_tool_README>`.
Make sure to install the latest release package with support for frameworks of the models you want to test. Make sure to install the latest release package with support for frameworks of the models you want to test.
For the most reliable performance benchmarks, :doc:`prepare the model for use with OpenVINO <openvino_docs_model_processing_introduction>`. For the most reliable performance benchmarks, :doc:`prepare the model for use with OpenVINO <openvino_docs_model_processing_introduction>`.
@ -87,7 +85,7 @@ slower than the subsequent ones, an aggregated value can be used for the executi
When comparing the OpenVINO Runtime performance with the framework or another reference code, make sure that both versions are as similar as possible: When comparing the OpenVINO Runtime performance with the framework or another reference code, make sure that both versions are as similar as possible:
- Wrap the exact inference execution (for examples, see :doc:`Benchmark app <openvino_inference_engine_samples_benchmark_app_README>`). - Wrap the exact inference execution (for examples, see :doc:`Benchmark app <openvino_sample_benchmark_tool>`).
- Do not include model loading time. - Do not include model loading time.
- Ensure that the inputs are identical for OpenVINO Runtime and the framework. For example, watch out for random values that can be used to populate the inputs. - Ensure that the inputs are identical for OpenVINO Runtime and the framework. For example, watch out for random values that can be used to populate the inputs.
- In situations when any user-side pre-processing should be tracked separately, consider :doc:`image pre-processing and conversion <openvino_docs_OV_UG_Preprocessing_Overview>`. - In situations when any user-side pre-processing should be tracked separately, consider :doc:`image pre-processing and conversion <openvino_docs_OV_UG_Preprocessing_Overview>`.
@ -98,7 +96,7 @@ Internal Inference Performance Counters and Execution Graphs
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
More detailed insights into inference performance breakdown can be achieved with device-specific performance counters and/or execution graphs. More detailed insights into inference performance breakdown can be achieved with device-specific performance counters and/or execution graphs.
Both :doc:`C++ <openvino_inference_engine_samples_benchmark_app_README>` and :doc:`Python <openvino_inference_engine_tools_benchmark_tool_README>` Both :doc:`C++ and Python <openvino_sample_benchmark_tool>`
versions of the *benchmark_app* support a ``-pc`` command-line parameter that outputs internal execution breakdown. versions of the *benchmark_app* support a ``-pc`` command-line parameter that outputs internal execution breakdown.
For example, the table shown below is part of performance counters for quantized For example, the table shown below is part of performance counters for quantized

View File

@ -31,10 +31,8 @@ Performance Information F.A.Q.
All of the performance benchmarks are generated using the All of the performance benchmarks are generated using the
open-source tool within the Intel® Distribution of OpenVINO™ toolkit open-source tool within the Intel® Distribution of OpenVINO™ toolkit
called ``benchmark_app``. This tool is available called :doc:`benchmark_app <openvino_sample_benchmark_tool>`.
:doc:`for C++ apps <openvino_inference_engine_samples_benchmark_app_README>`. This tool is available for Python and C++ apps.
as well as
:doc:`for Python apps <openvino_inference_engine_tools_benchmark_tool_README>`.
For a simple instruction on testing performance, see the :doc:`Getting Performance Numbers Guide <openvino_docs_MO_DG_Getting_Performance_Numbers>`. For a simple instruction on testing performance, see the :doc:`Getting Performance Numbers Guide <openvino_docs_MO_DG_Getting_Performance_Numbers>`.
@ -59,6 +57,10 @@ Performance Information F.A.Q.
- Meta AI - Meta AI
- Auto regressive language - Auto regressive language
- 4096 - 4096
* - `Mistral-7b <https://huggingface.co/mistralai/Mistral-7B-v0.1>`__
- Mistral AI
- Auto regressive language
- 4096
* - `Stable-Diffusion-V2-1 <https://huggingface.co/stabilityai/stable-diffusion-2-1>`__ * - `Stable-Diffusion-V2-1 <https://huggingface.co/stabilityai/stable-diffusion-2-1>`__
- Hugginface - Hugginface
- Latent Diffusion Model - Latent Diffusion Model
@ -71,18 +73,14 @@ Performance Information F.A.Q.
- BERT-large - BERT-large
- question / answer - question / answer
- 384 - 384
* - `deeplabv3 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/deeplabv3>`__
- DeepLab v3 Tf
- semantic segmentation
- 513x513
* - `efficientdet-d0 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/efficientdet-d0-tf>`__ * - `efficientdet-d0 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/efficientdet-d0-tf>`__
- Efficientdet - Efficientdet
- classification - classification
- 512x512 - 512x512
* - `faster_rcnn_resnet50_coco <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/faster_rcnn_resnet50_coco>`__ * - `mask_rcnn_resnet50_atrous_coco <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mask_rcnn_resnet50_atrous_coco>`__
- Faster RCNN TF - Mask R-CNN ResNet 50 Atrous
- object detection - object instance segmentation
- 600x1024 - 800x1365
* - `mobilenet-v2 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mobilenet-v2-pytorch>`__ * - `mobilenet-v2 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mobilenet-v2-pytorch>`__
- Mobilenet V2 PyTorch - Mobilenet V2 PyTorch
- classification - classification
@ -94,40 +92,36 @@ Performance Information F.A.Q.
* - `ssd-mobilenet-v1-coco <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssd_mobilenet_v1_coco>`__ * - `ssd-mobilenet-v1-coco <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssd_mobilenet_v1_coco>`__
- ssd-mobilenet-V1-coco onnx model - ssd-mobilenet-V1-coco onnx model
- object detection - object detection
- 300x300 - 300x300
* - `ssd-resnet34-1200-onnx <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssd-resnet34-1200-onnx>`__ * - `ssd-resnet34-1200-onnx <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssd-resnet34-1200-onnx>`__
- ssd-resnet34 onnx model - ssd-resnet34 onnx model
- object detection - object detection
- 1200x1200 - 1200x1200
* - `unet-camvid-onnx-0001 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/unet-camvid-onnx-0001>`__ * - `unet-camvid-onnx-0001 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/unet-camvid-onnx-0001>`__
- U-Net - U-Net
- semantic segmentation - semantic segmentation
- 368x480 - 368x480
* - `yolo-v3 <https://https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tf>`__
- YOLO v3
- object detection
- 416x416
* - `yolo-v3-tiny <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tiny-tf>`__ * - `yolo-v3-tiny <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tiny-tf>`__
- YOLO v3 Tiny - YOLO v3 Tiny
- object detection - object detection
- 416x416 - 416x416
* - `yolov8n <https://https://github.com/ultralytics/ultralytics>`__ * - `yolov8n <https://https://github.com/ultralytics/ultralytics>`__
- Yolov8nano - Yolov8nano
- object detection - object detection
- 608x608 - 608x608
.. dropdown:: Where can I purchase the specific hardware used in the benchmarking? .. dropdown:: Where can I purchase the specific hardware used in the benchmarking?
Intel partners with vendors all over the world. For a list of Hardware Manufacturers, see the Intel partners with vendors all over the world. For a list of Hardware Manufacturers, see the
`Intel® AI: In Production Partners & Solutions Catalog <https://www.intel.com/content/www/us/en/internet-of-things/ai-in-production/partners-solutions-catalog.html>`__. `Intel® AI: In Production Partners & Solutions Catalog <https://www.intel.com/content/www/us/en/internet-of-things/ai-in-production/partners-solutions-catalog.html>`__.
For more details, see the :doc:`Supported Devices <openvino_docs_OV_UG_supported_plugins_Supported_Devices>`. For more details, see the :doc:`Supported Devices <openvino_docs_OV_UG_supported_plugins_Supported_Devices>`.
documentation. Before purchasing any hardware, you can test and run documentation. Before purchasing any hardware, you can test and run
models remotely, using `Intel® DevCloud for the Edge <http://devcloud.intel.com/edge/>`__. models remotely, using `Intel® DevCloud for the Edge <http://devcloud.intel.com/edge/>`__.
.. dropdown:: How can I optimize my models for better performance or accuracy? .. dropdown:: How can I optimize my models for better performance or accuracy?
Set of guidelines and recommendations to optimize models are available in the Set of guidelines and recommendations to optimize models are available in the
:doc:`optimization guide <openvino_docs_deployment_optimization_guide_dldt_optimization_guide>`. :doc:`optimization guide <openvino_docs_deployment_optimization_guide_dldt_optimization_guide>`.
Join the conversation in the `Community Forum <https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit>`__ for further support. Join the conversation in the `Community Forum <https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit>`__ for further support.
@ -166,4 +160,3 @@ Performance Information F.A.Q.
for autonomous vehicles, where a quick response to the result of the for autonomous vehicles, where a quick response to the result of the
inference is required. inference is required.

View File

@ -5,15 +5,18 @@ Model Accuracy
The following two tables present the absolute accuracy drop calculated as the accuracy difference The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8 and FP16 between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
representations of a model on three platform architectures. Please also refer to notes below table FP16 representations of a model on three platform architectures. Please also refer to notes below
for more information. The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
the table for more information.
* A - Intel® Core™ i9-9000K (AVX2), INT8 and FP32 * A - Intel® Core™ i9-9000K (AVX2), INT8 and FP32
* B - Intel® Xeon® 6338, (VNNI), INT8 and FP32 * B - Intel® Xeon® 6338, (VNNI), INT8 and FP32
* C - Intel® Flex-170, INT8 and FP16 * C - Intel(R) Xeon 8490H (VNNI, AMX), INT8, BF16, FP32
* D - Intel® Flex-170, INT8 and FP16
.. list-table:: Model Accuracy for INT8 .. list-table:: Model Accuracy for INT8
:header-rows: 1 :header-rows: 1
@ -24,104 +27,114 @@ for more information.
- A, INT8 - A, INT8
- B, INT8 - B, INT8
- C, INT8 - C, INT8
- D, INT8
* - bert-base-cased * - bert-base-cased
- SST-2_bert_cased_padded - SST-2_bert_cased_padded
- accuracy - spearman@cosine
- -0.76% - 3.17%
- 2.42% - 2.68%
- 2.72% - 3.00%
- 2.73%
* - bert-large-uncased-whole-word-masking-squad-0001 * - bert-large-uncased-whole-word-masking-squad-0001
- SQUAD_v1_1_bert_msl384_mql64_ds128_lowercase - SQUAD_v1_1_bert_msl384_mql64_ds128_lowercase
- F1 - F1
- 0.07% - 0.07%
- -0.03% - -0.03%
- 0.13%
- 0.11% - 0.11%
* - deeplabv3
- VOC2012_segm
- mean_iou
- 0.49%
- 0.23%
- -0.16%
* - efficientdet-d0 * - efficientdet-d0
- COCO2017_detection_91cl - COCO2017_detection_91cl
- coco_precision - coco_precision
- -0.84% - -0.84%
- -0.59% - -0.59%
- -0.62%
- -0.63% - -0.63%
* - faster_rcnn_resnet50_coco * - mask_rcnn_resnet50_atrous_coco
- COCO2017_detection_91cl_bkgr - COCO2017_detection_91cl_bkgr
- coco_orig_precision - coco_orig_precision
- -0.19% - 0.03%
- -0.19% - 0.08%
- -0.04% - 0.11%
- 0.07%
* - mobilenet-v2 * - mobilenet-v2
- ImageNet2012 - ImageNet2012
- accuracy @ top1 - accuracy @ top1
- - %
- -0.97%
- -0.97% - -0.97%
- -0.95% - -0.95%
* - resnet-50 * - resnet-50
- ImageNet2012 - ImageNet2012
- accuracy @ top1 - accuracy @ top1
- -0.09% - -0.20%
- -0.12%
- -0.19% - -0.19%
* - ssd-mobilenet-v1-coco - -0.13%
- COCO2017_detection_80cl_bkgr - -0.15%
- coco-precision
- -2.97%
- -0.29%
- -0.26%
* - ssd-resnet34-1200 * - ssd-resnet34-1200
- COCO2017_detection_80cl_bkgr - COCO2017_detection_80cl_bkgr
- map - map
- -0.03% - -0.03%
- -0.06% - -0.06%
- -0.01%
- 0.04% - 0.04%
* - ssd-mobilenet-v1-coco
- COCO2017_detection_80cl_bkgr
- coco-precision
- -2.97%
- -0.29%
- -0.31%
- -0.26%
* - unet-camvid-onnx-0001 * - unet-camvid-onnx-0001
- CamVid_12cl - CamVid_12cl
- mean_iou @ mean - mean_iou @ mean
- -6.32% - -6.32%
- 6.40%
- 6.40% - 6.40%
* - yolo_v3 - 6.41%
- COCO2017_detection_80cl - 6.40%
- map
- -0.13%
- -0.26%
- -0.44%
* - yolo_v3_tiny * - yolo_v3_tiny
- COCO2017_detection_80cl - COCO2017_detection_80cl
- map - map
- -0.11% - %
- -0.13% - -0.23%
- -0.15% - -0.24%
- -0.66%
* - yolo_v8n * - yolo_v8n
- COCO2017_detection_80cl - COCO2017_detection_80cl
- map - map
- 0.27% - -0.02%
- 0.23% - -0.03%
- 0.17% - -0.06%
- -0.06%
* - chatGLM2-6b * - chatGLM2-6b
- lambada openai - lambada openai
- ppl - ppl
- -
- 17.595 - 17.38
- - 17.41
- 17.17
* - Llama-2-7b-chat * - Llama-2-7b-chat
- Wiki, StackExch, Crawl - Wiki, StackExch, Crawl
- ppl - ppl
- -
- 3.268 - 3.24
- - 3.24
- 3.25
* - Stable-Diffusion-V2-1 * - Stable-Diffusion-V2-1
- LIAON-5B - LIAON-5B
- CLIP
-
-
-
-
* - Mistral-7b
- proprietary Mistral.ai
- ppl - ppl
- -
- - 3.29
- - 3.47
- 3.49
.. list-table:: Model Accuracy for FP32 and FP16 (FP16: Flex-170 only) .. list-table:: Model Accuracy for BF16, FP32 and FP16 (FP16: Flex-170 only. BF16: Xeon(R) 8490H only)
:header-rows: 1 :header-rows: 1
* - OpenVINO™ Model name * - OpenVINO™ Model name
@ -129,53 +142,55 @@ for more information.
- Metric Name - Metric Name
- A, FP32 - A, FP32
- B, FP32 - B, FP32
- C, FP16 - C, FP32
- C, BF16
- D, FP16
* - bert-base-cased * - bert-base-cased
- SST-2_bert_cased_padded - SST-2_bert_cased_padded
- accuracy - spearman@cosine
- 0.00% - 0.00%
- 0.00% - 0.00%
- 0.00% - 0.00%
- -0.09%
- 0.00%
* - bert-large-uncased-whole-word-masking-squad-0001 * - bert-large-uncased-whole-word-masking-squad-0001
- SQUAD_v1_1_bert_msl384_mql64_ds128_lowercase - SQUAD_v1_1_bert_msl384_mql64_ds128_lowercase
- F1 - F1
- 0.04% - 0.04%
- 0.04% - 0.04%
- 0.04% - 0.04%
* - deeplabv3 - 0.06%
- VOC2012_segm - 0.04%
- mean_iou
- 0.00%
- 0.00%
- 0.00%
* - efficientdet-d0 * - efficientdet-d0
- COCO2017_detection_91cl - COCO2017_detection_91cl
- coco_precision - coco_precision
- -0.02% - -0.02%
- -0.02% - -0.02%
- -0.02% - -0.02%
* - faster_rcnn_resnet50_coco - -0.02%
- -0.03%
* - mask_rcnn_resnet50_atrous_coco
- COCO2017_detection_91cl_bkgr - COCO2017_detection_91cl_bkgr
- coco_orig_precision - coco_orig_precision
- 0.00% - -0.01%
- - -0.01%
- 0.00% - %
- -0.18%
- 0.02%
* - mobilenet-v2 * - mobilenet-v2
- ImageNet2012 - ImageNet2012
- accuracy @ top1 - accuracy @ top1
- 0.00% - 0.00%
- 0.00% - 0.00%
- 0.00% - 0.00%
- -0.04%
- 0.02%
* - resnet-50 * - resnet-50
- ImageNet2012 - ImageNet2012
- accuracy @ top1 - accuracy @ top1
- 0.02%
- 0.02%
- 0.00% - 0.00%
- 0.00%
- 0.00%
* - ssd-mobilenet-v1-coco
- COCO2017_detection_80cl_bkgr
- coco-precision
- 0.01%
- 0.01% - 0.01%
- 0.01% - 0.01%
* - ssd-resnet34-1200 * - ssd-resnet34-1200
@ -184,49 +199,74 @@ for more information.
- 0.00% - 0.00%
- 0.00% - 0.00%
- 0.00% - 0.00%
- -0.02%
- 0.02%
* - ssd-mobilenet-v1-coco
- COCO2017_detection_80cl_bkgr
- coco-precision
- 0.01%
- 0.01%
- 0.01%
- 0.05%
- -0.03%
* - unet-camvid-onnx-0001 * - unet-camvid-onnx-0001
- CamVid_12cl - CamVid_12cl
- mean_iou @ mean - mean_iou @ mean
- 0.00% - 0.00%
- 0.00%
- 0.00%
* - yolo_v3
- COCO2017_detection_80cl
- map
- 0.00%
- 0.00% - 0.00%
- 0.00% - 0.00%
- -0.03%
- -0.03%
* - yolo_v3_tiny * - yolo_v3_tiny
- COCO2017_detection_80cl - COCO2017_detection_80cl
- map - map
- -0.04% - %
- -0.04% - 0.00%
- 0.02% - 0.00%
- 0.00%
- -0.02%
* - yolo_v8n * - yolo_v8n
- COCO2017_detection_80cl - COCO2017_detection_80cl
- map - map
- 0.00% - 0.00%
- 0.00% - 0.00%
- 0.00% - 0.00%
- 0.05%
- -0.03%
* - chatGLM2-6b * - chatGLM2-6b
- lambada-openai - lambada openai
- ppl - ppl
- -
- 17.488 - 17.48
- - 17.56
-
- 17.49
* - Llama-2-7b-chat * - Llama-2-7b-chat
- Wiki, StackExch, Crawl - Wiki, StackExch, Crawl
- ppl - ppl
- -
- 3.262 - 3.26
- - 3.26
-
-
* - Stable-Diffusion-V2-1 * - Stable-Diffusion-V2-1
- LIAON-5B - LIAON-5B
- CLIP
-
-
-
-
- 22.48
* - Mistral-7b
- proprietary Mistral.ai
- ppl - ppl
- -
- - 3.19
- 3.18
-
- -
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop. Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity the values do not indicate a deviation from a reference but are the actual measured accuracy for the model. For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
accuracy for the model.

View File

@ -0,0 +1,175 @@
.. {#release_policy}
Release Policy
=============================================================================
OpenVINO offers releases of three different types, each targeting a different use case:
* `Regular releases <#regular-releases>`__
* `Long-Term Support <#long-term-support-releases>`__
* `Nightly <#nightly-releases>`__
Regular releases
####################
OpenVINO™ is published multiple times a year, when significant new features and bug fixes have
been completed and validated. For each regular release, a dedicated development branch in GitHub
is created, targeting changes such as:
* New features of gold quality, as well as Beta features, labeled as “preview.”
* Key bug fixes.
* Newest hardware support.
Each regular release is supported until the next version arrives, making it suitable for:
* Most typical use cases (the recommended release type).
* Products requiring frequent changes in supported hardware, libraries, operating systems, and models.
Long-Term Support releases
###########################
Each year's final release becomes a Long-Term Support (LTS) version, which continues to receive
bug fixes and security updates, even after newer versions are published. Therefore, LTS may be
used for production environments where:
* There is no need for frequent changes in hardware or model support.
* New optimizations are not prioritized.
* Upgrading is challenging, e.g., due to high software complexity.
* A legacy feature, discontinued in newer OpenVINO versions, is still required.
**LTS Lifecycle**
* LTS is typically published at the end of every year cycle.
* LTS uses the branch of the last yearly regular release.
* LTS aim to receive an update once a year.
* Security updates are offered for the duration of the entire LTS period, which is two years
(or until superseded by two consecutive LTS versions).
* Updates targeting newly discovered bugs are offered for the period of one year.
.. note::
LTS releases may offer limited distribution options.
**Components covered by LTS**
Not all components associated with the OpenVINO™ toolkit are covered by the LTS policy.
The following elements are not guaranteed to receive updates:
* Components in the deprecation period.
* Preview features (highlighted in the release notes).
* Components not directly connected to the OpenVINO™ workflow, such as: Samples, demos, and Jupyter notebooks.
* OpenVINO tools, such as NNCF and OVMS.
* Code samples used in component testing.
Nightly releases
###########################
OpenVINO nightly releases are the first source of newly added features and priority bug fixes
reported for the previous versions, as a preview of the most recent changes. They are:
* Released every workday.
* Based on the master branch of the OpenVINO GitHub repository.
* Not fit for production environments.
* Offered with limited distribution options:
Since their validation scope is limited, **they should never be used for production purposes**.
Instead, they may serve:
* Early integration testing.
* Community contribution development and integration.
* Tracking development progress.
.. tab-set::
.. tab-item:: Downloadable Archives
:sync: archives-s3
1. Go to `OpenVINO Nightly Packages <https://storage.openvinotoolkit.org/repositories/openvino/packages/nightly/>`__.
2. Select a package you want to install.
3. Download the archive for your platform.
4. Unpack the archive in a convenient location.
5. Once unpacked, proceed as with a regular OpenVINO archive (see :doc:`installation guides <openvino_docs_install_guides_overview>`).
.. tab-item:: OV Wheels on S3
:sync: wheels-s3
A PyPI repository deployed on AWS S3 (`see more details <https://peps.python.org/pep-0503/>`__)
enables the use of regular PyPI without the need to rename wheels. Installation commands vary depending
on the branch:
.. tab-set::
.. tab-item:: Master
:sync: master
.. code-block:: py
pip install --pre openvino --extra-index-url
https://storage.openvinotoolkit.org/simple/wheels/nightly
.. tab-item:: Release
:sync: release
* This command includes **Release Candidates**.
* To use ``extra-index-url``, you need to pass a link containing ``simple``.
* The ``--pre`` allows the installation of dev-builds.
.. code-block:: py
pip install --pre openvino --extra-index-url
https://storage.openvinotoolkit.org/simple/wheels/pre-release
.. tab-item:: OV Wheels on PyPi (not recommended)
:sync: wheels-pypi
Install OV Wheels from PyPI:
.. code-block:: py
pip install openvino-nightly
Additional Information
#########################
| **Determining the OpenVINO Version**
| If you need to operate on a specific OpenVINO release, and you are not sure which version
is included in the installed package, you can verify it in one of two ways:
.. tab-set::
.. tab-item:: Python
:sync: python
Execute the following command within the installed package:
.. code-block:: python
python3 -c "import openvino; print(openvino.__version__)"
.. tab-item:: Archives
:sync: archives
You can find the file version in:
.. code-block:: text
<UNZIPPED_ARCHIVE_ROOT>/runtime/version.txt
| **Issue Reporting**
| To report issues, use the `Intel® Premier Support <https://www.intel.com/content/www/us/en/design/support/ips/training/welcome.html>`__
clearly stating the issue, impact, and the expected timeline.
| **Distribution:**
* `Selector tool <https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/download.html>`__ of all distribution options.
* Source code distribution: `GitHub <https://github.com/openvinotoolkit/openvino>`__ and
`Gitee <https://gitee.com/openvinotoolkit-prc/openvino>`__ .
* Binary distribution:
* Download from `OpenVINO storage <https://storage.openvinotoolkit.org/repositories/openvino/packages/>`__
* `pypi.org <https://pypi.org/project/openvino-dev/>`__
* `DockerHub* <https://hub.docker.com/u/openvino>`__

View File

@ -1,314 +1,345 @@
.. {#openvino_release_notes} .. {#openvino_release_notes}
OpenVINO Release Notes OpenVINO Release Notes
====================== ========================================
The Intel® Distribution of OpenVINO™ toolkit is an open-source solution for optimizing 2023.3 (LTS) - 24.01.2024
and deploying AI inference in domains such as computer vision,automatic speech
recognition, natural language processing, recommendation systems, and generative AI.
With its plug-in architecture, OpenVINO enables developers to write once and deploy
anywhere. We are proud to announce the release of OpenVINO 2023.2 introducing a range
of new features, improvements, and deprecations aimed at enhancing the developer
experience.
New and changed in 2023.2
########################### ###########################
Summary of major features and improvements Summary of major features and improvements
++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++
* More Generative AI coverage and framework integrations to minimize code changes. * More Generative AI coverage and framework integrations to minimize code changes.
* **Expanded model support for direct PyTorch model conversion** - automatically convert * Introducing `OpenVINO Gen AI repository <https://github.com/openvinotoolkit/openvino.genai>`__
additional models directly from PyTorch or execute via ``torch.compile`` with OpenVINO on GitHub that demonstrates native C and C++ pipeline samples for Large Language Models
as the backend. (LLMs). String tensors are now supported as inputs and tokenizers natively to reduce
* **New and noteworthy models supported** - we have enabled models used for chatbots, overhead and ease production.
instruction following, code generation, and many more, including prominent models * New and noteworthy models validated; Mistral, Zephyr, Qwen, ChatGLM3, and Baichuan
like Llava, chatGLM, Bark (text to audio) and LCM (Latent Consistency Models, an * New Jupyter Notebooks for
optimized version of Stable Diffusion). `Latent Consistency Models (LCM) <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/263-latent-consistency-models-image-generation>`__
* **Easier optimization and conversion of Hugging Face models** - compress LLM models and `Distil-Whisper <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__.
to Int8 with the Hugging Face Optimum command line interface and export models to Updated `LLM Chatbot notebook <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot>`__
the OpenVINO IR format. to include LangChain, Neural Chat, TinyLlama, ChatGLM3, Qwen, Notus and Youri models.
* **OpenVINO is now available on Conan** - a package manager which allows more seamless * Torch.compile is now fully integrated with OpenVINO, which now includes a hardware
package management for large scale projects for C and C++ developers. 'options' parameter allowing for seamless inference hardware selection by leveraging
the plugin architecture in OpenVINO.
* Broader Large Language Model (LLM) support and more model compression techniques. * Broader Large Language Model (LLM) support and more model compression techniques.
* Accelerate inference for LLM models on Intel® CoreTM CPU and iGPU with the * As part of the Neural Network Compression Framework (NNCF), INT4 weight compression model
use of Int8 model weight compression. formats are now fully supported on Intel® Xeon® CPUs in addition to Intel® Core™ and iGPU,
* Expanded model support for dynamic shapes for improved performance on GPU. adding more performance, lower memory usage, and accuracy opportunity when using LLMs.
* Preview support for Int4 model format is now included. Int4 optimized model * Improved performance of transformer based LLM on CPU and GPU using stateful model technique
weights are now available to try on Intel® Core™ CPU and iGPU, to accelerate to increase memory efficiency where internal states are shared among multiple iterations of
models like Llama 2 and chatGLM2. inference.
* The following Int4 model compression formats are supported for inference * Easier optimization and conversion of Hugging Face models - compress LLM models to INT8
in runtime: and INT4 with Hugging Face Optimum command line interface and export models to OpenVINO
format. Note this is part of `Optimum-Intel <https://huggingface.co/docs/optimum/intel/index>`__
* Generative Pre-training Transformer Quantization (GPTQ); with GPTQ-compressed which needs to be installed separately.
models, you can access them through the Hugging Face repositories. * Tokenizer and TorchVision transform support is now available in the OpenVINO runtime
* Native Int4 compression through Neural Network Compression Framework (NNCF). (via new API) requiring less preprocessing code and enhancing performance by automatically
handling this model setup. More details on Tokenizers support in Ecosystem section.
* More portability and performance to run AI at the edge, in the cloud, or locally. * More portability and performance to run AI at the edge, in the cloud, or locally.
* **In 2023.1 we announced full support for ARM** architecture, now we have improved * Full support for 5th Gen Intel® Xeon® Scalable processors (codename Emerald Rapids).
performance by enabling FP16 model formats for LLMs and integrating additional * Further optimized performance on Intel® Core™ Ultra (codename Meteor Lake) CPU with
acceleration libraries to improve latency. latency hint, by leveraging both P-core and E-cores.
* Improved performance on ARM platforms using throughput hint, which increases efficiency
in utilization of CPU cores and memory bandwidth.
* Preview JavaScript API to enable node JS development to access JavaScript binding via
source code. See details below.
* Improved `model serving of LLMs <https://github.com/openvinotoolkit/model_server/tree/main/demos/python_demos/llm_text_generation>`__
through OpenVINO Model Server. This not only enables LLM serving over KServe v2 gRPC
and REST APIs for more flexibility, but also improves throughput by running processing
like tokenization on the server side. More details in the Ecosystem section.
Support Change and Deprecation Notices Support Change and Deprecation Notices
++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++
* The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated * The OpenVINO™ Development Tools package (pip install openvino-dev) is deprecated and will be
and will be removed from installation options and distribution channels with removed from installation options and distribution channels beginning with 2025.0.
2025.0. To learn more, refer to the For more details, refer to the :doc:`OpenVINO Legacy Features and Components <openvino_legacy_features>`
:doc:`OpenVINO Legacy Features and Components page <openvino_legacy_features>`. page.
To ensure optimal performance, install the OpenVINO package (pip install openvino), * Ubuntu 18.04 support will be discontinued in the 2023.3 LTS release. The recommended version
which includes essential components such as OpenVINO Runtime, OpenVINO Converter, of Ubuntu is 22.04.
and Benchmark Tool. * Starting in release 2023.3 OpenVINO will no longer support Python 3.7 due to the Python
community discontinuing support. Update to a newer version (currently 3.8-3.11) to avoid
interruptions.
* All ONNX Frontend legacy API (known as ONNX_IMPORTER_API) will no longer be available in 2024.0 release.
* ``PerfomanceMode.UNDEFINED`` property as part of the OpenVINO Python API will be
discontinued in the 2024.0 release.
* Tools: * Tools:
* :doc:`Deployment Manager <openvino_docs_install_guides_deployment_manager_tool>` * :doc:`Deployment Manager <openvino_docs_install_guides_deployment_manager_tool>`
is deprecated and will be removed in the 2024.0 release. is deprecated and will be supported for two years according to our LTS policy.
Visit our :doc:`selector tool <openvino_docs_install_guides_overview>` to see
package distribution options or our :doc:`deployment guide <openvino_deployment_guide>`
documentation.
* Accuracy Checker is deprecated and will be discontinued with 2024.0. * Accuracy Checker is deprecated and will be discontinued with 2024.0.
* Post-Training Optimization Tool (POT) is deprecated and will be * Post-Training Optimization Tool (POT) has been deprecated and the 2023.3 LTS will be
discontinued with 2024.0. the last release that will support the tool. Developers are encouraged to use the Neural
* Model Optimizer is deprecated and will be fully supported up until the 2025.0 Network Compression Framework (NNCF) for this feature.
release. Model conversion to the OpenVINO format should be performed through * Model Optimizer is deprecated and will be fully supported until the 2025.0 release.
OpenVINO Model Converter, which is part of the PyPI package. Follow the We encourage developers to perform model conversion through OpenVINO Model Converter
:doc:`Model Optimizer to OpenVINO Model Converter transition <openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition>` (API call: OVC). Follow the
guide for smoother transition. Known limitations are TensorFlow model with :doc:`model conversion transition guide <openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition>`
TF1 Control flow and object detection models. These limitations relate to for more details.
the gap in TensorFlow direct conversion capabilities which will be addressed * Deprecated support for a `git patch <https://github.com/openvinotoolkit/nncf/tree/develop/third_party_integration/huggingface_transformers>`__
in upcoming releases. for NNCF integration with `huggingface/transformers <https://github.com/huggingface/transformers>`__.
* PyTorch 1.13 support is deprecated in Neural Network Compression Framework (NNCF) The recommended approach is to use `huggingface/optimum-intel <https://github.com/huggingface/optimum-intel>`__
for applying NNCF optimization on top of models from Hugging Face.
* Support for Apache MXNet, Caffe, and Kaldi model formats is deprecated and will be
discontinued with the 2024.0 release.
* Runtime: * Runtime:
* Intel® Gaussian & Neural Accelerator (Intel®GNA) will be deprecated in a future * Intel® Gaussian & Neural Accelerator (Intel® GNA) will be deprecated in a future release.
release. We encourage developers to use the Neural Processing Unit (NPU) for We encourage developers to use the Neural Processing Unit (NPU) for low-powered systems
low powered systems like Intel® Core™ Ultra or 14th generation and beyond. like Intel® CoreTM Ultra or 14th generation and beyond.
* OpenVINO C++/C/Python 1.0 APIs will be discontinued with 2024.0. * OpenVINO C++/C/Python 1.0 APIs are deprecated and will be discontinued in the 2024.0 release.
* Python 3.7 support has been discontinued. Please use API 2.0 in your applications going forward to avoid disruption.
* OpenVINO property Affinity API will be deprecated from 2024.0 and will be discontinued in 2025.0.
It will be replaced with CPU binding configurations (``ov::hint::enable_cpu_pinning``).
OpenVINO™ Development Tools OpenVINO™ Development Tools
++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++
List of components and their changes:
------------------------------------------
* :doc:`OpenVINO Model Converter tool <openvino_docs_model_processing_introduction>`
now supports the original framework shape format.
* `Neural Network Compression Framework (NNCF) <https://github.com/openvinotoolkit/nncf>`__ * `Neural Network Compression Framework (NNCF) <https://github.com/openvinotoolkit/nncf>`__
* Added data-free Int4 weight compression support for LLMs in OpenVINO IR with
``nncf.compress_weights()``.
* Improved quantization time of LLMs with NNCF PTQ API for ``nncf.quantize()``
and ``nncf.quantize_with_accuracy_control()``.
* Added support for SmoothQuant and ChannelAlighnment algorithms in NNCF HyperParameter
Tuner for automatic optimization of their hyperparameters during quantization.
* Added quantization support for the ``IF`` operation of models in OpenVINO format
to speed up such models.
* NNCF Post-training Quantization for PyTorch backend is now supported with
``nncf.quantize()`` and the common implementation of quantization algorithms.
* Added support for PyTorch 2.1. PyTorch 1.13 support has been deprecated.
OpenVINO™ Runtime (previously known as Inference Engine) * Weight compression API, ``nncf.compress_weights()``, has been extended by:
---------------------------------------------------------
* OpenVINO Common * When using the 'all_layers' parameter, it compresses the model, including embeddings
and final layers, to the 4-bit format. This helps make the model footprint smaller
and improves performance, but it might impact the model accuracy. By default, this
parameter is disabled, and the backup precision (INT8) is assigned for the embeddings
and last layers.
* When using INT8_SYM compression mode for better performance of the compressed model
in case of 8-bit weight compression you might experience an impact on model accuracy
therefore by default, we use INT8_ASYM mode to better balance performance and accuracy.
* We implemented a 4-bit data-aware weight compression feature, introducing the 'dataset'
optional parameter in ``nncf.compress_weights()``. This parameter can be utilized to
mitigate accuracy loss in compressed models. It's important to note that enabling
this option will extend the compression time.
* Post-training Quantization with Accuracy Control, ``nncf.quantize_with_accuracy_control()``,
has been extended by the 'restore_mode' optional parameter to revert weights to INT8
instead of the original precision. This parameter helps to reduce the size of the
quantized model and improves its performance. By default, it is disabled and model
weights are reverted to the original precision in ``nncf.quantize_with_accuracy_control()``.
* Operations for reference implementations updated from legacy API to API 2.0. OpenVINO™ Runtime
* Symbolic transformation introduced the ability to remove Reshape operations ++++++++++++++++++++++++
surrounding MatMul operations.
* OpenVINO Python API
* Better support for the ``openvino.properties`` submodule, which now allows the use
of properties directly, without additional parenthesis. Example use-case:
``{openvino.properties.cache_dir: “./some_path/”}``.
* Added missing properties: ``execution_devices`` and ``loaded_from_cache``.
* Improved error propagation on imports from OpenVINO package.
* AUTO device plug-in (AUTO)
* o Provided additional option to improve performance of cumulative throughput
(or MULTI), where part of CPU resources can be reserved for GPU inference
when GPU and CPU are both used for inference (using ``ov::hint::enable_cpu_pinning(true)``).
This avoids the performance issue of CPU resource contention where there
is not enough CPU resources to schedule tasks for GPU
(`PR #19214 <https://github.com/openvinotoolkit/openvino/pull/19214>`__).
* CPU
* Introduced support of GPTQ quantized Int4 models, with improved performance
compared to Int8 weight-compressed or FP16 models. In the CPU plugin,
the gain in performance is achieved by FullyConnected acceleration with
4bit weight decompression
(`PR #20607 <https://github.com/openvinotoolkit/openvino/pull/20607>`__).
* Improved performance of Int8 weight-compressed large language models on
some platforms, such as 13th Gen Intel Core
(`PR #20607 <https://github.com/openvinotoolkit/openvino/pull/20607>`__).
* Further reduced memory consumption of select large language models on
CPU platforms with AMX and AVX512 ISA, by eliminating extra memory copy
with a unified weight layout
(`PR #19575 <https://github.com/openvinotoolkit/openvino/pull/19575>`__).
* Fixed performance issue observed in 2023.1 release on select Xeon CPU
platform with improved thread workload partitioning matching L2 cache
utilization
(`PR #20436 <https://github.com/openvinotoolkit/openvino/pull/20436>`__).
* Extended support of configuration (enable_cpu_pinning) on Windows
platforms to allow fine-grain control on CPU resource used for inference
workload, by binding inference thread to CPU cores
(`PR #19418 <https://github.com/openvinotoolkit/openvino/pull/19418>`__).
* Optimized YoloV8n and YoloV8s model performance for BF16/FP32 precision.
* Optimized Falcon model on 4th Gen Intel® Xeon® Scalable Processors.
* Enabled support for FP16 inference precision on ARM.
* GPU
* Enhanced inference performance for Large Language Models.
* Introduced int8 weight compression to boost LLM performance.
(`PR #19548 <https://github.com/openvinotoolkit/openvino/pull/19548>`__).
* Implemented Int4 GPTQ weight compression for improved LLM performance.
* Optimized constant weights for LLMs, resulting in better memory usage
and faster model loading.
* Optimized gemm (general matrix multiply) and fc (fully connected) for
enhanced performance on iGPU.
(`PR #19780 <https://github.com/openvinotoolkit/openvino/pull/19780>`__).
* Completed GPU plugin migration to API 2.0.
* Added support for oneDNN 3.3 version.
* Model Import Updates * Model Import Updates
* TensorFlow Framework Support * TensorFlow Framework Support
* Supported conversion of models from memory in keras.Model and tf.function formats. * Supported TF1 While Control flow construction w/o TensorArray operations
`PR #19903 <https://github.com/openvinotoolkit/openvino/pull/19903>`__ (`PR #20800 <https://github.com/openvinotoolkit/openvino/pull/20800>`__).
* Supported TF 2.14. * Support for complex tensors has been added
`PR #20385 <https://github.com/openvinotoolkit/openvino/pull/20385>`__ (`PR #20860 <https://github.com/openvinotoolkit/openvino/pull/20860>`__),
(`PR #21477 <https://github.com/openvinotoolkit/openvino/pull/21477>`__).
* Provided fixes for the following:
* PyTorch Framework Support * Accept any model file extension for frozen protobuf format
(`PR #21508 <https://github.com/openvinotoolkit/openvino/pull/21508>`__).
* Correct ArgMin/ArgMax translators for repeating elements case
(`PR #21364 <https://github.com/openvinotoolkit/openvino/pull/21364>`__).
* Correct PartitionedCall translator when numbers of external and internal
body inputs mismatch
(`PR #20825 <https://github.com/openvinotoolkit/openvino/pull/20825>`__).
* Supported Int4 GPTQ models. * PyTorch Framework Support
* New operations supported.
* ONNX Framework Support * Added support of nested dictionaries and lists as example input.
* Disabled ``torch.jit.freeze`` in default model tracing scenario and
improved support for models without freezing, extending model
coverage and improving accuracy for some models.
* Added support for ONNX version 1.14.1 * ONNX Framework Support
(`PR #18359 <https://github.com/openvinotoolkit/openvino/pull/18359>`__)
* Switched to ONNX 1.15.0 as a supported version of original framework
(`PR #20929 <https://github.com/openvinotoolkit/openvino/pull/20929>`__).
* CPU
* Full support for 5th Gen Intel® Xeon® Scalable processors (codename Emerald Rapids)
with sub-numa (SNC) and efficient core resource scheduling to improve performance.
* Further optimized performance on Intel® Core™ Ultra (codename Meteor Lake) CPU with
latency hint, by leveraging both P-core and E-cores.
* Further improved performance of LLMs in INT4 weight compression, especially on 1st
token latency and on 4th and 5th Gen of Intel Xeon platforms (codename Sapphire
Rapids and Emerald Rapids) with AMX capabilities.
* Improved performance of transformer-based LLM using stateful model technique to
increase memory efficiency where internal states (KV cache) are shared among multiple
iterations of inference. The stateful model implementation supports both greedy search
and beam search (preview) for LLMs. This technique also reduces the memory footprint
of LLMs, where Intel Core and Ultra platforms like Raptor Lake and Meteor Lake can
run INT4 models, such as Llama v2 7B.
* Improved performance on ARM platforms with throughput hint, by increasing
efficiency in usage of the CPU cores and memory bandwidth.
* GPU
* Full support for Intel® Core™ Ultra (codename Meteor Lake) integrated graphics.
* For LLMs, the first inference latency for INT8 and INT4 weight-compressed models has
been improved on iGPU thanks to more efficient context processing. Overall average
token latency for INT8 and INT4 has also been enhanced on iGPU with graph compilation
optimization, various host overhead optimization, and dynamic padding support for GEMM.
* Stateful model is functionally supported for LLMs.
* Model caching for dynamically shaped models is now supported. Model loading time is
improved for these models, including LLMs.
* API for switching between size mode (model caching) and speed mode (kernel caching)
is introduced.
* The model cache file name is changed to be independent of GPU driver versions.
The GPU will not generate separate model cache files when the driver is updated.
* Compilation time for Stable Diffusion models has been improved.
* NPU
* NPU plugin is available as part of OpenVINO. With the Intel(R) Core Ultra NPU driver
installed, inference can run on the NPU device.
* AUTO device plug-in (AUTO)
* Introduced the round-robin policy to AUTO cumulative throughput hint, which dispatches
inference requests to multiple devices (such as multiple GPU devices) in the round-robin
sequence, instead of in the device priority sequence. The device priority sequence
remains as the default configuration.
* AUTO loads stateful models to GPU or CPU per device priority, since GPU now supports
stateful model inference.
* OpenVINO Common
* Enhanced support of String tensors has been implemented, enabling the use of operators
and models that rely on string tensors. This update also enhances the capability in
the torchvision preprocessing (`PR #21244 <https://github.com/openvinotoolkit/openvino/pull/20929>`__).
* A new feature has been added that enables the selection of P-Cores for model compilation
on CPU device(s) with hybrid architecture (i.e. Intel® Core™ 12th Gen and beyond).
This will reduce compilation time compared to previous implementation where P-cores
and E-cores are used randomly by OS scheduling.
* OpenVINO JavaScript API (preview feature)
* We've introduced a preview version of
`JS API <https://github.com/openvinotoolkit/openvino/tree/master/src/bindings/js>`__
for OpenVINO runtime in this release. We hope that you will try this feature and
provide your feedback through GitHub `issues <https://github.com/openvinotoolkit/openvino/issues>`__.
* Known limitations:
* Only supported in manylinux and x86 (Windows, ARM, ARM64, and macOS have not been tested)
* Node.js version >= 18.16
* CMake version < 3.14 is not supported
* gcc compiler version < 7 is not supported
* OpenVINO Python API
* Introducing string tensor support for Python API.
* Added support for the following:
* Create ov.Tensor from Python lists
* Create ov.Tensor from empty numpy arrays.
* Constants from empty numpy arrays.
* Autogenerated get/set methods for Node attributes.
* Inference functions (``InferRequest.infer/start_async``, ``CompiledModel.__call__`` etc.) support OVDict as the input.
* PILLOW interpolation modes bindings. (`PR #21188 <https://github.com/openvinotoolkit/openvino/pull/21188>`__ external contribution: @meetpatel0963)
* Torchvision to :doc:`OpenVINO preprocessing <openvino_docs_OV_UG_string_tensors>`
converter documentation has been added to OpenVINO docs.
OpenVINO Ecosystem OpenVINO Ecosystem
+++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++
OpenVINO Model Server * OpenVINO Tokenizer (Preview feature)
--------------------------
Introduced an extension of the KServe gRPC API, enabling streaming input and * OpenVINO Tokenizer adds text processing operations to OpenVINO:
output for servables with Mediapipe graphs. This extension ensures the persistence
of Mediapipe graphs within a user session, improving processing performance.
This enhancement supports stateful graphs, such as tracking algorithms, and
enables the use of source calculators.
(`see additional documentation <https://github.com/openvinotoolkit/model_server/blob/main/docs/streaming_endpoints.md>`__)
* Mediapipe framework has been updated to the version 0.10.3. * Text PrePostprocessing without third-party dependencies
* model_api used in the openvino inference Mediapipe calculator has been updated * Convert a HuggingFace tokenizer into the OpenVINO model tokenizer and the
and included with all its features. detokenizer using a CLI tool or Python API
* Added a demo showcasing gRPC streaming with Mediapipe graph. * Connect a tokenizer and a model to get a single model with text input
(`see here <https://github.com/openvinotoolkit/model_server/tree/main/demos/mediapipe/holistic_tracking>`__)
* Added parameters for gRPC quota configuration and changed default gRPC channel
arguments to add rate limits. It will minimize the risks of impact of the service
from uncontrolled flow of requests.
* Updated python clients requirements to match wide range of python versions from 3.6 to 3.11
Learn more about the changes in https://github.com/openvinotoolkit/model_server/releases * OpenVINO Tokenizer models work only on the CPU device
* Supported platforms: Linux (x86 and ARM), Windows and Mac (x86 and ARM)
Jupyter Notebook Tutorials
-----------------------------
* The following notebooks have been updated or newly added:
* `LaBSE <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/220-cross-lingual-books-alignment>`__
Cross-lingual Books Alignment With Transformers
* `LLM chatbot <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot>`__
Create LLM-powered Chatbot
* Updated to include Int4 weight compression and Zephyr 7B model
* `Bark Text-to-Speech <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/256-bark-text-to-audio>`__
Text-to-Speech generation using Bark
* `LLaVA Multimodal Chatbot <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/257-llava-multimodal-chatbot>`__
Visual-language assistant with LLaVA
* `BLIP-Diffusion - Subject-Driven Generation <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/258-blip-diffusion-subject-generation>`__
Subject-driven image generation and editing using BLIP Diffusion
* `DeciDiffusion <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/259-decidiffusion-image-generation>`__
Image generation with DeciDiffusion
* `Fast Segment Anything <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/261-fast-segment-anything>`__
Object segmentations with FastSAM
* `SoftVC VITS Singing Voice Conversion <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/262-softvc-voice-conversion>`__
* `QR Code Monster <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/264-qrcode-monster>`__
Generate creative QR codes with ControlNet QR Code Monster
* `Würstchen <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/265-wuerstchen-image-generation>`__
Text-to-image generation with Würstchen
* `Distil-Whisper <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/267-distil-whisper-asr>`__
Automatic speech recognition using Distil-Whisper and OpenVINO™
* Added optimization support (8-bit quantization, weight compression) * OpenVINO Model Server
by NNCF for the following notebooks:
* Added support for serving pipelines with custom nodes implemented as a
`python code <https://github.com/openvinotoolkit/model_server/blob/main/docs/python_support/quickstart.md>`__
This greatly simplifies exposing GenAI algorithms based on Hugging Face
and Optimum libraries. It can be also applied for arbitrary pre and
post-processing in model serving pipelines.
* Included a new set of model serving demos that use custom nodes with python
code. These include LLM `text generation <https://github.com/openvinotoolkit/model_server/tree/main/demos/python_demos/llm_text_generation>`__,
`stable diffusion <https://github.com/openvinotoolkit/model_server/tree/main/demos/python_demos/stable_diffusion>`__,
and `seq2seq translation <https://github.com/openvinotoolkit/model_server/tree/main/demos/python_demos/seq2seq_translation>`__.
* Improved video stream analysis `demo <https://github.com/openvinotoolkit/model_server/tree/main/demos/real_time_stream_analysis/python>`__.
A simple client example can now process the
video stream from a local camera, video file or RTSP stream.
* Learn more about these changes on
`GitHub <https://github.com/openvinotoolkit/model_server/releases>`__.
* Jupyter Notebook Tutorials
* The following notebooks have been updated or newly added:
* `Sound generation with AudioLDM2 and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/270-sound-generation-audioldm2>`__.
* `Single-step image generation using SDXL-turbo and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/271-sdxl-turbo>`__.
* `Paint by Example using Diffusion models and OpenVINO™ <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/272-paint-by-example>`__.
* `LLM-powered chatbot using Stable-Zephyr-3b and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/273-stable-zephyr-3b-chatbot>`__.
* `Object segmentations with EfficientSAM and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/274-efficient-sam>`__.
* `Create an LLM-powered RAG system using OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot/254-rag-chatbot.ipynb>`__
- Demonstrates an integration with LangChain.
* `High-resolution image generation with Segmind-VegaRT and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/248-stable-diffusion-xl/248-segmind-vegart.ipynb>`__.
* `Text-to-Image Generation with LCM LoRA and ControlNet Conditioning <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/263-latent-consistency-models-image-generation/263-lcm-lora-controlnet.ipynb>`__.
* `LLM Instruction-following pipeline with OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/275-llm-question-answering>`__ -
Demonstrates how to run an instruction-following text generation pipeline using
tiny-llama-1b-chat, phi-2, dolly-v2-3b, red-pajama-3b-instruct and mistral-7b models.
* `LLM chabot notebook <https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot>`__
updated with support for new LLMs and INT4/INT8 Weight Compression: TinyLlama-1b-chat,
Mistral-7B, neural-chat-7b, notus-7b, ChatGLM3, youri-7b-chat (for Japanese language).
* Added optimization support (8-bit quantization, weight compression) by NNCF for the following notebooks:
* `Image generation with Würstchen and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/265-wuerstchen-image-generation>`__
* `QR-code monster <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/264-qrcode-monster>`__
* `INT4-compression support for LLaVA multimodal chatbot <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/257-llava-multimodal-chatbot>`__
* `Distil-whisper quantization <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__
* `Image generation with DeepFloyd IF <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/238-deepfloyd-if>`__
* `Instruction following using Databricks Dolly 2.0 <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following>`__
* `Visual Question Answering and Image Captioning using BLIP <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/233-blip-visual-language-processing>`__
* `Grammatical Error Correction <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/214-grammar-correction>`__
* `Universal segmentation with OneFormer <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/249-oneformer-segmentation>`__
* `Visual-language assistant with LLaVA and OpenVINO <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/257-llava-multimodal-chatbot>`__
* `Image editing with InstructPix2Pix <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/231-instruct-pix2pix-image-editing>`__
* `MMS: Scaling Speech Technology to 1000+ languages <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/255-mms-massively-multilingual-speech>`__
* `Image generation with Latent Consistency Model <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/263-latent-consistency-models-image-generation>`__
* `Object segmentations with FastSAM <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/261-fast-segment-anything>`__
* `Automatic speech recognition using Distil-Whisper <https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr>`__
Known issues Known issues
++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++
| **ID - 118179** | **ID - 127202**
| *Component* - Python API, Plugins | *Component* - CPU Plugin
| *Description:* | *Description:*
| When input byte sizes are matching, inference methods accept incorrect inputs | Deeplabv3 model from TF framework shows lower performance than previous
in copy mode (share_inputs=False). Example: [1, 4, 512, 512] is allowed when release. This is because the TopK layer in the model is now correctly
[1, 512, 512, 4] is required by the model. conducting the stable sort as specified by the model, slower than the
previous unstable sort.
| *Workaround:* | *Workaround:*
| Pass inputs which shape and layout match model ones. | This release has the correct behavior. If performance is critical,
please use the previous version of OpenVINO, or tune the model.
| **ID - 124181**
| *Component* - CPU plugin
| *Description:*
| On CPU platform with L2 cache size less than 256KB, such as i3 series of 8th
Gen Intel CORE platforms, some models may hang during model loading.
| *Workaround:*
| Rebuild the software from OpenVINO master or use the next OpenVINO release.
| **ID - 121959**
| *Component* - CPU plugin
| *Description:*
| During inference using latency hint on selected hybrid CPU platforms
(such as 12th or 13th Gen Intel CORE), there is a sporadic occurrence of
increased latency caused by the operating system scheduling of P-cores or
E-cores during OpenVINO initialization.
| *Workaround:*
| This will be fixed in the next OpenVINO release.
| **ID - 123101** | **ID - 123101**
| *Component* - GPU plugin | *Component* - GPU plugin
| *Description:* | *Description:*
| Hung up of GPU plugin on A770 Graphics (dGPU) in case of | Hung up of GPU plugin on A770 Graphics (dGPU) in case of large
large batch size (1750). batch size (1750).
| *Workaround:* | *Workaround:*
| Decrease the batch size, wait for fixed driver released. | Decrease the batch size, and wait for the fixed driver released.
Included in This Release Included in This Release
+++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++
@ -320,19 +351,19 @@ three types of operating systems: Windows, Linux, and macOS.
|| Component || License | Location | || Component || License | Location |
+================================+===================================+=================+=================+=======================+=================================================+ +================================+===================================+=================+=================+=======================+=================================================+
|| OpenVINO (Inference Engine) C++ Runtime || Dual licensing: || <install_root>/runtime/* | || OpenVINO (Inference Engine) C++ Runtime || Dual licensing: || <install_root>/runtime/* |
|| Unified API to integrate the inference with application logic || Intel® OpenVINO™ Distribution License (Version May 2021) || <install_root>/runtime/include/* | || Unified API to integrate the inference with application logic || Intel® OpenVINO™ Distribution License (Version May 2021) || <install_root>/runtime/include/* |
|| OpenVINO (Inference Engine) Headers || Apache 2.0 || | || OpenVINO (Inference Engine) Headers || Apache 2.0 || |
+--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+
|| OpenVINO (Inference Engine) Pythion API || Apache 2.0 || <install_root>/python/* | || OpenVINO (Inference Engine) Python API || Apache 2.0 || <install_root>/python/* |
+--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+
|| OpenVINO (Inference Engine) Samples || Apache 2.0 || <install_root>/samples/* | || OpenVINO (Inference Engine) Samples || Apache 2.0 || <install_root>/samples/* |
|| Samples that illustrate OpenVINO C++/ Python API usage || || | || Samples that illustrate OpenVINO C++/ Python API usage || || |
+--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+
|| [Deprecated] Deployment manager || Apache 2.0 || <install_root>/tools/deployment_manager/* | || [Deprecated] Deployment manager || Apache 2.0 || <install_root>/tools/deployment_manager/* |
|| The Deployment Manager is a Python* command-line tool that || || | || The Deployment Manager is a Python command-line tool that || || |
|| creates a deployment package by assembling the model, IR files, || || | || creates a deployment package by assembling the model, IR files, || || |
|| your application, and associated dependencies into a runtime || || | || your application, and associated dependencies into a runtime || || |
|| package for your target device. || || | || package for your target device. || || |
+--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+ +--------------------------------------------------------------------+-----------------------------------------------------------+-------------------------------------------------+
@ -360,7 +391,7 @@ enabled hardware, software or service activation. Learn more at
`http://www.intel.com/ <http://www.intel.com/>`__ `http://www.intel.com/ <http://www.intel.com/>`__
or from the OEM or retailer. or from the OEM or retailer.
No computer system can be absolutely secure. No computer system can be absolutely secure.
Intel, Atom, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks Intel, Atom, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks
of Intel Corporation in the U.S. and/or other countries. of Intel Corporation in the U.S. and/or other countries.
@ -371,18 +402,18 @@ Other names and brands may be claimed as the property of others.
Copyright © 2023, Intel Corporation. All rights reserved. Copyright © 2023, Intel Corporation. All rights reserved.
For more complete information about compiler optimizations, see our Optimization Notice. For more complete information about compiler optimizations, see our Optimization Notice.
Performance varies by use, configuration and other factors. Learn more at Performance varies by use, configuration and other factors. Learn more at
`www.Intel.com/PerformanceIndex <www.Intel.com/PerformanceIndex>`__. `www.Intel.com/PerformanceIndex <www.Intel.com/PerformanceIndex>`__.
Download Download
+++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++
`The OpenVINO product selector tool <https://docs.openvino.ai/install>`__ `The OpenVINO product selector tool <https://docs.openvino.ai/install>`__
provides easy access to the right packages that match your desired OS, version, provides easy access to the right packages that match your desired OS, version,
and distribution options. and distribution options.

View File

@ -2,13 +2,16 @@
System Requirements System Requirements
=================== ===================
Certain hardware requires specific drivers to work properly with OpenVINO. .. note::
These drivers, including Linux* kernels, might require updates to your operating system,
which is not part of OpenVINO installation. Refer to your hardware's documentation Certain hardware (including but not limited to GPU, GNA, and latest CPUs) requires manual
for updating instructions. installation of specific drivers and/or other software components to work correctly and/or
to utilize hardware capabilities at their best. This might require updates to operating
system, including but not limited to Linux kernel, please refer to their documentation
for details. These modifications should be handled by user and are not part of OpenVINO
installation.
CPU CPU
@ -20,26 +23,24 @@ CPU
* Intel Atom® processor with Intel® SSE4.2 support * Intel Atom® processor with Intel® SSE4.2 support
* Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics * Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics
* 6th - 13th generation Intel® Core™ processors * 6th - 14th generation Intel® Core™ processors
* Intel®Core™Ultra (codenameMeteor Lake) * Intel® Core™ Ultra (codename Meteor Lake)
* Intel® Xeon® Scalable Processors (code name Skylake) * 1st - 5th generation Intel® Xeon® Scalable Processors
* 2nd Generation Intel® Xeon® Scalable Processors (code name Cascade Lake) * ARM and ARM64 CPUs; Apple M1, M2, and Raspberry Pi
* 3rd Generation Intel® Xeon® Scalable Processors(code nameCooper Lakeand Ice Lake)
* 4th Generation Intel® Xeon® Scalable Processors(code name Sapphire Rapids)
* ARM* and ARM64 CPUs; Apple M1, M2 and Raspberry Pi
.. tab-item:: Supported Operating Systems .. tab-item:: Supported Operating Systems
* Ubuntu 22.04 long-term support (LTS), 64-bit (Kernel 5.15+) * Ubuntu 22.04 long-term support (LTS), 64-bit (Kernel 5.15+)
* Ubuntu 20.04 long-term support (LTS), 64-bit (Kernel 5.15+) * Ubuntu 20.04 long-term support (LTS), 64-bit (Kernel 5.15+)
* Ubuntu 18.04 long-term support (LTS) with limitations, 64-bit (Kernel 5.4+) * Ubuntu 18.04 long-term support (LTS) with limitations, 64-bit (Kernel 5.4+)
* Windows* 10 * Windows 10
* Windows* 11 * Windows 11
* macOS* 10.15 and above, 64-bit * macOS 10.15 and above, 64-bit
* macOS 11 and above, ARM64 * macOS 11 and above, ARM64
* Red Hat Enterprise Linux* 8, 64-bit * CentOS 7
* Debian 9 ARM64 and ARM * Red Hat Enterprise Linux 8, 64-bit
* CentOS 7 64-bit * Ubuntu 18 ARM64
* Debian 9 ARM
GPU GPU
########## ##########
@ -53,7 +54,7 @@ GPU
* Intel® Iris® Pro Graphics * Intel® Iris® Pro Graphics
* Intel® Iris® Xe Graphics * Intel® Iris® Xe Graphics
* Intel® Iris® Xe Max Graphics * Intel® Iris® Xe Max Graphics
* Intel® Arc ™ GPU Series * Intel® Arc™ GPU Series
* Intel® Data Center GPU Flex Series * Intel® Data Center GPU Flex Series
* Intel® Data Center GPU Max Series * Intel® Data Center GPU Max Series
@ -63,7 +64,7 @@ GPU
* Ubuntu 20.04 long-term support (LTS), 64-bit * Ubuntu 20.04 long-term support (LTS), 64-bit
* Windows 10, 64-bit * Windows 10, 64-bit
* Windows 11, 64-bit * Windows 11, 64-bit
* Centos 7 * CentOS 7
* Red Hat Enterprise Linux 8, 64-bit * Red Hat Enterprise Linux 8, 64-bit
.. tab-item:: Additional considerations .. tab-item:: Additional considerations
@ -72,24 +73,24 @@ GPU
Distribution of OpenVINO™ toolkit package. Distribution of OpenVINO™ toolkit package.
* A chipset that supports processor graphics is required for Intel® Xeon® * A chipset that supports processor graphics is required for Intel® Xeon®
processors. Processor graphics are not included in all processors. See processors. Processor graphics are not included in all processors. See
`Product Specifications <https://ark.intel.com/>`__ `Product Specifications <https://ark.intel.com/>`__
for information about your processor. for information about your processor.
* Although this release works with Ubuntu 20.04 for discrete graphic cards, * While this release of OpenVINO supports Ubuntu 20.04, the driver stack
Ubuntu 20.04 is not POR for discrete graphics drivers, so OpenVINO support for Intel discrete graphic cards does not fully support Ubuntu 20.04.
is limited. We recommend using Ubuntu 22.04 when executing on discrete graphics.
* The following minimum (i.e., used for old hardware) OpenCL™ driver's versions * The following minimum (i.e., used for old hardware) OpenCL™ driver's versions
were used during OpenVINO internal validation: 22.43 for Ubuntu 22.04, 21.48 were used during OpenVINO internal validation: 22.43 for Ubuntu 22.04, 21.48
for Ubuntu 20.04 and 21.49 for Red Hat Enterprise Linux 8. for Ubuntu 20.04 and 21.49 for Red Hat Enterprise Linux 8.
NPU and GNA Intel® Neural Processing Unit
############################# ################################
.. tab-set:: .. tab-set::
.. tab-item:: Operating Systems for NPU .. tab-item:: Operating Systems for NPU
* Ubuntu 22.04 long-term support (LTS), 64-bit * Ubuntu 22.04 long-term support (LTS), 64-bit
* Windows 11, 64-bit * Windows 11, 64-bit (22H2, 23H2)
.. tab-item:: Operating Systems for GNA .. tab-item:: Operating Systems for GNA
@ -100,10 +101,23 @@ NPU and GNA
.. tab-item:: Additional considerations .. tab-item:: Additional considerations
* These Accelerators require drivers that are not included in the * These Accelerators require :doc:`drivers <openvino_docs_install_guides_configurations_for_intel_npu>`
Intel® Distribution of OpenVINO™ toolkit package. that are not included in the Intel® Distribution of OpenVINO™ toolkit package.
* Users can access the NPU plugin through the OpenVINO archives on * Users can access the NPU plugin through the OpenVINO archives on
the download page. the :doc:`download page <openvino_docs_install_guides_overview>`.
Intel® Gaussian & Neural Accelerator
##########################################
.. tab-set::
.. tab-item:: Operating Systems for GNA
* Ubuntu 22.04 long-term support (LTS), 64-bit
* Ubuntu 20.04 long-term support (LTS), 64-bit
* Windows 10, 64-bit
* Windows 11, 64-bit
Operating systems and developer environment Operating systems and developer environment
@ -111,7 +125,7 @@ Operating systems and developer environment
.. tab-set:: .. tab-set::
.. tab-item:: Linux .. tab-item:: Linux OS
* Ubuntu 22.04 with Linux kernel 5.15+ * Ubuntu 22.04 with Linux kernel 5.15+
* Ubuntu 20.04 with Linux kernel 5.15+ * Ubuntu 20.04 with Linux kernel 5.15+
@ -127,16 +141,13 @@ Operating systems and developer environment
* `GNU Compiler Collection (GCC) <https://www.gnu.org/software/gcc/>`__ 7.5 and above * `GNU Compiler Collection (GCC) <https://www.gnu.org/software/gcc/>`__ 7.5 and above
* `CMake <https://cmake.org/download/>`__ 3.10 or higher * `CMake <https://cmake.org/download/>`__ 3.10 or higher
Higher versions of kernel might be required for 10th Gen Intel® Core™ Processors, Higher versions of kernel might be required for 10th Gen Intel® Core™ Processors, 11th Gen
11th Gen Intel® Core™ Processors, 11th Gen Intel® Core™ Processors S-Series Processors, Intel® Core™ Processors, 11th Gen Intel® Core™ Processors S-Series Processors, 12th Gen
12th Gen Intel® Core™ Processors, 13th Gen Intel® Core™ Processors, Intel® Core™ Ultra Intel® Core™ Processors, 13th Gen Intel® Core™ Processors, 14th Gen
Processors, or 4th Gen Intel® Xeon® Scalable Processors to support CPU, GPU, GNA or Intel® Core™ Processors, Intel® Core™ Ultra Processors, 4th Gen Intel® Xeon® Scalable Processors
hybrid-cores CPU capabilities. or 5th Gen Intel® Xeon® Scalable Processors to support CPU, GPU, GNA or hybrid-cores CPU capabilities.
.. tab-item:: Windows .. tab-item:: Windows 10 and 11
* Windows 10
* Windows 11
Build environment components: Build environment components:
@ -158,26 +169,25 @@ Operating systems and developer environment
.. tab-item:: DL frameworks versions: .. tab-item:: DL frameworks versions:
* TensorFlow* 1.15, 2.12 * TensorFlow 1.15, 2.12
* MxNet* 1.9.0 * MxNet 1.9.0
* ONNX* 1.14.1 * ONNX 1.14.1
* PaddlePaddle* 2.4 * PaddlePaddle 2.4
This package can be installed on other versions of DL Framework This package can be installed on other versions of DL Frameworks
but only the version specified here is fully validated. but only the versions specified here are fully validated.
.. note:: .. note::
OpenVINO Python binaries and binaries on Windows/CentOS7/MACOS(x86) are built OpenVINO Python binaries and binaries on Windows, CentOS 7, and macOS (x86) are built
with oneTBB libraries. Other binaries on Ubuntu and Redhat OSes are built with with oneTBB libraries, and others on Ubuntu and RedHat systems are built with
legacy TBB which is released by OS distribution. OpenVINO can be built with legacy TBB which is released by OS distribution. OpenVINO can be built from source
either oneTBB or legacy TBB by the user on all OS systems listed. System with either oneTBB or legacy TBB on all the systems listed here. System
compatibility and performance are improved on Hybrid CPUs, compatibility and performance are improved on Hybrid CPUs
such as 12th Gen Intel Core and above. such as 12th Gen Intel Core and above.
Legal Information Legal Information
+++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++
@ -202,7 +212,7 @@ enabled hardware, software or service activation. Learn more at
`http://www.intel.com/ <http://www.intel.com/>`__ `http://www.intel.com/ <http://www.intel.com/>`__
or from the OEM or retailer. or from the OEM or retailer.
No computer system can be absolutely secure. No computer system can be absolutely secure.
Intel, Atom, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks Intel, Atom, Arria, Core, Movidius, Xeon, OpenVINO, and the Intel logo are trademarks
of Intel Corporation in the U.S. and/or other countries. of Intel Corporation in the U.S. and/or other countries.
@ -213,9 +223,9 @@ Other names and brands may be claimed as the property of others.
Copyright © 2023, Intel Corporation. All rights reserved. Copyright © 2023, Intel Corporation. All rights reserved.
For more complete information about compiler optimizations, see our Optimization Notice. For more complete information about compiler optimizations, see our Optimization Notice.
Performance varies by use, configuration and other factors. Learn more at Performance varies by use, configuration and other factors. Learn more at
`www.Intel.com/PerformanceIndex <www.Intel.com/PerformanceIndex>`__. `www.Intel.com/PerformanceIndex <www.Intel.com/PerformanceIndex>`__.

View File

@ -5,8 +5,8 @@ OpenVINO Extensibility Mechanism
.. meta:: .. meta::
:description: Explore OpenVINO™ Extensibility API, which allows adding :description: Explore OpenVINO™ Extensibility API, which allows adding
support for models with custom operations and their further implementation support for models with custom operations and their further implementation
in applications. in applications.
.. toctree:: .. toctree::
@ -16,18 +16,18 @@ OpenVINO Extensibility Mechanism
openvino_docs_Extensibility_UG_add_openvino_ops openvino_docs_Extensibility_UG_add_openvino_ops
openvino_docs_Extensibility_UG_Frontend_Extensions openvino_docs_Extensibility_UG_Frontend_Extensions
openvino_docs_Extensibility_UG_GPU openvino_docs_Extensibility_UG_GPU
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1
:hidden: :hidden:
openvino_docs_transformations openvino_docs_transformations
OpenVINO Plugin Developer Guide <openvino_docs_ie_plugin_dg_overview> OpenVINO Plugin Developer Guide <openvino_docs_ie_plugin_dg_overview>
The Intel® Distribution of OpenVINO™ toolkit supports neural-network models trained with various frameworks, including The Intel® Distribution of OpenVINO™ toolkit supports neural-network models trained with various frameworks, including
TensorFlow, PyTorch, ONNX, TensorFlow Lite, and PaddlePaddle (OpenVINO support for Apache MXNet, Caffe, and Kaldi is currently TensorFlow, PyTorch, ONNX, TensorFlow Lite, and PaddlePaddle (OpenVINO support for Apache MXNet, Caffe, and Kaldi is currently
being deprecated and will be removed entirely in the future). The list of supported operations is different for each of the supported frameworks. being deprecated and will be removed entirely in the future). The list of supported operations is different for each of the supported frameworks.
To see the operations supported by your framework, refer to :doc:`Supported Framework Operations <openvino_resources_supported_operations_frontend>`. To see the operations supported by your framework, refer to :doc:`Supported Framework Operations <openvino_resources_supported_operations_frontend>`.
Custom operations, which are not included in the list, are not recognized by OpenVINO out-of-the-box. The need for custom operation may appear in two cases: Custom operations, which are not included in the list, are not recognized by OpenVINO out-of-the-box. The need for custom operation may appear in two cases:
@ -80,32 +80,32 @@ Registering Extensions
A custom operation class and a new mapping frontend extension class object should be registered to be usable in OpenVINO runtime. A custom operation class and a new mapping frontend extension class object should be registered to be usable in OpenVINO runtime.
.. note:: .. note::
This documentation is derived from the `Template extension <https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension/new>`__, which demonstrates the details of extension development. It is based on minimalistic ``Identity`` operation that is a placeholder for your real custom operation. Review the complete, fully compilable code to see how it works. This documentation is derived from the `Template extension <https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension/new>`__, which demonstrates the details of extension development. It is based on minimalistic ``Identity`` operation that is a placeholder for your real custom operation. Review the complete, fully compilable code to see how it works.
Use the ``:ref:`ov::Core::add_extension <doxid-classov_1_1_core_1a68d0dea1cbcd42a67bea32780e32acea>``` method to load the extensions to the ``:ref:`ov::Core <doxid-classov_1_1_core>``` object. This method allows loading library with extensions or extensions from the code. Use the ``ov::Core::add_extension`` method to load the extensions to the ``ov::Core`` object. This method allows loading library with extensions or extensions from the code.
Load Extensions to Core Load Extensions to Core
+++++++++++++++++++++++ +++++++++++++++++++++++
Extensions can be loaded from a code with the ``:ref:`ov::Core::add_extension <doxid-classov_1_1_core_1a68d0dea1cbcd42a67bea32780e32acea>``` method: Extensions can be loaded from a code with the ``ov::Core::add_extension`` method:
.. tab-set:: .. tab-set::
.. tab-item:: Python .. tab-item:: Python
:sync: py :sync: py
.. doxygensnippet:: docs/snippets/ov_extensions.py .. doxygensnippet:: docs/snippets/ov_extensions.py
:language: python :language: python
:fragment: [add_extension] :fragment: [add_extension]
.. tab-item:: C++ .. tab-item:: C++
:sync: cpp :sync: cpp
.. doxygensnippet:: docs/snippets/ov_extensions.cpp .. doxygensnippet:: docs/snippets/ov_extensions.cpp
:language: cpp :language: cpp
:fragment: [add_extension] :fragment: [add_extension]
The ``Identity`` is a custom operation class defined in :doc:`Custom Operation Guide <openvino_docs_Extensibility_UG_add_openvino_ops>`. This is sufficient to enable reading OpenVINO IR which uses the ``Identity`` extension operation emitted by Model Optimizer. In order to load original model directly to the runtime, add a mapping extension: The ``Identity`` is a custom operation class defined in :doc:`Custom Operation Guide <openvino_docs_Extensibility_UG_add_openvino_ops>`. This is sufficient to enable reading OpenVINO IR which uses the ``Identity`` extension operation emitted by Model Optimizer. In order to load original model directly to the runtime, add a mapping extension:
@ -130,7 +130,7 @@ When Python API is used, there is no way to implement a custom OpenVINO operatio
Python can still be used to map and decompose operations when only operations from the standard OpenVINO operation set are used. Python can still be used to map and decompose operations when only operations from the standard OpenVINO operation set are used.
.. _create_a_library_with_extensions: .. _create_a_library_with_extensions:
Create a Library with Extensions Create a Library with Extensions
++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++
@ -142,7 +142,7 @@ An extension library should be created in the following cases:
To create an extension library, for example, to load the extensions into Model Optimizer, perform the following: To create an extension library, for example, to load the extensions into Model Optimizer, perform the following:
1. Create an entry point for extension library. OpenVINO provides the ``:ref:`OPENVINO_CREATE_EXTENSIONS() <doxid-core_2include_2openvino_2core_2extension_8hpp_1acdadcfa0eff763d8b4dadb8a9cb6f6e6>``` macro, which allows to define an entry point to a library with OpenVINO Extensions. 1. Create an entry point for extension library. OpenVINO provides the ``OPENVINO_CREATE_EXTENSIONS()`` macro, which allows to define an entry point to a library with OpenVINO Extensions.
This macro should have a vector of all OpenVINO Extensions as an argument. This macro should have a vector of all OpenVINO Extensions as an argument.
Based on that, the declaration of an extension class might look like the following: Based on that, the declaration of an extension class might look like the following:
@ -162,7 +162,7 @@ This CMake script finds OpenVINO, using the ``find_package`` CMake command.
3. Build the extension library, running the commands below: 3. Build the extension library, running the commands below:
.. code-block:: sh .. code-block:: sh
$ cd src/core/template_extension/new $ cd src/core/template_extension/new
$ mkdir build $ mkdir build
$ cd build $ cd build
@ -173,16 +173,16 @@ This CMake script finds OpenVINO, using the ``find_package`` CMake command.
4. After the build, you may use the path to your extension library to load your extensions to OpenVINO Runtime: 4. After the build, you may use the path to your extension library to load your extensions to OpenVINO Runtime:
.. tab-set:: .. tab-set::
.. tab-item:: Python .. tab-item:: Python
:sync: py :sync: py
.. doxygensnippet:: docs/snippets/ov_extensions.py .. doxygensnippet:: docs/snippets/ov_extensions.py
:language: python :language: python
:fragment: [add_extension_lib] :fragment: [add_extension_lib]
.. tab-item:: C++ .. tab-item:: C++
:sync: cpp :sync: cpp
.. doxygensnippet:: docs/snippets/ov_extensions.cpp .. doxygensnippet:: docs/snippets/ov_extensions.cpp
:language: cpp :language: cpp
@ -194,5 +194,5 @@ See Also
* :doc:`OpenVINO Transformations <openvino_docs_transformations>` * :doc:`OpenVINO Transformations <openvino_docs_transformations>`
* :doc:`Using OpenVINO Runtime Samples <openvino_docs_OV_UG_Samples_Overview>` * :doc:`Using OpenVINO Runtime Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Hello Shape Infer SSD sample <openvino_inference_engine_samples_hello_reshape_ssd_README>` * :doc:`Hello Shape Infer SSD sample <openvino_sample_hello_reshape_ssd>`

View File

@ -5,7 +5,7 @@ How to Implement Custom GPU Operations
.. meta:: .. meta::
:description: Learn the details of custom kernel support for the GPU device to :description: Learn the details of custom kernel support for the GPU device to
enable operations not supported by OpenVINO. enable operations not supported by OpenVINO.
@ -16,20 +16,20 @@ The GPU codepath abstracts many details about OpenCL. You need to provide the ke
There are two options for using the custom operation configuration file: There are two options for using the custom operation configuration file:
* Include a section with your kernels into the automatically-loaded ``<lib_path>/cldnn_global_custom_kernels/cldnn_global_custom_kernels.xml`` file. * Include a section with your kernels into the automatically-loaded ``<lib_path>/cldnn_global_custom_kernels/cldnn_global_custom_kernels.xml`` file.
* Call the ``:ref:`ov::Core::set_property() <doxid-classov_1_1_core_1aa953cb0a1601dbc9a34ef6ba82b8476e>``` method from your application with the ``"CONFIG_FILE"`` key and the configuration file name as a value before loading the network that uses custom operations to the plugin: * Call the ``ov::Core::set_property()`` method from your application with the ``"CONFIG_FILE"`` key and the configuration file name as a value before loading the network that uses custom operations to the plugin:
.. tab-set:: .. tab-set::
.. tab-item:: Python .. tab-item:: Python
:sync: py :sync: py
.. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.py .. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.py
:language: python :language: python
:fragment: [part0] :fragment: [part0]
.. tab-item:: C++ .. tab-item:: C++
:sync: cpp :sync: cpp
.. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.cpp .. doxygensnippet:: docs/snippets/gpu/custom_kernels_api.cpp
:language: cpp :language: cpp
:fragment: [part0] :fragment: [part0]
@ -43,10 +43,10 @@ feature a dedicated command-line option ``-c`` to load custom kernels. For examp
$ ./classification_sample -m <path_to_model>/bvlc_alexnet_fp16.xml -i ./validation_set/daily/227x227/apron.bmp -d GPU $ ./classification_sample -m <path_to_model>/bvlc_alexnet_fp16.xml -i ./validation_set/daily/227x227/apron.bmp -d GPU
-c <absolute_path_to_config>/custom_layer_example.xml -c <absolute_path_to_config>/custom_layer_example.xml
.. _config-file-format: .. _config-file-format:
Configuration File Format Configuration File Format
######################### #########################
The configuration file is expected to follow the ``.xml`` file structure The configuration file is expected to follow the ``.xml`` file structure
with a node of the type ``CustomLayer`` for every custom operation you provide. with a node of the type ``CustomLayer`` for every custom operation you provide.
@ -344,7 +344,7 @@ Example Kernel
.. _debugging-tips: .. _debugging-tips:
.. note:: .. note::
As described in the previous section, all items such as the ``INPUT0_TYPE`` are actually defined as OpenCL (pre-)compiler inputs by OpenVINO for efficiency reasons. See the `Debugging Tips <#debugging-tips>`__ below for information on debugging the results. As described in the previous section, all items such as the ``INPUT0_TYPE`` are actually defined as OpenCL (pre-)compiler inputs by OpenVINO for efficiency reasons. See the `Debugging Tips <#debugging-tips>`__ below for information on debugging the results.
Debugging Tips Debugging Tips

View File

@ -5,8 +5,8 @@ Custom OpenVINO™ Operations
.. meta:: .. meta::
:description: Explore OpenVINO™ Extension API which enables registering :description: Explore OpenVINO™ Extension API which enables registering
custom operations to support models with operations custom operations to support models with operations
not supported by OpenVINO. not supported by OpenVINO.
OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box. This capability requires writing code in C++, so if you are using Python to develop your application you need to build a separate shared library implemented in C++ first and load it in Python using ``add_extension`` API. Please refer to :ref:`Create library with extensions <create_library_with_extensions>` for more details on library creation and usage. The remining part of this document describes how to implement an operation class. OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box. This capability requires writing code in C++, so if you are using Python to develop your application you need to build a separate shared library implemented in C++ first and load it in Python using ``add_extension`` API. Please refer to :ref:`Create library with extensions <create_library_with_extensions>` for more details on library creation and usage. The remining part of this document describes how to implement an operation class.
@ -14,7 +14,7 @@ OpenVINO™ Extension API allows you to register custom operations to support mo
Operation Class Operation Class
############### ###############
To add your custom operation, create a new class that extends ``ov::Op``, which is in turn derived from ``:ref:`ov::Node <doxid-classov_1_1_node>```, the base class for all graph operations in OpenVINO™. To add ``ov::Op``, include the next file: To add your custom operation, create a new class that extends ``ov::Op``, which is in turn derived from ``ov::Node``, the base class for all graph operations in OpenVINO™. To add ``ov::Op``, include the next file:
.. doxygensnippet:: ./src/core/template_extension/new/identity.hpp .. doxygensnippet:: ./src/core/template_extension/new/identity.hpp
:language: cpp :language: cpp
@ -24,9 +24,9 @@ Follow the steps below to add a custom operation:
1. Add the ``OPENVINO_OP`` macro which defines a ``NodeTypeInfo`` object that identifies the type of the operation to the graph users and helps with dynamic type resolution. The type info of an operation currently consists of a string operation identifier and a string for operation version. 1. Add the ``OPENVINO_OP`` macro which defines a ``NodeTypeInfo`` object that identifies the type of the operation to the graph users and helps with dynamic type resolution. The type info of an operation currently consists of a string operation identifier and a string for operation version.
2. Implement default constructor and constructors that optionally take the operation inputs and attributes as parameters. 2. Implement default constructor and constructors that optionally take the operation inputs and attributes as parameters.
3. Override the shape inference method ``validate_and_infer_types``. This method is called multiple times during graph manipulations to determine the shapes and element types of the operations outputs. To access the input shapes and input element types, use the ``get_input_partial_shape()`` and ``get_input_element_type()`` methods of ``:ref:`ov::Node <doxid-classov_1_1_node>```. Set the inferred shape and element type of the output using ``set_output_type``. 3. Override the shape inference method ``validate_and_infer_types``. This method is called multiple times during graph manipulations to determine the shapes and element types of the operations outputs. To access the input shapes and input element types, use the ``get_input_partial_shape()`` and ``get_input_element_type()`` methods of ``ov::Node``. Set the inferred shape and element type of the output using ``set_output_type``.
4. Override the ``clone_with_new_inputs`` method, which enables graph manipulation routines to create copies of this operation and connect it to different nodes during optimization. 4. Override the ``clone_with_new_inputs`` method, which enables graph manipulation routines to create copies of this operation and connect it to different nodes during optimization.
@ -40,9 +40,9 @@ Based on that, declaration of an operation class can look as follows:
Operation Constructors Operation Constructors
++++++++++++++++++++++ ++++++++++++++++++++++
OpenVINO™ operation contains two constructors: OpenVINO™ operation contains two constructors:
* Default constructor, which enables you to create an operation without attributes * Default constructor, which enables you to create an operation without attributes
* Constructor that creates and validates an operation with specified inputs and attributes * Constructor that creates and validates an operation with specified inputs and attributes
.. doxygensnippet:: ./src/core/template_extension/new/identity.cpp .. doxygensnippet:: ./src/core/template_extension/new/identity.cpp
@ -52,7 +52,7 @@ OpenVINO™ operation contains two constructors:
``validate_and_infer_types()`` ``validate_and_infer_types()``
++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++
``:ref:`ov::Node::validate_and_infer_types <doxid-classov_1_1_node_1ac5224b5be848ec670d2078d9816d12e7>``` method validates operation attributes and calculates output shapes using attributes of the operation. ``ov::Node::validate_and_infer_types`` method validates operation attributes and calculates output shapes using attributes of the operation.
.. doxygensnippet:: ./src/core/template_extension/new/identity.cpp .. doxygensnippet:: ./src/core/template_extension/new/identity.cpp
:language: cpp :language: cpp
@ -61,7 +61,7 @@ OpenVINO™ operation contains two constructors:
``clone_with_new_inputs()`` ``clone_with_new_inputs()``
+++++++++++++++++++++++++++ +++++++++++++++++++++++++++
``:ref:`ov::Node::clone_with_new_inputs <doxid-classov_1_1_node_1a04cb103fa069c3b7944ab7c44d94f5ff>``` method creates a copy of the operation with new inputs. ``ov::Node::clone_with_new_inputs`` method creates a copy of the operation with new inputs.
.. doxygensnippet:: ./src/core/template_extension/new/identity.cpp .. doxygensnippet:: ./src/core/template_extension/new/identity.cpp
:language: cpp :language: cpp
@ -70,7 +70,7 @@ OpenVINO™ operation contains two constructors:
``visit_attributes()`` ``visit_attributes()``
++++++++++++++++++++++ ++++++++++++++++++++++
``:ref:`ov::Node::visit_attributes <doxid-classov_1_1_node_1a9743b56d352970486d17dae2416d958e>``` method enables you to visit all operation attributes. ``ov::Node::visit_attributes`` method enables you to visit all operation attributes.
.. doxygensnippet:: ./src/core/template_extension/new/identity.cpp .. doxygensnippet:: ./src/core/template_extension/new/identity.cpp
:language: cpp :language: cpp
@ -79,7 +79,7 @@ OpenVINO™ operation contains two constructors:
``evaluate() and has_evaluate()`` ``evaluate() and has_evaluate()``
+++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++
``:ref:`ov::Node::evaluate <doxid-classov_1_1_node_1acfb82acc8349d7138aeaa05217c7014e>``` method enables you to apply constant folding to an operation. ``ov::Node::evaluate`` method enables you to apply constant folding to an operation.
.. doxygensnippet:: ./src/core/template_extension/new/identity.cpp .. doxygensnippet:: ./src/core/template_extension/new/identity.cpp
:language: cpp :language: cpp

View File

@ -96,6 +96,6 @@ Detailed Guides
API References API References
############## ##############
* `OpenVINO Plugin API <https://docs.openvino.ai/2023.2/groupov_dev_api.html>`__ * `OpenVINO Plugin API <https://docs.openvino.ai/2023.3/api/c_cpp_api/group__ov__dev__api.html>`__
* `OpenVINO Transformation API <https://docs.openvino.ai/2023.2/groupie_transformation_api.html>`__ * `OpenVINO Transformation API <https://docs.openvino.ai/2023.3/api/c_cpp_api/group__ie__transformation__api.html>`__

View File

@ -329,7 +329,7 @@ After that you should quantize model by the :doc:`Model Quantizer <omz_tools_dow
Inference Inference
+++++++++ +++++++++
The simplest way to infer the model and collect performance counters is :doc:`Benchmark Application <openvino_inference_engine_samples_benchmark_app_README>`. The simplest way to infer the model and collect performance counters is :doc:`Benchmark Application <openvino_sample_benchmark_tool>`.
.. code-block:: sh .. code-block:: sh

View File

@ -5,7 +5,7 @@ Step 2. Markup Transformations
.. meta:: .. meta::
:description: Learn about markup transformations, which are used to create :description: Learn about markup transformations, which are used to create
attributes for input and output ports and operations during runtime. attributes for input and output ports and operations during runtime.
.. toctree:: .. toctree::
@ -21,7 +21,7 @@ Step 2. Markup Transformations
PropagatePrecisions <openvino_docs_OV_UG_lpt_PropagatePrecisions> PropagatePrecisions <openvino_docs_OV_UG_lpt_PropagatePrecisions>
AlignQuantizationIntervals <openvino_docs_OV_UG_lpt_AlignQuantizationIntervals> AlignQuantizationIntervals <openvino_docs_OV_UG_lpt_AlignQuantizationIntervals>
AlignQuantizationParameters <openvino_docs_OV_UG_lpt_AlignQuantizationParameters> AlignQuantizationParameters <openvino_docs_OV_UG_lpt_AlignQuantizationParameters>
CreateAttribute <openvino_docs_OV_UG_lpt_CreateAttribute> CreateAttribute <openvino_docs_OV_UG_lpt_CreateAttribute>
CreatePrecisionsDependentAttribute <openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute> CreatePrecisionsDependentAttribute <openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute>
PropagateThroughPrecisionPreserved <openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved> PropagateThroughPrecisionPreserved <openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved>
@ -31,13 +31,13 @@ Step 2. Markup Transformations
This step defines the optimal ``FakeQuantize`` decomposition precisions for the best inference performance via operations markup with runtime attribute instances. Attributes are created for input and output ports and operations. Transformations do not change the operation output port precisions. A model markup low precision logic is decomposed and implemented into the following common markup transformations. The order of transformations is important: This step defines the optimal ``FakeQuantize`` decomposition precisions for the best inference performance via operations markup with runtime attribute instances. Attributes are created for input and output ports and operations. Transformations do not change the operation output port precisions. A model markup low precision logic is decomposed and implemented into the following common markup transformations. The order of transformations is important:
1. :doc:`MarkupBias <openvino_docs_OV_UG_lpt_MarkupBias>` 1. :doc:`MarkupBias <openvino_docs_OV_UG_lpt_MarkupBias>`
2. :doc:`MarkupCanBeQuantized <openvino_docs_OV_UG_lpt_MarkupCanBeQuantized>` 2. :doc:`MarkupCanBeQuantized <openvino_docs_OV_UG_lpt_MarkupCanBeQuantized>`
3. :doc:`MarkupPrecisions <openvino_docs_OV_UG_lpt_MarkupPrecisions>` 3. :doc:`MarkupPrecisions <openvino_docs_OV_UG_lpt_MarkupPrecisions>`
4. :doc:`MarkupPerTensorQuantization <openvino_docs_OV_UG_lpt_MarkupPerTensorQuantization>` 4. :doc:`MarkupPerTensorQuantization <openvino_docs_OV_UG_lpt_MarkupPerTensorQuantization>`
5. :doc:`MarkupAvgPoolPrecisionPreserved <openvino_docs_OV_UG_lpt_MarkupAvgPoolPrecisionPreserved>` 5. :doc:`MarkupAvgPoolPrecisionPreserved <openvino_docs_OV_UG_lpt_MarkupAvgPoolPrecisionPreserved>`
6. :doc:`PropagatePrecisions <openvino_docs_OV_UG_lpt_PropagatePrecisions>` 6. :doc:`PropagatePrecisions <openvino_docs_OV_UG_lpt_PropagatePrecisions>`
7. :doc:`AlignQuantizationIntervals <openvino_docs_OV_UG_lpt_AlignQuantizationIntervals>` 7. :doc:`AlignQuantizationIntervals <openvino_docs_OV_UG_lpt_AlignQuantizationIntervals>`
8. :doc:`AlignQuantizationParameters <openvino_docs_OV_UG_lpt_AlignQuantizationParameters>` 8. :doc:`AlignQuantizationParameters <openvino_docs_OV_UG_lpt_AlignQuantizationParameters>`
.. list-table:: .. list-table::
:header-rows: 1 :header-rows: 1
@ -47,16 +47,16 @@ This step defines the optimal ``FakeQuantize`` decomposition precisions for the
- Use attributes - Use attributes
* - MarkupBias * - MarkupBias
- Bias - Bias
- -
* - MarkupCanBeQuantized * - MarkupCanBeQuantized
- Precisions - Precisions
- -
* - MarkupPrecisions * - MarkupPrecisions
- Precisions,PrecisionPreserved - Precisions,PrecisionPreserved
- -
* - MarkupPerTensorQuantization * - MarkupPerTensorQuantization
- PerTensorQuantization - PerTensorQuantization
- -
* - MarkupAvgPoolPrecisionPreserved * - MarkupAvgPoolPrecisionPreserved
- AvgPoolPrecisionPreserved - AvgPoolPrecisionPreserved
- Precisions, PrecisionPreserved - Precisions, PrecisionPreserved
@ -70,16 +70,16 @@ This step defines the optimal ``FakeQuantize`` decomposition precisions for the
- QuantizationAlignment - QuantizationAlignment
- PrecisionPreserved, PerTensorQuantization - PrecisionPreserved, PerTensorQuantization
.. note:: .. note::
The same type of attribute instances can be created in different transformations. This approach is the result of the transformation single-responsibility principle. For example, ``Precision`` attribute instances are created in ``MarkupCanBeQuantized`` and ``MarkupPrecisions`` transformations, but the reasons for their creation are different The same type of attribute instances can be created in different transformations. This approach is the result of the transformation single-responsibility principle. For example, ``Precision`` attribute instances are created in ``MarkupCanBeQuantized`` and ``MarkupPrecisions`` transformations, but the reasons for their creation are different
Common markup transformations can be decomposed into simpler utility markup transformations. The order of Markup utility transformations is not important: Common markup transformations can be decomposed into simpler utility markup transformations. The order of Markup utility transformations is not important:
* :doc:`CreateAttribute <openvino_docs_OV_UG_lpt_CreateAttribute>` * :doc:`CreateAttribute <openvino_docs_OV_UG_lpt_CreateAttribute>`
* :doc:`CreatePrecisionsDependentAttribute <openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute>` * :doc:`CreatePrecisionsDependentAttribute <openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute>`
* :doc:`PropagateThroughPrecisionPreserved <openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved>` * :doc:`PropagateThroughPrecisionPreserved <openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved>`
* :doc:`PropagateToInput <openvino_docs_OV_UG_lpt_PropagateToInput>` * :doc:`PropagateToInput <openvino_docs_OV_UG_lpt_PropagateToInput>`
* :doc:`UpdateSharedPrecisionPreserved <openvino_docs_OV_UG_lpt_UpdateSharedPrecisionPreserved>` * :doc:`UpdateSharedPrecisionPreserved <openvino_docs_OV_UG_lpt_UpdateSharedPrecisionPreserved>`
Let's explore all transformations and their relations in detail, using one and the same model: Let's explore all transformations and their relations in detail, using one and the same model:
@ -90,12 +90,12 @@ The original model key features:
* The first ``concat1`` concatenation operation has not quantized ``convolution1`` consumer. * The first ``concat1`` concatenation operation has not quantized ``convolution1`` consumer.
* The second ``concat2`` concatenation operation has quantized ``convolution2`` consumer with requirements: * The second ``concat2`` concatenation operation has quantized ``convolution2`` consumer with requirements:
* support ``unsigned int8`` on activations, * support ``unsigned int8`` on activations,
* per-tensor quantization. * per-tensor quantization.
* Between the ``concat2`` concatenation operation and ``Convolution`` there is an ``AvgPool`` operation, which mathematically should return an ``f32`` tensor. But the ``MarkupAvgPoolPrecisionPreserved`` transformation is active. This allows the low precision transformation, that goes after the ``AvgPool``, to propagate low precision tensor to the next consumer. * Between the ``concat2`` concatenation operation and ``Convolution`` there is an ``AvgPool`` operation, which mathematically should return an ``f32`` tensor. But the ``MarkupAvgPoolPrecisionPreserved`` transformation is active. This allows the low precision transformation, that goes after the ``AvgPool``, to propagate low precision tensor to the next consumer.
Transformations are run with the following parameters: Transformations are run with the following parameters:
@ -127,8 +127,8 @@ Model display features (here and below):
The transformation is required and includes two tasks: The transformation is required and includes two tasks:
1. Mark operation input ports (create ``Precision`` attribute instance) by provided restrictions: input port index and required precisions. Restrictions are provided as input argument in ``:ref:`ov::pass::low_precision::LowPrecision <doxid-classov_1_1pass_1_1low__precision_1_1_low_precision>``` constructor. 1. Mark operation input ports (create ``Precision`` attribute instance) by provided restrictions: input port index and required precisions. Restrictions are provided as input argument in ``ov::pass::low_precision::LowPrecision`` constructor.
2. Mark precision preserved operations. 2. Mark precision preserved operations.
No attributes are required before the transformation. Changes in the example model after ``MarkupPrecisions`` transformation: No attributes are required before the transformation. Changes in the example model after ``MarkupPrecisions`` transformation:
@ -191,7 +191,7 @@ Result model:
.. image:: _static/images/step2_markup5.svg .. image:: _static/images/step2_markup5.svg
:alt: PropagatePrecisions :alt: PropagatePrecisions
.. note:: .. note::
``AlignQuantizationIntervals`` and ``AlignQuantizationParameters`` transformations are required if the model has quantized concatenation operations. ``AlignQuantizationIntervals`` and ``AlignQuantizationParameters`` transformations are required if the model has quantized concatenation operations.
6. AlignQuantizationIntervals 6. AlignQuantizationIntervals

View File

@ -12,11 +12,11 @@ Plugin API Reference
:maxdepth: 1 :maxdepth: 1
:hidden: :hidden:
../groupov_dev_api ../api/c_cpp_api/group__ov__dev__api
../groupie_transformation_api ../api/c_cpp_api/group__ie__transformation__api
The guides below provides extra API references needed for OpenVINO plugin development: The guides below provides extra API references needed for OpenVINO plugin development:
* `OpenVINO Plugin API <https://docs.openvino.ai/2023.2/groupov_dev_api.html>`__ * `OpenVINO Plugin API <https://docs.openvino.ai/2023.3/api/c_cpp_api/group__ov__dev__api.html>`__
* `OpenVINO Transformation API <https://docs.openvino.ai/2023.2/groupie_transformation_api.html>`__ * `OpenVINO Transformation API <https://docs.openvino.ai/2023.3/api/c_cpp_api/group__ie__transformation__api.html>`__

View File

@ -5,7 +5,7 @@ Overview of Transformations API
.. meta:: .. meta::
:description: Learn how to apply additional model optimizations or transform :description: Learn how to apply additional model optimizations or transform
unsupported subgraphs and operations, using OpenVINO™ Transformations API. unsupported subgraphs and operations, using OpenVINO™ Transformations API.
@ -17,19 +17,19 @@ Overview of Transformations API
openvino_docs_Extensibility_UG_matcher_pass openvino_docs_Extensibility_UG_matcher_pass
openvino_docs_Extensibility_UG_graph_rewrite_pass openvino_docs_Extensibility_UG_graph_rewrite_pass
OpenVINO Transformation mechanism allows to develop transformation passes to modify ``:ref:`ov::Model <doxid-classov_1_1_model>```. You can use this mechanism to apply additional optimizations to the original Model or transform unsupported subgraphs and operations to new operations which are supported by the plugin. OpenVINO Transformation mechanism allows to develop transformation passes to modify ``ov::Model``. You can use this mechanism to apply additional optimizations to the original Model or transform unsupported subgraphs and operations to new operations which are supported by the plugin.
This guide contains all necessary information that you need to start implementing OpenVINO™ transformations. This guide contains all necessary information that you need to start implementing OpenVINO™ transformations.
Working with Model Working with Model
################## ##################
Before the moving to transformation part it is needed to say several words about functions which allow to modify ``:ref:`ov::Model <doxid-classov_1_1_model>```. Before the moving to transformation part it is needed to say several words about functions which allow to modify ``ov::Model``.
This chapter extends the :doc:`model representation guide <openvino_docs_OV_UG_Model_Representation>` and shows an API that allows us to manipulate with ``:ref:`ov::Model <doxid-classov_1_1_model>```. This chapter extends the :doc:`model representation guide <openvino_docs_OV_UG_Model_Representation>` and shows an API that allows us to manipulate with ``ov::Model``.
Working with node input and output ports Working with node input and output ports
++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++
First of all let's talk about ``:ref:`ov::Node <doxid-classov_1_1_node>``` input/output ports. Each OpenVINO™ operation has input and output ports except cases when operation has ``Parameter`` or ``Constant`` type. First of all let's talk about ``ov::Node`` input/output ports. Each OpenVINO™ operation has input and output ports except cases when operation has ``Parameter`` or ``Constant`` type.
Every port belongs to its node, so using a port we can access parent node, get shape and type for particular input/output, get all consumers in case of output port, and get producer node in case of input port. Every port belongs to its node, so using a port we can access parent node, get shape and type for particular input/output, get all consumers in case of output port, and get producer node in case of input port.
With output port we can set inputs for newly created operations. With output port we can set inputs for newly created operations.
@ -49,13 +49,13 @@ Let's start with OpenVINO™ helper functions. The most popular function is ``ov
We will review real replacement case where Negative operation is replaced with Multiply. We will review real replacement case where Negative operation is replaced with Multiply.
.. image:: ./_static/images/ngraph_replace_node.png .. image:: ./_static/images/ngraph_replace_node.png
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp :language: cpp
:fragment: [ov:replace_node] :fragment: [ov:replace_node]
``:ref:`ov::replace_node <doxid-namespaceov_1a75d84ee654edb73fe4fb18936a5dca6d>``` has a constraint that number of output ports for both of ops must be the same; otherwise, it raises an exception. ``ov::replace_node`` has a constraint that number of output ports for both of ops must be the same; otherwise, it raises an exception.
The alternative way to do the same replacement is the following: The alternative way to do the same replacement is the following:
@ -71,7 +71,7 @@ Another transformation example is insertion.
:language: cpp :language: cpp
:fragment: [ov:insert_node] :fragment: [ov:insert_node]
The alternative way to the insert operation is to make a node copy and use ``:ref:`ov::replace_node() <doxid-namespaceov_1a75d84ee654edb73fe4fb18936a5dca6d>```: The alternative way to the insert operation is to make a node copy and use ``ov::replace_node()``:
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp :language: cpp
@ -88,16 +88,16 @@ To eliminate operation, OpenVINO™ has special method that considers all limita
:language: cpp :language: cpp
:fragment: [ov:eliminate_node] :fragment: [ov:eliminate_node]
``:ref:`ov::replace_output_update_name() <doxid-namespaceov_1a75ba2120e573883bd96bb19c887c6a1d>``` in case of successful replacement it automatically preserves friendly name and runtime info. ``ov::replace_output_update_name()`` in case of successful replacement it automatically preserves friendly name and runtime info.
.. _transformations_types: .. _transformations_types:
Transformations types Transformations types
##################### #####################
OpenVINO™ Runtime has three main transformation types: OpenVINO™ Runtime has three main transformation types:
* :doc:`Model pass <openvino_docs_Extensibility_UG_model_pass>` - straightforward way to work with ``:ref:`ov::Model <doxid-classov_1_1_model>``` directly * :doc:`Model pass <openvino_docs_Extensibility_UG_model_pass>` - straightforward way to work with ``ov::Model`` directly
* :doc:`Matcher pass <openvino_docs_Extensibility_UG_matcher_pass>` - pattern-based transformation approach * :doc:`Matcher pass <openvino_docs_Extensibility_UG_matcher_pass>` - pattern-based transformation approach
* :doc:`Graph rewrite pass <openvino_docs_Extensibility_UG_graph_rewrite_pass>` - container for matcher passes needed for efficient execution * :doc:`Graph rewrite pass <openvino_docs_Extensibility_UG_graph_rewrite_pass>` - container for matcher passes needed for efficient execution
@ -108,12 +108,12 @@ Transformation conditional compilation
Transformation library has two internal macros to support conditional compilation feature. Transformation library has two internal macros to support conditional compilation feature.
* ``:ref:`MATCHER_SCOPE(region) <doxid-conditional__compilation_2include_2openvino_2cc_2pass_2itt_8hpp_1a3d1377542bcf3e305c33a1b683cc77df>``` - allows to disable the MatcherPass if matcher isn't used. The region name should be unique. This macro creates a local variable ``matcher_name`` which you should use as a matcher name. * ``MATCHER_SCOPE(region)`` - allows to disable the MatcherPass if matcher isn't used. The region name should be unique. This macro creates a local variable ``matcher_name`` which you should use as a matcher name.
* ``:ref:`RUN_ON_MODEL_SCOPE(region) <doxid-conditional__compilation_2include_2openvino_2cc_2pass_2itt_8hpp_1ab308561b849d47b9c820506ec73c4a30>``` - allows to disable run_on_model pass if it isn't used. The region name should be unique. * ``RUN_ON_MODEL_SCOPE(region)`` - allows to disable run_on_model pass if it isn't used. The region name should be unique.
.. _transformation_writing_essentials: .. _transformation_writing_essentials:
Transformation writing essentials Transformation writing essentials
################################# #################################
When developing a transformation, you need to follow these transformation rules: When developing a transformation, you need to follow these transformation rules:
@ -121,7 +121,7 @@ When developing a transformation, you need to follow these transformation rules:
1. Friendly Names 1. Friendly Names
+++++++++++++++++ +++++++++++++++++
Each ``:ref:`ov::Node <doxid-classov_1_1_node>``` has an unique name and a friendly name. In transformations we care only about friendly name because it represents the name from the model. Each ``ov::Node`` has an unique name and a friendly name. In transformations we care only about friendly name because it represents the name from the model.
To avoid losing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below. To avoid losing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below.
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
@ -133,8 +133,8 @@ In more advanced cases, when replaced operation has several outputs and we add a
2. Runtime Info 2. Runtime Info
+++++++++++++++ +++++++++++++++
Runtime info is a map ``std::map<std::string, :ref:`ov::Any <doxid-classov_1_1_any>`>`` located inside ``:ref:`ov::Node <doxid-classov_1_1_node>``` class. It represents additional attributes in ``:ref:`ov::Node <doxid-classov_1_1_node>```. Runtime info is a map ``std::map<std::string, ov::Any>`` located inside ``ov::Node`` class. It represents additional attributes in ``ov::Node>``.
These attributes can be set by users or by plugins and when executing transformation that changes ``:ref:`ov::Model <doxid-classov_1_1_model>``` we need to preserve these attributes as they will not be automatically propagated. These attributes can be set by users or by plugins and when executing transformation that changes ``ov::Model`` we need to preserve these attributes as they will not be automatically propagated.
In most cases, transformations have the following types: 1:1 (replace node with another node), 1:N (replace node with a sub-graph), N:1 (fuse sub-graph into a single node), N:M (any other transformation). In most cases, transformations have the following types: 1:1 (replace node with another node), 1:N (replace node with a sub-graph), N:1 (fuse sub-graph into a single node), N:M (any other transformation).
Currently, there is no mechanism that automatically detects transformation types, so we need to propagate this runtime information manually. See the examples below. Currently, there is no mechanism that automatically detects transformation types, so we need to propagate this runtime information manually. See the examples below.
@ -143,21 +143,21 @@ Currently, there is no mechanism that automatically detects transformation types
:language: cpp :language: cpp
:fragment: [ov:copy_runtime_info] :fragment: [ov:copy_runtime_info]
When transformation has multiple fusions or decompositions, ``:ref:`ov::copy_runtime_info <doxid-namespaceov_1a3bb5969a95703b4b4fd77f6f58837207>``` must be called multiple times for each case. When transformation has multiple fusions or decompositions, ``ov::copy_runtime_info`` must be called multiple times for each case.
.. note:: ``copy_runtime_info`` removes ``rt_info`` from destination nodes. If you want to keep it, you need to specify them in source nodes like this: ``copy_runtime_info({a, b, c}, {a, b})`` .. note:: ``copy_runtime_info`` removes ``rt_info`` from destination nodes. If you want to keep it, you need to specify them in source nodes like this: ``copy_runtime_info({a, b, c}, {a, b})``
3. Constant Folding 3. Constant Folding
+++++++++++++++++++ +++++++++++++++++++
If your transformation inserts constant sub-graphs that need to be folded, do not forget to use ``:ref:`ov::pass::ConstantFolding() <doxid-classov_1_1pass_1_1_constant_folding>``` after your transformation or call constant folding directly for operation. If your transformation inserts constant sub-graphs that need to be folded, do not forget to use ``ov::pass::ConstantFolding()`` after your transformation or call constant folding directly for operation.
The example below shows how constant subgraph can be constructed. The example below shows how constant subgraph can be constructed.
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp :language: cpp
:fragment: [ov:constant_subgraph] :fragment: [ov:constant_subgraph]
Manual constant folding is more preferable than ``:ref:`ov::pass::ConstantFolding() <doxid-classov_1_1pass_1_1_constant_folding>``` because it is much faster. Manual constant folding is more preferable than ``ov::pass::ConstantFolding()`` because it is much faster.
Below you can find an example of manual constant folding: Below you can find an example of manual constant folding:
@ -167,28 +167,28 @@ Below you can find an example of manual constant folding:
.. _common_mistakes: .. _common_mistakes:
Common mistakes in transformations Common mistakes in transformations
################################## ##################################
In transformation development process: In transformation development process:
* Do not use deprecated OpenVINO™ API. Deprecated methods has the ``OPENVINO_DEPRECATED`` macros in its definition. * Do not use deprecated OpenVINO™ API. Deprecated methods has the ``OPENVINO_DEPRECATED`` macros in its definition.
* Do not pass ``shared_ptr<Node>`` as an input for other node if type of node is unknown or it has multiple outputs. Use explicit output port. * Do not pass ``shared_ptr<Node>`` as an input for other node if type of node is unknown or it has multiple outputs. Use explicit output port.
* If you replace node with another node that produces different shape, remember that new shape will not be propagated until the first ``validate_nodes_and_infer_types`` call for ``:ref:`ov::Model <doxid-classov_1_1_model>```. If you are using ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>```, it will automatically call this method after each transformation execution. * If you replace node with another node that produces different shape, remember that new shape will not be propagated until the first ``validate_nodes_and_infer_types`` call for ``ov::Model``. If you are using ``ov::pass::Manager``, it will automatically call this method after each transformation execution.
* Do not forget to call the ``:ref:`ov::pass::ConstantFolding <doxid-classov_1_1pass_1_1_constant_folding>``` pass if your transformation creates constant subgraphs. * Do not forget to call the ``ov::pass::ConstantFolding`` pass if your transformation creates constant subgraphs.
* Use latest OpSet if you are not developing downgrade transformation pass. * Use latest OpSet if you are not developing downgrade transformation pass.
* When developing a callback for ``:ref:`ov::pass::MatcherPass <doxid-classov_1_1pass_1_1_matcher_pass>```, do not change nodes that come after the root node in topological order. * When developing a callback for ``ov::pass::MatcherPass``, do not change nodes that come after the root node in topological order.
.. _using_pass_manager: .. _using_pass_manager:
Using pass manager Using pass manager
################## ##################
``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` is a container class that can store the list of transformations and execute them. The main idea of this class is to have high-level representation for grouped list of transformations. ``ov::pass::Manager`` is a container class that can store the list of transformations and execute them. The main idea of this class is to have high-level representation for grouped list of transformations.
It can register and apply any `transformation pass <#transformations_types>`__ on model. It can register and apply any `transformation pass <#transformations_types>`__ on model.
In addition, ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` has extended debug capabilities (find more information in the `how to debug transformations <#how_to_debug_transformations>`__ section). In addition, ``ov::pass::Manager`` has extended debug capabilities (find more information in the `how to debug transformations <#how_to_debug_transformations>`__ section).
The example below shows basic usage of ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` The example below shows basic usage of ``ov::pass::Manager``
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp .. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp :language: cpp
@ -199,16 +199,16 @@ Another example shows how multiple matcher passes can be united into single Grap
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp .. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp :language: cpp
:fragment: [matcher_pass:manager2] :fragment: [matcher_pass:manager2]
.. _how_to_debug_transformations:
How to debug transformations .. _how_to_debug_transformations:
How to debug transformations
############################ ############################
If you are using ``ngraph::pass::Manager`` to run sequence of transformations, you can get additional debug capabilities by using the following environment variables: If you are using ``ngraph::pass::Manager`` to run sequence of transformations, you can get additional debug capabilities by using the following environment variables:
.. code-block:: cpp .. code-block:: cpp
OV_PROFILE_PASS_ENABLE=1 - enables performance measurement for each transformation and prints execution status OV_PROFILE_PASS_ENABLE=1 - enables performance measurement for each transformation and prints execution status
OV_ENABLE_VISUALIZE_TRACING=1 - enables visualization after each transformation. By default, it saves dot and svg files. OV_ENABLE_VISUALIZE_TRACING=1 - enables visualization after each transformation. By default, it saves dot and svg files.
@ -218,6 +218,6 @@ If you are using ``ngraph::pass::Manager`` to run sequence of transformations, y
See Also See Also
######## ########
* :doc:`OpenVINO™ Model Representation <openvino_docs_OV_UG_Model_Representation>` * :doc:`OpenVINO™ Model Representation <openvino_docs_OV_UG_Model_Representation>`
* :doc:`OpenVINO™ Extensions <openvino_docs_Extensibility_UG_Intro>` * :doc:`OpenVINO™ Extensions <openvino_docs_Extensibility_UG_Intro>`

View File

@ -5,11 +5,11 @@ OpenVINO Graph Rewrite Pass
.. meta:: .. meta::
:description: Get to know how Graph Rewrite handles running multiple matcher passes on :description: Get to know how Graph Rewrite handles running multiple matcher passes on
ov::Model in a single graph traversal. ov::Model in a single graph traversal.
``:ref:`ov::pass::GraphRewrite <doxid-classov_1_1pass_1_1_graph_rewrite>``` serves for running multiple matcher passes on ``:ref:`ov::Model <doxid-classov_1_1_model>``` in a single graph traversal. ``ov::pass::GraphRewrite`` serves for running multiple matcher passes on ``ov::Model`` in a single graph traversal.
Example: Example:
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp .. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
@ -18,13 +18,13 @@ Example:
In addition, GraphRewrite handles nodes that were registered by MatcherPasses during their execution. This nodes will be added to the beginning of the sequence with nodes for pattern matching. In addition, GraphRewrite handles nodes that were registered by MatcherPasses during their execution. This nodes will be added to the beginning of the sequence with nodes for pattern matching.
.. note:: .. note::
When using ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` temporary GraphRewrite is used to execute single MatcherPass. When using ``ov::pass::Manager`` temporary GraphRewrite is used to execute single MatcherPass.
GraphRewrite has two algorithms for MatcherPasses execution. First algorithm is straightforward. It applies each MatcherPass in registration order to current node. GraphRewrite has two algorithms for MatcherPasses execution. First algorithm is straightforward. It applies each MatcherPass in registration order to current node.
.. image:: ./_static/images/graph_rewrite_execution.png .. image:: ./_static/images/graph_rewrite_execution.png
But it is not really efficient when you have a lot of registered passes. So first of all GraphRewrite checks that all MatcherPass patterns has type-based root node (it means that type of this node is not hidden into predicate). But it is not really efficient when you have a lot of registered passes. So first of all GraphRewrite checks that all MatcherPass patterns has type-based root node (it means that type of this node is not hidden into predicate).
And then creates map from registered MatcherPasses. That helps to avoid additional cost of applying each MatcherPass for each node. And then creates map from registered MatcherPasses. That helps to avoid additional cost of applying each MatcherPass for each node.

View File

@ -5,11 +5,11 @@ OpenVINO Matcher Pass
.. meta:: .. meta::
:description: Learn how to create a pattern, implement a callback, register :description: Learn how to create a pattern, implement a callback, register
the pattern and Matcher to execute MatcherPass transformation the pattern and Matcher to execute MatcherPass transformation
on a model. on a model.
``:ref:`ov::pass::MatcherPass <doxid-classov_1_1pass_1_1_matcher_pass>``` is used for pattern-based transformations. ``ov::pass::MatcherPass`` is used for pattern-based transformations.
Template for MatcherPass transformation class Template for MatcherPass transformation class
@ -22,7 +22,7 @@ Template for MatcherPass transformation class
:fragment: [graph_rewrite:template_transformation_cpp] :fragment: [graph_rewrite:template_transformation_cpp]
To use ``:ref:`ov::pass::MatcherPass <doxid-classov_1_1pass_1_1_matcher_pass>```, you need to complete these steps: To use ``ov::pass::MatcherPass``, you need to complete these steps:
1. Create a pattern 1. Create a pattern
2. Implement a callback 2. Implement a callback
@ -34,10 +34,10 @@ So let's go through each of these steps.
Create a pattern Create a pattern
################ ################
Pattern is a single root ``:ref:`ov::Model <doxid-classov_1_1_model>```. But the only difference is that you do not need to create a model object, you just need to create and connect opset or special pattern operations. Pattern is a single root ``ov::Model``. But the only difference is that you do not need to create a model object, you just need to create and connect opset or special pattern operations.
Then you need to take the last created operation and put it as a root of the pattern. This root node will be used as a root node in pattern matching. Then you need to take the last created operation and put it as a root of the pattern. This root node will be used as a root node in pattern matching.
.. note:: .. note::
Any nodes in a pattern that have no consumers and are not registered as root will not be used in pattern matching. Any nodes in a pattern that have no consumers and are not registered as root will not be used in pattern matching.
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
@ -60,13 +60,13 @@ Callback is an action applied to every pattern entrance. In general, callback is
The example above shows the callback structure and how Matcher can be used for accessing nodes detected by pattern. The example above shows the callback structure and how Matcher can be used for accessing nodes detected by pattern.
Callback return value is ``true`` if root node was replaced and another pattern cannot be applied to the same root node; otherwise, it is ``false``. Callback return value is ``true`` if root node was replaced and another pattern cannot be applied to the same root node; otherwise, it is ``false``.
.. note:: .. note::
It is not recommended to manipulate with nodes that are under root node. This may affect GraphRewrite execution as it is expected that all nodes that come after root node in topological order are valid and can be used in pattern matching. It is not recommended to manipulate with nodes that are under root node. This may affect GraphRewrite execution as it is expected that all nodes that come after root node in topological order are valid and can be used in pattern matching.
MatcherPass also provides functionality that allows reporting of the newly created nodes that can be used in additional pattern matching. MatcherPass also provides functionality that allows reporting of the newly created nodes that can be used in additional pattern matching.
If MatcherPass was registered in ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` or ``:ref:`ov::pass::GraphRewrite <doxid-classov_1_1pass_1_1_graph_rewrite>```, these registered nodes will be added for additional pattern matching. If MatcherPass was registered in ``ov::pass::Manager`` or ``ov::pass::GraphRewrite``, these registered nodes will be added for additional pattern matching.
That means that matcher passes registered in ``:ref:`ov::pass::GraphRewrite <doxid-classov_1_1pass_1_1_graph_rewrite>``` will be applied to these nodes. That means that matcher passes registered in ``ov::pass::GraphRewrite`` will be applied to these nodes.
The example below shows how single MatcherPass can fuse sequence of operations using the ``register_new_node`` method. The example below shows how single MatcherPass can fuse sequence of operations using the ``register_new_node`` method.
@ -74,7 +74,7 @@ The example below shows how single MatcherPass can fuse sequence of operations u
:language: cpp :language: cpp
:fragment: [matcher_pass:relu_fusion] :fragment: [matcher_pass:relu_fusion]
.. note:: .. note::
If you register multiple nodes, please add them in topological order. We do not topologically sort these nodes as it is a time-consuming operation. If you register multiple nodes, please add them in topological order. We do not topologically sort these nodes as it is a time-consuming operation.
Register pattern and Matcher Register pattern and Matcher
@ -82,12 +82,12 @@ Register pattern and Matcher
The last step is to register Matcher and callback inside the MatcherPass pass. To do this, call the ``register_matcher`` method. The last step is to register Matcher and callback inside the MatcherPass pass. To do this, call the ``register_matcher`` method.
.. note:: .. note::
Only one matcher can be registered for a single MatcherPass class. Only one matcher can be registered for a single MatcherPass class.
.. code-block:: cpp .. code-block:: cpp
// Register matcher and callback // Register matcher and callback
register_matcher(m, callback); register_matcher(m, callback);
@ -103,13 +103,13 @@ MatcherPass has multiple ways to be executed:
:language: cpp :language: cpp
:fragment: [matcher_pass:run_on_node] :fragment: [matcher_pass:run_on_node]
* Run on ``:ref:`ov::Model <doxid-classov_1_1_model>``` using GraphRewrite - this approach gives ability to run MatcherPass on whole ``:ref:`ov::Model <doxid-classov_1_1_model>```. Moreover, multiple MatcherPass transformation can be registered in a single GraphRewite to be executed in a single graph traversal. * Run on ``ov::Model`` using GraphRewrite - this approach gives ability to run MatcherPass on whole ``ov::Model``. Moreover, multiple MatcherPass transformation can be registered in a single GraphRewite to be executed in a single graph traversal.
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp .. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp :language: cpp
:fragment: [matcher_pass:graph_rewrite] :fragment: [matcher_pass:graph_rewrite]
* Run on ``:ref:`ov::Model <doxid-classov_1_1_model>``` using ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` - this approach helps you to register MatcherPass for execution on ``:ref:`ov::Model <doxid-classov_1_1_model>``` as another transformation types. * Run on ``ov::Model`` using ``ov::pass::Manager`` - this approach helps you to register MatcherPass for execution on ``ov::Model`` as another transformation types.
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp .. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp :language: cpp
@ -125,8 +125,8 @@ And for these cases OpenVINO™ provides additional helpers to construct pattern
There are two main helpers: There are two main helpers:
1. ``:ref:`ov::pass::pattern::any_input <doxid-namespaceov_1_1pass_1_1pattern_1a8ed84c3eed4610f117ee10d86d500e02>``` - helps to express inputs if their types are undefined. 1. ``ov::pass::pattern::any_input`` - helps to express inputs if their types are undefined.
2. ``:ref:`ov::pass::pattern::wrap_type <doxid-namespaceov_1_1pass_1_1pattern_1adfcd6031c95d7bace5f084e2aa105af8>`<T>`` - helps to express nodes of pattern without specifying node attributes. 2. ``ov::pass::pattern::wrap_type <T>`` - helps to express nodes of pattern without specifying node attributes.
Let's go through the example to have better understanding of how it works: Let's go through the example to have better understanding of how it works:

View File

@ -5,25 +5,25 @@ OpenVINO Model Pass
.. meta:: .. meta::
:description: Learn how to use Model Pass transformation class to take entire :description: Learn how to use Model Pass transformation class to take entire
ov::Model as input and process it. ov::Model as input and process it.
``:ref:`ov::pass::ModelPass <doxid-classov_1_1pass_1_1_model_pass>``` is used for transformations that take entire ``:ref:`ov::Model <doxid-classov_1_1_model>``` as an input and process it. ``ov::pass::ModelPass`` is used for transformations that take entire ``ov::Model`` as an input and process it.
Template for ModelPass transformation class Template for ModelPass transformation class
.. doxygensnippet:: docs/snippets/template_model_transformation.hpp .. doxygensnippet:: docs/snippets/template_model_transformation.hpp
:language: cpp :language: cpp
:fragment: [model_pass:template_transformation_hpp] :fragment: [model_pass:template_transformation_hpp]
.. doxygensnippet:: docs/snippets/template_model_transformation.cpp .. doxygensnippet:: docs/snippets/template_model_transformation.cpp
:language: cpp :language: cpp
:fragment: [model_pass:template_transformation_cpp] :fragment: [model_pass:template_transformation_cpp]
Using ``:ref:`ov::pass::ModelPass <doxid-classov_1_1pass_1_1_model_pass>```, you need to override the ``run_on_model`` method where you will write the transformation code. Using ``ov::pass::ModelPass``, you need to override the ``run_on_model`` method where you will write the transformation code.
Return value is ``true`` if the original model has changed during transformation (new operation was added, or operations replacement was made, or node attributes were changed); otherwise, it is ``false``. Return value is ``true`` if the original model has changed during transformation (new operation was added, or operations replacement was made, or node attributes were changed); otherwise, it is ``false``.
Also ``:ref:`ov::pass::ModelPass <doxid-classov_1_1pass_1_1_model_pass>``` based transformations can be executed via ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>```. Also ``ov::pass::ModelPass`` based transformations can be executed via ``ov::pass::Manager``.
See Also See Also
######## ########

View File

@ -19,6 +19,7 @@ Operation Specifications
Asin-1 <openvino_docs_ops_arithmetic_Asin_1> Asin-1 <openvino_docs_ops_arithmetic_Asin_1>
Asinh-3 <openvino_docs_ops_arithmetic_Asinh_3> Asinh-3 <openvino_docs_ops_arithmetic_Asinh_3>
Assign-3 <openvino_docs_ops_infrastructure_Assign_3> Assign-3 <openvino_docs_ops_infrastructure_Assign_3>
Assign-6 <openvino_docs_ops_infrastructure_Assign_6>
Atan-1 <openvino_docs_ops_arithmetic_Atan_1> Atan-1 <openvino_docs_ops_arithmetic_Atan_1>
Atanh-3 <openvino_docs_ops_arithmetic_Atanh_3> Atanh-3 <openvino_docs_ops_arithmetic_Atanh_3>
AvgPool-1 <openvino_docs_ops_pooling_AvgPool_1> AvgPool-1 <openvino_docs_ops_pooling_AvgPool_1>
@ -162,6 +163,7 @@ Operation Specifications
Range-4 <openvino_docs_ops_generation_Range_4> Range-4 <openvino_docs_ops_generation_Range_4>
RDFT-9<openvino_docs_ops_signals_RDFT_9> RDFT-9<openvino_docs_ops_signals_RDFT_9>
ReadValue-3 <openvino_docs_ops_infrastructure_ReadValue_3> ReadValue-3 <openvino_docs_ops_infrastructure_ReadValue_3>
ReadValue-6 <openvino_docs_ops_infrastructure_ReadValue_6>
ReLU-1 <openvino_docs_ops_activation_ReLU_1> ReLU-1 <openvino_docs_ops_activation_ReLU_1>
ReduceL1-4 <openvino_docs_ops_reduction_ReduceL1_4> ReduceL1-4 <openvino_docs_ops_reduction_ReduceL1_4>
ReduceL2-4 <openvino_docs_ops_reduction_ReduceL2_4> ReduceL2-4 <openvino_docs_ops_reduction_ReduceL2_4>

View File

@ -5,7 +5,7 @@ Swish
.. meta:: .. meta::
:description: Learn about Swish-4 - an element-wise, activation operation, which :description: Learn about Swish-4 - an element-wise, activation operation, which
can be performed on a single tensor in OpenVINO. can be performed on a single tensor in OpenVINO.
**Versioned name**: *Swish-4* **Versioned name**: *Swish-4*
@ -55,7 +55,7 @@ Example: Second input ``beta`` provided
<dim>256</dim> <dim>256</dim>
<dim>56</dim> <dim>56</dim>
</port> </port>
<port id="1"> < !-- beta value: 2.0 --> <port id="1"> <!-- beta value: 2.0 -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ CumSum
.. meta:: .. meta::
:description: Learn about CumSum-3 - an element-wise, arithmetic operation, which :description: Learn about CumSum-3 - an element-wise, arithmetic operation, which
can be performed on a single tensor in OpenVINO. can be performed on a single tensor in OpenVINO.
**Versioned name**: *CumSum-3* **Versioned name**: *CumSum-3*
@ -24,7 +24,7 @@ To perform the summation in the opposite direction of the axis, set reverse attr
* **Description**: If the attribute is set to ``true``, then exclusive sums are returned, the ``j-th`` element is not included in the ``j-th`` sum. Otherwise, the inclusive sum of the first ``j`` elements for the ``j-th`` element is calculated. * **Description**: If the attribute is set to ``true``, then exclusive sums are returned, the ``j-th`` element is not included in the ``j-th`` sum. Otherwise, the inclusive sum of the first ``j`` elements for the ``j-th`` element is calculated.
* **Range of values**: * **Range of values**:
* ``false`` - include the top element * ``false`` - include the top element
* ``true`` - do not include the top element * ``true`` - do not include the top element
* **Type**: ``boolean`` * **Type**: ``boolean``
@ -35,7 +35,7 @@ To perform the summation in the opposite direction of the axis, set reverse attr
* **Description**: If set to ``true`` will perform the sums in reverse direction. * **Description**: If set to ``true`` will perform the sums in reverse direction.
* **Range of values**: * **Range of values**:
* ``false`` - do not perform sums in reverse direction * ``false`` - do not perform sums in reverse direction
* ``true`` - perform sums in reverse direction * ``true`` - perform sums in reverse direction
* **Type**: ``boolean`` * **Type**: ``boolean``
@ -63,16 +63,16 @@ To perform the summation in the opposite direction of the axis, set reverse attr
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... type="CumSum" exclusive="0" reverse="0"> <layer ... type="CumSum" exclusive="0" reverse="0">
<input> <input>
<port id="0"> < !-- input value is: [1., 2., 3., 4., 5.] --> <port id="0"> <!-- input value is: [1., 2., 3., 4., 5.] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
<port id="1"/> < !-- axis value is: 0 --> <port id="1"/> <!-- axis value is: 0 -->
</input> </input>
<output> <output>
<port id="2"> < !-- output value is: [1., 3., 6., 10., 15.] --> <port id="2"> <!-- output value is: [1., 3., 6., 10., 15.] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
</output> </output>
@ -82,16 +82,16 @@ To perform the summation in the opposite direction of the axis, set reverse attr
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... type="CumSum" exclusive="1" reverse="0"> <layer ... type="CumSum" exclusive="1" reverse="0">
<input> <input>
<port id="0"> < !-- input value is: [1., 2., 3., 4., 5.] --> <port id="0"> <!-- input value is: [1., 2., 3., 4., 5.] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
<port id="1"/> < !-- axis value is: 0 --> <port id="1"/> <!-- axis value is: 0 -->
</input> </input>
<output> <output>
<port id="2"> < !-- output value is: [0., 1., 3., 6., 10.] --> <port id="2"> <!-- output value is: [0., 1., 3., 6., 10.] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
</output> </output>
@ -101,16 +101,16 @@ To perform the summation in the opposite direction of the axis, set reverse attr
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... type="CumSum" exclusive="0" reverse="1"> <layer ... type="CumSum" exclusive="0" reverse="1">
<input> <input>
<port id="0"> < !-- input value is: [1., 2., 3., 4., 5.] --> <port id="0"> <!-- input value is: [1., 2., 3., 4., 5.] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
<port id="1"/> < !-- axis value is: 0 --> <port id="1"/> <!-- axis value is: 0 -->
</input> </input>
<output> <output>
<port id="2"> < !-- output value is: [15., 14., 12., 9., 5.] --> <port id="2"> <!-- output value is: [15., 14., 12., 9., 5.] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
</output> </output>
@ -120,7 +120,7 @@ To perform the summation in the opposite direction of the axis, set reverse attr
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... type="CumSum" exclusive="1" reverse="1"> <layer ... type="CumSum" exclusive="1" reverse="1">
<input> <input>
<port id="0"> < -- input value is: [1., 2., 3., 4., 5.] --> <port id="0"> < -- input value is: [1., 2., 3., 4., 5.] -->

View File

@ -5,7 +5,7 @@ Sqrt
.. meta:: .. meta::
:description: Learn about Sqrt-1 - an element-wise, arithmetic operation, which :description: Learn about Sqrt-1 - an element-wise, arithmetic operation, which
can be performed on a single tensor in OpenVINO. can be performed on a single tensor in OpenVINO.
**Versioned name**: *Sqrt-1* **Versioned name**: *Sqrt-1*
@ -48,12 +48,12 @@ Sqrt
<layer ... type="Sqrt"> <layer ... type="Sqrt">
<input> <input>
<port id="0"> <port id="0">
<dim>4</dim> < !-- float input values: [4.0, 7.0, 9.0, 10.0] --> <dim>4</dim> <!-- float input values: [4.0, 7.0, 9.0, 10.0] -->
</port> </port>
</input> </input>
<output> <output>
<port id="1"> <port id="1">
<dim>4</dim> < !-- float output values: [2.0, 2.6457512, 3.0, 3.1622777] --> <dim>4</dim> <!-- float output values: [2.0, 2.6457512, 3.0, 3.1622777] -->
</port> </port>
</output> </output>
</layer> </layer>
@ -66,12 +66,12 @@ Sqrt
<layer ... type="Sqrt"> <layer ... type="Sqrt">
<input> <input>
<port id="0"> <port id="0">
<dim>4</dim> < !-- int input values: [4, 7, 9, 10] --> <dim>4</dim> <!-- int input values: [4, 7, 9, 10] -->
</port> </port>
</input> </input>
<output> <output>
<port id="1"> <port id="1">
<dim>4</dim> < !-- int output values: [2, 3, 3, 3] --> <dim>4</dim> <!-- int output values: [2, 3, 3, 3] -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ IsFinite
.. meta:: .. meta::
:description: Learn about IsFinite-10 - an element-wise, comparison operation, which :description: Learn about IsFinite-10 - an element-wise, comparison operation, which
can be performed on a single tensor in OpenVINO. can be performed on a single tensor in OpenVINO.
**Versioned name**: *IsFinite-10* **Versioned name**: *IsFinite-10*
@ -64,12 +64,12 @@ IsFinite
<layer ... type="IsFinite"> <layer ... type="IsFinite">
<input> <input>
<port id="0" precision="FP32"> <port id="0" precision="FP32">
<dim>4</dim> < !-- Input value is: [NaN, 2.1, 3.7, Inf] --> <dim>4</dim> <!-- Input value is: [NaN, 2.1, 3.7, Inf] -->
</port> </port>
</input> </input>
<output> <output>
<port id="1" precision="BOOL"> <port id="1" precision="BOOL">
<dim>4</dim> < !-- Output value is: [False, True, True, False] --> <dim>4</dim> <!-- Output value is: [False, True, True, False] -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ Select
.. meta:: .. meta::
:description: Learn about Select-1 - an element-wise, condition operation, which :description: Learn about Select-1 - an element-wise, condition operation, which
can be performed on three given tensors in OpenVINO. can be performed on three given tensors in OpenVINO.
**Versioned name**: *Select-1* **Versioned name**: *Select-1*
@ -58,21 +58,21 @@ Select
<layer ... type="Select"> <layer ... type="Select">
<input> <input>
<port id="0"> < !-- cond value is: [[false, false], [true, false], [true, true]] --> <port id="0"> <!-- cond value is: [[false, false], [true, false], [true, true]] -->
<dim>3</dim> <dim>3</dim>
<dim>2</dim> <dim>2</dim>
</port> </port>
<port id="1"> < !-- then value is: [[-1, 0], [1, 2], [3, 4]] --> <port id="1"> <!-- then value is: [[-1, 0], [1, 2], [3, 4]] -->
<dim>3</dim> <dim>3</dim>
<dim>2</dim> <dim>2</dim>
</port> </port>
<port id="2"> < !-- else value is: [[11, 10], [9, 8], [7, 6]] --> <port id="2"> <!-- else value is: [[11, 10], [9, 8], [7, 6]] -->
<dim>3</dim> <dim>3</dim>
<dim>2</dim> <dim>2</dim>
</port> </port>
</input> </input>
<output> <output>
<port id="1"> < !-- output value is: [[11, 10], [1, 8], [3, 4]] --> <port id="1"> <!-- output value is: [[11, 10], [1, 8], [3, 4]] -->
<dim>3</dim> <dim>3</dim>
<dim>2</dim> <dim>2</dim>
</port> </port>

View File

@ -5,7 +5,7 @@ ConvolutionBackpropData
.. meta:: .. meta::
:description: Learn about ConvolutionBackpropData-1 - a 1D, 2D or 3D convolution operation, which :description: Learn about ConvolutionBackpropData-1 - a 1D, 2D or 3D convolution operation, which
can be performed on input and kernel tensors in OpenVINO. can be performed on input and kernel tensors in OpenVINO.
**Versioned name**: *ConvolutionBackpropData-1* **Versioned name**: *ConvolutionBackpropData-1*
@ -24,11 +24,11 @@ When output shape is specified as an input tensor ``output_shape`` then it speci
.. code-block:: xml .. code-block:: xml
:force: :force:
if auto_pads != None: if auto_pads != None:
pads_begin[i] = 0 pads_begin[i] = 0
pads_end[i] = 0 pads_end[i] = 0
Y_i = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - pads_begin[i] - pads_end[i] + output_padding[i] Y_i = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - pads_begin[i] - pads_end[i] + output_padding[i]
where ``K_i`` filter kernel dimension along spatial axis ``i``. where ``K_i`` filter kernel dimension along spatial axis ``i``.
@ -37,7 +37,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
.. code-block:: xml .. code-block:: xml
:force: :force:
total_padding[i] = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - output_shape[i] + output_padding[i] total_padding[i] = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - output_shape[i] + output_padding[i]
if auto_pads != SAME_UPPER: if auto_pads != SAME_UPPER:
pads_begin[i] = total_padding[i] // 2 pads_begin[i] = total_padding[i] // 2
@ -81,7 +81,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
* *auto_pad* * *auto_pad*
* **Description**: *auto_pad* has the same definition as *auto_pad* for a regular Convolution but applied in the backward way, for the output tensor. * **Description**: *auto_pad* has the same definition as *auto_pad* for a regular Convolution but applied in the backward way, for the output tensor.
* *explicit*: use explicit padding values from ``pads_begin`` and ``pads_end``. * *explicit*: use explicit padding values from ``pads_begin`` and ``pads_end``.
* *same_upper* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end. * *same_upper* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end.
* *same_lower* the input is padded to match the output size. In case of odd padding value an extra padding is added at the beginning. * *same_lower* the input is padded to match the output size. In case of odd padding value an extra padding is added at the beginning.
@ -105,7 +105,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
* **2**: Convolution kernel tensor of type *T1* and rank 3, 4 or 5. Layout is ``[C_INPUT, C_OUTPUT, Z, Y, X]`` (number of input channels, number of output channels, spatial axes Z, Y, X). Spatial size of the kernel is derived from the shape of this input and aren't specified by any attribute. **Required.** * **2**: Convolution kernel tensor of type *T1* and rank 3, 4 or 5. Layout is ``[C_INPUT, C_OUTPUT, Z, Y, X]`` (number of input channels, number of output channels, spatial axes Z, Y, X). Spatial size of the kernel is derived from the shape of this input and aren't specified by any attribute. **Required.**
* **3**: ``output_shape`` is 1D tensor of type *T2* that specifies spatial shape of the output. If specified, *padding amount* is deduced from relation of input and output spatial shapes according to formulas in the description. If not specified, *output shape* is calculated based on the ``pads_begin`` and ``pads_end`` or completely according to ``auto_pad``. **Optional.** * **3**: ``output_shape`` is 1D tensor of type *T2* that specifies spatial shape of the output. If specified, *padding amount* is deduced from relation of input and output spatial shapes according to formulas in the description. If not specified, *output shape* is calculated based on the ``pads_begin`` and ``pads_end`` or completely according to ``auto_pad``. **Optional.**
* **Note**: Type of the convolution (1D, 2D or 3D) is derived from the rank of the input tensors and not specified by any attribute: * **Note**: Type of the convolution (1D, 2D or 3D) is derived from the rank of the input tensors and not specified by any attribute:
* 1D convolution (input tensors rank 3) means that there is only one spatial axis X, * 1D convolution (input tensors rank 3) means that there is only one spatial axis X,
* 2D convolution (input tensors rank 4) means that there are two spatial axes Y, X, * 2D convolution (input tensors rank 4) means that there are two spatial axes Y, X,
* 3D convolution (input tensors rank 5) means that there are three spatial axes Z, Y, X. * 3D convolution (input tensors rank 5) means that there are three spatial axes Z, Y, X.
@ -125,7 +125,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer id="5" name="upsampling_node" type="ConvolutionBackpropData"> <layer id="5" name="upsampling_node" type="ConvolutionBackpropData">
<data dilations="1,1" pads_begin="1,1" pads_end="1,1" strides="2,2" output_padding="0,0" auto_pad="explicit"/> <data dilations="1,1" pads_begin="1,1" pads_end="1,1" strides="2,2" output_padding="0,0" auto_pad="explicit"/>
<input> <input>
@ -156,7 +156,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer id="5" name="upsampling_node" type="ConvolutionBackpropData"> <layer id="5" name="upsampling_node" type="ConvolutionBackpropData">
<data dilations="1,1" pads_begin="0,0" pads_end="0,0" strides="3,3" output_padding="2,2" auto_pad="explicit"/> <data dilations="1,1" pads_begin="0,0" pads_end="0,0" strides="3,3" output_padding="2,2" auto_pad="explicit"/>
<input> <input>
@ -187,7 +187,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer id="5" name="upsampling_node" type="ConvolutionBackpropData"> <layer id="5" name="upsampling_node" type="ConvolutionBackpropData">
<data dilations="1,1" pads_begin="1,1" pads_end="1,1" strides="1,1" output_padding="0,0" auto_pad="valid"/> <data dilations="1,1" pads_begin="1,1" pads_end="1,1" strides="1,1" output_padding="0,0" auto_pad="valid"/>
<input> <input>
@ -204,7 +204,7 @@ If ``output_shape`` is specified, ``pads_begin`` and ``pads_end`` are ignored, a
<dim>3</dim> <dim>3</dim>
</port> </port>
<port id="2"> <port id="2">
<dim>2</dim> < !-- output_shape value is: [450, 450]--> <dim>2</dim> <!-- output_shape value is: [450, 450]-->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ PriorBoxClustered
.. meta:: .. meta::
:description: Learn about PriorBoxClustered-1 - an object detection operation, :description: Learn about PriorBoxClustered-1 - an object detection operation,
which can be performed on two 1D input tensors. which can be performed on two 1D input tensors.
**Versioned name**: *PriorBoxClustered-1* **Versioned name**: *PriorBoxClustered-1*
@ -94,7 +94,7 @@ If *clip* is defined, the coordinates of prior boxes are recalculated with the f
* *step (step_w, step_h)* * *step (step_w, step_h)*
* **Description**: *step (step_w, step_h)* is a distance between box centers. For example, *step* equal 85 means that the distance between neighborhood prior boxes centers is 85. If both *step_h* and *step_w* are 0 then they are updated with value of *step*. If after that they are still 0 then they are calculated as input image width(height) divided with first input width(height). * **Description**: *step (step_w, step_h)* is a distance between box centers. For example, *step* equal 85 means that the distance between neighborhood prior boxes centers is 85. If both *step_h* and *step_w* are 0 then they are updated with value of *step*. If after that they are still 0 then they are calculated as input image width(height) divided with first input width(height).
* **Range of values**: floating-point positive number * **Range of values**: floating-point positive number
* **Type**: ``float`` * **Type**: ``float``
* **Default value**: 0.0 * **Default value**: 0.0
@ -139,10 +139,10 @@ If *clip* is defined, the coordinates of prior boxes are recalculated with the f
<data clip="false" height="44.0,10.0,30.0,19.0,94.0,32.0,61.0,53.0,17.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2" width="86.0,13.0,57.0,39.0,68.0,34.0,142.0,50.0,23.0"/> <data clip="false" height="44.0,10.0,30.0,19.0,94.0,32.0,61.0,53.0,17.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2" width="86.0,13.0,57.0,39.0,68.0,34.0,142.0,50.0,23.0"/>
<input> <input>
<port id="0"> <port id="0">
<dim>2</dim> < !-- [10, 19] --> <dim>2</dim> <!-- [10, 19] -->
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim> < !-- [180, 320] --> <dim>2</dim> <!-- [180, 320] -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -6,7 +6,7 @@ PriorBox
.. meta:: .. meta::
:description: Learn about PriorBox-1 - an object detection operation, :description: Learn about PriorBox-1 - an object detection operation,
which can be performed on two required input tensors. which can be performed on two required input tensors.
**Versioned name**: *PriorBox-1* **Versioned name**: *PriorBox-1*
@ -22,44 +22,44 @@ PriorBox
1. First calculates *center_x* and *center_y* of prior box: 1. First calculates *center_x* and *center_y* of prior box:
.. math:: .. math::
W \equiv Width \quad Of \quad Image \\ H \equiv Height \quad Of \quad Image W \equiv Width \quad Of \quad Image \\ H \equiv Height \quad Of \quad Image
* If step equals 0: * If step equals 0:
.. math:: .. math::
center_x=(w+0.5) \\ center_y=(h+0.5) center_x=(w+0.5) \\ center_y=(h+0.5)
* else: * else:
.. math:: .. math::
center_x=(w+offset)*step \\ center_y=(h+offset)*step \\ w \subset \left( 0, W \right ) \\ h \subset \left( 0, H \right ) center_x=(w+offset)*step \\ center_y=(h+offset)*step \\ w \subset \left( 0, W \right ) \\ h \subset \left( 0, H \right )
2. Then, for each :math:`s \subset \left( 0, min\_sizes \right )` calculates coordinates of prior boxes: 2. Then, for each :math:`s \subset \left( 0, min\_sizes \right )` calculates coordinates of prior boxes:
.. math:: .. math::
xmin = \frac{\frac{center_x - s}{2}}{W} xmin = \frac{\frac{center_x - s}{2}}{W}
.. math:: .. math::
ymin = \frac{\frac{center_y - s}{2}}{H} ymin = \frac{\frac{center_y - s}{2}}{H}
.. math:: .. math::
xmax = \frac{\frac{center_x + s}{2}}{W} xmax = \frac{\frac{center_x + s}{2}}{W}
.. math:: .. math::
ymin = \frac{\frac{center_y + s}{2}}{H} ymin = \frac{\frac{center_y + s}{2}}{H}
3. If *clip* attribute is set to true, each output value is clipped between :math:`\left< 0, 1 \right>`. 3. If *clip* attribute is set to true, each output value is clipped between :math:`\left< 0, 1 \right>`.
@ -186,10 +186,10 @@ PriorBox
<data aspect_ratio="2.0" clip="false" density="" fixed_ratio="" fixed_size="" flip="true" max_size="38.46" min_size="16.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2"/> <data aspect_ratio="2.0" clip="false" density="" fixed_ratio="" fixed_size="" flip="true" max_size="38.46" min_size="16.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2"/>
<input> <input>
<port id="0"> <port id="0">
<dim>2</dim> < !-- values: [24, 42] --> <dim>2</dim> <!-- values: [24, 42] -->
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim> < !-- values: [384, 672] --> <dim>2</dim> <!-- values: [384, 672] -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ PriorBox
.. meta:: .. meta::
:description: Learn about PriorBox-8 - an object detection operation, :description: Learn about PriorBox-8 - an object detection operation,
which can be performed on two required input tensors. which can be performed on two required input tensors.
**Versioned name**: *PriorBox-8* **Versioned name**: *PriorBox-8*
@ -21,41 +21,41 @@ PriorBox
1. First, it calculates *center_x* and *center_y* of a prior box: 1. First, it calculates *center_x* and *center_y* of a prior box:
.. math:: .. math::
W \equiv Width \quad Of \quad Image \\ H \equiv Height \quad Of \quad Image W \equiv Width \quad Of \quad Image \\ H \equiv Height \quad Of \quad Image
* If step equals 0: * If step equals 0:
.. math:: .. math::
center_x=(w+0.5) \\ center_y=(h+0.5) center_x=(w+0.5) \\ center_y=(h+0.5)
* else: * else:
.. math:: .. math::
center_x=(w+offset)*step \\ center_y=(h+offset)*step \\ w \subset \left( 0, W \right ) \\ h \subset \left( 0, H \right ) center_x=(w+offset)*step \\ center_y=(h+offset)*step \\ w \subset \left( 0, W \right ) \\ h \subset \left( 0, H \right )
2. Then, it calculates coordinates of prior boxes for each :math:`s \subset \left( 0, min\_sizes \right )` : 2. Then, it calculates coordinates of prior boxes for each :math:`s \subset \left( 0, min\_sizes \right )` :
.. math:: .. math::
xmin = \frac{\frac{center_x - s}{2}}{W} xmin = \frac{\frac{center_x - s}{2}}{W}
.. math:: .. math::
ymin = \frac{\frac{center_y - s}{2}}{H} ymin = \frac{\frac{center_y - s}{2}}{H}
.. math:: .. math::
xmax = \frac{\frac{center_x + s}{2}}{W} xmax = \frac{\frac{center_x + s}{2}}{W}
.. math:: .. math::
ymin = \frac{\frac{center_y + s}{2}}{H} ymin = \frac{\frac{center_y + s}{2}}{H}
3. If *clip* attribute is set to true, each output value is clipped between :math:`\left< 0, 1 \right>`. 3. If *clip* attribute is set to true, each output value is clipped between :math:`\left< 0, 1 \right>`.
@ -82,7 +82,7 @@ PriorBox
* **Description**: *flip* is a flag that denotes that each *aspect_ratio* is duplicated and flipped. For example, *flip* equals 1 and *aspect_ratio* equals ``[4.0,2.0]``, meaning that the aspect_ratio is equal to ``[4.0,2.0,0.25,0.5]``. * **Description**: *flip* is a flag that denotes that each *aspect_ratio* is duplicated and flipped. For example, *flip* equals 1 and *aspect_ratio* equals ``[4.0,2.0]``, meaning that the aspect_ratio is equal to ``[4.0,2.0,0.25,0.5]``.
* **Range of values**: * **Range of values**:
* false or 0 - each *aspect_ratio* is flipped * false or 0 - each *aspect_ratio* is flipped
* true or 1 - each *aspect_ratio* is not flipped * true or 1 - each *aspect_ratio* is not flipped
* **Type**: ``boolean`` * **Type**: ``boolean``
@ -193,10 +193,10 @@ PriorBox
<data aspect_ratio="2.0" clip="false" density="" fixed_ratio="" fixed_size="" flip="true" max_size="38.46" min_size="16.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2"/> <data aspect_ratio="2.0" clip="false" density="" fixed_ratio="" fixed_size="" flip="true" max_size="38.46" min_size="16.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2"/>
<input> <input>
<port id="0"> <port id="0">
<dim>2</dim> < !-- values: [24, 42] --> <dim>2</dim> <!-- values: [24, 42] -->
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim> < !-- values: [384, 672] --> <dim>2</dim> <!-- values: [384, 672] -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ RegionYolo
.. meta:: .. meta::
:description: Learn about RegionYolo-1 - an object detection operation, :description: Learn about RegionYolo-1 - an object detection operation,
which can be performed on a 4D input tensor. which can be performed on a 4D input tensor.
**Versioned name**: *RegionYolo-1* **Versioned name**: *RegionYolo-1*
@ -65,7 +65,7 @@ RegionYolo
* **Description**: *do_softmax* is a flag that specifies the inference method and affects how the number of regions is determined. It also affects output shape. If it is 0, then output shape is 4D, and 2D otherwise. * **Description**: *do_softmax* is a flag that specifies the inference method and affects how the number of regions is determined. It also affects output shape. If it is 0, then output shape is 4D, and 2D otherwise.
* **Range of values**: * **Range of values**:
* *false* - do not perform softmax * *false* - do not perform softmax
* *true* - perform softmax * *true* - perform softmax
* **Type**: ``boolean`` * **Type**: ``boolean``
@ -100,7 +100,7 @@ RegionYolo
.. code-block:: xml .. code-block:: xml
:force: :force:
< !-- YOLO V3 example --> <!-- YOLO V3 example -->
<layer type="RegionYolo" ... > <layer type="RegionYolo" ... >
<data anchors="10,14,23,27,37,58,81,82,135,169,344,319" axis="1" classes="80" coords="4" do_softmax="0" end_axis="3" mask="0,1,2" num="6"/> <data anchors="10,14,23,27,37,58,81,82,135,169,344,319" axis="1" classes="80" coords="4" do_softmax="0" end_axis="3" mask="0,1,2" num="6"/>
<input> <input>
@ -120,8 +120,8 @@ RegionYolo
</port> </port>
</output> </output>
</layer> </layer>
< !-- YOLO V2 Example --> <!-- YOLO V2 Example -->
<layer type="RegionYolo" ... > <layer type="RegionYolo" ... >
<data anchors="1.08,1.19,3.42,4.41,6.63,11.38,9.42,5.11,16.62,10.52" axis="1" classes="20" coords="4" do_softmax="1" end_axis="3" num="5"/> <data anchors="1.08,1.19,3.42,4.41,6.63,11.38,9.42,5.11,16.62,10.52" axis="1" classes="20" coords="4" do_softmax="1" end_axis="3" num="5"/>
<input> <input>

View File

@ -5,7 +5,7 @@ Eye
.. meta:: .. meta::
:description: Learn about Eye-9 - a generation operation, which can be :description: Learn about Eye-9 - a generation operation, which can be
performed on three required and one optional input tensors. performed on three required and one optional input tensors.
**Versioned name**: *Eye-9* **Versioned name**: *Eye-9*
@ -23,13 +23,13 @@ Example 1. *Eye* output with ``output_type`` = ``i32``:
.. code-block:: xml .. code-block:: xml
:force: :force:
num_rows = 3 num_rows = 3
num_columns = 4 num_columns = 4
diagonal_index = 2 diagonal_index = 2
output = [[0 0 1 0] output = [[0 0 1 0]
[0 0 0 1] [0 0 0 1]
[0 0 0 0]] [0 0 0 0]]
@ -38,13 +38,13 @@ Example 2. *Eye* output with ``output_type`` = ``i32``:
.. code-block:: xml .. code-block:: xml
:force: :force:
num_rows = 3 num_rows = 3
num_columns = 4 num_columns = 4
diagonal_index = -1 diagonal_index = -1
output = [[0 0 0 0] output = [[0 0 0 0]
[1 0 0 0] [1 0 0 0]
[0 1 0 0]] [0 1 0 0]]
@ -53,13 +53,13 @@ Example 3. *Eye* output with ``output_type`` = ``f16``:
.. code-block:: xml .. code-block:: xml
:force: :force:
num_rows = 2 num_rows = 2
diagonal_index = 5 diagonal_index = 5
batch_shape = [1, 2] batch_shape = [1, 2]
output = [[[[0. 0.] output = [[[[0. 0.]
[0. 0.]] [0. 0.]]
[[0. 0.] [[0. 0.]
@ -97,13 +97,13 @@ Example 3. *Eye* output with ``output_type`` = ``f16``:
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... name="Eye" type="Eye"> <layer ... name="Eye" type="Eye">
<data output_type="i8"/> <data output_type="i8"/>
<input> <input>
<port id="0" precision="I32"/> < !-- num rows: 5 --> <port id="0" precision="I32"/> <!-- num rows: 5 -->
<port id="1" precision="I32"/> < !-- num columns: 5 --> <port id="1" precision="I32"/> <!-- num columns: 5 -->
<port id="2" precision="I32"/> < !-- diagonal index --> <port id="2" precision="I32"/> <!-- diagonal index -->
</input> </input>
<output> <output>
<port id="3" precision="I8" names="Eye:0"> <port id="3" precision="I8" names="Eye:0">
@ -117,14 +117,14 @@ Example 3. *Eye* output with ``output_type`` = ``f16``:
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... name="Eye" type="Eye"> <layer ... name="Eye" type="Eye">
<data output_type="f32"/> <data output_type="f32"/>
<input> <input>
<port id="0" precision="I32"/> < !-- num rows --> <port id="0" precision="I32"/> <!-- num rows -->
<port id="1" precision="I32"/> < !-- num columns --> <port id="1" precision="I32"/> <!-- num columns -->
<port id="2" precision="I32"/> < !-- diagonal index --> <port id="2" precision="I32"/> <!-- diagonal index -->
<port id="3" precision="I32"/> < !-- batch_shape : [2, 3] --> <port id="3" precision="I32"/> <!-- batch_shape : [2, 3] -->
</input> </input>
<output> <output>
<port id="3" precision="F32" names="Eye:0"> <port id="3" precision="F32" names="Eye:0">

View File

@ -91,10 +91,10 @@ Example 3 - 2D tensor, without replacement
* **Description**: controls whether to sample with replacement (classes can be sampled multiple times). * **Description**: controls whether to sample with replacement (classes can be sampled multiple times).
* **Range of values**: `true`, `false` * **Range of values**: `true`, `false`
* ``true`` - class indices can be sampled multiple times. * ``true`` - class indices can be sampled multiple times.
* ``false`` - class indices will not repeat in the output and the size of ``probs``' ``class_size`` dimension is required to be larger or equal to *num_samples* value. Might affect performance. * ``false`` - class indices will not repeat in the output and the size of ``probs``' ``class_size`` dimension is required to be larger or equal to *num_samples* value. Might affect performance.
* **Type**: `bool` * **Type**: `bool`
* **Required**: *Yes* * **Required**: *Yes*
@ -149,16 +149,16 @@ Example 3 - 2D tensor, without replacement
<layer ... name="Multinomial" type="Multinomial"> <layer ... name="Multinomial" type="Multinomial">
<data convert_type="f32", with_replacement="true", log_probs="false", global_seed="234", op_seed="148"/> <data convert_type="f32", with_replacement="true", log_probs="false", global_seed="234", op_seed="148"/>
<input> <input>
<port id="0" precision="FP32"> < !-- probs value: [[0.1, 0.5, 0.4]] --> <port id="0" precision="FP32"> <!-- probs value: [[0.1, 0.5, 0.4]] -->
<dim>1</dim> < !-- batch size of 2 --> <dim>1</dim> <!-- batch size of 2 -->
<dim>3</dim> <dim>3</dim>
</port> </port>
<port id="1" precision="I32"/> < !-- num_samples value: 5 --> <port id="1" precision="I32"/> <!-- num_samples value: 5 -->
</input> </input>
<output> <output>
<port id="3" precision="I32" names="Multinomial:0"> <port id="3" precision="I32" names="Multinomial:0">
<dim>1</dim> < !--dimension depends on input batch size --> <dim>1</dim> <!--dimension depends on input batch size -->
<dim>5</dim> < !--dimension depends on num_samples --> <dim>5</dim> <!--dimension depends on num_samples -->
</port> </port>
</output> </output>
</layer> </layer>
@ -171,16 +171,16 @@ Example 3 - 2D tensor, without replacement
<layer ... name="Multinomial" type="Multinomial"> <layer ... name="Multinomial" type="Multinomial">
<data convert_type="f32", with_replacement="true", log_probs="true", global_seed="234", op_seed="148"/> <data convert_type="f32", with_replacement="true", log_probs="true", global_seed="234", op_seed="148"/>
<input> <input>
<port id="0" precision="FP32"> < !-- probs value: [[-1, 1, 2], [50, 1, 21]] --> <port id="0" precision="FP32"> <!-- probs value: [[-1, 1, 2], [50, 1, 21]] -->
<dim>2</dim> < !-- batch size of 2 --> <dim>2</dim> <!-- batch size of 2 -->
<dim>3</dim> <dim>3</dim>
</port> </port>
<port id="1" precision="I32"/> < !-- num_samples value: 10 --> <port id="1" precision="I32"/> <!-- num_samples value: 10 -->
</input> </input>
<output> <output>
<port id="3" precision="I32" names="Multinomial:0"> <port id="3" precision="I32" names="Multinomial:0">
<dim>2</dim> < !--dimension depends on input batch size --> <dim>2</dim> <!--dimension depends on input batch size -->
<dim>10</dim> < !--dimension depends on num_samples --> <dim>10</dim> <!--dimension depends on num_samples -->
</port> </port>
</output> </output>
</layer> </layer>
@ -193,16 +193,16 @@ Example 3 - 2D tensor, without replacement
<layer ... name="Multinomial" type="Multinomial"> <layer ... name="Multinomial" type="Multinomial">
<data convert_type="f32", with_replacement="false", log_probs="false", global_seed="234", op_seed="148"/> <data convert_type="f32", with_replacement="false", log_probs="false", global_seed="234", op_seed="148"/>
<input> <input>
<port id="0" precision="FP32"> < !-- probs value: [[0.1, 0.5, 0.4]] --> <port id="0" precision="FP32"> <!-- probs value: [[0.1, 0.5, 0.4]] -->
<dim>2</dim> < !-- batch size of 2 --> <dim>2</dim> <!-- batch size of 2 -->
<dim>3</dim> <dim>3</dim>
</port> </port>
<port id="1" precision="I32"/> < !-- num_samples value: 2 --> <port id="1" precision="I32"/> <!-- num_samples value: 2 -->
</input> </input>
<output> <output>
<port id="3" precision="I32" names="Multinomial:0"> <port id="3" precision="I32" names="Multinomial:0">
<dim>2</dim> < !-- batch size of 2 --> <dim>2</dim> <!-- batch size of 2 -->
<dim>2</dim> < !-- 2 unique samples of classes --> <dim>2</dim> <!-- 2 unique samples of classes -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ RandomUniform
.. meta:: .. meta::
:description: Learn about RandomUniform-8 - a generation operation, which can be :description: Learn about RandomUniform-8 - a generation operation, which can be
performed on three required input tensors. performed on three required input tensors.
**Versioned name**: *RandomUniform-8* **Versioned name**: *RandomUniform-8*
@ -16,10 +16,10 @@ RandomUniform
**Detailed description**: **Detailed description**:
*RandomUniform* operation generates random numbers from a uniform distribution in the range ``[minval, maxval)``. *RandomUniform* operation generates random numbers from a uniform distribution in the range ``[minval, maxval)``.
The generation algorithm is based on underlying random integer generator that uses Philox algorithm. Philox algorithm The generation algorithm is based on underlying random integer generator that uses Philox algorithm. Philox algorithm
is a counter-based pseudo-random generator, which produces uint32 values. Single invocation of Philox algorithm returns is a counter-based pseudo-random generator, which produces uint32 values. Single invocation of Philox algorithm returns
four result random values, depending on the given *key* and *counter* values. *Key* and *counter* are initialized four result random values, depending on the given *key* and *counter* values. *Key* and *counter* are initialized
with *global_seed* and *op_seed* attributes respectively. with *global_seed* and *op_seed* attributes respectively.
If both seed values equal to zero, RandomUniform generates non-deterministic sequence. If both seed values equal to zero, RandomUniform generates non-deterministic sequence.
@ -32,7 +32,7 @@ If both seed values equal to zero, RandomUniform generates non-deterministic seq
Link to the original paper `Parallel Random Numbers: As Easy as 1, 2, 3 <https://www.thesalmons.org/john/random123/papers/random123sc11.pdf>`__. Link to the original paper `Parallel Random Numbers: As Easy as 1, 2, 3 <https://www.thesalmons.org/john/random123/papers/random123sc11.pdf>`__.
The result of Philox is calculated by applying a fixed number of *key* and *counter* updating so-called "rounds". The result of Philox is calculated by applying a fixed number of *key* and *counter* updating so-called "rounds".
This implementation uses 4x32_10 version of Philox algorithm, where number of rounds = 10. This implementation uses 4x32_10 version of Philox algorithm, where number of rounds = 10.
Suppose we have *n* which determines *n*-th 4 elements of random sequence. Suppose we have *n* which determines *n*-th 4 elements of random sequence.
@ -43,7 +43,7 @@ In each round *key*, *counter* and *n* are splitted to pairs of uint32 values:
R = cast\_to\_uint32(value)\\ R = cast\_to\_uint32(value)\\
L = cast\_to\_uint32(value >> 32), L = cast\_to\_uint32(value >> 32),
where *cast\_to\_uint32* - static cast to uint32, *value* - uint64 input value, *L*, *R* - uint32 where *cast\_to\_uint32* - static cast to uint32, *value* - uint64 input value, *L*, *R* - uint32
result values, >> - bitwise right shift. result values, >> - bitwise right shift.
Then *n* and *counter* are updated with the following formula: Then *n* and *counter* are updated with the following formula:
@ -68,7 +68,7 @@ Values :math:`L'_{n}, R'_{n}, L'_{counter}, R'_{counter}` are resulting four ran
Float values between [0..1) are obtained from 32-bit integers by the following rules. Float values between [0..1) are obtained from 32-bit integers by the following rules.
Float16 is formatted as follows: *sign* (1 bit) *exponent* (5 bits) *mantissa* (10 bits). The value is interpreted Float16 is formatted as follows: *sign* (1 bit) *exponent* (5 bits) *mantissa* (10 bits). The value is interpreted
using following formula: using following formula:
.. math:: .. math::
@ -99,7 +99,7 @@ where x is uint32 generated random value.
Float32 is formatted as follows: *sign* (1 bit) *exponent* (8 bits) *mantissa* (23 bits). The value is interpreted using following formula: Float32 is formatted as follows: *sign* (1 bit) *exponent* (8 bits) *mantissa* (23 bits). The value is interpreted using following formula:
.. math:: .. math::
(-1)^{sign} * 1, mantissa * 2 ^{exponent - 127} (-1)^{sign} * 1, mantissa * 2 ^{exponent - 127}
@ -117,7 +117,7 @@ So the resulting float value is:
.. code-block:: xml .. code-block:: xml
:force: :force:
val = ((exponent << 23) | x & 0x7fffffu) - 1.0, val = ((exponent << 23) | x & 0x7fffffu) - 1.0,
where x is uint32 generated random value. where x is uint32 generated random value.
@ -125,7 +125,7 @@ where x is uint32 generated random value.
Double is formatted as follows: *sign* (1 bit) *exponent* (11 bits) *mantissa* (52 bits). The value is interpreted using following formula: Double is formatted as follows: *sign* (1 bit) *exponent* (11 bits) *mantissa* (52 bits). The value is interpreted using following formula:
.. math:: .. math::
(-1)^{sign} * 1, mantissa * 2 ^{exponent - 1023} (-1)^{sign} * 1, mantissa * 2 ^{exponent - 1023}
@ -133,7 +133,7 @@ so to obtain double values *sign*, *exponent* and *mantissa* are set as follows:
.. code-block:: xml .. code-block:: xml
:force: :force:
sign = 0 sign = 0
exponent = 1023 - representation of a zero exponent. exponent = 1023 - representation of a zero exponent.
mantissa = 52 right bits from two concatinated uint32 values from random integer generator. mantissa = 52 right bits from two concatinated uint32 values from random integer generator.
@ -143,7 +143,7 @@ So the resulting double is obtained as follows:
.. code-block:: xml .. code-block:: xml
:force: :force:
mantissa_h = x0 & 0xfffffu; // upper 20 bits of mantissa mantissa_h = x0 & 0xfffffu; // upper 20 bits of mantissa
mantissa_l = x1; // lower 32 bits of mantissa mantissa_l = x1; // lower 32 bits of mantissa
mantissa = (mantissa_h << 32) | mantissa_l; mantissa = (mantissa_h << 32) | mantissa_l;
@ -156,7 +156,7 @@ To obtain a value in a specified range each value is processed with the followin
For float values: For float values:
.. math:: .. math::
result = x * (maxval - minval) + minval, result = x * (maxval - minval) + minval,
where *x* is random float or double value between [0..1). where *x* is random float or double value between [0..1).
@ -174,7 +174,7 @@ Example 1. *RandomUniform* output with ``global_seed`` = 150, ``op_seed`` = 10,
.. code-block:: xml .. code-block:: xml
:force: :force:
input_shape = [ 3, 3 ] input_shape = [ 3, 3 ]
output = [[0.7011236 0.30539632 0.93931055] output = [[0.7011236 0.30539632 0.93931055]
[0.9456035 0.11694777 0.50770056] [0.9456035 0.11694777 0.50770056]
@ -185,7 +185,7 @@ Example 2. *RandomUniform* output with ``global_seed`` = 80, ``op_seed`` = 100,
.. code-block:: xml .. code-block:: xml
:force: :force:
input_shape = [ 2, 2 ] input_shape = [ 2, 2 ]
minval = 2 minval = 2
@ -200,7 +200,7 @@ Example 3. *RandomUniform* output with ``global_seed`` = 80, ``op_seed`` = 100,
.. code-block:: xml .. code-block:: xml
:force: :force:
input_shape = [ 2, 3 ] input_shape = [ 2, 3 ]
minval = 50 minval = 50
@ -261,11 +261,11 @@ Example 3. *RandomUniform* output with ``global_seed`` = 80, ``op_seed`` = 100,
<layer ... name="RandomUniform" type="RandomUniform"> <layer ... name="RandomUniform" type="RandomUniform">
<data output_type="f32" global_seed="234" op_seed="148"/> <data output_type="f32" global_seed="234" op_seed="148"/>
<input> <input>
<port id="0" precision="I32"> < !-- shape value: [2, 3, 10] --> <port id="0" precision="I32"> <!-- shape value: [2, 3, 10] -->
<dim>3</dim> <dim>3</dim>
</port> </port>
<port id="1" precision="FP32"/> < !-- min value --> <port id="1" precision="FP32"/> <!-- min value -->
<port id="2" precision="FP32"/> < !-- max value --> <port id="2" precision="FP32"/> <!-- max value -->
</input> </input>
<output> <output>
<port id="3" precision="FP32" names="RandomUniform:0"> <port id="3" precision="FP32" names="RandomUniform:0">

View File

@ -5,7 +5,7 @@ Range
.. meta:: .. meta::
:description: Learn about Range-1 - a generation operation, which can be :description: Learn about Range-1 - a generation operation, which can be
performed on three required input tensors. performed on three required input tensors.
**Versioned name**: *Range-1* **Versioned name**: *Range-1*
@ -46,7 +46,7 @@ For a positive ``step``:
for a negative ``step``: for a negative ``step``:
.. math:: .. math::
start>=val[i]>stop, start>=val[i]>stop,
@ -66,16 +66,16 @@ where
<layer ... type="Range"> <layer ... type="Range">
<input> <input>
<port id="0"> < !-- start value: 2 --> <port id="0"> <!-- start value: 2 -->
</port> </port>
<port id="1"> < !-- stop value: 23 --> <port id="1"> <!-- stop value: 23 -->
</port> </port>
<port id="2"> < !-- step value: 3 --> <port id="2"> <!-- step value: 3 -->
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>7</dim> < !-- [ 2, 5, 8, 11, 14, 17, 20] --> <dim>7</dim> <!-- [ 2, 5, 8, 11, 14, 17, 20] -->
</port> </port>
</output> </output>
</layer> </layer>
@ -88,16 +88,16 @@ where
<layer ... type="Range"> <layer ... type="Range">
<input> <input>
<port id="0"> < !-- start value: 23 --> <port id="0"> <!-- start value: 23 -->
</port> </port>
<port id="1"> < !-- stop value: 2 --> <port id="1"> <!-- stop value: 2 -->
</port> </port>
<port id="2"> < !-- step value: -3 --> <port id="2"> <!-- step value: -3 -->
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>7</dim> < !-- [23, 20, 17, 14, 11, 8, 5] --> <dim>7</dim> <!-- [23, 20, 17, 14, 11, 8, 5] -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ Range
.. meta:: .. meta::
:description: Learn about Range-4 - a generation operation, which can be :description: Learn about Range-4 - a generation operation, which can be
performed on three required input tensors. performed on three required input tensors.
**Versioned name**: *Range-4* **Versioned name**: *Range-4*
@ -81,16 +81,16 @@ This is aligned with PyTorch's operation ``torch.arange``, to align with tensorf
<layer ... type="Range"> <layer ... type="Range">
<data output_type="i32"> <data output_type="i32">
<input> <input>
<port id="0"> < !-- start value: 2 --> <port id="0"> <!-- start value: 2 -->
</port> </port>
<port id="1"> < !-- stop value: 23 --> <port id="1"> <!-- stop value: 23 -->
</port> </port>
<port id="2"> < !-- step value: 3 --> <port id="2"> <!-- step value: 3 -->
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>7</dim> < !-- [ 2, 5, 8, 11, 14, 17, 20] --> <dim>7</dim> <!-- [ 2, 5, 8, 11, 14, 17, 20] -->
</port> </port>
</output> </output>
</layer> </layer>
@ -104,16 +104,16 @@ This is aligned with PyTorch's operation ``torch.arange``, to align with tensorf
<layer ... type="Range"> <layer ... type="Range">
<data output_type="i32"> <data output_type="i32">
<input> <input>
<port id="0"> < !-- start value: 23 --> <port id="0"> <!-- start value: 23 -->
</port> </port>
<port id="1"> < !-- stop value: 2 --> <port id="1"> <!-- stop value: 2 -->
</port> </port>
<port id="2"> < !-- step value: -3 --> <port id="2"> <!-- step value: -3 -->
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>7</dim> < !-- [23, 20, 17, 14, 11, 8, 5] --> <dim>7</dim> <!-- [23, 20, 17, 14, 11, 8, 5] -->
</port> </port>
</output> </output>
</layer> </layer>
@ -127,16 +127,16 @@ This is aligned with PyTorch's operation ``torch.arange``, to align with tensorf
<layer ... type="Range"> <layer ... type="Range">
<data output_type="f32"> <data output_type="f32">
<input> <input>
<port id="0"> < !-- start value: 1 --> <port id="0"> <!-- start value: 1 -->
</port> </port>
<port id="1"> < !-- stop value: 2.5 --> <port id="1"> <!-- stop value: 2.5 -->
</port> </port>
<port id="2"> < !-- step value: 0.5 --> <port id="2"> <!-- step value: 0.5 -->
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>3</dim> < !-- [ 1.0, 1.5, 2.0] --> <dim>3</dim> <!-- [ 1.0, 1.5, 2.0] -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ I420toBGR
.. meta:: .. meta::
:description: Learn about I420toBGR-8 - an image processing operation, which :description: Learn about I420toBGR-8 - an image processing operation, which
can be performed to convert image from I420 to BGR format. can be performed to convert image from I420 to BGR format.
**Versioned name**: *I420toBGR-8* **Versioned name**: *I420toBGR-8*
@ -70,19 +70,19 @@ Same as specified for :doc:`I420toRGB <openvino_docs_ops_image_I420toRGB_8>` ope
<layer ... type="I420toBGR"> <layer ... type="I420toBGR">
<input> <input>
<port id="0">  < !-- Y plane --> <port id="0">  <!-- Y plane -->
<dim>1</dim> <dim>1</dim>
<dim>480</dim> <dim>480</dim>
<dim>640</dim> <dim>640</dim>
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1">  < !-- U plane --> <port id="1">  <!-- U plane -->
<dim>1</dim> <dim>1</dim>
<dim>240</dim> <dim>240</dim>
<dim>320</dim> <dim>320</dim>
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="2">  < !-- V plane --> <port id="2">  <!-- V plane -->
<dim>1</dim> <dim>1</dim>
<dim>240</dim> <dim>240</dim>
<dim>320</dim> <dim>320</dim>

View File

@ -5,7 +5,7 @@ I420toRGB
.. meta:: .. meta::
:description: Learn about I420toRGB-8 - an image processing operation, which :description: Learn about I420toRGB-8 - an image processing operation, which
can be performed to convert image from I420 to RGB format. can be performed to convert image from I420 to RGB format.
**Versioned name**: *I420toRGB-8* **Versioned name**: *I420toRGB-8*
@ -113,19 +113,19 @@ Input I420 image tensor shall have ``NHWC (also known as NYXC)`` layout and can
<layer ... type="I420toRGB"> <layer ... type="I420toRGB">
<input> <input>
<port id="0">  < !-- Y plane --> <port id="0">  <!-- Y plane -->
<dim>1</dim> <dim>1</dim>
<dim>480</dim> <dim>480</dim>
<dim>640</dim> <dim>640</dim>
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1">  < !-- U plane --> <port id="1">  <!-- U plane -->
<dim>1</dim> <dim>1</dim>
<dim>240</dim> <dim>240</dim>
<dim>320</dim> <dim>320</dim>
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="2">  < !-- V plane --> <port id="2">  <!-- V plane -->
<dim>1</dim> <dim>1</dim>
<dim>240</dim> <dim>240</dim>
<dim>320</dim> <dim>320</dim>

View File

@ -5,7 +5,7 @@ Interpolate
.. meta:: .. meta::
:description: Learn about I420toRGB-8 - an image processing operation, which :description: Learn about I420toRGB-8 - an image processing operation, which
can be performed on two required tensors. can be performed on two required tensors.
**Versioned name**: *Interpolate-1* **Versioned name**: *Interpolate-1*
@ -91,7 +91,7 @@ This is a scalar that specifies padding for each spatial dimension.
<dim>80</dim> <dim>80</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim>  < !--The values in this input are [50, 60] --> <dim>2</dim>  <!--The values in this input are [50, 60] -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ Interpolate
.. meta:: .. meta::
:description: Learn about Interpolate-11 - an image processing operation, which :description: Learn about Interpolate-11 - an image processing operation, which
can be performed on two required and one optional tensor. can be performed on two required and one optional tensor.
**Versioned name**: *Interpolate-11* **Versioned name**: *Interpolate-11*
@ -129,13 +129,13 @@ Interpolate
<dim>80</dim> <dim>80</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim> < !--The values in this input are [24, 160] --> <dim>2</dim> <!--The values in this input are [24, 160] -->
</port> </port>
<port id="2"> <port id="2">
<dim>2</dim> < !--The values in this input are [0.5, 2.0] --> <dim>2</dim> <!--The values in this input are [0.5, 2.0] -->
</port> </port>
<port id="3"> <port id="3">
<dim>2</dim> < !--The values in this input are [2, 3] (axes). --> <dim>2</dim> <!--The values in this input are [2, 3] (axes). -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ Interpolate
.. meta:: .. meta::
:description: Learn about Interpolate-4 - an image processing operation, which :description: Learn about Interpolate-4 - an image processing operation, which
can be performed on three required and one optional tensor. can be performed on three required and one optional tensor.
**Versioned name**: *Interpolate-4* **Versioned name**: *Interpolate-4*
@ -128,7 +128,7 @@ Calculations are performed according to the following rules.
import math import math
import numpy as np import numpy as np
from enum import Enum, unique from enum import Enum, unique
class GetNearestPixel: class GetNearestPixel:
def __init__(self, mode: str): def __init__(self, mode: str):
self.func = { self.func = {
@ -138,37 +138,37 @@ Calculations are performed according to the following rules.
'ceil': GetNearestPixel.ceil_func, 'ceil': GetNearestPixel.ceil_func,
'simple': GetNearestPixel.simple_func 'simple': GetNearestPixel.simple_func
}[mode] }[mode]
def __call__(self, x_original, is_downsample): def __call__(self, x_original, is_downsample):
return self.func(x_original, is_downsample) return self.func(x_original, is_downsample)
@staticmethod @staticmethod
def prefer_floor_func(x_original, is_downsample): def prefer_floor_func(x_original, is_downsample):
if x_original == int(x_original) + 0.5: if x_original == int(x_original) + 0.5:
return int(math.floor(x_original)) return int(math.floor(x_original))
else: else:
return int(round(x_original)) return int(round(x_original))
@staticmethod @staticmethod
def prefer_ceil_func(x_original, is_downsample): def prefer_ceil_func(x_original, is_downsample):
return int(round(x_original)) return int(round(x_original))
@staticmethod @staticmethod
def floor_func(x_original, is_downsample): def floor_func(x_original, is_downsample):
return int(math.floor(x_original)) return int(math.floor(x_original))
@staticmethod @staticmethod
def ceil_func(x_original, is_downsample): def ceil_func(x_original, is_downsample):
return int(math.ceil(x_original)) return int(math.ceil(x_original))
@staticmethod @staticmethod
def simple_func(x_original, is_downsample): def simple_func(x_original, is_downsample):
if is_downsample: if is_downsample:
return int(math.ceil(x_original)) return int(math.ceil(x_original))
else: else:
return int(x_original) return int(x_original)
class GetOriginalCoordinate: class GetOriginalCoordinate:
def __init__(self, mode: str): def __init__(self, mode: str):
self.func = { self.func = {
@ -178,31 +178,31 @@ Calculations are performed according to the following rules.
'tf_half_pixel_for_nn': GetOriginalCoordinate.tf_half_pixel_for_nn_func, 'tf_half_pixel_for_nn': GetOriginalCoordinate.tf_half_pixel_for_nn_func,
'align_corners': GetOriginalCoordinate.align_corners_func 'align_corners': GetOriginalCoordinate.align_corners_func
}[mode] }[mode]
def __call__(self, x_resized, x_scale, length_resized, length_original): def __call__(self, x_resized, x_scale, length_resized, length_original):
return self.func(x_resized, x_scale, length_resized, length_original) return self.func(x_resized, x_scale, length_resized, length_original)
@staticmethod @staticmethod
def half_pixel_func(x_resized, x_scale, length_resized, length_original): def half_pixel_func(x_resized, x_scale, length_resized, length_original):
return ((x_resized + 0.5) / x_scale) - 0.5 return ((x_resized + 0.5) / x_scale) - 0.5
@staticmethod @staticmethod
def pytorch_half_pixel_func(x_resized, x_scale, length_resized, length_original): def pytorch_half_pixel_func(x_resized, x_scale, length_resized, length_original):
return (x_resized + 0.5) / x_scale - 0.5 if length_resized > 1 else 0.0 return (x_resized + 0.5) / x_scale - 0.5 if length_resized > 1 else 0.0
@staticmethod @staticmethod
def asymmetric_func(x_resized, x_scale, length_resized, length_original): def asymmetric_func(x_resized, x_scale, length_resized, length_original):
return x_resized / x_scale return x_resized / x_scale
@staticmethod @staticmethod
def tf_half_pixel_for_nn_func(x_resized, x_scale, length_resized, length_original): def tf_half_pixel_for_nn_func(x_resized, x_scale, length_resized, length_original):
return (x_resized + 0.5) / x_scale return (x_resized + 0.5) / x_scale
@staticmethod @staticmethod
def align_corners_func(x_resized, x_scale, length_resized, length_original): def align_corners_func(x_resized, x_scale, length_resized, length_original):
return 0 if length_resized == 1 else x_resized * (length_original - 1) / (length_resized - 1) return 0 if length_resized == 1 else x_resized * (length_original - 1) / (length_resized - 1)
def get_cubic_coeff(s, a): def get_cubic_coeff(s, a):
abs_s = abs(s) abs_s = abs(s)
coeff = np.zeros(4) coeff = np.zeros(4)
@ -211,18 +211,18 @@ Calculations are performed according to the following rules.
coeff[2] = (((-a -2.0) * abs_s+ (2.0 * a + 3.0)) * abs_s - a) * abs_s coeff[2] = (((-a -2.0) * abs_s+ (2.0 * a + 3.0)) * abs_s - a) * abs_s
coeff[3] = - a * abs_s * abs_s * (abs_s - 1.0) coeff[3] = - a * abs_s * abs_s * (abs_s - 1.0)
return coeff return coeff
def triangle_coeffs(dz): def triangle_coeffs(dz):
return np.maximum(0.0, 1.0 - np.abs(dz)) return np.maximum(0.0, 1.0 - np.abs(dz))
@unique @unique
class ShapeCalculationMode(Enum): class ShapeCalculationMode(Enum):
SIZES = 0 SIZES = 0
SCALES = 1 SCALES = 1
class InterpolateCalculation: class InterpolateCalculation:
def __init__(self, attrs: dict): def __init__(self, attrs: dict):
self.mode = attrs['mode'] self.mode = attrs['mode']
@ -233,38 +233,38 @@ Calculations are performed according to the following rules.
'linear_onnx': self.onnx_linear_interpolation 'linear_onnx': self.onnx_linear_interpolation
}[self.mode] }[self.mode]
self.attrs = attrs self.attrs = attrs
self.pads_begin = attrs.get('pads_begin', [0]) self.pads_begin = attrs.get('pads_begin', [0])
self.pads_end = attrs.get('pads_end', [0]) self.pads_end = attrs.get('pads_end', [0])
self.coordinate_transformation_mode = attrs.get('coordinate_transformation_mode', 'half_pixel') self.coordinate_transformation_mode = attrs.get('coordinate_transformation_mode', 'half_pixel')
self.nearest_mode = attrs.get('nearest_mode', 'round_prefer_floor') self.nearest_mode = attrs.get('nearest_mode', 'round_prefer_floor')
self.cube_coeff = attrs.get('cube_coeff', -0.75) self.cube_coeff = attrs.get('cube_coeff', -0.75)
self.antialias = attrs.get('antialias', False) self.antialias = attrs.get('antialias', False)
self.shape_calculation_mode = { self.shape_calculation_mode = {
'sizes': ShapeCalculationMode.SIZES, 'sizes': ShapeCalculationMode.SIZES,
'scales': ShapeCalculationMode.SCALES 'scales': ShapeCalculationMode.SCALES
}[attrs['shape_calculation_mode']] }[attrs['shape_calculation_mode']]
self.get_original_coordinate = self.get_coordinate_transformation_mode() self.get_original_coordinate = self.get_coordinate_transformation_mode()
self.get_nearest_pixel = GetNearestPixel(self.nearest_mode) self.get_nearest_pixel = GetNearestPixel(self.nearest_mode)
def get_coordinate_transformation_mode(self): def get_coordinate_transformation_mode(self):
return GetOriginalCoordinate(self.coordinate_transformation_mode) return GetOriginalCoordinate(self.coordinate_transformation_mode)
def shape_infer(self, input_data, sizes, scales): def shape_infer(self, input_data, sizes, scales):
result = input_data.shape + self.pads_begin + self.pads_end result = input_data.shape + self.pads_begin + self.pads_end
if self.shape_calculation_mode == ShapeCalculationMode.SIZES: if self.shape_calculation_mode == ShapeCalculationMode.SIZES:
for i, axis in enumerate(self.axes): for i, axis in enumerate(self.axes):
result[axis] = sizes[i] result[axis] = sizes[i]
else: else:
for i, axis in enumerate(self.axes): for i, axis in enumerate(self.axes):
result[axis] = math.floor(scales[i] * result[axis]) result[axis] = math.floor(scales[i] * result[axis])
return result return result
@staticmethod @staticmethod
def correct_pad(pad, rank): def correct_pad(pad, rank):
pad_len = len(pad) pad_len = len(pad)
@ -274,17 +274,17 @@ Calculations are performed according to the following rules.
return np.array(pad[: rank - 1]).astype(np.int64) return np.array(pad[: rank - 1]).astype(np.int64)
else: else:
return np.array(pad, dtype=np.int64) return np.array(pad, dtype=np.int64)
def __call__(self, input_data, sizes, scales, axes): def __call__(self, input_data, sizes, scales, axes):
rank = input_data.ndim rank = input_data.ndim
self.pads_begin = InterpolateCalculation.correct_pad(self.pads_begin, rank) self.pads_begin = InterpolateCalculation.correct_pad(self.pads_begin, rank)
self.pads_end = InterpolateCalculation.correct_pad(self.pads_end, rank) self.pads_end = InterpolateCalculation.correct_pad(self.pads_end, rank)
self.pads = list(zip(self.pads_begin, self.pads_end)) self.pads = list(zip(self.pads_begin, self.pads_end))
self.axes = np.array(axes).astype(np.int64) self.axes = np.array(axes).astype(np.int64)
self.output_shape = self.shape_infer(input_data, sizes, scales) self.output_shape = self.shape_infer(input_data, sizes, scales)
padded_data = np.pad(input_data, self.pads, 'constant') padded_data = np.pad(input_data, self.pads, 'constant')
if self.shape_calculation_mode == ShapeCalculationMode.SIZES: if self.shape_calculation_mode == ShapeCalculationMode.SIZES:
num_of_axes = len(self.axes) num_of_axes = len(self.axes)
self.scales = np.zeros(num_of_axes) self.scales = np.zeros(num_of_axes)
@ -292,18 +292,18 @@ Calculations are performed according to the following rules.
self.scales[i] = self.output_shape[axis] / padded_data.shape[axis] self.scales[i] = self.output_shape[axis] / padded_data.shape[axis]
else: else:
self.scales = scales self.scales = scales
if self.mode == 'nearest': if self.mode == 'nearest':
self.all_scales = np.ones(rank).astype(np.float) self.all_scales = np.ones(rank).astype(np.float)
for i, axis in enumerate(self.axes): for i, axis in enumerate(self.axes):
self.all_scales[axis] = self.scales[i] self.all_scales[axis] = self.scales[i]
self.input_shape = padded_data.shape self.input_shape = padded_data.shape
return self.func(padded_data) return self.func(padded_data)
def clip_coord(self, coord, axis): def clip_coord(self, coord, axis):
return max(0, min(coord, self.input_shape[axis] - 1)) return max(0, min(coord, self.input_shape[axis] - 1))
def cubic_interpolation(self, input_data): def cubic_interpolation(self, input_data):
rank = len(self.input_shape) rank = len(self.input_shape)
result = np.zeros(self.output_shape) result = np.zeros(self.output_shape)
@ -328,28 +328,28 @@ Calculations are performed according to the following rules.
summa += coeffs_prod * input_data[tuple(coords_for_sum)] summa += coeffs_prod * input_data[tuple(coords_for_sum)]
result[coordinates] = summa result[coordinates] = summa
return result return result
def linear_interpolation(self, input_data): def linear_interpolation(self, input_data):
result = np.zeros(self.output_shape) result = np.zeros(self.output_shape)
num_of_axes = len(self.axes) num_of_axes = len(self.axes)
is_downsample = False is_downsample = False
for scale in self.scales: for scale in self.scales:
is_downsample = is_downsample or (scale < 1) is_downsample = is_downsample or (scale < 1)
antialias = is_downsample and self.antialias antialias = is_downsample and self.antialias
a = np.zeros(num_of_axes) a = np.zeros(num_of_axes)
for i, _ in enumerate(self.axes): for i, _ in enumerate(self.axes):
a[i] = self.scales[i] if antialias else 1.0 a[i] = self.scales[i] if antialias else 1.0
prod_of_a = np.prod(a) prod_of_a = np.prod(a)
r = np.zeros(num_of_axes).astype(np.int64) r = np.zeros(num_of_axes).astype(np.int64)
for i, _ in enumerate(self.axes): for i, _ in enumerate(self.axes):
r[i] = 2 if self.scales[i] > 1.0 else int(math.ceil(2.0/a[i])) r[i] = 2 if self.scales[i] > 1.0 else int(math.ceil(2.0/a[i]))
indices = [tuple(np.array(ind).astype(np.int64) - r) for ind in np.ndindex(tuple(2 * r + 1))] indices = [tuple(np.array(ind).astype(np.int64) - r) for ind in np.ndindex(tuple(2 * r + 1))]
for coordinates in np.ndindex(tuple(self.output_shape)): for coordinates in np.ndindex(tuple(self.output_shape)):
icoords = np.array(coordinates).astype(np.float64) icoords = np.array(coordinates).astype(np.float64)
icoords_r = np.array(coordinates).astype(np.float64) icoords_r = np.array(coordinates).astype(np.float64)
@ -357,51 +357,51 @@ Calculations are performed according to the following rules.
in_coord = self.get_original_coordinate(coordinates[axis], self.scales[i], self.output_shape[axis], self.input_shape[axis]) in_coord = self.get_original_coordinate(coordinates[axis], self.scales[i], self.output_shape[axis], self.input_shape[axis])
icoords[axis] = in_coord icoords[axis] = in_coord
icoords_r[axis] = round(in_coord) icoords_r[axis] = round(in_coord)
summa = 0.0 summa = 0.0
wsum = 0.0 wsum = 0.0
for index in indices: for index in indices:
inner_coords = np.array(coordinates) inner_coords = np.array(coordinates)
for i, axis in enumerate(self.axes): for i, axis in enumerate(self.axes):
inner_coords[axis] = index[i] + icoords_r[axis] inner_coords[axis] = index[i] + icoords_r[axis]
conditions = [inner_coords[axis] >= 0 and inner_coords[axis] < self.input_shape[axis] for axis in self.axes] conditions = [inner_coords[axis] >= 0 and inner_coords[axis] < self.input_shape[axis] for axis in self.axes]
if not all(conditions): if not all(conditions):
continue continue
dz = np.zeros(num_of_axes) dz = np.zeros(num_of_axes)
for i, axis in enumerate(self.axes): for i, axis in enumerate(self.axes):
dz[i] = icoords[axis] - inner_coords[axis] dz[i] = icoords[axis] - inner_coords[axis]
w = prod_of_a * np.prod(triangle_coeffs(a * dz)) w = prod_of_a * np.prod(triangle_coeffs(a * dz))
wsum += w wsum += w
summa += w * input_data[tuple(inner_coords)] summa += w * input_data[tuple(inner_coords)]
if wsum == 0: if wsum == 0:
result[coordinates] = 0.0 result[coordinates] = 0.0
else: else:
result[coordinates] = summa / wsum result[coordinates] = summa / wsum
return result return result
def onnx_linear_interpolation5D(self, input_data): def onnx_linear_interpolation5D(self, input_data):
rank = len(self.input_shape) rank = len(self.input_shape)
assert rank in [3, 5], "mode 'linear_onnx' supports only 3D or 5D tensors" assert rank in [3, 5], "mode 'linear_onnx' supports only 3D or 5D tensors"
assert set(self.axes) == {2, 3, 4} or set(self.axes) == {0, 1, 2}, \ assert set(self.axes) == {2, 3, 4} or set(self.axes) == {0, 1, 2}, \
"mode 'linear_onnx' supports only case when axes = {2, 3, 4} or axes = {0, 1, 2}" "mode 'linear_onnx' supports only case when axes = {2, 3, 4} or axes = {0, 1, 2}"
result = np.zeros(self.output_shape) result = np.zeros(self.output_shape)
if rank == 3: if rank == 3:
reshaped_data = np.reshape(input_data, (1, 1, self.input_shape[0], self.input_shape[1], self.input_shape[2])) reshaped_data = np.reshape(input_data, (1, 1, self.input_shape[0], self.input_shape[1], self.input_shape[2]))
result = np.reshape(result, (1, 1, self.output_shape[0], self.output_shape[1], self.output_shape[2])) result = np.reshape(result, (1, 1, self.output_shape[0], self.output_shape[1], self.output_shape[2]))
else: else:
reshaped_data = input_data reshaped_data = input_data
input_shape = np.array(reshaped_data.shape).astype(np.int64) input_shape = np.array(reshaped_data.shape).astype(np.int64)
output_shape = np.array(result.shape).astype(np.int64) output_shape = np.array(result.shape).astype(np.int64)
batch_size = input_shape[0]; batch_size = input_shape[0];
num_channels = input_shape[1]; num_channels = input_shape[1];
input_depth = input_shape[2]; input_depth = input_shape[2];
@ -410,31 +410,31 @@ Calculations are performed according to the following rules.
output_depth = output_shape[2]; output_depth = output_shape[2];
output_height = output_shape[3]; output_height = output_shape[3];
output_width = output_shape[4]; output_width = output_shape[4];
depth_scale = self.scales[0]; depth_scale = self.scales[0];
height_scale = self.scales[1]; height_scale = self.scales[1];
width_scale = self.scales[2]; width_scale = self.scales[2];
z_original = np.zeros(output_depth).astype(np.float) z_original = np.zeros(output_depth).astype(np.float)
y_original = np.zeros(output_height).astype(np.float) y_original = np.zeros(output_height).astype(np.float)
x_original = np.zeros(output_width).astype(np.float) x_original = np.zeros(output_width).astype(np.float)
in_z1 = np.zeros(output_depth).astype(np.int64) in_z1 = np.zeros(output_depth).astype(np.int64)
in_z2 = np.zeros(output_depth).astype(np.int64) in_z2 = np.zeros(output_depth).astype(np.int64)
in_y1 = np.zeros(output_height).astype(np.int64) in_y1 = np.zeros(output_height).astype(np.int64)
in_y2 = np.zeros(output_height).astype(np.int64) in_y2 = np.zeros(output_height).astype(np.int64)
in_x1 = np.zeros(output_width).astype(np.int64) in_x1 = np.zeros(output_width).astype(np.int64)
in_x2 = np.zeros(output_width).astype(np.int64) in_x2 = np.zeros(output_width).astype(np.int64)
dz1 = np.zeros(output_depth).astype(np.float) dz1 = np.zeros(output_depth).astype(np.float)
dz2 = np.zeros(output_depth).astype(np.float) dz2 = np.zeros(output_depth).astype(np.float)
dy1 = np.zeros(output_height).astype(np.float) dy1 = np.zeros(output_height).astype(np.float)
dy2 = np.zeros(output_height).astype(np.float) dy2 = np.zeros(output_height).astype(np.float)
dx1 = np.zeros(output_width).astype(np.float) dx1 = np.zeros(output_width).astype(np.float)
dx2 = np.zeros(output_width).astype(np.float) dx2 = np.zeros(output_width).astype(np.float)
for z in range(0, output_depth): for z in range(0, output_depth):
in_z = self.get_original_coordinate(z, depth_scale, output_depth, input_depth) in_z = self.get_original_coordinate(z, depth_scale, output_depth, input_depth)
z_original[z] = in_z z_original[z] = in_z
@ -443,11 +443,11 @@ Calculations are performed according to the following rules.
in_z2[z] = min(in_z1[z] + 1, input_depth - 1) in_z2[z] = min(in_z1[z] + 1, input_depth - 1)
dz1[z] = abs(in_z - in_z1[z]) dz1[z] = abs(in_z - in_z1[z])
dz2[z] = abs(in_z - in_z2[z]) dz2[z] = abs(in_z - in_z2[z])
if in_z1[z] == in_z2[z]: if in_z1[z] == in_z2[z]:
dz1[z] = 0.5 dz1[z] = 0.5
dz2[z] = 0.5 dz2[z] = 0.5
for y in range(0, output_height): for y in range(0, output_height):
in_y = self.get_original_coordinate(y, height_scale, output_height, input_height) in_y = self.get_original_coordinate(y, height_scale, output_height, input_height)
y_original[y] = in_y y_original[y] = in_y
@ -456,19 +456,19 @@ Calculations are performed according to the following rules.
in_y2[y] = min(in_y1[y] + 1, input_height - 1) in_y2[y] = min(in_y1[y] + 1, input_height - 1)
dy1[y] = abs(in_y - in_y1[y]) dy1[y] = abs(in_y - in_y1[y])
dy2[y] = abs(in_y - in_y2[y]) dy2[y] = abs(in_y - in_y2[y])
if in_y1[y] == in_y2[y]: if in_y1[y] == in_y2[y]:
dy1[y] = 0.5 dy1[y] = 0.5
dy2[y] = 0.5 dy2[y] = 0.5
for x in range(0, output_width): for x in range(0, output_width):
in_x = self.get_original_coordinate(x, width_scale, output_width, input_width); in_x = self.get_original_coordinate(x, width_scale, output_width, input_width);
x_original[x] = in_x x_original[x] = in_x
in_x = max(0.0, min(in_x, input_width - 1)); in_x = max(0.0, min(in_x, input_width - 1));
in_x1[x] = min(in_x, input_width - 1); in_x1[x] = min(in_x, input_width - 1);
in_x2[x] = min(in_x1[x] + 1, input_width - 1); in_x2[x] = min(in_x1[x] + 1, input_width - 1);
dx1[x] = abs(in_x - in_x1[x]); dx1[x] = abs(in_x - in_x1[x]);
dx2[x] = abs(in_x - in_x2[x]); dx2[x] = abs(in_x - in_x2[x]);
if in_x1[x] == in_x2[x]: if in_x1[x] == in_x2[x]:
@ -487,33 +487,33 @@ Calculations are performed according to the following rules.
x212 = reshaped_data[n, c, in_z2[z], in_y1[y], in_x2[x]] x212 = reshaped_data[n, c, in_z2[z], in_y1[y], in_x2[x]]
x122 = reshaped_data[n, c, in_z2[z], in_y2[y], in_x1[x]] x122 = reshaped_data[n, c, in_z2[z], in_y2[y], in_x1[x]]
x222 = reshaped_data[n, c, in_z2[z], in_y2[y], in_x2[x]] x222 = reshaped_data[n, c, in_z2[z], in_y2[y], in_x2[x]]
temp = dx2[x] * dy2[y] * dz2[z] * x111 + dx1[x] * dy2[y] * dz2[z] * x211 temp = dx2[x] * dy2[y] * dz2[z] * x111 + dx1[x] * dy2[y] * dz2[z] * x211
temp += dx2[x] * dy1[y] * dz2[z] * x121 + dx1[x] * dy1[y] * dz2[z] * x221 temp += dx2[x] * dy1[y] * dz2[z] * x121 + dx1[x] * dy1[y] * dz2[z] * x221
temp += dx2[x] * dy2[y] * dz1[z] * x112 + dx1[x] * dy2[y] * dz1[z] * x212 temp += dx2[x] * dy2[y] * dz1[z] * x112 + dx1[x] * dy2[y] * dz1[z] * x212
temp += dx2[x] * dy1[y] * dz1[z] * x122 + dx1[x] * dy1[y] * dz1[z] * x222 temp += dx2[x] * dy1[y] * dz1[z] * x122 + dx1[x] * dy1[y] * dz1[z] * x222
result[n, c, z, y, x] = temp result[n, c, z, y, x] = temp
return np.reshape(result, self.output_shape) return np.reshape(result, self.output_shape)
def onnx_linear_interpolation4D(self, input_data): def onnx_linear_interpolation4D(self, input_data):
rank = len(self.input_shape) rank = len(self.input_shape)
assert rank in [2, 4], "mode 'linear_onnx' supports only 2D or 4D tensors" assert rank in [2, 4], "mode 'linear_onnx' supports only 2D or 4D tensors"
assert set(self.axes) == {2, 3} or set(self.axes) == {0, 1}, \ assert set(self.axes) == {2, 3} or set(self.axes) == {0, 1}, \
"mode 'linear_onnx' supports only case when axes = {2, 3} or axes = {0, 1}" "mode 'linear_onnx' supports only case when axes = {2, 3} or axes = {0, 1}"
result = np.zeros(self.output_shape) result = np.zeros(self.output_shape)
if rank == 2: if rank == 2:
reshaped_data = np.reshape(input_data, (1, 1, self.input_shape[0], self.input_shape[1])) reshaped_data = np.reshape(input_data, (1, 1, self.input_shape[0], self.input_shape[1]))
result = np.reshape(result, (1, 1, self.output_shape[0], self.output_shape[1])) result = np.reshape(result, (1, 1, self.output_shape[0], self.output_shape[1]))
else: else:
reshaped_data = input_data reshaped_data = input_data
input_shape = np.array(reshaped_data.shape).astype(np.int64) input_shape = np.array(reshaped_data.shape).astype(np.int64)
output_shape = np.array(result.shape).astype(np.int64) output_shape = np.array(result.shape).astype(np.int64)
output_height = output_shape[2] output_height = output_shape[2]
output_width = output_shape[3] output_width = output_shape[3]
input_height = input_shape[2] input_height = input_shape[2]
@ -522,21 +522,21 @@ Calculations are performed according to the following rules.
width_scale = self.scales[1] width_scale = self.scales[1]
batch_size = input_shape[0] batch_size = input_shape[0]
num_channels = input_shape[1] num_channels = input_shape[1]
y_original = np.zeros(output_height).astype(np.float) y_original = np.zeros(output_height).astype(np.float)
x_original = np.zeros(output_width).astype(np.float) x_original = np.zeros(output_width).astype(np.float)
in_y1 = np.zeros(output_height).astype(np.int64) in_y1 = np.zeros(output_height).astype(np.int64)
in_y2 = np.zeros(output_height).astype(np.int64) in_y2 = np.zeros(output_height).astype(np.int64)
in_x1 = np.zeros(output_width).astype(np.int64) in_x1 = np.zeros(output_width).astype(np.int64)
in_x2 = np.zeros(output_width).astype(np.int64) in_x2 = np.zeros(output_width).astype(np.int64)
dy1 = np.zeros(output_height).astype(np.float) dy1 = np.zeros(output_height).astype(np.float)
dy2 = np.zeros(output_height).astype(np.float) dy2 = np.zeros(output_height).astype(np.float)
dx1 = np.zeros(output_width).astype(np.float) dx1 = np.zeros(output_width).astype(np.float)
dx2 = np.zeros(output_width).astype(np.float) dx2 = np.zeros(output_width).astype(np.float)
for y in range(0, output_height): for y in range(0, output_height):
in_y = self.get_original_coordinate(y, height_scale, output_height, input_height) in_y = self.get_original_coordinate(y, height_scale, output_height, input_height)
y_original[y] = in_y y_original[y] = in_y
@ -545,25 +545,25 @@ Calculations are performed according to the following rules.
in_y2[y] = min(in_y1[y] + 1, input_height - 1) in_y2[y] = min(in_y1[y] + 1, input_height - 1)
dy1[y] = abs(in_y - in_y1[y]) dy1[y] = abs(in_y - in_y1[y])
dy2[y] = abs(in_y - in_y2[y]) dy2[y] = abs(in_y - in_y2[y])
if in_y1[y] == in_y2[y]: if in_y1[y] == in_y2[y]:
dy1[y] = 0.5 dy1[y] = 0.5
dy2[y] = 0.5 dy2[y] = 0.5
for x in range(0, output_width): for x in range(0, output_width):
in_x = self.get_original_coordinate(x, width_scale, output_width, input_width); in_x = self.get_original_coordinate(x, width_scale, output_width, input_width);
x_original[x] = in_x x_original[x] = in_x
in_x = max(0.0, min(in_x, input_width - 1)); in_x = max(0.0, min(in_x, input_width - 1));
in_x1[x] = min(in_x, input_width - 1); in_x1[x] = min(in_x, input_width - 1);
in_x2[x] = min(in_x1[x] + 1, input_width - 1); in_x2[x] = min(in_x1[x] + 1, input_width - 1);
dx1[x] = abs(in_x - in_x1[x]); dx1[x] = abs(in_x - in_x1[x]);
dx2[x] = abs(in_x - in_x2[x]); dx2[x] = abs(in_x - in_x2[x]);
if in_x1[x] == in_x2[x]: if in_x1[x] == in_x2[x]:
dx1[x] = 0.5 dx1[x] = 0.5
dx2[x] = 0.5 dx2[x] = 0.5
for n in range(0, batch_size): for n in range(0, batch_size):
for c in range(0, num_channels): for c in range(0, num_channels):
for y in range(0, output_height): for y in range(0, output_height):
@ -574,21 +574,21 @@ Calculations are performed according to the following rules.
x22 = reshaped_data[n, c, in_y2[y], in_x2[x]] x22 = reshaped_data[n, c, in_y2[y], in_x2[x]]
temp = dx2[x] * dy2[y] * x11 + dx1[x] * dy2[y] * x21 + dx2[x] * dy1[y] * x12 + dx1[x] * dy1[y] * x22 temp = dx2[x] * dy2[y] * x11 + dx1[x] * dy2[y] * x21 + dx2[x] * dy1[y] * x12 + dx1[x] * dy1[y] * x22
result[n, c, y, x] = temp result[n, c, y, x] = temp
return np.reshape(result, self.output_shape) return np.reshape(result, self.output_shape)
def onnx_linear_interpolation(self, input_data): def onnx_linear_interpolation(self, input_data):
rank = len(self.input_shape) rank = len(self.input_shape)
assert rank in [2, 3, 4, 5], "mode 'linear_onnx' supports only 2D, 3D, 4D, or 5D tensors" assert rank in [2, 3, 4, 5], "mode 'linear_onnx' supports only 2D, 3D, 4D, or 5D tensors"
if rank in [2, 4]: if rank in [2, 4]:
self.onnx_linear_interpolation4D(input_data) self.onnx_linear_interpolation4D(input_data)
else: else:
self.onnx_linear_interpolation5D(input_data) self.onnx_linear_interpolation5D(input_data)
def nearest_interpolation(self, input_data): def nearest_interpolation(self, input_data):
result = np.zeros(self.output_shape) result = np.zeros(self.output_shape)
num_of_axes = len(self.axes) num_of_axes = len(self.axes)
for coordinates in np.ndindex(tuple(self.output_shape)): for coordinates in np.ndindex(tuple(self.output_shape)):
input_coords = np.array(coordinates, dtype=np.int64) input_coords = np.array(coordinates, dtype=np.int64)
@ -597,7 +597,7 @@ Calculations are performed according to the following rules.
nearest_pixel = self.get_nearest_pixel(in_coord, scale < 1) nearest_pixel = self.get_nearest_pixel(in_coord, scale < 1)
input_coords[axis] = max(0, min(nearest_pixel, self.input_shape[axis] - 1)) input_coords[axis] = max(0, min(nearest_pixel, self.input_shape[axis] - 1))
result[coordinates] = input_data[tuple(input_coords)] result[coordinates] = input_data[tuple(input_coords)]
return result return result
@ -617,13 +617,13 @@ Calculations are performed according to the following rules.
<dim>80</dim> <dim>80</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim>  < !--The values in this input are [24, 160] --> <dim>2</dim>  <!--The values in this input are [24, 160] -->
</port> </port>
<port id="2"> <port id="2">
<dim>2</dim>  < !--The values in this input are [0.5, 2.0] --> <dim>2</dim>  <!--The values in this input are [0.5, 2.0] -->
</port> </port>
<port id="3"> <port id="3">
<dim>2</dim>  < !--The values in this input are [2, 3] (axes). --> <dim>2</dim>  <!--The values in this input are [2, 3] (axes). -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ NV12toBGR
.. meta:: .. meta::
:description: Learn about NV12toBGR-8 - an image processing operation, which :description: Learn about NV12toBGR-8 - an image processing operation, which
can be performed to convert an image from NV12 to BGR format. can be performed to convert an image from NV12 to BGR format.
**Versioned name**: *NV12toBGR-8* **Versioned name**: *NV12toBGR-8*
@ -70,13 +70,13 @@ Same as specified for :doc:`NV12toRGB <openvino_docs_ops_image_NV12toRGB_8>` ope
<layer ... type="NV12toBGR"> <layer ... type="NV12toBGR">
<input> <input>
<port id="0"> < !-- Y plane --> <port id="0"> <!-- Y plane -->
<dim>1</dim> <dim>1</dim>
<dim>480</dim> <dim>480</dim>
<dim>640</dim> <dim>640</dim>
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1"> < !-- UV plane --> <port id="1"> <!-- UV plane -->
<dim>1</dim> <dim>1</dim>
<dim>240</dim> <dim>240</dim>
<dim>320</dim> <dim>320</dim>

View File

@ -5,7 +5,7 @@ NV12toRGB
.. meta:: .. meta::
:description: Learn about NV12toRGB-8 - an image processing operation, which :description: Learn about NV12toRGB-8 - an image processing operation, which
can be performed to convert an image from NV12 to RGB format. can be performed to convert an image from NV12 to RGB format.
**Versioned name**: *NV12toRGB-8* **Versioned name**: *NV12toRGB-8*
@ -102,13 +102,13 @@ Input NV12 image tensor shall have ``NHWC (also known as NYXC)`` layout and can
<layer ... type="NV12toRGB"> <layer ... type="NV12toRGB">
<input> <input>
<port id="0"> < !-- Y plane --> <port id="0"> <!-- Y plane -->
<dim>1</dim> <dim>1</dim>
<dim>480</dim> <dim>480</dim>
<dim>640</dim> <dim>640</dim>
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1"> < !-- UV plane --> <port id="1"> <!-- UV plane -->
<dim>1</dim> <dim>1</dim>
<dim>240</dim> <dim>240</dim>
<dim>320</dim> <dim>320</dim>

View File

@ -0,0 +1,66 @@
.. {#openvino_docs_ops_infrastructure_Assign_6}
Assign
======
.. meta::
:description: Learn about Assign-6 - an infrastructure operation, which
can be performed on a single input tensor to set a value to variable_id.
**Versioned name**: *Assign-6*
**Category**: *Infrastructure*
**Short description**: *Assign* sets an input value to the ``variable_id`` variable.
**Detailed description**:
ReadValue, Assign, and Variable define a coherent mechanism for reading, writing and
storing a memory buffer between inference calls. More details can be found on the
:doc:`StateAPI<openvino_docs_OV_UG_stateful_models_intro>` documentation page.
*Assign* sets an input value to the ``variable_id`` variable. This value will be read
by the *ReadValue* operation on the next inference call if it has not been reset.
The operation checks if the shape and type specified in ``variable_id`` extend (relax)
the shape and type inferred from the 1st input. If not, it returns an error. For example,
if the type in the variable is specified as dynamic, it means that any type for 1st
input is allowed but if it is specified as f32, only f32 type is allowed.
Only one pair of ReadValue and Assign operations is expected for each Variable in the model.
**Attributes**:
* *variable_id*
* **Description**: identifier of the variable to be updated
* **Range of values**: any non-empty string
* **Type**: string
* **Required**: *yes*
**Inputs**
* **1**: ``new_value`` - input tensor of any supported type. **Required.**
**Outputs**
* **1**: tensor with the same shape and type as ``new_value``.
**Example**
.. code-block:: xml
:force:
<layer ... type="Assign" ...>
<data variable_id="lstm_state_1"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>224</dim>
<dim>224</dim>
</port>
</input>
</layer>

View File

@ -0,0 +1,96 @@
.. {#openvino_docs_ops_infrastructure_ReadValue_6}
ReadValue
=========
.. meta::
:description: Learn about ReadValue-6 - an infrastructure operation, which
can be performed on a single input tensor or without input tensors
to return the value of variable_id.
**Versioned name**: *ReadValue-6*
**Category**: *Infrastructure*
**Short description**: *ReadValue* returns value of the ``variable_id`` variable.
**Detailed description**:
*ReadValue*, *Assign*, and *Variable* define a coherent mechanism for reading, writing,
and storing some memory buffer between inference calls. More details can be found on the
:doc:`StateAPI<openvino_docs_OV_UG_stateful_models_intro>` documentation page.
If the 1st input is provided and this is the first inference or reset has been called,
*ReadValue* returns the value from the 1st input.
If the 1st input is not provided and this is the first inference or reset has been called,
*ReadValue* returns the tensor with the ``variable_shape`` and ``variable_type`` and zero values.
In all other cases *ReadValue* returns the value from the corresponding ``variable_id`` variable.
If the 1st input has been provided, the operation checks if ``variable_shape`` and ``variable_type``
extend (relax) the shape and type inferred from the 1st input. If not, it returns an error.
For example, if ``variable_type`` is specified as dynamic, it means that any type for 1st input
is allowed but if it is specified as f32, only f32 type is allowed.
Only one pair of ReadValue and Assign operations is expected for each Variable in the model.
**Attributes**:
* *variable_id*
* **Description**: identifier of the variable to be read.
* **Range of values**: any non-empty string
* **Type**: string
* **Required**: *yes*
* *variable_type*
* **Description**: the type of the variable
* **Range of values**: : u1, u4, u8, u16, u32, u64, i4, i8, i16, i32, i64, f16, f32, boolean, bf16
* **Type**: ``string``
* **Required**: *yes*
* *variable_shape*
* **Description**: the shape of the variable
* **Range of values**: list of integers, empty list is allowed, which means 0D or scalar tensor
* **Type**: ``int[]``
* **Required**: *yes*
**Inputs**
* **1**: ``init_value`` - input tensor whose values are used in the first inference or after a reset call. **Optional.**
**Outputs**
* **1**: tensor with the same shape and type as specified in *variable_type*, *variable_shape*.
**Example**
.. code-block:: xml
:force:
<layer ... type="ReadValue" ...>
<data variable_id="lstm_state_1" variable_type="f32" variable_shape="1,3,224,224"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>224</dim>
<dim>224</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>224</dim>
<dim>224</dim>
</port>
</output>
</layer>

View File

@ -5,7 +5,7 @@ BatchToSpace
.. meta:: .. meta::
:description: Learn about BatchToSpace-2 - a data movement operation, :description: Learn about BatchToSpace-2 - a data movement operation,
which can be performed on four required input tensors. which can be performed on four required input tensors.
**Versioned name**: *BatchToSpace-2* **Versioned name**: *BatchToSpace-2*
@ -21,25 +21,25 @@ BatchToSpace
1. Reshape ``data`` input to produce a tensor of shape :math:`[B_1, \dots, B_{N - 1}, \frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1, D_2, \dots, D_{N - 1}]` 1. Reshape ``data`` input to produce a tensor of shape :math:`[B_1, \dots, B_{N - 1}, \frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1, D_2, \dots, D_{N - 1}]`
.. math:: .. math::
x^{\prime} = reshape(data, [B_1, \dots, B_{N - 1}, \frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1, D_2, \dots, D_{N - 1}]) x^{\prime} = reshape(data, [B_1, \dots, B_{N - 1}, \frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1, D_2, \dots, D_{N - 1}])
2. Permute dimensions of :math:`x^{\prime}` to produce a tensor of shape :math:`[\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1, B_1, D_2, B_2, \dots, D_{N-1}, B_{N - 1}]` 2. Permute dimensions of :math:`x^{\prime}` to produce a tensor of shape :math:`[\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1, B_1, D_2, B_2, \dots, D_{N-1}, B_{N - 1}]`
.. math:: .. math::
x^{\prime\prime} = transpose(x', [N, N + 1, 0, N + 2, 1, \dots, N + N - 1, N - 1]) x^{\prime\prime} = transpose(x', [N, N + 1, 0, N + 2, 1, \dots, N + N - 1, N - 1])
3. Reshape :math:`x^{\prime\prime}` to produce a tensor of shape :math:`[\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1 \times B_1, D_2 \times B_2, \dots, D_{N - 1} \times B_{N - 1}]` 3. Reshape :math:`x^{\prime\prime}` to produce a tensor of shape :math:`[\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1 \times B_1, D_2 \times B_2, \dots, D_{N - 1} \times B_{N - 1}]`
.. math:: .. math::
x^{\prime\prime\prime} = reshape(x^{\prime\prime}, [\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1 \times B_1, D_2 \times B_2, \dots, D_{N - 1} \times B_{N - 1}]) x^{\prime\prime\prime} = reshape(x^{\prime\prime}, [\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, D_1 \times B_1, D_2 \times B_2, \dots, D_{N - 1} \times B_{N - 1}])
4. Crop the start and end of spatial dimensions of :math:`x^{\prime\prime\prime}` according to ``crops_begin`` and ``crops_end`` inputs to produce the output :math:`y` of shape: 4. Crop the start and end of spatial dimensions of :math:`x^{\prime\prime\prime}` according to ``crops_begin`` and ``crops_end`` inputs to produce the output :math:`y` of shape:
.. math:: .. math::
\left[\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, crop(D_1 \times B_1, CB_1, CE_1), crop(D_2 \times B_2, CB_2, CE_2), \dots , crop(D_{N - 1} \times B_{N - 1}, CB_{N - 1}, CE_{N - 1})\right] \left[\frac{batch}{\left(B_1 \times \dots \times B_{N - 1}\right)}, crop(D_1 \times B_1, CB_1, CE_1), crop(D_2 \times B_2, CB_2, CE_2), \dots , crop(D_{N - 1} \times B_{N - 1}, CB_{N - 1}, CE_{N - 1})\right]
Where Where
@ -80,27 +80,27 @@ Example: 2D input tensor ``data``
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer type="BatchToSpace" ...> <layer type="BatchToSpace" ...>
<input> <input>
<port id="0"> < !-- data --> <port id="0"> <!-- data -->
<dim>10</dim> < !-- batch --> <dim>10</dim> <!-- batch -->
<dim>2</dim> < !-- spatial dimension 1 --> <dim>2</dim> <!-- spatial dimension 1 -->
</port> </port>
<port id="1"> < !-- block_shape value: [1, 5] --> <port id="1"> <!-- block_shape value: [1, 5] -->
<dim>2</dim> <dim>2</dim>
</port> </port>
<port id="2"> < !-- crops_begin value: [0, 2] --> <port id="2"> <!-- crops_begin value: [0, 2] -->
<dim>2</dim> <dim>2</dim>
</port> </port>
<port id="3"> < !-- crops_end value: [0, 0] --> <port id="3"> <!-- crops_end value: [0, 0] -->
<dim>2</dim> <dim>2</dim>
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>2</dim> < !-- data.shape[0] / (block_shape.shape[0] * block_shape.shape[1]) --> <dim>2</dim> <!-- data.shape[0] / (block_shape.shape[0] * block_shape.shape[1]) -->
<dim>8</dim> < !-- data.shape[1] * block_shape.shape[1] - crops_begin[1] - crops_end[1]--> <dim>8</dim> <!-- data.shape[1] * block_shape.shape[1] - crops_begin[1] - crops_end[1]-->
</port> </port>
</output> </output>
</layer> </layer>
@ -109,33 +109,33 @@ Example: 5D input tensor ``data``
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer type="BatchToSpace" ...> <layer type="BatchToSpace" ...>
<input> <input>
<port id="0"> < !-- data --> <port id="0"> <!-- data -->
<dim>48</dim> < !-- batch --> <dim>48</dim> <!-- batch -->
<dim>3</dim> < !-- spatial dimension 1 --> <dim>3</dim> <!-- spatial dimension 1 -->
<dim>3</dim> < !-- spatial dimension 2 --> <dim>3</dim> <!-- spatial dimension 2 -->
<dim>1</dim> < !-- spatial dimension 3 --> <dim>1</dim> <!-- spatial dimension 3 -->
<dim>3</dim> < !-- spatial dimension 4 --> <dim>3</dim> <!-- spatial dimension 4 -->
</port> </port>
<port id="1"> < !-- block_shape value: [1, 2, 4, 3, 1] --> <port id="1"> <!-- block_shape value: [1, 2, 4, 3, 1] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
<port id="2"> < !-- crops_begin value: [0, 0, 1, 0, 0] --> <port id="2"> <!-- crops_begin value: [0, 0, 1, 0, 0] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
<port id="3"> < !-- crops_end value: [0, 0, 1, 0, 0] --> <port id="3"> <!-- crops_end value: [0, 0, 1, 0, 0] -->
<dim>5</dim> <dim>5</dim>
</port> </port>
</input> </input>
<output> <output>
<port id="3"> <port id="3">
<dim>2</dim> < !-- data.shape[0] / (block_shape.shape[0] * block_shape.shape[1] * ... * block_shape.shape[4]) --> <dim>2</dim> <!-- data.shape[0] / (block_shape.shape[0] * block_shape.shape[1] * ... * block_shape.shape[4]) -->
<dim>6</dim> < !-- data.shape[1] * block_shape.shape[1] - crops_begin[1] - crops_end[1]--> <dim>6</dim> <!-- data.shape[1] * block_shape.shape[1] - crops_begin[1] - crops_end[1]-->
<dim>10</dim> < !-- data.shape[2] * block_shape.shape[2] - crops_begin[2] - crops_end[2] --> <dim>10</dim> <!-- data.shape[2] * block_shape.shape[2] - crops_begin[2] - crops_end[2] -->
<dim>3</dim> < !-- data.shape[3] * block_shape.shape[3] - crops_begin[3] - crops_end[3] --> <dim>3</dim> <!-- data.shape[3] * block_shape.shape[3] - crops_begin[3] - crops_end[3] -->
<dim>3</dim> < !-- data.shape[4] * block_shape.shape[4] - crops_begin[4] - crops_end[4] --> <dim>3</dim> <!-- data.shape[4] * block_shape.shape[4] - crops_begin[4] - crops_end[4] -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ Broadcast
.. meta:: .. meta::
:description: Learn about Broadcast-1 - a data movement operation, :description: Learn about Broadcast-1 - a data movement operation,
which can be performed on two required and one optional input tensor. which can be performed on two required and one optional input tensor.
**Versioned name**: *Broadcast-1* **Versioned name**: *Broadcast-1*
@ -53,7 +53,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="numpy"/> <data mode="numpy"/>
<input> <input>
@ -63,9 +63,9 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 16, 50, 50] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
</port> </port>
< !-- the 3rd input shouldn't be provided with mode="numpy" --> <!-- the 3rd input shouldn't be provided with mode="numpy" -->
</input> </input>
<output> <output>
<port id="2"> <port id="2">
@ -76,7 +76,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
</port> </port>
</output> </output>
</layer> </layer>
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="explicit"/> <data mode="explicit"/>
<input> <input>
@ -84,10 +84,10 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>16</dim> <dim>16</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 16, 50, 50] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
</port> </port>
<port id="1"> <port id="1">
<dim>1</dim> < !--The tensor contains 1 elements: [1] --> <dim>1</dim> <!--The tensor contains 1 elements: [1] -->
</port> </port>
</input> </input>
<output> <output>
@ -99,7 +99,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
</port> </port>
</output> </output>
</layer> </layer>
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="explicit"/> <data mode="explicit"/>
<input> <input>
@ -108,10 +108,10 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>50</dim> <dim>50</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 50, 50, 16] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 50, 50, 16] -->
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim> < !--The tensor contains 2 elements: [1, 2] --> <dim>2</dim> <!--The tensor contains 2 elements: [1, 2] -->
</port> </port>
</input> </input>
<output> <output>

View File

@ -5,7 +5,7 @@ Broadcast
.. meta:: .. meta::
:description: Learn about Broadcast-3 - a data movement operation, :description: Learn about Broadcast-3 - a data movement operation,
which can be performed on two required and one optional input tensor. which can be performed on two required and one optional input tensor.
**Versioned name**: *Broadcast-3* **Versioned name**: *Broadcast-3*
@ -61,7 +61,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="numpy"/> <data mode="numpy"/>
<input> <input>
@ -71,9 +71,9 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 16, 50, 50] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
</port> </port>
< !-- the 3rd input shouldn't be provided with mode="numpy" --> <!-- the 3rd input shouldn't be provided with mode="numpy" -->
</input> </input>
<output> <output>
<port id="2"> <port id="2">
@ -84,7 +84,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
</port> </port>
</output> </output>
</layer> </layer>
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="explicit"/> <data mode="explicit"/>
<input> <input>
@ -92,10 +92,10 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>16</dim> <dim>16</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 16, 50, 50] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
</port> </port>
<port id="1"> <port id="1">
<dim>1</dim> < !--The tensor contains 1 elements: [1] --> <dim>1</dim> <!--The tensor contains 1 elements: [1] -->
</port> </port>
</input> </input>
<output> <output>
@ -107,7 +107,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
</port> </port>
</output> </output>
</layer> </layer>
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="explicit"/> <data mode="explicit"/>
<input> <input>
@ -116,10 +116,10 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>50</dim> <dim>50</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 50, 50, 16] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 50, 50, 16] -->
</port> </port>
<port id="1"> <port id="1">
<dim>2</dim> < !--The tensor contains 2 elements: [1, 2] --> <dim>2</dim> <!--The tensor contains 2 elements: [1, 2] -->
</port> </port>
</input> </input>
<output> <output>
@ -131,7 +131,7 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
</port> </port>
</output> </output>
</layer> </layer>
<layer ... type="Broadcast" ...> <layer ... type="Broadcast" ...>
<data mode="bidirectional"/> <data mode="bidirectional"/>
<input> <input>
@ -141,9 +141,9 @@ For example, ``axes_mapping = [1]`` enables broadcasting of a tensor with shape
<dim>1</dim> <dim>1</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>4</dim> < !--The tensor contains 4 elements: [1, 1, 50, 50] --> <dim>4</dim> <!--The tensor contains 4 elements: [1, 1, 50, 50] -->
</port> </port>
< !-- the 3rd input shouldn't be provided with mode="bidirectional" --> <!-- the 3rd input shouldn't be provided with mode="bidirectional" -->
</input> </input>
<output> <output>
<port id="2"> <port id="2">

View File

@ -5,7 +5,7 @@ Concat
.. meta:: .. meta::
:description: Learn about Concat-1 - a data movement operation, :description: Learn about Concat-1 - a data movement operation,
which can be performed on arbitrary number of input tensors. which can be performed on arbitrary number of input tensors.
**Versioned name**: *Concat-1* **Versioned name**: *Concat-1*
@ -39,25 +39,25 @@ Concat
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer id="1" type="Concat"> <layer id="1" type="Concat">
<data axis="1" /> <data axis="1" />
<input> <input>
<port id="0"> <port id="0">
<dim>1</dim> <dim>1</dim>
<dim>8</dim> < !-- axis for concatenation --> <dim>8</dim> <!-- axis for concatenation -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>1</dim> <dim>1</dim>
<dim>16</dim> < !-- axis for concatenation --> <dim>16</dim> <!-- axis for concatenation -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
<port id="2"> <port id="2">
<dim>1</dim> <dim>1</dim>
<dim>32</dim> < !-- axis for concatenation --> <dim>32</dim> <!-- axis for concatenation -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
@ -65,7 +65,7 @@ Concat
<output> <output>
<port id="0"> <port id="0">
<dim>1</dim> <dim>1</dim>
<dim>56</dim> < !-- concatenated axis: 8 + 16 + 32 = 48 --> <dim>56</dim> <!-- concatenated axis: 8 + 16 + 32 = 48 -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
@ -75,25 +75,25 @@ Concat
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer id="1" type="Concat"> <layer id="1" type="Concat">
<data axis="-3" /> <data axis="-3" />
<input> <input>
<port id="0"> <port id="0">
<dim>1</dim> <dim>1</dim>
<dim>8</dim> < !-- axis for concatenation --> <dim>8</dim> <!-- axis for concatenation -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
<port id="1"> <port id="1">
<dim>1</dim> <dim>1</dim>
<dim>16</dim> < !-- axis for concatenation --> <dim>16</dim> <!-- axis for concatenation -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
<port id="2"> <port id="2">
<dim>1</dim> <dim>1</dim>
<dim>32</dim> < !-- axis for concatenation --> <dim>32</dim> <!-- axis for concatenation -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>
@ -101,7 +101,7 @@ Concat
<output> <output>
<port id="0"> <port id="0">
<dim>1</dim> <dim>1</dim>
<dim>56</dim> < !-- concatenated axis: 8 + 16 + 32 = 48 --> <dim>56</dim> <!-- concatenated axis: 8 + 16 + 32 = 48 -->
<dim>50</dim> <dim>50</dim>
<dim>50</dim> <dim>50</dim>
</port> </port>

View File

@ -5,7 +5,7 @@ DepthToSpace
.. meta:: .. meta::
:description: Learn about DepthToSpace-1 - a data movement operation, :description: Learn about DepthToSpace-1 - a data movement operation,
which can be performed on a single input tensor. which can be performed on a single input tensor.
**Versioned name**: *DepthToSpace-1* **Versioned name**: *DepthToSpace-1*
@ -21,7 +21,7 @@ DepthToSpace
The operation is equivalent to the following transformation of the input tensor ``data`` with ``K`` spatial dimensions of shape ``[N, C, D1, D2, ..., DK]`` to *Y* output tensor. If ``mode = blocks_first``: The operation is equivalent to the following transformation of the input tensor ``data`` with ``K`` spatial dimensions of shape ``[N, C, D1, D2, ..., DK]`` to *Y* output tensor. If ``mode = blocks_first``:
.. code-block:: cpp .. code-block:: cpp
x' = reshape(data, [N, block_size, block_size, ..., block_size, C / (block_size ^ K), D1, D2, ..., DK]) x' = reshape(data, [N, block_size, block_size, ..., block_size, C / (block_size ^ K), D1, D2, ..., DK])
x'' = transpose(x', [0, K + 1, K + 2, 1, K + 3, 2, K + 4, 3, ..., K + (K + 1), K]) x'' = transpose(x', [0, K + 1, K + 2, 1, K + 3, 2, K + 4, 3, ..., K + (K + 1), K])
y = reshape(x'', [N, C / (block_size ^ K), D1 * block_size, D2 * block_size, D3 * block_size, ..., DK * block_size]) y = reshape(x'', [N, C / (block_size ^ K), D1 * block_size, D2 * block_size, D3 * block_size, ..., DK * block_size])
@ -29,7 +29,7 @@ The operation is equivalent to the following transformation of the input tensor
If ``mode = depth_first``: If ``mode = depth_first``:
.. code-block:: cpp .. code-block:: cpp
x' = reshape(data, [N, C / (block_size ^ K), block_size, block_size, ..., block_size, D1, D2, ..., DK]) x' = reshape(data, [N, C / (block_size ^ K), block_size, block_size, ..., block_size, D1, D2, ..., DK])
x'' = transpose(x', [0, 1, K + 2, 2, K + 3, 3, K + 4, 4, ..., K + (K + 1), K + 1]) x'' = transpose(x', [0, 1, K + 2, 2, K + 3, 3, K + 4, 4, ..., K + (K + 1), K + 1])
y = reshape(x'', [N, C / (block_size ^ K), D1 * block_size, D2 * block_size, D3 * block_size, ..., DK * block_size]) y = reshape(x'', [N, C / (block_size ^ K), D1 * block_size, D2 * block_size, D3 * block_size, ..., DK * block_size])
@ -70,7 +70,7 @@ If ``mode = depth_first``:
.. code-block:: xml .. code-block:: xml
:force: :force:
<layer type="DepthToSpace" ...> <layer type="DepthToSpace" ...>
<data block_size="2" mode="blocks_first"/> <data block_size="2" mode="blocks_first"/>
<input> <input>
@ -83,10 +83,10 @@ If ``mode = depth_first``:
</input> </input>
<output> <output>
<port id="1"> <port id="1">
<dim>5</dim> < !-- data.shape[0] --> <dim>5</dim> <!-- data.shape[0] -->
<dim>7</dim> < !-- data.shape[1] / (block_size ^ 2) --> <dim>7</dim> <!-- data.shape[1] / (block_size ^ 2) -->
<dim>4</dim> < !-- data.shape[2] * block_size --> <dim>4</dim> <!-- data.shape[2] * block_size -->
<dim>6</dim> < !-- data.shape[3] * block_size --> <dim>6</dim> <!-- data.shape[3] * block_size -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,14 +5,14 @@ Gather
.. meta:: .. meta::
:description: Learn about Gather-1 - a data movement operation, :description: Learn about Gather-1 - a data movement operation,
which can be performed on three required input tensors. which can be performed on three required input tensors.
**Versioned name:** *Gather-1* **Versioned name:** *Gather-1*
**Category:** *Data movement* **Category:** *Data movement*
**Short description:** *Gather* operation takes slices of data in the first input tensor according **Short description:** *Gather* operation takes slices of data in the first input tensor according
to the indices specified in the second input tensor and axis from the third input. to the indices specified in the second input tensor and axis from the third input.
**Detailed description** **Detailed description**
@ -30,13 +30,13 @@ Where ``axis`` is the value from the third input.
* **1**: Tensor with arbitrary data. **Required.** * **1**: Tensor with arbitrary data. **Required.**
* **2**: Tensor with indices to gather. The values for indices are in the range ``[0, input1[axis] - 1]``. **Required.** * **2**: Tensor with indices to gather. The values for indices are in the range ``[0, input1[axis] - 1]``. **Required.**
* **3**: Scalar or 1D tensor *axis* is a dimension index to gather data from. For example, *axis* equal * **3**: Scalar or 1D tensor *axis* is a dimension index to gather data from. For example, *axis* equal
to 1 means that gathering is performed over the first dimension. Negative value means reverse indexing. to 1 means that gathering is performed over the first dimension. Negative value means reverse indexing.
Allowed values are from ``[-len(input1.shape), len(input1.shape) - 1]``. **Required.** Allowed values are from ``[-len(input1.shape), len(input1.shape) - 1]``. **Required.**
**Outputs** **Outputs**
* **1**: The resulting tensor that consists of elements from the first input tensor gathered by indices * **1**: The resulting tensor that consists of elements from the first input tensor gathered by indices
from the second input tensor. Shape of the tensor is ``[input1.shape[:axis], input2.shape, input1.shape[axis + 1:]]`` from the second input tensor. Shape of the tensor is ``[input1.shape[:axis], input2.shape, input1.shape[axis + 1:]]``
**Example** **Example**
@ -58,17 +58,17 @@ Where ``axis`` is the value from the third input.
<dim>20</dim> <dim>20</dim>
<dim>28</dim> <dim>28</dim>
</port> </port>
<port id="2"/> < !-- axis = 1 --> <port id="2"/> <!-- axis = 1 -->
</input> </input>
<output> <output>
<port id="2"> <port id="2">
<dim>6</dim> < !-- embedded dimension from the 1st input --> <dim>6</dim> <!-- embedded dimension from the 1st input -->
<dim>15</dim> < !-- embedded dimension from the 2nd input --> <dim>15</dim> <!-- embedded dimension from the 2nd input -->
<dim>4</dim> < !-- embedded dimension from the 2nd input --> <dim>4</dim> <!-- embedded dimension from the 2nd input -->
<dim>20</dim> < !-- embedded dimension from the 2nd input --> <dim>20</dim> <!-- embedded dimension from the 2nd input -->
<dim>28</dim> < !-- embedded dimension from the 2nd input --> <dim>28</dim> <!-- embedded dimension from the 2nd input -->
<dim>10</dim> < !-- embedded dimension from the 1st input --> <dim>10</dim> <!-- embedded dimension from the 1st input -->
<dim>24</dim> < !-- embedded dimension from the 1st input --> <dim>24</dim> <!-- embedded dimension from the 1st input -->
</port> </port>
</output> </output>
</layer> </layer>

View File

@ -5,7 +5,7 @@ Gather
.. meta:: .. meta::
:description: Learn about Gather-7 - a data movement operation, :description: Learn about Gather-7 - a data movement operation,
which can be performed on three required input tensors. which can be performed on three required input tensors.
**Versioned name**: *Gather-7* **Versioned name**: *Gather-7*
@ -29,12 +29,12 @@ the number of batch dimensions. ``N`` and ``M`` are numbers of dimensions of ``d
**Attributes**: **Attributes**:
* *batch_dims* * *batch_dims*
* **Description**: *batch_dims* (also denoted as ``b``) is a leading number of dimensions of ``data`` * **Description**: *batch_dims* (also denoted as ``b``) is a leading number of dimensions of ``data``
tensor and ``indices`` representing the batches, and *Gather* starts to gather from the ``b`` tensor and ``indices`` representing the batches, and *Gather* starts to gather from the ``b``
dimension. It requires the first ``b`` dimensions in `data` and `indices` tensors to be equal. dimension. It requires the first ``b`` dimensions in `data` and `indices` tensors to be equal.
If ``batch_dims`` is less than zero, the normalized value is used ``batch_dims = indices.rank + batch_dims``. If ``batch_dims`` is less than zero, the normalized value is used ``batch_dims = indices.rank + batch_dims``.
* **Range of values**: ``[-min(data.rank, indices.rank); min(data.rank, indices.rank)]`` and * **Range of values**: ``[-min(data.rank, indices.rank); min(data.rank, indices.rank)]`` and
``batch_dims' <= axis'``. Where ``batch_dims'`` and ``axis'`` stand for normalized ``batch_dims`` and ``axis`` values. ``batch_dims' <= axis'``. Where ``batch_dims'`` and ``axis'`` stand for normalized ``batch_dims`` and ``axis`` values.
* **Type**: *T_AXIS* * **Type**: *T_AXIS*
* **Default value**: 0 * **Default value**: 0
@ -46,7 +46,7 @@ Example 1 with default *batch_dims* value:
batch_dims = 0 batch_dims = 0
axis = 0 axis = 0
indices = [0, 0, 4] indices = [0, 0, 4]
data = [1, 2, 3, 4, 5] data = [1, 2, 3, 4, 5]
output = [1, 1, 5] output = [1, 1, 5]
@ -58,15 +58,15 @@ Example 2 with non-default *batch_dims* value:
batch_dims = 1 batch_dims = 1
axis = 1 axis = 1
indices = [[0, 0, 4], <-- this is applied to the first batch indices = [[0, 0, 4], <-- this is applied to the first batch
[4, 0, 0]] <-- this is applied to the second batch [4, 0, 0]] <-- this is applied to the second batch
indices_shape = (2, 3) indices_shape = (2, 3)
data = [[1, 2, 3, 4, 5], <-- the first batch data = [[1, 2, 3, 4, 5], <-- the first batch
[6, 7, 8, 9, 10]] <-- the second batch [6, 7, 8, 9, 10]] <-- the second batch
data_shape = (2, 5) data_shape = (2, 5)
output = [[ 1, 1, 5], output = [[ 1, 1, 5],
[10, 6, 6]] [10, 6, 6]]
output_shape = (2, 3) output_shape = (2, 3)
@ -78,24 +78,24 @@ Example 3 with non-default *batch_dims* value:
batch_dims = 2 batch_dims = 2
axis = 2 axis = 2
indices = [[[0, 0, 4], <-- this is applied to the first batch, index = (0, 0) indices = [[[0, 0, 4], <-- this is applied to the first batch, index = (0, 0)
[4, 0, 0]], <-- this is applied to the second batch, index = (0, 1) [4, 0, 0]], <-- this is applied to the second batch, index = (0, 1)
[[1, 2, 4], <-- this is applied to the third batch, index = (1, 0) [[1, 2, 4], <-- this is applied to the third batch, index = (1, 0)
[4, 3, 2]]] <-- this is applied to the fourth batch, index = (1, 1) [4, 3, 2]]] <-- this is applied to the fourth batch, index = (1, 1)
indices_shape = (2, 2, 3) indices_shape = (2, 2, 3)
data = [[[1, 2, 3, 4, 5], <-- the first batch, index = (0, 0) data = [[[1, 2, 3, 4, 5], <-- the first batch, index = (0, 0)
[6, 7, 8, 9, 10]], <-- the second batch, index = (0, 1) [6, 7, 8, 9, 10]], <-- the second batch, index = (0, 1)
[[11, 12, 13, 14, 15], <-- the third batch, index = (1, 0) [[11, 12, 13, 14, 15], <-- the third batch, index = (1, 0)
[16, 17, 18, 19, 20]]] <-- the fourth batch, index = (1, 1) [16, 17, 18, 19, 20]]] <-- the fourth batch, index = (1, 1)
data_shape = (2, 2, 5) data_shape = (2, 2, 5)
output = [[[ 1, 1, 5], output = [[[ 1, 1, 5],
[10, 6, 6]], [10, 6, 6]],
[[12, 13, 15], [[12, 13, 15],
[20, 19, 18]]] [20, 19, 18]]]
output_shape = (2, 2, 3) output_shape = (2, 2, 3)
@ -106,28 +106,28 @@ Example 4 with *axis* > *batch_dims*:
batch_dims = 1 batch_dims = 1
axis = 2 axis = 2
indices = [[1, 2, 4], <-- this is applied to the first batch indices = [[1, 2, 4], <-- this is applied to the first batch
[4, 3, 2]] <-- this is applied to the second batch [4, 3, 2]] <-- this is applied to the second batch
indices_shape = (2, 3) indices_shape = (2, 3)
data = [[[[ 1, 2, 3, 4], <-- first batch data = [[[[ 1, 2, 3, 4], <-- first batch
[ 5, 6, 7, 8], [ 5, 6, 7, 8],
[ 9, 10, 11, 12], [ 9, 10, 11, 12],
[13, 14, 15, 16], [13, 14, 15, 16],
[17, 18, 19, 20]]], [17, 18, 19, 20]]],
[[[21, 22, 23, 24], <-- second batch [[[21, 22, 23, 24], <-- second batch
[25, 26, 27, 28], [25, 26, 27, 28],
[29, 30, 31, 32], [29, 30, 31, 32],
[33, 34, 35, 36], [33, 34, 35, 36],
[37, 38, 39, 40]]]] [37, 38, 39, 40]]]]
data_shape = (2, 1, 5, 4) data_shape = (2, 1, 5, 4)
output = [[[[ 5, 6, 7, 8], output = [[[[ 5, 6, 7, 8],
[ 9, 10, 11, 12], [ 9, 10, 11, 12],
[17, 18, 19, 20]]], [17, 18, 19, 20]]],
[[[37, 38, 39, 40], [[[37, 38, 39, 40],
[33, 34, 35, 36], [33, 34, 35, 36],
[29, 30, 31, 32]]]] [29, 30, 31, 32]]]]
@ -140,15 +140,15 @@ Example 5 with negative *batch_dims* value:
batch_dims = -1 <-- normalized value will be indices.rank + batch_dims = 2 - 1 = 1 batch_dims = -1 <-- normalized value will be indices.rank + batch_dims = 2 - 1 = 1
axis = 1 axis = 1
indices = [[0, 0, 4], <-- this is applied to the first batch indices = [[0, 0, 4], <-- this is applied to the first batch
[4, 0, 0]] <-- this is applied to the second batch [4, 0, 0]] <-- this is applied to the second batch
indices_shape = (2, 3) indices_shape = (2, 3)
data = [[1, 2, 3, 4, 5], <-- the first batch data = [[1, 2, 3, 4, 5], <-- the first batch
[6, 7, 8, 9, 10]] <-- the second batch [6, 7, 8, 9, 10]] <-- the second batch
data_shape = (2, 5) data_shape = (2, 5)
output = [[ 1, 1, 5], output = [[ 1, 1, 5],
[10, 6, 6]] [10, 6, 6]]
output_shape = (2, 3) output_shape = (2, 3)
@ -167,7 +167,7 @@ Example 5 with negative *batch_dims* value:
**Outputs** **Outputs**
* **1**: The resulting tensor of type *T* that consists of elements from ``data`` tensor gathered by ``indices``. * **1**: The resulting tensor of type *T* that consists of elements from ``data`` tensor gathered by ``indices``.
The shape of the output tensor is ``data.shape[:axis] + indices.shape[batch_dims:] + data.shape[axis + 1:]`` The shape of the output tensor is ``data.shape[:axis] + indices.shape[batch_dims:] + data.shape[axis + 1:]``
**Types** **Types**
@ -193,7 +193,7 @@ Example 5 with negative *batch_dims* value:
<dim>32</dim> <dim>32</dim>
<dim>21</dim> <dim>21</dim>
</port> </port>
<port id="2"/> < !-- axis = 1 --> <port id="2"/> <!-- axis = 1 -->
</input> </input>
<output> <output>
<port id="2"> <port id="2">

Some files were not shown because too many files have changed in this diff Show More