[DOC] Conditional compilation developer guide review (#14340)
* conditional compilation review * additional fixes * apply comment from review
This commit is contained in:
Tatiana Savina
GitHub
parent
1af95fb3f8
commit
b76168b32b
|
|
@ -9,21 +9,21 @@
|
|||
|
||||
## Introduction
|
||||
|
||||
Conditional compilation can significantly reduce OpenVINO™ binaries size by excluding unnecessary components for particular models inference.
|
||||
Conditional compilation can significantly reduce OpenVINO™ binaries size by excluding unnecessary components for particular model inference.
|
||||
The following components can be excluded from the build:
|
||||
* Openrations and graph transformations in OpenVINO and plugins
|
||||
* jit kernels in a CPU plugin
|
||||
* arbitrary code that is not used for particular model inference
|
||||
* Operations and graph transformations in OpenVINO and plugins,
|
||||
* jit kernels in a CPU plugin,
|
||||
* arbitrary code that is not used for particular model inference.
|
||||
|
||||
However, conditional compilation has a significant drawback - **the resulting OpenVINO Runtime will work only with a limited set of models and devices.**
|
||||
However, the conditional compilation has a significant drawback - **the resulting OpenVINO Runtime will work only with a limited set of models and devices.**
|
||||
|
||||
There are 2 conditional compilation build stages: `SELECTIVE_BUILD=COLLECT` and `SELECTIVE_BUILD=ON`.
|
||||
There are two conditional compilation build stages: `SELECTIVE_BUILD=COLLECT` and `SELECTIVE_BUILD=ON`.
|
||||
|
||||
`SELECTIVE_BUILD=COLLECT` enables analyzed mode for annotated code regions, it can provide capability with the help of itt to collect some statistic data. Once this stage is completed, user need run openvino sample(such as benchmark_app) to generate actual statistic data and run `itt_collector tool` to dump them into cvs files. This statge can be enabled during build OpenVINO with options `-DSELECTIVE_BUILD=COLLECT -DENABLE_PROFILING_ITT=ON`
|
||||
`SELECTIVE_BUILD=COLLECT` enables analysis mode for annotated code regions. It can be used to collect statistics data using `itt`. Once this stage is completed, run an OpenVINO sample, such as benchmark_app, to generate actual statistic data. Then run `itt_collector tool` to dump the data into cvs files. This statge can be enabled during the OpenVINO build with the options `-DSELECTIVE_BUILD=COLLECT -DENABLE_PROFILING_ITT=ON`.
|
||||
|
||||
`SELECTIVE_BUILD=ON` excludes all annotated inactive code region out of compiled binaries with help of `cvs files` generated by `itt_collector tool`. It is done by a header file which generated from the `cvs files`. This stage need build OpenVINO with option `-DSELECTIVE_BUILD=ON -DENABLE_PROFILING_ITT=OFF -DSELECTIVE_BUILD_STAT=<cc_data.csv>`. Notice: `-DENABLE_PROFILING_ITT=OFF` is not must, if user wants to have ITT counters in final application, he can set it ON: `-DENABLE_PROFILING_ITT=ON`.
|
||||
`SELECTIVE_BUILD=ON` excludes all inactive annotated code regions from the compiled binaries using `cvs files` generated by the `itt_collector tool`. It is done by a header file which generated from the `cvs files`. This stage need build OpenVINO with option `-DSELECTIVE_BUILD=ON -DENABLE_PROFILING_ITT=OFF -DSELECTIVE_BUILD_STAT=<cc_data.csv>`. Notice: `-DENABLE_PROFILING_ITT=OFF` is not must, if user wants to have ITT counters in final application, he can set it ON: `-DENABLE_PROFILING_ITT=ON`.
|
||||
|
||||
Note: If above options are not enabled, conditional compilation will be OFF and default behavior is kept, all features of OpenVINO are enabled. You can ignore `SELECTIVE_BUILD` or set option `-DSELECTIVE_BUILD=OFF`
|
||||
> **NOTE**: If the above options are not enabled, conditional compilation will be OFF and the default behavior is kept, all OpenVINO features are enabled. You can ignore `SELECTIVE_BUILD` or set option `-DSELECTIVE_BUILD=OFF`.
|
||||
|
||||
### Setup environment
|
||||
|
||||
|
|
|
|||
|
|
@ -1,22 +1,21 @@
|
|||
# OpenVINO Conditional Compilation
|
||||
|
||||
|
||||
OpenVINO Conditional Compilation(CC) is a very useful feature for the scenarios that is senstive for binaries size, it can significantly optimize OpenVINO™ binaries size by excluding unnecessary code regions by the help of ITT profiler, especially when build application with static OpenVINO package.
|
||||
OpenVINO Conditional Compilation(CC) feature can significantly optimize OpenVINO™ binaries size by excluding unnecessary code regions with ITT profiler, especially when building an application with a static OpenVINO package.
|
||||
|
||||
## Key contacts
|
||||
|
||||
People from [openvino-ie-maintainers]https://github.com/orgs/openvinotoolkit/teams/openvino-ie-maintainers group have the rights to approve and merge PRs related with conditional compilation.
|
||||
People from [openvino-ie-maintainers](https://github.com/orgs/openvinotoolkit/teams/openvino-ie-maintainers) group have the rights to approve and merge PRs related to conditional compilation.
|
||||
|
||||
## Components
|
||||
|
||||
* [docs](./docs/) contains documentation for conditional compilation.
|
||||
* [include](./include/) contains header files that define class and macros for conditional compilation.
|
||||
* [scripts](./scripts/) script tools that used in condition compilation.
|
||||
* [scripts](./scripts/) script tools used in conditional compilation.
|
||||
|
||||
## Tutorials
|
||||
|
||||
* [Conditional Compilation Introduction](../../../docs/dev/conditional_compilation.md)
|
||||
* [Develop CC for New Components](./docs/develop_cc_for_new_component.md)
|
||||
* [Develop Conditional Compilation for New Components](./docs/develop_cc_for_new_component.md)
|
||||
|
||||
## How to contribute
|
||||
|
||||
|
|
|
|||
|
|
@ -2,14 +2,13 @@
|
|||
|
||||
## Contents
|
||||
|
||||
- [Introduce conditional compilation macro](#Introduction-of-conditional-compilation-macro)
|
||||
- [Enable conditional complication for a new component or domain](#building-for-different-models)
|
||||
- [How to verify the new conditional compilation feature](#building-for-devices-with-different-isa)
|
||||
|
||||
- [Introduce conditional compilation macro](#introduce-conditional-compilation-macro)
|
||||
- [Enable conditional compilation for a new component or domain](#enable-conditional-compilation-for-a-new-component-or-domain)
|
||||
- [Verify the new conditional compilation feature](#verify-the-new-conditional-compilation-feature)
|
||||
|
||||
## Introduce conditional compilation macro
|
||||
|
||||
There are several macros to help enable conditional compilation(CC) feature for new component, and in each CC mode(SELECTIVE_BUILD_ANALYZER and SELECTIVE_BUILD) they must be defined with same name but difference macro content. Developer can apply or reference these macros for own component's conditional compilation enablement. You can find these macros in [selective_buld.h](https://github.com/openvinotoolkit/openvino/blob/master/src/common/conditional_compilation/include/openvino/cc/selective_build.h) and [itt.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp).
|
||||
There are several macros to help enable the conditional compilation (CC) feature for a new component. They must be defined with the same name but different macro content in each CC mode (SELECTIVE BUILD ANALYZER and SELECTIVE BUILD). You can apply or reference these macros to enable conditional compilation for your own component. You can find these macros in [selective_buld.h](https://github.com/openvinotoolkit/openvino/blob/master/src/common/conditional_compilation/include/openvino/cc/selective_build.h) and [itt.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp).
|
||||
|
||||
|
||||
| Macros | SELECTIVE_BUILD_ANALYZER | SELECTIVE_BUILD | Off Conditional Compilation |
|
||||
|
|
@ -23,49 +22,50 @@ There are several macros to help enable conditional compilation(CC) feature for
|
|||
|
||||
### 1. Macro in SELECTIVE_BUILD_ANALYZER mode
|
||||
#### OV_CC_DOMAINS
|
||||
It defines domains for conditional compilation, in which contains 3 macros to define detail domains(simple scope, switch/cases and factories). The implement code is:
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/713eb9683f67a5eb07374e96b7bbbfcd971ca69e/src/common/conditional_compilation/include/openvino/cc/selective_build.h#L113-L116
|
||||
It defines domains for conditional compilation, which contains three macros to define detail domains (simple scope, switch/cases, and factories).
|
||||
|
||||
Notice: macro `OV_PP_CAT` will connect 2 symbols together, and macro `OV_ITT_DOMAIN` is used to declare domain with a given name for ITT profiling.
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/713eb9683f67a5eb07374e96b7bbbfcd971ca69e/src/common/conditional_compilation/include/openvino/cc/selective_build.h#L113-L116)
|
||||
|
||||
> **NOTE**: macro `OV_PP_CAT` connects two symbols together, and macro `OV_ITT_DOMAIN` is used to declare a domain with a given name for ITT profiling.
|
||||
|
||||
#### OV_SCOPE
|
||||
It will leverage `OV_ITT_SCOPED_TASK` to help annotate code region until scope exit, which will be profiled whether need to set to be active code region or not, the active code region will generate a macro definition with prefix of “SIMPLE_” in the header file to make it can be built in `SELECTIVE_BUILD` mode, while the inactive code region will be excluded in the following build.
|
||||
It leverages `OV_ITT_SCOPED_TASK` to help annotate a code region until the scope exit, which will be profiled depending on whether the active code area needs to be set or not. The active code region will generate a macro definition with the prefix “SIMPLE_” in the header file so that it can be built in the `SELECTIVE_BUILD` mode, while the inactive code region will be excluded in the following build.
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/selective_build.h#L159-L160
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/selective_build.h#L159-L160)
|
||||
|
||||
|
||||
#### OV_SWITCH
|
||||
It is mainly used for template class to verify specified input parameter whether match template parameter types. It will check each template parameter types (`Cases`) and return true if is matched or return false if not match any one. If matched, it will be labeled as active code region. The `Cases` can be defined by macro *OV_CASE*, which is a `case_wrapper` structure to store data type and its value. Most of case, `OV_SWITCH` will combine `OV_CASE` to work together, there is a simple example:
|
||||
It is mainly used for the template class to verify whether the specified input parameter matches template parameter types. It checks each template parameter type (`Cases`) and returns true if it matches or false if there is no match. If matched, it is labeled as an active code region. The `Cases` can be defined by macro *OV_CASE*, which is a `case_wrapper` structure to store data type and its value. In most cases, `OV_SWITCH` combines `OV_CASE` to work together:
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/plugins/intel_cpu/src/nodes/one_hot.cpp#L158-L161
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/plugins/intel_cpu/src/nodes/one_hot.cpp#L158-L161)
|
||||
|
||||
And there is another macro named **OV_CASE2**, which can support 2 pairs of type and value.
|
||||
There is another macro named **OV_CASE2**, which can support two pairs of type and value.
|
||||
|
||||
#### MATCHER_SCOPE
|
||||
It defines a string to represent the matcher name in this region.
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp#L21-L21
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp#L21-L21)
|
||||
|
||||
|
||||
#### RUN_ON_FUNCTION_SCOPE and RUN_ON_MODEL_SCOPE
|
||||
They are very similar except for different region suffix, one is run on function, another is run on model. Both them leverage OV_SCOPE with ov_pass as its module name.
|
||||
They are very similar except for different region suffix. One is run on function, and another is run on model. Both of them leverage OV_SCOPE with ov_pass as its module name.
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp#L20-L22
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp#L20-L22)
|
||||
|
||||
|
||||
### 2. Macro in SELECTIVE_BUILD mode
|
||||
|
||||
#### OV_CC_DOMAINS
|
||||
It is empty in this mode.
|
||||
Empty in this mode.
|
||||
|
||||
#### OV_CC_SCOPE_IS_ENABLED
|
||||
It checks whether the condition is true or false, it always is embedded in **OV_SCOPE** or **MATCHER_SCOPE** to decide whether the following code region need be excluded out of final binaries.
|
||||
It checks whether the condition is true or false. It is always embedded in **OV_SCOPE** or **MATCHER_SCOPE** to decide whether the following code region needs to be excluded from the final binaries.
|
||||
|
||||
#### OV_SCOPE
|
||||
It will check whether the code region in this module is active or inactive by the macros in header file(`conditional_compilation_gen.h`) generated in previous analyzed mode. The generated macros name format is <module>_<region>, in which module name is passed by the parameters of OV_SCOPE.
|
||||
It checks whether the code region in this module is active or inactive by the macros in the header file(`conditional_compilation_gen.h`) generated in the previous analyzed mode. The generated macros name format is <module>_<region>, in which the module name is passed by the parameters of OV_SCOPE.
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/selective_build.h#L181-L183
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/selective_build.h#L181-L183)
|
||||
|
||||
There is an example of `conditional_compilation_gen.h`:
|
||||
```
|
||||
|
|
@ -92,68 +92,68 @@ There is an example of `conditional_compilation_gen.h`:
|
|||
```
|
||||
|
||||
#### OV_SWITCH
|
||||
It will decide whether the template class parameter match operation need be excluded from final binaries. If the match operation is false in analyzed mode, the code region will be excluded in SELECTIVE_BUILD mode.
|
||||
It decides whether the template class parameter match operation needs to be excluded from the final binaries. If the match operation is false in analyzed mode, the code region will be excluded in SELECTIVE_BUILD mode.
|
||||
|
||||
#### OV_CASE and OV_CASE2
|
||||
They are used with OV_SWITCH to provide match cases.
|
||||
|
||||
#### MATCHER_SCOPE
|
||||
It defines a string to represent the matcher name in this region, and then decide whether the code region is inactivated and excluded from final binaries, according to the OV_SCOPE result in analyzed mode.
|
||||
It defines a string to represent the matcher name in this region. Then decides whether the code region is inactivated and excluded from the final binaries according to the OV_SCOPE result in analyzed mode.
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp#L30-L33
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/34b76584f724ad07ed27c8ea27777ac46df92c23/src/common/conditional_compilation/include/openvino/cc/pass/itt.hpp#L30-L33)
|
||||
|
||||
#### RUN_ON_FUNCTION_SCOPE and RUN_ON_MODEL_SCOPE
|
||||
They are very similar, and functionality is same with MATCHER_SCOPE plus throw error.
|
||||
They are very similar, and functionality are the same with MATCHER_SCOPE plus throw error.
|
||||
|
||||
### 3. Non Conditional Compilation
|
||||
It means that conditional compilation is disabled.
|
||||
|
||||
#### OV_CC_DOMAINS
|
||||
It is empty in this mode
|
||||
Empty in this mode.
|
||||
|
||||
#### OV_SCOPE
|
||||
It is empty in this mode
|
||||
Empty in this mode.
|
||||
|
||||
#### OV_SCOPE and OV_CASE
|
||||
They will do match operation without itt scope task.
|
||||
They do the match operation without the itt scope task.
|
||||
|
||||
#### MATCHER_SCOPE
|
||||
It is empty.
|
||||
Empty.
|
||||
|
||||
#### RUN_ON_FUNCTION_SCOPE and RUN_ON_MODEL_SCOPE
|
||||
They are empty.
|
||||
Empty.
|
||||
|
||||
## Enable conditional compilation for a new component or domain
|
||||
|
||||
## Enable conditional complication for a new component or domain
|
||||
### step 1: requirement analyze
|
||||
Analyze which code region need conditional complication and figure out requirement for conditional complication.
|
||||
### Step 1: analyze requirements
|
||||
|
||||
### step 2: create itt.hpp if possible
|
||||
Create itt.hpp file in the target component directory, in which you need define some particular macro needed by conditional compilation if existed macro cannot meet requirement. An example [itt.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/core/src/itt.hpp).
|
||||
Analyze which code region need conditional compilation and figure out requirements for conditional compilation.
|
||||
|
||||
- Add macro into itt.hpp to define new CC domain:
|
||||
### Step 2: create itt.hpp if possible
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/be277ab9772de827739ccf960bf7cebf1c6f06b0/src/core/src/itt.hpp#L12-L28
|
||||
|
||||
- Define new macro according to the actual requirement by target component
|
||||
The same macro should be defined in the 3 modes separately, below is an example:
|
||||
Create itt.hpp file in the target component directory, in which you need to define a particular macro needed by the conditional compilation, if the existing macro cannot meet the requirement. An example [itt.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/core/src/itt.hpp).
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/be277ab9772de827739ccf960bf7cebf1c6f06b0/src/core/src/itt.hpp#L34-L57
|
||||
- Add macro into itt.hpp to define a new CC domain:
|
||||
|
||||
[Open the code](https://github.com/openvinotoolkit/openvino/blob/be277ab9772de827739ccf960bf7cebf1c6f06b0/src/core/src/itt.hpp#L12-L28)
|
||||
|
||||
- Define a new macro according to the requirement of the target component. The same macro should be defined in the three modes separately.
|
||||
|
||||
### step 3: apply CC macros
|
||||
Apply the CC macros into your code region that need conditional compilation.
|
||||
An example:
|
||||
[Open the code example](https://github.com/openvinotoolkit/openvino/blob/be277ab9772de827739ccf960bf7cebf1c6f06b0/src/core/src/itt.hpp#L34-L57)
|
||||
|
||||
https://github.com/openvinotoolkit/openvino/blob/807279276b78e1fdf9e1e0babd427e1e8dd9a07b/src/core/src/op/abs.cpp#L19-L23
|
||||
### Step 3: apply CC macros
|
||||
|
||||
Then code region in ov::op::v0::Abs::clone_with_new_inputs() is implement conditional compilation.
|
||||
In `SELECTIVE_BUILD=COLLECT` stage, if it was not called, then in `SELECTIVE_BUILD=ON` stage, the code region in this function will be stripped out.
|
||||
Apply the CC macros for your code region that needs conditional compilation.
|
||||
|
||||
[Open the code example](https://github.com/openvinotoolkit/openvino/blob/807279276b78e1fdf9e1e0babd427e1e8dd9a07b/src/core/src/op/abs.cpp#L19-L23)
|
||||
|
||||
## How to verify the new conditional compilation feature
|
||||
Then the code region in ov::op::v0::Abs::clone_with_new_inputs() is implemented with conditional compilation.
|
||||
In the `SELECTIVE_BUILD=COLLECT` stage, or if it was not called, in the `SELECTIVE_BUILD=ON` stage, the code region in this function will be stripped out.
|
||||
|
||||
## Verify the new conditional compilation feature
|
||||
|
||||
#### Build OpenVINO in SELECTIVE_BUILD_ANALYZER mode
|
||||
|
||||
```
|
||||
cmake \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
|
|
@ -165,6 +165,7 @@ In `SELECTIVE_BUILD=COLLECT` stage, if it was not called, then in `SELECTIVE_BUI
|
|||
```
|
||||
|
||||
#### Generate and collect analyzed data
|
||||
|
||||
```
|
||||
cmake --build build --target sea_itt_lib
|
||||
mkdir -p cc_data
|
||||
|
|
@ -183,13 +184,15 @@ In `SELECTIVE_BUILD=COLLECT` stage, if it was not called, then in `SELECTIVE_BUI
|
|||
-S .
|
||||
cmake --build build -j `nproc`
|
||||
```
|
||||
Note: It will benefit more if build OpenVINO static library with conditional complication, you can add option of `-DBUILD_SHARED_LIBS=OFF`.
|
||||
> **NOTE**: It is more beneficial to build OpenVINO static library with conditional compilation. For that, add the `-DBUILD_SHARED_LIBS=OFF` option.
|
||||
|
||||
#### Run test sample
|
||||
Run test samples built in SELECTIVE_BUILD and check whether there will be any error reported.
|
||||
#### Run a test sample
|
||||
|
||||
Run test samples built in SELECTIVE_BUILD and check for any errors.
|
||||
```
|
||||
./benchmark_app -m <your_model.xml> -d CPU
|
||||
```
|
||||
|
||||
#### Check the binaries size benefit.
|
||||
Get the binaries size and compare with master branch result(need do as step 1 to 4), check whether the binaries is smaller than master.
|
||||
#### Check the binaries size benefit
|
||||
|
||||
Get the binaries size and compare with the master branch result (need to do as step 1 to 4), check whether the binaries are smaller than in the master.
|
||||
Loading…
Reference in New Issue