[DOCS] Fix links for 24.0 (#23150)
### Details: - *item1* - *...* ### Tickets: - *ticket-id*
This commit is contained in:
Tatiana Savina
GitHub
parent
e03ea569de
commit
79c8d38174
|
|
@ -24,7 +24,7 @@ OpenVINO (Open Visual Inference and Neural network Optimization) is an open-sour
|
|||
Features
|
||||
##############################################################
|
||||
|
||||
One of the main purposes of OpenVINO is to streamline the deployment of deep learning models in user applications. It optimizes and accelerates model inference, which is crucial for such domains as Generative AI, Large Language models, and use cases like object detection, classification, segmentation, and many others.
|
||||
One of the main purposes of OpenVINO is to streamline the deployment of deep learning models in user applications. It optimizes and accelerates model inference, which is crucial for such domains as Generative AI, Large Language models, and use cases like object detection, classification, segmentation, and many others.
|
||||
|
||||
* :doc:`Model Optimization <openvino-workflow/model-optimization>`
|
||||
|
||||
|
|
@ -32,7 +32,7 @@ OpenVINO provides multiple optimization methods for both the training and post-t
|
|||
|
||||
* :doc:`Model Conversion and Framework Compatibility <openvino-workflow/model-preparation>`
|
||||
|
||||
Supported models can be loaded directly or converted to the OpenVINO format to achieve better performance. Supported frameworks include ONNX, PyTorch, TensorFlow, TensorFlow Lite, Keras, and PaddlePaddle.
|
||||
Supported models can be loaded directly or converted to the OpenVINO format to achieve better performance. Supported frameworks include ONNX, PyTorch, TensorFlow, TensorFlow Lite, Keras, and PaddlePaddle.
|
||||
|
||||
* :doc:`Model Inference <openvino-workflow/running-inference>`
|
||||
|
||||
|
|
@ -40,20 +40,20 @@ OpenVINO accelerates deep learning models on various hardware platforms, ensurin
|
|||
|
||||
* `Deployment on a server <https://github.com/openvinotoolkit/model_server>`__
|
||||
|
||||
A model can be deployed either locally using OpenVINO Runtime or on a model server. Runtime is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions. The model server enables quick model inference using external about-openvino/additional-resources.
|
||||
A model can be deployed either locally using OpenVINO Runtime or on a model server. Runtime is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions. The model server enables quick model inference using external about-openvino/additional-resources.
|
||||
|
||||
Architecture
|
||||
##############################################################
|
||||
|
||||
To learn more about how OpenVINO works, read the Developer documentation on its `architecture <https://github.com/openvinotoolkit/openvino/blob/master/src/docs/architecture.md>`__ and `core components <https://github.com/openvinotoolkit/openvino/blob/master/src/README.md>`__.
|
||||
|
||||
OpenVINO Ecosystem
|
||||
OpenVINO Ecosystem
|
||||
##############################################################
|
||||
|
||||
Along with the primary components of model optimization and runtime, the toolkit also includes:
|
||||
|
||||
* `Neural Network Compression Framework (NNCF) <https://github.com/openvinotoolkit/nncf>`__ - a tool for enhanced OpenVINO™ inference to get performance boost with minimal accuracy drop.
|
||||
* :doc:`Openvino Notebooks <learn-openvino/interactive-tutorials-python>`- Jupyter Python notebook learn-openvino/interactive-tutorials-python, which demonstrate key features of the toolkit.
|
||||
* :doc:`Openvino Notebooks <learn-openvino/interactive-tutorials-python>`- Jupyter Python notebook, which demonstrate key features of the toolkit.
|
||||
* `OpenVINO Model Server <https://github.com/openvinotoolkit/model_server>`__ - a server that enables scalability via a serving microservice.
|
||||
* :doc:`OpenVINO Training Extensions <documentation/openvino-ecosystem/openvino-training-extensions>` – a convenient environment to train Deep Learning models and convert them using the OpenVINO™ toolkit for optimized inference.
|
||||
* :doc:`Dataset Management Framework (Datumaro) <documentation/openvino-ecosystem/datumaro>` - a tool to build, transform, and analyze datasets.
|
||||
|
|
@ -66,7 +66,7 @@ OpenVINO community plays a vital role in the growth and development of the open-
|
|||
* `OpenVINO GitHub issues, discussions and pull requests <https://github.com/openvinotoolkit/openvino>`__
|
||||
* `OpenVINO Blog <https://blog.openvino.ai/>`__
|
||||
* `Community Forum <https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit>`__
|
||||
* `OpenVINO video learn-openvino/interactive-tutorials-python <https://www.youtube.com/watch?v=_Jnjt21ZDS8&list=PLg-UKERBljNxdIQir1wrirZJ50yTp4eHv>`__
|
||||
* `OpenVINO video <https://www.youtube.com/watch?v=_Jnjt21ZDS8&list=PLg-UKERBljNxdIQir1wrirZJ50yTp4eHv>`__
|
||||
* `Support Information <https://www.intel.com/content/www/us/en/support/products/96066/software/development-software/openvino-toolkit.html>`__
|
||||
|
||||
Case Studies
|
||||
|
|
|
|||
|
|
@ -115,7 +115,7 @@ Glossary of terms used in OpenVINO™
|
|||
|
||||
| *plugin / Inference Device / Inference Mode*
|
||||
| OpenVINO makes hardware available for inference based on several core components.
|
||||
They used to be called "plugins" in earlier versions of ../../documentation and you may
|
||||
They used to be called "plugins" in earlier versions of documentation and you may
|
||||
still find this term in some articles. Because of their role in the software,
|
||||
they are now referred to as Devices and Modes ("virtual" devices). For a detailed
|
||||
description of the concept, refer to
|
||||
|
|
|
|||
|
|
@ -69,39 +69,39 @@ Performance Information F.A.Q.
|
|||
- BERT
|
||||
- question / answer
|
||||
- 128
|
||||
* - `bert-large-uncased-whole-word-masking-squad-int8-0001 <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001>`__
|
||||
* - `bert-large-uncased-whole-word-masking-squad-int8-0001 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001>`__
|
||||
- BERT-large
|
||||
- question / answer
|
||||
- 384
|
||||
* - `efficientdet-d0 <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/efficientdet-d0-tf>`__
|
||||
* - `efficientdet-d0 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/efficientdet-d0-tf>`__
|
||||
- Efficientdet
|
||||
- classification
|
||||
- 512x512
|
||||
* - `mask_rcnn_resnet50_atrous_coco <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/mask_rcnn_resnet50_atrous_coco>`__
|
||||
* - `mask_rcnn_resnet50_atrous_coco <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mask_rcnn_resnet50_atrous_coco>`__
|
||||
- Mask R-CNN ResNet 50 Atrous
|
||||
- object instance segmentation
|
||||
- 800x1365
|
||||
* - `mobilenet-v2 <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/mobilenet-v2-pytorch>`__
|
||||
* - `mobilenet-v2 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mobilenet-v2-pytorch>`__
|
||||
- Mobilenet V2 PyTorch
|
||||
- classification
|
||||
- 224x224
|
||||
* - `resnet-50 <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/resnet-50-tf>`__
|
||||
* - `resnet-50 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/resnet-50-tf>`__
|
||||
- ResNet-50_v1_ILSVRC-2012
|
||||
- classification
|
||||
- 224x224
|
||||
* - `ssd-mobilenet-v1-coco <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/ssd_mobilenet_v1_coco>`__
|
||||
* - `ssd-mobilenet-v1-coco <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssd_mobilenet_v1_coco>`__
|
||||
- ssd-mobilenet-V1-coco onnx model
|
||||
- object detection
|
||||
- 300x300
|
||||
* - `ssd-resnet34-1200-onnx <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/ssd-resnet34-1200-onnx>`__
|
||||
* - `ssd-resnet34-1200-onnx <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssd-resnet34-1200-onnx>`__
|
||||
- ssd-resnet34 onnx model
|
||||
- object detection
|
||||
- 1200x1200
|
||||
* - `unet-camvid-onnx-0001 <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/intel/unet-camvid-onnx-0001>`__
|
||||
* - `unet-camvid-onnx-0001 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/unet-camvid-onnx-0001>`__
|
||||
- U-Net
|
||||
- semantic segmentation
|
||||
- 368x480
|
||||
* - `yolo-v3-tiny <https://github.com/openvinotoolkit/open_../../documentation/legacy-features/model-zoo/tree/master/models/public/yolo-v3-tiny-tf>`__
|
||||
* - `yolo-v3-tiny <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tiny-tf>`__
|
||||
- YOLO v3 Tiny
|
||||
- object detection
|
||||
- 416x416
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ Registering Extensions
|
|||
A custom operation class and a new mapping frontend extension class object should be registered to be usable in OpenVINO runtime.
|
||||
|
||||
.. note::
|
||||
This ../documentation is derived from the `Template extension <https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension/new>`__, which demonstrates the details of extension development. It is based on minimalistic ``Identity`` operation that is a placeholder for your real custom operation. Review the complete, fully compilable code to see how it works.
|
||||
This documentation is derived from the `Template extension <https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension>`__, which demonstrates the details of extension development. It is based on minimalistic ``Identity`` operation that is a placeholder for your real custom operation. Review the complete, fully compilable code to see how it works.
|
||||
|
||||
Use the ``ov::Core::add_extension`` method to load the extensions to the ``ov::Core`` object. This method allows loading library with extensions or extensions from the code.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
.. {#openvino_docs_Extensibility_UG_add_openvino_ops}
|
||||
|
||||
Custom OpenVINO™ Operations
|
||||
Custom OpenVINO Operations
|
||||
=============================
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,26 +5,26 @@ Frontend Extensions
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn how to use frontend extension classes to facilitate the mapping
|
||||
of custom operations from the framework model representation to the OpenVINO
|
||||
:description: Learn how to use frontend extension classes to facilitate the mapping
|
||||
of custom operations from the framework model representation to the OpenVINO
|
||||
representation.
|
||||
|
||||
|
||||
The goal of this chapter is to explain how to use Frontend extension classes to facilitate
|
||||
mapping of custom operations from framework model representation to OpenVINO representation.
|
||||
Refer to :doc:`Introduction to OpenVINO Extension <../openvino-extensibility>` to
|
||||
The goal of this chapter is to explain how to use Frontend extension classes to facilitate
|
||||
mapping of custom operations from framework model representation to OpenVINO representation.
|
||||
Refer to :doc:`Introduction to OpenVINO Extension <../openvino-extensibility>` to
|
||||
understand the entire flow.
|
||||
|
||||
This API is applicable to new frontends only, which exist for ONNX, TensorFlow Lite, PaddlePaddle, and TensorFlow.
|
||||
If a different model format is used, follow legacy
|
||||
:doc:`Model Optimizer Extensions <../legacy-features/transition-legacy-conversion-api/legacy-model-optimizer-extensibility>`
|
||||
This API is applicable to new frontends only, which exist for ONNX, TensorFlow Lite, PaddlePaddle, and TensorFlow.
|
||||
If a different model format is used, follow legacy
|
||||
:doc:`Model Optimizer Extensions <../legacy-features/transition-legacy-conversion-api/legacy-model-optimizer-extensibility>`
|
||||
guide.
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
This ../../documentation is written based on the `Template extension <https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension/new>`__,
|
||||
which demonstrates extension development details based on minimalistic ``Identity``
|
||||
operation that is a placeholder for your real custom operation. You can review the complete code,
|
||||
This documentation is written based on the `Template extension <https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension>`__,
|
||||
which demonstrates extension development details based on minimalistic ``Identity``
|
||||
operation that is a placeholder for your real custom operation. You can review the complete code,
|
||||
which is fully compilable, to see how it works.
|
||||
|
||||
|
||||
|
|
@ -35,26 +35,26 @@ guide.
|
|||
Single Operation Mapping with OpExtension
|
||||
#########################################
|
||||
|
||||
This section covers the case when a single operation in framework representation is mapped to a single
|
||||
operation in OpenVINO representation. This is called *one-to-one mapping*. There is ``OpExtension``
|
||||
This section covers the case when a single operation in framework representation is mapped to a single
|
||||
operation in OpenVINO representation. This is called *one-to-one mapping*. There is ``OpExtension``
|
||||
class that works well if all the following conditions are satisfied:
|
||||
|
||||
1. Number of inputs to operation in the Framework representation is the same as in the OpenVINO representation.
|
||||
2. Number of outputs is also the same in both representations.
|
||||
3. Inputs can be indexed and are mapped in order correspondingly, e.g.
|
||||
3. Inputs can be indexed and are mapped in order correspondingly, e.g.
|
||||
input with index 0 in framework representation maps to input with index 0 in OpenVINO representation and so on.
|
||||
4. The same for outputs.
|
||||
5. Each attribute in OpenVINO operation can be initialized from one of the attributes of original operation or by
|
||||
some predefined constant value. Value of copied attributes cannot contain expressions, value is accepted as-is,
|
||||
5. Each attribute in OpenVINO operation can be initialized from one of the attributes of original operation or by
|
||||
some predefined constant value. Value of copied attributes cannot contain expressions, value is accepted as-is,
|
||||
so type of a value should be compatible.
|
||||
|
||||
.. note::
|
||||
|
||||
``OpExtension`` class is currently available for ONNX and TensorFlow frontends.
|
||||
PaddlePaddle frontend has named inputs and outputs for operation (not indexed)
|
||||
``OpExtension`` class is currently available for ONNX and TensorFlow frontends.
|
||||
PaddlePaddle frontend has named inputs and outputs for operation (not indexed)
|
||||
therefore OpExtension mapping is not applicable for this case.
|
||||
|
||||
The following example maps ONNX operation with the type of `Identity <https://github.com/onnx/onnx/blob/main/docs/Operators.md#Identity>`__
|
||||
The following example maps ONNX operation with the type of `Identity <https://github.com/onnx/onnx/blob/main/docs/Operators.md#Identity>`__
|
||||
to OpenVINO template extension ``Identity`` class.
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -67,38 +67,38 @@ to OpenVINO template extension ``Identity`` class.
|
|||
|
||||
The mapping doesn’t involve any attributes, as operation Identity doesn’t have them.
|
||||
|
||||
Extension objects, like just constructed ``extension`` can be used to add to the
|
||||
Extension objects, like just constructed ``extension`` can be used to add to the
|
||||
OpenVINO runtime just before the loading a model that contains custom operations:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_read_model]
|
||||
|
||||
Or extensions can be constructed in a separately compiled shared library.
|
||||
Separately compiled library can be used in Model Optimizer or ``benchmark_app``.
|
||||
Read about how to build and load such a library in the chapter of “Create library with extensions” in
|
||||
Or extensions can be constructed in a separately compiled shared library.
|
||||
Separately compiled library can be used in Model Optimizer or ``benchmark_app``.
|
||||
Read about how to build and load such a library in the chapter of “Create library with extensions” in
|
||||
:doc:`Introduction to OpenVINO Extension <../openvino-extensibility>`.
|
||||
|
||||
If operation have multiple inputs and/or outputs they will be mapped in order.
|
||||
The type of elements in input/output tensors should match expected types in the surrounding operations.
|
||||
For example, if a custom operation produces the ``f32`` data type, the operation that consumes this output
|
||||
If operation have multiple inputs and/or outputs they will be mapped in order.
|
||||
The type of elements in input/output tensors should match expected types in the surrounding operations.
|
||||
For example, if a custom operation produces the ``f32`` data type, the operation that consumes this output
|
||||
should also support ``f32``. Otherwise, model conversion fails with an error, as no automatic type conversion is performed.
|
||||
|
||||
Converting to Standard OpenVINO Operation
|
||||
+++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
``OpExtension`` class can be used when mapping to one of the operations from standard OpenVINO
|
||||
``OpExtension`` class can be used when mapping to one of the operations from standard OpenVINO
|
||||
operation set is what you need and there is no class like ``TemplateExtension::Identity`` implemented.
|
||||
|
||||
Here is an example of a custom framework operation 'MyRelu'. Assume it is mathematically equivalent
|
||||
to standard ``Relu`` that exists in the OpenVINO operation set, but for some reason has the type name of 'MyRelu'.
|
||||
Here is an example of a custom framework operation 'MyRelu'. Assume it is mathematically equivalent
|
||||
to standard ``Relu`` that exists in the OpenVINO operation set, but for some reason has the type name of 'MyRelu'.
|
||||
In this case, you can directly say that 'MyRelu' -> ``Relu`` mapping should be used:
|
||||
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: [py_frontend_extension_MyRelu]
|
||||
|
|
@ -111,27 +111,27 @@ In this case, you can directly say that 'MyRelu' -> ``Relu`` mapping should be u
|
|||
:fragment: [frontend_extension_MyRelu]
|
||||
|
||||
|
||||
In the resulting converted OpenVINO model, “MyRelu” operation will be replaced by the standard operation
|
||||
``Relu`` from the latest available OpenVINO operation set. Notice that when standard operation is used,
|
||||
it can be specified using just a type string (“Relu”) instead of using a ``ov::opset8::Relu`` class name as a
|
||||
template parameter for ``OpExtension``. This method is available for operations from the standard operation set only.
|
||||
For a user custom OpenVINO operation the corresponding class should be always specified as a template parameter
|
||||
In the resulting converted OpenVINO model, “MyRelu” operation will be replaced by the standard operation
|
||||
``Relu`` from the latest available OpenVINO operation set. Notice that when standard operation is used,
|
||||
it can be specified using just a type string (“Relu”) instead of using a ``ov::opset8::Relu`` class name as a
|
||||
template parameter for ``OpExtension``. This method is available for operations from the standard operation set only.
|
||||
For a user custom OpenVINO operation the corresponding class should be always specified as a template parameter
|
||||
as it was demonstrated with ``TemplateExtension::Identity``.
|
||||
|
||||
Attribute Mapping
|
||||
++++++++++++++++++
|
||||
|
||||
As described above, ``OpExtension`` is useful when attributes can be mapped one by one or initialized by a constant.
|
||||
Attributes in OpenVINO operators are identified by their names, so for frameworks that also have named attributes (like TensorFlow, PaddlePaddle, ONNX),
|
||||
you can specify name to name mapping. For frameworks where OpenVINO operator's attributes can be mapped to one of the framework
|
||||
Attributes in OpenVINO operators are identified by their names, so for frameworks that also have named attributes (like TensorFlow, PaddlePaddle, ONNX),
|
||||
you can specify name to name mapping. For frameworks where OpenVINO operator's attributes can be mapped to one of the framework
|
||||
operator inputs (like PyTorch), there's a name to input index mapping.
|
||||
|
||||
|
||||
Named attributes mapping
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If the set of attributes in framework representation and OpenVINO representation completely match by their names and types,
|
||||
no attribute mapping has to be specified in OpExtension constructor parameters. The attributes are discovered and mapped automatically
|
||||
If the set of attributes in framework representation and OpenVINO representation completely match by their names and types,
|
||||
no attribute mapping has to be specified in OpExtension constructor parameters. The attributes are discovered and mapped automatically
|
||||
based on ``visit_attributes`` method that should be defined for any OpenVINO operation.
|
||||
|
||||
Imagine you have CustomOperation class implementation that has two attributes with names: ``attr1`` and ``attr2``.
|
||||
|
|
@ -140,7 +140,7 @@ Imagine you have CustomOperation class implementation that has two attributes wi
|
|||
:language: cpp
|
||||
:fragment: [frontend_extension_CustomOperation]
|
||||
|
||||
And original model in framework representation also has operation with name ``CustomOperation`` with the same
|
||||
And original model in framework representation also has operation with name ``CustomOperation`` with the same
|
||||
``attr1`` and ``attr2`` attributes. Then with the following code:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -149,7 +149,7 @@ And original model in framework representation also has operation with name ``Cu
|
|||
|
||||
Both ``attr1`` and ``attr2`` are copied from framework representation to OpenVINO representation automatically.
|
||||
|
||||
If for some reason names of attributes are different but values still can be copied “as-is” you can pass attribute
|
||||
If for some reason names of attributes are different but values still can be copied “as-is” you can pass attribute
|
||||
names mapping in ``OpExtension`` constructor:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -158,8 +158,8 @@ names mapping in ``OpExtension`` constructor:
|
|||
|
||||
Where ``fw_attr1`` and ``fw_attr2`` are names for corresponding attributes in framework operation representation.
|
||||
|
||||
If copying of an attribute is not what you need, ``OpExtension`` also can set attribute to predefined constant value.
|
||||
For the same ``CustomOperation``, imagine you want to set ``attr2`` to value 5 instead of copying from ``fw_attr2``,
|
||||
If copying of an attribute is not what you need, ``OpExtension`` also can set attribute to predefined constant value.
|
||||
For the same ``CustomOperation``, imagine you want to set ``attr2`` to value 5 instead of copying from ``fw_attr2``,
|
||||
to achieve that do the following:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -179,20 +179,20 @@ This is achieved by specifying maps as arguments for ``OpExtension`` constructor
|
|||
Attribute mapping with named inputs and outputs
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Mappings in previous examples assume that inputs and outputs of an operator in framework model representation come
|
||||
with a particular order so you can directly map framework operation input ``0`` to OpenVINO operation input ``0`` and so on.
|
||||
That's not always the case, for frameworks like PaddlePaddle, operation inputs and outputs are identified by their names
|
||||
and may be defined in any order. So to map it to OpenVINO operation inputs and outputs, you have to specify that order yourself.
|
||||
This can be done by creating two vector of strings, one for input and one for output, where framework operation
|
||||
Mappings in previous examples assume that inputs and outputs of an operator in framework model representation come
|
||||
with a particular order so you can directly map framework operation input ``0`` to OpenVINO operation input ``0`` and so on.
|
||||
That's not always the case, for frameworks like PaddlePaddle, operation inputs and outputs are identified by their names
|
||||
and may be defined in any order. So to map it to OpenVINO operation inputs and outputs, you have to specify that order yourself.
|
||||
This can be done by creating two vector of strings, one for input and one for output, where framework operation
|
||||
input name at position ``i`` maps to OpenVINO operation input at position ``i`` (and similarly for outputs).
|
||||
|
||||
|
||||
Let's see the following example. Like previously, we'd like to map ``CustomOperation`` in the original model,
|
||||
to OpenVINO ``CustomOperation`` as is (so their name and attributes names match). This time, that framework operation
|
||||
inputs and outputs are not strictly ordered and can be identified by their names ``A``, ``B``, ``C`` for inputs
|
||||
and ``X``, ``Y`` for outputs. Those inputs and outputs can be mapped to OpenVINO operation, such that inputs
|
||||
``A``, ``B``, ``C`` map to OpenVINO ``CustomOperation`` first, second and third input and ``X`` and ``Y``
|
||||
outputs map to OpenVINO ``CustomOperation`` first and second output respectively.
|
||||
Let's see the following example. Like previously, we'd like to map ``CustomOperation`` in the original model,
|
||||
to OpenVINO ``CustomOperation`` as is (so their name and attributes names match). This time, that framework operation
|
||||
inputs and outputs are not strictly ordered and can be identified by their names ``A``, ``B``, ``C`` for inputs
|
||||
and ``X``, ``Y`` for outputs. Those inputs and outputs can be mapped to OpenVINO operation, such that inputs
|
||||
``A``, ``B``, ``C`` map to OpenVINO ``CustomOperation`` first, second and third input and ``X`` and ``Y``
|
||||
outputs map to OpenVINO ``CustomOperation`` first and second output respectively.
|
||||
|
||||
Given that, such custom operation can be registered by the following:
|
||||
|
||||
|
|
@ -208,7 +208,7 @@ Second example shows how to map the operation with named inputs and outputs, but
|
|||
:fragment: [frontend_extension_CustomOperation_rename_paddle]
|
||||
|
||||
|
||||
and the last one shows how to map the operation with named inputs and outputs, but when (in order to correctly map framework
|
||||
and the last one shows how to map the operation with named inputs and outputs, but when (in order to correctly map framework
|
||||
operation to OpenVINO operation) one of the attributes has to be set to predefined value:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -220,7 +220,7 @@ Mapping attributes from operation inputs
|
|||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
For models (like PyTorch models), where operations have attributes on the input list, you can specify name to input index mapping.
|
||||
For example, imagine you have created a custom OpenVINO operation that implements a variant of ELU activation function
|
||||
For example, imagine you have created a custom OpenVINO operation that implements a variant of ELU activation function
|
||||
with two attributes ``alpha`` and ``beta``:
|
||||
|
||||
.. math::
|
||||
|
|
@ -232,15 +232,15 @@ with two attributes ``alpha`` and ``beta``:
|
|||
\end{array}
|
||||
\right.
|
||||
|
||||
Below is a snippet of ``CustomElu`` class showing how to define its attributes:
|
||||
Below is a snippet of ``CustomElu`` class showing how to define its attributes:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_framework_map_CustomElu]
|
||||
|
||||
Let's see an example of how you can map ``CustomElu`` to PyTorch `aten::elu <https://pytorch.org/docs/stable/generated/torch.nn.functional.elu.html>`_
|
||||
Let's see an example of how you can map ``CustomElu`` to PyTorch `aten::elu <https://pytorch.org/docs/stable/generated/torch.nn.functional.elu.html>`_
|
||||
(note that if ``beta`` is equal to ``1``, ``CustomElu`` works the same as ``aten::elu``).
|
||||
``aten::elu`` has ``alpha`` attribute second on the input list, but it doesn't have ``beta``.
|
||||
``aten::elu`` has ``alpha`` attribute second on the input list, but it doesn't have ``beta``.
|
||||
So in order to map it to ``CustomElu`` you can use the following:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -254,11 +254,11 @@ Such created extension can be used, e.g. in dynamic library, please refer to :re
|
|||
Mapping custom operations to frontends with OPENVINO_FRAMEWORK_MAP macro
|
||||
########################################################################
|
||||
|
||||
``OPENVINO_FRAMEWORK_MAP`` is a macro that should be used inside OpenVINO operation's class definition and that lets you specify
|
||||
``OPENVINO_FRAMEWORK_MAP`` is a macro that should be used inside OpenVINO operation's class definition and that lets you specify
|
||||
the mapping between this operation to a frontend operation.
|
||||
|
||||
Let's consider the following example. Imagine you have an ONNX model with ``CustomOp`` operation (and this operation has ``mode`` attribute),
|
||||
a TensorFlow model with ``CustomOpV3`` operation (this operation has ``axis`` attribute) and a PaddlePaddle model with ``CustomOp`` (with ``mode`` attribute)
|
||||
Let's consider the following example. Imagine you have an ONNX model with ``CustomOp`` operation (and this operation has ``mode`` attribute),
|
||||
a TensorFlow model with ``CustomOpV3`` operation (this operation has ``axis`` attribute) and a PaddlePaddle model with ``CustomOp`` (with ``mode`` attribute)
|
||||
that has input named "X" and output named "Out" and all of them can be implemented with a single OpenVINO operation ``CustomOp`` like follows:
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
|
|
@ -269,7 +269,7 @@ that has input named "X" and output named "Out" and all of them can be implement
|
|||
:language: cpp
|
||||
:fragment: [frontend_extension_framework_map_macro_CustomOp]
|
||||
|
||||
Let's take a closer look at the parameters this macro takes (note that there are two flavors - the second one is to map
|
||||
Let's take a closer look at the parameters this macro takes (note that there are two flavors - the second one is to map
|
||||
for PaddlePaddle operations where input and output names have to be specified).
|
||||
|
||||
.. code-block:: cpp
|
||||
|
|
@ -278,27 +278,27 @@ for PaddlePaddle operations where input and output names have to be specified).
|
|||
OPENVINO_FRAMEWORK_MAP(framework, input_names, output_names, name, attributes_map, attributes_values)
|
||||
|
||||
- ``framework`` - framework name.
|
||||
- ``name`` - the framework operation name. It's optional if the OpenVINO custom operation name
|
||||
(that is the name that is passed as the first parameter to ``OPENVINO_OP`` macro) is the same
|
||||
- ``name`` - the framework operation name. It's optional if the OpenVINO custom operation name
|
||||
(that is the name that is passed as the first parameter to ``OPENVINO_OP`` macro) is the same
|
||||
as the framework operation name and both ``attributes_map`` and ``attributes_values`` are not provided.
|
||||
- ``input_names`` - vector of strings that specify the names of inputs (needed to map PaddlePaddle to OpenVINO operations),
|
||||
- ``output_names`` - vector of strings that specify the names of outputs (needed to map PaddlePaddle to OpenVINO operations),
|
||||
- ``attributes_map`` - used to provide a mapping between OpenVINO operation attribute and
|
||||
framework operation attribute. Contains key-value pairs, where key is an OpenVINO operation
|
||||
attribute name and value is its corresponding framework operation attribute name.
|
||||
This parameter is optional if the number of OpenVINO operation attributes and their names
|
||||
- ``attributes_map`` - used to provide a mapping between OpenVINO operation attribute and
|
||||
framework operation attribute. Contains key-value pairs, where key is an OpenVINO operation
|
||||
attribute name and value is its corresponding framework operation attribute name.
|
||||
This parameter is optional if the number of OpenVINO operation attributes and their names
|
||||
match one-to-one with framework operation attributes.
|
||||
- ``attributes_values`` - used to provide default values for OpenVINO operation attributes
|
||||
that are not specified in ``attributes_map``. Contains key-value pairs, where key is an OpenVINO
|
||||
operation attribute name and the value is this attribute value. This parameter cannot be provided
|
||||
- ``attributes_values`` - used to provide default values for OpenVINO operation attributes
|
||||
that are not specified in ``attributes_map``. Contains key-value pairs, where key is an OpenVINO
|
||||
operation attribute name and the value is this attribute value. This parameter cannot be provided
|
||||
if ``attributes_map`` contains all of OpenVINO operation attributes or if ``attributes_map`` is not provided.
|
||||
|
||||
In the example above, ``OPENVINO_FRAMEWORK_MAP`` is used three times.
|
||||
First, OpenVINO ``CustomOp`` is mapped to ONNX ``CustomOp`` operation, ``m_mode`` attribute is mapped to ``mode``
|
||||
attribute, while ``m_axis`` attribute gets the default value ``-1``. Secondly, OpenVINO ``CustomOp`` is mapped
|
||||
to TensorFlow ``CustomOpV3`` operation, ``m_axis`` attribute is mapped to ``axis`` attribute, while ``m_mode``
|
||||
attribute gets the default value ``"linear"``. Thirdly, OpenVINO ``CustomOp`` is mapped to PaddlePaddle ``CustomOp`` operation,
|
||||
``m_mode`` attribute is mapped to ``mode`` attribute, while ``m_axis`` attribute gets the default value ``-1``.
|
||||
First, OpenVINO ``CustomOp`` is mapped to ONNX ``CustomOp`` operation, ``m_mode`` attribute is mapped to ``mode``
|
||||
attribute, while ``m_axis`` attribute gets the default value ``-1``. Secondly, OpenVINO ``CustomOp`` is mapped
|
||||
to TensorFlow ``CustomOpV3`` operation, ``m_axis`` attribute is mapped to ``axis`` attribute, while ``m_mode``
|
||||
attribute gets the default value ``"linear"``. Thirdly, OpenVINO ``CustomOp`` is mapped to PaddlePaddle ``CustomOp`` operation,
|
||||
``m_mode`` attribute is mapped to ``mode`` attribute, while ``m_axis`` attribute gets the default value ``-1``.
|
||||
This mapping also specifies the input name "X" and output name "Out".
|
||||
|
||||
The last step is to register this custom operation by following:
|
||||
|
|
@ -309,7 +309,7 @@ The last step is to register this custom operation by following:
|
|||
|
||||
.. important::
|
||||
|
||||
To map an operation on a specific framework, you have to link to a respective
|
||||
To map an operation on a specific framework, you have to link to a respective
|
||||
frontend (``openvino::frontend::onnx``, ``openvino::frontend::tensorflow``, ``openvino::frontend::paddle``) in the ``CMakeLists.txt`` file:
|
||||
|
||||
.. code-block:: sh
|
||||
|
|
@ -320,27 +320,27 @@ The last step is to register this custom operation by following:
|
|||
Mapping to Multiple Operations with ConversionExtension
|
||||
#######################################################
|
||||
|
||||
Previous sections cover the case when a single operation is mapped to a single operation with optional
|
||||
adjustment in names and attribute values. That is likely enough for your own custom operation with existing
|
||||
C++ kernel implementation. In this case your framework representation and OpenVINO representation for the
|
||||
Previous sections cover the case when a single operation is mapped to a single operation with optional
|
||||
adjustment in names and attribute values. That is likely enough for your own custom operation with existing
|
||||
C++ kernel implementation. In this case your framework representation and OpenVINO representation for the
|
||||
operation are under your control and inputs/outpus/attributes can be aligned to make ``OpExtension`` usable.
|
||||
|
||||
In case if one-to-one mapping is not possible, *decomposition to multiple operations* should be considered.
|
||||
It is achieved by using more verbose and less automated ``ConversionExtension`` class.
|
||||
It enables writing arbitrary code to replace a single framework operation by multiple connected OpenVINO
|
||||
In case if one-to-one mapping is not possible, *decomposition to multiple operations* should be considered.
|
||||
It is achieved by using more verbose and less automated ``ConversionExtension`` class.
|
||||
It enables writing arbitrary code to replace a single framework operation by multiple connected OpenVINO
|
||||
operations constructing dependency graph of any complexity.
|
||||
|
||||
``ConversionExtension`` maps a single operation to a function which builds a graph using OpenVINO
|
||||
operation classes. Follow chapter :ref:`Build a Model in OpenVINO Runtime <ov_ug_build_model>` to
|
||||
``ConversionExtension`` maps a single operation to a function which builds a graph using OpenVINO
|
||||
operation classes. Follow chapter :ref:`Build a Model in OpenVINO Runtime <ov_ug_build_model>` to
|
||||
learn how to use OpenVINO operation classes to build a fragment of model for replacement.
|
||||
|
||||
Below example illustrates using ``ConversionExtension`` for conversion of “ThresholdedRelu”
|
||||
Below example illustrates using ``ConversionExtension`` for conversion of “ThresholdedRelu”
|
||||
from ONNX according to the formula: ``ThresholdedRelu(x, alpha) -> Multiply(x, Convert(Greater(x, alpha), type=float))``.
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
``ThresholdedRelu`` is one of the standard ONNX operators which is supported by ONNX frontend
|
||||
natively out-of-the-box. Here we are re-implementing it to illustrate how you can add a similar
|
||||
``ThresholdedRelu`` is one of the standard ONNX operators which is supported by ONNX frontend
|
||||
natively out-of-the-box. Here we are re-implementing it to illustrate how you can add a similar
|
||||
support for your custom operation instead of ``ThresholdedRelu``.
|
||||
|
||||
.. tab-set::
|
||||
|
|
@ -354,7 +354,7 @@ from ONNX according to the formula: ``ThresholdedRelu(x, alpha) -> Multiply(x, C
|
|||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_ThresholdedReLU_header]
|
||||
|
|
@ -363,21 +363,21 @@ from ONNX according to the formula: ``ThresholdedRelu(x, alpha) -> Multiply(x, C
|
|||
|
||||
.. tab-item:: Python
|
||||
:sync: py
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: [py_frontend_extension_ThresholdedReLU]
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: [frontend_extension_ThresholdedReLU]
|
||||
|
||||
|
||||
The next example shows how to use ``ConversionExtension`` to convert PyTorch
|
||||
`aten::hardtanh <https://pytorch.org/docs/stable/generated/torch.nn.functional.hardtanh.html>`_
|
||||
The next example shows how to use ``ConversionExtension`` to convert PyTorch
|
||||
`aten::hardtanh <https://pytorch.org/docs/stable/generated/torch.nn.functional.hardtanh.html>`_
|
||||
to demonstrate how to use ``get_values_from_const_input`` function to fetch an attribute value from input:
|
||||
|
||||
|
||||
|
|
@ -394,7 +394,7 @@ To access original framework operation attribute value and connect to inputs, ``
|
|||
|
||||
* ``NodeContext::get_values_from_const_input`` to get an attribute with a given input index.
|
||||
|
||||
The conversion function should return a vector of node outputs that are mapped to
|
||||
The conversion function should return a vector of node outputs that are mapped to
|
||||
corresponding outputs of the original framework operation in the same order.
|
||||
|
||||
Some frameworks require output names of the operation to be provided during conversion.
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ OpenVINO plugin dynamic library consists of several main components:
|
|||
|
||||
.. note::
|
||||
|
||||
This ../../documentation is written based on the ``Template`` plugin, which demonstrates plugin development details. Find the complete code of the ``Template``, which is fully compilable and up-to-date, at ``<openvino source dir>/src/plugins/template``.
|
||||
This documentation is written based on the ``Template`` plugin, which demonstrates plugin development details. Find the complete code of the ``Template``, which is fully compilable and up-to-date, at ``<openvino source dir>/src/plugins/template``.
|
||||
|
||||
|
||||
Detailed Guides
|
||||
|
|
|
|||
|
|
@ -313,7 +313,7 @@ This step is optional. It modifies the transformation function to a device-speci
|
|||
Result model overview
|
||||
#####################
|
||||
|
||||
Let's explore quantized `TensorFlow implementation of ResNet-50 <https://github.com/openvinotoolkit/open_../../../legacy-features/model-zoo/tree/master/models/public/resnet-50-tf>`__ model. Use :doc:`Model Downloader <../../../../omz_tools_downloader>` tool to download the ``fp16`` model from `OpenVINO™ Toolkit - Open Model Zoo repository <https://github.com/openvinotoolkit/open_../../../legacy-features/model-zoo>`__:
|
||||
Let's explore quantized `TensorFlow implementation of ResNet-50 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/resnet-50-tf>`__ model. Use :doc:`Model Downloader <../../../../omz_tools_downloader>` tool to download the ``fp16`` model from `OpenVINO™ Toolkit - Open Model Zoo repository <https://github.com/openvinotoolkit/open_model_zoo>`__:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
|
@ -347,7 +347,7 @@ Result model depends on different factors:
|
|||
|
||||
|
||||
Information about layer precision is stored in the performance counters that are
|
||||
available from the OpenVINO Runtime API. For example, the part of performance counters table for quantized `TensorFlow implementation of ResNet-50 <https://github.com/openvinotoolkit/open_../../../legacy-features/model-zoo/tree/master/models/public/resnet-50-tf>`__ model inference on CPU Plugin looks as follows:
|
||||
available from the OpenVINO Runtime API. For example, the part of performance counters table for quantized `TensorFlow implementation of ResNet-50 <https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/resnet-50-tf>`__ model inference on CPU Plugin looks as follows:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ Step 3. Main Transformations
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about main transformations, which are mostly low
|
||||
precision transformations that handle decomposition and
|
||||
:description: Learn about main transformations, which are mostly low
|
||||
precision transformations that handle decomposition and
|
||||
dequantization operations.
|
||||
|
||||
.. toctree::
|
||||
|
|
@ -50,41 +50,41 @@ Step 3. Main Transformations
|
|||
|
||||
|
||||
Main transformations are the majority of low precision transformations. Transformations operate with dequantization operations. Main transformations include:
|
||||
|
||||
* :doc:`AddTransformation <step3-main/arithmetic/add>`
|
||||
* :doc:`AvgPoolTransformation <step3-main/pooling/avg-pool>`
|
||||
* :doc:`BatchToSpaceTransformation <step3-main/shape/batch-to-space>`
|
||||
* :doc:`ClampTransformation <step3-main/activation/clamp>`
|
||||
* :doc:`ConcatTransformation <step3-main/movement/concat>`
|
||||
* :doc:`ConvolutionTransformation <step3-main/convolution/convolution>`
|
||||
* :doc:`ConvolutionBackpropDataTransformation <step3-main/convolution/convolution-backprop-data>`
|
||||
* :doc:`DepthToSpaceTransformation <step3-main/movement/depth-to-space>`
|
||||
* :doc:`FakeQuantizeDecompositionTransformation <step4-cleanup/fake-quantize-decomposition>`
|
||||
* :doc:`FakeQuantizeTransformation <step3-main/quantization/fake-quantize>`
|
||||
* :doc:`InterpolateTransformation <step3-main/image/interpolate>`
|
||||
* :doc:`GroupConvolutionTransformation <step3-main/convolution/group-convolution>`
|
||||
* :doc:`GatherTransformation <step3-main/movement/gather>`
|
||||
* :doc:`MatMulTransformation <step3-main/matrix/mat-mul>`
|
||||
* :doc:`MaxPoolTransformation <step3-main/pooling/max-pool>`
|
||||
* :doc:`MultiplyPartialTransformation <step3-main/arithmetic/multiply-partial>`
|
||||
* :doc:`MVNTransformation <step3-main/normalization/mvn>`
|
||||
* :doc:`NormalizeL2Transformation <step3-main/normalization/normalize-l2>`
|
||||
|
||||
* :doc:`AddTransformation <step3-main/arithmetic/add>`
|
||||
* :doc:`AvgPoolTransformation <step3-main/pooling/avg-pool>`
|
||||
* :doc:`BatchToSpaceTransformation <step3-main/shape/batch-to-space>`
|
||||
* :doc:`ClampTransformation <step3-main/activation/clamp>`
|
||||
* :doc:`ConcatTransformation <step3-main/movement/concat>`
|
||||
* :doc:`ConvolutionTransformation <step3-main/convolution/convolution>`
|
||||
* :doc:`ConvolutionBackpropDataTransformation <step3-main/convolution/convolution-backprop-data>`
|
||||
* :doc:`DepthToSpaceTransformation <step3-main/movement/depth-to-space>`
|
||||
* :doc:`FakeQuantizeDecompositionTransformation <step4-cleanup/fake-quantize-decomposition>`
|
||||
* :doc:`FakeQuantizeTransformation <step3-main/quantization/fake-quantize>`
|
||||
* :doc:`InterpolateTransformation <step3-main/image/interpolate>`
|
||||
* :doc:`GroupConvolutionTransformation <step3-main/convolution/group-convolution>`
|
||||
* :doc:`GatherTransformation <step3-main/movement/gather>`
|
||||
* :doc:`MatMulTransformation <step3-main/matrix/mat-mul>`
|
||||
* :doc:`MaxPoolTransformation <step3-main/pooling/max-pool>`
|
||||
* :doc:`MultiplyPartialTransformation <step3-main/arithmetic/multiply-partial>`
|
||||
* :doc:`MVNTransformation <step3-main/normalization/mvn>`
|
||||
* :doc:`NormalizeL2Transformation <step3-main/normalization/normalize-l2>`
|
||||
* :doc:`PadTransformation<step3-main/movement/pad>`
|
||||
* :doc:`PReluTransformation <step3-main/activation/prelu>`
|
||||
* :doc:`ReduceMaxTransformation <step3-main/reduction/reduce-max>`
|
||||
* :doc:`ReduceMeanTransformation <step3-main/reduction/reduce-mean>`
|
||||
* :doc:`ReduceMinTransformation <step3-main/reduction/reduce-min>`
|
||||
* :doc:`ReduceSumTransformation <step3-main/reduction/reduce-sum>`
|
||||
* :doc:`ReluTransformation <step3-main/activation/relu>`
|
||||
* :doc:`ReshapeTransformation <step3-main/shape/reshape>`
|
||||
* :doc:`SpaceToBatchTransformation <step3-main/shape/space-to-batch>`
|
||||
* :doc:`SqueezeTransformation <step3-main/shape/squeeze>`
|
||||
* :doc:`ShuffleChannelsTransformation <step3-main/movement/shuffle-channels>`
|
||||
* :doc:`SplitTransformation <step3-main/movement/split>`
|
||||
* :doc:`StridedSliceTransformation <step3-main/movement/strided-slice>`
|
||||
* :doc:`TransposeTransformation <step3-main/movement/transpose>`
|
||||
* :doc:`UnsqueezeTransformation <step3-main/shape/unsqueeze>`
|
||||
* :doc:`VariadicSplitTransformation <step3-main/movement/variadic-split>`
|
||||
* :doc:`PReluTransformation <step3-main/activation/prelu>`
|
||||
* :doc:`ReduceMaxTransformation <step3-main/reduction/reduce-max>`
|
||||
* :doc:`ReduceMeanTransformation <step3-main/reduction/reduce-mean>`
|
||||
* :doc:`ReduceMinTransformation <step3-main/reduction/reduce-min>`
|
||||
* :doc:`ReduceSumTransformation <step3-main/reduction/reduce-sum>`
|
||||
* :doc:`ReluTransformation <step3-main/activation/relu>`
|
||||
* :doc:`ReshapeTransformation <step3-main/shape/reshape>`
|
||||
* :doc:`SpaceToBatchTransformation <step3-main/shape/space-to-batch>`
|
||||
* :doc:`SqueezeTransformation <step3-main/shape/squeeze>`
|
||||
* :doc:`ShuffleChannelsTransformation <step3-main/movement/shuffle-channels>`
|
||||
* :doc:`SplitTransformation <step3-main/movement/split>`
|
||||
* :doc:`StridedSliceTransformation <step3-main/movement/strided-slice>`
|
||||
* :doc:`TransposeTransformation <step3-main/movement/transpose>`
|
||||
* :doc:`UnsqueezeTransformation <step3-main/shape/unsqueeze>`
|
||||
* :doc:`VariadicSplitTransformation <step3-main/movement/variadic-split>`
|
||||
|
||||
Let's explore some main transformations on the example model. Original model:
|
||||
|
||||
|
|
@ -102,10 +102,10 @@ Changes in the example model after main transformation:
|
|||
|
||||
* original ``FakeQuantize`` operations were replaced with new operations with other output intervals and output port precision,
|
||||
* dequantization operations.
|
||||
|
||||
* Dequantization operations were moved via precision preserved (``concat1`` and ``concat2``) and quantized (``convolution2``) operations.
|
||||
|
||||
.. note::
|
||||
|
||||
The left branch (branch #1) does not require per-tensor quantization. As a result, the ``fakeQuantize1``output interval is [0, 255]. But quantized ``convolution2`` requires per-tensor quantization on the right branch (branch #2). Then all connected ``FakeQuantize`` interval operations (``fakeQuantize1`` and ``fakeQuantize2``) are aligned to have per-tensor quantization after the concatenation (``concat2``) operation.
|
||||
* Dequantization operations were moved via precision preserved (``concat1`` and ``concat2``) and quantized (``convolution2``) operations.
|
||||
|
||||
.. note::
|
||||
|
||||
The left branch (branch #1) does not require per-tensor quantization. As a result, the ``fakeQuantize1`` output interval is [0, 255]. But quantized ``convolution2`` requires per-tensor quantization on the right branch (branch #2). Then all connected ``FakeQuantize`` interval operations (``fakeQuantize1`` and ``fakeQuantize2``) are aligned to have per-tensor quantization after the concatenation (``concat2``) operation.
|
||||
|
||||
|
|
|
|||
|
|
@ -25,36 +25,45 @@ If dequantization operations on the full branch have a `FakeQuantize` operation
|
|||
|
||||
Depending on the plugin instruction set, low precision inference for the `Add` operation can be implemented in two logical steps in one plugin kernel:
|
||||
|
||||
* Inference step #1: Operations in the full branch, for example, `Convolution` and `FakeQuantize` with fused dequantization operations, and `Add` can be inferred in the original precision.
|
||||
* Inference step #1: Operations in the full branch, for example, `Convolution` and `FakeQuantize` with fused dequantization operations, and `Add` can be inferred in the original precision.
|
||||
|
||||
* Inference step #2: Inference step #1 result can be added with the empty branch tensor in low precision.
|
||||
* Inference step #2: Inference step #1 result can be added with the empty branch tensor in low precision.
|
||||
|
||||
This approach allows to infer the `Add` operation in the optimal way.
|
||||
|
||||
## Subgraph before transformation
|
||||
Subgraph before transformation
|
||||
#############################################
|
||||
|
||||
The subgraph with quantized `Add` operation before transformation:
|
||||
|
||||
\f[
|
||||
y_{ch,i}=(scale1_{ch} * (x1_{ch,i} - shift1_{ch})) + (scale2_{ch} * (x2_{ch,i} - shift2_{ch}))
|
||||
\f]
|
||||
.. math::
|
||||
|
||||
|
||||
y_{ch,i}=(scale1_{ch} * (x1_{ch,i} - shift1_{ch})) + (scale2_{ch} * (x2_{ch,i} - shift2_{ch}))
|
||||
|
||||
|
||||
.. image:: /../../../docs/sphinx_setup/_static/images/add.common.png
|
||||
|
||||
|
||||
Subgraph after transformation
|
||||
#############################################
|
||||
|
||||
## Subgraph after transformation
|
||||
The subgraph with the `Add` operation after the transformation:
|
||||
|
||||
\f[
|
||||
y_{ch,i}=scale2_{ch} * (scale1_{ch}' * (x1_{ch,i} - shift1_{ch}') + x2_{ch,i})
|
||||
\f]
|
||||
.. math::
|
||||
|
||||
y_{ch,i}=scale2_{ch} * (scale1_{ch}' * (x1_{ch,i} - shift1_{ch}') + x2_{ch,i})
|
||||
|
||||
|
||||
where:
|
||||
|
||||
\f[
|
||||
scale1_{ch}' = scale1_{ch} / scale2_{ch}
|
||||
\f]
|
||||
.. math::
|
||||
|
||||
\f[
|
||||
shift1_{ch}' = shift1_{ch} + scale2_{ch} * shift2_{ch} / scale1_{ch}
|
||||
\f]
|
||||
scale1_{ch}' = scale1_{ch} / scale2_{ch}
|
||||
|
||||
|
||||
|
||||
.. math::
|
||||
|
||||
shift1_{ch}' = shift1_{ch} + scale2_{ch} * shift2_{ch} / scale1_{ch}
|
||||
|
||||
|
||||
.. image:: /../../../docs/sphinx_setup/_static/images/add.transformed.png
|
||||
|
|
@ -6,32 +6,41 @@ ConvolutionTransformation transformation
|
|||
ov::pass::low_precision::ConvolutionTransformation class represents the `Convolution` operation transformation.
|
||||
|
||||
The transformation propagates dequantization operations on activations and weights through the `Convolution` operation. The transformation supports several weights quantization approaches:
|
||||
|
||||
* quantized weights in low precision with dequantization operations,
|
||||
* weights in original precision with `FakeQuantize` operation.
|
||||
|
||||
Result dequantization `Multiply` constant value *result* is calculated as multiplication for dequantization `Multiply` constant value on activations *a* and dequantization `Multiply` constant value on weights *b* :
|
||||
Result dequantization ``Multiply`` constant value *result* is calculated as multiplication for dequantization `Multiply` constant value on activations *a* and dequantization `Multiply` constant value on weights *b* :
|
||||
|
||||
\f[
|
||||
result_{i} = a_{i} \cdot b_{i}
|
||||
\f]
|
||||
.. math::
|
||||
|
||||
## Limitations
|
||||
result_{i} = a_{i} \cdot b_{i}
|
||||
|
||||
Limitations
|
||||
##############################
|
||||
|
||||
* Dequantization on activations must be per-tensor. It means that dequantization `Multiply` constant value on activations must be scalar.
|
||||
|
||||
## Subgraph before transformation
|
||||
Subgraph before transformation
|
||||
#############################################
|
||||
|
||||
### Quantized weights in low precision with dequantization operations
|
||||
The subgraph with quantized `Convolution` before transformation with quantized weights in low precision constant and dequantization operations:
|
||||
Quantized weights in low precision with dequantization operations
|
||||
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
|
||||
The subgraph with quantized ``Convolution`` before transformation with quantized weights in low precision constant and dequantization operations:
|
||||
|
||||
### Weights in original precision with FakeQuantize operation
|
||||
The subgraph with quantized `Convolution` before transformation with weights in original precision and `FakeQuantize` operation:
|
||||
.. image:: /../../../docs/sphinx_setup/_static/images/fq_and_convolution.common.png
|
||||
|
||||
|
||||
Weights in original precision with FakeQuantize operation
|
||||
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
The subgraph with quantized ``Convolution`` before transformation with weights in original precision and `FakeQuantize` operation:
|
||||
|
||||
.. image:: /../../../docs/sphinx_setup/_static/images/fq_fq_and_convolution.common.png
|
||||
|
||||
Subgraph after transformation
|
||||
#############################################
|
||||
|
||||
## Subgraph after transformation
|
||||
The subgraph with `Convolution` operation after the transformation:
|
||||
|
||||
|
||||
.. image:: /../../../docs/sphinx_setup/_static/images/fq_and_convolution.transformed.png
|
||||
|
|
@ -5,14 +5,14 @@ Compiled Model
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Use the ov::CompiledModel class as the base class for a compiled
|
||||
:description: Use the ov::CompiledModel class as the base class for a compiled
|
||||
model and to create an arbitrary number of ov::InferRequest objects.
|
||||
|
||||
ov::CompiledModel class functionality:
|
||||
|
||||
* Compile an ov::Model instance to a backend specific graph representation
|
||||
* Create an arbitrary number of ov::InferRequest objects
|
||||
* Hold some common ../../../about-openvino/additional-resources shared between different instances of ov::InferRequest. For example:
|
||||
* Hold some common resources shared between different instances of ov::InferRequest. For example:
|
||||
|
||||
* ov::ICompiledModel::m_task_executor task executor to implement asynchronous execution
|
||||
* ov::ICompiledModel::m_callback_executor task executor to run an asynchronous inference request callback in a separate thread
|
||||
|
|
@ -20,7 +20,7 @@ ov::CompiledModel class functionality:
|
|||
CompiledModel Class
|
||||
###################
|
||||
|
||||
OpenVINO Plugin API provides the interface ov::ICompiledModel which should be used as a base class for a compiled model. Based on that, a declaration of an compiled model class can look as follows:
|
||||
OpenVINO Plugin API provides the interface ov::ICompiledModel which should be used as a base class for a compiled model. Based on that, a declaration of an compiled model class can look as follows:
|
||||
|
||||
.. doxygensnippet:: src/plugins/template/src/compiled_model.hpp
|
||||
:language: cpp
|
||||
|
|
@ -58,8 +58,8 @@ The function accepts a const shared pointer to ``ov::Model`` object and applies
|
|||
:fragment: [compiled_model:compile_model]
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
.. note::
|
||||
|
||||
After all these steps, the backend specific graph is ready to create inference requests and perform inference.
|
||||
|
||||
export_model()
|
||||
|
|
@ -88,8 +88,8 @@ While the public OpenVINO API has a single interface for inference request, whic
|
|||
|
||||
* For single-stage pipelines, there is no need to define this method and create a class derived from ov::IAsyncInferRequest. For single stage pipelines, a default implementation of this method creates ov::IAsyncInferRequest wrapping a synchronous inference request and runs it asynchronously in the ``m_request_executor`` executor.
|
||||
* For pipelines with multiple stages, such as performing some preprocessing on host, uploading input data to a device, running inference on a device, or downloading and postprocessing output data, schedule stages on several task executors to achieve better device use and performance. You can do it by creating a sufficient number of inference requests running in parallel. In this case, device stages of different inference requests are overlapped with preprocessing and postprocessing stage giving better performance.
|
||||
|
||||
.. important::
|
||||
|
||||
.. important::
|
||||
|
||||
It is up to you to decide how many task executors you need to optimally execute a device pipeline.
|
||||
|
||||
|
|
|
|||
|
|
@ -71,6 +71,6 @@ This topic provides a complete list of available sets of operations supported in
|
|||
See Also
|
||||
########
|
||||
|
||||
:doc:`Operation Sets in OpenVINO <../operation-sets>`
|
||||
:doc:`OpenVINO IR format <../../openvino-ir-format>`
|
||||
* :doc:`Operation Sets in OpenVINO <../operation-sets>`
|
||||
* :doc:`OpenVINO IR format <../../openvino-ir-format>`
|
||||
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ IsFinite
|
|||
*IsFinite* operation has no attributes.
|
||||
|
||||
**Detailed description**: *IsFinite* returns the boolean mask of the input tensor in which ``False`` corresponds to ``NaN`` and ``Infinity`` and ``True`` to all other values.
|
||||
|
||||
* The output tensor has the same shape as the input tensor.
|
||||
* The ``i``'th element of the output tensor is ``False`` if ``i``'th element of the input tensor is ``NaN`` or ``Inf``. Otherwise, it is ``True``.
|
||||
* For example, for a given input tensor ``[NaN, 2.1, 3.7, Inf]`` the output tensor is ``[False, True, True, False]``.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ IsInf
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about IsInf - an element-wise, comparison operation, which
|
||||
:description: Learn about IsInf - an element-wise, comparison operation, which
|
||||
can be performed on a single tensor in OpenVINO.
|
||||
|
||||
**Versioned name**: *IsInf-10*
|
||||
|
|
@ -22,6 +22,7 @@ IsInf
|
|||
|
||||
* **Description**: specifies rules used for mapping values with negative infinity.
|
||||
* **Range of values**:
|
||||
|
||||
* ``false`` - map negative infinity to ``false``
|
||||
* ``true`` - map negative infinity to ``true``
|
||||
* **Type**: ``boolean``
|
||||
|
|
@ -32,6 +33,7 @@ IsInf
|
|||
|
||||
* **Description**: specifies rules used for mapping values with positive infinity.
|
||||
* **Range of values**:
|
||||
|
||||
* ``false`` - map positive infinity to ``false``
|
||||
* ``true`` - map positive infinity to ``true``
|
||||
* **Type**: ``boolean``
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ BinaryConvolution
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about BinaryConvolution-1 - a 2D convolution operation, which
|
||||
:description: Learn about BinaryConvolution-1 - a 2D convolution operation, which
|
||||
can be performed on binary input and binary kernel tensors in OpenVINO.
|
||||
|
||||
**Versioned name**: *BinaryConvolution-1*
|
||||
|
|
@ -21,7 +21,7 @@ Computation algorithm for mode *xnor-popcount*:
|
|||
|
||||
- ``X = XNOR(input_patch, filter)``, where XNOR is bitwise operation on two bit streams
|
||||
- ``P = popcount(X)``, where popcount is the number of ``1`` bits in the ``X`` bit stream
|
||||
- ``Output = 2 \* P - B``, where ``B`` is the total number of bits in the ``P`` bit stream
|
||||
- ``Output = 2 * P - B``, where ``B`` is the total number of bits in the ``P`` bit stream
|
||||
|
||||
**Attributes**:
|
||||
|
||||
|
|
@ -107,7 +107,7 @@ Computation algorithm for mode *xnor-popcount*:
|
|||
|
||||
.. code-block:: xml
|
||||
:force:
|
||||
|
||||
|
||||
<layer type="BinaryConvolution" ...>
|
||||
<data dilations="1,1" pads_begin="2,2" pads_end="2,2" strides="1,1" mode="xnor-popcount" pad_value="0" auto_pad="explicit"/>
|
||||
<input>
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ ExperimentalDetectronPriorGridGenerator
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about ExperimentalDetectronPriorGridGenerator-6 -
|
||||
an object detection operation, which can be performed on three
|
||||
:description: Learn about ExperimentalDetectronPriorGridGenerator-6 -
|
||||
an object detection operation, which can be performed on three
|
||||
required input tensors.
|
||||
|
||||
**Versioned name**: *ExperimentalDetectronPriorGridGenerator-6*
|
||||
|
|
@ -27,7 +27,7 @@ Numbers of generated cells is ``featmap_height`` and ``featmap_width`` if *h* an
|
|||
|
||||
* **Description**: The *flatten* attribute specifies whether the output tensor should be 2D or 4D.
|
||||
* **Range of values**:
|
||||
|
||||
|
||||
* ``true`` - the output tensor should be a 2D tensor
|
||||
* ``false`` - the output tensor should be a 4D tensor
|
||||
* **Type**: ``boolean``
|
||||
|
|
@ -69,7 +69,7 @@ Numbers of generated cells is ``featmap_height`` and ``featmap_width`` if *h* an
|
|||
**Inputs**
|
||||
|
||||
* **1**: A 2D tensor of type *T* with shape ``[number_of_priors, 4]`` contains priors. **Required.**
|
||||
* **2**: A 4D tensor of type *T* with input feature map ``[1, number_of_channels, featmap_height, featmap_width]``. This operation uses only sizes of this input tensor, not its data.**Required.**
|
||||
* **2**: A 4D tensor of type *T* with input feature map ``[1, number_of_channels, featmap_height, featmap_width]``. This operation uses only sizes of this input tensor, not its data. **Required.**
|
||||
* **3**: A 4D tensor of type *T* with input image ``[1, number_of_channels, image_height, image_width]``. The number of channels of both feature map and input image tensors must match. This operation uses only sizes of this input tensor, not its data. **Required.**
|
||||
|
||||
**Outputs**
|
||||
|
|
@ -84,7 +84,7 @@ Numbers of generated cells is ``featmap_height`` and ``featmap_width`` if *h* an
|
|||
|
||||
.. code-block:: xml
|
||||
:force:
|
||||
|
||||
|
||||
<layer ... type="ExperimentalDetectronPriorGridGenerator" version="opset6">
|
||||
<data flatten="true" h="0" stride_x="32.0" stride_y="32.0" w="0"/>
|
||||
<input>
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ ExperimentalDetectronROIFeatureExtractor
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about ExperimentalDetectronROIFeatureExtractor-6 -
|
||||
an object detection operation, which can be performed on two
|
||||
:description: Learn about ExperimentalDetectronROIFeatureExtractor-6 -
|
||||
an object detection operation, which can be performed on two
|
||||
required input tensors.
|
||||
|
||||
**Versioned name**: *ExperimentalDetectronROIFeatureExtractor-6*
|
||||
|
|
@ -61,7 +61,7 @@ For more details please see the following source: `Feature Pyramid Networks for
|
|||
|
||||
* **Description**: The *aligned* attribute specifies add offset (``-0.5``) to ROIs sizes or not.
|
||||
* **Range of values**:
|
||||
|
||||
|
||||
* ``true`` - add offset to ROIs sizes
|
||||
* ``false`` - do not add offset to ROIs sizes
|
||||
* **Type**: boolean
|
||||
|
|
@ -70,7 +70,7 @@ For more details please see the following source: `Feature Pyramid Networks for
|
|||
|
||||
**Inputs**:
|
||||
|
||||
* **1**: 2D input tensor of type *T* with shape ``[number_of_ROIs, 4]`` providing the ROIs as 4-tuples: [x 1, y 1, x<sub>2</sub>, y<sub>2</sub>]. Coordinates *x* and *y* are refer to the network's input *image_size*. **Required.**
|
||||
* **1**: 2D input tensor of type *T* with shape ``[number_of_ROIs, 4]`` providing the ROIs as 4-tuples: .. math:: [x_{1}, y_{1}, x_{2}, y_{2}]. Coordinates *x* and *y* are refer to the network's input *image_size*. **Required.**
|
||||
* **2**, ..., **L**: Pyramid of 4D input tensors with feature maps. Shape must be ``[1, number_of_channels, layer_size[l], layer_size[l]]``. The number of channels must be the same for all layers of the pyramid. The layer width and height must equal to the ``layer_size[l] = image_size / pyramid_scales[l]``. **Required.**
|
||||
|
||||
**Outputs**:
|
||||
|
|
@ -86,7 +86,7 @@ For more details please see the following source: `Feature Pyramid Networks for
|
|||
|
||||
.. code-block:: xml
|
||||
:force:
|
||||
|
||||
|
||||
<layer ... type="ExperimentalDetectronROIFeatureExtractor" version="opset6">
|
||||
<data aligned="false" output_size="7" pyramid_scales="4,8,16,32,64" sampling_ratio="2"/>
|
||||
<input>
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ Assign
|
|||
|
||||
ReadValue, Assign, and Variable define a coherent mechanism for reading, writing and
|
||||
storing a memory buffer between inference calls. More details can be found on the
|
||||
:doc:`StateAPI<../../../../../openvino-workflow/running-inference/stateful-models-intro>` ../../../../../documentation page.
|
||||
:doc:`State API <../../../../../../openvino-workflow/running-inference/stateful-models-intro>` documentation page.
|
||||
|
||||
*Assign* sets an input value to the ``variable_id`` variable. This value will be read
|
||||
by the *ReadValue* operation on the next inference call if it has not been reset.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ Loop
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about Loop-5 - an infrastructure operation, which
|
||||
:description: Learn about Loop-5 - an infrastructure operation, which
|
||||
can be performed on two required and one optional input tensor.
|
||||
|
||||
**Versioned name**: *Loop-5*
|
||||
|
|
@ -115,6 +115,7 @@ Loop operation description in the IR also has several special sections: ``body``
|
|||
|
||||
* **Description**: if *axis* is specified for ``output`` entry, then it is an axis to concatenate the body ``Result`` output across all iterations.
|
||||
If *axis* is specified for ``input`` entry, then it is an axis to iterate through, it triggers the slicing of the input tensor.
|
||||
|
||||
* **Range of values**: an integer. Negative value means counting dimension from the end.
|
||||
* **Type**: ``int``
|
||||
* **Default value**: None
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ ReadValue
|
|||
|
||||
*ReadValue*, *Assign*, and *Variable* define a coherent mechanism for reading, writing,
|
||||
and storing some memory buffer between inference calls. More details can be found on the
|
||||
:doc:`StateAPI<../../../../../openvino-workflow/running-inference/stateful-models-intro>` ../../../../../documentation page.
|
||||
:doc:`StateAPI <../../../../../../openvino-workflow/running-inference/stateful-models-intro>` documentation page.
|
||||
|
||||
If the 1st input is provided and this is the first inference or reset has been called,
|
||||
*ReadValue* returns the value from the 1st input.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ ExtractImagePatches
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about ExtractImagePatches-3 - a data movement operation,
|
||||
:description: Learn about ExtractImagePatches-3 - a data movement operation,
|
||||
which can be performed on a 4D input tensor.
|
||||
|
||||
**Versioned name**: *ExtractImagePatches-3*
|
||||
|
|
@ -71,7 +71,7 @@ The "auto_pad" attribute has no effect on the size of each patch, it determines
|
|||
|
||||
.. code-block:: xml
|
||||
:force:
|
||||
|
||||
|
||||
<layer type="ExtractImagePatches" ...>
|
||||
<data sizes="3,3" strides="5,5" rates="1,1" auto_pad="valid"/>
|
||||
<input>
|
||||
|
|
@ -97,7 +97,7 @@ Image is a ``1 x 1 x 10 x 10`` array that contains the numbers 1 through 100. We
|
|||
1. ``sizes="3,3", strides="5,5", rates="1,1", auto_pad="valid"``
|
||||
|
||||
.. math::
|
||||
|
||||
|
||||
\begin{bmatrix}
|
||||
x & x & x & 4 & 5 & x & x & x & 9 & 10 \\
|
||||
x & x & x & 14 & 15 & x & x & x & 19 & 20 \\
|
||||
|
|
@ -110,101 +110,101 @@ Image is a ``1 x 1 x 10 x 10`` array that contains the numbers 1 through 100. We
|
|||
81 & 82 & 83 & 84 & 85 & 86 & 87 & 88 & 89 & 90 \\
|
||||
91 & 92 & 93 & 94 & 95 & 96 & 79 & 98 & 99 & 100
|
||||
\end{bmatrix}
|
||||
|
||||
|
||||
|
||||
|
||||
output:
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
|
||||
[[[[ 1 6]
|
||||
[51 56]]
|
||||
|
||||
|
||||
[[ 2 7]
|
||||
[52 57]]
|
||||
|
||||
|
||||
[[ 3 8]
|
||||
[53 58]]
|
||||
|
||||
|
||||
[[11 16]
|
||||
[61 66]]
|
||||
|
||||
|
||||
[[12 17]
|
||||
[62 67]]
|
||||
|
||||
|
||||
[[13 18]
|
||||
[63 68]]
|
||||
|
||||
|
||||
[[21 26]
|
||||
[71 76]]
|
||||
|
||||
|
||||
[[22 27]
|
||||
[72 77]]
|
||||
|
||||
|
||||
[[23 28]
|
||||
[73 78]]]]
|
||||
|
||||
|
||||
output shape: `[1, 9, 2, 2]`
|
||||
|
||||
2. ``sizes="4,4", strides="8,8", rates="1,1", auto_pad="valid"``
|
||||
|
||||
|
||||
.. math::
|
||||
|
||||
\begin{bmatrix}
|
||||
x & x & x & x & 5 & 6 & 7 & 8 & 9 & 10 \\
|
||||
x & x & x & x & 15 & 16 & 17 & 18 & 19 & 20 \\
|
||||
x & x & x & x & 25 & 26 & 27 & 28 & 29 & 30 \\
|
||||
x & x & x & x & 35 & 36 & 37 & 38 & 39 & 40 \\
|
||||
41 & 42 & 43 & 44 & 45 & 46 & 47 & 48 & 49 & 50 \\
|
||||
51 & 52 & 53 & 54 & 55 & 56 & 57 & 58 & 59 & 60 \\
|
||||
61 & 62 & 63 & 64 & 65 & 66 & 67 & 68 & 69 & 70 \\
|
||||
71 & 72 & 73 & 74 & 75 & 76 & 77 & 78 & 79 & 80 \\
|
||||
81 & 82 & 83 & 84 & 85 & 86 & 87 & 88 & 89 & 90 \\
|
||||
91 & 92 & 93 & 94 & 95 & 96 & 79 & 98 & 99 & 100
|
||||
\end{bmatrix}
|
||||
|
||||
|
||||
|
||||
\begin{bmatrix}
|
||||
x & x & x & x & 5 & 6 & 7 & 8 & 9 & 10 \\
|
||||
x & x & x & x & 15 & 16 & 17 & 18 & 19 & 20 \\
|
||||
x & x & x & x & 25 & 26 & 27 & 28 & 29 & 30 \\
|
||||
x & x & x & x & 35 & 36 & 37 & 38 & 39 & 40 \\
|
||||
41 & 42 & 43 & 44 & 45 & 46 & 47 & 48 & 49 & 50 \\
|
||||
51 & 52 & 53 & 54 & 55 & 56 & 57 & 58 & 59 & 60 \\
|
||||
61 & 62 & 63 & 64 & 65 & 66 & 67 & 68 & 69 & 70 \\
|
||||
71 & 72 & 73 & 74 & 75 & 76 & 77 & 78 & 79 & 80 \\
|
||||
81 & 82 & 83 & 84 & 85 & 86 & 87 & 88 & 89 & 90 \\
|
||||
91 & 92 & 93 & 94 & 95 & 96 & 79 & 98 & 99 & 100
|
||||
\end{bmatrix}
|
||||
|
||||
|
||||
output:
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
|
||||
[[[[ 1]]
|
||||
|
||||
|
||||
[[ 2]]
|
||||
|
||||
|
||||
[[ 3]]
|
||||
|
||||
|
||||
[[ 4]]
|
||||
|
||||
|
||||
[[11]]
|
||||
|
||||
|
||||
[[12]]
|
||||
|
||||
|
||||
[[13]]
|
||||
|
||||
|
||||
[[14]]
|
||||
|
||||
|
||||
[[21]]
|
||||
|
||||
|
||||
[[22]]
|
||||
|
||||
|
||||
[[23]]
|
||||
|
||||
|
||||
[[24]]
|
||||
|
||||
|
||||
[[31]]
|
||||
|
||||
|
||||
[[32]]
|
||||
|
||||
|
||||
[[33]]
|
||||
|
||||
|
||||
[[34]]]]
|
||||
|
||||
|
||||
output shape: ``[1, 16, 1, 1]``
|
||||
|
||||
3. ``sizes="4,4", strides="9,9", rates="1,1", auto_pad="same_upper"``
|
||||
|
||||
|
||||
.. math::
|
||||
|
||||
|
||||
\begin{bmatrix}
|
||||
x & x & x & x & 0 & 0 & 0 & 0 & 0 & x & x & x & x\\
|
||||
x & x & x & x & 4 & 5 & 6 & 7 & 8 & x & x & x & x\\
|
||||
|
|
@ -220,67 +220,67 @@ Image is a ``1 x 1 x 10 x 10`` array that contains the numbers 1 through 100. We
|
|||
x & x & x & x & 0 & 0 & 0 & 0 & 0 & x & x & x & x\\
|
||||
x & x & x & x & 0 & 0 & 0 & 0 & 0 & x & x & x & x
|
||||
\end{bmatrix}
|
||||
|
||||
|
||||
output:
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
|
||||
[[[[ 0 0]
|
||||
[ 0 89]]
|
||||
|
||||
|
||||
[[ 0 0]
|
||||
[ 81 90]]
|
||||
|
||||
|
||||
[[ 0 0]
|
||||
[ 82 0]]
|
||||
|
||||
|
||||
[[ 0 0]
|
||||
[ 83 0]]
|
||||
|
||||
|
||||
[[ 0 9]
|
||||
[ 0 99]]
|
||||
|
||||
|
||||
[[ 1 10]
|
||||
[ 91 100]]
|
||||
|
||||
|
||||
[[ 2 0]
|
||||
[ 92 0]]
|
||||
|
||||
|
||||
[[ 3 0]
|
||||
[ 93 0]]
|
||||
|
||||
|
||||
[[ 0 19]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 11 20]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 12 0]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 13 0]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 0 29]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 21 30]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 22 0]
|
||||
[ 0 0]]
|
||||
|
||||
|
||||
[[ 23 0]
|
||||
[ 0 0]]]]
|
||||
|
||||
|
||||
output shape: ``[1, 16, 2, 2]``
|
||||
|
||||
4. ``sizes="3,3", strides="5,5", rates="2,2", auto_pad="valid"``
|
||||
|
||||
|
||||
This time we use the symbols ``x``, ``y``, ``z`` and ``k`` to distinguish the patches:
|
||||
|
||||
|
||||
.. math::
|
||||
|
||||
|
||||
\begin{bmatrix}
|
||||
x & 2 & x & 4 & x & y & 7 & y & 9 & y \\
|
||||
11 & 12 & 13 & 14 & 15 & 16 & 17 & 18 & 19 & 20 \\
|
||||
|
|
@ -293,46 +293,46 @@ Image is a ``1 x 1 x 10 x 10`` array that contains the numbers 1 through 100. We
|
|||
81 & 82 & 83 & 84 & 85 & 86 & 87 & 88 & 89 & 90 \\
|
||||
z & 92 & z & 94 & z & k & 79 & k & 99 & k
|
||||
\end{bmatrix}
|
||||
|
||||
|
||||
output:
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
|
||||
[[[[ 1 6]
|
||||
[ 51 56]]
|
||||
|
||||
|
||||
[[ 3 8]
|
||||
[ 53 58]]
|
||||
|
||||
|
||||
[[ 5 10]
|
||||
[ 55 60]]
|
||||
|
||||
|
||||
[[ 21 26]
|
||||
[ 71 76]]
|
||||
|
||||
|
||||
[[ 23 28]
|
||||
[ 73 78]]
|
||||
|
||||
|
||||
[[ 25 30]
|
||||
[ 75 80]]
|
||||
|
||||
|
||||
[[ 41 46]
|
||||
[ 91 96]]
|
||||
|
||||
|
||||
[[ 43 48]
|
||||
[ 93 98]]
|
||||
|
||||
|
||||
[[ 45 50]
|
||||
[ 95 100]]]]
|
||||
|
||||
|
||||
output_shape: ``[1, 9, 2, 2]``
|
||||
|
||||
5. ``sizes="2,2", strides="3,3", rates="1,1", auto_pad="valid"``
|
||||
|
||||
|
||||
Image is a ``1 x 2 x 5 x 5`` array that contains two feature maps where feature map with coordinate 0 contains numbers in a range ``[1, 25]`` and feature map with coordinate 1 contains numbers in a range ``[26, 50]``
|
||||
|
||||
|
||||
.. math::
|
||||
|
||||
|
||||
\begin{bmatrix}
|
||||
x & x & 3 & x & x\\
|
||||
x & x & 8 & x & x\\
|
||||
|
|
@ -347,35 +347,35 @@ Image is a ``1 x 1 x 10 x 10`` array that contains the numbers 1 through 100. We
|
|||
x & x & 43 & x & x\\
|
||||
x & x & 48 & x & x
|
||||
\end{bmatrix}
|
||||
|
||||
|
||||
output:
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
|
||||
[[[[ 1 4]
|
||||
[16 19]]
|
||||
|
||||
|
||||
[[26 29]
|
||||
[41 44]]
|
||||
|
||||
|
||||
[[ 2 5]
|
||||
[17 20]]
|
||||
|
||||
|
||||
[[27 30]
|
||||
[42 45]]
|
||||
|
||||
|
||||
[[ 6 9]
|
||||
[21 24]]
|
||||
|
||||
|
||||
[[31 34]
|
||||
[46 49]]
|
||||
|
||||
|
||||
[[ 7 10]
|
||||
[22 25]]
|
||||
|
||||
|
||||
[[32 35]
|
||||
[47 50]]]]
|
||||
|
||||
|
||||
output shape: ``[1, 8, 2, 2]``
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ GatherElements
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about GatherElements-6 - a data movement operation,
|
||||
:description: Learn about GatherElements-6 - a data movement operation,
|
||||
which can be performed on two required input tensors.
|
||||
|
||||
**Versioned name**: *GatherElements-6*
|
||||
|
|
@ -15,10 +15,10 @@ GatherElements
|
|||
|
||||
**Short description**: *GatherElements* takes elements from the input ``data`` tensor at positions specified in the ``indices`` tensor.
|
||||
|
||||
**Detailed description** *GatherElements* takes elements from the ``data`` tensor at positions specified
|
||||
in the ``indices`` tensor. The ``data`` and ``indices`` tensors have the same rank ``r >= 1``. Optional
|
||||
attribute ``axis`` determines along which axis elements with indices specified in ``indices`` are taken.
|
||||
The ``indices`` tensor has the same shape as the ``data`` tensor except for the ``axis`` dimension.
|
||||
**Detailed description** *GatherElements* takes elements from the ``data`` tensor at positions specified
|
||||
in the ``indices`` tensor. The ``data`` and ``indices`` tensors have the same rank ``r >= 1``. Optional
|
||||
attribute ``axis`` determines along which axis elements with indices specified in ``indices`` are taken.
|
||||
The ``indices`` tensor has the same shape as the ``data`` tensor except for the ``axis`` dimension.
|
||||
Output consists of values (gathered from the ``data`` tensor) for each element in the ``indices`` tensor
|
||||
and has the same shape as ``indices``.
|
||||
|
||||
|
|
@ -90,6 +90,7 @@ Example 3 ``indices`` has lesser (than ``data``) shape:
|
|||
**Attributes**:
|
||||
|
||||
* *axis*
|
||||
|
||||
* **Description**: Which axis to gather on. Negative value means counting dimensions from the back.
|
||||
* **Range of values**: ``[-r, r-1]`` where ``r = rank(data)``.
|
||||
* **Type**: int
|
||||
|
|
@ -99,7 +100,7 @@ Example 3 ``indices`` has lesser (than ``data``) shape:
|
|||
**Inputs**:
|
||||
|
||||
* **1**: Tensor of type *T*. This is a tensor of a ``rank >= 1``. **Required.**
|
||||
* **2**: Tensor of type *T_IND* with the same rank as the input. All index values are expected to be
|
||||
* **2**: Tensor of type *T_IND* with the same rank as the input. All index values are expected to be
|
||||
within bounds ``[0, s-1]``, where ``s`` is size along ``axis`` dimension of the ``data`` tensor. **Required.**
|
||||
|
||||
**Outputs**:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ GroupNormalization
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about GroupNormalization-12 - a normalization operation,
|
||||
:description: Learn about GroupNormalization-12 - a normalization operation,
|
||||
which can be performed on three required input tensors.
|
||||
|
||||
**Versioned name**: *GroupNormalization-12*
|
||||
|
|
@ -61,31 +61,31 @@ The operation is applied per batch, per group of channels. This means that the e
|
|||
.. code-block:: xml
|
||||
:force:
|
||||
|
||||
<layer ... type="GroupNormalization">
|
||||
<data epsilon="1e-5" num_groups="4"/>
|
||||
<input>
|
||||
<port id="0">
|
||||
<dim>3</dim>
|
||||
<dim>12</dim>
|
||||
<dim>100</dim>
|
||||
<dim>100</dim>
|
||||
</port>
|
||||
<port id="1">
|
||||
<dim>12</dim> <!-- 12 scale values, 1 for each channel -->
|
||||
</port>
|
||||
<port id="2">
|
||||
<dim>12</dim> <!-- 12 bias values, 1 for each channel -->
|
||||
</port>
|
||||
</input>
|
||||
<output>
|
||||
<port id="3">
|
||||
<dim>3</dim>
|
||||
<dim>12</dim>
|
||||
<dim>100</dim>
|
||||
<dim>100</dim>
|
||||
</port>
|
||||
</output>
|
||||
</layer>
|
||||
<layer ... type="GroupNormalization">
|
||||
<data epsilon="1e-5" num_groups="4"/>
|
||||
<input>
|
||||
<port id="0">
|
||||
<dim>3</dim>
|
||||
<dim>12</dim>
|
||||
<dim>100</dim>
|
||||
<dim>100</dim>
|
||||
</port>
|
||||
<port id="1">
|
||||
<dim>12</dim> <!-- 12 scale values, 1 for each channel -->
|
||||
</port>
|
||||
<port id="2">
|
||||
<dim>12</dim> <!-- 12 bias values, 1 for each channel -->
|
||||
</port>
|
||||
</input>
|
||||
<output>
|
||||
<port id="3">
|
||||
<dim>3</dim>
|
||||
<dim>12</dim>
|
||||
<dim>100</dim>
|
||||
<dim>100</dim>
|
||||
</port>
|
||||
</output>
|
||||
</layer>
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ MaxPool
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about MaxPool-8 - a pooling operation, which can
|
||||
:description: Learn about MaxPool-8 - a pooling operation, which can
|
||||
be performed on a 3D, 4D or 5D input tensor.
|
||||
|
||||
**Versioned name**: *MaxPool-8*
|
||||
|
|
@ -103,18 +103,18 @@ MaxPool
|
|||
|
||||
**Outputs**:
|
||||
|
||||
* **1**: Input shape can be either ``[N, C, H]``, ``[N, C, H, W]``, or ``[N, C, H, W, D]``. The corresponding output shape is ``[N, C, H_out]``, ``[N, C, H_out, W_out]`` or ``[N, C, H_out, W_out, D_out]``. Output tensor has the same data type as the input tensor.
|
||||
* **1**: Input shape can be either ``[N, C, H]``, ``[N, C, H, W]``, or ``[N, C, H, W, D]``. The corresponding output shape is ``[N, C, H_out]``, ``[N, C, H_out, W_out]`` or ``[N, C, H_out, W_out, D_out]``. Output tensor has the same data type as the input tensor.
|
||||
|
||||
* **2**: Output tensor of type *T_IND* with indices of values selected by the pooling operation.
|
||||
Shape of this output matches the first output. The type of this output can be specified using the ``index_element_type`` attribute.
|
||||
Values are computed as indices in a tensor flattened to 1D, not considering padding. Examples for a 5D input tensor:
|
||||
* **2**: Output tensor of type *T_IND* with indices of values selected by the pooling operation.
|
||||
Shape of this output matches the first output. The type of this output can be specified using the ``index_element_type`` attribute.
|
||||
Values are computed as indices in a tensor flattened to 1D, not considering padding. Examples for a 5D input tensor:
|
||||
|
||||
* When ``axis == 0``, the values are in the range ``[0, N * C * H * W * D)``.
|
||||
* When ``axis == 2``, the values are in the range ``[0, H * W * D)``.
|
||||
* When ``axis == 0``, the values are in the range ``[0, N * C * H * W * D)``.
|
||||
* When ``axis == 2``, the values are in the range ``[0, H * W * D)``.
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
The values of this output can only be calculated correctly if ``pads_value`` is set to ``-infinity``.
|
||||
The values of this output can only be calculated correctly if ``pads_value`` is set to ``-infinity``.
|
||||
|
||||
|
||||
**Types**
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ GRUCell
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about GRUCell-3 - a sequence processing operation, which
|
||||
:description: Learn about GRUCell-3 - a sequence processing operation, which
|
||||
can be performed on four required and one optional input tensor.
|
||||
|
||||
**Versioned name**: *GRUCell-3*
|
||||
|
|
@ -65,8 +65,8 @@ GRUCell
|
|||
|
||||
* *linear_before_reset*
|
||||
|
||||
* **Description**: *linear_before_reset* flag denotes if the layer behaves according to the modification
|
||||
of *GRUCell* described in the formula in the `ONNX ../../../../../documentation <https://github.com/onnx/onnx/blob/master/docs/Operators.md#GRU>`__.
|
||||
* **Description**: *linear_before_reset* flag denotes if the layer behaves according to the modification
|
||||
of *GRUCell* described in the formula in the `ONNX documentation <https://github.com/onnx/onnx/blob/master/docs/Operators.md#GRU>`__.
|
||||
* **Range of values**: true or false
|
||||
* **Type**: ``boolean``
|
||||
* **Default value**: false
|
||||
|
|
@ -78,8 +78,8 @@ GRUCell
|
|||
* **2**: ``initial_hidden_state`` - 2D tensor of type *T* ``[batch_size, hidden_size]``. **Required.**
|
||||
* **3**: ``W`` - 2D tensor of type *T* ``[3 * hidden_size, input_size]``, the weights for matrix multiplication, gate order: zrh. **Required.**
|
||||
* **4**: ``R`` - 2D tensor of type *T* ``[3 * hidden_size, hidden_size]``, the recurrence weights for matrix multiplication, gate order: zrh. **Required.**
|
||||
* **5**: ``B`` - 1D tensor of type *T*. If *linear_before_reset* is set to 1, then the shape is ``[4 * hidden_size]`` -
|
||||
the sum of biases for z and r gates (weights and recurrence weights), the biases for h gate are placed separately.
|
||||
* **5**: ``B`` - 1D tensor of type *T*. If *linear_before_reset* is set to 1, then the shape is ``[4 * hidden_size]`` -
|
||||
the sum of biases for z and r gates (weights and recurrence weights), the biases for h gate are placed separately.
|
||||
Otherwise the shape is ``[3 * hidden_size]``, the sum of biases (weights and recurrence weights). **Optional.**
|
||||
|
||||
**Outputs**
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ GRUSequence
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about GRUSequence-5 - a sequence processing operation, which
|
||||
:description: Learn about GRUSequence-5 - a sequence processing operation, which
|
||||
can be performed on six required input tensors.
|
||||
|
||||
**Versioned name**: *GRUSequence-5*
|
||||
|
|
@ -16,11 +16,11 @@ GRUSequence
|
|||
|
||||
**Detailed description**
|
||||
|
||||
A single cell in the sequence is implemented in the same way as in *GRUCell* operation. *GRUSequence*
|
||||
represents a sequence of GRU cells. The sequence can be connected differently depending on
|
||||
``direction`` attribute that specifies the direction of traversing of input data along sequence
|
||||
dimension or specifies whether it should be a bidirectional sequence. The most of the attributes
|
||||
are in sync with the specification of ONNX GRU operator defined
|
||||
A single cell in the sequence is implemented in the same way as in *GRUCell* operation. *GRUSequence*
|
||||
represents a sequence of GRU cells. The sequence can be connected differently depending on
|
||||
``direction`` attribute that specifies the direction of traversing of input data along sequence
|
||||
dimension or specifies whether it should be a bidirectional sequence. The most of the attributes
|
||||
are in sync with the specification of ONNX GRU operator defined
|
||||
`GRUCell <https://github.com/onnx/onnx/blob/master/docs/Operators.md#gru>`__
|
||||
|
||||
|
||||
|
|
@ -35,7 +35,7 @@ are in sync with the specification of ONNX GRU operator defined
|
|||
|
||||
* *activations*
|
||||
|
||||
* **Description**: *activations* specifies activation functions for gates, there are two gates,
|
||||
* **Description**: *activations* specifies activation functions for gates, there are two gates,
|
||||
so two activation functions should be specified as a value for this attributes
|
||||
* **Range of values**: any combination of *relu*, *sigmoid*, *tanh*
|
||||
* **Type**: a list of strings
|
||||
|
|
@ -44,7 +44,7 @@ are in sync with the specification of ONNX GRU operator defined
|
|||
|
||||
* *activations_alpha, activations_beta*
|
||||
|
||||
* **Description**: *activations_alpha, activations_beta* attributes of functions;
|
||||
* **Description**: *activations_alpha, activations_beta* attributes of functions;
|
||||
applicability and meaning of these attributes depends on chosen activation functions
|
||||
* **Range of values**: a list of floating-point numbers
|
||||
* **Type**: ``float[]``
|
||||
|
|
@ -61,8 +61,8 @@ are in sync with the specification of ONNX GRU operator defined
|
|||
|
||||
* *direction*
|
||||
|
||||
* **Description**: Specify if the RNN is forward, reverse, or bidirectional. If it is one of *forward* or *reverse*
|
||||
then ``num_directions = 1``, if it is *bidirectional*, then ``num_directions = 2``. This ``num_directions``
|
||||
* **Description**: Specify if the RNN is forward, reverse, or bidirectional. If it is one of *forward* or *reverse*
|
||||
then ``num_directions = 1``, if it is *bidirectional*, then ``num_directions = 2``. This ``num_directions``
|
||||
value specifies input/output shape requirements.
|
||||
* **Range of values**: *forward*, *reverse*, *bidirectional*
|
||||
* **Type**: ``string``
|
||||
|
|
@ -70,8 +70,8 @@ are in sync with the specification of ONNX GRU operator defined
|
|||
|
||||
* *linear_before_reset*
|
||||
|
||||
* **Description**: *linear_before_reset* flag denotes if the layer behaves according to the modification
|
||||
of *GRUCell* described in the formula in the `ONNX ../../../../../documentation <https://github.com/onnx/onnx/blob/master/docs/Operators.md#GRU>`__.
|
||||
* **Description**: *linear_before_reset* flag denotes if the layer behaves according to the modification
|
||||
of *GRUCell* described in the formula in the `ONNX documentation <https://github.com/onnx/onnx/blob/master/docs/Operators.md#GRU>`__.
|
||||
* **Range of values**: True or False
|
||||
* **Type**: ``boolean``
|
||||
* **Default value**: False
|
||||
|
|
@ -79,19 +79,19 @@ are in sync with the specification of ONNX GRU operator defined
|
|||
|
||||
**Inputs**
|
||||
|
||||
* **1**: ``X`` - 3D tensor of type *T1* ``[batch_size, seq_length, input_size]``, input data.
|
||||
* **1**: ``X`` - 3D tensor of type *T1* ``[batch_size, seq_length, input_size]``, input data.
|
||||
It differs from GRUCell 1st input only by additional axis with size ``seq_length``. **Required.**
|
||||
* **2**: ``initial_hidden_state`` - 3D tensor of type *T1* ``[batch_size, num_directions, hidden_size]``,
|
||||
* **2**: ``initial_hidden_state`` - 3D tensor of type *T1* ``[batch_size, num_directions, hidden_size]``,
|
||||
input hidden state data. **Required.**
|
||||
* **3**: ``sequence_lengths`` - 1D tensor of type *T2* ``[batch_size]``, specifies real sequence lengths
|
||||
* **3**: ``sequence_lengths`` - 1D tensor of type *T2* ``[batch_size]``, specifies real sequence lengths
|
||||
for each batch element. In case of negative values in this input, the operation behavior is undefined. **Required.**
|
||||
* **4**: ``W`` - 3D tensor of type *T1* ``[num_directions, 3 * hidden_size, input_size]``,
|
||||
* **4**: ``W`` - 3D tensor of type *T1* ``[num_directions, 3 * hidden_size, input_size]``,
|
||||
the weights for matrix multiplication, gate order: zrh. **Required.**
|
||||
* **5**: ``R`` - 3D tensor of type *T1* ``[num_directions, 3 * hidden_size, hidden_size]``,
|
||||
* **5**: ``R`` - 3D tensor of type *T1* ``[num_directions, 3 * hidden_size, hidden_size]``,
|
||||
the recurrence weights for matrix multiplication, gate order: zrh. **Required.**
|
||||
* **6**: ``B`` - 2D tensor of type *T*. If *linear_before_reset* is set to 1, then the shape
|
||||
is ``[num_directions, 4 * hidden_size]`` - the sum of biases for z and r gates (weights and recurrence weights),
|
||||
the biases for h gate are placed separately. Otherwise the shape is ``[num_directions, 3 * hidden_size]``,
|
||||
* **6**: ``B`` - 2D tensor of type *T*. If *linear_before_reset* is set to 1, then the shape
|
||||
is ``[num_directions, 4 * hidden_size]`` - the sum of biases for z and r gates (weights and recurrence weights),
|
||||
the biases for h gate are placed separately. Otherwise the shape is ``[num_directions, 3 * hidden_size]``,
|
||||
the sum of biases (weights and recurrence weights). **Required.**
|
||||
|
||||
**Outputs**
|
||||
|
|
|
|||
|
|
@ -5,18 +5,18 @@ LSTMSequence
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about LSTMSequence-1 - a sequence processing operation, which
|
||||
:description: Learn about LSTMSequence-1 - a sequence processing operation, which
|
||||
can be performed on seven required input tensors.
|
||||
|
||||
**Versioned name**: *LSTMSequence-1*
|
||||
|
||||
**Category**: *Sequence processing*
|
||||
|
||||
**Short description**: *LSTMSequence* operation represents a series of LSTM cells. Each cell is implemented as ref:`LSTMCell <LSTMCell>` operation.
|
||||
**Short description**: *LSTMSequence* operation represents a series of LSTM cells. Each cell is implemented as :doc:`LSTM Cell <lstm-cell-1>` operation.
|
||||
|
||||
**Detailed description**
|
||||
|
||||
A single cell in the sequence is implemented in the same way as in ref:`LSTMCell <LSTMCell>` operation. *LSTMSequence* represents a sequence of LSTM cells. The sequence can be connected differently depending on ``direction`` attribute that specifies the direction of traversing of input data along sequence dimension or specifies whether it should be a bidirectional sequence. The most of the attributes are in sync with the specification of ONNX LSTM operator defined `LSTMCell <https://github.com/onnx/onnx/blob/master/docs/Operators.md#lstm>`__ .
|
||||
A single cell in the sequence is implemented in the same way as in :doc:`LSTM Cell <lstm-cell-1>` operation. *LSTMSequence* represents a sequence of LSTM cells. The sequence can be connected differently depending on ``direction`` attribute that specifies the direction of traversing of input data along sequence dimension or specifies whether it should be a bidirectional sequence. The most of the attributes are in sync with the specification of ONNX LSTM operator defined `LSTMCell <https://github.com/onnx/onnx/blob/master/docs/Operators.md#lstm>`__ .
|
||||
|
||||
|
||||
**Attributes**
|
||||
|
|
|
|||
|
|
@ -5,18 +5,18 @@ RNNSequence
|
|||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about RNNSequence-5 - a sequence processing operation, which
|
||||
:description: Learn about RNNSequence-5 - a sequence processing operation, which
|
||||
can be performed on six required input tensors.
|
||||
|
||||
**Versioned name**: *RNNSequence-5*
|
||||
|
||||
**Category**: *Sequence processing*
|
||||
|
||||
**Short description**: *RNNSequence* operation represents a series of RNN cells. Each cell is implemented as `RNNCell <#RNNCell>`__ operation.
|
||||
**Short description**: *RNNSequence* operation represents a series of RNN cells. Each cell is implemented as :doc:`RNNCell <rnn-cell-3>` operation.
|
||||
|
||||
**Detailed description**
|
||||
|
||||
A single cell in the sequence is implemented in the same way as in `RNNCell <#RNNCell>`__ operation. *RNNSequence* represents a sequence of RNN cells. The sequence can be connected differently depending on `direction` attribute that specifies the direction of traversing of input data along sequence dimension or specifies whether it should be a bidirectional sequence. The most of the attributes are in sync with the specification of ONNX RNN operator defined `RNNCell <https://github.com/onnx/onnx/blob/master/docs/Operators.md#rnn>`__.
|
||||
A single cell in the sequence is implemented in the same way as in :doc:`RNNCell <rnn-cell-3>` operation. *RNNSequence* represents a sequence of RNN cells. The sequence can be connected differently depending on `direction` attribute that specifies the direction of traversing of input data along sequence dimension or specifies whether it should be a bidirectional sequence. The most of the attributes are in sync with the specification of ONNX RNN operator defined `RNNCell <https://github.com/onnx/onnx/blob/master/docs/Operators.md#rnn>`__.
|
||||
|
||||
|
||||
**Attributes**
|
||||
|
|
|
|||
|
|
@ -60,9 +60,9 @@ omitting training-related parameter.
|
|||
|
||||
* **4**: ``attention_mask`` - two options available. ``attention_mask`` is ignored if ``causal`` is set to ``True``. **Optional.**
|
||||
|
||||
** at least 3 dimensional tensor of type *T* or ``boolean`` and shape ``[N, ..., L, S]``.
|
||||
* at least 3 dimensional tensor of type *T* or ``boolean`` and shape ``[N, ..., L, S]``.
|
||||
|
||||
** a scalar of type *T* with value ``0``. Scalar zero value signals that applying an attention mask is not necessary (similar to specifying attention_mask=None in the provided pseudo-code).
|
||||
* a scalar of type *T* with value ``0``. Scalar zero value signals that applying an attention mask is not necessary (similar to specifying attention_mask=None in the provided pseudo-code).
|
||||
|
||||
* **5**: ``scale`` a scalar tensor of type *T*, an alternative scale factor instead of 1/sqrt(query.shape[-1]) used by default in the pseudo-code above. **Optional.**
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d8d3621c4be5d3382cb164a19676253412f85b5f47fac27b024c726f1571647e
|
||||
size 380663
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ff2d26dc0b86f339458a2fafbbd6a88daf3d3dc6fcefb636243f42a6e91bc328
|
||||
size 492066
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9e5bfd5ca52ea6660e0ff67afefc98d64941eab6e8b464116242a6e044f318f5
|
||||
size 207602
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:756c225ee8e1da046e0210bf0696185b3939378f10b4ed6d757e43070d379436
|
||||
size 135804
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:08d4116490ab329636fced24c292636fbe00856976b19e5219e433bc2c6e4e16
|
||||
size 190590
|
||||
Loading…
Reference in New Issue