[Spec][Internal][Op] Specification for RMS internal op (#24564)
### Details: - Dev documentation for the existing [internal::RMS](70142121c1/src/common/transformations/include/ov_ops/rms.hpp) op (specified as is without any additional features, to be developed and updated if needed) - RMS was created initially as a [custom gpu RMS](9fadb5ac64/src/plugins/intel_gpu/include/intel_gpu/op/rms.hpp) operation, and it's going to be moved to op::internal::RMS to be available for common transformations - Similar approach as it was done for AUGRUCell/AUGRUSeqence, so the place for such documents has been already agreed and it's not a part of the official web docs tree ### Tickets: - 134914, dicsussion 129027
This commit is contained in:
parent
248c841e85
commit
b71eab2555
|
|
@ -0,0 +1,84 @@
|
|||
.. {#openvino_docs_ops_internal_RMS}
|
||||
|
||||
RMS
|
||||
===
|
||||
|
||||
|
||||
.. meta::
|
||||
:description: Learn about RMS a normalization operation.
|
||||
|
||||
**Versioned name**: *RMS*
|
||||
|
||||
**Category**: *Normalization*
|
||||
|
||||
**Short description**: Calculates Root Mean Square (RMS) normalization of the input tensor.
|
||||
|
||||
**Detailed description**
|
||||
|
||||
*RMS* operation performs Root Mean Square (RMS) normalization on a given input ``data`` along the last dimension of the input.
|
||||
`Reference <https://arxiv.org/abs/1910.07467>`__.
|
||||
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
(x / Sqrt(ReduceMean(x^2, -1) + eps)) * scale
|
||||
|
||||
|
||||
**Attributes**
|
||||
|
||||
* *epsilon*
|
||||
|
||||
* **Description**: A very small value added to the variance for numerical stability. Ensures that division by zero does not occur for any normalized element.
|
||||
* **Range of values**: a positive floating-point number
|
||||
* **Type**: ``float``
|
||||
* **Required**: *yes*
|
||||
|
||||
* *output_type*
|
||||
|
||||
* **Description**: The precision for output type conversion, after scaling. It's used for output type compression to f16.
|
||||
* **Range of values**: Supported floating point type: "f16", "undefined"
|
||||
* **Type**: ``string``
|
||||
* **Default value**: "undefined" (means that output type is set to the same as the input type)
|
||||
* **Required**: *no*
|
||||
|
||||
|
||||
**Inputs**
|
||||
|
||||
* **1**: ``data`` - Input data to be normalized. A tensor of type *T* and arbitrary shape. **Required.**
|
||||
|
||||
* **2**: ``scale`` - A tensor of type *T* containing the scale values for . The shape should be broadcastable to the shape of ``data`` tensor. **Required.**
|
||||
|
||||
|
||||
**Outputs**
|
||||
|
||||
* **1**: Output tensor of the same shape as the ``data`` input tensor and type specified by *output_type* attribute.
|
||||
|
||||
**Types**
|
||||
|
||||
* *T*: any floating point type.
|
||||
|
||||
**Example**
|
||||
|
||||
.. code-block:: xml
|
||||
:force:
|
||||
|
||||
<layer ... type="RMS"> <!-- normalization always over the last dimension [-1] -->
|
||||
<data eps="1e-6"/>
|
||||
<input>
|
||||
<port id="0">
|
||||
<dim>12</dim>
|
||||
<dim>25</dim>
|
||||
<dim>512</dim>
|
||||
</port>
|
||||
<port id="1">
|
||||
<dim>512</dim>
|
||||
</port>
|
||||
</input>
|
||||
<output>
|
||||
<port id="2">
|
||||
<dim>12</dim>
|
||||
<dim>25</dim>
|
||||
<dim>512</dim>
|
||||
</port>
|
||||
</output>
|
||||
</layer>
|
||||
Loading…
Reference in New Issue