chamgj
/
FT_DSP.gitlink.net
forked from FT_DSP/FT_DSP.gitlink.net
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
FT_DSP.gitlink.net/master/html/functionlib/dsplib/layernormgrad.html

223 lines
30 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html class="writer-html5" lang="zh-CN" data-content_root="../../">
<head>
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Layernormgrad &mdash; MindSpore Signal+ 使用手册 alpha 文档</title>
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=03e43079" />
<link rel="stylesheet" type="text/css" href="../../_static/css/theme.css?v=e59714d7" />
<link rel="stylesheet" type="text/css" href="../../_static/table.css?v=72b86d3e" />
<script src="../../_static/jquery.js?v=5d32c60e"></script>
<script src="../../_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
<script src="../../_static/documentation_options.js?v=406e4f49"></script>
<script src="../../_static/doctools.js?v=9bcbadda"></script>
<script src="../../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../../_static/translations.js?v=beaddf03"></script>
<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<script src="../../_static/table.js?v=99ed15ba"></script>
<script src="../../_static/js/theme.js"></script>
<link rel="index" title="索引" href="../../genindex.html" />
<link rel="search" title="搜索" href="../../search.html" />
<link rel="next" title="Less" href="less.html" />
<link rel="prev" title="L2norm" href="l2norm.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../../index.html" class="icon icon-home">
MindSpore Signal+ 使用手册
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
<input type="text" name="q" placeholder="搜索文档" aria-label="搜索文档" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="导航菜单">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../quickstart/index.html">快速入门</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../appdevelop/index.html">应用开发示例</a></li>
<li class="toctree-l1"><a class="reference internal" href="../index.html">算子库支持</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../refdoc/index.html">参考资料</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="移动版导航菜单" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../../index.html">MindSpore Signal+ 使用手册</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="页面导航">
<ul class="wy-breadcrumbs">
<li><a href="../../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="../index.html">算子库支持</a></li>
<li class="breadcrumb-item"><a href="dsplib_index.html">DSP Library C API Reference</a></li>
<li class="breadcrumb-item active">Layernormgrad</li>
<li class="wy-breadcrumbs-aside">
<a href="../../_sources/functionlib/dsplib/layernormgrad.rst.txt" rel="nofollow"> 查看页面源码</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="layernormgrad">
<h1>Layernormgrad<a class="headerlink" href="#layernormgrad" title="Link to this heading"></a></h1>
<p>计算 Layer Normalization 操作的梯度。该算子是 Layer Normalization 的反向传播部分，用于计算损失函数相对于输入 x、以及可学习参数 gamma 和 beta 的梯度。</p>
<div class="math notranslate nohighlight">
\[ \begin{align}\begin{aligned}\text{dg}_i = \sum_{j} \text{dy}_j \cdot \frac{x_j - \mu}{\sqrt{\sigma^2 + \epsilon}}\\\text{db}_i = \sum_{j} \text{dy}_j\\\text{dx}_i = f(\text{dy}, x, \gamma, \mu, \sigma^2)\end{aligned}\end{align} \]</div>
<p>其中 :math:mu 是均值，:math:sigma^2 是方差，:math:epsilon 是一个为了防止除零而添加的极小值。dx 的计算较为复杂，它依赖于 dy、x 和 gamma。</p>
<dl class="simple">
<dt>输入：</dt><dd><ul class="simple">
<li><p><strong>x</strong> - 前向传播时的输入数据地址。</p></li>
<li><p><strong>dy</strong> - 后续层反向传播回来的梯度数据地址。</p></li>
<li><p><strong>var</strong> - 前向传播时计算出的方差（variance）地址。</p></li>
<li><p><strong>mean</strong> - 前向传播时计算出的均值（mean）地址。</p></li>
<li><p><strong>gamma</strong> - 前向传播时使用的可学习缩放参数 :math:gamma 地址。</p></li>
<li><p><strong>param_num</strong> - 特征维度的大小，也是 gamma 和 beta 的大小。</p></li>
<li><p><strong>param_size</strong> - 进行独立归一化的单元数量（例如批处理大小 Batch Size）。</p></li>
<li><p><strong>block_num</strong> - 块的数量（通常等于 param_size）。</p></li>
<li><p><strong>block_size</strong> - 每个块的大小（通常等于 param_num）。</p></li>
<li><p><strong>core_mask</strong> - 核掩码（仅共享存储版本需要）。</p></li>
</ul>
</dd>
<dt>输出：</dt><dd><ul class="simple">
<li><p><strong>dx</strong> - 计算出的关于输入 x 的梯度地址。</p></li>
<li><p><strong>dg</strong> - 计算出的关于参数 gamma 的梯度地址。</p></li>
<li><p><strong>db</strong> - 计算出的关于参数 beta 的梯度地址。</p></li>
</ul>
</dd>
<dt>支持平台：</dt><dd><p><code class="docutils literal notranslate"><span class="pre">FT78NE</span></code>
<code class="docutils literal notranslate"><span class="pre">MT7004</span></code></p>
</dd>
</dl>
<div class="admonition note">
<p class="admonition-title">备注</p>
<ul class="simple">
<li><p>FT78NE 支持fp32</p></li>
<li><p>MT7004 支持fp16, fp32</p></li>
</ul>
</div>
<p><strong>共享存储版本:</strong></p>
<dl class="c function">
<dt class="sig sig-object c" id="c.hp_layernormgrad_s">
<span class="kt"><span class="pre">void</span></span><span class="w"> </span><span class="sig-name descname"><span class="n"><span class="pre">hp_layernormgrad_s</span></span></span><span class="sig-paren">(</span><span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">x</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dy</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">var</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">mean</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">gamma</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_size</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_size</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dx</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dg</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">db</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">core_mask</span></span><span class="sig-paren">)</span><a class="headerlink" href="#c.hp_layernormgrad_s" title="Link to this definition"></a><br /></dt>
<dd></dd></dl>
<dl class="c function">
<dt class="sig sig-object c" id="c.fp_layernormgrad_s">
<span class="kt"><span class="pre">void</span></span><span class="w"> </span><span class="sig-name descname"><span class="n"><span class="pre">fp_layernormgrad_s</span></span></span><span class="sig-paren">(</span><span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">x</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dy</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">var</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">mean</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">gamma</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_size</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_size</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dx</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dg</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">db</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">core_mask</span></span><span class="sig-paren">)</span><a class="headerlink" href="#c.fp_layernormgrad_s" title="Link to this definition"></a><br /></dt>
<dd></dd></dl>
<p><strong>C调用示例：</strong></p>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="linenos"> 1</span><span class="c1">//FT78NE示例</span>
<span class="linenos"> 2</span><span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;stdio.h&gt;</span>
<span class="linenos"> 3</span><span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;layernormgrad.h&gt;</span><span class="c1"> // 假设头文件名为 layernormgrad.h</span>
<span class="linenos"> 4</span>
<span class="linenos"> 5</span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="kt">char</span><span class="o">*</span><span class="w"> </span><span class="n">argv</span><span class="p">[])</span><span class="w"> </span><span class="p">{</span>
<span class="linenos"> 6</span><span class="w"> </span><span class="c1">// 假设在DDR空间</span>
<span class="linenos"> 7</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xA0000000</span><span class="p">;</span>
<span class="linenos"> 8</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">dy</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xA1000000</span><span class="p">;</span>
<span class="linenos"> 9</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">var</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xA2000000</span><span class="p">;</span>
<span class="linenos">10</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">mean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xA3000000</span><span class="p">;</span>
<span class="linenos">11</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">gamma</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xA4000000</span><span class="p">;</span>
<span class="linenos">12</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">dx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xB0000000</span><span class="p">;</span>
<span class="linenos">13</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">dg</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xB1000000</span><span class="p">;</span>
<span class="linenos">14</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">db</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0xB2000000</span><span class="p">;</span>
<span class="linenos">15</span>
<span class="linenos">16</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">param_num</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">256</span><span class="p">;</span><span class="w"> </span><span class="c1">// 特征维度</span>
<span class="linenos">17</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">param_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">64</span><span class="p">;</span><span class="w"> </span><span class="c1">// Batch Size</span>
<span class="linenos">18</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">core_mask</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mh">0xff</span><span class="p">;</span>
<span class="linenos">19</span>
<span class="hll"><span class="linenos">20</span><span class="w"> </span><span class="n">fp_layernormgrad_s</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="w"> </span><span class="n">dy</span><span class="p">,</span><span class="w"> </span><span class="n">var</span><span class="p">,</span><span class="w"> </span><span class="n">mean</span><span class="p">,</span><span class="w"> </span><span class="n">gamma</span><span class="p">,</span><span class="w"> </span><span class="n">param_num</span><span class="p">,</span><span class="w"> </span><span class="n">param_size</span><span class="p">,</span><span class="w"> </span><span class="n">param_size</span><span class="p">,</span><span class="w"> </span><span class="n">param_num</span><span class="p">,</span><span class="w"> </span><span class="n">dx</span><span class="p">,</span><span class="w"> </span><span class="n">dg</span><span class="p">,</span><span class="w"> </span><span class="n">db</span><span class="p">,</span><span class="w"> </span><span class="n">core_mask</span><span class="p">);</span>
</span><span class="linenos">21</span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="linenos">22</span><span class="p">}</span>
</pre></div>
</div>
<p><strong>私有存储版本:</strong></p>
<dl class="c function">
<dt class="sig sig-object c" id="c.hp_layernormgrad_p">
<span class="kt"><span class="pre">void</span></span><span class="w"> </span><span class="sig-name descname"><span class="n"><span class="pre">hp_layernormgrad_p</span></span></span><span class="sig-paren">(</span><span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">x</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dy</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">var</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">mean</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">gamma</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_size</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_size</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dx</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dg</span></span>, <span class="n"><span class="pre">half</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">db</span></span><span class="sig-paren">)</span><a class="headerlink" href="#c.hp_layernormgrad_p" title="Link to this definition"></a><br /></dt>
<dd></dd></dl>
<dl class="c function">
<dt class="sig sig-object c" id="c.fp_layernormgrad_p">
<span class="kt"><span class="pre">void</span></span><span class="w"> </span><span class="sig-name descname"><span class="n"><span class="pre">fp_layernormgrad_p</span></span></span><span class="sig-paren">(</span><span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">x</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dy</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">var</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">mean</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">gamma</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">param_size</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_num</span></span>, <span class="kt"><span class="pre">int</span></span><span class="w"> </span><span class="n"><span class="pre">block_size</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dx</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">dg</span></span>, <span class="kt"><span class="pre">float</span></span><span class="w"> </span><span class="p"><span class="pre">*</span></span><span class="n"><span class="pre">db</span></span><span class="sig-paren">)</span><a class="headerlink" href="#c.fp_layernormgrad_p" title="Link to this definition"></a><br /></dt>
<dd></dd></dl>
<p><strong>C调用示例：</strong></p>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="linenos"> 1</span><span class="c1">//FT78NE示例</span>
<span class="linenos"> 2</span><span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;stdio.h&gt;</span>
<span class="linenos"> 3</span><span class="cp">#include</span><span class="w"> </span><span class="cpf">&lt;layernormgrad.h&gt;</span><span class="c1"> // 假设头文件名为 layernormgrad.h</span>
<span class="linenos"> 4</span>
<span class="linenos"> 5</span><span class="kt">int</span><span class="w"> </span><span class="nf">main</span><span class="p">(</span><span class="kt">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="kt">char</span><span class="o">*</span><span class="w"> </span><span class="n">argv</span><span class="p">[])</span><span class="w"> </span><span class="p">{</span>
<span class="linenos"> 6</span><span class="w"> </span><span class="c1">// 假设在L2空间</span>
<span class="linenos"> 7</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x10000000</span><span class="p">;</span>
<span class="linenos"> 8</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">dy</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x11000000</span><span class="p">;</span>
<span class="linenos"> 9</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">var</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x12000000</span><span class="p">;</span>
<span class="linenos">10</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">mean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x13000000</span><span class="p">;</span>
<span class="linenos">11</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">gamma</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x14000000</span><span class="p">;</span>
<span class="linenos">12</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">dx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x15000000</span><span class="p">;</span>
<span class="linenos">13</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">dg</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x16000000</span><span class="p">;</span>
<span class="linenos">14</span><span class="w"> </span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="n">db</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">float</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="mh">0x17000000</span><span class="p">;</span>
<span class="linenos">15</span>
<span class="linenos">16</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">param_num</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">256</span><span class="p">;</span><span class="w"> </span><span class="c1">// 特征维度</span>
<span class="linenos">17</span><span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">param_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">64</span><span class="p">;</span><span class="w"> </span><span class="c1">// Batch Size</span>
<span class="linenos">18</span>
<span class="hll"><span class="linenos">19</span><span class="w"> </span><span class="n">fp_layernormgrad_p</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="w"> </span><span class="n">dy</span><span class="p">,</span><span class="w"> </span><span class="n">var</span><span class="p">,</span><span class="w"> </span><span class="n">mean</span><span class="p">,</span><span class="w"> </span><span class="n">gamma</span><span class="p">,</span><span class="w"> </span><span class="n">param_num</span><span class="p">,</span><span class="w"> </span><span class="n">param_size</span><span class="p">,</span><span class="w"> </span><span class="n">param_size</span><span class="p">,</span><span class="w"> </span><span class="n">param_num</span><span class="p">,</span><span class="w"> </span><span class="n">dx</span><span class="p">,</span><span class="w"> </span><span class="n">dg</span><span class="p">,</span><span class="w"> </span><span class="n">db</span><span class="p">);</span>
</span><span class="linenos">20</span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="linenos">21</span><span class="p">}</span>
</pre></div>
</div>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="页脚">
<a href="l2norm.html" class="btn btn-neutral float-left" title="L2norm" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> 上一页</a>
<a href="less.html" class="btn btn-neutral float-right" title="Less" accesskey="n" rel="next">下一页 <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>&#169; 版权所有 2025 - 2026, NUDT-674。</p>
</div>
利用 <a href="https://www.sphinx-doc.org/">Sphinx</a> 构建，使用的
<a href="https://github.com/readthedocs/sphinx_rtd_theme">主题</a>
由 <a href="https://readthedocs.org">Read the Docs</a> 开发.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink