class_docstr: **Transformer** 的 Decoder 层
  
  :param d_model: 输入、输出的维度
  :param n_head: **多头注意力** head 的数目，需要能被 ``d_model`` 整除
  :param dim_ff:  FFN 中间映射的维度
  :param dropout: Dropout 的大小
  :param layer_idx: layer的编号
class_docstr_tok: ["*","*","Transformer","*","*","\u7684","Decoder","\u5c42",":","param","d_model",":","\u8f93\u5165\u3001\u8f93\u51fa\u7684\u7ef4\u5ea6",":","param","n_head",":","*","*","\u591a\u5934\u6ce8\u610f\u529b","*","*","head","\u7684\u6570\u76ee\uff0c\u9700\u8981\u80fd\u88ab","``","d_model","``","\u6574\u9664",":","param","dim_ff",":","FFN","\u4e2d\u95f4\u6620\u5c04\u7684\u7ef4\u5ea6",":","param","dropout",":","Dropout","\u7684\u5927\u5c0f",":","param","layer_idx",":","layer\u7684\u7f16\u53f7"]
docstr: :param x: ``decoder`` 端的输入，形状为 ``[batch_size, seq_len, dim]`` 
  :param encoder_output: ``encoder`` 的输出，形状为 ``[batch_size, src_seq_len, dim]``
  :param encoder_mask: 掩码，形状为 ``[batch_size, src_seq_len]``，为 **1** 的地方表示需要 attend
  :param self_attn_mask: 下三角的mask矩阵，只在训练时传入。形状为 ``[seq_len, seq_len]``
  :param state: 只在 inference 阶段传入，记录了 ``encoder`` 和 ``decoder`` 的状态
  :return:
docstr_tok: [":","param","x",":","``","decoder","``","\u7aef\u7684\u8f93\u5165\uff0c\u5f62\u72b6\u4e3a","``","[","batch_size",",","seq_len",",","dim","]","``",":","param","encoder_output",":","``","encoder","``","\u7684\u8f93\u51fa\uff0c\u5f62\u72b6\u4e3a","``","[","batch_size",",","src_seq_len",",","dim","]","``",":","param","encoder_mask",":","\u63a9\u7801\uff0c\u5f62\u72b6\u4e3a","``","[","batch_size",",","src_seq_len","]","``","\uff0c\u4e3a","*","*","1","*","*","\u7684\u5730\u65b9\u8868\u793a\u9700\u8981","attend",":","param","self_attn_mask",":","\u4e0b\u4e09\u89d2\u7684mask\u77e9\u9635\uff0c\u53ea\u5728\u8bad\u7ec3\u65f6\u4f20\u5165\u3002\u5f62\u72b6\u4e3a","``","[","seq_len",",","seq_len","]","``",":","param","state",":","\u53ea\u5728","inference","\u9636\u6bb5\u4f20\u5165\uff0c\u8bb0\u5f55\u4e86","``","encoder","``","\u548c","``","decoder","``","\u7684\u72b6\u6001",":","return",":"]
code: def forward(
      self,
      x,
      encoder_output,
      encoder_mask=None,
      self_attn_mask=None,
      state: TransformerState = None,
  ):
      """
  
      :param x: ``decoder`` 端的输入，形状为 ``[batch_size, seq_len, dim]``
      :param encoder_output: ``encoder`` 的输出，形状为 ``[batch_size, src_seq_len, dim]``
      :param encoder_mask: 掩码，形状为 ``[batch_size, src_seq_len]``，为 **1** 的地方表示需要 attend
      :param self_attn_mask: 下三角的mask矩阵，只在训练时传入。形状为 ``[seq_len, seq_len]``
      :param state: 只在 inference 阶段传入，记录了 ``encoder`` 和 ``decoder`` 的状态
      :return:
      """
  
      # self attention part
      residual = x
      x = self.self_attn_layer_norm(x)
      x, _ = self.self_attn(
          query=x, key=x, value=x, attn_mask=self_attn_mask, state=state
      )
  
      x = F.dropout(x, p=self.dropout, training=self.training)
      x = residual + x
  
      # encoder attention part
      residual = x
      x = self.encoder_attn_layer_norm(x)
      x, attn_weight = self.encoder_attn(
          query=x,
          key=encoder_output,
          value=encoder_output,
          key_mask=encoder_mask,
          state=state,
      )
      x = F.dropout(x, p=self.dropout, training=self.training)
      x = residual + x
  
      # ffn
      residual = x
      x = self.final_layer_norm(x)
      x = self.ffn(x)
      x = residual + x
  
      return x, attn_weight
  
imports: from typing import Union, Tuple
  import math
  import torch
  from torch import nn
  import torch.nn.functional
  from ..attention import AttentionLayer, MultiHeadAttention
  from ....embeddings.torch.utils import get_embeddings
  from ....embeddings.torch.static_embedding import StaticEmbedding
  from .seq2seq_state import State, LSTMState, TransformerState
label_desc: Use image node_id 2 for calling the TransformerSeq2SeqDecoderLayer obj's underlying member method code with example usage: obj.forward(x, encoder_output, encoder_mask, _attn_mask, state) and returns: x,  attn_weight