class_docstr: Rewrite fairseq.modules.TransformerDecoderLayer to avoid forwarding
  model multiple times and be compatible with checkpoint_activations.
  
  The input is expected to be a list of tensors from different channels,
  each is forwarded to the same model (shared attention weights).
  
  In the original paper each operation (multi-head attention, encoder
  attention or FFN) is postprocessed with: `dropout -> add residual ->
  layernorm`. In the tensor2tensor code they suggest that learning is more
  robust when preprocessing each layer with layernorm and postprocessing with:
  `dropout -> add residual`. We default to the approach in the paper, but the
  tensor2tensor approach can be enabled by setting
  *args.decoder_normalize_before* to ``True``.
  
  Args:
      args (argparse.Namespace): parsed command-line arguments
      no_encoder_attn (bool, optional): whether to attend to encoder outputs
          (default: False).
class_docstr_tok: ["Rewrite","fairseq.modules.TransformerDecoderLayer","to","avoid","forwarding","model","multiple","times","and","be","compatible","with","checkpoint_activations",".","The","input","is","expected","to","be","a","list","of","tensors","from","different","channels",",","each","is","forwarded","to","the","same","model","(","shared","attention","weights",")",".","In","the","original","paper","each","operation","(","multi-head","attention",",","encoder","attention","or","FFN",")","is","postprocessed","with",":","`","dropout","-",">","add","residual","-",">","layernorm","`",".","In","the","tensor2tensor","code","they","suggest","that","learning","is","more","robust","when","preprocessing","each","layer","with","layernorm","and","postprocessing","with",":","`","dropout","-",">","add","residual","`",".","We","default","to","the","approach","in","the","paper",",","but","the","tensor2tensor","approach","can","be","enabled","by","setting","*","args.decoder_normalize_before","*","to","``","True","``",".","Args",":","args","(","argparse.Namespace",")",":","parsed","command-line","arguments","no_encoder_attn","(","bool",",","optional",")",":","whether","to","attend","to","encoder","outputs","(","default",":","False",")","."]
code: def __init__(
      self,
      args,
      no_encoder_attn=False,
      add_bias_kv=False,
      add_zero_attn=False,
  ):
      super().__init__()
      self.embed_dim = args.decoder_embed_dim
      self.dropout_module = FairseqDropout(
          args.dropout, module_name=self.__class__.__name__
      )
      self.quant_noise = getattr(args, "quant_noise_pq", 0)
      self.quant_noise_block_size = getattr(
          args, "quant_noise_pq_block_size", 8
      )
  
      self.cross_self_attention = getattr(
          args, "cross_self_attention", False
      )
  
      self.self_attn = self.build_self_attention(
          self.embed_dim,
          args,
          add_bias_kv=add_bias_kv,
          add_zero_attn=add_zero_attn,
      )
  
      self.activation_fn = utils.get_activation_fn(
          activation=str(args.activation_fn)
          if getattr(args, "activation_fn", None) is not None
          else "relu"
      )
      activation_dropout_p = getattr(args, "activation_dropout", 0) or 0
      if activation_dropout_p == 0:
          # for backwards compatibility with models that use args.relu_dropout
          activation_dropout_p = getattr(args, "relu_dropout", 0) or 0
      self.activation_dropout_module = FairseqDropout(
          float(activation_dropout_p),
          module_name=self.__class__.__name__,
      )
      self.normalize_before = args.decoder_normalize_before
  
      # use layerNorm rather than FusedLayerNorm for exporting.
      # char_inputs can be used to determint this.
      # TODO  remove this once we update apex with the fix
      export = getattr(args, "char_inputs", False)
      self.self_attn_layer_norm = LayerNorm(
          self.embed_dim, export=export
      )
  
      if no_encoder_attn:
          self.encoder_attn = None
          self.encoder_attn_layer_norm = None
      else:
          self.encoder_attn = self.build_encoder_attention(
              self.embed_dim, args
          )
          self.encoder_attn_layer_norm = LayerNorm(
              self.embed_dim, export=export
          )
  
      self.fc1 = self.build_fc1(
          self.embed_dim,
          args.decoder_ffn_embed_dim,
          self.quant_noise,
          self.quant_noise_block_size,
      )
      self.fc2 = self.build_fc2(
          args.decoder_ffn_embed_dim,
          self.embed_dim,
          self.quant_noise,
          self.quant_noise_block_size,
      )
  
      self.final_layer_norm = LayerNorm(self.embed_dim, export=export)
      self.need_attn = True
  
      self.onnx_trace = False
  
imports: from typing import Dict, List, Tuple, Optional
  import torch
  import torch.nn
  from fairseq import utils
  from fairseq.modules import LayerNorm, MultiheadAttention
  from fairseq.modules.fairseq_dropout import FairseqDropout
  from fairseq.modules.quant_noise import quant_noise
  from torch import Tensor
label_desc: Use image node_id 1 to create a new StandardTransformerDecoderLayer object from inherited base classes: nn with example: obj = StandardTransformerDecoderLayer(args, no_encoder_attn, add_bias_kv, add_zero_attn)