class_docstr: Transformer decoder that applies multi-head self-attention for
  sequence decoding.
  
  It is a stack of :class:`~texar.tf.modules.encoders.MultiheadAttentionEncoder`,
  :class:`~texar.tf.modules.FeedForwardNetwork` and residual connections.
  
  Args:
      vocab_size (int, optional): Vocabulary size. Required if
          :attr:`output_layer` is `None`.
      output_layer (optional): An output layer that transforms cell output
          to logits. This can be:
  
          - A callable layer, e.g., an instance             of :tf_main:`tf.layers.Layer <layers/Layer>`.
          - A tensor. A dense layer will be created using the tensor             as the kernel weights. The bias of the dense layer is determined by            `hparams.output_layer_bias`. This can be used to tie the output             layer with the input embedding matrix, as proposed in             https://arxiv.org/pdf/1608.05859.pdf
          - `None`. A dense layer will be created based on attr:`vocab_size`            and `hparams.output_layer_bias`.
          - If no output layer in the end is needed, set             `(vocab_size=None, output_layer=tf.identity)`.
  
  .. document private functions
  .. automethod:: _build
class_docstr_tok: ["Transformer","decoder","that","applies","multi-head","self-attention","for","sequence","decoding",".","It","is","a","stack","of",":","class",":","`","~texar.tf.modules.encoders.MultiheadAttentionEncoder","`",",",":","class",":","`","~texar.tf.modules.FeedForwardNetwork","`","and","residual","connections",".","Args",":","vocab_size","(","int",",","optional",")",":","Vocabulary","size",".","Required","if",":","attr",":","`","output_layer","`","is","`","None","`",".","output_layer","(","optional",")",":","An","output","layer","that","transforms","cell","output","to","logits",".","This","can","be",":","-","A","callable","layer",",","e.g.",",","an","instance","of",":","tf_main",":","`","tf.layers.Layer","<","layers\/Layer",">","`",".","-","A","tensor",".","A","dense","layer","will","be","created","using","the","tensor","as","the","kernel","weights",".","The","bias","of","the","dense","layer","is","determined","by","`","hparams.output_layer_bias","`",".","This","can","be","used","to","tie","the","output","layer","with","the","input","embedding","matrix",",","as","proposed","in","https",":","\/\/arxiv.org\/pdf\/1608.05859.pdf","-","`","None","`",".","A","dense","layer","will","be","created","based","on","attr",":","`","vocab_size","`","and","`","hparams.output_layer_bias","`",".","-","If","no","output","layer","in","the","end","is","needed",",","set","`","(","vocab_size=None",",","output_layer=tf.identity",")","`",".","..","document","private","functions","..","automethod",":",":","_build"]
docstr: Performs decoding.
  
  The interface is mostly the same with that of RNN decoders
  (see :meth:`~texar.tf.modules.RNNDecoderBase._build`). The main
  difference is that, here, `sequence_length` is not needed, and
  continuation generation is additionally supported.
  
  The function provides **3 ways** to specify the decoding method, with
  varying flexibility:
  
  1. The :attr:`decoding_strategy` argument.
  
      - **"train_greedy"**: decoding in teacher-forcing fashion (i.e.,
        feeding ground truth to decode the next step), and for each step
        sample is obtained by taking the `argmax` of logits.
        Argument :attr:`inputs` is required for this strategy.
      - **"infer_greedy"**: decoding in inference fashion (i.e., feeding
        `generated` sample to decode the next step), and for each step
        sample is obtained by taking the `argmax` of logits.
        Arguments :attr:`(start_tokens, end_token, embedding)` are
        required for this strategy, and argument
        :attr:`max_decoding_length` is optional.
      - **"infer_sample"**: decoding in inference fashion, and for each
        step sample is obtained by `random sampling` from the logits.
        Arguments :attr:`(start_tokens, end_token, embedding)` are
        required for this strategy, and argument
        :attr:`max_decoding_length` is optional.
  
    This argument is used only when arguments :attr:`helper` and
    :attr:`beam_width` are both `None`.
  
  2. The :attr:`helper` argument: An instance of subclass of
     :class:`texar.tf.modules.Helper`.
     This provides a superset of decoding strategies than above.
     The interface is the same as in RNN decoders.
     Please refer to :meth:`texar.tf.modules.RNNDecoderBase._build` for
     detailed usage and examples.
  
     Note that, here, though using a
     :class:`~texar.tf.modules.TrainingHelper` corresponds to the
     "train_greedy" strategy above and will get the same output results,
     the implementation is *slower* than
     directly setting `decoding_strategy = "train_greedy"`.
  
     Argument :attr:`max_decoding_length` is optional.
  
  3. **Beam search**: set :attr:`beam_width` to use beam search decoding.
     Arguments :attr:`(start_tokens, end_token)` are required,
     and argument :attr:`max_decoding_length` is optional.
  
  Args:
      memory (optional): The memory to attend, e.g., the output of an RNN
          encoder. A Tensor of shape `[batch_size, memory_max_time, dim]`.
      memory_sequence_length (optional): A Tensor of shape `[batch_size]`
          containing the sequence lengths for the batch entries in
          memory. Used to create attention bias of
          :attr:`memory_attention_bias` is not given. Ignored if
          `memory_attention_bias` is provided.
      memory_attention_bias (optional): A Tensor of shape
          `[batch_size, num_heads, memory_max_time, dim]`.
          An attention bias typically sets the value of a padding
          position to a large negative value for masking. If not given,
          :attr:`memory_sequence_length` is used to automatically
          create an attention bias.
      inputs (optional): Input tensor for teacher forcing decoding, of
          shape `[batch_size, target_max_time, emb_dim]` containing the
          target sequence word embeddings.
          Used when :attr:`decoding_strategy` is set to "train_greedy".
      decoding_strategy (str): A string specifying the decoding
          strategy, including "train_greedy", "infer_greedy",
          "infer_sample".
          Different arguments are required based on the
          strategy. See above for details. Ignored if
          :attr:`beam_width` or :attr:`helper` is set.
      beam_width (int): Set to use beam search. If given,
          :attr:`decoding_strategy` is ignored.
      length_penalty (float): Length penalty coefficient used in beam
          search decoding. Refer to https://arxiv.org/abs/1609.08144
          for more details.
          It Should be larger if longer sentences are wanted.
      start_tokens (optional): An int Tensor of shape `[batch_size]`,
          containing the start tokens.
          Used when :attr:`decoding_strategy` = "infer_greedy" or
          "infer_sample", or :attr:`beam_width` is set.
          Ignored when context is set.
      end_token (optional): An int 0D Tensor, the token that marks end
          of decoding.
          Used when :attr:`decoding_strategy` = "infer_greedy" or
          "infer_sample", or :attr:`beam_width` is set.
      context (optional): An int Tensor of shape `[batch_size, length]`,
          containing the starting tokens for decoding.
          If context is set, the start_tokens will be ignored.
      context_sequence_length (optional): specify the length of context.
      softmax_temperature (optional): A float 0D Tensor, value to divide
          the logits by before computing the softmax. Larger values
          (above 1.0) result in more random samples. Must > 0. If `None`,
          1.0 is used.
          Used when `decoding_strategy = "infer_sample"`.
      max_decoding_length (optional): An int scalar Tensor indicating
          the maximum allowed number of decoding steps.
          If `None` (default), use `"max_decoding_length"` defined in
          :attr:`hparams`. Ignored in "train_greedy" decoding.
      impute_finished (bool): If `True`, then states for batch
          entries which are marked as finished get copied through and
          the corresponding outputs get zeroed out.  This causes some
          slowdown at each time step, but ensures that the final state
          and outputs have the correct values and that backprop ignores
          time steps that were marked as finished. Ignored in
          "train_greedy" decoding.
      embedding (optional): Embedding used when
          "infer_greedy" or "infer_sample" `decoding_strategy`, or
          beam search, is used. This can be
          a callable or the `params` argument for
          :tf_main:`embedding_lookup <nn/embedding_lookup>`.
          If a callable, it can take a vector tensor of token `ids`,
          or take two arguments (`ids`, `times`), where `ids`
          is a vector tensor of token ids, and `times` is a vector tensor
          of time steps (i.e., position ids). The latter case can be used
          when attr:`embedding` is a combination of word embedding and
          position embedding.
      helper (optional): An instance of
          :tf_main:`Helper <contrib/seq2seq/Helper>` that defines the
          decoding strategy. If given, :attr:`decoding_strategy` is
          ignored.
      mode (optional): A tensor taking value in
          :tf_main:`tf.estimator.ModeKeys <estimator/ModeKeys>`, including
          `TRAIN`, `EVAL`, and `PREDICT`. Controls dropout mode.
          If `None` (default), :func:`texar.tf.global_mode`
          is used.
  
  Returns:
  
      - For **"train_greedy"** decoding, returns an instance of
        :class:`~texar.tf.modules.TransformerDecoderOutput` which contains
        `sample_id` and `logits`.
  
      - For **"infer_greedy"** and **"infer_sample"** decoding or
        decoding with :attr:`helper`, returns
        a tuple `(outputs, sequence_lengths)`, where `outputs` is an
        instance of :class:`~texar.tf.modules.TransformerDecoderOutput` as
        in "train_greedy", and `sequence_lengths` is a Tensor of shape
        `[batch_size]` containing the length of each sample.
  
      - For **beam search** decoding, returns a `dict` containing keys
        "sample_id" and "log_prob".
  
          - **"sample_id"** is an int Tensor of shape
            `[batch_size, max_time, beam_width]` containing generated
            token indexes. `sample_id[:,:,0]` is the highest-probable
            sample.
          - **"log_prob"** is a float Tensor of shape
            `[batch_size, beam_width]` containing the log probability
            of each sequence sample.
docstr_tok: ["Performs","decoding",".","The","interface","is","mostly","the","same","with","that","of","RNN","decoders","(","see",":","meth",":","`","~texar.tf.modules.RNNDecoderBase._build","`",")",".","The","main","difference","is","that",",","here",",","`","sequence_length","`","is","not","needed",",","and","continuation","generation","is","additionally","supported",".","The","function","provides","*","*","3","ways","*","*","to","specify","the","decoding","method",",","with","varying","flexibility",":","1",".","The",":","attr",":","`","decoding_strategy","`","argument",".","-","*","*","''","train_greedy","''","*","*",":","decoding","in","teacher-forcing","fashion","(","i.e.",",","feeding","ground","truth","to","decode","the","next","step",")",",","and","for","each","step","sample","is","obtained","by","taking","the","`","argmax","`","of","logits",".","Argument",":","attr",":","`","inputs","`","is","required","for","this","strategy",".","-","*","*","''","infer_greedy","''","*","*",":","decoding","in","inference","fashion","(","i.e.",",","feeding","`","generated","`","sample","to","decode","the","next","step",")",",","and","for","each","step","sample","is","obtained","by","taking","the","`","argmax","`","of","logits",".","Arguments",":","attr",":","`","(","start_tokens",",","end_token",",","embedding",")","`","are","required","for","this","strategy",",","and","argument",":","attr",":","`","max_decoding_length","`","is","optional",".","-","*","*","''","infer_sample","''","*","*",":","decoding","in","inference","fashion",",","and","for","each","step","sample","is","obtained","by","`","random","sampling","`","from","the","logits",".","Arguments",":","attr",":","`","(","start_tokens",",","end_token",",","embedding",")","`","are","required","for","this","strategy",",","and","argument",":","attr",":","`","max_decoding_length","`","is","optional",".","This","argument","is","used","only","when","arguments",":","attr",":","`","helper","`","and",":","attr",":","`","beam_width","`","are","both","`","None","`",".","2",".","The",":","attr",":","`","helper","`","argument",":","An","instance","of","subclass","of",":","class",":","`","texar.tf.modules.Helper","`",".","This","provides","a","superset","of","decoding","strategies","than","above",".","The","interface","is","the","same","as","in","RNN","decoders",".","Please","refer","to",":","meth",":","`","texar.tf.modules.RNNDecoderBase._build","`","for","detailed","usage","and","examples",".","Note","that",",","here",",","though","using","a",":","class",":","`","~texar.tf.modules.TrainingHelper","`","corresponds","to","the","``","train_greedy","''","strategy","above","and","will","get","the","same","output","results",",","the","implementation","is","*","slower","*","than","directly","setting","`","decoding_strategy","=","``","train_greedy","''","`",".","Argument",":","attr",":","`","max_decoding_length","`","is","optional",".","3",".","*","*","Beam","search","*","*",":","set",":","attr",":","`","beam_width","`","to","use","beam","search","decoding",".","Arguments",":","attr",":","`","(","start_tokens",",","end_token",")","`","are","required",",","and","argument",":","attr",":","`","max_decoding_length","`","is","optional",".","Args",":","memory","(","optional",")",":","The","memory","to","attend",",","e.g.",",","the","output","of","an","RNN","encoder",".","A","Tensor","of","shape","`","[","batch_size",",","memory_max_time",",","dim","]","`",".","memory_sequence_length","(","optional",")",":","A","Tensor","of","shape","`","[","batch_size","]","`","containing","the","sequence","lengths","for","the","batch","entries","in","memory",".","Used","to","create","attention","bias","of",":","attr",":","`","memory_attention_bias","`","is","not","given",".","Ignored","if","`","memory_attention_bias","`","is","provided",".","memory_attention_bias","(","optional",")",":","A","Tensor","of","shape","`","[","batch_size",",","num_heads",",","memory_max_time",",","dim","]","`",".","An","attention","bias","typically","sets","the","value","of","a","padding","position","to","a","large","negative","value","for","masking",".","If","not","given",",",":","attr",":","`","memory_sequence_length","`","is","used","to","automatically","create","an","attention","bias",".","inputs","(","optional",")",":","Input","tensor","for","teacher","forcing","decoding",",","of","shape","`","[","batch_size",",","target_max_time",",","emb_dim","]","`","containing","the","target","sequence","word","embeddings",".","Used","when",":","attr",":","`","decoding_strategy","`","is","set","to","``","train_greedy","''",".","decoding_strategy","(","str",")",":","A","string","specifying","the","decoding","strategy",",","including","``","train_greedy","''",",","``","infer_greedy","''",",","``","infer_sample","''",".","Different","arguments","are","required","based","on","the","strategy",".","See","above","for","details",".","Ignored","if",":","attr",":","`","beam_width","`","or",":","attr",":","`","helper","`","is","set",".","beam_width","(","int",")",":","Set","to","use","beam","search",".","If","given",",",":","attr",":","`","decoding_strategy","`","is","ignored",".","length_penalty","(","float",")",":","Length","penalty","coefficient","used","in","beam","search","decoding",".","Refer","to","https",":","\/\/arxiv.org\/abs\/1609.08144","for","more","details",".","It","Should","be","larger","if","longer","sentences","are","wanted",".","start_tokens","(","optional",")",":","An","int","Tensor","of","shape","`","[","batch_size","]","`",",","containing","the","start","tokens",".","Used","when",":","attr",":","`","decoding_strategy","`","=","``","infer_greedy","''","or","``","infer_sample","''",",","or",":","attr",":","`","beam_width","`","is","set",".","Ignored","when","context","is","set",".","end_token","(","optional",")",":","An","int","0D","Tensor",",","the","token","that","marks","end","of","decoding",".","Used","when",":","attr",":","`","decoding_strategy","`","=","``","infer_greedy","''","or","``","infer_sample","''",",","or",":","attr",":","`","beam_width","`","is","set",".","context","(","optional",")",":","An","int","Tensor","of","shape","`","[","batch_size",",","length","]","`",",","containing","the","starting","tokens","for","decoding",".","If","context","is","set",",","the","start_tokens","will","be","ignored",".","context_sequence_length","(","optional",")",":","specify","the","length","of","context",".","softmax_temperature","(","optional",")",":","A","float","0D","Tensor",",","value","to","divide","the","logits","by","before","computing","the","softmax",".","Larger","values","(","above","1.0",")","result","in","more","random","samples",".","Must",">","0",".","If","`","None","`",",","1.0","is","used",".","Used","when","`","decoding_strategy","=","``","infer_sample","''","`",".","max_decoding_length","(","optional",")",":","An","int","scalar","Tensor","indicating","the","maximum","allowed","number","of","decoding","steps",".","If","`","None","`","(","default",")",",","use","`","``","max_decoding_length","''","`","defined","in",":","attr",":","`","hparams","`",".","Ignored","in","``","train_greedy","''","decoding",".","impute_finished","(","bool",")",":","If","`","True","`",",","then","states","for","batch","entries","which","are","marked","as","finished","get","copied","through","and","the","corresponding","outputs","get","zeroed","out",".","This","causes","some","slowdown","at","each","time","step",",","but","ensures","that","the","final","state","and","outputs","have","the","correct","values","and","that","backprop","ignores","time","steps","that","were","marked","as","finished",".","Ignored","in","``","train_greedy","''","decoding",".","embedding","(","optional",")",":","Embedding","used","when","``","infer_greedy","''","or","``","infer_sample","''","`","decoding_strategy","`",",","or","beam","search",",","is","used",".","This","can","be","a","callable","or","the","`","params","`","argument","for",":","tf_main",":","`","embedding_lookup","<","nn\/embedding_lookup",">","`",".","If","a","callable",",","it","can","take","a","vector","tensor","of","token","`","ids","`",",","or","take","two","arguments","(","`","ids","`",",","`","times","`",")",",","where","`","ids","`","is","a","vector","tensor","of","token","ids",",","and","`","times","`","is","a","vector","tensor","of","time","steps","(","i.e.",",","position","ids",")",".","The","latter","case","can","be","used","when","attr",":","`","embedding","`","is","a","combination","of","word","embedding","and","position","embedding",".","helper","(","optional",")",":","An","instance","of",":","tf_main",":","`","Helper","<","contrib\/seq2seq\/Helper",">","`","that","defines","the","decoding","strategy",".","If","given",",",":","attr",":","`","decoding_strategy","`","is","ignored",".","mode","(","optional",")",":","A","tensor","taking","value","in",":","tf_main",":","`","tf.estimator.ModeKeys","<","estimator\/ModeKeys",">","`",",","including","`","TRAIN","`",",","`","EVAL","`",",","and","`","PREDICT","`",".","Controls","dropout","mode",".","If","`","None","`","(","default",")",",",":","func",":","`","texar.tf.global_mode","`","is","used",".","Returns",":","-","For","*","*","''","train_greedy","''","*","*","decoding",",","returns","an","instance","of",":","class",":","`","~texar.tf.modules.TransformerDecoderOutput","`","which","contains","`","sample_id","`","and","`","logits","`",".","-","For","*","*","''","infer_greedy","''","*","*","and","*","*","''","infer_sample","''","*","*","decoding","or","decoding","with",":","attr",":","`","helper","`",",","returns","a","tuple","`","(","outputs",",","sequence_lengths",")","`",",","where","`","outputs","`","is","an","instance","of",":","class",":","`","~texar.tf.modules.TransformerDecoderOutput","`","as","in","``","train_greedy","''",",","and","`","sequence_lengths","`","is","a","Tensor","of","shape","`","[","batch_size","]","`","containing","the","length","of","each","sample",".","-","For","*","*","beam","search","*","*","decoding",",","returns","a","`","dict","`","containing","keys","``","sample_id","''","and","``","log_prob","''",".","-","*","*","''","sample_id","''","*","*","is","an","int","Tensor","of","shape","`","[","batch_size",",","max_time",",","beam_width","]","`","containing","generated","token","indexes",".","`","sample_id","[",":",",",":",",0","]","`","is","the","highest-probable","sample",".","-","*","*","''","log_prob","''","*","*","is","a","float","Tensor","of","shape","`","[","batch_size",",","beam_width","]","`","containing","the","log","probability","of","each","sequence","sample","."]
code: def _build(
      self,  # pylint: disable=arguments-differ, too-many-statements
      decoding_strategy="train_greedy",
      inputs=None,
      memory=None,
      memory_sequence_length=None,
      memory_attention_bias=None,
      beam_width=None,
      length_penalty=0.0,
      start_tokens=None,
      end_token=None,
      context=None,
      context_sequence_length=None,
      softmax_temperature=None,
      max_decoding_length=None,
      impute_finished=False,
      embedding=None,
      helper=None,
      mode=None,
  ):
      """Performs decoding.
  
      The interface is mostly the same with that of RNN decoders
      (see :meth:`~texar.tf.modules.RNNDecoderBase._build`). The main
      difference is that, here, `sequence_length` is not needed, and
      continuation generation is additionally supported.
  
      The function provides **3 ways** to specify the decoding method, with
      varying flexibility:
  
      1. The :attr:`decoding_strategy` argument.
  
          - **"train_greedy"**: decoding in teacher-forcing fashion (i.e.,
            feeding ground truth to decode the next step), and for each step
            sample is obtained by taking the `argmax` of logits.
            Argument :attr:`inputs` is required for this strategy.
          - **"infer_greedy"**: decoding in inference fashion (i.e., feeding
            `generated` sample to decode the next step), and for each step
            sample is obtained by taking the `argmax` of logits.
            Arguments :attr:`(start_tokens, end_token, embedding)` are
            required for this strategy, and argument
            :attr:`max_decoding_length` is optional.
          - **"infer_sample"**: decoding in inference fashion, and for each
            step sample is obtained by `random sampling` from the logits.
            Arguments :attr:`(start_tokens, end_token, embedding)` are
            required for this strategy, and argument
            :attr:`max_decoding_length` is optional.
  
        This argument is used only when arguments :attr:`helper` and
        :attr:`beam_width` are both `None`.
  
      2. The :attr:`helper` argument: An instance of subclass of
         :class:`texar.tf.modules.Helper`.
         This provides a superset of decoding strategies than above.
         The interface is the same as in RNN decoders.
         Please refer to :meth:`texar.tf.modules.RNNDecoderBase._build` for
         detailed usage and examples.
  
         Note that, here, though using a
         :class:`~texar.tf.modules.TrainingHelper` corresponds to the
         "train_greedy" strategy above and will get the same output results,
         the implementation is *slower* than
         directly setting `decoding_strategy = "train_greedy"`.
  
         Argument :attr:`max_decoding_length` is optional.
  
      3. **Beam search**: set :attr:`beam_width` to use beam search decoding.
         Arguments :attr:`(start_tokens, end_token)` are required,
         and argument :attr:`max_decoding_length` is optional.
  
      Args:
          memory (optional): The memory to attend, e.g., the output of an RNN
              encoder. A Tensor of shape `[batch_size, memory_max_time, dim]`.
          memory_sequence_length (optional): A Tensor of shape `[batch_size]`
              containing the sequence lengths for the batch entries in
              memory. Used to create attention bias of
              :attr:`memory_attention_bias` is not given. Ignored if
              `memory_attention_bias` is provided.
          memory_attention_bias (optional): A Tensor of shape
              `[batch_size, num_heads, memory_max_time, dim]`.
              An attention bias typically sets the value of a padding
              position to a large negative value for masking. If not given,
              :attr:`memory_sequence_length` is used to automatically
              create an attention bias.
          inputs (optional): Input tensor for teacher forcing decoding, of
              shape `[batch_size, target_max_time, emb_dim]` containing the
              target sequence word embeddings.
              Used when :attr:`decoding_strategy` is set to "train_greedy".
          decoding_strategy (str): A string specifying the decoding
              strategy, including "train_greedy", "infer_greedy",
              "infer_sample".
              Different arguments are required based on the
              strategy. See above for details. Ignored if
              :attr:`beam_width` or :attr:`helper` is set.
          beam_width (int): Set to use beam search. If given,
              :attr:`decoding_strategy` is ignored.
          length_penalty (float): Length penalty coefficient used in beam
              search decoding. Refer to https://arxiv.org/abs/1609.08144
              for more details.
              It Should be larger if longer sentences are wanted.
          start_tokens (optional): An int Tensor of shape `[batch_size]`,
              containing the start tokens.
              Used when :attr:`decoding_strategy` = "infer_greedy" or
              "infer_sample", or :attr:`beam_width` is set.
              Ignored when context is set.
          end_token (optional): An int 0D Tensor, the token that marks end
              of decoding.
              Used when :attr:`decoding_strategy` = "infer_greedy" or
              "infer_sample", or :attr:`beam_width` is set.
          context (optional): An int Tensor of shape `[batch_size, length]`,
              containing the starting tokens for decoding.
              If context is set, the start_tokens will be ignored.
          context_sequence_length (optional): specify the length of context.
          softmax_temperature (optional): A float 0D Tensor, value to divide
              the logits by before computing the softmax. Larger values
              (above 1.0) result in more random samples. Must > 0. If `None`,
              1.0 is used.
              Used when `decoding_strategy = "infer_sample"`.
          max_decoding_length (optional): An int scalar Tensor indicating
              the maximum allowed number of decoding steps.
              If `None` (default), use `"max_decoding_length"` defined in
              :attr:`hparams`. Ignored in "train_greedy" decoding.
          impute_finished (bool): If `True`, then states for batch
              entries which are marked as finished get copied through and
              the corresponding outputs get zeroed out.  This causes some
              slowdown at each time step, but ensures that the final state
              and outputs have the correct values and that backprop ignores
              time steps that were marked as finished. Ignored in
              "train_greedy" decoding.
          embedding (optional): Embedding used when
              "infer_greedy" or "infer_sample" `decoding_strategy`, or
              beam search, is used. This can be
              a callable or the `params` argument for
              :tf_main:`embedding_lookup <nn/embedding_lookup>`.
              If a callable, it can take a vector tensor of token `ids`,
              or take two arguments (`ids`, `times`), where `ids`
              is a vector tensor of token ids, and `times` is a vector tensor
              of time steps (i.e., position ids). The latter case can be used
              when attr:`embedding` is a combination of word embedding and
              position embedding.
          helper (optional): An instance of
              :tf_main:`Helper <contrib/seq2seq/Helper>` that defines the
              decoding strategy. If given, :attr:`decoding_strategy` is
              ignored.
          mode (optional): A tensor taking value in
              :tf_main:`tf.estimator.ModeKeys <estimator/ModeKeys>`, including
              `TRAIN`, `EVAL`, and `PREDICT`. Controls dropout mode.
              If `None` (default), :func:`texar.tf.global_mode`
              is used.
  
      Returns:
  
          - For **"train_greedy"** decoding, returns an instance of
            :class:`~texar.tf.modules.TransformerDecoderOutput` which contains
            `sample_id` and `logits`.
  
          - For **"infer_greedy"** and **"infer_sample"** decoding or
            decoding with :attr:`helper`, returns
            a tuple `(outputs, sequence_lengths)`, where `outputs` is an
            instance of :class:`~texar.tf.modules.TransformerDecoderOutput` as
            in "train_greedy", and `sequence_lengths` is a Tensor of shape
            `[batch_size]` containing the length of each sample.
  
          - For **beam search** decoding, returns a `dict` containing keys
            "sample_id" and "log_prob".
  
              - **"sample_id"** is an int Tensor of shape
                `[batch_size, max_time, beam_width]` containing generated
                token indexes. `sample_id[:,:,0]` is the highest-probable
                sample.
              - **"log_prob"** is a float Tensor of shape
                `[batch_size, beam_width]` containing the log probability
                of each sequence sample.
      """
  
      if memory is not None:
          if memory_attention_bias is None:
              if memory_sequence_length is None:
                  raise ValueError(
                      "`memory_sequence_length` is required if "
                      "`memory_attention_bias` is not given."
                  )
  
              enc_padding = 1 - tf.sequence_mask(
                  memory_sequence_length,
                  shape_list(memory)[1],
                  dtype=tf.float32,
              )
              memory_attention_bias = (
                  attn.attention_bias_ignore_padding(enc_padding)
              )
  
      # record the context, which will be used in step function
      # for dynamic_decode
      if context is not None:
          start_tokens = context[:, 0]
          self.context = context[:, 1:]
          self.context_sequence_length = context_sequence_length - 1
      else:
          self.context = None
  
      self.embedding = embedding
  
      if (
          helper is None
          and beam_width is None
          and decoding_strategy == "train_greedy"
      ):  # Teacher-forcing
          decoder_self_attention_bias = (
              attn.attention_bias_lower_triangle(shape_list(inputs)[1])
          )
  
          decoder_output = self._self_attention_stack(
              inputs,
              memory,
              decoder_self_attention_bias=decoder_self_attention_bias,
              memory_attention_bias=memory_attention_bias,
              cache=None,
              mode=mode,
          )
          logits = self._output_layer(decoder_output)
          preds = tf.cast(tf.argmax(logits, axis=-1), tf.int32)
          rets = TransformerDecoderOutput(
              logits=logits, sample_id=preds
          )
  
      else:
          if max_decoding_length is None:
              max_decoding_length = self._hparams.max_decoding_length
          self.max_decoding_length = max_decoding_length
          if beam_width is None:  # Inference-like decoding
              # Prepare helper
              if helper is None:
                  if decoding_strategy == "infer_greedy":
                      helper = tx_helper.GreedyEmbeddingHelper(
                          embedding, start_tokens, end_token
                      )
                  elif decoding_strategy == "infer_sample":
                      helper = tx_helper.SampleEmbeddingHelper(
                          embedding,
                          start_tokens,
                          end_token,
                          softmax_temperature,
                      )
                  else:
                      raise ValueError(
                          "Unknown decoding strategy: {}".format(
                              decoding_strategy
                          )
                      )
              self._helper = helper
  
              self._cache = self._init_cache(
                  memory,
                  memory_attention_bias,
                  beam_search_decoding=False,
              )
              if context is not None:
                  self.context = tf.pad(
                      self.context,
                      [
                          [0, 0],
                          [
                              0,
                              max_decoding_length
                              - shape_list(self.context)[1],
                          ],
                      ],
                  )
  
              outputs, _, sequence_lengths = dynamic_decode(
                  decoder=self,
                  impute_finished=impute_finished,
                  maximum_iterations=max_decoding_length,
                  output_time_major=False,
                  scope=self.variable_scope,
              )
  
              if context is not None:
                  # Here the length of sample_id will be larger than that
                  # of logit by 1, because there will be a additional
                  # start_token in the returned sample_id.
                  # the start_id should be the first token of the
                  # given context
                  outputs = TransformerDecoderOutput(
                      logits=outputs.logits,
                      sample_id=tf.concat(
                          [
                              tf.expand_dims(start_tokens, 1),
                              outputs.sample_id,
                          ],
                          axis=1,
                      ),
                  )
                  sequence_lengths = sequence_lengths + 1
              rets = outputs, sequence_lengths
  
          else:  # Beam-search decoding
              # Ignore `decoding_strategy`; Assume `helper` is not set
              if helper is not None:
                  raise ValueError(
                      "Must not set 'beam_width' and 'helper' "
                      "simultaneously."
                  )
              _batch_size = shape_list(start_tokens)[0]
              self._cache = self._init_cache(
                  memory,
                  memory_attention_bias,
                  beam_search_decoding=True,
                  batch_size=_batch_size,
              )
  
              # The output format is different when running beam search
              sample_id, log_prob = self._beam_decode(
                  start_tokens,
                  end_token,
                  beam_width=beam_width,
                  length_penalty=length_penalty,
                  decode_length=max_decoding_length,
              )
              rets = {"sample_id": sample_id, "log_prob": log_prob}
  
      if not self._built:
          self._add_internal_trainable_variables()
          self._built = True
  
      return rets
  
imports: import collections
  import tensorflow
  from tensorflow.contrib.seq2seq import Decoder
  from texar.tf.core import layers
  from texar.tf.module_base import ModuleBase
  from texar.tf.modules.networks.networks import FeedForwardNetwork
  from texar.tf.modules.encoders.transformer_encoders import default_transformer_poswise_net_hparams
  from texar.tf.modules.encoders.multihead_attention import MultiheadAttentionEncoder
  from texar.tf.modules.decoders.rnn_decoder_base import _make_output_layer
  from texar.tf.modules.decoders import tf_helpers
  from texar.tf.utils import beam_search, transformer_attentions
  from texar.tf.utils.shapes import shape_list
  from texar.tf.utils.mode import is_train_mode
  from texar.tf.modules.decoders.dynamic_decode import dynamic_decode
label_desc: Use image node_id 5 for calling the TransformerDecoder obj's underlying member method code with example usage: obj._build(decoding_strategy, inputs, memory, memory_sequence_length, memory_attention_bias, beam_width, length_penalty, start_tokens, end_token, context, context_sequence_length, softmax_temperature, max_decoding_length, impute_finished, embedding, helper, mode) and returns: rets