class_docstr: Base class for RNN-with-attention speech decoders.
  
  The decoder takes encoder_outputs, a NestedMap generated by the encoder, as
  input. The NestedMap is expected to contain the following fields:
  
    - 'encoded': encoded features
    - 'padding': padding for encoded features
  
  The decoder operates in a 'step-by-step' fashion. The model encapsulates all
  information which should persist from one step to the next in the
  DecoderStepState NestedMap, which provides a 'misc_states' NestedMap which can
  store arbitrary information required by the specific decoder sub-class.
  
  A 'step' in training consists of the following sequence of steps which compute
  the outputs from the decoder given the current input target (and the state of
  the model after making the previous predictions):
  
  1. Compute the input target at the current time step::
  
      cur_target_info = self.TargetsToBeFedAtCurrentDecodeStep(...)
  
  2. Update state and compute outputs by running SingleDecodeStep::
  
      step_outs, new_state = self.SingleDecodeStep(...)
  
  3. Update state based on the logits computed at this step::
  
      new_state = self.PostStepDecoderStateUpdate(old_state, logits)
  
  4. Display summaries based on the accumulated information across all steps::
  
      self.AddAdditionalDecoderSummaries(seq_out_tas)
  
  Sub-classes can customize behavior by implementing the following functions,
  which will modify the behavior of the decoder:
  
  For beam search decoder:
    - _InitBeamSearchStateCallback
    - _PreBeamSearchStepCallback
    - _PostBeamSearchStepCallback
  For EMBR training:
    - ComputeHypsWithBeamSearch
  
  - MiscZeroState: NestedMap which represents the initial state for the
    'misc_states' in the DecoderStepState. The default implementation returns
    an empty NestedMap.
  
  - SingleDecodeStep: This corresponds to the computation which happens in each
    step of the model. The function should return the outputs of the decoder
    as well as the updated state.
  
  - PostStepDecoderStateUpdate: A function which updates the DecoderStepState
    after the output logits from the decoder have been computed. By default,
    this returns the DecoderStepState unchanged.
  
  - TargetsToBeFedAtCurrentDecodeStep: Returns a TargetInfo namedtuple, which
    represents information about the targets which should be input at the
    current step, as well as the output label which should be predicted.
    The default implementation uses the values in the batched 'targets'
    provided by the InputGenerator.
  
  - AddAdditionalDecoderSummaries: A function which can be used to add any
    decoder specific information as part of the summaries displayed during
    training. By default this is a no-op.
  
  - CreateTargetInfoMisc: A function which can be used to store arbitrary
    information as required by a sub-classes in the target info arrays used
    to determine the current label at each step during training. By default,
    this creates an empty NestedMap.
  
  A few other functions that control how the decoder initializes and computes
  attention during the initial step, and during each step can also be
  modified, if need be:
  
  - _GetAttenContextDim: The dimensionality of the attention context vector.
  - _CreateAtten: Controls how the attention module is configured. Most
    subclasses will not have to change this unless it changes how attention
    works.
  - BaseZeroState: Returns initial state of RNNs, and attention.
  - _InitAttention: Initializes Tensors used by the attention module.
  - _GetInitialSeqStateTensorArrays: Get intitial tensor arrays for
    ComputePredictionsDynamic.
  - _GetNewAttenProbs: Update atten probs for a timestep and return the
    updated tensor array.
class_docstr_tok: ["Base","class","for","RNN-with-attention","speech","decoders",".","The","decoder","takes","encoder_outputs",",","a","NestedMap","generated","by","the","encoder",",","as","input",".","The","NestedMap","is","expected","to","contain","the","following","fields",":","-","'encoded","'",":","encoded","features","-","'padding","'",":","padding","for","encoded","features","The","decoder","operates","in","a","'step-by-step","'","fashion",".","The","model","encapsulates","all","information","which","should","persist","from","one","step","to","the","next","in","the","DecoderStepState","NestedMap",",","which","provides","a","'misc_states","'","NestedMap","which","can","store","arbitrary","information","required","by","the","specific","decoder","sub-class",".","A","'step","'","in","training","consists","of","the","following","sequence","of","steps","which","compute","the","outputs","from","the","decoder","given","the","current","input","target","(","and","the","state","of","the","model","after","making","the","previous","predictions",")",":","1",".","Compute","the","input","target","at","the","current","time","step",":",":","cur_target_info","=","self.TargetsToBeFedAtCurrentDecodeStep","(","...",")","2",".","Update","state","and","compute","outputs","by","running","SingleDecodeStep",":",":","step_outs",",","new_state","=","self.SingleDecodeStep","(","...",")","3",".","Update","state","based","on","the","logits","computed","at","this","step",":",":","new_state","=","self.PostStepDecoderStateUpdate","(","old_state",",","logits",")","4",".","Display","summaries","based","on","the","accumulated","information","across","all","steps",":",":","self.AddAdditionalDecoderSummaries","(","seq_out_tas",")","Sub-classes","can","customize","behavior","by","implementing","the","following","functions",",","which","will","modify","the","behavior","of","the","decoder",":","For","beam","search","decoder",":","-","_InitBeamSearchStateCallback","-","_PreBeamSearchStepCallback","-","_PostBeamSearchStepCallback","For","EMBR","training",":","-","ComputeHypsWithBeamSearch","-","MiscZeroState",":","NestedMap","which","represents","the","initial","state","for","the","'misc_states","'","in","the","DecoderStepState",".","The","default","implementation","returns","an","empty","NestedMap",".","-","SingleDecodeStep",":","This","corresponds","to","the","computation","which","happens","in","each","step","of","the","model",".","The","function","should","return","the","outputs","of","the","decoder","as","well","as","the","updated","state",".","-","PostStepDecoderStateUpdate",":","A","function","which","updates","the","DecoderStepState","after","the","output","logits","from","the","decoder","have","been","computed",".","By","default",",","this","returns","the","DecoderStepState","unchanged",".","-","TargetsToBeFedAtCurrentDecodeStep",":","Returns","a","TargetInfo","namedtuple",",","which","represents","information","about","the","targets","which","should","be","input","at","the","current","step",",","as","well","as","the","output","label","which","should","be","predicted",".","The","default","implementation","uses","the","values","in","the","batched","'targets'","provided","by","the","InputGenerator",".","-","AddAdditionalDecoderSummaries",":","A","function","which","can","be","used","to","add","any","decoder","specific","information","as","part","of","the","summaries","displayed","during","training",".","By","default","this","is","a","no-op",".","-","CreateTargetInfoMisc",":","A","function","which","can","be","used","to","store","arbitrary","information","as","required","by","a","sub-classes","in","the","target","info","arrays","used","to","determine","the","current","label","at","each","step","during","training",".","By","default",",","this","creates","an","empty","NestedMap",".","A","few","other","functions","that","control","how","the","decoder","initializes","and","computes","attention","during","the","initial","step",",","and","during","each","step","can","also","be","modified",",","if","need","be",":","-","_GetAttenContextDim",":","The","dimensionality","of","the","attention","context","vector",".","-","_CreateAtten",":","Controls","how","the","attention","module","is","configured",".","Most","subclasses","will","not","have","to","change","this","unless","it","changes","how","attention","works",".","-","BaseZeroState",":","Returns","initial","state","of","RNNs",",","and","attention",".","-","_InitAttention",":","Initializes","Tensors","used","by","the","attention","module",".","-","_GetInitialSeqStateTensorArrays",":","Get","intitial","tensor","arrays","for","ComputePredictionsDynamic",".","-","_GetNewAttenProbs",":","Update","atten","probs","for","a","timestep","and","return","the","updated","tensor","array","."]
docstr: Computes one 'step' of computation for the decoder.
  
  Must be implemented by sub-classes. Residual connections must also be taken
  care of in sub-classes.
  
  Args:
    theta: A NestedMap object containing weights' values of this
      layer and its children layers.
    packed_src: A NestedMap to represent the packed source tensors generated
      by the attention model.
    cur_target_info: TargetInfo namedtuple, which represents the targets
      which represents information about the target at this step. It is up
      to the various sub-classes to determine how to process the current
      target.
    decoder_step_state: DecoderStepState which encapsulates the state of the
      decoder before computing outputs at the current step.
    per_step_src_padding: Optional padding to be applied to the source_encs
      which overrides the default padding in source_paddings. Used, for
      example, by the Neural Transducer (NT) decoder.
    use_deterministic_random: whether to use deterministic random numbers when
      needed. Must be set to True if called from functional recurrent.
  
  Returns:
    A tuple (step_out, new_decoder_state) which represent the outputs of the
    decoder (usually logits), and the new decoder state after processing the
    current step.
docstr_tok: ["Computes","one","'step","'","of","computation","for","the","decoder",".","Must","be","implemented","by","sub-classes",".","Residual","connections","must","also","be","taken","care","of","in","sub-classes",".","Args",":","theta",":","A","NestedMap","object","containing","weights","'","values","of","this","layer","and","its","children","layers",".","packed_src",":","A","NestedMap","to","represent","the","packed","source","tensors","generated","by","the","attention","model",".","cur_target_info",":","TargetInfo","namedtuple",",","which","represents","the","targets","which","represents","information","about","the","target","at","this","step",".","It","is","up","to","the","various","sub-classes","to","determine","how","to","process","the","current","target",".","decoder_step_state",":","DecoderStepState","which","encapsulates","the","state","of","the","decoder","before","computing","outputs","at","the","current","step",".","per_step_src_padding",":","Optional","padding","to","be","applied","to","the","source_encs","which","overrides","the","default","padding","in","source_paddings",".","Used",",","for","example",",","by","the","Neural","Transducer","(","NT",")","decoder",".","use_deterministic_random",":","whether","to","use","deterministic","random","numbers","when","needed",".","Must","be","set","to","True","if","called","from","functional","recurrent",".","Returns",":","A","tuple","(","step_out",",","new_decoder_state",")","which","represent","the","outputs","of","the","decoder","(","usually","logits",")",",","and","the","new","decoder","state","after","processing","the","current","step","."]
code: def SingleDecodeStep(
      self,
      theta,
      packed_src,
      cur_target_info,
      decoder_step_state,
      per_step_src_padding=None,
      use_deterministic_random=False,
  ):
      """Computes one 'step' of computation for the decoder.
  
      Must be implemented by sub-classes. Residual connections must also be taken
      care of in sub-classes.
  
      Args:
        theta: A NestedMap object containing weights' values of this
          layer and its children layers.
        packed_src: A NestedMap to represent the packed source tensors generated
          by the attention model.
        cur_target_info: TargetInfo namedtuple, which represents the targets
          which represents information about the target at this step. It is up
          to the various sub-classes to determine how to process the current
          target.
        decoder_step_state: DecoderStepState which encapsulates the state of the
          decoder before computing outputs at the current step.
        per_step_src_padding: Optional padding to be applied to the source_encs
          which overrides the default padding in source_paddings. Used, for
          example, by the Neural Transducer (NT) decoder.
        use_deterministic_random: whether to use deterministic random numbers when
          needed. Must be set to True if called from functional recurrent.
  
      Returns:
        A tuple (step_out, new_decoder_state) which represent the outputs of the
        decoder (usually logits), and the new decoder state after processing the
        current step.
      """
      # TODO(syzhang): unify the API to always pass in packed_src.
      raise NotImplementedError("Must be implemented by sub-classes.")
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import math
  import lingvo.compat
  from lingvo.core import attention
  from lingvo.core import base_decoder
  from lingvo.core import base_layer
  from lingvo.core import cluster_factory
  from lingvo.core import layers
  from lingvo.core import plot
  from lingvo.core import py_utils
  from lingvo.core import recurrent
  from lingvo.core import rnn_cell
  from lingvo.core import summary_utils
  from lingvo.core import symbolic
  from lingvo.tasks.asr import contextualizer_base
  from lingvo.tasks.asr import decoder_utils
  from lingvo.tasks.asr import fusion
  from matplotlib import font_manager
  import six
  from six.moves import range
label_desc: Use image node_id 29 for calling the AsrDecoderBase obj's underlying member method code with example usage: obj.SingleDecodeStep(theta, packed_src, cur_target_info, decoder_step_state, per_step_src_padding, use_deterministic_random) without return types