class_docstr: A class containing all functions for auto-regressive text generation, to be used as a mixin in [`PreTrainedModel`].
  
  The class exposes [`~generation.GenerationMixin.generate`], which can be used for:
      - *greedy decoding* by calling [`~generation.GenerationMixin.greedy_search`] if `num_beams=1` and
        `do_sample=False`
      - *contrastive search* by calling [`~generation.GenerationMixin.contrastive_search`] if `penalty_alpha>0` and
        `top_k>1`
      - *multinomial sampling* by calling [`~generation.GenerationMixin.sample`] if `num_beams=1` and
        `do_sample=True`
      - *beam-search decoding* by calling [`~generation.GenerationMixin.beam_search`] if `num_beams>1` and
        `do_sample=False`
      - *beam-search multinomial sampling* by calling [`~generation.GenerationMixin.beam_sample`] if `num_beams>1`
        and `do_sample=True`
      - *diverse beam-search decoding* by calling [`~generation.GenerationMixin.group_beam_search`], if `num_beams>1`
        and `num_beam_groups>1`
      - *constrained beam-search decoding* by calling [`~generation.GenerationMixin.constrained_beam_search`], if
        `constraints!=None` or `force_words_ids!=None`
  
  You do not need to call any of the above methods directly. Pass custom parameter values to 'generate' instead. To
  learn more about decoding strategies refer to the [text generation strategies guide](../generation_strategies).
class_docstr_tok: ["A","class","containing","all","functions","for","auto-regressive","text","generation",",","to","be","used","as","a","mixin","in","[","`","PreTrainedModel","`","]",".","The","class","exposes","[","`","~generation.GenerationMixin.generate","`","]",",","which","can","be","used","for",":","-","*","greedy","decoding","*","by","calling","[","`","~generation.GenerationMixin.greedy_search","`","]","if","`","num_beams=1","`","and","`","do_sample=False","`","-","*","contrastive","search","*","by","calling","[","`","~generation.GenerationMixin.contrastive_search","`","]","if","`","penalty_alpha",">","0","`","and","`","top_k",">","1","`","-","*","multinomial","sampling","*","by","calling","[","`","~generation.GenerationMixin.sample","`","]","if","`","num_beams=1","`","and","`","do_sample=True","`","-","*","beam-search","decoding","*","by","calling","[","`","~generation.GenerationMixin.beam_search","`","]","if","`","num_beams",">","1","`","and","`","do_sample=False","`","-","*","beam-search","multinomial","sampling","*","by","calling","[","`","~generation.GenerationMixin.beam_sample","`","]","if","`","num_beams",">","1","`","and","`","do_sample=True","`","-","*","diverse","beam-search","decoding","*","by","calling","[","`","~generation.GenerationMixin.group_beam_search","`","]",",","if","`","num_beams",">","1","`","and","`","num_beam_groups",">","1","`","-","*","constrained","beam-search","decoding","*","by","calling","[","`","~generation.GenerationMixin.constrained_beam_search","`","]",",","if","`","constraints","!","=None","`","or","`","force_words_ids","!","=None","`","You","do","not","need","to","call","any","of","the","above","methods","directly",".","Pass","custom","parameter","values","to","'generate","'","instead",".","To","learn","more","about","decoding","strategies","refer","to","the","[","text","generation","strategies","guide","]","(","..","\/generation_strategies",")","."]
docstr: Generates sequences of token ids for models with a language modeling head using **beam search decoding** and
  can be used for text-decoder, text-to-text, speech-to-text, and vision-to-text models.
  
  <Tip warning={true}>
  
  In most cases, you do not need to call [`~generation.GenerationMixin.beam_search`] directly. Use generate()
  instead. For an overview of generation strategies and code examples, check the [following
  guide](../generation_strategies).
  
  </Tip>
  
  Parameters:
      input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
          The sequence used as a prompt for the generation.
      beam_scorer (`BeamScorer`):
          An derived instance of [`BeamScorer`] that defines how beam hypotheses are constructed, stored and
          sorted during generation. For more information, the documentation of [`BeamScorer`] should be read.
      logits_processor (`LogitsProcessorList`, *optional*):
          An instance of [`LogitsProcessorList`]. List of instances of class derived from [`LogitsProcessor`]
          used to modify the prediction scores of the language modeling head applied at each generation step.
      stopping_criteria (`StoppingCriteriaList`, *optional*):
          An instance of [`StoppingCriteriaList`]. List of instances of class derived from [`StoppingCriteria`]
          used to tell if the generation loop should stop.
      max_length (`int`, *optional*, defaults to 20):
          **DEPRECATED**. Use `logits_processor` or `stopping_criteria` directly to cap the number of generated
          tokens. The maximum length of the sequence to be generated.
      pad_token_id (`int`, *optional*):
          The id of the *padding* token.
      eos_token_id (`Union[int, List[int]]`, *optional*):
          The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens.
      output_attentions (`bool`, *optional*, defaults to `False`):
          Whether or not to return the attentions tensors of all attention layers. See `attentions` under
          returned tensors for more details.
      output_hidden_states (`bool`, *optional*, defaults to `False`):
          Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
          for more details.
      output_scores (`bool`, *optional*, defaults to `False`):
          Whether or not to return the prediction scores. See `scores` under returned tensors for more details.
      return_dict_in_generate (`bool`, *optional*, defaults to `False`):
          Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
      synced_gpus (`bool`, *optional*, defaults to `False`):
          Whether to continue running the while loop until max_length (needed for ZeRO stage 3)
      model_kwargs:
          Additional model specific kwargs will be forwarded to the `forward` function of the model. If model is
          an encoder-decoder model the kwargs should include `encoder_outputs`.
  
  Return:
      [`generation.GenerateBeamDecoderOnlyOutput`], [`~generation.GenerateBeamEncoderDecoderOutput`] or
      `torch.LongTensor`: A `torch.LongTensor` containing the generated tokens (default behaviour) or a
      [`~generation.GenerateBeamDecoderOnlyOutput`] if `model.config.is_encoder_decoder=False` and
      `return_dict_in_generate=True` or a [`~generation.GenerateBeamEncoderDecoderOutput`] if
      `model.config.is_encoder_decoder=True`.
  
  
  Examples:
  
  ```python
  >>> from transformers import (
  ...     AutoTokenizer,
  ...     AutoModelForSeq2SeqLM,
  ...     LogitsProcessorList,
  ...     MinLengthLogitsProcessor,
  ...     BeamSearchScorer,
  ... )
  >>> import torch
  
  >>> tokenizer = AutoTokenizer.from_pretrained("t5-base")
  >>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
  
  >>> encoder_input_str = "translate English to German: How old are you?"
  >>> encoder_input_ids = tokenizer(encoder_input_str, return_tensors="pt").input_ids
  
  
  >>> # lets run beam search using 3 beams
  >>> num_beams = 3
  >>> # define decoder start token ids
  >>> input_ids = torch.ones((num_beams, 1), device=model.device, dtype=torch.long)
  >>> input_ids = input_ids * model.config.decoder_start_token_id
  
  >>> # add encoder_outputs to model keyword arguments
  >>> model_kwargs = {
  ...     "encoder_outputs": model.get_encoder()(
  ...         encoder_input_ids.repeat_interleave(num_beams, dim=0), return_dict=True
  ...     )
  ... }
  
  >>> # instantiate beam scorer
  >>> beam_scorer = BeamSearchScorer(
  ...     batch_size=1,
  ...     num_beams=num_beams,
  ...     device=model.device,
  ... )
  
  >>> # instantiate logits processors
  >>> logits_processor = LogitsProcessorList(
  ...     [
  ...         MinLengthLogitsProcessor(5, eos_token_id=model.config.eos_token_id),
  ...     ]
  ... )
  
  >>> outputs = model.beam_search(input_ids, beam_scorer, logits_processor=logits_processor, **model_kwargs)
  
  >>> tokenizer.batch_decode(outputs, skip_special_tokens=True)
  ['Wie alt bist du?']
  ```
docstr_tok: ["Generates","sequences","of","token","ids","for","models","with","a","language","modeling","head","using","*","*","beam","search","decoding","*","*","and","can","be","used","for","text-decoder",",","text-to-text",",","speech-to-text",",","and","vision-to-text","models",".","<","Tip","warning=","{","true","}",">","In","most","cases",",","you","do","not","need","to","call","[","`","~generation.GenerationMixin.beam_search","`","]","directly",".","Use","generate","(",")","instead",".","For","an","overview","of","generation","strategies","and","code","examples",",","check","the","[","following","guide","]","(","..","\/generation_strategies",")",".","<","\/Tip",">","Parameters",":","input_ids","(","`","torch.LongTensor","`","of","shape","`","(","batch_size",",","sequence_length",")","`",")",":","The","sequence","used","as","a","prompt","for","the","generation",".","beam_scorer","(","`","BeamScorer","`",")",":","An","derived","instance","of","[","`","BeamScorer","`","]","that","defines","how","beam","hypotheses","are","constructed",",","stored","and","sorted","during","generation",".","For","more","information",",","the","documentation","of","[","`","BeamScorer","`","]","should","be","read",".","logits_processor","(","`","LogitsProcessorList","`",",","*","optional","*",")",":","An","instance","of","[","`","LogitsProcessorList","`","]",".","List","of","instances","of","class","derived","from","[","`","LogitsProcessor","`","]","used","to","modify","the","prediction","scores","of","the","language","modeling","head","applied","at","each","generation","step",".","stopping_criteria","(","`","StoppingCriteriaList","`",",","*","optional","*",")",":","An","instance","of","[","`","StoppingCriteriaList","`","]",".","List","of","instances","of","class","derived","from","[","`","StoppingCriteria","`","]","used","to","tell","if","the","generation","loop","should","stop",".","max_length","(","`","int","`",",","*","optional","*",",","defaults","to","20",")",":","*","*","DEPRECATED","*","*",".","Use","`","logits_processor","`","or","`","stopping_criteria","`","directly","to","cap","the","number","of","generated","tokens",".","The","maximum","length","of","the","sequence","to","be","generated",".","pad_token_id","(","`","int","`",",","*","optional","*",")",":","The","id","of","the","*","padding","*","token",".","eos_token_id","(","`","Union","[","int",",","List","[","int","]","]","`",",","*","optional","*",")",":","The","id","of","the","*","end-of-sequence","*","token",".","Optionally",",","use","a","list","to","set","multiple","*","end-of-sequence","*","tokens",".","output_attentions","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","or","not","to","return","the","attentions","tensors","of","all","attention","layers",".","See","`","attentions","`","under","returned","tensors","for","more","details",".","output_hidden_states","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","or","not","to","return","the","hidden","states","of","all","layers",".","See","`","hidden_states","`","under","returned","tensors","for","more","details",".","output_scores","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","or","not","to","return","the","prediction","scores",".","See","`","scores","`","under","returned","tensors","for","more","details",".","return_dict_in_generate","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","or","not","to","return","a","[","`","~utils.ModelOutput","`","]","instead","of","a","plain","tuple",".","synced_gpus","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","to","continue","running","the","while","loop","until","max_length","(","needed","for","ZeRO","stage","3",")","model_kwargs",":","Additional","model","specific","kwargs","will","be","forwarded","to","the","`","forward","`","function","of","the","model",".","If","model","is","an","encoder-decoder","model","the","kwargs","should","include","`","encoder_outputs","`",".","Return",":","[","`","generation.GenerateBeamDecoderOnlyOutput","`","]",",","[","`","~generation.GenerateBeamEncoderDecoderOutput","`","]","or","`","torch.LongTensor","`",":","A","`","torch.LongTensor","`","containing","the","generated","tokens","(","default","behaviour",")","or","a","[","`","~generation.GenerateBeamDecoderOnlyOutput","`","]","if","`","model.config.is_encoder_decoder=False","`","and","`","return_dict_in_generate=True","`","or","a","[","`","~generation.GenerateBeamEncoderDecoderOutput","`","]","if","`","model.config.is_encoder_decoder=True","`",".","Examples",":","``","`","python",">",">",">","from","transformers","import","(","...","AutoTokenizer",",","...","AutoModelForSeq2SeqLM",",","...","LogitsProcessorList",",","...","MinLengthLogitsProcessor",",","...","BeamSearchScorer",",","...",")",">",">",">","import","torch",">",">",">","tokenizer","=","AutoTokenizer.from_pretrained","(","``","t5-base","''",")",">",">",">","model","=","AutoModelForSeq2SeqLM.from_pretrained","(","``","t5-base","''",")",">",">",">","encoder_input_str","=","``","translate","English","to","German",":","How","old","are","you","?","''",">",">",">","encoder_input_ids","=","tokenizer","(","encoder_input_str",",","return_tensors=","''","pt","''",")",".input_ids",">",">",">","#","lets","run","beam","search","using","3","beams",">",">",">","num_beams","=","3",">",">",">","#","define","decoder","start","token","ids",">",">",">","input_ids","=","torch.ones","(","(","num_beams",",","1",")",",","device=model.device",",","dtype=torch.long",")",">",">",">","input_ids","=","input_ids","*","model.config.decoder_start_token_id",">",">",">","#","add","encoder_outputs","to","model","keyword","arguments",">",">",">","model_kwargs","=","{","...","``","encoder_outputs","''",":","model.get_encoder","(",")","(","...","encoder_input_ids.repeat_interleave","(","num_beams",",","dim=0",")",",","return_dict=True","...",")","...","}",">",">",">","#","instantiate","beam","scorer",">",">",">","beam_scorer","=","BeamSearchScorer","(","...","batch_size=1",",","...","num_beams=num_beams",",","...","device=model.device",",","...",")",">",">",">","#","instantiate","logits","processors",">",">",">","logits_processor","=","LogitsProcessorList","(","...","[","...","MinLengthLogitsProcessor","(","5",",","eos_token_id=model.config.eos_token_id",")",",","...","]","...",")",">",">",">","outputs","=","model.beam_search","(","input_ids",",","beam_scorer",",","logits_processor=logits_processor",",","*","*","model_kwargs",")",">",">",">","tokenizer.batch_decode","(","outputs",",","skip_special_tokens=True",")","[","'Wie","alt","bist","du","?","'","]","``","`"]
code: def beam_search(
      self,
      input_ids: torch.LongTensor,
      beam_scorer: BeamScorer,
      logits_processor: Optional[LogitsProcessorList] = None,
      stopping_criteria: Optional[StoppingCriteriaList] = None,
      max_length: Optional[int] = None,
      pad_token_id: Optional[int] = None,
      eos_token_id: Optional[Union[int, List[int]]] = None,
      output_attentions: Optional[bool] = None,
      output_hidden_states: Optional[bool] = None,
      output_scores: Optional[bool] = None,
      return_dict_in_generate: Optional[bool] = None,
      synced_gpus: bool = False,
      **model_kwargs,
  ) -> Union[GenerateBeamOutput, torch.LongTensor]:
      r"""
      Generates sequences of token ids for models with a language modeling head using **beam search decoding** and
      can be used for text-decoder, text-to-text, speech-to-text, and vision-to-text models.
  
      <Tip warning={true}>
  
      In most cases, you do not need to call [`~generation.GenerationMixin.beam_search`] directly. Use generate()
      instead. For an overview of generation strategies and code examples, check the [following
      guide](../generation_strategies).
  
      </Tip>
  
      Parameters:
          input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
              The sequence used as a prompt for the generation.
          beam_scorer (`BeamScorer`):
              An derived instance of [`BeamScorer`] that defines how beam hypotheses are constructed, stored and
              sorted during generation. For more information, the documentation of [`BeamScorer`] should be read.
          logits_processor (`LogitsProcessorList`, *optional*):
              An instance of [`LogitsProcessorList`]. List of instances of class derived from [`LogitsProcessor`]
              used to modify the prediction scores of the language modeling head applied at each generation step.
          stopping_criteria (`StoppingCriteriaList`, *optional*):
              An instance of [`StoppingCriteriaList`]. List of instances of class derived from [`StoppingCriteria`]
              used to tell if the generation loop should stop.
          max_length (`int`, *optional*, defaults to 20):
              **DEPRECATED**. Use `logits_processor` or `stopping_criteria` directly to cap the number of generated
              tokens. The maximum length of the sequence to be generated.
          pad_token_id (`int`, *optional*):
              The id of the *padding* token.
          eos_token_id (`Union[int, List[int]]`, *optional*):
              The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens.
          output_attentions (`bool`, *optional*, defaults to `False`):
              Whether or not to return the attentions tensors of all attention layers. See `attentions` under
              returned tensors for more details.
          output_hidden_states (`bool`, *optional*, defaults to `False`):
              Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
              for more details.
          output_scores (`bool`, *optional*, defaults to `False`):
              Whether or not to return the prediction scores. See `scores` under returned tensors for more details.
          return_dict_in_generate (`bool`, *optional*, defaults to `False`):
              Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
          synced_gpus (`bool`, *optional*, defaults to `False`):
              Whether to continue running the while loop until max_length (needed for ZeRO stage 3)
          model_kwargs:
              Additional model specific kwargs will be forwarded to the `forward` function of the model. If model is
              an encoder-decoder model the kwargs should include `encoder_outputs`.
  
      Return:
          [`generation.GenerateBeamDecoderOnlyOutput`], [`~generation.GenerateBeamEncoderDecoderOutput`] or
          `torch.LongTensor`: A `torch.LongTensor` containing the generated tokens (default behaviour) or a
          [`~generation.GenerateBeamDecoderOnlyOutput`] if `model.config.is_encoder_decoder=False` and
          `return_dict_in_generate=True` or a [`~generation.GenerateBeamEncoderDecoderOutput`] if
          `model.config.is_encoder_decoder=True`.
  
  
      Examples:
  
      ```python
      >>> from transformers import (
      ...     AutoTokenizer,
      ...     AutoModelForSeq2SeqLM,
      ...     LogitsProcessorList,
      ...     MinLengthLogitsProcessor,
      ...     BeamSearchScorer,
      ... )
      >>> import torch
  
      >>> tokenizer = AutoTokenizer.from_pretrained("t5-base")
      >>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
  
      >>> encoder_input_str = "translate English to German: How old are you?"
      >>> encoder_input_ids = tokenizer(encoder_input_str, return_tensors="pt").input_ids
  
  
      >>> # lets run beam search using 3 beams
      >>> num_beams = 3
      >>> # define decoder start token ids
      >>> input_ids = torch.ones((num_beams, 1), device=model.device, dtype=torch.long)
      >>> input_ids = input_ids * model.config.decoder_start_token_id
  
      >>> # add encoder_outputs to model keyword arguments
      >>> model_kwargs = {
      ...     "encoder_outputs": model.get_encoder()(
      ...         encoder_input_ids.repeat_interleave(num_beams, dim=0), return_dict=True
      ...     )
      ... }
  
      >>> # instantiate beam scorer
      >>> beam_scorer = BeamSearchScorer(
      ...     batch_size=1,
      ...     num_beams=num_beams,
      ...     device=model.device,
      ... )
  
      >>> # instantiate logits processors
      >>> logits_processor = LogitsProcessorList(
      ...     [
      ...         MinLengthLogitsProcessor(5, eos_token_id=model.config.eos_token_id),
      ...     ]
      ... )
  
      >>> outputs = model.beam_search(input_ids, beam_scorer, logits_processor=logits_processor, **model_kwargs)
  
      >>> tokenizer.batch_decode(outputs, skip_special_tokens=True)
      ['Wie alt bist du?']
      ```"""
      # init values
      logits_processor = (
          logits_processor
          if logits_processor is not None
          else LogitsProcessorList()
      )
      stopping_criteria = (
          stopping_criteria
          if stopping_criteria is not None
          else StoppingCriteriaList()
      )
      if max_length is not None:
          warnings.warn(
              "`max_length` is deprecated in this function, use"
              " `stopping_criteria=StoppingCriteriaList([MaxLengthCriteria(max_length=max_length)])` instead.",
              UserWarning,
          )
          stopping_criteria = validate_stopping_criteria(
              stopping_criteria, max_length
          )
      if len(stopping_criteria) == 0:
          warnings.warn(
              "You don't have defined any stopping_criteria, this will likely loop forever",
              UserWarning,
          )
      pad_token_id = (
          pad_token_id
          if pad_token_id is not None
          else self.generation_config.pad_token_id
      )
      eos_token_id = (
          eos_token_id
          if eos_token_id is not None
          else self.generation_config.eos_token_id
      )
      if isinstance(eos_token_id, int):
          eos_token_id = [eos_token_id]
      output_scores = (
          output_scores
          if output_scores is not None
          else self.generation_config.output_scores
      )
      output_attentions = (
          output_attentions
          if output_attentions is not None
          else self.generation_config.output_attentions
      )
      output_hidden_states = (
          output_hidden_states
          if output_hidden_states is not None
          else self.generation_config.output_hidden_states
      )
      return_dict_in_generate = (
          return_dict_in_generate
          if return_dict_in_generate is not None
          else self.generation_config.return_dict_in_generate
      )
  
      batch_size = len(beam_scorer._beam_hyps)
      num_beams = beam_scorer.num_beams
  
      batch_beam_size, cur_len = input_ids.shape
  
      if num_beams * batch_size != batch_beam_size:
          raise ValueError(
              f"Batch dimension of `input_ids` should be {num_beams * batch_size}, but is {batch_beam_size}."
          )
  
      # init attention / hidden states / scores tuples
      scores = (
          () if (return_dict_in_generate and output_scores) else None
      )
      beam_indices = (
          tuple(() for _ in range(batch_beam_size))
          if (return_dict_in_generate and output_scores)
          else None
      )
      decoder_attentions = (
          ()
          if (return_dict_in_generate and output_attentions)
          else None
      )
      cross_attentions = (
          ()
          if (return_dict_in_generate and output_attentions)
          else None
      )
      decoder_hidden_states = (
          ()
          if (return_dict_in_generate and output_hidden_states)
          else None
      )
  
      # if model is an encoder-decoder, retrieve encoder attention weights and hidden states
      if return_dict_in_generate and self.config.is_encoder_decoder:
          encoder_attentions = (
              model_kwargs["encoder_outputs"].get("attentions")
              if output_attentions
              else None
          )
          encoder_hidden_states = (
              model_kwargs["encoder_outputs"].get("hidden_states")
              if output_hidden_states
              else None
          )
  
      # initialise score of first beam with 0 and the rest with -1e9. This makes sure that only tokens
      # of the first beam are considered to avoid sampling the exact same tokens across all beams.
      beam_scores = torch.zeros(
          (batch_size, num_beams),
          dtype=torch.float,
          device=input_ids.device,
      )
      beam_scores[:, 1:] = -1e9
      beam_scores = beam_scores.view((batch_size * num_beams,))
  
      this_peer_finished = False  # used by synced_gpus only
  
      decoder_prompt_len = input_ids.shape[
          -1
      ]  # record the prompt length of decoder
      while True:
          if synced_gpus:
              # Under synced_gpus the `forward` call must continue until all gpus complete their sequence.
              # The following logic allows an early break if all peers finished generating their sequence
              this_peer_finished_flag = torch.tensor(
                  0.0 if this_peer_finished else 1.0
              ).to(input_ids.device)
              # send 0.0 if we finished, 1.0 otherwise
              dist.all_reduce(
                  this_peer_finished_flag, op=dist.ReduceOp.SUM
              )
              # did all peers finish? the reduced sum will be 0.0 then
              if this_peer_finished_flag.item() == 0.0:
                  break
  
          model_inputs = self.prepare_inputs_for_generation(
              input_ids, **model_kwargs
          )
  
          outputs = self(
              **model_inputs,
              return_dict=True,
              output_attentions=output_attentions,
              output_hidden_states=output_hidden_states,
          )
  
          if synced_gpus and this_peer_finished:
              cur_len = cur_len + 1
              continue  # don't waste resources running the code we don't need
  
          next_token_logits = outputs.logits[:, -1, :]
          next_token_scores = nn.functional.log_softmax(
              next_token_logits, dim=-1
          )  # (batch_size * num_beams, vocab_size)
  
          next_token_scores_processed = logits_processor(
              input_ids, next_token_scores
          )
          next_token_scores = next_token_scores_processed + beam_scores[
              :, None
          ].expand_as(next_token_scores_processed)
  
          # Store scores, attentions and hidden_states when required
          if return_dict_in_generate:
              if output_scores:
                  scores += (next_token_scores_processed,)
              if output_attentions:
                  decoder_attentions += (
                      (outputs.decoder_attentions,)
                      if self.config.is_encoder_decoder
                      else (outputs.attentions,)
                  )
                  if self.config.is_encoder_decoder:
                      cross_attentions += (outputs.cross_attentions,)
  
              if output_hidden_states:
                  decoder_hidden_states += (
                      (outputs.decoder_hidden_states,)
                      if self.config.is_encoder_decoder
                      else (outputs.hidden_states,)
                  )
  
          # reshape for beam search
          vocab_size = next_token_scores.shape[-1]
          next_token_scores = next_token_scores.view(
              batch_size, num_beams * vocab_size
          )
  
          # Sample 1 + len(eos_token_id) next tokens for each beam so we have at least 1 non eos token per beam.
          n_eos_tokens = len(eos_token_id) if eos_token_id else 0
          next_token_scores, next_tokens = torch.topk(
              next_token_scores,
              max(2, 1 + n_eos_tokens) * num_beams,
              dim=1,
              largest=True,
              sorted=True,
          )
  
          next_indices = torch.div(
              next_tokens, vocab_size, rounding_mode="floor"
          )
          next_tokens = next_tokens % vocab_size
  
          # stateless
          beam_outputs = beam_scorer.process(
              input_ids,
              next_token_scores,
              next_tokens,
              next_indices,
              pad_token_id=pad_token_id,
              eos_token_id=eos_token_id,
              beam_indices=beam_indices,
              decoder_prompt_len=decoder_prompt_len,
          )
  
          beam_scores = beam_outputs["next_beam_scores"]
          beam_next_tokens = beam_outputs["next_beam_tokens"]
          beam_idx = beam_outputs["next_beam_indices"]
  
          input_ids = torch.cat(
              [input_ids[beam_idx, :], beam_next_tokens.unsqueeze(-1)],
              dim=-1,
          )
  
          model_kwargs = self._update_model_kwargs_for_generation(
              outputs,
              model_kwargs,
              is_encoder_decoder=self.config.is_encoder_decoder,
          )
          if model_kwargs["past_key_values"] is not None:
              model_kwargs[
                  "past_key_values"
              ] = self._temporary_reorder_cache(
                  model_kwargs["past_key_values"], beam_idx
              )
  
          if return_dict_in_generate and output_scores:
              beam_indices = tuple(
                  (
                      beam_indices[beam_idx[i]] + (beam_idx[i],)
                      for i in range(len(beam_indices))
                  )
              )
  
          # increase cur_len
          cur_len = cur_len + 1
  
          if beam_scorer.is_done or stopping_criteria(
              input_ids, scores
          ):
              if not synced_gpus:
                  break
              else:
                  this_peer_finished = True
  
      sequence_outputs = beam_scorer.finalize(
          input_ids,
          beam_scores,
          next_tokens,
          next_indices,
          pad_token_id=pad_token_id,
          eos_token_id=eos_token_id,
          max_length=stopping_criteria.max_length,
          beam_indices=beam_indices,
          decoder_prompt_len=decoder_prompt_len,
      )
  
      if return_dict_in_generate:
          if not output_scores:
              sequence_outputs["sequence_scores"] = None
  
          if self.config.is_encoder_decoder:
              return GenerateBeamEncoderDecoderOutput(
                  sequences=sequence_outputs["sequences"],
                  sequences_scores=sequence_outputs["sequence_scores"],
                  scores=scores,
                  beam_indices=sequence_outputs["beam_indices"],
                  encoder_attentions=encoder_attentions,
                  encoder_hidden_states=encoder_hidden_states,
                  decoder_attentions=decoder_attentions,
                  cross_attentions=cross_attentions,
                  decoder_hidden_states=decoder_hidden_states,
                  past_key_values=model_kwargs.get("past_key_values"),
              )
          else:
              return GenerateBeamDecoderOnlyOutput(
                  sequences=sequence_outputs["sequences"],
                  sequences_scores=sequence_outputs["sequence_scores"],
                  scores=scores,
                  beam_indices=sequence_outputs["beam_indices"],
                  attentions=decoder_attentions,
                  hidden_states=decoder_hidden_states,
                  past_key_values=model_kwargs.get("past_key_values"),
              )
      else:
          return sequence_outputs["sequences"]
  
imports: import copy
  import inspect
  import warnings
  from dataclasses import dataclass
  from typing import TYPE_CHECKING, Any, Callable, Dict, List, Optional, Tuple, Union
  import torch
  import torch.distributed
  from torch import nn
  from ..cache_utils import Cache, DynamicCache
  from ..integrations.deepspeed import is_deepspeed_zero3_enabled
  from ..modeling_outputs import CausalLMOutputWithPast, Seq2SeqLMOutput
  from ..models.auto import MODEL_FOR_CAUSAL_IMAGE_MODELING_MAPPING, MODEL_FOR_CAUSAL_LM_MAPPING, MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING, MODEL_FOR_SPEECH_SEQ_2_SEQ_MAPPING, MODEL_FOR_VISION_2_SEQ_MAPPING
  from ..utils import ExplicitEnum, ModelOutput, is_accelerate_available, logging
  from .beam_constraints import DisjunctiveConstraint, PhrasalConstraint
  from .beam_search import BeamScorer, BeamSearchScorer, ConstrainedBeamSearchScorer
  from .candidate_generator import AssistedCandidateGenerator, CandidateGenerator, PromptLookupCandidateGenerator, _crop_past_key_values, _prepare_attention_mask, _prepare_token_type_ids
  from .configuration_utils import GenerationConfig
  from .logits_process import EncoderNoRepeatNGramLogitsProcessor, EncoderRepetitionPenaltyLogitsProcessor, EpsilonLogitsWarper, EtaLogitsWarper, ExponentialDecayLengthPenalty, ForcedBOSTokenLogitsProcessor, ForcedEOSTokenLogitsProcessor, ForceTokensLogitsProcessor, HammingDiversityLogitsProcessor, InfNanRemoveLogitsProcessor, LogitNormalization, LogitsProcessorList, MinLengthLogitsProcessor, MinNewTokensLengthLogitsProcessor, NoBadWordsLogitsProcessor, NoRepeatNGramLogitsProcessor, PrefixConstrainedLogitsProcessor, RepetitionPenaltyLogitsProcessor, SequenceBiasLogitsProcessor, SuppressTokensAtBeginLogitsProcessor, SuppressTokensLogitsProcessor, TemperatureLogitsWarper, TopKLogitsWarper, TopPLogitsWarper, TypicalLogitsWarper, UnbatchedClassifierFreeGuidanceLogitsProcessor
  from .stopping_criteria import MaxLengthCriteria, MaxTimeCriteria, StoppingCriteria, StoppingCriteriaList, validate_stopping_criteria
label_desc: Use image node_id 27 for calling the GenerationMixin obj's underlying member method code with example usage: obj.beam_search(input_ids, beam_scorer, logits_processor, stopping_criteria, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus) and returns: sequence_outputs, GenerateBeamEncoderDecoderOutput, GenerateBeamDecoderOnlyOutput