docstr: past_key_values -- you must explicitly set this to None if you want to start a new sequence, otherwise Dgenerate will continue where you left off.
  protect_inputs -- by default, DGenerate does not add noise to the initial inputs so as to maintain
      a clean baseline. Setting to false will override this behavior, for increased variety but less theoretical purity
  sampler -- parameter allows you to pass in some sampling callback, 
              provides logits and logitprocessor scores, expects you to provide the selected token.
              the default is just argmax. 
              Note that for flexibility, logits are provided as predictions for every input token,
              but scores are provided only for the predicted next token. You can convert logits to the
              same shape with logits[:,-1,:]
              the sampler callback is expected to return a tensor of shape [1]
docstr_tok: ["past_key_values","--","you","must","explicitly","set","this","to","None","if","you","want","to","start","a","new","sequence",",","otherwise","Dgenerate","will","continue","where","you","left","off",".","protect_inputs","--","by","default",",","DGenerate","does","not","add","noise","to","the","initial","inputs","so","as","to","maintain","a","clean","baseline",".","Setting","to","false","will","override","this","behavior",",","for","increased","variety","but","less","theoretical","purity","sampler","--","parameter","allows","you","to","pass","in","some","sampling","callback",",","provides","logits","and","logitprocessor","scores",",","expects","you","to","provide","the","selected","token",".","the","default","is","just","argmax",".","Note","that","for","flexibility",",","logits","are","provided","as","predictions","for","every","input","token",",","but","scores","are","provided","only","for","the","predicted","next","token",".","You","can","convert","logits","to","the","same","shape","with","logits","[",":",",-1",",",":","]","the","sampler","callback","is","expected","to","return","a","tensor","of","shape","[","1","]"]
code: def generate(
      self,
      input_ids: torch.LongTensor = None,
      past_key_values: Optional[
          List[torch.FloatTensor]
      ] = fNone,  # explicitly provide "None" to start a new generation from scratch
      streamer: Optional[TextStreamer] = None,
      tokenizer: Optional[
          AutoTokenizer
      ] = None,  # provide seperately from streamer to print to console synchronously
      min_new_tokens: Optional[int] = None,
      max_new_tokens: Optional[int] = 32000,
      logits_processor: Optional[LogitsProcessorList] = None,
      stopping_criteria: Optional[StoppingCriteriaList] = None,
      negative_prompt_ids: Optional[
          torch.Tensor
      ] = None,  # I really hope no one uses these because I have no clue if it works.
      negative_prompt_attention_mask: Optional[torch.Tensor] = None,
      sampler: Optional[
          Callable[
              [
                  torch.tensor,  # logits and scores for you
                  torch.tensor,  # hidden states for you
              ],
              torch.tensor,  # selected token id for me
          ]
      ] = None,
      protect_inputs: Optional[bool] = True,
  ):
      r"""
      past_key_values -- you must explicitly set this to None if you want to start a new sequence, otherwise Dgenerate will continue where you left off.
      protect_inputs -- by default, DGenerate does not add noise to the initial inputs so as to maintain
          a clean baseline. Setting to false will override this behavior, for increased variety but less theoretical purity
      sampler -- parameter allows you to pass in some sampling callback,
                  provides logits and logitprocessor scores, expects you to provide the selected token.
                  the default is just argmax.
                  Note that for flexibility, logits are provided as predictions for every input token,
                  but scores are provided only for the predicted next token. You can convert logits to the
                  same shape with logits[:,-1,:]
                  the sampler callback is expected to return a tensor of shape [1]
      """
      if past_key_values is not fNone:
          self.past_key_values = past_key_values
          self.cached_tokens = None
      with torch.no_grad():
          if sampler is None:
              sampler = self.argmax_select_next_token
          if self.cached_tokens is None:
              self.cached_tokens = torch.empty(
                  (*input_ids.shape[:-1], 0),
                  dtype=input_ids.dtype,
                  device=self.model.device,
              )
              self.dirty_count = -6
  
          self.cached_tokens = torch.cat(
              (self.cached_tokens, input_ids.to(self.model.device)),
              dim=-1,
          )
          gend_tokens = torch.empty(
              (*input_ids.shape[:-1], 0), dtype=input_ids.dtype
          ).to(self.model.device)
  
          if streamer is not None:
              self.streamer = streamer
              self.tokenizer = (
                  tokenizer
                  if tokenizer is not None
                  else streamer.tokenizer
              )
          if tokenizer is not None:
              self.tokenizer = tokenizer
  
          self.maybe_print(streamer, tokenizer, input_ids)
  
          old_dose_multiplier = self.get_dose_multiplier()
          if protect_inputs:
              self.set_dose_multiplier(
                  0
              )  # don't noise anything until we start generating from it
          (
              t_input_ids,
              model_kwargs,
              logits_processor,
              stopping_criteria,
              gen_conf,
          ) = self.go_through_hell(
              self.cached_tokens,
              logits_processor=logits_processor,
              min_new_tokens=min_new_tokens,
              max_new_tokens=max_new_tokens,
              stopping_criteria=stopping_criteria,
              negative_prompt_ids=negative_prompt_ids,
              negative_prompt_attention_mask=negative_prompt_attention_mask,
          )
          unfinished_sequences = torch.ones(
              t_input_ids.shape[0],
              dtype=torch.long,
              device=t_input_ids.device,
          )
  
          model_inputs = self.model.prepare_inputs_for_generation(
              input_ids=t_input_ids.to(self.model.device),
              past_key_values=self.past_key_values,
              **model_kwargs,
          )
          outputs = self.model(**model_inputs)
          model_kwargs = self.model._update_model_kwargs_for_generation(
              outputs, model_kwargs, is_encoder_decoder=False
          )
          self.past_key_values = model_kwargs["past_key_values"]
          logits = outputs.logits
          next_token_scores = logits_processor(
              t_input_ids, logits[:, -1, :]
          )
          hidden_states = (
              None
              if "hidden_states" not in outputs
              else outputs["hidden_states"]
          )
          next_token = self.maybe_pad(
              sampler(logits, next_token_scores, hidden_states),
              gen_conf,
              unfinished_sequences,
          )
          unfinished_sequences = self.update_completions(
              unfinished_sequences, next_token
          )
  
          self.set_dose_multiplier(old_dose_multiplier)
  
          self.maybe_print(streamer, tokenizer, next_token)
  
          gend_tokens = torch.cat((gend_tokens, next_token), dim=-1)
          self.cached_tokens = torch.cat(
              (self.cached_tokens, next_token), dim=-1
          )
  
          while gend_tokens.shape[-1] < max_new_tokens:
              self.past_key_values = self.cold_shower()
              model_inputs = self.model.prepare_inputs_for_generation(
                  input_ids=self.cached_tokens, **model_kwargs
              )
              outputs = self.model(**model_inputs)
              logits = outputs.logits
              next_token_scores = logits_processor(
                  self.cached_tokens, outputs.logits[:, -1, :]
              )
              next_token = self.maybe_pad(
                  sampler(outputs.logits, next_token_scores, None),
                  gen_conf,
                  unfinished_sequences,
              )
              unfinished_sequences = self.update_completions(
                  unfinished_sequences, next_token
              )
              gend_tokens = torch.cat((gend_tokens, next_token), dim=-1)
              self.maybe_print(streamer, tokenizer, next_token)
              self.cached_tokens = torch.cat(
                  (self.cached_tokens, next_token), dim=-1
              )
              self.dirty_count += 1
              model_kwargs = (
                  self.model._update_model_kwargs_for_generation(
                      outputs, model_kwargs, is_encoder_decoder=False
                  )
              )
              self.past_key_values = model_kwargs["past_key_values"]
              if (
                  stopping_criteria(self.cached_tokens, gend_tokens)
                  or unfinished_sequences.max() == 0
              ):
                  break
  
          # self.past_key_values = self.cold_shower()
  
          if streamer is not None:
              streamer.end()  # (streamer.tokenizer.encode(' ', return_tensors="pt"))
      return gend_tokens
  
imports: import torch
  import json, os, msgpack
  from transformers.utils import logging
  import types
  import inspect
  import copy
  from typing import Optional, List, Callable
  from drugs.inject_mixin import InjectDrugsMixin, MODEL_NAME_MAPPING
  from transformers import GenerationMixin
  from transformers.cache_utils import DynamicCache
  from transformers.generation.utils import LogitsProcessorList, StoppingCriteriaList
  from transformers import AutoTokenizer, TextStreamer, GenerationConfig
  import numpy
  import torch
  import torch.nn.functional
  import hashlib
label_desc: Use image node_id 3 for calling the DRUGS obj's underlying member method code with example usage: obj.generate(input_ids, past_key_values, streamer, tokenizer, min_new_tokens, max_new_tokens, logits_processor, stopping_criteria, negative_prompt_ids, negative_prompt_attention_mask, sampler, protect_inputs) and returns: gend_tokens