docstr: For all batches, no matter it is a new batch or a mini batch, we need to do prefill first.
docstr_tok: ["For","all","batches",",","no","matter","it","is","a","new","batch","or","a","mini","batch",",","we","need","to","do","prefill","first","."]
code: def _prefill_batch(self, batch):
      """
      For all batches, no matter it is a new batch or a mini batch, we need to do prefill first.
      """
      self._init_batch(batch)
  
      # TODO: figure out if cache and batch id is needed
      ans = self.engine._prefill_batch(batch.batch_id)
      req_to_out_token_id = ans
      self._add_token_id_to_req(batch, req_to_out_token_id)
      has_new_finished_req = batch.mark_finished_req(
          self.eos_id, self.engine.max_output_len
      )
      yield from self._handle_finish_req(batch, has_new_finished_req)
  
imports: import time
  from typing import List
  from .dynamic_batching.get_tokenizer import get_tokenizer
  from .dynamic_batching.infer_batch import InferBatch
  from .dynamic_batching.io_struct import Batch, Req
  from .dynamic_batching.req_queue import ReqQueue
  from .dynamic_batching.sampling_params import SamplingParams
  from .dynamic_batching.stats import Stats
  from .tensor_parallel import TPInferEngine
label_desc: Use image node_id 8 for calling the DynamicBatchManager obj's underlying member method code with example usage: obj._prefill_batch(batch) without return types