class_docstr: An asynchronous wrapper for LLMEngine.
  
  This class is used to wrap the LLMEngine class to make it asynchronous. It
  uses asyncio to create a background loop that keeps processing incoming
  requests. The LLMEngine is kicked by the generate method when there
  are requests in the waiting queue. The generate method yields the outputs
  from the LLMEngine to the caller.
  
  NOTE: For the comprehensive list of arguments, see `LLMEngine`.
  
  Args:
      worker_use_ray: Whether to use Ray for model workers. Required for
          distributed execution. Should be the same as
          `parallel_config.worker_use_ray`.
      log_requests: Whether to log the requests.
      start_engine_loop: If True, the background task to run the engine
          will be automatically started in the generate call.
      *args, *kwargs: Arguments for LLMEngine.
class_docstr_tok: ["An","asynchronous","wrapper","for","LLMEngine",".","This","class","is","used","to","wrap","the","LLMEngine","class","to","make","it","asynchronous",".","It","uses","asyncio","to","create","a","background","loop","that","keeps","processing","incoming","requests",".","The","LLMEngine","is","kicked","by","the","generate","method","when","there","are","requests","in","the","waiting","queue",".","The","generate","method","yields","the","outputs","from","the","LLMEngine","to","the","caller",".","NOTE",":","For","the","comprehensive","list","of","arguments",",","see","`","LLMEngine","`",".","Args",":","worker_use_ray",":","Whether","to","use","Ray","for","model","workers",".","Required","for","distributed","execution",".","Should","be","the","same","as","`","parallel_config.worker_use_ray","`",".","log_requests",":","Whether","to","log","the","requests",".","start_engine_loop",":","If","True",",","the","background","task","to","run","the","engine","will","be","automatically","started","in","the","generate","call",".","*","args",",","*","kwargs",":","Arguments","for","LLMEngine","."]
code: def __init__(
      self,
      # worker_use_ray: bool,
      # engine_use_ray: bool,
      *args,
      log_requests: bool = True,
      max_log_len: Optional[int] = None,
      start_engine_loop: bool = True,
      **kwargs,
  ) -> None:
      # self.worker_use_ray = worker_use_ray
      # self.engine_use_ray = engine_use_ray
      self.log_requests = log_requests
      self.max_log_len = max_log_len
      self.engine = self._init_engine(*args, **kwargs)
  
      self.background_loop = None
      # We need to keep a reference to unshielded
      # task as well to prevent it from being garbage
      # collected
      self._background_loop_unshielded = None
      self.start_engine_loop = start_engine_loop
      self._request_tracker = RequestTracker()
  
imports: import asyncio
  import time
  from functools import partial
  from typing import Any, Dict, Iterable, List, Optional, Set, Tuple, Type, Union
  from bigdl.llm.vllm.config import ModelConfig
  from bigdl.llm.vllm.engine.arg_utils import AsyncEngineArgs
  from bigdl.llm.vllm.engine.llm_engine import LLMEngine
  from bigdl.llm.vllm.logger import init_logger
  from bigdl.llm.vllm.outputs import RequestOutput
  from bigdl.llm.vllm.sampling_params import SamplingParams
  from bigdl.llm.utils.common import invalidInputError
label_desc: Use image node_id 1 to create a new AsyncLLMEngine object  with example: obj = AsyncLLMEngine()