class_docstr: A class representing RL environment episodes for individual agents.
  
  SingleAgentEpisode stores observations, info dicts, actions, rewards, and all
  module outputs (e.g. state outs, action logp, etc..) for an individual agent within
  some single-agent or multi-agent environment.
  The two main APIs to add data to an ongoing episode are the `add_env_reset()`
  and `add_env_step()` methods, which should be called passing the outputs of the
  respective gym.Env API calls: `env.reset()` and `env.step()`.
  
  A SingleAgentEpisode might also only represent a chunk of an episode, which is
  useful for cases, in which partial (non-complete episode) sampling is performed
  and collected episode data has to be returned before the actual gym.Env episode has
  finished (see `SingleAgentEpisode.cut()`). In order to still maintain visibility
  onto past experiences within such a "cut" episode, SingleAgentEpisode instances
  can have a "lookback buffer" of n timesteps at their beginning (left side), which
  solely exists for the purpose of compiling extra data (e.g. "prev. reward"), but
  is not considered part of the finished/packaged episode (b/c the data in the
  lookback buffer is already part of a previous episode chunk).
  
  Powerful getter methods, such as `get_observations()` help collect different types
  of data from the episode at individual time indices or time ranges, including the
  "lookback buffer" range described above. For example, to extract the last 4 rewards
  of an ongoing episode, one can call `self.get_rewards(slice(-4, None))` or
  `self.rewards[-4:]`. This would work, even if the ongoing SingleAgentEpisode is
  a continuation chunk from a much earlier started episode, as long as it has a
  lookback buffer size of sufficient size.
  
  Examples:
  
  .. testcode::
  
      import gymnasium as gym
  
      from ray.rllib.env.single_agent_episode import SingleAgentEpisode
  
      # Construct a new episode (without any data in it yet).
      episode = SingleAgentEpisode()
      assert len(episode) == 0
  
      # Fill the episode with some data (10 timesteps).
      env = gym.make("CartPole-v1")
      obs, infos = env.reset()
      episode.add_env_reset(obs, infos)
  
      # Even with the initial obs/infos, the episode is still considered len=0.
      assert len(episode) == 0
      for _ in range(5):
          action = env.action_space.sample()
          obs, reward, term, trunc, infos = env.step(action)
          episode.add_env_step(
              observation=obs,
              action=action,
              reward=reward,
              terminated=term,
              truncated=trunc,
              infos=infos,
          )
      assert len(episode) == 5
  
      # We can now access information from the episode via the getter APIs.
  
      # Get the last 3 rewards (in a batch of size 3).
      episode.get_rewards(slice(-3, None))  # same as `episode.rewards[-3:]`
  
      # Get the most recent action (single item, not batched).
      # This works regardless of the action space or whether the episode has
      # been finalized or not (see below).
      episode.get_actions(-1)  # same as episode.actions[-1]
  
      # Looking back from ts=1, get the previous 4 rewards AND fill with 0.0
      # in case we go over the beginning (ts=0). So we would expect
      # [0.0, 0.0, 0.0, r0] to be returned here, where r0 is the very first received
      # reward in the episode:
      episode.get_rewards(slice(-4, 0), neg_indices_left_of_zero=True, fill=0.0)
  
      # Note the use of fill=0.0 here (fill everything that's out of range with this
      # value) AND the argument `neg_indices_left_of_zero=True`, which interprets
      # negative indices as being left of ts=0 (e.g. -1 being the timestep before
      # ts=0).
  
      # Assuming we had a complex action space (nested gym.spaces.Dict) with one or
      # more elements being Discrete or MultiDiscrete spaces:
      # 1) The `fill=...` argument would still work, filling all spaces (Boxes,
      # Discrete) with that provided value.
      # 2) Setting the flag `one_hot_discrete=True` would convert those discrete
      # sub-components automatically into one-hot (or multi-one-hot) tensors.
      # This simplifies the task of having to provide the previous 4 (nested and
      # partially discrete/multi-discrete) actions for each timestep within a training
      # batch, thereby filling timesteps before the episode started with 0.0s and
      # one-hot'ing the discrete/multi-discrete components in these actions:
      episode = SingleAgentEpisode(action_space=gym.spaces.Dict({
          "a": gym.spaces.Discrete(3),
          "b": gym.spaces.MultiDiscrete([2, 3]),
          "c": gym.spaces.Box(-1.0, 1.0, (2,)),
      }))
  
      # ... fill episode with data ...
      episode.add_env_reset(observation=0)
      # ... from a few steps.
      episode.add_env_step(
          observation=1,
          action={"a":0, "b":np.array([1, 2]), "c":np.array([.5, -.5], np.float32)},
          reward=1.0,
      )
  
      # In your connector
      prev_4_a = []
      # Note here that len(episode) does NOT include the lookback buffer.
      for ts in range(len(episode)):
          prev_4_a.append(
              episode.get_actions(
                  indices=slice(ts - 4, ts),
                  # Make sure negative indices are interpreted as
                  # "into lookback buffer"
                  neg_indices_left_of_zero=True,
                  # Zero-out everything even further before the lookback buffer.
                  fill=0.0,
                  # Take care of discrete components (get ready as NN input).
                  one_hot_discrete=True,
              )
          )
  
      # Finally, convert from list of batch items to a struct (same as action space)
      # of batched (numpy) arrays, in which all leafs have B==len(prev_4_a).
      from ray.rllib.utils.spaces.space_utils import batch
  
      prev_4_actions_col = batch(prev_4_a)
class_docstr_tok: ["A","class","representing","RL","environment","episodes","for","individual","agents",".","SingleAgentEpisode","stores","observations",",","info","dicts",",","actions",",","rewards",",","and","all","module","outputs","(","e.g",".","state","outs",",","action","logp",",","etc","..",")","for","an","individual","agent","within","some","single-agent","or","multi-agent","environment",".","The","two","main","APIs","to","add","data","to","an","ongoing","episode","are","the","`","add_env_reset","(",")","`","and","`","add_env_step","(",")","`","methods",",","which","should","be","called","passing","the","outputs","of","the","respective","gym.Env","API","calls",":","`","env.reset","(",")","`","and","`","env.step","(",")","`",".","A","SingleAgentEpisode","might","also","only","represent","a","chunk","of","an","episode",",","which","is","useful","for","cases",",","in","which","partial","(","non-complete","episode",")","sampling","is","performed","and","collected","episode","data","has","to","be","returned","before","the","actual","gym.Env","episode","has","finished","(","see","`","SingleAgentEpisode.cut","(",")","`",")",".","In","order","to","still","maintain","visibility","onto","past","experiences","within","such","a","``","cut","''","episode",",","SingleAgentEpisode","instances","can","have","a","``","lookback","buffer","''","of","n","timesteps","at","their","beginning","(","left","side",")",",","which","solely","exists","for","the","purpose","of","compiling","extra","data","(","e.g",".","``","prev",".","reward","''",")",",","but","is","not","considered","part","of","the","finished\/packaged","episode","(","b\/c","the","data","in","the","lookback","buffer","is","already","part","of","a","previous","episode","chunk",")",".","Powerful","getter","methods",",","such","as","`","get_observations","(",")","`","help","collect","different","types","of","data","from","the","episode","at","individual","time","indices","or","time","ranges",",","including","the","''","lookback","buffer","''","range","described","above",".","For","example",",","to","extract","the","last","4","rewards","of","an","ongoing","episode",",","one","can","call","`","self.get_rewards","(","slice","(","-4",",","None",")",")","`","or","`","self.rewards","[","-4",":","]","`",".","This","would","work",",","even","if","the","ongoing","SingleAgentEpisode","is","a","continuation","chunk","from","a","much","earlier","started","episode",",","as","long","as","it","has","a","lookback","buffer","size","of","sufficient","size",".","Examples",":","..","testcode",":",":","import","gymnasium","as","gym","from","ray.rllib.env.single_agent_episode","import","SingleAgentEpisode","#","Construct","a","new","episode","(","without","any","data","in","it","yet",")",".","episode","=","SingleAgentEpisode","(",")","assert","len","(","episode",")","==","0","#","Fill","the","episode","with","some","data","(","10","timesteps",")",".","env","=","gym.make","(","``","CartPole-v1","''",")","obs",",","infos","=","env.reset","(",")","episode.add_env_reset","(","obs",",","infos",")","#","Even","with","the","initial","obs\/infos",",","the","episode","is","still","considered","len=0",".","assert","len","(","episode",")","==","0","for","_","in","range","(","5",")",":","action","=","env.action_space.sample","(",")","obs",",","reward",",","term",",","trunc",",","infos","=","env.step","(","action",")","episode.add_env_step","(","observation=obs",",","action=action",",","reward=reward",",","terminated=term",",","truncated=trunc",",","infos=infos",",",")","assert","len","(","episode",")","==","5","#","We","can","now","access","information","from","the","episode","via","the","getter","APIs",".","#","Get","the","last","3","rewards","(","in","a","batch","of","size","3",")",".","episode.get_rewards","(","slice","(","-3",",","None",")",")","#","same","as","`","episode.rewards","[","-3",":","]","`","#","Get","the","most","recent","action","(","single","item",",","not","batched",")",".","#","This","works","regardless","of","the","action","space","or","whether","the","episode","has","#","been","finalized","or","not","(","see","below",")",".","episode.get_actions","(","-1",")","#","same","as","episode.actions","[","-1","]","#","Looking","back","from","ts=1",",","get","the","previous","4","rewards","AND","fill","with","0.0","#","in","case","we","go","over","the","beginning","(","ts=0",")",".","So","we","would","expect","#","[","0.0",",","0.0",",","0.0",",","r0","]","to","be","returned","here",",","where","r0","is","the","very","first","received","#","reward","in","the","episode",":","episode.get_rewards","(","slice","(","-4",",","0",")",",","neg_indices_left_of_zero=True",",","fill=0.0",")","#","Note","the","use","of","fill=0.0","here","(","fill","everything","that","'s","out","of","range","with","this","#","value",")","AND","the","argument","`","neg_indices_left_of_zero=True","`",",","which","interprets","#","negative","indices","as","being","left","of","ts=0","(","e.g",".","-1","being","the","timestep","before","#","ts=0",")",".","#","Assuming","we","had","a","complex","action","space","(","nested","gym.spaces.Dict",")","with","one","or","#","more","elements","being","Discrete","or","MultiDiscrete","spaces",":","#","1",")","The","`","fill=","...","`","argument","would","still","work",",","filling","all","spaces","(","Boxes",",","#","Discrete",")","with","that","provided","value",".","#","2",")","Setting","the","flag","`","one_hot_discrete=True","`","would","convert","those","discrete","#","sub-components","automatically","into","one-hot","(","or","multi-one-hot",")","tensors",".","#","This","simplifies","the","task","of","having","to","provide","the","previous","4","(","nested","and","#","partially","discrete\/multi-discrete",")","actions","for","each","timestep","within","a","training","#","batch",",","thereby","filling","timesteps","before","the","episode","started","with","0.0s","and","#","one-hot'ing","the","discrete\/multi-discrete","components","in","these","actions",":","episode","=","SingleAgentEpisode","(","action_space=gym.spaces.Dict","(","{","``","a","''",":","gym.spaces.Discrete","(","3",")",",","``","b","''",":","gym.spaces.MultiDiscrete","(","[","2",",","3","]",")",",","``","c","''",":","gym.spaces.Box","(","-1.0",",","1.0",",","(","2",",",")",")",",","}",")",")","#","...","fill","episode","with","data","...","episode.add_env_reset","(","observation=0",")","#","...","from","a","few","steps",".","episode.add_env_step","(","observation=1",",","action=","{","``","a","''",":0",",","``","b","''",":","np.array","(","[","1",",","2","]",")",",","``","c","''",":","np.array","(","[",".5",",","-.5","]",",","np.float32",")","}",",","reward=1.0",",",")","#","In","your","connector","prev_4_a","=","[","]","#","Note","here","that","len","(","episode",")","does","NOT","include","the","lookback","buffer",".","for","ts","in","range","(","len","(","episode",")",")",":","prev_4_a.append","(","episode.get_actions","(","indices=slice","(","ts","-","4",",","ts",")",",","#","Make","sure","negative","indices","are","interpreted","as","#","``","into","lookback","buffer","''","neg_indices_left_of_zero=True",",","#","Zero-out","everything","even","further","before","the","lookback","buffer",".","fill=0.0",",","#","Take","care","of","discrete","components","(","get","ready","as","NN","input",")",".","one_hot_discrete=True",",",")",")","#","Finally",",","convert","from","list","of","batch","items","to","a","struct","(","same","as","action","space",")","#","of","batched","(","numpy",")","arrays",",","in","which","all","leafs","have","B==len","(","prev_4_a",")",".","from","ray.rllib.utils.spaces.space_utils","import","batch","prev_4_actions_col","=","batch","(","prev_4_a",")"]
docstr: Initializes a SingleAgentEpisode instance.
  
  This constructor can be called with or without already sampled data, part of
  which might then go into the lookback buffer.
  
  Args:
      id_: Unique identifier for this episode. If no ID is provided the
          constructor generates a unique hexadecimal code for the id.
      observations: Either a list of individual observations from a sampling or
          an already instantiated `BufferWithInfiniteLookback` object (possibly
          with observation data in it). If a list, will construct the buffer
          automatically (given the data and the `len_lookback_buffer` argument).
      observation_space: An optional gym.Space, which all individual observations
          should abide to. If data is appended or set in this SingleAgentEpisode
          and a space has been provided during construction, any new observation
          data will be checked for correctness.
      actions: Either a list of individual info dicts from a sampling or
          an already instantiated `BufferWithInfiniteLookback` object (possibly
          with info dict] data in it). If a list, will construct the buffer
          automatically (given the data and the `len_lookback_buffer` argument).
      actions: Either a list of individual actions from a sampling or
          an already instantiated `BufferWithInfiniteLookback` object (possibly
          with action data in it). If a list, will construct the buffer
          automatically (given the data and the `len_lookback_buffer` argument).
      action_space: An optional gym.Space, which all individual actions
          should abide to. If data is appended or set in this SingleAgentEpisode
          and a space has been provided during construction, any new action
          data will be checked for correctness.
      rewards: Either a list of individual rewards from a sampling or
          an already instantiated `BufferWithInfiniteLookback` object (possibly
          with reward data in it). If a list, will construct the buffer
          automatically (given the data and the `len_lookback_buffer` argument).
      actions: A dict mapping string keys to either lists of individual extra
          model output tensors (e.g. `action_logp` or state outs) from a
          sampling or to an already instantiated `BufferWithInfiniteLookback`
          object (possibly with extra model output data in it). If mapping is to
          a list, will construct the buffer automatically (given the data and
          the `len_lookback_buffer` argument).
      render_images: An optional list of RGB uint8 images from rendering
          the environment.
      terminated: A boolean indicating, if the episode is already terminated.
      truncated: A boolean indicating, if the episode has been truncated.
      t_started: Optional. The starting timestep of the episode. The default
          is zero. If data is provided, the starting point is from the last
          observation onwards (i.e. `t_started = len(observations) - 1). If
          this parameter is provided the episode starts at the provided value.
      len_lookback_buffer: The size of the (optional) lookback buffers to keep in
          front of this Episode for each type of data (observations, actions,
          etc..). If larger 0, will interpret the first `len_lookback_buffer`
          items in each type of data as NOT part of this actual
          episode chunk, but instead serve as "historical" record that may be
          viewed and used to derive new data from. For example, it might be
          necessary to have a lookback buffer of four if you would like to do
          observation frame stacking and your episode has been cut and you are now
          operating on a new chunk (continuing from the cut one). Then, for the
          first 3 items, you would have to be able to look back into the old
          chunk's data.
          If `len_lookback_buffer` is None, will interpret all provided data in
          constructor as part of the lookback buffers.
docstr_tok: ["Initializes","a","SingleAgentEpisode","instance",".","This","constructor","can","be","called","with","or","without","already","sampled","data",",","part","of","which","might","then","go","into","the","lookback","buffer",".","Args",":","id_",":","Unique","identifier","for","this","episode",".","If","no","ID","is","provided","the","constructor","generates","a","unique","hexadecimal","code","for","the","id",".","observations",":","Either","a","list","of","individual","observations","from","a","sampling","or","an","already","instantiated","`","BufferWithInfiniteLookback","`","object","(","possibly","with","observation","data","in","it",")",".","If","a","list",",","will","construct","the","buffer","automatically","(","given","the","data","and","the","`","len_lookback_buffer","`","argument",")",".","observation_space",":","An","optional","gym.Space",",","which","all","individual","observations","should","abide","to",".","If","data","is","appended","or","set","in","this","SingleAgentEpisode","and","a","space","has","been","provided","during","construction",",","any","new","observation","data","will","be","checked","for","correctness",".","actions",":","Either","a","list","of","individual","info","dicts","from","a","sampling","or","an","already","instantiated","`","BufferWithInfiniteLookback","`","object","(","possibly","with","info","dict","]","data","in","it",")",".","If","a","list",",","will","construct","the","buffer","automatically","(","given","the","data","and","the","`","len_lookback_buffer","`","argument",")",".","actions",":","Either","a","list","of","individual","actions","from","a","sampling","or","an","already","instantiated","`","BufferWithInfiniteLookback","`","object","(","possibly","with","action","data","in","it",")",".","If","a","list",",","will","construct","the","buffer","automatically","(","given","the","data","and","the","`","len_lookback_buffer","`","argument",")",".","action_space",":","An","optional","gym.Space",",","which","all","individual","actions","should","abide","to",".","If","data","is","appended","or","set","in","this","SingleAgentEpisode","and","a","space","has","been","provided","during","construction",",","any","new","action","data","will","be","checked","for","correctness",".","rewards",":","Either","a","list","of","individual","rewards","from","a","sampling","or","an","already","instantiated","`","BufferWithInfiniteLookback","`","object","(","possibly","with","reward","data","in","it",")",".","If","a","list",",","will","construct","the","buffer","automatically","(","given","the","data","and","the","`","len_lookback_buffer","`","argument",")",".","actions",":","A","dict","mapping","string","keys","to","either","lists","of","individual","extra","model","output","tensors","(","e.g",".","`","action_logp","`","or","state","outs",")","from","a","sampling","or","to","an","already","instantiated","`","BufferWithInfiniteLookback","`","object","(","possibly","with","extra","model","output","data","in","it",")",".","If","mapping","is","to","a","list",",","will","construct","the","buffer","automatically","(","given","the","data","and","the","`","len_lookback_buffer","`","argument",")",".","render_images",":","An","optional","list","of","RGB","uint8","images","from","rendering","the","environment",".","terminated",":","A","boolean","indicating",",","if","the","episode","is","already","terminated",".","truncated",":","A","boolean","indicating",",","if","the","episode","has","been","truncated",".","t_started",":","Optional",".","The","starting","timestep","of","the","episode",".","The","default","is","zero",".","If","data","is","provided",",","the","starting","point","is","from","the","last","observation","onwards","(","i.e",".","`","t_started","=","len","(","observations",")","-","1",")",".","If","this","parameter","is","provided","the","episode","starts","at","the","provided","value",".","len_lookback_buffer",":","The","size","of","the","(","optional",")","lookback","buffers","to","keep","in","front","of","this","Episode","for","each","type","of","data","(","observations",",","actions",",","etc","..",")",".","If","larger","0",",","will","interpret","the","first","`","len_lookback_buffer","`","items","in","each","type","of","data","as","NOT","part","of","this","actual","episode","chunk",",","but","instead","serve","as","``","historical","''","record","that","may","be","viewed","and","used","to","derive","new","data","from",".","For","example",",","it","might","be","necessary","to","have","a","lookback","buffer","of","four","if","you","would","like","to","do","observation","frame","stacking","and","your","episode","has","been","cut","and","you","are","now","operating","on","a","new","chunk","(","continuing","from","the","cut","one",")",".","Then",",","for","the","first","3","items",",","you","would","have","to","be","able","to","look","back","into","the","old","chunk","'s","data",".","If","`","len_lookback_buffer","`","is","None",",","will","interpret","all","provided","data","in","constructor","as","part","of","the","lookback","buffers","."]
code: def __init__(
      self,
      id_: Optional[str] = None,
      *,
      observations: Optional[
          Union[List[ObsType], BufferWithInfiniteLookback]
      ] = None,
      observation_space: Optional[gym.Space] = None,
      actions: Optional[
          Union[List[ActType], BufferWithInfiniteLookback]
      ] = None,
      action_space: Optional[gym.Space] = None,
      rewards: Optional[
          Union[List[SupportsFloat], BufferWithInfiniteLookback]
      ] = None,
      infos: Optional[
          Union[List[Dict], BufferWithInfiniteLookback]
      ] = None,
      terminated: bool = False,
      truncated: bool = False,
      extra_model_outputs: Optional[Dict[str, Any]] = None,
      render_images: Optional[List[np.ndarray]] = None,
      t_started: Optional[int] = None,
      len_lookback_buffer: Optional[int] = 0,
  ) -> "SingleAgentEpisode":
      """Initializes a SingleAgentEpisode instance.
  
      This constructor can be called with or without already sampled data, part of
      which might then go into the lookback buffer.
  
      Args:
          id_: Unique identifier for this episode. If no ID is provided the
              constructor generates a unique hexadecimal code for the id.
          observations: Either a list of individual observations from a sampling or
              an already instantiated `BufferWithInfiniteLookback` object (possibly
              with observation data in it). If a list, will construct the buffer
              automatically (given the data and the `len_lookback_buffer` argument).
          observation_space: An optional gym.Space, which all individual observations
              should abide to. If data is appended or set in this SingleAgentEpisode
              and a space has been provided during construction, any new observation
              data will be checked for correctness.
          actions: Either a list of individual info dicts from a sampling or
              an already instantiated `BufferWithInfiniteLookback` object (possibly
              with info dict] data in it). If a list, will construct the buffer
              automatically (given the data and the `len_lookback_buffer` argument).
          actions: Either a list of individual actions from a sampling or
              an already instantiated `BufferWithInfiniteLookback` object (possibly
              with action data in it). If a list, will construct the buffer
              automatically (given the data and the `len_lookback_buffer` argument).
          action_space: An optional gym.Space, which all individual actions
              should abide to. If data is appended or set in this SingleAgentEpisode
              and a space has been provided during construction, any new action
              data will be checked for correctness.
          rewards: Either a list of individual rewards from a sampling or
              an already instantiated `BufferWithInfiniteLookback` object (possibly
              with reward data in it). If a list, will construct the buffer
              automatically (given the data and the `len_lookback_buffer` argument).
          actions: A dict mapping string keys to either lists of individual extra
              model output tensors (e.g. `action_logp` or state outs) from a
              sampling or to an already instantiated `BufferWithInfiniteLookback`
              object (possibly with extra model output data in it). If mapping is to
              a list, will construct the buffer automatically (given the data and
              the `len_lookback_buffer` argument).
          render_images: An optional list of RGB uint8 images from rendering
              the environment.
          terminated: A boolean indicating, if the episode is already terminated.
          truncated: A boolean indicating, if the episode has been truncated.
          t_started: Optional. The starting timestep of the episode. The default
              is zero. If data is provided, the starting point is from the last
              observation onwards (i.e. `t_started = len(observations) - 1). If
              this parameter is provided the episode starts at the provided value.
          len_lookback_buffer: The size of the (optional) lookback buffers to keep in
              front of this Episode for each type of data (observations, actions,
              etc..). If larger 0, will interpret the first `len_lookback_buffer`
              items in each type of data as NOT part of this actual
              episode chunk, but instead serve as "historical" record that may be
              viewed and used to derive new data from. For example, it might be
              necessary to have a lookback buffer of four if you would like to do
              observation frame stacking and your episode has been cut and you are now
              operating on a new chunk (continuing from the cut one). Then, for the
              first 3 items, you would have to be able to look back into the old
              chunk's data.
              If `len_lookback_buffer` is None, will interpret all provided data in
              constructor as part of the lookback buffers.
      """
      self.id_ = id_ or uuid.uuid4().hex
  
      # Lookback buffer length is not provided. Interpret already given data as
      # lookback buffer lengths for all data types.
      if len_lookback_buffer is None:
          len_lookback_buffer = len(rewards or [])
  
      infos = infos or [{} for _ in range(len(observations or []))]
  
      # Observations: t0 (initial obs) to T.
      self.observation_space = observation_space
      if isinstance(observations, BufferWithInfiniteLookback):
          self.observations = observations
      else:
          self.observations = BufferWithInfiniteLookback(
              data=observations,
              lookback=len_lookback_buffer,
              space=observation_space,
          )
      # Infos: t0 (initial info) to T.
      if isinstance(infos, BufferWithInfiniteLookback):
          self.infos = infos
      else:
          self.infos = BufferWithInfiniteLookback(
              data=infos,
              lookback=len_lookback_buffer,
          )
      # Actions: t1 to T.
      self.action_space = action_space
      if isinstance(actions, BufferWithInfiniteLookback):
          self.actions = actions
      else:
          self.actions = BufferWithInfiniteLookback(
              data=actions,
              lookback=len_lookback_buffer,
              space=action_space,
          )
      # Rewards: t1 to T.
      if isinstance(rewards, BufferWithInfiniteLookback):
          self.rewards = rewards
      else:
          self.rewards = BufferWithInfiniteLookback(
              data=rewards,
              lookback=len_lookback_buffer,
              space=gym.spaces.Box(
                  float("-inf"), float("inf"), (), np.float32
              ),
          )
  
      # obs[-1] is the final observation in the episode.
      self.is_terminated = terminated
      # obs[-1] is the last obs in a truncated-by-the-env episode (there will no more
      # observations in following chunks for this episode).
      self.is_truncated = truncated
  
      # Extra model outputs, e.g. `action_dist_input` needed in the batch.
      self.extra_model_outputs = defaultdict(
          functools.partial(
              BufferWithInfiniteLookback,
              lookback=len_lookback_buffer,
          ),
      )
      for k, v in (extra_model_outputs or {}).items():
          if isinstance(v, BufferWithInfiniteLookback):
              self.extra_model_outputs[k] = v
          else:
              self.extra_model_outputs[k].data = v
  
      # RGB uint8 images from rendering the env; the images include the corresponding
      # rewards.
      assert render_images is None or observations is not None
      self.render_images = render_images or []
  
      # The global last timestep of the episode and the timesteps when this chunk
      # started (excluding a possible lookback buffer).
      self.t_started = t_started or 0
  
      self.t = (
          (len(rewards) if rewards is not None else 0)
          - len_lookback_buffer
          + self.t_started
      )
  
      # Validate the episode data thus far.
      self.validate()
  
imports: import functools
  from collections import defaultdict
  import numpy
  import uuid
  import gymnasium
  from gymnasium.core import ActType, ObsType
  from typing import Any, Dict, List, Optional, SupportsFloat, Union
  from ray.rllib.policy.sample_batch import SampleBatch
  from ray.rllib.env.utils import BufferWithInfiniteLookback
label_desc: Use image node_id 1 to create a new SingleAgentEpisode object  with example: obj = SingleAgentEpisode(id_)