class_docstr: In Appendix F, Page 14 of the TracIn paper, they show that the calculation
  of the influence score of between a test example x' and a training example x,
  can be computed much more quickly than naive back-propagation in the special
  case when considering only gradients in the last fully-connected layer. This class
  computes influence scores for that special case. Note that the computed
  influence scores are exactly the same as when naive back-propagation is used -
  there is no loss in accuracy.
  
  In more detail regarding the influence score computation: let :math`x`
  and :math`\nabla_y f(y)` be the input and output-gradient of the last
  fully-connected layer, respectively, for a training example. Similarly, let
  :math`x'` and :math`\nabla_{y'} f(y')` be the corresponding quantities for
  a test example. Then, the influence score of the training example on the test
  example is the sum of the contribution from each checkpoint. The contribution from
  a given checkpoint is :math`(x^T x')(\nabla_y f(y)^T \nabla_{y'} f(y'))`.
class_docstr_tok: ["In","Appendix","F",",","Page","14","of","the","TracIn","paper",",","they","show","that","the","calculation","of","the","influence","score","of","between","a","test","example","x","'","and","a","training","example","x",",","can","be","computed","much","more","quickly","than","naive","back-propagation","in","the","special","case","when","considering","only","gradients","in","the","last","fully-connected","layer",".","This","class","computes","influence","scores","for","that","special","case",".","Note","that","the","computed","influence","scores","are","exactly","the","same","as","when","naive","back-propagation","is","used","-","there","is","no","loss","in","accuracy",".","In","more","detail","regarding","the","influence","score","computation",":","let",":","math","`","x","`","and",":","math","`","\\nabla_y","f","(","y",")","`","be","the","input","and","output-gradient","of","the","last","fully-connected","layer",",","respectively",",","for","a","training","example",".","Similarly",",","let",":","math","`","x","'","`","and",":","math","`","\\nabla_","{","y","'","}","f","(","y","'",")","`","be","the","corresponding","quantities","for","a","test","example",".","Then",",","the","influence","score","of","the","training","example","on","the","test","example","is","the","sum","of","the","contribution","from","each","checkpoint",".","The","contribution","from","a","given","checkpoint","is",":","math","`","(","x^T","x","'",")","(","\\nabla_y","f","(","y",")","^T","\\nabla_","{","y","'","}","f","(","y","'",")",")","`","."]
docstr: Computes self influence scores for the examples in `inputs`, which is
  either a single batch or a Pytorch `DataLoader` that yields batches. Therefore,
  the computed self influence scores are *not* for the examples in training
  dataset `train_dataset` (unlike when computing self influence scores using the
  `influence` method). Note that if `inputs` is a single batch, this
  will call `model` on that single batch, and if `inputs` yields
  batches, this will call `model` on each batch that is yielded. Therefore,
  please ensure that for both cases, the batch(es) that `model` is called
  with are not too large, so that there will not be an out-of-memory error. This
  implementation performs an outer iteration over checkpoints, and an inner
  iteration over all batches that `inputs` represents. The pros of this
  implementation are that the checkpoints do not need to be loaded too many
  times.
  
  Args:
      batches (tuple or DataLoader): Either a single tuple of any, or a
              `DataLoader`, where each batch yielded is a tuple of any. In
              either case, the tuple represents a single batch, where the last
              element is assumed to be the labels for the batch. That is,
              `model(*batch[0:-1])` produces the output for `model`,
              and `batch[-1]` are the labels, if any. This is the same
              assumption made for each batch yielded by training dataset
              `train_dataset`. Please see documentation for the
              `train_dataset` argument to `TracInCP.__init__` for
              more details on the assumed structure of a batch.
      show_progress (bool, optional): Computation of self influence scores can
              take a long time if `inputs` represents many examples. If
              `show_progress` is true, the progress of this computation will be
              displayed. In more detail, this computation will iterate over all
              checkpoints (provided as the `checkpoints` initialization argument)
              in an outer loop, and iterate over all batches that
              `inputs` represents in an inner loop. Thus if
              `show_progress` is True, the progress of both the outer
              iteration and the inner iterations will be displayed. To show
              progress, it will try to use tqdm if available for advanced
              features (e.g. time estimation). Otherwise, it will fallback to a
              simple output of progress.
              Default: False
  
  Returns:
      self_influence_scores (Tensor): This is a 1D tensor containing the self
              influence scores of all examples in `inputs`, regardless of
              whether it represents a single batch or a `DataLoader` that yields
              batches.
docstr_tok: ["Computes","self","influence","scores","for","the","examples","in","`","inputs","`",",","which","is","either","a","single","batch","or","a","Pytorch","`","DataLoader","`","that","yields","batches",".","Therefore",",","the","computed","self","influence","scores","are","*","not","*","for","the","examples","in","training","dataset","`","train_dataset","`","(","unlike","when","computing","self","influence","scores","using","the","`","influence","`","method",")",".","Note","that","if","`","inputs","`","is","a","single","batch",",","this","will","call","`","model","`","on","that","single","batch",",","and","if","`","inputs","`","yields","batches",",","this","will","call","`","model","`","on","each","batch","that","is","yielded",".","Therefore",",","please","ensure","that","for","both","cases",",","the","batch","(","es",")","that","`","model","`","is","called","with","are","not","too","large",",","so","that","there","will","not","be","an","out-of-memory","error",".","This","implementation","performs","an","outer","iteration","over","checkpoints",",","and","an","inner","iteration","over","all","batches","that","`","inputs","`","represents",".","The","pros","of","this","implementation","are","that","the","checkpoints","do","not","need","to","be","loaded","too","many","times",".","Args",":","batches","(","tuple","or","DataLoader",")",":","Either","a","single","tuple","of","any",",","or","a","`","DataLoader","`",",","where","each","batch","yielded","is","a","tuple","of","any",".","In","either","case",",","the","tuple","represents","a","single","batch",",","where","the","last","element","is","assumed","to","be","the","labels","for","the","batch",".","That","is",",","`","model","(","*","batch","[","0",":","-1","]",")","`","produces","the","output","for","`","model","`",",","and","`","batch","[","-1","]","`","are","the","labels",",","if","any",".","This","is","the","same","assumption","made","for","each","batch","yielded","by","training","dataset","`","train_dataset","`",".","Please","see","documentation","for","the","`","train_dataset","`","argument","to","`","TracInCP.__init__","`","for","more","details","on","the","assumed","structure","of","a","batch",".","show_progress","(","bool",",","optional",")",":","Computation","of","self","influence","scores","can","take","a","long","time","if","`","inputs","`","represents","many","examples",".","If","`","show_progress","`","is","true",",","the","progress","of","this","computation","will","be","displayed",".","In","more","detail",",","this","computation","will","iterate","over","all","checkpoints","(","provided","as","the","`","checkpoints","`","initialization","argument",")","in","an","outer","loop",",","and","iterate","over","all","batches","that","`","inputs","`","represents","in","an","inner","loop",".","Thus","if","`","show_progress","`","is","True",",","the","progress","of","both","the","outer","iteration","and","the","inner","iterations","will","be","displayed",".","To","show","progress",",","it","will","try","to","use","tqdm","if","available","for","advanced","features","(","e.g",".","time","estimation",")",".","Otherwise",",","it","will","fallback","to","a","simple","output","of","progress",".","Default",":","False","Returns",":","self_influence_scores","(","Tensor",")",":","This","is","a","1D","tensor","containing","the","self","influence","scores","of","all","examples","in","`","inputs","`",",","regardless","of","whether","it","represents","a","single","batch","or","a","`","DataLoader","`","that","yields","batches","."]
code: def _self_influence_by_checkpoints(
      self,
      inputs: Union[Tuple[Any, ...], DataLoader],
      show_progress: bool = False,
  ) -> Tensor:
      """
      Computes self influence scores for the examples in `inputs`, which is
      either a single batch or a Pytorch `DataLoader` that yields batches. Therefore,
      the computed self influence scores are *not* for the examples in training
      dataset `train_dataset` (unlike when computing self influence scores using the
      `influence` method). Note that if `inputs` is a single batch, this
      will call `model` on that single batch, and if `inputs` yields
      batches, this will call `model` on each batch that is yielded. Therefore,
      please ensure that for both cases, the batch(es) that `model` is called
      with are not too large, so that there will not be an out-of-memory error. This
      implementation performs an outer iteration over checkpoints, and an inner
      iteration over all batches that `inputs` represents. The pros of this
      implementation are that the checkpoints do not need to be loaded too many
      times.
  
      Args:
          batches (tuple or DataLoader): Either a single tuple of any, or a
                  `DataLoader`, where each batch yielded is a tuple of any. In
                  either case, the tuple represents a single batch, where the last
                  element is assumed to be the labels for the batch. That is,
                  `model(*batch[0:-1])` produces the output for `model`,
                  and `batch[-1]` are the labels, if any. This is the same
                  assumption made for each batch yielded by training dataset
                  `train_dataset`. Please see documentation for the
                  `train_dataset` argument to `TracInCP.__init__` for
                  more details on the assumed structure of a batch.
          show_progress (bool, optional): Computation of self influence scores can
                  take a long time if `inputs` represents many examples. If
                  `show_progress` is true, the progress of this computation will be
                  displayed. In more detail, this computation will iterate over all
                  checkpoints (provided as the `checkpoints` initialization argument)
                  in an outer loop, and iterate over all batches that
                  `inputs` represents in an inner loop. Thus if
                  `show_progress` is True, the progress of both the outer
                  iteration and the inner iterations will be displayed. To show
                  progress, it will try to use tqdm if available for advanced
                  features (e.g. time estimation). Otherwise, it will fallback to a
                  simple output of progress.
                  Default: False
  
      Returns:
          self_influence_scores (Tensor): This is a 1D tensor containing the self
                  influence scores of all examples in `inputs`, regardless of
                  whether it represents a single batch or a `DataLoader` that yields
                  batches.
      """
      # If `inputs` is not a `DataLoader`, turn it into one.
      inputs = _format_inputs_dataset(inputs)
  
      # If `show_progress` is true, create an outer progress bar that keeps track of
      # how many checkpoints have been processed
      if show_progress:
          # Try to determine length of inner progress bar if possible, with a default
          # of `None`.
          inputs_len = None
          try:
              inputs_len = len(inputs)
          except TypeError:
              warnings.warn(
                  "Unable to determine the number of batches in `inputs`. "
                  "Therefore, if showing the progress of the computation of self "
                  "influence scores, only the number of batches processed can be "
                  "displayed, and not the percentage completion of the computation, "
                  "nor any time estimates."
              )
  
      def get_checkpoint_contribution(checkpoint):
          # This function returns a 1D tensor representing the contribution to the
          # self influence score for the given checkpoint, for all batches in
          # `inputs`. The length of the 1D tensor is the total number of
          # examples in `inputs`.
          assert (
              checkpoint is not None
          ), "None returned from `checkpoints`, cannot load."
  
          learning_rate = self.checkpoints_load_func(
              self.model, checkpoint
          )
  
          # This will store a list of the contribution of the self influence score
          # from each batch. Each element is a 1D tensor of length batch_size - the
          # batch size of each batch in `inputs` (they do not need to be all
          # the same)
          checkpoint_contribution = []
  
          _inputs = inputs
          # If `show_progress` is true, create an inner progress bar that keeps track
          # of how many batches have been processed for the current checkpoint
          if show_progress:
              _inputs = progress(
                  inputs,
                  desc=(
                      f"Using {self.get_name()} to compute self "
                      "influence. Processing batch"
                  ),
                  total=inputs_len,
              )
  
          for batch in _inputs:
              (
                  batch_jacobian,
                  batch_layer_input,
              ) = _basic_computation_tracincp_fast(
                  self,
                  batch[0:-1],
                  batch[-1],
                  self.loss_fn,
                  self.reduction_type,
              )
  
              checkpoint_contribution.append(
                  torch.sum(batch_jacobian**2, dim=1)
                  * torch.sum(batch_layer_input**2, dim=1)
                  * learning_rate
              )
  
          # We concatenate the contributions from each batch into a single 1D tensor,
          # which represents the contributions for all batches in `inputs`
          return torch.cat(checkpoint_contribution, dim=0)
  
      if show_progress:
          checkpoints_progress = progress(
              desc=(
                  f"Using {self.get_name()} to compute self "
                  "influence. Processing checkpoint"
              ),
              total=len(self.checkpoints),
              mininterval=0.0,
          )
      else:
          checkpoints_progress = NullProgress()
  
      with checkpoints_progress:
          batches_self_tracin_scores = get_checkpoint_contribution(
              self.checkpoints[0]
          )
          checkpoints_progress.update()
          # The self influence score for all examples is the sum of contributions from
          # each checkpoint
          for checkpoint in self.checkpoints[1:]:
              batches_self_tracin_scores += get_checkpoint_contribution(
                  checkpoint
              )
              checkpoints_progress.update()
          return batches_self_tracin_scores
  
imports: import threading
  import warnings
  from collections import defaultdict
  from typing import Any, Callable, cast, Dict, Iterator, List, Optional, Tuple, Union
  import torch
  from captum._utils.common import _get_module_from_name, _sort_key_list
  from captum._utils.gradient import _gather_distributed_tensors
  from captum._utils.progress import NullProgress, progress
  from captum.influence._core.tracincp import _influence_route_to_helpers, KMostInfluentialResults, TracInCPBase
  from captum.influence._utils.common import _check_loss_fn, _format_inputs_dataset, _get_k_most_influential_helper, _jacobian_loss_wrt_inputs, _load_flexible_state_dict, _self_influence_by_batches_helper, _tensor_batch_dot
  from captum.influence._utils.nearest_neighbors import AnnoyNearestNeighbors, NearestNeighbors
  from captum.log import log_usage
  from torch import device, Tensor
  from torch.nn import Module
  from torch.utils.data import DataLoader, Dataset
label_desc: Use image node_id 6 for calling the TracInCPFast obj's underlying member method code with example usage: obj._self_influence_by_checkpoints(inputs, show_progress) and returns: torch, batches_self_tracin_scores