class_docstr: Measures the latency, number of estimated floating-point operations and parameters of each module in a PyTorch model.
  
  The flops-profiler profiles the forward pass of a PyTorch model and prints the model graph with the measured profile attached to each module. It shows how latency, flops and parameters are spent in the model and which modules or layers could be the bottleneck. It also outputs the names of the top k modules in terms of aggregated latency, flops, and parameters at depth l with k and l specified by the user. The output profile is computed for each batch of input.
  The DeepSpeed flops profiler can be used with the DeepSpeed runtime or as a standalone package.
  When using DeepSpeed for model training, the flops profiler can be configured in the deepspeed_config file and no user code change is required.
  
  If using the profiler as a standalone package, one imports the flops_profiler package and use the APIs.
  
  Here is an example for usage in a typical training workflow:
  
      .. code-block:: python
  
          model = Model()
          prof = FlopsProfiler(model)
  
          for step, batch in enumerate(data_loader):
              if step == profile_step:
                  prof.start_profile()
  
              loss = model(batch)
  
              if step == profile_step:
                  flops = prof.get_total_flops(as_string=True)
                  params = prof.get_total_params(as_string=True)
                  prof.print_model_profile(profile_step=profile_step)
                  prof.end_profile()
  
              loss.backward()
              optimizer.step()
  
  To profile a trained model in inference, use the `get_model_profile` API.
  
  Args:
      object (torch.nn.Module): The PyTorch model to profile.
class_docstr_tok: ["Measures","the","latency",",","number","of","estimated","floating-point","operations","and","parameters","of","each","module","in","a","PyTorch","model",".","The","flops-profiler","profiles","the","forward","pass","of","a","PyTorch","model","and","prints","the","model","graph","with","the","measured","profile","attached","to","each","module",".","It","shows","how","latency",",","flops","and","parameters","are","spent","in","the","model","and","which","modules","or","layers","could","be","the","bottleneck",".","It","also","outputs","the","names","of","the","top","k","modules","in","terms","of","aggregated","latency",",","flops",",","and","parameters","at","depth","l","with","k","and","l","specified","by","the","user",".","The","output","profile","is","computed","for","each","batch","of","input",".","The","DeepSpeed","flops","profiler","can","be","used","with","the","DeepSpeed","runtime","or","as","a","standalone","package",".","When","using","DeepSpeed","for","model","training",",","the","flops","profiler","can","be","configured","in","the","deepspeed_config","file","and","no","user","code","change","is","required",".","If","using","the","profiler","as","a","standalone","package",",","one","imports","the","flops_profiler","package","and","use","the","APIs",".","Here","is","an","example","for","usage","in","a","typical","training","workflow",":","..","code-block",":",":","python","model","=","Model","(",")","prof","=","FlopsProfiler","(","model",")","for","step",",","batch","in","enumerate","(","data_loader",")",":","if","step","==","profile_step",":","prof.start_profile","(",")","loss","=","model","(","batch",")","if","step","==","profile_step",":","flops","=","prof.get_total_flops","(","as_string=True",")","params","=","prof.get_total_params","(","as_string=True",")","prof.print_model_profile","(","profile_step=profile_step",")","prof.end_profile","(",")","loss.backward","(",")","optimizer.step","(",")","To","profile","a","trained","model","in","inference",",","use","the","`","get_model_profile","`","API",".","Args",":","object","(","torch.nn.Module",")",":","The","PyTorch","model","to","profile","."]
docstr: Prints the model graph with the measured profile attached to each module.
  
  Args:
      profile_step (int, optional): The global training step at which to profile. Note that warm up steps are needed for accurate time measurement.
      module_depth (int, optional): The depth of the model to which to print the aggregated module information. When set to -1, it prints information from the top to the innermost modules (the maximum depth).
      top_modules (int, optional): Limits the aggregated profile output to the number of top modules specified.
      detailed (bool, optional): Whether to print the detailed model profile.
      output_file (str, optional): Path to the output file. If None, the profiler prints to stdout.
docstr_tok: ["Prints","the","model","graph","with","the","measured","profile","attached","to","each","module",".","Args",":","profile_step","(","int",",","optional",")",":","The","global","training","step","at","which","to","profile",".","Note","that","warm","up","steps","are","needed","for","accurate","time","measurement",".","module_depth","(","int",",","optional",")",":","The","depth","of","the","model","to","which","to","print","the","aggregated","module","information",".","When","set","to","-1",",","it","prints","information","from","the","top","to","the","innermost","modules","(","the","maximum","depth",")",".","top_modules","(","int",",","optional",")",":","Limits","the","aggregated","profile","output","to","the","number","of","top","modules","specified",".","detailed","(","bool",",","optional",")",":","Whether","to","print","the","detailed","model","profile",".","output_file","(","str",",","optional",")",":","Path","to","the","output","file",".","If","None",",","the","profiler","prints","to","stdout","."]
code: def print_model_profile(
      self,
      profile_step=1,
      module_depth=-1,
      top_modules=1,
      detailed=True,
      output_file=None,
  ):
      """Prints the model graph with the measured profile attached to each module.
  
      Args:
          profile_step (int, optional): The global training step at which to profile. Note that warm up steps are needed for accurate time measurement.
          module_depth (int, optional): The depth of the model to which to print the aggregated module information. When set to -1, it prints information from the top to the innermost modules (the maximum depth).
          top_modules (int, optional): Limits the aggregated profile output to the number of top modules specified.
          detailed (bool, optional): Whether to print the detailed model profile.
          output_file (str, optional): Path to the output file. If None, the profiler prints to stdout.
      """
      if not self.started:
          return
      import sys
      import os.path
  
      original_stdout = None
      f = None
      if output_file and output_file != "":
          dir_path = os.path.dirname(os.path.abspath(output_file))
          if not os.path.exists(dir_path):
              os.makedirs(dir_path)
          original_stdout = sys.stdout
          f = open(output_file, "w")
          sys.stdout = f
  
      total_flops = self.get_total_flops()
      total_macs = self.get_total_macs()
      total_duration = self.get_total_duration()
      total_params = self.get_total_params()
  
      self.flops = total_flops
      self.macs = total_macs
      self.params = total_params
  
      print(
          "\n-------------------------- DeepSpeed Flops Profiler --------------------------"
      )
      print(f"Profile Summary at step {profile_step}:")
      print(
          "Notations:\ndata parallel size (dp_size), model parallel size(mp_size),\nnumber of parameters (params), number of multiply-accumulate operations(MACs),\nnumber of floating-point operations (flops), floating-point operations per second (FLOPS),\nfwd latency (forward propagation latency), bwd latency (backward propagation latency),\nstep (weights update latency), iter latency (sum of fwd, bwd and step latency)\n"
      )
      if self.ds_engine:
          print(
              "{:<60}  {:<8}".format(
                  "world size: ", self.ds_engine.world_size
              )
          )
          print(
              "{:<60}  {:<8}".format(
                  "data parallel size: ", self.ds_engine.dp_world_size
              )
          )
          print(
              "{:<60}  {:<8}".format(
                  "model parallel size: ", self.ds_engine.mp_world_size
              )
          )
          print(
              "{:<60}  {:<8}".format(
                  "batch size per GPU: ",
                  self.ds_engine.train_micro_batch_size_per_gpu(),
              )
          )
  
      print(
          "{:<60}  {:<8}".format(
              "params per gpu: ", params_to_string(total_params)
          )
      )
      print(
          "{:<60}  {:<8}".format(
              "params of model = params per GPU * mp_size: ",
              params_to_string(
                  total_params
                  * (
                      (self.ds_engine.mp_world_size)
                      if self.ds_engine
                      else 1
                  )
              ),
          )
      )
  
      print(
          "{:<60}  {:<8}".format(
              "fwd MACs per GPU: ", macs_to_string(total_macs)
          )
      )
  
      print(
          "{:<60}  {:<8}".format(
              "fwd flops per GPU: ", num_to_string(total_flops)
          )
      )
  
      print(
          "{:<60}  {:<8}".format(
              "fwd flops of model = fwd flops per GPU * mp_size: ",
              num_to_string(
                  total_flops
                  * (
                      (self.ds_engine.mp_world_size)
                      if self.ds_engine
                      else 1
                  )
              ),
          )
      )
  
      fwd_latency = self.get_total_duration()
      if self.ds_engine and self.ds_engine.wall_clock_breakdown():
          fwd_latency = (
              self.ds_engine.timers("forward").elapsed(False) / 1000.0
          )
      print(
          "{:<60}  {:<8}".format(
              "fwd latency: ", duration_to_string(fwd_latency)
          )
      )
      print(
          "{:<60}  {:<8}".format(
              "fwd FLOPS per GPU = fwd flops per GPU / fwd latency: ",
              flops_to_string(total_flops / fwd_latency),
          )
      )
  
      if self.ds_engine and self.ds_engine.wall_clock_breakdown():
          bwd_latency = (
              self.ds_engine.timers("backward").elapsed(False) / 1000.0
          )
          step_latency = (
              self.ds_engine.timers("step").elapsed(False) / 1000.0
          )
          print(
              "{:<60}  {:<8}".format(
                  "bwd latency: ", duration_to_string(bwd_latency)
              )
          )
          print(
              "{:<60}  {:<8}".format(
                  "bwd FLOPS per GPU = 2 * fwd flops per GPU / bwd latency: ",
                  flops_to_string(2 * total_flops / bwd_latency),
              )
          )
          print(
              "{:<60}  {:<8}".format(
                  "fwd+bwd FLOPS per GPU = 3 * fwd flops per GPU / (fwd+bwd latency): ",
                  flops_to_string(
                      3 * total_flops / (fwd_latency + bwd_latency)
                  ),
              )
          )
  
          print(
              "{:<60}  {:<8}".format(
                  "step latency: ", duration_to_string(step_latency)
              )
          )
  
          iter_latency = fwd_latency + bwd_latency + step_latency
          print(
              "{:<60}  {:<8}".format(
                  "iter latency: ", duration_to_string(iter_latency)
              )
          )
          print(
              "{:<60}  {:<8}".format(
                  "FLOPS per GPU = 3 * fwd flops per GPU / iter latency: ",
                  flops_to_string(3 * total_flops / iter_latency),
              )
          )
  
          samples_per_iter = (
              self.ds_engine.train_micro_batch_size_per_gpu()
              * self.ds_engine.world_size
          )
          print(
              "{:<60}  {:<8.2f}".format(
                  "samples/second: ", samples_per_iter / iter_latency
              )
          )
  
      def flops_repr(module):
          params = module.__params__
          flops = get_module_flops(module)
          macs = get_module_macs(module)
          items = [
              params_to_string(params),
              "{:.2%} Params".format(
                  params / total_params if total_params else 0
              ),
              macs_to_string(macs),
              "{:.2%} MACs".format(
                  0.0 if total_macs == 0 else macs / total_macs
              ),
          ]
          duration = get_module_duration(module)
  
          items.append(duration_to_string(duration))
          items.append(
              "{:.2%} latency".format(
                  0.0
                  if total_duration == 0
                  else duration / total_duration
              )
          )
          items.append(
              flops_to_string(
                  0.0 if duration == 0 else flops / duration
              )
          )
          items.append(module.original_extra_repr())
          return ", ".join(items)
  
      def add_extra_repr(module):
          flops_extra_repr = flops_repr.__get__(module)
          if module.extra_repr != flops_extra_repr:
              module.original_extra_repr = module.extra_repr
              module.extra_repr = flops_extra_repr
              assert module.extra_repr != module.original_extra_repr
  
      def del_extra_repr(module):
          if hasattr(module, "original_extra_repr"):
              module.extra_repr = module.original_extra_repr
              del module.original_extra_repr
  
      self.model.apply(add_extra_repr)
  
      print(
          "\n----------------------------- Aggregated Profile per GPU -----------------------------"
      )
      self.print_model_aggregated_profile(
          module_depth=module_depth, top_modules=top_modules
      )
  
      if detailed:
          print(
              "\n------------------------------ Detailed Profile per GPU ------------------------------"
          )
          print(
              "Each module profile is listed after its name in the following order: \nparams, percentage of total params, MACs, percentage of total MACs, fwd latency, percentage of total fwd latency, fwd FLOPS"
          )
          print(
              "\nNote: 1. A module can have torch.nn.module or torch.nn.functional to compute logits (e.g. CrossEntropyLoss). They are not counted as submodules, thus not to be printed out. However they make up the difference between a parent's MACs (or latency) and the sum of its submodules'.\n2. Number of floating-point operations is a theoretical estimation, thus FLOPS computed using that could be larger than the maximum system throughput.\n3. The fwd latency listed in the top module's profile is directly captured at the module forward function in PyTorch, thus it's less than the fwd latency shown above which is captured in DeepSpeed.\n"
          )
          print(self.model)
  
      self.model.apply(del_extra_repr)
  
      print(
          "------------------------------------------------------------------------------"
      )
  
      if output_file:
          sys.stdout = original_stdout
          f.close()
  
imports: import time
  import torch
  import torch.nn
  import torch.nn.functional
  from functools import partial
  from typing import List, Optional
  from collections import OrderedDict
  import numpy
label_desc: Use image node_id 10 for calling the FlopsProfiler obj's underlying member method code with example usage: obj.print_model_profile(profile_step, module_depth, top_modules, detailed, output_file) and returns: str