class_docstr: We use Pytorch's JIT tracer to run a forward pass and generate a trace graph, which
  is an internal representation of the model.  We then use ONNX to "clean" this
  representation.  After builiding a new representation of the graph, we can print
  it to a table, a PNG, add information to nodes, etc.
  
  The trace is a C++ component and the API is not documented, so we need to dig into some
  Pytorch internals code to understand how to get the info we need.
  https://github.com/pytorch/pytorch/blob/master/torch/onnx/__init__.py
  https://github.com/pytorch/pytorch/blob/master/torch/onnx/symbolic.py
  
  We think that using the trace output to generate a representation of the graph, is
  the best method available in Pytorch, due to the module's dynamic nature.
  Pytorch's module API naturally ignores layers/operations which are implemented as
  torch.autograd.Function, without an nn.Module.  For example:
      out = F.relu(self.bn1(self.conv1(x)))
  
  Another case where traversing the nn.Module API is not sufficient to create a
  representation of the graph, is the same nn.Module is used several times in the
  graph.  For example:
  
      out = self.conv1(x)
      out = self.bn1(out)
      out = self.relu(out)    <=== First use of self.relu
  
      out = self.conv2(out)
      out = self.bn2(out)
      out = self.relu(out)    <=== Second use of self.relu
class_docstr_tok: ["We","use","Pytorch","'s","JIT","tracer","to","run","a","forward","pass","and","generate","a","trace","graph",",","which","is","an","internal","representation","of","the","model",".","We","then","use","ONNX","to","``","clean","''","this","representation",".","After","builiding","a","new","representation","of","the","graph",",","we","can","print","it","to","a","table",",","a","PNG",",","add","information","to","nodes",",","etc",".","The","trace","is","a","C++","component","and","the","API","is","not","documented",",","so","we","need","to","dig","into","some","Pytorch","internals","code","to","understand","how","to","get","the","info","we","need",".","https",":","\/\/github.com\/pytorch\/pytorch\/blob\/master\/torch\/onnx\/__init__.py","https",":","\/\/github.com\/pytorch\/pytorch\/blob\/master\/torch\/onnx\/symbolic.py","We","think","that","using","the","trace","output","to","generate","a","representation","of","the","graph",",","is","the","best","method","available","in","Pytorch",",","due","to","the","module","'s","dynamic","nature",".","Pytorch","'s","module","API","naturally","ignores","layers\/operations","which","are","implemented","as","torch.autograd.Function",",","without","an","nn.Module",".","For","example",":","out","=","F.relu","(","self.bn1","(","self.conv1","(","x",")",")",")","Another","case","where","traversing","the","nn.Module","API","is","not","sufficient","to","create","a","representation","of","the","graph",",","is","the","same","nn.Module","is","used","several","times","in","the","graph",".","For","example",":","out","=","self.conv1","(","x",")","out","=","self.bn1","(","out",")","out","=","self.relu","(","out",")","<","===","First","use","of","self.relu","out","=","self.conv2","(","out",")","out","=","self.bn2","(","out",")","out","=","self.relu","(","out",")","<","===","Second","use","of","self.relu"]
code: def add_macs_attr(self):
      for op in self.ops.values():
          op["attrs"]["MACs"] = 0
          if op["type"] == "Conv":
              conv_out = op["outputs"][0]
              conv_in = op["inputs"][0]
              conv_w = op["attrs"]["kernel_shape"]
              groups = op["attrs"]["group"]
              ofm_vol = self.param_volume(conv_out)
              try:
                  # MACs = volume(OFM) * (#IFM * K^2) / #Groups
                  op["attrs"]["MACs"] = int(
                      ofm_vol
                      * SummaryGraph.volume(conv_w)
                      * self.params[conv_in]["shape"][1]
                      / groups
                  )
              except IndexError:
                  # Todo: change the method for calculating MACs
                  msglogger.error(
                      "An input to a Convolutional layer is missing shape information "
                      "(MAC values will be wrong)"
                  )
                  msglogger.error(
                      "For details see https://github.com/NervanaSystems/distiller/issues/168"
                  )
                  op["attrs"]["MACs"] = 0
          elif op["type"] == "Gemm":
              conv_out = op["outputs"][0]
              conv_in = op["inputs"][0]
              try:
                  n_ifm = self.param_shape(conv_in)[1]
                  n_ofm = self.param_shape(conv_out)[1]
              except IndexError:
                  msglogger.error(
                      "An input to a Convolutional layer is missing shape information."
                  )
                  msglogger.error(
                      "For details see https://github.com/NervanaSystems/distiller/issues/360"
                  )
                  n_ifm = n_ofm = 0
  
              # MACs = #IFM * #OFM
              op["attrs"]["MACs"] = n_ofm * n_ifm
  
imports: from .None import distiller
  import re
  import numpy
  import collections
  import torch
  import torch.nn
  import torch.jit
  import logging
  from collections import OrderedDict, defaultdict
  from collections.abc import MutableSequence, Iterable
label_desc: Use image node_id 11 for calling the SummaryGraph obj's underlying member method code with example usage: obj.add_macs_attr() without return types