code: def activation_channels_l1(activation):
      """Calculate the L1-norms of an activation's channels.
  
      The activation usually has the shape: (batch_size, num_channels, h, w).
  
      When the activations are computed on a distributed GPU system, different parts of the
      activation tensor might be computed by a differnt GPU. If this function is called from
      the forward-callback of some activation module in the graph, we will only witness part
      of the batch.  For example, if the batch_size is 256, and we are using 4 GPUS, instead
      of seeing activations with shape = (256, num_channels, h, w), we may see 4 calls with
      shape = (64, num_channels, h, w).
  
      Since we want to calculate the average of the L1-norm of each of the channels of the
      activation, we need to move the partial sums results to the CPU, where they will be
      added together.
  
      Returns - for each channel: the batch-mean of its L1 magnitudes (i.e. over all of the
      activations in the mini-batch, compute the mean of the L! magnitude of each channel).
      """
      if activation.dim() == 4:
          view_2d = activation.view(
              -1, activation.size(2) * activation.size(3)
          )  # (batch*channels) x (h*w)
          featuremap_norms = view_2d.norm(
              p=1, dim=1
          )  # (batch*channels) x 1
          featuremap_norms_mat = featuremap_norms.view(
              activation.size(0), activation.size(1)
          )  # batch x channels
      elif activation.dim() == 2:
          featuremap_norms_mat = activation.norm(
              p=1, dim=1
          )  # batch x 1
      else:
          raise ValueError(
              "activation_channels_l1: Unsupported shape: ".format(
                  activation.shape
              )
          )
      # We need to move the results back to the CPU
      return featuremap_norms_mat.mean(dim=0).cpu()
  
imports: import argparse
  from collections import OrderedDict
  from copy import deepcopy
  import logging
  import operator
  import random
  import numpy
  import torch
  import torch.nn
  import torch.backends.cudnn
  import yaml
  import inspect
label_desc: Use image node_id 32 for calling a global function with example usage: activation_channels_l1(activation) and returns: featuremap_norms_mat