class_docstr: Builds Non-local Neural Networks as a generic family of building
  blocks for capturing long-range dependencies. Non-local Network
  computes the response at a position as a weighted sum of the
  features at all positions. This building block can be plugged into
  many computer vision architectures.
  More details in the paper: https://arxiv.org/pdf/1711.07971.pdf
class_docstr_tok: ["Builds","Non-local","Neural","Networks","as","a","generic","family","of","building","blocks","for","capturing","long-range","dependencies",".","Non-local","Network","computes","the","response","at","a","position","as","a","weighted","sum","of","the","features","at","all","positions",".","This","building","block","can","be","plugged","into","many","computer","vision","architectures",".","More","details","in","the","paper",":","https",":","\/\/arxiv.org\/pdf\/1711.07971.pdf"]
code: def forward(self, x):
      x_identity = x
      N, C, T, H, W = x.size()
  
      theta = self.conv_theta(x)
  
      # Perform temporal-spatial pooling to reduce the computation.
      if self.use_pool:
          x = self.pool(x)
  
      phi = self.conv_phi(x)
      g = self.conv_g(x)
  
      theta = theta.view(N, self.dim_inner, -1)
      phi = phi.view(N, self.dim_inner, -1)
      g = g.view(N, self.dim_inner, -1)
  
      # (N, C, TxHxW) * (N, C, TxHxW) => (N, TxHxW, TxHxW).
      theta_phi = torch.einsum("nct,ncp->ntp", (theta, phi))
      # For original Non-local paper, there are two main ways to normalize
      # the affinity tensor:
      #   1) Softmax normalization (norm on exp).
      #   2) dot_product normalization.
      if self.instantiation == "softmax":
          # Normalizing the affinity tensor theta_phi before softmax.
          theta_phi = theta_phi * (self.dim_inner**-0.5)
          theta_phi = nn.functional.softmax(theta_phi, dim=2)
      elif self.instantiation == "dot_product":
          spatial_temporal_dim = theta_phi.shape[2]
          theta_phi = theta_phi / spatial_temporal_dim
      else:
          raise NotImplementedError(
              "Unknown norm type {}".format(self.instantiation)
          )
  
      # (N, TxHxW, TxHxW) * (N, C, TxHxW) => (N, C, TxHxW).
      theta_phi_g = torch.einsum("ntg,ncg->nct", (theta_phi, g))
  
      # (N, C, TxHxW) => (N, C, T, H, W).
      theta_phi_g = theta_phi_g.view(N, self.dim_inner, T, H, W)
  
      p = self.conv_out(theta_phi_g)
      p = self.bn(p)
      return x_identity + p
  
imports: import torch
  import torch.nn
label_desc: Use image node_id 3 for calling the Nonlocal obj's underlying member method code with example usage: obj.forward(x) and returns: unknown