class_docstr: Sinusoidal positional encoding for non-recurrent neural networks.
  Implementation based on "Attention Is All You Need"
  
  Args:
     channels (int): embedding size
     dropout_p (float): dropout rate applied to the output.
     max_len (int): maximum sequence length.
     use_scale (bool): whether to use a learnable scaling coefficient.
class_docstr_tok: ["Sinusoidal","positional","encoding","for","non-recurrent","neural","networks",".","Implementation","based","on","``","Attention","Is","All","You","Need","''","Args",":","channels","(","int",")",":","embedding","size","dropout_p","(","float",")",":","dropout","rate","applied","to","the","output",".","max_len","(","int",")",":","maximum","sequence","length",".","use_scale","(","bool",")",":","whether","to","use","a","learnable","scaling","coefficient","."]
code: def __init__(
      self, channels, dropout_p=0.0, max_len=5000, use_scale=False
  ):
      super().__init__()
      if channels % 2 != 0:
          raise ValueError(
              "Cannot use sin/cos positional encoding with "
              "odd channels (got channels={:d})".format(channels)
          )
      self.use_scale = use_scale
      if use_scale:
          self.scale = torch.nn.Parameter(torch.ones(1))
      pe = torch.zeros(max_len, channels)
      position = torch.arange(0, max_len).unsqueeze(1)
      div_term = torch.pow(
          10000, torch.arange(0, channels, 2).float() / channels
      )
      pe[:, 0::2] = torch.sin(position.float() * div_term)
      pe[:, 1::2] = torch.cos(position.float() * div_term)
      pe = pe.unsqueeze(0).transpose(1, 2)
      self.register_buffer("pe", pe)
      if dropout_p > 0:
          self.dropout = nn.Dropout(p=dropout_p)
      self.channels = channels
  
imports: import math
  import torch
  from torch import nn
label_desc: Use image node_id 1 to create a new PositionalEncoding object from inherited base classes: nn with example: obj = PositionalEncoding(channels, dropout_p, max_len, use_scale)