class_docstr: A `Highway layer <https://arxiv.org/abs/1505.00387>`_ does a gated combination of a linear
  transformation and a non-linear transformation of its input.  :math:`y = g * x + (1 - g) *
  f(A(x))`, where :math:`A` is a linear transformation, :math:`f` is an element-wise
  non-linearity, and :math:`g` is an element-wise gate, computed as :math:`sigmoid(B(x))`.
  
  This module will apply a fixed number of highway layers to its input, returning the final
  result.
  
  Parameters
  ----------
  input_dim : ``int``
      The dimensionality of :math:`x`.  We assume the input has shape ``(batch_size, ...,
      input_dim)``.
  num_layers : ``int``, optional (default=``1``)
      The number of highway layers to apply to the input.
  activation : ``Callable[[torch.Tensor], torch.Tensor]``, optional (default=``torch.nn.functional.relu``)
      The non-linearity to use in the highway layers.
class_docstr_tok: ["A","`","Highway","layer","<","https",":","\/\/arxiv.org\/abs\/1505.00387",">","`","_","does","a","gated","combination","of","a","linear","transformation","and","a","non-linear","transformation","of","its","input",".",":","math",":","`","y","=","g","*","x","+","(","1","-","g",")","*","f","(","A","(","x",")",")","`",",","where",":","math",":","`","A","`","is","a","linear","transformation",",",":","math",":","`","f","`","is","an","element-wise","non-linearity",",","and",":","math",":","`","g","`","is","an","element-wise","gate",",","computed","as",":","math",":","`","sigmoid","(","B","(","x",")",")","`",".","This","module","will","apply","a","fixed","number","of","highway","layers","to","its","input",",","returning","the","final","result",".","Parameters","--","--","--","--","--","input_dim",":","``","int","``","The","dimensionality","of",":","math",":","`","x","`",".","We","assume","the","input","has","shape","``","(","batch_size",",","...",",","input_dim",")","``",".","num_layers",":","``","int","``",",","optional","(","default=","``","1","``",")","The","number","of","highway","layers","to","apply","to","the","input",".","activation",":","``","Callable","[","[","torch.Tensor","]",",","torch.Tensor","]","``",",","optional","(","default=","``","torch.nn.functional.relu","``",")","The","non-linearity","to","use","in","the","highway","layers","."]
code: def __init__(
      self, input_dim, num_layers=1, activation=torch.nn.functional.relu
  ):
      super(Highway, self).__init__()
      self._input_dim = input_dim
      self._layers = torch.nn.ModuleList(
          [
              torch.nn.Linear(input_dim, input_dim * 2)
              for _ in range(num_layers)
          ]
      )
      self._activation = activation
      for layer in self._layers:
          # We should bias the highway layer to just carry its input forward.  We do that by
          # setting the bias on `B(x)` to be positive, because that means `g` will be biased to
          # be high, so we will carry the input forward.  The bias on `B(x)` is the second half
          # of the bias vector in each Linear layer.
          layer.bias[input_dim:].data.fill_(1)
  
imports: from __future__ import absolute_import
  import torch
label_desc: Use image node_id 1 to create a new Highway object from inherited base classes: torch.nn with example: obj = Highway(input_dim, num_layers, activation)