class_docstr: Implements AddSign algorithm.
  
  It has been proposed in `Neural Optimizer Search with Reinforcement Learning`_.
      
  :param params: (iterable): iterable of parameters to optimize or dicts defining
      parameter groups
  :param lr: (float, optional): learning rate (default: 1e-3)
  :param beta: (float, optional): coefficients used for computing
      running averages of gradient (default: 0.9)
  :param alpha: (float, optional): term added to
      the internal_decay * sign(g) * sign(m) (default: 1)
  :param sign_internal_decay: (callable, optional): a function that returns
      an internal decay calculated based on the current training step and
      the total number of training steps.
      If None, the internal decay is assumed to be 1.
      
  .. _Neural Optimizer Search with Reinforcement Learning:
      https://arxiv.org/abs/1709.07417
class_docstr_tok: ["Implements","AddSign","algorithm",".","It","has","been","proposed","in","`","Neural","Optimizer","Search","with","Reinforcement","Learning","`","_",".",":","param","params",":","(","iterable",")",":","iterable","of","parameters","to","optimize","or","dicts","defining","parameter","groups",":","param","lr",":","(","float",",","optional",")",":","learning","rate","(","default",":","1e-3",")",":","param","beta",":","(","float",",","optional",")",":","coefficients","used","for","computing","running","averages","of","gradient","(","default",":","0.9",")",":","param","alpha",":","(","float",",","optional",")",":","term","added","to","the","internal_decay","*","sign","(","g",")","*","sign","(","m",")","(","default",":","1",")",":","param","sign_internal_decay",":","(","callable",",","optional",")",":","a","function","that","returns","an","internal","decay","calculated","based","on","the","current","training","step","and","the","total","number","of","training","steps",".","If","None",",","the","internal","decay","is","assumed","to","be","1",".","..","_Neural","Optimizer","Search","with","Reinforcement","Learning",":","https",":","\/\/arxiv.org\/abs\/1709.07417"]
code: def __init__(
      self, params, lr=1e-3, beta=0.9, alpha=1, sign_internal_decay=None
  ):
      if sign_internal_decay is not None and not callable(
          sign_internal_decay
      ):
          raise TypeError(
              "{} is not a callable".format(
                  type(sign_internal_decay).__name__
              )
          )
      defaults = dict(
          lr=lr,
          beta=beta,
          alpha=alpha,
          sign_internal_decay=sign_internal_decay
          if sign_internal_decay is not None
          else lambda _: 1,
      )
      super(AddSign, self).__init__(params, defaults)
  
imports: import torch
  from torch.optim.optimizer import Optimizer
label_desc: Use image node_id 1 to create a new AddSign object from inherited base classes: Optimizer with example: obj = AddSign(params, lr, beta, alpha, sign_internal_decay)