class_docstr: Implements AddSign algorithm.
  
  It has been proposed in `Neural Optimizer Search with Reinforcement Learning`_.
      
  :param params: (iterable): iterable of parameters to optimize or dicts defining
      parameter groups
  :param lr: (float, optional): learning rate (default: 1e-3)
  :param beta: (float, optional): coefficients used for computing
      running averages of gradient (default: 0.9)
  :param alpha: (float, optional): term added to
      the internal_decay * sign(g) * sign(m) (default: 1)
  :param sign_internal_decay: (callable, optional): a function that returns
      an internal decay calculated based on the current training step and
      the total number of training steps.
      If None, the internal decay is assumed to be 1.
      
  .. _Neural Optimizer Search with Reinforcement Learning:
      https://arxiv.org/abs/1709.07417
class_docstr_tok: ["Implements","AddSign","algorithm",".","It","has","been","proposed","in","`","Neural","Optimizer","Search","with","Reinforcement","Learning","`","_",".",":","param","params",":","(","iterable",")",":","iterable","of","parameters","to","optimize","or","dicts","defining","parameter","groups",":","param","lr",":","(","float",",","optional",")",":","learning","rate","(","default",":","1e-3",")",":","param","beta",":","(","float",",","optional",")",":","coefficients","used","for","computing","running","averages","of","gradient","(","default",":","0.9",")",":","param","alpha",":","(","float",",","optional",")",":","term","added","to","the","internal_decay","*","sign","(","g",")","*","sign","(","m",")","(","default",":","1",")",":","param","sign_internal_decay",":","(","callable",",","optional",")",":","a","function","that","returns","an","internal","decay","calculated","based","on","the","current","training","step","and","the","total","number","of","training","steps",".","If","None",",","the","internal","decay","is","assumed","to","be","1",".","..","_Neural","Optimizer","Search","with","Reinforcement","Learning",":","https",":","\/\/arxiv.org\/abs\/1709.07417"]
docstr: Performs a single optimization step.
  
  :param closure: (callable, optional): A closure that reevaluates the model
      and returns the loss.
docstr_tok: ["Performs","a","single","optimization","step",".",":","param","closure",":","(","callable",",","optional",")",":","A","closure","that","reevaluates","the","model","and","returns","the","loss","."]
code: def step(self, closure=None):
      """Performs a single optimization step.
  
      :param closure: (callable, optional): A closure that reevaluates the model
          and returns the loss.
      """
      loss = None
      if closure is not None:
          loss = closure()
  
      for group in self.param_groups:
          for p in group["params"]:
              if p.grad is None:
                  continue
              grad = p.grad.data
  
              state = self.state[p]
  
              # State initialization
              if len(state) == 0:
                  state["step"] = 0
                  # Exponential moving average of gradient values
                  state["exp_avg"] = torch.zeros_like(p.data)
  
              exp_avg = state["exp_avg"]
              beta = group["beta"]
              alpha = group["alpha"]
  
              state["step"] += 1
              internal_decay = group["sign_internal_decay"](
                  state["step"] - 1
              )
  
              # Decay the first moment running average coefficient
              exp_avg.mul_(beta).add_(1 - beta, grad)
              add_sign = grad.mul(
                  alpha + internal_decay * grad.sign() * exp_avg.sign()
              )
              p.data.add_(-group["lr"], add_sign)
  
      return loss
  
imports: import torch
  from torch.optim.optimizer import Optimizer
label_desc: Use image node_id 2 for calling the AddSign obj's underlying member method code with example usage: obj.step(closure) and returns: loss