class_docstr: Implements stochastic gradient descent (optionally with momentum).
  
  Nesterov momentum is based on the formula from `On the importance of initialization and momentum in deep learning`__.
  
  Args:
      lr(float): learning rate
      momentum(float, optional): momentum factor(default: 0)
      weight_decay(float, optional): weight decay(L2 penalty)(default: 0)
      dampening(float, optional): dampening for momentum(default: 0)
      nesterov(bool, optional): enables Nesterov momentum(default: False)
  
  Typical usage example:
      >> > from singa import opt
      >> > optimizer = opt.SGD(lr=0.1, momentum=0.9)
      >> > optimizer.update()
  
  __ http: // www.cs.toronto.edu / %7Ehinton / absps / momentum.pdf
  
  .. note::
      The implementation of SGD with Momentum / Nesterov subtly differs from
      Sutskever et. al. and implementations in some other frameworks.
  
      Considering the specific case of Momentum, the update can be written as
  
      .. math::
                v = 
  ho * v + g \
                p = p - lr * v
  
      where p, g, v and: math: `
  ho` denote the parameters, gradient,
      velocity, and momentum respectively.
  
      This is in contrast to Sutskever et. al. and
      other frameworks which employ an update of the form
  
      .. math::
           v = 
  ho * v + lr * g \
           p = p - v
  
      The Nesterov version is analogously modified.
class_docstr_tok: ["Implements","stochastic","gradient","descent","(","optionally","with","momentum",")",".","Nesterov","momentum","is","based","on","the","formula","from","`","On","the","importance","of","initialization","and","momentum","in","deep","learning","`","__",".","Args",":","lr","(","float",")",":","learning","rate","momentum","(","float",",","optional",")",":","momentum","factor","(","default",":","0",")","weight_decay","(","float",",","optional",")",":","weight","decay","(","L2","penalty",")","(","default",":","0",")","dampening","(","float",",","optional",")",":","dampening","for","momentum","(","default",":","0",")","nesterov","(","bool",",","optional",")",":","enables","Nesterov","momentum","(","default",":","False",")","Typical","usage","example",":",">",">",">","from","singa","import","opt",">",">",">","optimizer","=","opt.SGD","(","lr=0.1",",","momentum=0.9",")",">",">",">","optimizer.update","(",")","__","http",":","\/\/","www.cs.toronto.edu","\/","%","7Ehinton","\/","absps","\/","momentum.pdf","..","note",":",":","The","implementation","of","SGD","with","Momentum","\/","Nesterov","subtly","differs","from","Sutskever","et",".","al",".","and","implementations","in","some","other","frameworks",".","Considering","the","specific","case","of","Momentum",",","the","update","can","be","written","as","..","math",":",":","v","=","ho","*","v","+","g","\\","p","=","p","-","lr","*","v","where","p",",","g",",","v","and",":","math",":","`","ho","`","denote","the","parameters",",","gradient",",","velocity",",","and","momentum","respectively",".","This","is","in","contrast","to","Sutskever","et",".","al",".","and","other","frameworks","which","employ","an","update","of","the","form","..","math",":",":","v","=","ho","*","v","+","lr","*","g","\\","p","=","p","-","v","The","Nesterov","version","is","analogously","modified","."]
code: def set_states(self, states):
      super().set_states(states)
      if "moments" in states:
          self.moments = states["moments"]
          self.mom_value = self.momentum(self.step_counter)
  
imports: from src.eva_engine.phase1.algo.alg_base import Evaluator
  from src.common.constant import Config
  from singa import singa_wrap
  from singa import device
  from singa import tensor
  from singa import opt
  from singa import autograd
  from singa.opt import Optimizer
  from singa.opt import DecayScheduler
  from singa.opt import Constant
  import numpy
  import time
  import argparse
  from PIL import Image
  from numpy import linalg
  import torch
label_desc: Use image node_id 5 for calling the MSSGD obj's underlying member method code with example usage: obj.set_states(states) without return types