class_docstr: Root Mean Square propagation.
  
  Root Mean Square (RMS) propagation protects against vanishing and
  exploding gradients. In RMSprop, the gradient is divided by a running
  average of recent gradients. Given the parameters :math:`\theta`, gradient :math:`\nabla J`,
  we keep a running average :math:`\mu` of the last :math:`1/\lambda` gradients squared.
  The update equations are then given by
  
  .. math::
  
      \mu' &= \lambda\mu + (1-\lambda)(\nabla J)^2
  
  .. math::
  
      \theta' &= \theta - \frac{\alpha}{\sqrt{\mu + \epsilon} + \epsilon}\nabla J
  
  where we use :math:`\epsilon` as a (small) smoothing factor to prevent from dividing by zero.
class_docstr_tok: ["Root","Mean","Square","propagation",".","Root","Mean","Square","(","RMS",")","propagation","protects","against","vanishing","and","exploding","gradients",".","In","RMSprop",",","the","gradient","is","divided","by","a","running","average","of","recent","gradients",".","Given","the","parameters",":","math",":","`","\\theta","`",",","gradient",":","math",":","`","\\nabla","J","`",",","we","keep","a","running","average",":","math",":","`","\\mu","`","of","the","last",":","math",":","`","1\/\\lambda","`","gradients","squared",".","The","update","equations","are","then","given","by","..","math",":",":","\\mu","'","&","=","\\lambda\\mu","+","(","1-\\lambda",")","(","\\nabla","J",")","^2","..","math",":",":","\\theta","'","&","=","\\theta","-","\\frac","{","\\alpha","}","{","\\sqrt","{","\\mu","+","\\epsilon","}","+","\\epsilon","}","\\nabla","J","where","we","use",":","math",":","`","\\epsilon","`","as","a","(","small",")","smoothing","factor","to","prevent","from","dividing","by","zero","."]
docstr: Class constructor.
  
  Arguments:
      stochastic_round (bool): Set this to True for stochastic rounding.
                               If False rounding will be to nearest.
                               If True will perform stochastic rounding using default width.
                               Only affects the gpu backend.
      decay_rate (float): decay rate of states
      learning_rate (float): the multiplication coefficent of updates
      epsilon (float): smoothing epsilon to avoid divide by zeros
      gradient_clip_norm (float, optional): Target gradient norm.
                                            Defaults to None.
      gradient_clip_value (float, optional): Value to element-wise clip
                                             gradients.
                                             Defaults to None.
      param_clip_value (float, optional): Value to element-wise clip
                                          parameters.
                                          Defaults to None.
      schedule (neon.optimizers.optimizer.Schedule, optional): Learning rate schedule.
                                                               Defaults to a constant.
  Notes:
      Only constant learning rate is supported currently.
docstr_tok: ["Class","constructor",".","Arguments",":","stochastic_round","(","bool",")",":","Set","this","to","True","for","stochastic","rounding",".","If","False","rounding","will","be","to","nearest",".","If","True","will","perform","stochastic","rounding","using","default","width",".","Only","affects","the","gpu","backend",".","decay_rate","(","float",")",":","decay","rate","of","states","learning_rate","(","float",")",":","the","multiplication","coefficent","of","updates","epsilon","(","float",")",":","smoothing","epsilon","to","avoid","divide","by","zeros","gradient_clip_norm","(","float",",","optional",")",":","Target","gradient","norm",".","Defaults","to","None",".","gradient_clip_value","(","float",",","optional",")",":","Value","to","element-wise","clip","gradients",".","Defaults","to","None",".","param_clip_value","(","float",",","optional",")",":","Value","to","element-wise","clip","parameters",".","Defaults","to","None",".","schedule","(","neon.optimizers.optimizer.Schedule",",","optional",")",":","Learning","rate","schedule",".","Defaults","to","a","constant",".","Notes",":","Only","constant","learning","rate","is","supported","currently","."]
code: def __init__(
      self,
      stochastic_round=False,
      decay_rate=0.95,
      learning_rate=2e-3,
      epsilon=1e-6,
      gradient_clip_norm=None,
      gradient_clip_value=None,
      param_clip_value=None,
      name=None,
      schedule=Schedule(),
  ):
      """
      Class constructor.
  
      Arguments:
          stochastic_round (bool): Set this to True for stochastic rounding.
                                   If False rounding will be to nearest.
                                   If True will perform stochastic rounding using default width.
                                   Only affects the gpu backend.
          decay_rate (float): decay rate of states
          learning_rate (float): the multiplication coefficent of updates
          epsilon (float): smoothing epsilon to avoid divide by zeros
          gradient_clip_norm (float, optional): Target gradient norm.
                                                Defaults to None.
          gradient_clip_value (float, optional): Value to element-wise clip
                                                 gradients.
                                                 Defaults to None.
          param_clip_value (float, optional): Value to element-wise clip
                                              parameters.
                                              Defaults to None.
          schedule (neon.optimizers.optimizer.Schedule, optional): Learning rate schedule.
                                                                   Defaults to a constant.
      Notes:
          Only constant learning rate is supported currently.
      """
      super(RMSProp, self).__init__(name=name)
      self.state_list = None
  
      self.epsilon = epsilon
      self.decay_rate = decay_rate
      self.learning_rate = learning_rate
      self.schedule = schedule
      self.gradient_clip_norm = gradient_clip_norm
      self.gradient_clip_value = gradient_clip_value
      self.param_clip_value = param_clip_value
      self.stochastic_round = stochastic_round
  
imports: from __future__ import division
  from neon import NervanaObject
  from neon.util.persist import load_class
  import logging
  import numpy
label_desc: Use image node_id 1 to create a new RMSProp object from inherited base classes: Optimizer with example: obj = RMSProp(stochastic_round, decay_rate, learning_rate, epsilon, gradient_clip_norm, gradient_clip_value, param_clip_value, name, schedule)