class_docstr: Root Mean Square propagation.
  
  Root Mean Square (RMS) propagation protects against vanishing and
  exploding gradients. In RMSprop, the gradient is divided by a running
  average of recent gradients. Given the parameters :math:`\theta`, gradient :math:`\nabla J`,
  we keep a running average :math:`\mu` of the last :math:`1/\lambda` gradients squared.
  The update equations are then given by
  
  .. math::
  
      \mu' &= \lambda\mu + (1-\lambda)(\nabla J)^2
  
  .. math::
  
      \theta' &= \theta - \frac{\alpha}{\sqrt{\mu + \epsilon} + \epsilon}\nabla J
  
  where we use :math:`\epsilon` as a (small) smoothing factor to prevent from dividing by zero.
class_docstr_tok: ["Root","Mean","Square","propagation",".","Root","Mean","Square","(","RMS",")","propagation","protects","against","vanishing","and","exploding","gradients",".","In","RMSprop",",","the","gradient","is","divided","by","a","running","average","of","recent","gradients",".","Given","the","parameters",":","math",":","`","\\theta","`",",","gradient",":","math",":","`","\\nabla","J","`",",","we","keep","a","running","average",":","math",":","`","\\mu","`","of","the","last",":","math",":","`","1\/\\lambda","`","gradients","squared",".","The","update","equations","are","then","given","by","..","math",":",":","\\mu","'","&","=","\\lambda\\mu","+","(","1-\\lambda",")","(","\\nabla","J",")","^2","..","math",":",":","\\theta","'","&","=","\\theta","-","\\frac","{","\\alpha","}","{","\\sqrt","{","\\mu","+","\\epsilon","}","+","\\epsilon","}","\\nabla","J","where","we","use",":","math",":","`","\\epsilon","`","as","a","(","small",")","smoothing","factor","to","prevent","from","dividing","by","zero","."]
docstr: Apply the learning rule to all the layers and update the states.
  
  Arguments:
      layer_list (list): a list of Layer objects to optimize.
      epoch (int): the current epoch, needed for the Schedule object.
docstr_tok: ["Apply","the","learning","rule","to","all","the","layers","and","update","the","states",".","Arguments",":","layer_list","(","list",")",":","a","list","of","Layer","objects","to","optimize",".","epoch","(","int",")",":","the","current","epoch",",","needed","for","the","Schedule","object","."]
code: def optimize(self, layer_list, epoch):
      """
      Apply the learning rule to all the layers and update the states.
  
      Arguments:
          layer_list (list): a list of Layer objects to optimize.
          epoch (int): the current epoch, needed for the Schedule object.
      """
      lrate = self.schedule.get_learning_rate(self.learning_rate, epoch)
      epsilon, decay = (self.epsilon, self.decay_rate)
      param_list = get_param_list(layer_list)
  
      scale_factor = self.clip_gradient_norm(
          param_list, self.gradient_clip_norm
      )
  
      for (param, grad), states in param_list:
          param.rounding = self.stochastic_round
          if len(states) == 0:
              states.append(self.be.zeros_like(grad))
  
          grad = grad / self.be.bsz
          grad = self.clip_value(grad, self.gradient_clip_value)
  
          # update state
          state = states[0]
          state[:] = decay * state + self.be.square(grad) * (
              1.0 - decay
          )
  
          param[:] = self.clip_value(
              param
              - (scale_factor * grad * lrate)
              / (self.be.sqrt(state + epsilon) + epsilon),
              self.param_clip_value,
          )
  
imports: from __future__ import division
  from neon import NervanaObject
  from neon.util.persist import load_class
  import logging
  import numpy
label_desc: Use image node_id 2 for calling the RMSProp obj's underlying member method code with example usage: obj.optimize(layer_list, epoch) without return types