class_docstr: Implements a natural gradient descent optimizer for variational models
  that are based on a distribution q(u) = N(q_mu, q_sqrt q_sqrtᵀ) that is
  parameterized by mean q_mu and lower-triangular Cholesky factor q_sqrt
  of the covariance.
  
  Note that this optimizer does not implement the standard API of
  tf.optimizers.Optimizer. Its only public method is minimize(), which has
  a custom signature (var_list needs to be a list of (q_mu, q_sqrt) tuples,
  where q_mu and q_sqrt are gpflow.Parameter instances, not tf.Variable).
  
  Note furthermore that the natural gradients are implemented only for the
  full covariance case (i.e., q_diag=True is NOT supported).
  
  When using in your work, please cite :cite:t:`salimbeni18`.
class_docstr_tok: ["Implements","a","natural","gradient","descent","optimizer","for","variational","models","that","are","based","on","a","distribution","q","(","u",")","=","N","(","q_mu",",","q_sqrt","q_sqrt\u1d40",")","that","is","parameterized","by","mean","q_mu","and","lower-triangular","Cholesky","factor","q_sqrt","of","the","covariance",".","Note","that","this","optimizer","does","not","implement","the","standard","API","of","tf.optimizers.Optimizer",".","Its","only","public","method","is","minimize","(",")",",","which","has","a","custom","signature","(","var_list","needs","to","be","a","list","of","(","q_mu",",","q_sqrt",")","tuples",",","where","q_mu","and","q_sqrt","are","gpflow.Parameter","instances",",","not","tf.Variable",")",".","Note","furthermore","that","the","natural","gradients","are","implemented","only","for","the","full","covariance","case","(","i.e.",",","q_diag=True","is","NOT","supported",")",".","When","using","in","your","work",",","please","cite",":","cite",":","t",":","`","salimbeni18","`","."]
docstr: Computes gradients of loss_fn() w.r.t. q_mu and q_sqrt, and updates
  these parameters using the natgrad backwards step, for all sets of
  variational parameters passed in.
  
  :param loss_fn: Loss function.
  :param parameters: List of tuples (q_mu, q_sqrt, xi_transform)
docstr_tok: ["Computes","gradients","of","loss_fn","(",")","w.r.t",".","q_mu","and","q_sqrt",",","and","updates","these","parameters","using","the","natgrad","backwards","step",",","for","all","sets","of","variational","parameters","passed","in",".",":","param","loss_fn",":","Loss","function",".",":","param","parameters",":","List","of","tuples","(","q_mu",",","q_sqrt",",","xi_transform",")"]
code: def _natgrad_steps(
      self,
      loss_fn: LossClosure,
      parameters: Sequence[
          Tuple[Parameter, Parameter, Optional[XiTransform]]
      ],
  ) -> None:
      """
      Computes gradients of loss_fn() w.r.t. q_mu and q_sqrt, and updates
      these parameters using the natgrad backwards step, for all sets of
      variational parameters passed in.
  
      :param loss_fn: Loss function.
      :param parameters: List of tuples (q_mu, q_sqrt, xi_transform)
      """
      q_mus, q_sqrts, xis = zip(*parameters)
      q_mu_vars = [p.unconstrained_variable for p in q_mus]
      q_sqrt_vars = [p.unconstrained_variable for p in q_sqrts]
  
      with tf.GradientTape(watch_accessed_variables=False) as tape:
          tape.watch(q_mu_vars + q_sqrt_vars)
          loss = loss_fn()
  
      q_mu_grads, q_sqrt_grads = tape.gradient(
          loss, [q_mu_vars, q_sqrt_vars]
      )
      # NOTE that these are the gradients in *unconstrained* space
  
      with tf.name_scope(f"{self.natgrad_name}/natural_gradient_steps"):
          for q_mu_grad, q_sqrt_grad, q_mu, q_sqrt, xi_transform in zip(
              q_mu_grads, q_sqrt_grads, q_mus, q_sqrts, xis
          ):
              self._natgrad_apply_gradients(
                  q_mu_grad, q_sqrt_grad, q_mu, q_sqrt, xi_transform
              )
  
imports: import abc
  import functools
  from typing import Any, Callable, Dict, Optional, Sequence, Tuple, Union
  import tensorflow
  from check_shapes import check_shapes
  from ..base import AnyNDArray, Parameter, _to_constrained
label_desc: Use image node_id 3 for calling the NaturalGradient obj's underlying member method code with example usage: obj._natgrad_steps(loss_fn, parameters) without return types