class_docstr: Marginal distribution of a Gaussian process at finitely many points.
  
  A Gaussian process (GP) is an indexed collection of random variables, any
  finite collection of which are jointly Gaussian. While this definition applies
  to finite index sets, it is typically implicit that the index set is infinite;
  in applications, it is often some finite dimensional real or complex vector
  space. In such cases, the GP may be thought of as a distribution over
  (real- or complex-valued) functions defined over the index set.
  
  Just as Gaussian distributions are fully specified by their first and second
  moments, a Gaussian process can be completely specified by a mean and
  covariance function. Let `S` denote the index set and `K` the space in which
  each indexed random variable takes its values (again, often R or C). The mean
  function is then a map `m: S -> K`, and the covariance function, or kernel, is
  a positive-definite function `k: (S x S) -> K`. The properties of functions
  drawn from a GP are entirely dictated (up to translation) by the form of the
  kernel function.
  
  This `Distribution` represents the marginal joint distribution over function
  values at a given finite collection of points `[x[1], ..., x[N]]` from the
  index set `S`. By definition, this marginal distribution is just a
  multivariate normal distribution, whose mean is given by the vector
  `[ m(x[1]), ..., m(x[N]) ]` and whose covariance matrix is constructed from
  pairwise applications of the kernel function to the given inputs:
  
  ```none
      | k(x[1], x[1])    k(x[1], x[2])  ...  k(x[1], x[N]) |
      | k(x[2], x[1])    k(x[2], x[2])  ...  k(x[2], x[N]) |
      |      ...              ...                 ...      |
      | k(x[N], x[1])    k(x[N], x[2])  ...  k(x[N], x[N]) |
  ```
  
  For this to be a valid covariance matrix, it must be symmetric and positive
  definite; hence the requirement that `k` be a positive definite function
  (which, by definition, says that the above procedure will yield PD matrices).
  
  We also support the inclusion of zero-mean Gaussian noise in the model, via
  the `observation_noise_variance` parameter. This augments the generative model
  to
  
  ```none
  f ~ GP(m, k)
  (y[i] | f, x[i]) ~ Normal(f(x[i]), s)
  ```
  
  where
  
    * `m` is the mean function
    * `k` is the covariance kernel function
    * `f` is the function drawn from the GP
    * `x[i]` are the index points at which the function is observed
    * `y[i]` are the observed values at the index points
    * `s` is the scale of the observation noise.
  
  Note that this class represents an *unconditional* Gaussian process; it does
  not implement posterior inference conditional on observed function
  evaluations. This class is useful, for example, if one wishes to combine a GP
  prior with a non-conjugate likelihood using MCMC to sample from the posterior.
  
  #### Mathematical Details
  
  The probability density function (pdf) is a multivariate normal whose
  parameters are derived from the GP's properties:
  
  ```none
  pdf(x; index_points, mean_fn, kernel) = exp(-0.5 * y) / Z
  K = (kernel.matrix(index_points, index_points) +
       observation_noise_variance * eye(N))
  y = (x - mean_fn(index_points))^T @ K @ (x - mean_fn(index_points))
  Z = (2 * pi)**(.5 * N) |det(K)|**(.5)
  ```
  
  where:
  
  * `index_points` are points in the index set over which the GP is defined,
  * `mean_fn` is a callable mapping the index set to the GP's mean values,
  * `kernel` is `PositiveSemidefiniteKernel`-like and represents the covariance
    function of the GP,
  * `observation_noise_variance` represents (optional) observation noise.
  * `eye(N)` is an N-by-N identity matrix.
  
  #### Examples
  
  ##### Draw joint samples from a GP prior
  
  ```python
  import numpy as np
  import tensorflow.compat.v2 as tf
  import tensorflow_probability as tfp
  
  tfd = tfp.distributions
  psd_kernels = tfp.math.psd_kernels
  
  num_points = 100
  # Index points should be a collection (100, here) of feature vectors. In this
  # example, we're using 1-d vectors, so we just need to reshape the output from
  # np.linspace, to give a shape of (100, 1).
  index_points = np.expand_dims(np.linspace(-1., 1., num_points), -1)
  
  # Define a kernel with default parameters.
  kernel = psd_kernels.ExponentiatedQuadratic()
  
  gp = tfd.GaussianProcess(kernel, index_points)
  
  samples = gp.sample(10)
  # ==> 10 independently drawn, joint samples at `index_points`
  
  noisy_gp = tfd.GaussianProcess(
      kernel=kernel,
      index_points=index_points,
      observation_noise_variance=.05)
  noisy_samples = noisy_gp.sample(10)
  # ==> 10 independently drawn, noisy joint samples at `index_points`
  ```
  
  ##### Optimize kernel parameters via maximum marginal likelihood.
  
  ```python
  # Suppose we have some data from a known function. Note the index points in
  # general have shape `[b1, ..., bB, f1, ..., fF]` (here we assume `F == 1`),
  # so we need to explicitly consume the feature dimensions (just the last one
  # here).
  f = lambda x: np.sin(10*x[..., 0]) * np.exp(-x[..., 0]**2)
  observed_index_points = np.expand_dims(np.random.uniform(-1., 1., 50), -1)
  # Squeeze to take the shape from [50, 1] to [50].
  observed_values = f(observed_index_points)
  
  # Define a kernel with trainable parameters.
  kernel = psd_kernels.ExponentiatedQuadratic(
      amplitude=tf.Variable(1., dtype=np.float64, name='amplitude'),
      length_scale=tf.Variable(1., dtype=np.float64, name='length_scale'))
  
  gp = tfd.GaussianProcess(kernel, observed_index_points)
  
  optimizer = tf_keras.optimizers.Adam()
  
  @tf.function
  def optimize():
    with tf.GradientTape() as tape:
      loss = -gp.log_prob(observed_values)
    grads = tape.gradient(loss, gp.trainable_variables)
    optimizer.apply_gradients(zip(grads, gp.trainable_variables))
    return loss
  
  for i in range(1000):
    neg_log_likelihood = optimize()
    if i % 100 == 0:
      print("Step {}: NLL = {}".format(i, neg_log_likelihood))
  print("Final NLL = {}".format(neg_log_likelihood))
  ```
class_docstr_tok: ["Marginal","distribution","of","a","Gaussian","process","at","finitely","many","points",".","A","Gaussian","process","(","GP",")","is","an","indexed","collection","of","random","variables",",","any","finite","collection","of","which","are","jointly","Gaussian",".","While","this","definition","applies","to","finite","index","sets",",","it","is","typically","implicit","that","the","index","set","is","infinite",";","in","applications",",","it","is","often","some","finite","dimensional","real","or","complex","vector","space",".","In","such","cases",",","the","GP","may","be","thought","of","as","a","distribution","over","(","real-","or","complex-valued",")","functions","defined","over","the","index","set",".","Just","as","Gaussian","distributions","are","fully","specified","by","their","first","and","second","moments",",","a","Gaussian","process","can","be","completely","specified","by","a","mean","and","covariance","function",".","Let","`","S","`","denote","the","index","set","and","`","K","`","the","space","in","which","each","indexed","random","variable","takes","its","values","(","again",",","often","R","or","C",")",".","The","mean","function","is","then","a","map","`","m",":","S","-",">","K","`",",","and","the","covariance","function",",","or","kernel",",","is","a","positive-definite","function","`","k",":","(","S","x","S",")","-",">","K","`",".","The","properties","of","functions","drawn","from","a","GP","are","entirely","dictated","(","up","to","translation",")","by","the","form","of","the","kernel","function",".","This","`","Distribution","`","represents","the","marginal","joint","distribution","over","function","values","at","a","given","finite","collection","of","points","`","[","x","[","1","]",",","...",",","x","[","N","]","]","`","from","the","index","set","`","S","`",".","By","definition",",","this","marginal","distribution","is","just","a","multivariate","normal","distribution",",","whose","mean","is","given","by","the","vector","`","[","m","(","x","[","1","]",")",",","...",",","m","(","x","[","N","]",")","]","`","and","whose","covariance","matrix","is","constructed","from","pairwise","applications","of","the","kernel","function","to","the","given","inputs",":","``","`","none","|","k","(","x","[","1","]",",","x","[","1","]",")","k","(","x","[","1","]",",","x","[","2","]",")","...","k","(","x","[","1","]",",","x","[","N","]",")","|","|","k","(","x","[","2","]",",","x","[","1","]",")","k","(","x","[","2","]",",","x","[","2","]",")","...","k","(","x","[","2","]",",","x","[","N","]",")","|","|","...","...","...","|","|","k","(","x","[","N","]",",","x","[","1","]",")","k","(","x","[","N","]",",","x","[","2","]",")","...","k","(","x","[","N","]",",","x","[","N","]",")","|","``","`","For","this","to","be","a","valid","covariance","matrix",",","it","must","be","symmetric","and","positive","definite",";","hence","the","requirement","that","`","k","`","be","a","positive","definite","function","(","which",",","by","definition",",","says","that","the","above","procedure","will","yield","PD","matrices",")",".","We","also","support","the","inclusion","of","zero-mean","Gaussian","noise","in","the","model",",","via","the","`","observation_noise_variance","`","parameter",".","This","augments","the","generative","model","to","``","`","none","f","~","GP","(","m",",","k",")","(","y","[","i","]","|","f",",","x","[","i","]",")","~","Normal","(","f","(","x","[","i","]",")",",","s",")","``","`","where","*","`","m","`","is","the","mean","function","*","`","k","`","is","the","covariance","kernel","function","*","`","f","`","is","the","function","drawn","from","the","GP","*","`","x","[","i","]","`","are","the","index","points","at","which","the","function","is","observed","*","`","y","[","i","]","`","are","the","observed","values","at","the","index","points","*","`","s","`","is","the","scale","of","the","observation","noise",".","Note","that","this","class","represents","an","*","unconditional","*","Gaussian","process",";","it","does","not","implement","posterior","inference","conditional","on","observed","function","evaluations",".","This","class","is","useful",",","for","example",",","if","one","wishes","to","combine","a","GP","prior","with","a","non-conjugate","likelihood","using","MCMC","to","sample","from","the","posterior",".","#","#","#","#","Mathematical","Details","The","probability","density","function","(","pdf",")","is","a","multivariate","normal","whose","parameters","are","derived","from","the","GP","'s","properties",":","``","`","none","pdf","(","x",";","index_points",",","mean_fn",",","kernel",")","=","exp","(","-0.5","*","y",")","\/","Z","K","=","(","kernel.matrix","(","index_points",",","index_points",")","+","observation_noise_variance","*","eye","(","N",")",")","y","=","(","x","-","mean_fn","(","index_points",")",")","^T","@","K","@","(","x","-","mean_fn","(","index_points",")",")","Z","=","(","2","*","pi",")","*","*","(",".5","*","N",")","|det","(","K",")","|","*","*","(",".5",")","``","`","where",":","*","`","index_points","`","are","points","in","the","index","set","over","which","the","GP","is","defined",",","*","`","mean_fn","`","is","a","callable","mapping","the","index","set","to","the","GP","'s","mean","values",",","*","`","kernel","`","is","`","PositiveSemidefiniteKernel","`","-like","and","represents","the","covariance","function","of","the","GP",",","*","`","observation_noise_variance","`","represents","(","optional",")","observation","noise",".","*","`","eye","(","N",")","`","is","an","N-by-N","identity","matrix",".","#","#","#","#","Examples","#","#","#","#","#","Draw","joint","samples","from","a","GP","prior","``","`","python","import","numpy","as","np","import","tensorflow.compat.v2","as","tf","import","tensorflow_probability","as","tfp","tfd","=","tfp.distributions","psd_kernels","=","tfp.math.psd_kernels","num_points","=","100","#","Index","points","should","be","a","collection","(","100",",","here",")","of","feature","vectors",".","In","this","#","example",",","we","'re","using","1-d","vectors",",","so","we","just","need","to","reshape","the","output","from","#","np.linspace",",","to","give","a","shape","of","(","100",",","1",")",".","index_points","=","np.expand_dims","(","np.linspace","(","-1.",",","1.",",","num_points",")",",","-1",")","#","Define","a","kernel","with","default","parameters",".","kernel","=","psd_kernels.ExponentiatedQuadratic","(",")","gp","=","tfd.GaussianProcess","(","kernel",",","index_points",")","samples","=","gp.sample","(","10",")","#","==",">","10","independently","drawn",",","joint","samples","at","`","index_points","`","noisy_gp","=","tfd.GaussianProcess","(","kernel=kernel",",","index_points=index_points",",","observation_noise_variance=.05",")","noisy_samples","=","noisy_gp.sample","(","10",")","#","==",">","10","independently","drawn",",","noisy","joint","samples","at","`","index_points","`","``","`","#","#","#","#","#","Optimize","kernel","parameters","via","maximum","marginal","likelihood",".","``","`","python","#","Suppose","we","have","some","data","from","a","known","function",".","Note","the","index","points","in","#","general","have","shape","`","[","b1",",","...",",","bB",",","f1",",","...",",","fF","]","`","(","here","we","assume","`","F","==","1","`",")",",","#","so","we","need","to","explicitly","consume","the","feature","dimensions","(","just","the","last","one","#","here",")",".","f","=","lambda","x",":","np.sin","(","10","*","x","[","...",",","0","]",")","*","np.exp","(","-x","[","...",",","0","]","*","*","2",")","observed_index_points","=","np.expand_dims","(","np.random.uniform","(","-1.",",","1.",",","50",")",",","-1",")","#","Squeeze","to","take","the","shape","from","[","50",",","1","]","to","[","50","]",".","observed_values","=","f","(","observed_index_points",")","#","Define","a","kernel","with","trainable","parameters",".","kernel","=","psd_kernels.ExponentiatedQuadratic","(","amplitude=tf.Variable","(","1.",",","dtype=np.float64",",","name='amplitude","'",")",",","length_scale=tf.Variable","(","1.",",","dtype=np.float64",",","name='length_scale","'",")",")","gp","=","tfd.GaussianProcess","(","kernel",",","observed_index_points",")","optimizer","=","tf_keras.optimizers.Adam","(",")","@","tf.function","def","optimize","(",")",":","with","tf.GradientTape","(",")","as","tape",":","loss","=","-gp.log_prob","(","observed_values",")","grads","=","tape.gradient","(","loss",",","gp.trainable_variables",")","optimizer.apply_gradients","(","zip","(","grads",",","gp.trainable_variables",")",")","return","loss","for","i","in","range","(","1000",")",":","neg_log_likelihood","=","optimize","(",")","if","i","%","100","==","0",":","print","(","``","Step","{","}",":","NLL","=","{","}","''",".format","(","i",",","neg_log_likelihood",")",")","print","(","``","Final","NLL","=","{","}","''",".format","(","neg_log_likelihood",")",")","``","`"]
code: def index_points(self):
      return self._index_points
  
imports: import functools
  import numpy
  import tensorflow.compat.v2
  from tensorflow_probability.python.bijectors import identity
  from tensorflow_probability.python.bijectors import softplus
  from tensorflow_probability.python.distributions import cholesky_util
  from tensorflow_probability.python.distributions import distribution
  from tensorflow_probability.python.distributions import kullback_leibler
  from tensorflow_probability.python.distributions import mvn_linear_operator
  from tensorflow_probability.python.distributions.internal import stochastic_process_util
  from tensorflow_probability.python.internal import auto_composite_tensor
  from tensorflow_probability.python.internal import batch_shape_lib
  from tensorflow_probability.python.internal import distribution_util
  from tensorflow_probability.python.internal import dtype_util
  from tensorflow_probability.python.internal import nest_util
  from tensorflow_probability.python.internal import parameter_properties
  from tensorflow_probability.python.internal import prefer_static
  from tensorflow_probability.python.internal import reparameterization
  from tensorflow_probability.python.internal import tensor_util
  from tensorflow_probability.python.math import linalg
  from tensorflow.python.util import deprecation
  from tensorflow.python.util import nest
label_desc: Use image node_id 6 for calling the GaussianProcess obj's underlying member method code with example usage: obj.index_points() and returns: self