class_docstr: Run bayesian optimisation loop.
  
  An `Optimizer` represents the steps of a bayesian optimisation loop. To
  use it you need to provide your own loop mechanism. The various
  optimisers provided by `skopt` use this class under the hood.
  
  Use this class directly if you want to control the iterations of your
  bayesian optimisation loop.
  
  Parameters
  ----------
  dimensions : list, shape (n_dims,)
      List of search space dimensions.
      Each search dimension can be defined either as
  
      - a `(lower_bound, upper_bound)` tuple (for `Real` or `Integer`
        dimensions),
      - a `(lower_bound, upper_bound, "prior")` tuple (for `Real`
        dimensions),
      - as a list of categories (for `Categorical` dimensions), or
      - an instance of a `Dimension` object (`Real`, `Integer` or
        `Categorical`).
  
  base_estimator : `"GP"`, `"RF"`, `"ET"`, `"GBRT"` or sklearn regressor,             default: `"GP"`
      Should inherit from :obj:`sklearn.base.RegressorMixin`.
      In addition the `predict` method, should have an optional `return_std`
      argument, which returns `std(Y | x)` along with `E[Y | x]`.
      If base_estimator is one of ["GP", "RF", "ET", "GBRT"], a default
      surrogate model of the corresponding type is used corresponding to what
      is used in the minimize functions.
  
  n_random_starts : int, default: 10
      .. deprecated:: 0.6
          use `n_initial_points` instead.
  
  n_initial_points : int, default: 10
      Number of evaluations of `func` with initialization points
      before approximating it with `base_estimator`. Initial point
      generator can be changed by setting `initial_point_generator`.
  
  initial_point_generator : str, InitialPointGenerator instance,             default: `"random"`
      Sets a initial points generator. Can be either
  
      - `"random"` for uniform random numbers,
      - `"sobol"` for a Sobol' sequence,
      - `"halton"` for a Halton sequence,
      - `"hammersly"` for a Hammersly sequence,
      - `"lhs"` for a latin hypercube sequence,
      - `"grid"` for a uniform grid sequence
  
  acq_func : string, default: `"gp_hedge"`
      Function to minimize over the posterior distribution. Can be either
  
      - `"LCB"` for lower confidence bound.
      - `"EI"` for negative expected improvement.
      - `"PI"` for negative probability of improvement.
      - `"gp_hedge"` Probabilistically choose one of the above three
        acquisition functions at every iteration.
  
        - The gains `g_i` are initialized to zero.
        - At every iteration,
  
          - Each acquisition function is optimised independently to
            propose an candidate point `X_i`.
          - Out of all these candidate points, the next point `X_best` is
            chosen by :math:`softmax(\eta g_i)`
          - After fitting the surrogate model with `(X_best, y_best)`,
            the gains are updated such that :math:`g_i -= \mu(X_i)`
  
      - `"EIps"` for negated expected improvement per second to take into
        account the function compute time. Then, the objective function is
        assumed to return two values, the first being the objective value and
        the second being the time taken in seconds.
      - `"PIps"` for negated probability of improvement per second. The
        return type of the objective function is assumed to be similar to
        that of `"EIps"`
  
  acq_optimizer : string, `"sampling"` or `"lbfgs"`, default: `"auto"`
      Method to minimize the acquisition function. The fit model
      is updated with the optimal value obtained by optimizing `acq_func`
      with `acq_optimizer`.
  
      - If set to `"auto"`, then `acq_optimizer` is configured on the
        basis of the base_estimator and the space searched over.
        If the space is Categorical or if the estimator provided based on
        tree-models then this is set to be `"sampling"`.
      - If set to `"sampling"`, then `acq_func` is optimized by computing
        `acq_func` at `n_points` randomly sampled points.
      - If set to `"lbfgs"`, then `acq_func` is optimized by
  
        - Sampling `n_restarts_optimizer` points randomly.
        - `"lbfgs"` is run for 20 iterations with these points as initial
          points to find local minima.
        - The optimal of these local minima is used to update the prior.
  
  random_state : int, RandomState instance, or None (default)
      Set random state to something other than None for reproducible
      results.
  
  n_jobs : int, default: 1
      The number of jobs to run in parallel in the base_estimator,
      if the base_estimator supports n_jobs as parameter and
      base_estimator was given as string.
      If -1, then the number of jobs is set to the number of cores.
  
  acq_func_kwargs : dict
      Additional arguments to be passed to the acquisition function.
  
  acq_optimizer_kwargs : dict
      Additional arguments to be passed to the acquisition optimizer.
  
  model_queue_size : int or None, default: None
      Keeps list of models only as long as the argument given. In the
      case of None, the list has no capped length.
  
  Attributes
  ----------
  Xi : list
      Points at which objective has been evaluated.
  yi : scalar
      Values of objective at corresponding points in `Xi`.
  models : list
      Regression models used to fit observations and compute acquisition
      function.
  space : Space
      An instance of :class:`skopt.space.Space`. Stores parameter search
      space used to sample points, bounds, and type of parameters.
class_docstr_tok: ["Run","bayesian","optimisation","loop",".","An","`","Optimizer","`","represents","the","steps","of","a","bayesian","optimisation","loop",".","To","use","it","you","need","to","provide","your","own","loop","mechanism",".","The","various","optimisers","provided","by","`","skopt","`","use","this","class","under","the","hood",".","Use","this","class","directly","if","you","want","to","control","the","iterations","of","your","bayesian","optimisation","loop",".","Parameters","--","--","--","--","--","dimensions",":","list",",","shape","(","n_dims",",",")","List","of","search","space","dimensions",".","Each","search","dimension","can","be","defined","either","as","-","a","`","(","lower_bound",",","upper_bound",")","`","tuple","(","for","`","Real","`","or","`","Integer","`","dimensions",")",",","-","a","`","(","lower_bound",",","upper_bound",",","``","prior","''",")","`","tuple","(","for","`","Real","`","dimensions",")",",","-","as","a","list","of","categories","(","for","`","Categorical","`","dimensions",")",",","or","-","an","instance","of","a","`","Dimension","`","object","(","`","Real","`",",","`","Integer","`","or","`","Categorical","`",")",".","base_estimator",":","`","``","GP","''","`",",","`","``","RF","''","`",",","`","``","ET","''","`",",","`","``","GBRT","''","`","or","sklearn","regressor",",","default",":","`","``","GP","''","`","Should","inherit","from",":","obj",":","`","sklearn.base.RegressorMixin","`",".","In","addition","the","`","predict","`","method",",","should","have","an","optional","`","return_std","`","argument",",","which","returns","`","std","(","Y","|","x",")","`","along","with","`","E","[","Y","|","x","]","`",".","If","base_estimator","is","one","of","[","``","GP","''",",","``","RF","''",",","``","ET","''",",","``","GBRT","''","]",",","a","default","surrogate","model","of","the","corresponding","type","is","used","corresponding","to","what","is","used","in","the","minimize","functions",".","n_random_starts",":","int",",","default",":","10","..","deprecated",":",":","0.6","use","`","n_initial_points","`","instead",".","n_initial_points",":","int",",","default",":","10","Number","of","evaluations","of","`","func","`","with","initialization","points","before","approximating","it","with","`","base_estimator","`",".","Initial","point","generator","can","be","changed","by","setting","`","initial_point_generator","`",".","initial_point_generator",":","str",",","InitialPointGenerator","instance",",","default",":","`","``","random","''","`","Sets","a","initial","points","generator",".","Can","be","either","-","`","``","random","''","`","for","uniform","random","numbers",",","-","`","``","sobol","''","`","for","a","Sobol","'","sequence",",","-","`","``","halton","''","`","for","a","Halton","sequence",",","-","`","``","hammersly","''","`","for","a","Hammersly","sequence",",","-","`","``","lhs","''","`","for","a","latin","hypercube","sequence",",","-","`","``","grid","''","`","for","a","uniform","grid","sequence","acq_func",":","string",",","default",":","`","``","gp_hedge","''","`","Function","to","minimize","over","the","posterior","distribution",".","Can","be","either","-","`","``","LCB","''","`","for","lower","confidence","bound",".","-","`","``","EI","''","`","for","negative","expected","improvement",".","-","`","``","PI","''","`","for","negative","probability","of","improvement",".","-","`","``","gp_hedge","''","`","Probabilistically","choose","one","of","the","above","three","acquisition","functions","at","every","iteration",".","-","The","gains","`","g_i","`","are","initialized","to","zero",".","-","At","every","iteration",",","-","Each","acquisition","function","is","optimised","independently","to","propose","an","candidate","point","`","X_i","`",".","-","Out","of","all","these","candidate","points",",","the","next","point","`","X_best","`","is","chosen","by",":","math",":","`","softmax","(","\\eta","g_i",")","`","-","After","fitting","the","surrogate","model","with","`","(","X_best",",","y_best",")","`",",","the","gains","are","updated","such","that",":","math",":","`","g_i","-=","\\mu","(","X_i",")","`","-","`","``","EIps","''","`","for","negated","expected","improvement","per","second","to","take","into","account","the","function","compute","time",".","Then",",","the","objective","function","is","assumed","to","return","two","values",",","the","first","being","the","objective","value","and","the","second","being","the","time","taken","in","seconds",".","-","`","``","PIps","''","`","for","negated","probability","of","improvement","per","second",".","The","return","type","of","the","objective","function","is","assumed","to","be","similar","to","that","of","`","``","EIps","''","`","acq_optimizer",":","string",",","`","``","sampling","''","`","or","`","``","lbfgs","''","`",",","default",":","`","``","auto","''","`","Method","to","minimize","the","acquisition","function",".","The","fit","model","is","updated","with","the","optimal","value","obtained","by","optimizing","`","acq_func","`","with","`","acq_optimizer","`",".","-","If","set","to","`","``","auto","''","`",",","then","`","acq_optimizer","`","is","configured","on","the","basis","of","the","base_estimator","and","the","space","searched","over",".","If","the","space","is","Categorical","or","if","the","estimator","provided","based","on","tree-models","then","this","is","set","to","be","`","``","sampling","''","`",".","-","If","set","to","`","``","sampling","''","`",",","then","`","acq_func","`","is","optimized","by","computing","`","acq_func","`","at","`","n_points","`","randomly","sampled","points",".","-","If","set","to","`","``","lbfgs","''","`",",","then","`","acq_func","`","is","optimized","by","-","Sampling","`","n_restarts_optimizer","`","points","randomly",".","-","`","``","lbfgs","''","`","is","run","for","20","iterations","with","these","points","as","initial","points","to","find","local","minima",".","-","The","optimal","of","these","local","minima","is","used","to","update","the","prior",".","random_state",":","int",",","RandomState","instance",",","or","None","(","default",")","Set","random","state","to","something","other","than","None","for","reproducible","results",".","n_jobs",":","int",",","default",":","1","The","number","of","jobs","to","run","in","parallel","in","the","base_estimator",",","if","the","base_estimator","supports","n_jobs","as","parameter","and","base_estimator","was","given","as","string",".","If","-1",",","then","the","number","of","jobs","is","set","to","the","number","of","cores",".","acq_func_kwargs",":","dict","Additional","arguments","to","be","passed","to","the","acquisition","function",".","acq_optimizer_kwargs",":","dict","Additional","arguments","to","be","passed","to","the","acquisition","optimizer",".","model_queue_size",":","int","or","None",",","default",":","None","Keeps","list","of","models","only","as","long","as","the","argument","given",".","In","the","case","of","None",",","the","list","has","no","capped","length",".","Attributes","--","--","--","--","--","Xi",":","list","Points","at","which","objective","has","been","evaluated",".","yi",":","scalar","Values","of","objective","at","corresponding","points","in","`","Xi","`",".","models",":","list","Regression","models","used","to","fit","observations","and","compute","acquisition","function",".","space",":","Space","An","instance","of",":","class",":","`","skopt.space.Space","`",".","Stores","parameter","search","space","used","to","sample","points",",","bounds",",","and","type","of","parameters","."]
docstr: Suggest next point at which to evaluate the objective.
  
  Return a random point while not at least `n_initial_points`
  observations have been `tell`ed, after that `base_estimator` is used
  to determine the next point.
docstr_tok: ["Suggest","next","point","at","which","to","evaluate","the","objective",".","Return","a","random","point","while","not","at","least","`","n_initial_points","`","observations","have","been","`","tell","`","ed",",","after","that","`","base_estimator","`","is","used","to","determine","the","next","point","."]
code: def _ask(self):
      """Suggest next point at which to evaluate the objective.
  
      Return a random point while not at least `n_initial_points`
      observations have been `tell`ed, after that `base_estimator` is used
      to determine the next point.
      """
      if self._n_initial_points > 0 or self.base_estimator_ is None:
          # this will not make a copy of `self.rng` and hence keep advancing
          # our random state.
          if self._initial_samples is None:
              return self.space.rvs(random_state=self.rng)[0]
          else:
              # The samples are evaluated starting form initial_samples[0]
              return self._initial_samples[
                  len(self._initial_samples) - self._n_initial_points
              ]
  
      else:
          if not self.models:
              raise RuntimeError(
                  "Random evaluations exhausted and no "
                  "model has been fit."
              )
  
          next_x = self._next_x
          min_delta_x = min(
              [self.space.distance(next_x, xi) for xi in self.Xi]
          )
          if abs(min_delta_x) <= 1e-8:
              warnings.warn(
                  "The objective has been evaluated "
                  "at this point before."
              )
  
          # return point computed from last call to tell()
          return next_x
  
imports: import sys
  import warnings
  from math import log
  from numbers import Number
  import numpy
  from scipy.optimize import fmin_l_bfgs_b
  from sklearn.base import clone
  from sklearn.base import is_regressor
  from joblib import Parallel, delayed
  from sklearn.multioutput import MultiOutputRegressor
  from sklearn.utils import check_random_state
  from ..acquisition import _gaussian_acquisition
  from ..acquisition import gaussian_acquisition_1D
  from ..learning import GaussianProcessRegressor
  from ..space import Categorical
  from ..space import Space
  from ..utils import check_x_in_space
  from ..utils import cook_estimator
  from ..utils import create_result
  from ..utils import has_gradients
  from ..utils import is_listlike
  from ..utils import is_2Dlistlike
  from ..utils import normalize_dimensions
  from ..utils import cook_initial_point_generator
label_desc: Use image node_id 4 for calling the Optimizer obj's underlying member method code with example usage: obj._ask() and returns: next_x, self, self