class_docstr: Abstract base class for hyper parameter search with cross-validation.
class_docstr_tok: ["Abstract","base","class","for","hyper","parameter","search","with","cross-validation","."]
docstr: Repeatedly calls `evaluate_candidates` to conduct a search.
  
  This method, implemented in sub-classes, makes it possible to
  customize the scheduling of evaluations: GridSearchCV and
  RandomizedSearchCV schedule evaluations for their whole parameter
  search space at once but other more sequential approaches are also
  possible: for instance is possible to iteratively schedule evaluations
  for new regions of the parameter search space based on previously
  collected evaluation results. This makes it possible to implement
  Bayesian optimization or more generally sequential model-based
  optimization by deriving from the BaseSearchCV abstract base class.
  For example, Successive Halving is implemented by calling
  `evaluate_candidates` multiples times (once per iteration of the SH
  process), each time passing a different set of candidates with `X`
  and `y` of increasing sizes.
  
  Parameters
  ----------
  evaluate_candidates : callable
      This callback accepts:
          - a list of candidates, where each candidate is a dict of
            parameter settings.
          - an optional `cv` parameter which can be used to e.g.
            evaluate candidates on different dataset splits, or
            evaluate candidates on subsampled data (as done in the
            SucessiveHaling estimators). By default, the original `cv`
            parameter is used, and it is available as a private
            `_checked_cv_orig` attribute.
          - an optional `more_results` dict. Each key will be added to
            the `cv_results_` attribute. Values should be lists of
            length `n_candidates`
  
      It returns a dict of all results so far, formatted like
      ``cv_results_``.
  
      Important note (relevant whether the default cv is used or not):
      in randomized splitters, and unless the random_state parameter of
      cv was set to an int, calling cv.split() multiple times will
      yield different splits. Since cv.split() is called in
      evaluate_candidates, this means that candidates will be evaluated
      on different splits each time evaluate_candidates is called. This
      might be a methodological issue depending on the search strategy
      that you're implementing. To prevent randomized splitters from
      being used, you may use _split._yields_constant_splits()
  
  Examples
  --------
  
  ::
  
      def _run_search(self, evaluate_candidates):
          'Try C=0.1 only if C=1 is better than C=10'
          all_results = evaluate_candidates([{'C': 1}, {'C': 10}])
          score = all_results['mean_test_score']
          if score[0] < score[1]:
              evaluate_candidates([{'C': 0.1}])
docstr_tok: ["Repeatedly","calls","`","evaluate_candidates","`","to","conduct","a","search",".","This","method",",","implemented","in","sub-classes",",","makes","it","possible","to","customize","the","scheduling","of","evaluations",":","GridSearchCV","and","RandomizedSearchCV","schedule","evaluations","for","their","whole","parameter","search","space","at","once","but","other","more","sequential","approaches","are","also","possible",":","for","instance","is","possible","to","iteratively","schedule","evaluations","for","new","regions","of","the","parameter","search","space","based","on","previously","collected","evaluation","results",".","This","makes","it","possible","to","implement","Bayesian","optimization","or","more","generally","sequential","model-based","optimization","by","deriving","from","the","BaseSearchCV","abstract","base","class",".","For","example",",","Successive","Halving","is","implemented","by","calling","`","evaluate_candidates","`","multiples","times","(","once","per","iteration","of","the","SH","process",")",",","each","time","passing","a","different","set","of","candidates","with","`","X","`","and","`","y","`","of","increasing","sizes",".","Parameters","--","--","--","--","--","evaluate_candidates",":","callable","This","callback","accepts",":","-","a","list","of","candidates",",","where","each","candidate","is","a","dict","of","parameter","settings",".","-","an","optional","`","cv","`","parameter","which","can","be","used","to","e.g",".","evaluate","candidates","on","different","dataset","splits",",","or","evaluate","candidates","on","subsampled","data","(","as","done","in","the","SucessiveHaling","estimators",")",".","By","default",",","the","original","`","cv","`","parameter","is","used",",","and","it","is","available","as","a","private","`","_checked_cv_orig","`","attribute",".","-","an","optional","`","more_results","`","dict",".","Each","key","will","be","added","to","the","`","cv_results_","`","attribute",".","Values","should","be","lists","of","length","`","n_candidates","`","It","returns","a","dict","of","all","results","so","far",",","formatted","like","``","cv_results_","``",".","Important","note","(","relevant","whether","the","default","cv","is","used","or","not",")",":","in","randomized","splitters",",","and","unless","the","random_state","parameter","of","cv","was","set","to","an","int",",","calling","cv.split","(",")","multiple","times","will","yield","different","splits",".","Since","cv.split","(",")","is","called","in","evaluate_candidates",",","this","means","that","candidates","will","be","evaluated","on","different","splits","each","time","evaluate_candidates","is","called",".","This","might","be","a","methodological","issue","depending","on","the","search","strategy","that","you","'re","implementing",".","To","prevent","randomized","splitters","from","being","used",",","you","may","use","_split._yields_constant_splits","(",")","Examples","--","--","--","--",":",":","def","_run_search","(","self",",","evaluate_candidates",")",":","'Try","C=0.1","only","if","C=1","is","better","than","C=10'","all_results","=","evaluate_candidates","(","[","{","'","C","'",":","1","}",",","{","'","C","'",":","10","}","]",")","score","=","all_results","[","'mean_test_score","'","]","if","score","[","0","]","<","score","[","1","]",":","evaluate_candidates","(","[","{","'","C","'",":","0.1","}","]",")"]
code: def _run_search(self, evaluate_candidates):
      """Repeatedly calls `evaluate_candidates` to conduct a search.
  
      This method, implemented in sub-classes, makes it possible to
      customize the scheduling of evaluations: GridSearchCV and
      RandomizedSearchCV schedule evaluations for their whole parameter
      search space at once but other more sequential approaches are also
      possible: for instance is possible to iteratively schedule evaluations
      for new regions of the parameter search space based on previously
      collected evaluation results. This makes it possible to implement
      Bayesian optimization or more generally sequential model-based
      optimization by deriving from the BaseSearchCV abstract base class.
      For example, Successive Halving is implemented by calling
      `evaluate_candidates` multiples times (once per iteration of the SH
      process), each time passing a different set of candidates with `X`
      and `y` of increasing sizes.
  
      Parameters
      ----------
      evaluate_candidates : callable
          This callback accepts:
              - a list of candidates, where each candidate is a dict of
                parameter settings.
              - an optional `cv` parameter which can be used to e.g.
                evaluate candidates on different dataset splits, or
                evaluate candidates on subsampled data (as done in the
                SucessiveHaling estimators). By default, the original `cv`
                parameter is used, and it is available as a private
                `_checked_cv_orig` attribute.
              - an optional `more_results` dict. Each key will be added to
                the `cv_results_` attribute. Values should be lists of
                length `n_candidates`
  
          It returns a dict of all results so far, formatted like
          ``cv_results_``.
  
          Important note (relevant whether the default cv is used or not):
          in randomized splitters, and unless the random_state parameter of
          cv was set to an int, calling cv.split() multiple times will
          yield different splits. Since cv.split() is called in
          evaluate_candidates, this means that candidates will be evaluated
          on different splits each time evaluate_candidates is called. This
          might be a methodological issue depending on the search strategy
          that you're implementing. To prevent randomized splitters from
          being used, you may use _split._yields_constant_splits()
  
      Examples
      --------
  
      ::
  
          def _run_search(self, evaluate_candidates):
              'Try C=0.1 only if C=1 is better than C=10'
              all_results = evaluate_candidates([{'C': 1}, {'C': 10}])
              score = all_results['mean_test_score']
              if score[0] < score[1]:
                  evaluate_candidates([{'C': 0.1}])
      """
      raise NotImplementedError("_run_search not implemented.")
  
imports: import numbers
  import operator
  import time
  import warnings
  from abc import ABCMeta, abstractmethod
  from collections import defaultdict
  from collections.abc import Iterable, Mapping, Sequence
  from functools import partial, reduce
  from itertools import product
  import numpy
  from numpy.ma import MaskedArray
  from scipy.stats import rankdata
  from ..base import BaseEstimator, MetaEstimatorMixin, _fit_context, clone, is_classifier
  from ..exceptions import NotFittedError
  from ..metrics import check_scoring
  from ..metrics._scorer import _check_multimetric_scoring, get_scorer_names
  from ..utils import check_random_state
  from ..utils._param_validation import HasMethods, Interval, StrOptions
  from ..utils._tags import _safe_tags
  from ..utils.metaestimators import available_if
  from ..utils.parallel import Parallel, delayed
  from ..utils.random import sample_without_replacement
  from ..utils.validation import _check_fit_params, check_is_fitted, indexable
  from ._split import check_cv
  from ._validation import _aggregate_score_dicts, _fit_and_score, _insert_error_scores, _normalize_score_results, _warn_or_raise_about_fit_failures
label_desc: Use image node_id 14 for calling the BaseSearchCV obj's underlying member method code with example usage: obj._run_search(evaluate_candidates) without return types