code: def expected_calibration_error_evaluator(
      test_data: pd.DataFrame,
      prediction_column: str = "prediction",
      target_column: str = "target",
      eval_name: str = None,
      n_bins: int = 100,
      bin_choice: str = "count",
  ) -> EvalReturnType:
      """
      Computes the expected calibration error (ECE), given true label and prediction scores.
      See "On Calibration of Modern Neural Networks"(https://arxiv.org/abs/1706.04599) for more information.
  
      The ECE is the distance between the actuals observed frequency and the predicted probabilities,
      for a given choice of bins.
  
      Perfect calibration results in a score of 0.
  
      For example, if for the bin [0, 0.1] we have the three data points:
        1. prediction: 0.1, actual: 0
        2. prediction: 0.05, actual: 1
        3. prediction: 0.0, actual 0
  
      Then the predicted average is (0.1 + 0.05 + 0.00)/3 = 0.05, and the empirical frequency is (0 + 1 + 0)/3 = 1/3.
      Therefore, the distance for this bin is::
  
          |1/3 - 0.05| ~= 0.28.
  
      Graphical intuition::
  
          Actuals (empirical frequency between 0 and 1)
          |     *
          |   *
          | *
           ______ Predictions (probabilties between 0 and 1)
  
      Parameters
      ----------
      test_data : Pandas' DataFrame
          A Pandas' DataFrame with target and prediction scores.
  
      prediction_column : Strings
          The name of the column in `test_data` with the prediction scores.
  
      target_column : String
          The name of the column in `test_data` with the binary target.
  
      eval_name : String, optional (default=None)
          The name of the evaluator as it will appear in the logs.
  
      n_bins: Int (default=100)
          The number of bins.
          This is a trade-off between the number of points in each bin and the probability range they span.
          You want a small enough range that still contains a significant number of points for the distance to work.
  
      bin_choice: String (default="count")
          Two possibilities:
          "count" for equally populated bins (e.g. uses `pandas.qcut` for the bins)
          "prob" for equally spaced probabilities (e.g. uses `pandas.cut` for the bins),
          with distance weighed by the number of samples in each bin.
  
      Returns
      -------
      log: dict
         A log-like dictionary with the expected calibration error.
      """
  
      if eval_name is None:
          eval_name = (
              "expected_calibration_error_evaluator__" + target_column
          )
  
      if bin_choice == "count":
          bins = pd.qcut(test_data[prediction_column], q=n_bins)
      elif bin_choice == "prob":
          bins = pd.cut(test_data[prediction_column], bins=n_bins)
      else:
          raise AttributeError("Invalid bin_choice")
  
      metric_df = pd.DataFrame(
          {
              "bins": bins,
              "predictions": test_data[prediction_column],
              "actuals": test_data[target_column],
          }
      )
  
      agg_df = metric_df.groupby("bins").agg(
          {"bins": "count", "predictions": "mean", "actuals": "mean"}
      )
  
      sample_weight = None
      if bin_choice == "prob":
          sample_weight = agg_df["bins"].values
  
      distance = mean_absolute_error(
          agg_df["actuals"].values,
          agg_df["predictions"].values,
          sample_weight=sample_weight,
      )
  
      return {eval_name: distance}
  
imports: import warnings
  from typing import Any, Callable, Iterable, List
  import numpy
  import pandas
  import toolz
  from pandas.util import hash_pandas_object
  from sklearn.metrics import average_precision_score, brier_score_loss, fbeta_score, log_loss, mean_absolute_error, mean_squared_error, precision_score, r2_score, recall_score, roc_auc_score
  from toolz import curry, last, first
  from scipy import optimize
  from sklearn.linear_model import LogisticRegression
  from fklearn.types import EvalFnType, EvalReturnType, PredictFnType, UncurriedEvalFnType
label_desc: Use image node_id 10 for calling a global function with example usage: expected_calibration_error_evaluator(test_data, prediction_column, target_column, eval_name, n_bins, bin_choice) and returns: dict