class_docstr: Provide data silos for holdout evaluation.
  
  For this, we first combine the
  instances from all the sets or just some of the sets, then allow to create a different data silo
  instance with a train/test split.
  Calling DataSiloForHoldout.make() creates a list of holdout splits
class_docstr_tok: ["Provide","data","silos","for","holdout","evaluation",".","For","this",",","we","first","combine","the","instances","from","all","the","sets","or","just","some","of","the","sets",",","then","allow","to","create","a","different","data","silo","instance","with","a","train\/test","split",".","Calling","DataSiloForHoldout.make","(",")","creates","a","list","of","holdout","splits"]
docstr: Create number of folds data-silo-like objects which can be used for training from the
  original data silo passed on.
  
  NOTE: does not support question answering.
  
  :param datasilo: the data silo that contains the original data
  :type datasilo: DataSilo
  :param sets: which sets to use to create the xval folds (strings)
  :type sets: list
  :param n_splits: number of folds to create
  :type n_splits: int
  :param train_split: portion of all data to use for training, rest will be used for testing.
      The `dev_split` portion of the training set as defined in the processor of the original
      data silo is created from the training set portion, the dev set split is stratified if
      the stratification parameter is True.
  :type train_split: float
  :param random_state: random state for shuffling, either an integer for repeatable randomness or
      None to use whatever the current state is.
  :type random_state: int
  :param stratification: If class stratification should be done for the train/test and the train/dev splits.
  :type stratification: bool
docstr_tok: ["Create","number","of","folds","data-silo-like","objects","which","can","be","used","for","training","from","the","original","data","silo","passed","on",".","NOTE",":","does","not","support","question","answering",".",":","param","datasilo",":","the","data","silo","that","contains","the","original","data",":","type","datasilo",":","DataSilo",":","param","sets",":","which","sets","to","use","to","create","the","xval","folds","(","strings",")",":","type","sets",":","list",":","param","n_splits",":","number","of","folds","to","create",":","type","n_splits",":","int",":","param","train_split",":","portion","of","all","data","to","use","for","training",",","rest","will","be","used","for","testing",".","The","`","dev_split","`","portion","of","the","training","set","as","defined","in","the","processor","of","the","original","data","silo","is","created","from","the","training","set","portion",",","the","dev","set","split","is","stratified","if","the","stratification","parameter","is","True",".",":","type","train_split",":","float",":","param","random_state",":","random","state","for","shuffling",",","either","an","integer","for","repeatable","randomness","or","None","to","use","whatever","the","current","state","is",".",":","type","random_state",":","int",":","param","stratification",":","If","class","stratification","should","be","done","for","the","train\/test","and","the","train\/dev","splits",".",":","type","stratification",":","bool"]
code: def make(
      cls,
      datasilo,
      sets=["train", "dev", "test"],
      n_splits=5,
      train_split=0.80,
      random_state=None,
      stratification=True,
  ):
      """
      Create number of folds data-silo-like objects which can be used for training from the
      original data silo passed on.
  
      NOTE: does not support question answering.
  
      :param datasilo: the data silo that contains the original data
      :type datasilo: DataSilo
      :param sets: which sets to use to create the xval folds (strings)
      :type sets: list
      :param n_splits: number of folds to create
      :type n_splits: int
      :param train_split: portion of all data to use for training, rest will be used for testing.
          The `dev_split` portion of the training set as defined in the processor of the original
          data silo is created from the training set portion, the dev set split is stratified if
          the stratification parameter is True.
      :type train_split: float
      :param random_state: random state for shuffling, either an integer for repeatable randomness or
          None to use whatever the current state is.
      :type random_state: int
      :param stratification: If class stratification should be done for the train/test and the train/dev splits.
      :type stratification: bool
      """
      if "question_answering" in datasilo.processor.tasks:
          raise NotImplementedError(
              "Holdout evaluation not supported for question answering"
          )
      else:
          return cls._make(
              datasilo,
              sets,
              n_splits,
              train_split,
              random_state,
              stratification,
          )
  
imports: import copy
  import logging
  import torch.multiprocessing
  from contextlib import ExitStack
  from functools import partial
  import random
  from pathlib import Path
  from itertools import chain, groupby
  import numpy
  from sklearn.utils.class_weight import compute_class_weight
  from torch.utils.data import Dataset, Subset, IterableDataset
  from torch.utils.data.distributed import DistributedSampler
  from torch.utils.data.sampler import RandomSampler, SequentialSampler
  import torch
  from sklearn.model_selection import StratifiedKFold, KFold, ShuffleSplit, StratifiedShuffleSplit
  from tqdm import tqdm
  from farm.data_handler.dataloader import NamedDataLoader
  from farm.data_handler.processor import Processor, BertStyleLMProcessor
  from farm.data_handler.utils import grouper
  from farm.data_handler.dataset import ConcatTensorDataset
  from farm.modeling.tokenization import EmbeddingTokenizer
  from farm.utils import MLFlowLogger
  from farm.utils import log_ascii_workers, calc_chunksize
  from farm.utils import get_dict_checksum
  from farm.visual.ascii.images import TRACTOR_SMALL
label_desc: Use image node_id 3 for calling the DataSiloForHoldout obj's underlying member method code with example usage: obj.make(cls, datasilo, sets, n_splits, train_split, random_state, stratification) and returns: cls