class_docstr: Handles loading/featurizing of data from disk.
  
  The main use of `DataLoader` and its child classes is to make it
  easier to load large datasets into `Dataset` objects.`
  
  `DataLoader` is an abstract superclass that provides a
  general framework for loading data into DeepChem. This class should
  never be instantiated directly.  To load your own type of data, make
  a subclass of `DataLoader` and provide your own implementation for
  the `create_dataset()` method.
  
  To construct a `Dataset` from input data, first instantiate a
  concrete data loader (that is, an object which is an instance of a
  subclass of `DataLoader`) with a given `Featurizer` object. Then
  call the data loader's `create_dataset()` method on a list of input
  files that hold the source data to process. Note that each subclass
  of `DataLoader` is specialized to handle one type of input data so
  you will have to pick the loader class suitable for your input data
  type.
  
  Note that it isn't necessary to use a data loader to process input
  data. You can directly use `Featurizer` objects to featurize
  provided input into numpy arrays, but note that this calculation
  will be performed in memory, so you will have to write generators
  that walk the source files and write featurized data to disk
  yourself. `DataLoader` and its subclasses make this process easier
  for you by performing this work under the hood.
class_docstr_tok: ["Handles","loading\/featurizing","of","data","from","disk",".","The","main","use","of","`","DataLoader","`","and","its","child","classes","is","to","make","it","easier","to","load","large","datasets","into","`","Dataset","`","objects.","`","`","DataLoader","`","is","an","abstract","superclass","that","provides","a","general","framework","for","loading","data","into","DeepChem",".","This","class","should","never","be","instantiated","directly",".","To","load","your","own","type","of","data",",","make","a","subclass","of","`","DataLoader","`","and","provide","your","own","implementation","for","the","`","create_dataset","(",")","`","method",".","To","construct","a","`","Dataset","`","from","input","data",",","first","instantiate","a","concrete","data","loader","(","that","is",",","an","object","which","is","an","instance","of","a","subclass","of","`","DataLoader","`",")","with","a","given","`","Featurizer","`","object",".","Then","call","the","data","loader","'s","`","create_dataset","(",")","`","method","on","a","list","of","input","files","that","hold","the","source","data","to","process",".","Note","that","each","subclass","of","`","DataLoader","`","is","specialized","to","handle","one","type","of","input","data","so","you","will","have","to","pick","the","loader","class","suitable","for","your","input","data","type",".","Note","that","it","is","n't","necessary","to","use","a","data","loader","to","process","input","data",".","You","can","directly","use","`","Featurizer","`","objects","to","featurize","provided","input","into","numpy","arrays",",","but","note","that","this","calculation","will","be","performed","in","memory",",","so","you","will","have","to","write","generators","that","walk","the","source","files","and","write","featurized","data","to","disk","yourself",".","`","DataLoader","`","and","its","subclasses","make","this","process","easier","for","you","by","performing","this","work","under","the","hood","."]
docstr: Creates and returns a `Dataset` object by featurizing provided files.
  
  Reads in `inputs` and uses `self.featurizer` to featurize the
  data in these inputs.  For large files, automatically shards
  into smaller chunks of `shard_size` datapoints for convenience.
  Returns a `Dataset` object that contains the featurized dataset.
  
  This implementation assumes that the helper methods `_get_shards`
  and `_featurize_shard` are implemented and that each shard
  returned by `_get_shards` is a pandas dataframe.  You may choose
  to reuse or override this method in your subclass implementations.
  
  Parameters
  ----------
  inputs: List
      List of inputs to process. Entries can be filenames or arbitrary objects.
  data_dir: str, optional (default None)
      Directory to store featurized dataset.
  shard_size: int, optional (default 8192)
      Number of examples stored in each shard.
  
  Returns
  -------
  DiskDataset
      A `DiskDataset` object containing a featurized representation of data
      from `inputs`.
docstr_tok: ["Creates","and","returns","a","`","Dataset","`","object","by","featurizing","provided","files",".","Reads","in","`","inputs","`","and","uses","`","self.featurizer","`","to","featurize","the","data","in","these","inputs",".","For","large","files",",","automatically","shards","into","smaller","chunks","of","`","shard_size","`","datapoints","for","convenience",".","Returns","a","`","Dataset","`","object","that","contains","the","featurized","dataset",".","This","implementation","assumes","that","the","helper","methods","`","_get_shards","`","and","`","_featurize_shard","`","are","implemented","and","that","each","shard","returned","by","`","_get_shards","`","is","a","pandas","dataframe",".","You","may","choose","to","reuse","or","override","this","method","in","your","subclass","implementations",".","Parameters","--","--","--","--","--","inputs",":","List","List","of","inputs","to","process",".","Entries","can","be","filenames","or","arbitrary","objects",".","data_dir",":","str",",","optional","(","default","None",")","Directory","to","store","featurized","dataset",".","shard_size",":","int",",","optional","(","default","8192",")","Number","of","examples","stored","in","each","shard",".","Returns","--","--","--","-","DiskDataset","A","`","DiskDataset","`","object","containing","a","featurized","representation","of","data","from","`","inputs","`","."]
code: def create_dataset(
      self,
      inputs: OneOrMany[Any],
      data_dir: Optional[str] = None,
      shard_size: Optional[int] = 8192,
  ) -> Dataset:
      """Creates and returns a `Dataset` object by featurizing provided files.
  
      Reads in `inputs` and uses `self.featurizer` to featurize the
      data in these inputs.  For large files, automatically shards
      into smaller chunks of `shard_size` datapoints for convenience.
      Returns a `Dataset` object that contains the featurized dataset.
  
      This implementation assumes that the helper methods `_get_shards`
      and `_featurize_shard` are implemented and that each shard
      returned by `_get_shards` is a pandas dataframe.  You may choose
      to reuse or override this method in your subclass implementations.
  
      Parameters
      ----------
      inputs: List
          List of inputs to process. Entries can be filenames or arbitrary objects.
      data_dir: str, optional (default None)
          Directory to store featurized dataset.
      shard_size: int, optional (default 8192)
          Number of examples stored in each shard.
  
      Returns
      -------
      DiskDataset
          A `DiskDataset` object containing a featurized representation of data
          from `inputs`.
      """
      logger.info("Loading raw samples now.")
      logger.info("shard_size: %s" % str(shard_size))
  
      # Special case handling of single input
      if not isinstance(inputs, list):
          inputs = [inputs]
  
      def shard_generator():
          for shard_num, shard in enumerate(
              self._get_shards(inputs, shard_size)
          ):
              time1 = time.time()
              X, valid_inds = self._featurize_shard(shard)
              ids = shard[self.id_field].values
              ids = ids[valid_inds]
              if len(self.tasks) > 0:
                  # Featurize task results iff they exist.
                  y, w = _convert_df_to_numpy(shard, self.tasks)
                  # Filter out examples where featurization failed.
                  y, w = (y[valid_inds], w[valid_inds])
                  assert len(X) == len(ids) == len(y) == len(w)
              else:
                  # For prospective data where results are unknown, it
                  # makes no sense to have y values or weights.
                  y, w = (None, None)
                  assert len(X) == len(ids)
  
              time2 = time.time()
              logger.info(
                  "TIMING: featurizing shard %d took %0.3f s"
                  % (shard_num, time2 - time1)
              )
              yield X, y, w, ids
  
      return DiskDataset.create_dataset(
          shard_generator(), data_dir, self.tasks
      )
  
imports: import os
  import tempfile
  import zipfile
  import time
  import logging
  import warnings
  from typing import List, Optional, Tuple, Any, Sequence, Union, Iterator
  import pandas
  import numpy
  from deepchem.utils.typing import OneOrMany
  from deepchem.utils.data_utils import load_image_files, load_csv_files, load_json_files, load_sdf_files, unzip_file
  from deepchem.feat import UserDefinedFeaturizer, Featurizer
  from deepchem.data import Dataset, DiskDataset, NumpyDataset, ImageDataset
  from deepchem.feat.molecule_featurizers import OneHotFeaturizer
  from deepchem.utils.genomics_utils import encode_bio_sequence
label_desc: Use image node_id 3 for calling the DataLoader obj's underlying member method code with example usage: obj.create_dataset(inputs, data_dir, shard_size) and returns: DiskDataset