class_docstr: Creates `Dataset` objects from input image files.
  
  This class allows for loading of images in various formats.
  For user convenience, also accepts zip-files and directories
  of images and uses some limited intelligence to attempt to
  traverse subdirectories which contain images.
  
  Currently, only .png and .tif files are supported. If the
  inputs or labels are given as a list of files, the list must contain
  only image files.
  
  Examples
  --------
  For this example, we will be using the BBBC001 Dataset. This dataset contains 6 images
  of human HT29 colon cancer cells. We will use the images as inputs and we will assign
  the labels as integers ranging from 1 to 6 for the sake of simplicity.
  
  To learn more about this dataset, please visit: https://data.broadinstitute.org/bbbc/BBBC001/
  and also see our loader for this dataset: `deepchem.molnet.loadbbbc001`.
  
  Let's begin by importing the necessary modules and downloading the dataset.
  >>> import os
  >>> import deepchem as dc
  >>> data_dir = dc.utils.data_utils.get_data_dir()
  >>> dataset_file = os.path.join(data_dir, "BBBC001_v1_images_tif.zip")
  >>> BBBC1_IMAGE_URL = 'https://data.broadinstitute.org/bbbc/BBBC001/BBBC001_v1_images_tif.zip'
  >>> if not os.path.exists(dataset_file):
  ...    dc.utils.data_utils.download_url(url=BBBC1_IMAGE_URL, dest_dir=data_dir)
  
  Now that we have the dataset, let's create a list of labels for each image.
  
  >>> labels = np.array([1,2,3,4,5,6])
  
  Let's now write this to disk somewhere. We can now use `ImageLoader` to process
  this Image dataset. We do not use a featurizer here, hence the `UserDefinedFeaturizer`
  with an empty list.
  
  >>> featurizer = dc.feat.UserDefinedFeaturizer([])
  >>> loader = dc.data.ImageLoader(tasks=['demo-task'], sorting=False)
  >>> dataset = loader.create_dataset(inputs=(dataset_file, labels),
  ...                                 in_memory=False)
  
  We can confirm that we have 6 images in our dataset and 6 labels. The images are
  of size 512x512 while the labels are just integers.
  
  >>> len(dataset)
  6
  >>> dataset.X.shape
  (6, 512, 512)
  >>> dataset.y.shape
  (6,)
  
  The label files can also be images similar to the inputs, in which case we
  can provide a list of label files instead of a list of labels.
  
  To show this, we will use the input data as the ground truths, this is often
  seen when making autoencoders. Similar to the above example, let's use `ImageLoader`
  to process this Image dataset.
  
  >>> featurizer = dc.feat.UserDefinedFeaturizer([])
  >>> loader = dc.data.ImageLoader(tasks=['demo-task'], sorting=False)
  >>> dataset = loader.create_dataset(inputs=(dataset_file, dataset_file),
  ...                                 in_memory=False)
  
  We can confirm that we have 6 images in our dataset and 6 labels. The images are
  of size 512x512 while the labels are also images of size 512x512.
  
  >>> len(dataset)
  6
  >>> dataset.X.shape
  (6, 512, 512)
  >>> dataset.y.shape
  (6, 512, 512)
class_docstr_tok: ["Creates","`","Dataset","`","objects","from","input","image","files",".","This","class","allows","for","loading","of","images","in","various","formats",".","For","user","convenience",",","also","accepts","zip-files","and","directories","of","images","and","uses","some","limited","intelligence","to","attempt","to","traverse","subdirectories","which","contain","images",".","Currently",",","only",".png","and",".tif","files","are","supported",".","If","the","inputs","or","labels","are","given","as","a","list","of","files",",","the","list","must","contain","only","image","files",".","Examples","--","--","--","--","For","this","example",",","we","will","be","using","the","BBBC001","Dataset",".","This","dataset","contains","6","images","of","human","HT29","colon","cancer","cells",".","We","will","use","the","images","as","inputs","and","we","will","assign","the","labels","as","integers","ranging","from","1","to","6","for","the","sake","of","simplicity",".","To","learn","more","about","this","dataset",",","please","visit",":","https",":","\/\/data.broadinstitute.org\/bbbc\/BBBC001\/","and","also","see","our","loader","for","this","dataset",":","`","deepchem.molnet.loadbbbc001","`",".","Let","'s","begin","by","importing","the","necessary","modules","and","downloading","the","dataset",".",">",">",">","import","os",">",">",">","import","deepchem","as","dc",">",">",">","data_dir","=","dc.utils.data_utils.get_data_dir","(",")",">",">",">","dataset_file","=","os.path.join","(","data_dir",",","``","BBBC001_v1_images_tif.zip","''",")",">",">",">","BBBC1_IMAGE_URL","=","'https",":","\/\/data.broadinstitute.org\/bbbc\/BBBC001\/BBBC001_v1_images_tif.zip'",">",">",">","if","not","os.path.exists","(","dataset_file",")",":","...","dc.utils.data_utils.download_url","(","url=BBBC1_IMAGE_URL",",","dest_dir=data_dir",")","Now","that","we","have","the","dataset",",","let","'s","create","a","list","of","labels","for","each","image",".",">",">",">","labels","=","np.array","(","[","1,2,3,4,5,6","]",")","Let","'s","now","write","this","to","disk","somewhere",".","We","can","now","use","`","ImageLoader","`","to","process","this","Image","dataset",".","We","do","not","use","a","featurizer","here",",","hence","the","`","UserDefinedFeaturizer","`","with","an","empty","list",".",">",">",">","featurizer","=","dc.feat.UserDefinedFeaturizer","(","[","]",")",">",">",">","loader","=","dc.data.ImageLoader","(","tasks=","[","'demo-task","'","]",",","sorting=False",")",">",">",">","dataset","=","loader.create_dataset","(","inputs=","(","dataset_file",",","labels",")",",","...","in_memory=False",")","We","can","confirm","that","we","have","6","images","in","our","dataset","and","6","labels",".","The","images","are","of","size","512x512","while","the","labels","are","just","integers",".",">",">",">","len","(","dataset",")","6",">",">",">","dataset.X.shape","(","6",",","512",",","512",")",">",">",">","dataset.y.shape","(","6",",",")","The","label","files","can","also","be","images","similar","to","the","inputs",",","in","which","case","we","can","provide","a","list","of","label","files","instead","of","a","list","of","labels",".","To","show","this",",","we","will","use","the","input","data","as","the","ground","truths",",","this","is","often","seen","when","making","autoencoders",".","Similar","to","the","above","example",",","let","'s","use","`","ImageLoader","`","to","process","this","Image","dataset",".",">",">",">","featurizer","=","dc.feat.UserDefinedFeaturizer","(","[","]",")",">",">",">","loader","=","dc.data.ImageLoader","(","tasks=","[","'demo-task","'","]",",","sorting=False",")",">",">",">","dataset","=","loader.create_dataset","(","inputs=","(","dataset_file",",","dataset_file",")",",","...","in_memory=False",")","We","can","confirm","that","we","have","6","images","in","our","dataset","and","6","labels",".","The","images","are","of","size","512x512","while","the","labels","are","also","images","of","size","512x512",".",">",">",">","len","(","dataset",")","6",">",">",">","dataset.X.shape","(","6",",","512",",","512",")",">",">",">","dataset.y.shape","(","6",",","512",",","512",")"]
docstr: Creates and returns a `Dataset` object by featurizing provided image files and labels/weights.
  
  Parameters
  ----------
  inputs: `Union[OneOrMany[str], Tuple[Any]]`
      The inputs provided should be one of the following
  
        - filename
        - list of filenames
        - Tuple (list of filenames, labels)
        - Tuple (list of filenames, list of label filenames)
        - Tuple (list of filenames, labels, weights)
        - Tuple (list of filenames, list of label filenames, weights)
  
      Each file in a given list of filenames should either be of a supported
      image format (.png, .tif only for now) or of a compressed folder of
      image files (only .zip for now). If `labels` or `weights` are provided,
      they must correspond to the sorted order of all filenames provided, with
      one label/weight per file. Labels can be filenames too, in which case the
      labels are loaded as images.
  
  data_dir: str, optional (default None)
      Directory to store featurized dataset.
  shard_size: int, optional (default 8192)
      Shard size when loading data.
  in_memory: bool, optioanl (default False)
      If true, return in-memory `NumpyDataset`. Else return `ImageDataset`.
  
  Returns
  -------
  `ImageDataset` or `NumpyDataset` or `DiskDataset`
      - if `in_memory == False`, the return value is `ImageDataset`.
      - if `in_memory == True` and `data_dir is None`, the return value is `NumpyDataset`.
      - if `in_memory == True` and `data_dir is not None`, the return value is `DiskDataset`.
docstr_tok: ["Creates","and","returns","a","`","Dataset","`","object","by","featurizing","provided","image","files","and","labels\/weights",".","Parameters","--","--","--","--","--","inputs",":","`","Union","[","OneOrMany","[","str","]",",","Tuple","[","Any","]","]","`","The","inputs","provided","should","be","one","of","the","following","-","filename","-","list","of","filenames","-","Tuple","(","list","of","filenames",",","labels",")","-","Tuple","(","list","of","filenames",",","list","of","label","filenames",")","-","Tuple","(","list","of","filenames",",","labels",",","weights",")","-","Tuple","(","list","of","filenames",",","list","of","label","filenames",",","weights",")","Each","file","in","a","given","list","of","filenames","should","either","be","of","a","supported","image","format","(",".png",",",".tif","only","for","now",")","or","of","a","compressed","folder","of","image","files","(","only",".zip","for","now",")",".","If","`","labels","`","or","`","weights","`","are","provided",",","they","must","correspond","to","the","sorted","order","of","all","filenames","provided",",","with","one","label\/weight","per","file",".","Labels","can","be","filenames","too",",","in","which","case","the","labels","are","loaded","as","images",".","data_dir",":","str",",","optional","(","default","None",")","Directory","to","store","featurized","dataset",".","shard_size",":","int",",","optional","(","default","8192",")","Shard","size","when","loading","data",".","in_memory",":","bool",",","optioanl","(","default","False",")","If","true",",","return","in-memory","`","NumpyDataset","`",".","Else","return","`","ImageDataset","`",".","Returns","--","--","--","-","`","ImageDataset","`","or","`","NumpyDataset","`","or","`","DiskDataset","`","-","if","`","in_memory","==","False","`",",","the","return","value","is","`","ImageDataset","`",".","-","if","`","in_memory","==","True","`","and","`","data_dir","is","None","`",",","the","return","value","is","`","NumpyDataset","`",".","-","if","`","in_memory","==","True","`","and","`","data_dir","is","not","None","`",",","the","return","value","is","`","DiskDataset","`","."]
code: def create_dataset(
      self,
      inputs: Union[OneOrMany[str], Tuple[Any], Tuple[str, Any]],
      data_dir: Optional[str] = None,
      shard_size: Optional[int] = 8192,
      in_memory: bool = False,
  ) -> Dataset:
      """Creates and returns a `Dataset` object by featurizing provided image files and labels/weights.
  
      Parameters
      ----------
      inputs: `Union[OneOrMany[str], Tuple[Any]]`
          The inputs provided should be one of the following
  
            - filename
            - list of filenames
            - Tuple (list of filenames, labels)
            - Tuple (list of filenames, list of label filenames)
            - Tuple (list of filenames, labels, weights)
            - Tuple (list of filenames, list of label filenames, weights)
  
          Each file in a given list of filenames should either be of a supported
          image format (.png, .tif only for now) or of a compressed folder of
          image files (only .zip for now). If `labels` or `weights` are provided,
          they must correspond to the sorted order of all filenames provided, with
          one label/weight per file. Labels can be filenames too, in which case the
          labels are loaded as images.
  
      data_dir: str, optional (default None)
          Directory to store featurized dataset.
      shard_size: int, optional (default 8192)
          Shard size when loading data.
      in_memory: bool, optioanl (default False)
          If true, return in-memory `NumpyDataset`. Else return `ImageDataset`.
  
      Returns
      -------
      `ImageDataset` or `NumpyDataset` or `DiskDataset`
          - if `in_memory == False`, the return value is `ImageDataset`.
          - if `in_memory == True` and `data_dir is None`, the return value is `NumpyDataset`.
          - if `in_memory == True` and `data_dir is not None`, the return value is `DiskDataset`.
      """
      labels, weights = None, None
      if isinstance(inputs, tuple):
          if len(inputs) == 1:
              input_files = inputs[0]
              if isinstance(inputs, str):
                  input_files = [inputs]
          elif len(inputs) == 2:
              input_files, labels = inputs
          elif len(inputs) == 3:
              input_files, labels, weights = inputs
          else:
              raise ValueError(
                  "Input must be a tuple of length 1, 2, or 3"
              )
      else:
          input_files = inputs
      if isinstance(input_files, str):
          input_files = [input_files]
  
      image_files = []
      # Sometimes zip files contain directories within. Traverse directories
      while len(input_files) > 0:
          remainder = []
          for input_file in input_files:
              filename, extension = os.path.splitext(input_file)
              extension = extension.lower()
              if os.path.isdir(input_file):
                  dirfiles = [
                      os.path.join(input_file, subfile)
                      for subfile in os.listdir(input_file)
                  ]
                  remainder += dirfiles
              elif extension == ".zip":
                  zip_dir = tempfile.mkdtemp()
                  zip_ref = zipfile.ZipFile(input_file, "r")
                  zip_ref.extractall(path=zip_dir)
                  zip_ref.close()
                  zip_files = [
                      os.path.join(zip_dir, name)
                      for name in zip_ref.namelist()
                  ]
                  for zip_file in zip_files:
                      _, extension = os.path.splitext(zip_file)
                      extension = extension.lower()
                      if extension in [".png", ".tif"]:
                          image_files.append(zip_file)
              elif extension in [".png", ".tif"]:
                  image_files.append(input_file)
              else:
                  raise ValueError("Unsupported file format")
          input_files = remainder
  
      # Sort image files
      if self.sorting:
          image_files = sorted(image_files)
  
      if isinstance(labels, str):
          label_files = [labels]
      else:
          label_files = []
  
      label_image_files = []
      # Sometimes zip files contain directories within. Traverse directories
      while len(label_files) > 0:
          remainder = []
          for label_file in label_files:
              filename, extension = os.path.splitext(label_file)
              extension = extension.lower()
              if os.path.isdir(label_file):
                  dirfiles = [
                      os.path.join(label_file, subfile)
                      for subfile in os.listdir(label_file)
                  ]
                  remainder += dirfiles
              elif extension == ".zip":
                  zip_dir = tempfile.mkdtemp()
                  zip_ref = zipfile.ZipFile(label_file, "r")
                  zip_ref.extractall(path=zip_dir)
                  zip_ref.close()
                  zip_files = [
                      os.path.join(zip_dir, name)
                      for name in zip_ref.namelist()
                  ]
                  for zip_file in zip_files:
                      _, extension = os.path.splitext(zip_file)
                      extension = extension.lower()
                      if extension in [".png", ".tif"]:
                          label_image_files.append(zip_file)
              elif extension in [".png", ".tif"]:
                  label_image_files.append(label_file)
              else:
                  raise ValueError("Unsupported file format")
          label_files = remainder
  
      # Sort label image files
      if self.sorting:
          label_image_files = sorted(label_image_files)
  
      if in_memory:
          if data_dir is None:
              if isinstance(labels, str):
                  return NumpyDataset(
                      load_image_files(image_files),
                      y=load_image_files(label_image_files),
                      w=weights,
                      ids=image_files,
                  )
              else:
                  return NumpyDataset(
                      load_image_files(image_files),
                      y=labels,
                      w=weights,
                      ids=image_files,
                  )
          else:
              if isinstance(labels, str):
                  dataset = DiskDataset.from_numpy(
                      load_image_files(image_files),
                      y=load_image_files(label_image_files),
                      w=weights,
                      ids=image_files,
                      tasks=self.tasks,
                      data_dir=data_dir,
                  )
              else:
                  dataset = DiskDataset.from_numpy(
                      load_image_files(image_files),
                      y=labels,
                      w=weights,
                      ids=image_files,
                      tasks=self.tasks,
                      data_dir=data_dir,
                  )
              if shard_size is not None:
                  dataset.reshard(shard_size)
              return dataset
      else:
          if isinstance(labels, str):
              return ImageDataset(
                  image_files,
                  y=label_image_files,
                  w=weights,
                  ids=image_files,
              )
          else:
              return ImageDataset(
                  image_files, y=labels, w=weights, ids=image_files
              )
  
imports: import os
  import tempfile
  import zipfile
  import time
  import logging
  import warnings
  from typing import List, Optional, Tuple, Any, Sequence, Union, Iterator
  import pandas
  import numpy
  from deepchem.utils.typing import OneOrMany
  from deepchem.utils.data_utils import load_image_files, load_csv_files, load_json_files, load_sdf_files, unzip_file
  from deepchem.feat import UserDefinedFeaturizer, Featurizer
  from deepchem.data import Dataset, DiskDataset, NumpyDataset, ImageDataset
  from deepchem.feat.molecule_featurizers import OneHotFeaturizer
  from deepchem.utils.genomics_utils import encode_bio_sequence
label_desc: Use image node_id 2 for calling the ImageLoader obj's underlying member method code with example usage: obj.create_dataset(inputs, data_dir, shard_size, in_memory) and returns: dataset, ImageDataset, ImageDataset, NumpyDataset, NumpyDataset