class_docstr: Ivis is a technique that uses an artificial neural network for
  dimensionality reduction, often useful for the purposes of visualization.
  The network trains on triplets of data-points at a time and pulls positive
  points together, while pushing more distant points away from each other.
  Triplets are sampled from the original data using KNN approximation using
  the Annoy library.
  
  :param int embedding_dims: Number of dimensions in the embedding space
  :param int k: The number of neighbours to retrieve for each point.
      Must be less than one minus the number of rows in the dataset.
  :param Union[str,Callable] distance: The loss function used to train
      the neural network.
  
      *   If string: a registered loss function name. Predefined losses are:
          "pn", "euclidean", "manhattan_pn", "manhattan", "chebyshev",
          "chebyshev_pn", "softmax_ratio_pn", "softmax_ratio", "cosine", "cosine_pn".
      *   If Callable, must have two parameters, (y_true, y_pred).
          y_pred denotes the batch of triplets, and y_true are any corresponding labels.
          y_pred is expected to be of shape: (3, batch_size, embedding_dims).
  
              * When loading model loaded with a custom loss, provide the loss to the
                constructor of the new Ivis instance before loading the saved model.
  :param int batch_size: The size of mini-batches used during gradient
      descent while training the neural network. Must be less than the
      num_rows in the dataset.
  :param int epochs: The maximum number of epochs to train the model for.
      Each epoch the network will see a triplet based on each data-point
      once.
  :param int n_epochs_without_progress: After n number of epochs without an
      improvement to the loss, terminate training early.
  :param int n_trees: The number of random projections trees built by Annoy to
      approximate KNN. The more trees the higher the memory usage, but the
      better the accuracy of results.
  :param int ntrees: Deprecated. Use `n_trees` instead.
  :param str knn_distance_metric: The distance metric used to retrieve nearest
      neighbours. Supports "angular" (default), "euclidean", "manhattan",
      "hamming", or "dot".
  :param int search_k: The maximum number of nodes inspected during a nearest
      neighbour query by Annoy. The higher, the more computation time
      required, but the higher the accuracy. The default is n_trees * k,
      where k is the number of neighbours to retrieve. If this is set too
      low, a variable number of neighbours may be retrieved per data-point.
  :param bool precompute: Whether to pre-compute the nearest neighbours.
      Pre-computing is a little faster, but requires more memory. If memory
      is limited, try setting this to False.
  :param Union[str,tf.keras.models.Model] model: The keras model to train using
      triplet loss.
  
      *   If a model object is provided, an embedding layer of size
          'embedding_dims' will be appended to the end of the network.
      *   If a string, a pre-defined network by that name will be used. Possible
          options are: 'szubert', 'hinton', 'maaten'. By default the 'szubert'
          network will be created, which is a selu network composed of 3 dense
          layers of 128 neurons each, followed by an embedding layer of size
          'embedding_dims'.
  :param str supervision_metric: The supervision metric to
      optimize when training keras in supervised mode. Supports all of the
      classification or regression losses included with keras, so long as
      the labels are provided in the correct format. A list of keras' loss
      functions can be found at https://keras.io/losses/ .
  :param float supervision_weight: Float between 0 and 1 denoting the
      weighting to give to classification vs triplet loss when training
      in supervised mode. The higher the weight, the more classification
      influences training. Ignored if using Ivis in unsupervised mode.
  :param str annoy_index_path: The filepath of a pre-trained annoy index file
      saved on disk. If provided, the annoy index file will be loaded and used.
      Otherwise, a new index will be generated and saved to disk in a temporary directory.
  :param [keras.callbacks.Callback] callbacks: List of keras Callbacks to
      pass model during training, such as the TensorBoard callback. A set of
      ivis-specific callbacks are provided in the ivis.nn.callbacks module.
  :param bool build_index_on_disk: Whether to build the annoy index directly
      on disk. Building on disk should allow for bigger datasets to be indexed,
      but may cause issues.
  :param Union[np.array,collections.abc.Sequence] neighbour_matrix:
      Providing a neighbour matrix will cause Ivis to skip computing the Annoy KNN index
      and instead use the provided neighbour_matrix.
  
      - A pre-computed neighbour matrix can be provided as a numpy array. Indexing the array
        should retrieve a list of neighbours for the data point associated with that index.
  
      - Alternatively, dynamic computation of neighbours can be done by providing a
        class than implements the collections.abc.Sequence class, specifically the
        `__getitem__` and `__len__` methods.
  
          - See the ivis.data.neighbour_retrieval.AnnoyKnnMatrix class for an example.
  :param int verbose: Controls the volume of logging output the model
      produces when training. When set to 0, silences outputs, when above 0
      will print outputs.
class_docstr_tok: ["Ivis","is","a","technique","that","uses","an","artificial","neural","network","for","dimensionality","reduction",",","often","useful","for","the","purposes","of","visualization",".","The","network","trains","on","triplets","of","data-points","at","a","time","and","pulls","positive","points","together",",","while","pushing","more","distant","points","away","from","each","other",".","Triplets","are","sampled","from","the","original","data","using","KNN","approximation","using","the","Annoy","library",".",":","param","int","embedding_dims",":","Number","of","dimensions","in","the","embedding","space",":","param","int","k",":","The","number","of","neighbours","to","retrieve","for","each","point",".","Must","be","less","than","one","minus","the","number","of","rows","in","the","dataset",".",":","param","Union","[","str",",","Callable","]","distance",":","The","loss","function","used","to","train","the","neural","network",".","*","If","string",":","a","registered","loss","function","name",".","Predefined","losses","are",":","``","pn","''",",","``","euclidean","''",",","``","manhattan_pn","''",",","``","manhattan","''",",","``","chebyshev","''",",","``","chebyshev_pn","''",",","``","softmax_ratio_pn","''",",","``","softmax_ratio","''",",","``","cosine","''",",","``","cosine_pn","''",".","*","If","Callable",",","must","have","two","parameters",",","(","y_true",",","y_pred",")",".","y_pred","denotes","the","batch","of","triplets",",","and","y_true","are","any","corresponding","labels",".","y_pred","is","expected","to","be","of","shape",":","(","3",",","batch_size",",","embedding_dims",")",".","*","When","loading","model","loaded","with","a","custom","loss",",","provide","the","loss","to","the","constructor","of","the","new","Ivis","instance","before","loading","the","saved","model",".",":","param","int","batch_size",":","The","size","of","mini-batches","used","during","gradient","descent","while","training","the","neural","network",".","Must","be","less","than","the","num_rows","in","the","dataset",".",":","param","int","epochs",":","The","maximum","number","of","epochs","to","train","the","model","for",".","Each","epoch","the","network","will","see","a","triplet","based","on","each","data-point","once",".",":","param","int","n_epochs_without_progress",":","After","n","number","of","epochs","without","an","improvement","to","the","loss",",","terminate","training","early",".",":","param","int","n_trees",":","The","number","of","random","projections","trees","built","by","Annoy","to","approximate","KNN",".","The","more","trees","the","higher","the","memory","usage",",","but","the","better","the","accuracy","of","results",".",":","param","int","ntrees",":","Deprecated",".","Use","`","n_trees","`","instead",".",":","param","str","knn_distance_metric",":","The","distance","metric","used","to","retrieve","nearest","neighbours",".","Supports","``","angular","''","(","default",")",",","``","euclidean","''",",","``","manhattan","''",",","``","hamming","''",",","or","``","dot","''",".",":","param","int","search_k",":","The","maximum","number","of","nodes","inspected","during","a","nearest","neighbour","query","by","Annoy",".","The","higher",",","the","more","computation","time","required",",","but","the","higher","the","accuracy",".","The","default","is","n_trees","*","k",",","where","k","is","the","number","of","neighbours","to","retrieve",".","If","this","is","set","too","low",",","a","variable","number","of","neighbours","may","be","retrieved","per","data-point",".",":","param","bool","precompute",":","Whether","to","pre-compute","the","nearest","neighbours",".","Pre-computing","is","a","little","faster",",","but","requires","more","memory",".","If","memory","is","limited",",","try","setting","this","to","False",".",":","param","Union","[","str",",","tf.keras.models.Model","]","model",":","The","keras","model","to","train","using","triplet","loss",".","*","If","a","model","object","is","provided",",","an","embedding","layer","of","size","'embedding_dims","'","will","be","appended","to","the","end","of","the","network",".","*","If","a","string",",","a","pre-defined","network","by","that","name","will","be","used",".","Possible","options","are",":","'szubert","'",",","'hinton","'",",","'maaten","'",".","By","default","the","'szubert'","network","will","be","created",",","which","is","a","selu","network","composed","of","3","dense","layers","of","128","neurons","each",",","followed","by","an","embedding","layer","of","size","'embedding_dims","'",".",":","param","str","supervision_metric",":","The","supervision","metric","to","optimize","when","training","keras","in","supervised","mode",".","Supports","all","of","the","classification","or","regression","losses","included","with","keras",",","so","long","as","the","labels","are","provided","in","the","correct","format",".","A","list","of","keras","'","loss","functions","can","be","found","at","https",":","\/\/keras.io\/losses\/",".",":","param","float","supervision_weight",":","Float","between","0","and","1","denoting","the","weighting","to","give","to","classification","vs","triplet","loss","when","training","in","supervised","mode",".","The","higher","the","weight",",","the","more","classification","influences","training",".","Ignored","if","using","Ivis","in","unsupervised","mode",".",":","param","str","annoy_index_path",":","The","filepath","of","a","pre-trained","annoy","index","file","saved","on","disk",".","If","provided",",","the","annoy","index","file","will","be","loaded","and","used",".","Otherwise",",","a","new","index","will","be","generated","and","saved","to","disk","in","a","temporary","directory",".",":","param","[","keras.callbacks.Callback","]","callbacks",":","List","of","keras","Callbacks","to","pass","model","during","training",",","such","as","the","TensorBoard","callback",".","A","set","of","ivis-specific","callbacks","are","provided","in","the","ivis.nn.callbacks","module",".",":","param","bool","build_index_on_disk",":","Whether","to","build","the","annoy","index","directly","on","disk",".","Building","on","disk","should","allow","for","bigger","datasets","to","be","indexed",",","but","may","cause","issues",".",":","param","Union","[","np.array",",","collections.abc.Sequence","]","neighbour_matrix",":","Providing","a","neighbour","matrix","will","cause","Ivis","to","skip","computing","the","Annoy","KNN","index","and","instead","use","the","provided","neighbour_matrix",".","-","A","pre-computed","neighbour","matrix","can","be","provided","as","a","numpy","array",".","Indexing","the","array","should","retrieve","a","list","of","neighbours","for","the","data","point","associated","with","that","index",".","-","Alternatively",",","dynamic","computation","of","neighbours","can","be","done","by","providing","a","class","than","implements","the","collections.abc.Sequence","class",",","specifically","the","`","__getitem__","`","and","`","__len__","`","methods",".","-","See","the","ivis.data.neighbour_retrieval.AnnoyKnnMatrix","class","for","an","example",".",":","param","int","verbose",":","Controls","the","volume","of","logging","output","the","model","produces","when","training",".","When","set","to","0",",","silences","outputs",",","when","above","0","will","print","outputs","."]
docstr: Fit to data then transform
  
  Parameters
  ----------
  X : np.array, ivis.data.sequence.IndexableDataset, tensorflow.keras.utils.HDF5Matrix
      Data to train on and then embedded.
      Needs to have a `.shape` attribute and a `__getitem__` method.
  Y : array, shape (n_samples)
      Optional array for supervised dimensionality reduction.
      If Y contains -1 labels, and 'sparse_categorical_crossentropy'
      is the loss function, semi-supervised learning will be used.
  
  Returns
  -------
  X_new : array, shape (n_samples, embedding_dims)
      Embedding of the data in low-dimensional space.
docstr_tok: ["Fit","to","data","then","transform","Parameters","--","--","--","--","--","X",":","np.array",",","ivis.data.sequence.IndexableDataset",",","tensorflow.keras.utils.HDF5Matrix","Data","to","train","on","and","then","embedded",".","Needs","to","have","a","`",".shape","`","attribute","and","a","`","__getitem__","`","method",".","Y",":","array",",","shape","(","n_samples",")","Optional","array","for","supervised","dimensionality","reduction",".","If","Y","contains","-1","labels",",","and","'sparse_categorical_crossentropy'","is","the","loss","function",",","semi-supervised","learning","will","be","used",".","Returns","--","--","--","-","X_new",":","array",",","shape","(","n_samples",",","embedding_dims",")","Embedding","of","the","data","in","low-dimensional","space","."]
code: def fit_transform(self, X, Y=None, shuffle_mode=True):
      """Fit to data then transform
  
      Parameters
      ----------
      X : np.array, ivis.data.sequence.IndexableDataset, tensorflow.keras.utils.HDF5Matrix
          Data to train on and then embedded.
          Needs to have a `.shape` attribute and a `__getitem__` method.
      Y : array, shape (n_samples)
          Optional array for supervised dimensionality reduction.
          If Y contains -1 labels, and 'sparse_categorical_crossentropy'
          is the loss function, semi-supervised learning will be used.
  
      Returns
      -------
      X_new : array, shape (n_samples, embedding_dims)
          Embedding of the data in low-dimensional space.
      """
  
      self.fit(X, Y, shuffle_mode)
      return self.transform(X)
  
imports: import json
  import os
  import shutil
  import tempfile
  import weakref
  from copy import copy
  import dill
  import numpy
  import tensorflow
  from sklearn.base import BaseEstimator, TransformerMixin
  from sklearn.exceptions import NotFittedError
  from tensorflow import keras
  from tensorflow.keras.callbacks import EarlyStopping
  from tensorflow.keras.models import load_model, Model
  from .data.generators import generator_from_neighbour_matrix, KerasSequence
  from .data.neighbour_retrieval import AnnoyKnnMatrix
  from .nn.network import build_supervised_layer, triplet_network, base_network
  from .nn.callbacks import ModelCheckpoint
  from .nn.losses import triplet_loss, semi_supervised_loss
  from .utils.deprecation import check_deprecated_ntrees, deprecate_positional_args
label_desc: Use image node_id 8 for calling the Ivis obj's underlying member method code with example usage: obj.fit_transform(X, Y, shuffle_mode) and returns: self