class_docstr: Restricted  Boltzmann Machine
class_docstr_tok: ["Restricted","Boltzmann","Machine"]
docstr: Implementation of a multinomial Restricted Boltzmann Machine for collaborative filtering
  in numpy/pandas/tensorflow
  
  Based on the article by Ruslan Salakhutdinov, Andriy Mnih and Geoffrey Hinton
  https://www.cs.toronto.edu/~rsalakhu/papers/rbmcf.pdf
  
  In this implementation we use multinomial units instead of the one-hot-encoded used in
  the paper. This means that the weights are rank 2 (matrices) instead of rank 3 tensors.
  
  Basic mechanics:
  
  1) A computational graph is created when the RBM class is instantiated.
  For an item based recommender this consists of:
  visible units: The number n_visible of visible units equals the number of items
  hidden units : hyperparameter to fix during training
  
  2) Gibbs Sampling:
  
  2.1) for each training epoch, the visible units are first clamped on the data
  
  2.2) The activation probability of the hidden units, given a linear combination of
  the visibles, is evaluated P(h=1|phi_v). The latter is then used to sample the
  value of the hidden units.
  
  2.3) The probability P(v=l|phi_h) is evaluated, where l=1,..,r are the ratings (e.g.
  r=5 for the movielens dataset). In general, this is a multinomial distribution,
  from which we sample the value of v.
  
  2.4) This step is repeated k times, where k increases as optimization converges. It is
  essential to fix to zero the original unrated items during the all learning process.
  
  3) Optimization:
  The free energy of the visible units given the hidden is evaluated at the beginning (F_0)
  and after k steps of Bernoulli sampling (F_k). The weights and biases are updated by
  minimizing the differene F_0 - F_k.
  
  4) Inference:
  Once the joint probability distribution P(v,h) is learned, this is used to generate ratings
  for unrated items for all users
docstr_tok: ["Implementation","of","a","multinomial","Restricted","Boltzmann","Machine","for","collaborative","filtering","in","numpy\/pandas\/tensorflow","Based","on","the","article","by","Ruslan","Salakhutdinov",",","Andriy","Mnih","and","Geoffrey","Hinton","https",":","\/\/www.cs.toronto.edu\/~rsalakhu\/papers\/rbmcf.pdf","In","this","implementation","we","use","multinomial","units","instead","of","the","one-hot-encoded","used","in","the","paper",".","This","means","that","the","weights","are","rank","2","(","matrices",")","instead","of","rank","3","tensors",".","Basic","mechanics",":","1",")","A","computational","graph","is","created","when","the","RBM","class","is","instantiated",".","For","an","item","based","recommender","this","consists","of",":","visible","units",":","The","number","n_visible","of","visible","units","equals","the","number","of","items","hidden","units",":","hyperparameter","to","fix","during","training","2",")","Gibbs","Sampling",":","2.1",")","for","each","training","epoch",",","the","visible","units","are","first","clamped","on","the","data","2.2",")","The","activation","probability","of","the","hidden","units",",","given","a","linear","combination","of","the","visibles",",","is","evaluated","P","(","h=1|phi_v",")",".","The","latter","is","then","used","to","sample","the","value","of","the","hidden","units",".","2.3",")","The","probability","P","(","v=l|phi_h",")","is","evaluated",",","where","l=1",",","..",",","r","are","the","ratings","(","e.g",".","r=5","for","the","movielens","dataset",")",".","In","general",",","this","is","a","multinomial","distribution",",","from","which","we","sample","the","value","of","v.","2.4",")","This","step","is","repeated","k","times",",","where","k","increases","as","optimization","converges",".","It","is","essential","to","fix","to","zero","the","original","unrated","items","during","the","all","learning","process",".","3",")","Optimization",":","The","free","energy","of","the","visible","units","given","the","hidden","is","evaluated","at","the","beginning","(","F_0",")","and","after","k","steps","of","Bernoulli","sampling","(","F_k",")",".","The","weights","and","biases","are","updated","by","minimizing","the","differene","F_0","-","F_k",".","4",")","Inference",":","Once","the","joint","probability","distribution","P","(","v",",","h",")","is","learned",",","this","is","used","to","generate","ratings","for","unrated","items","for","all","users"]
code: def __init__(
      self,
      possible_ratings,
      visible_units,
      hidden_units=500,
      keep_prob=0.7,
      init_stdv=0.1,
      learning_rate=0.004,
      minibatch_size=100,
      training_epoch=20,
      display_epoch=10,
      sampling_protocol=[50, 70, 80, 90, 100],
      debug=False,
      with_metrics=False,
      seed=42,
  ):
      """Implementation of a multinomial Restricted Boltzmann Machine for collaborative filtering
      in numpy/pandas/tensorflow
  
      Based on the article by Ruslan Salakhutdinov, Andriy Mnih and Geoffrey Hinton
      https://www.cs.toronto.edu/~rsalakhu/papers/rbmcf.pdf
  
      In this implementation we use multinomial units instead of the one-hot-encoded used in
      the paper. This means that the weights are rank 2 (matrices) instead of rank 3 tensors.
  
      Basic mechanics:
  
      1) A computational graph is created when the RBM class is instantiated.
      For an item based recommender this consists of:
      visible units: The number n_visible of visible units equals the number of items
      hidden units : hyperparameter to fix during training
  
      2) Gibbs Sampling:
  
      2.1) for each training epoch, the visible units are first clamped on the data
  
      2.2) The activation probability of the hidden units, given a linear combination of
      the visibles, is evaluated P(h=1|phi_v). The latter is then used to sample the
      value of the hidden units.
  
      2.3) The probability P(v=l|phi_h) is evaluated, where l=1,..,r are the ratings (e.g.
      r=5 for the movielens dataset). In general, this is a multinomial distribution,
      from which we sample the value of v.
  
      2.4) This step is repeated k times, where k increases as optimization converges. It is
      essential to fix to zero the original unrated items during the all learning process.
  
      3) Optimization:
      The free energy of the visible units given the hidden is evaluated at the beginning (F_0)
      and after k steps of Bernoulli sampling (F_k). The weights and biases are updated by
      minimizing the differene F_0 - F_k.
  
      4) Inference:
      Once the joint probability distribution P(v,h) is learned, this is used to generate ratings
      for unrated items for all users
      """
  
      # RBM parameters
      self.n_hidden = hidden_units  # number of hidden units
      self.keep = (
          keep_prob  # keep probability for dropout regularization
      )
  
      # standard deviation used to initialize the weights matrices
      self.stdv = init_stdv
  
      # learning rate used in the update method of the optimizer
      self.learning_rate = learning_rate
  
      # size of the minibatch used in the random minibatches training; setting to 1 corresponds to
      # stochastic gradient descent, and it is considerably slower. Good performance is achieved
      # for a size of ~100.
      self.minibatch = minibatch_size
      self.epochs = (
          training_epoch + 1
      )  # number of epochs used to train the model
  
      # number of epochs to show the mse error during training
      self.display_epoch = display_epoch
  
      # protocol to increase Gibbs sampling's step. Array containing the
      # percentage of the total training epoch when the step increases by 1
      self.sampling_protocol = sampling_protocol
  
      # if true, functions print their control paramters and/or outputs
      self.debug = debug
  
      # if true, compute msre and accuracy during training
      self.with_metrics = with_metrics
  
      # Seed
      self.seed = seed
      np.random.seed(self.seed)
      tf.compat.v1.set_random_seed(self.seed)
  
      self.n_visible = visible_units  # number of items
  
      tf.compat.v1.reset_default_graph()
  
      # ----------------------Initializers-------------------------------------
  
      # create a sorted list of all the unique ratings (of float type)
      self.possible_ratings = possible_ratings
  
      # create a lookup table to map integer indices to float ratings
      self.ratings_lookup_table = tf.lookup.StaticHashTable(
          tf.lookup.KeyValueTensorInitializer(
              tf.constant(
                  list(range(len(self.possible_ratings))),
                  dtype=tf.int32,
              ),
              tf.constant(
                  list(self.possible_ratings), dtype=tf.float32
              ),
          ),
          default_value=0,
      )
  
      self.generate_graph()
      self.init_metrics()
      self.init_gpu()
      init_graph = tf.compat.v1.global_variables_initializer()
  
      # Start TF training session on default graph
      self.sess = tf.compat.v1.Session(config=self.config_gpu)
      self.sess.run(init_graph)
  
imports: import numpy
  import tensorflow
  import logging
  import os
  from pathlib import Path
label_desc: Use image node_id 1 to create a new RBM object  with example: obj = RBM(possible_ratings, visible_units, hidden_units, keep_prob, init_stdv, learning_rate, minibatch_size, training_epoch, display_epoch, sampling_protocol, debug, with_metrics, seed)