class_docstr: An implementation of linear chain conditional random field (CRF).
  An linear chain CRF is defined to maximize the following likelihood function:
  $$ L(W, U, b; y_1, ..., y_n) := rac{1}{Z} \sum_{y_1, ..., y_n} \exp(-a_1' y_1 - a_n' y_n
      - \sum_{k=1^n}((f(x_k' W + b) y_k) + y_1' U y_2)), $$
  where:
      $Z$: normalization constant
      $x_k, y_k$:  inputs and outputs
  This implementation has two modes for optimization:
  1. (`join mode`) optimized by maximizing join likelihood, which is optimal in theory of statistics.
     Note that in this case, CRF must be the output/last layer.
  2. (`marginal mode`) return marginal probabilities on each time step and optimized via composition
     likelihood (product of marginal likelihood), i.e., using `categorical_crossentropy` loss.
     Note that in this case, CRF can be either the last layer or an intermediate layer (though not explored).
  For prediction (test phrase), one can choose either Viterbi best path (class indices) or marginal
  probabilities if probabilities are needed. However, if one chooses *join mode* for training,
  Viterbi output is typically better than marginal output, but the marginal output will still perform
  reasonably close, while if *marginal mode* is used for training, marginal output usually performs
  much better. The default behavior is set according to this observation.
  In addition, this implementation supports masking and accepts either onehot or sparse target.
  # Examples
  ```python
      model = Sequential()
      model.add(Embedding(3001, 300, mask_zero=True)(X)
      # use learn_mode = 'join', test_mode = 'viterbi', sparse_target = True (label indice output)
      crf = CRF(10, sparse_target=True)
      model.add(crf)
      # crf.accuracy is default to Viterbi acc if using join-mode (default).
      # One can add crf.marginal_acc if interested, but may slow down learning
      model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])
      # y must be label indices (with shape 1 at dim 3) here, since `sparse_target=True`
      model.fit(x, y)
      # prediction give onehot representation of Viterbi best path
      y_hat = model.predict(x_test)
  ```
  # Arguments
      units: Positive integer, dimensionality of the output space.
      learn_mode: Either 'join' or 'marginal'.
          The former train the model by maximizing join likelihood while the latter
          maximize the product of marginal likelihood over all time steps.
      test_mode: Either 'viterbi' or 'marginal'.
          The former is recommended and as default when `learn_mode = 'join'` and
          gives one-hot representation of the best path at test (prediction) time,
          while the latter is recommended and chosen as default when `learn_mode = 'marginal'`,
          which produces marginal probabilities for each time step.
      sparse_target: Boolean (default False) indicating if provided labels are one-hot or
          indices (with shape 1 at dim 3).
      use_boundary: Boolean (default True) indicating if trainable start-end chain energies
          should be added to model.
      use_bias: Boolean, whether the layer uses a bias vector.
      kernel_initializer: Initializer for the `kernel` weights matrix,
          used for the linear transformation of the inputs.
          (see [initializers](../initializers.md)).
      chain_initializer: Initializer for the `chain_kernel` weights matrix,
          used for the CRF chain energy.
          (see [initializers](../initializers.md)).
      boundary_initializer: Initializer for the `left_boundary`, 'right_boundary' weights vectors,
          used for the start/left and end/right boundary energy.
          (see [initializers](../initializers.md)).
      bias_initializer: Initializer for the bias vector
          (see [initializers](../initializers.md)).
      activation: Activation function to use
          (see [activations](../activations.md)).
          If you pass None, no activation is applied
          (ie. "linear" activation: `a(x) = x`).
      kernel_regularizer: Regularizer function applied to
          the `kernel` weights matrix
          (see [regularizer](../regularizers.md)).
      chain_regularizer: Regularizer function applied to
          the `chain_kernel` weights matrix
          (see [regularizer](../regularizers.md)).
      boundary_regularizer: Regularizer function applied to
          the 'left_boundary', 'right_boundary' weight vectors
          (see [regularizer](../regularizers.md)).
      bias_regularizer: Regularizer function applied to the bias vector
          (see [regularizer](../regularizers.md)).
      kernel_constraint: Constraint function applied to
          the `kernel` weights matrix
          (see [constraints](../constraints.md)).
      chain_constraint: Constraint function applied to
          the `chain_kernel` weights matrix
          (see [constraints](../constraints.md)).
      boundary_constraint: Constraint function applied to
          the `left_boundary`, `right_boundary` weights vectors
          (see [constraints](../constraints.md)).
      bias_constraint: Constraint function applied to the bias vector
          (see [constraints](../constraints.md)).
      input_dim: dimensionality of the input (integer).
          This argument (or alternatively, the keyword argument `input_shape`)
          is required when using this layer as the first layer in a model.
      unroll: Boolean (default False). If True, the network will be unrolled, else a symbolic loop will be used.
          Unrolling can speed-up a RNN, although it tends to be more memory-intensive.
          Unrolling is only suitable for short sequences.
  # Input shape
      3D tensor with shape `(nb_samples, timesteps, input_dim)`.
  # Output shape
      3D tensor with shape `(nb_samples, timesteps, units)`.
  # Masking
      This layer supports masking for input data with a variable number
      of timesteps. To introduce masks to your data,
      use an [Embedding](embeddings.md) layer with the `mask_zero` parameter
      set to `True`.
class_docstr_tok: ["An","implementation","of","linear","chain","conditional","random","field","(","CRF",")",".","An","linear","chain","CRF","is","defined","to","maximize","the","following","likelihood","function",":","$","$","L","(","W",",","U",",","b",";","y_1",",","...",",","y_n",")",":","=","rac","{","1","}","{","Z","}","\\sum_","{","y_1",",","...",",","y_n","}","\\exp","(","-a_1","'","y_1","-","a_n","'","y_n","-","\\sum_","{","k=1^n","}","(","(","f","(","x_k","'","W","+","b",")","y_k",")","+","y_1","'","U","y_2",")",")",",","$","$","where",":","$","Z","$",":","normalization","constant","$","x_k",",","y_k","$",":","inputs","and","outputs","This","implementation","has","two","modes","for","optimization",":","1",".","(","`","join","mode","`",")","optimized","by","maximizing","join","likelihood",",","which","is","optimal","in","theory","of","statistics",".","Note","that","in","this","case",",","CRF","must","be","the","output\/last","layer",".","2",".","(","`","marginal","mode","`",")","return","marginal","probabilities","on","each","time","step","and","optimized","via","composition","likelihood","(","product","of","marginal","likelihood",")",",","i.e.",",","using","`","categorical_crossentropy","`","loss",".","Note","that","in","this","case",",","CRF","can","be","either","the","last","layer","or","an","intermediate","layer","(","though","not","explored",")",".","For","prediction","(","test","phrase",")",",","one","can","choose","either","Viterbi","best","path","(","class","indices",")","or","marginal","probabilities","if","probabilities","are","needed",".","However",",","if","one","chooses","*","join","mode","*","for","training",",","Viterbi","output","is","typically","better","than","marginal","output",",","but","the","marginal","output","will","still","perform","reasonably","close",",","while","if","*","marginal","mode","*","is","used","for","training",",","marginal","output","usually","performs","much","better",".","The","default","behavior","is","set","according","to","this","observation",".","In","addition",",","this","implementation","supports","masking","and","accepts","either","onehot","or","sparse","target",".","#","Examples","``","`","python","model","=","Sequential","(",")","model.add","(","Embedding","(","3001",",","300",",","mask_zero=True",")","(","X",")","#","use","learn_mode","=","'join","'",",","test_mode","=","'viterbi","'",",","sparse_target","=","True","(","label","indice","output",")","crf","=","CRF","(","10",",","sparse_target=True",")","model.add","(","crf",")","#","crf.accuracy","is","default","to","Viterbi","acc","if","using","join-mode","(","default",")",".","#","One","can","add","crf.marginal_acc","if","interested",",","but","may","slow","down","learning","model.compile","(","'adam","'",",","loss=crf.loss_function",",","metrics=","[","crf.accuracy","]",")","#","y","must","be","label","indices","(","with","shape","1","at","dim","3",")","here",",","since","`","sparse_target=True","`","model.fit","(","x",",","y",")","#","prediction","give","onehot","representation","of","Viterbi","best","path","y_hat","=","model.predict","(","x_test",")","``","`","#","Arguments","units",":","Positive","integer",",","dimensionality","of","the","output","space",".","learn_mode",":","Either","'join","'","or","'marginal","'",".","The","former","train","the","model","by","maximizing","join","likelihood","while","the","latter","maximize","the","product","of","marginal","likelihood","over","all","time","steps",".","test_mode",":","Either","'viterbi","'","or","'marginal","'",".","The","former","is","recommended","and","as","default","when","`","learn_mode","=","'join","'","`","and","gives","one-hot","representation","of","the","best","path","at","test","(","prediction",")","time",",","while","the","latter","is","recommended","and","chosen","as","default","when","`","learn_mode","=","'marginal","'","`",",","which","produces","marginal","probabilities","for","each","time","step",".","sparse_target",":","Boolean","(","default","False",")","indicating","if","provided","labels","are","one-hot","or","indices","(","with","shape","1","at","dim","3",")",".","use_boundary",":","Boolean","(","default","True",")","indicating","if","trainable","start-end","chain","energies","should","be","added","to","model",".","use_bias",":","Boolean",",","whether","the","layer","uses","a","bias","vector",".","kernel_initializer",":","Initializer","for","the","`","kernel","`","weights","matrix",",","used","for","the","linear","transformation","of","the","inputs",".","(","see","[","initializers","]","(","..","\/initializers.md",")",")",".","chain_initializer",":","Initializer","for","the","`","chain_kernel","`","weights","matrix",",","used","for","the","CRF","chain","energy",".","(","see","[","initializers","]","(","..","\/initializers.md",")",")",".","boundary_initializer",":","Initializer","for","the","`","left_boundary","`",",","'right_boundary","'","weights","vectors",",","used","for","the","start\/left","and","end\/right","boundary","energy",".","(","see","[","initializers","]","(","..","\/initializers.md",")",")",".","bias_initializer",":","Initializer","for","the","bias","vector","(","see","[","initializers","]","(","..","\/initializers.md",")",")",".","activation",":","Activation","function","to","use","(","see","[","activations","]","(","..","\/activations.md",")",")",".","If","you","pass","None",",","no","activation","is","applied","(","ie",".","``","linear","''","activation",":","`","a","(","x",")","=","x","`",")",".","kernel_regularizer",":","Regularizer","function","applied","to","the","`","kernel","`","weights","matrix","(","see","[","regularizer","]","(","..","\/regularizers.md",")",")",".","chain_regularizer",":","Regularizer","function","applied","to","the","`","chain_kernel","`","weights","matrix","(","see","[","regularizer","]","(","..","\/regularizers.md",")",")",".","boundary_regularizer",":","Regularizer","function","applied","to","the","'left_boundary","'",",","'right_boundary","'","weight","vectors","(","see","[","regularizer","]","(","..","\/regularizers.md",")",")",".","bias_regularizer",":","Regularizer","function","applied","to","the","bias","vector","(","see","[","regularizer","]","(","..","\/regularizers.md",")",")",".","kernel_constraint",":","Constraint","function","applied","to","the","`","kernel","`","weights","matrix","(","see","[","constraints","]","(","..","\/constraints.md",")",")",".","chain_constraint",":","Constraint","function","applied","to","the","`","chain_kernel","`","weights","matrix","(","see","[","constraints","]","(","..","\/constraints.md",")",")",".","boundary_constraint",":","Constraint","function","applied","to","the","`","left_boundary","`",",","`","right_boundary","`","weights","vectors","(","see","[","constraints","]","(","..","\/constraints.md",")",")",".","bias_constraint",":","Constraint","function","applied","to","the","bias","vector","(","see","[","constraints","]","(","..","\/constraints.md",")",")",".","input_dim",":","dimensionality","of","the","input","(","integer",")",".","This","argument","(","or","alternatively",",","the","keyword","argument","`","input_shape","`",")","is","required","when","using","this","layer","as","the","first","layer","in","a","model",".","unroll",":","Boolean","(","default","False",")",".","If","True",",","the","network","will","be","unrolled",",","else","a","symbolic","loop","will","be","used",".","Unrolling","can","speed-up","a","RNN",",","although","it","tends","to","be","more","memory-intensive",".","Unrolling","is","only","suitable","for","short","sequences",".","#","Input","shape","3D","tensor","with","shape","`","(","nb_samples",",","timesteps",",","input_dim",")","`",".","#","Output","shape","3D","tensor","with","shape","`","(","nb_samples",",","timesteps",",","units",")","`",".","#","Masking","This","layer","supports","masking","for","input","data","with","a","variable","number","of","timesteps",".","To","introduce","masks","to","your","data",",","use","an","[","Embedding","]","(","embeddings.md",")","layer","with","the","`","mask_zero","`","parameter","set","to","`","True","`","."]
code: def viterbi_acc(self):
      def acc(y_true, y_pred):
          X = self._inbound_nodes[0].input_tensors[0]
          mask = self._inbound_nodes[0].input_masks[0]
          y_pred = self.viterbi_decoding(X, mask)
          return self._get_accuracy(
              y_true, y_pred, mask, self.sparse_target
          )
  
      acc.func_name = "viterbi_acc"
      return acc
  
imports: from __future__ import absolute_import, division
  from ....imports import *
label_desc: Use image node_id 10 for calling the CRF obj's underlying member method code with example usage: obj.viterbi_acc() and returns: acc, self