class_docstr: Multi-head classifier with separate heads for each task.
  
  Typically used in task-incremental benchmarks where task labels are
  available and provided to the model.
  
  .. note::
      Each output head may have a different shape, and the number of
      classes can be determined automatically.
  
      However, since pytorch doest not support jagged tensors, when you
      compute a minibatch's output you must ensure that each sample
      has the same output size, otherwise the model will fail to
      concatenate the samples together.
  
      These can be easily ensured in two possible ways:
  
      - each minibatch contains a single task, which is the case in most
          common benchmarks in Avalanche. Some exceptions to this setting
          are multi-task replay or cumulative strategies.
      - each head has the same size, which can be enforced by setting a
          large enough `initial_out_features`.
class_docstr_tok: ["Multi-head","classifier","with","separate","heads","for","each","task",".","Typically","used","in","task-incremental","benchmarks","where","task","labels","are","available","and","provided","to","the","model",".","..","note",":",":","Each","output","head","may","have","a","different","shape",",","and","the","number","of","classes","can","be","determined","automatically",".","However",",","since","pytorch","doest","not","support","jagged","tensors",",","when","you","compute","a","minibatch","'s","output","you","must","ensure","that","each","sample","has","the","same","output","size",",","otherwise","the","model","will","fail","to","concatenate","the","samples","together",".","These","can","be","easily","ensured","in","two","possible","ways",":","-","each","minibatch","contains","a","single","task",",","which","is","the","case","in","most","common","benchmarks","in","Avalanche",".","Some","exceptions","to","this","setting","are","multi-task","replay","or","cumulative","strategies",".","-","each","head","has","the","same","size",",","which","can","be","enforced","by","setting","a","large","enough","`","initial_out_features","`","."]
docstr: Init.
  
  :param in_features: number of input features.
  :param initial_out_features: initial number of classes (can be
      dynamically expanded).
  :param masking: whether unused units should be masked (default=True).
  :param mask_value: the value used for masked units (default=-1000).
docstr_tok: ["Init",".",":","param","in_features",":","number","of","input","features",".",":","param","initial_out_features",":","initial","number","of","classes","(","can","be","dynamically","expanded",")",".",":","param","masking",":","whether","unused","units","should","be","masked","(","default=True",")",".",":","param","mask_value",":","the","value","used","for","masked","units","(","default=-1000",")","."]
code: def __init__(
      self,
      in_features,
      initial_out_features=2,
      masking=True,
      mask_value=-1000,
  ):
      """Init.
  
      :param in_features: number of input features.
      :param initial_out_features: initial number of classes (can be
          dynamically expanded).
      :param masking: whether unused units should be masked (default=True).
      :param mask_value: the value used for masked units (default=-1000).
      """
      super().__init__()
      self.masking = masking
      self.mask_value = mask_value
      self.in_features = in_features
      self.starting_out_features = initial_out_features
      self.classifiers = torch.nn.ModuleDict()
  
      # needs to create the first head because pytorch optimizers
      # fail when model.parameters() is empty.
      # masking in IncrementalClassifier is unaware of task labels
      # so we do masking here instead.
      first_head = IncrementalClassifier(
          self.in_features, self.starting_out_features, masking=False
      )
      self.classifiers["0"] = first_head
      self.max_class_label = max(
          self.max_class_label, initial_out_features
      )
  
      au_init = torch.zeros(initial_out_features, dtype=torch.int8)
      self.register_buffer("active_units_T0", au_init)
  
imports: import torch
  from torch.nn import Module
  from typing import Optional
  from avalanche.benchmarks.utils.flat_data import ConstantSequence
  from avalanche.benchmarks.scenarios import CLExperience
label_desc: Use image node_id 1 to create a new MultiHeadClassifier object from inherited base classes: MultiTaskModule with example: obj = MultiHeadClassifier(in_features, initial_out_features, masking, mask_value)