class_docstr: PackNet implements the PackNet algorithm for parameter isolation. It
  is designed to automatically upgrade most models to support PackNet.
  But because of the nature of the strategy, it is not possible to use it
  with every model or PyTorch module. Furthermore, PackNet not everything
  has been implemented yet. Here are some basic guidelines:
  
   - Stateless modules like :class:`torch.nn.ReLU`, :class:`torch.nn.Flatten`,
      or `torch.nn.Dropout` should work fine.
   - Many normalization layers currently do not work.
   - Supports: :class:`nn.Linear`, :class:`nn.Conv1d`, :class:`nn.Conv2d`,
      :class:`nn.Conv3d`, :class:`nn.ConvTranspose1d`, :class:`nn.ConvTranspose2d`,
      :class:`nn.ConvTranspose3d`
   - If you want to use a custom module with state or parameters, ensure it
      implements :class:`PackNetModule`.
  
  
  Mallya, A., & Lazebnik, S. (2018). PackNet: Adding Multiple Tasks to a
  Single Network by Iterative Pruning. 2018 IEEE/CVF Conference on Computer
  Vision and Pattern Recognition, 7765-7773.
  https://doi.org/10.1109/CVPR.2018.00810
class_docstr_tok: ["PackNet","implements","the","PackNet","algorithm","for","parameter","isolation",".","It","is","designed","to","automatically","upgrade","most","models","to","support","PackNet",".","But","because","of","the","nature","of","the","strategy",",","it","is","not","possible","to","use","it","with","every","model","or","PyTorch","module",".","Furthermore",",","PackNet","not","everything","has","been","implemented","yet",".","Here","are","some","basic","guidelines",":","-","Stateless","modules","like",":","class",":","`","torch.nn.ReLU","`",",",":","class",":","`","torch.nn.Flatten","`",",","or","`","torch.nn.Dropout","`","should","work","fine",".","-","Many","normalization","layers","currently","do","not","work",".","-","Supports",":",":","class",":","`","nn.Linear","`",",",":","class",":","`","nn.Conv1d","`",",",":","class",":","`","nn.Conv2d","`",",",":","class",":","`","nn.Conv3d","`",",",":","class",":","`","nn.ConvTranspose1d","`",",",":","class",":","`","nn.ConvTranspose2d","`",",",":","class",":","`","nn.ConvTranspose3d","`","-","If","you","want","to","use","a","custom","module","with","state","or","parameters",",","ensure","it","implements",":","class",":","`","PackNetModule","`",".","Mallya",",","A.",",","&","Lazebnik",",","S.","(","2018",")",".","PackNet",":","Adding","Multiple","Tasks","to","a","Single","Network","by","Iterative","Pruning",".","2018","IEEE\/CVF","Conference","on","Computer","Vision","and","Pattern","Recognition",",","7765-7773.","https",":","\/\/doi.org\/10.1109\/CVPR.2018.00810"]
docstr: Upgrade a PyTorch module and all of its submodules to be PackNet
  compatible. This is a recursive function that will wrap all submodules
  in a PackNet compatible module.
  
  :param wrappee: The module to wrap
  :raises ValueError: If the module is not supported
  :return: A PackNet compatible module
docstr_tok: ["Upgrade","a","PyTorch","module","and","all","of","its","submodules","to","be","PackNet","compatible",".","This","is","a","recursive","function","that","will","wrap","all","submodules","in","a","PackNet","compatible","module",".",":","param","wrappee",":","The","module","to","wrap",":","raises","ValueError",":","If","the","module","is","not","supported",":","return",":","A","PackNet","compatible","module"]
code: def wrap(wrappee: nn.Module):
      """Upgrade a PyTorch module and all of its submodules to be PackNet
      compatible. This is a recursive function that will wrap all submodules
      in a PackNet compatible module.
  
      :param wrappee: The module to wrap
      :raises ValueError: If the module is not supported
      :return: A PackNet compatible module
      """
      # Weight norm is not supported
      if hasattr(wrappee, "weight_g") and hasattr(wrappee, "weight_v"):
          raise ValueError("PackNet does not support weight norm")
      # Other norms are not supported
      if hasattr(wrappee, "running_mean") or hasattr(
          wrappee, "running_var"
      ):
          raise ValueError(
              "The PackNet implementation does not yet support norms "
              f"{wrappee.__class__.__name__}"
          )
  
      # Recursive cases
      if isinstance(wrappee, PackNetModule):
          return wrappee
      elif isinstance(wrappee, nn.Linear):
          return _PnLinear(wrappee)
      elif isinstance(wrappee, (nn.Conv1d, nn.Conv2d, nn.Conv3d)):
          return _PnConvNd(wrappee)
      elif isinstance(
          wrappee,
          (nn.ConvTranspose1d, nn.ConvTranspose2d, nn.ConvTranspose3d),
      ):
          return _PnConvTransposedNd(wrappee)
      elif isinstance(wrappee, nn.Sequential):
          # Wrap each submodule
          for i, x in enumerate(wrappee):
              wrappee[i] = PackNetModel.wrap(x)
          return wrappee
  
      # If the module has parameters and has not been wrapped yet, then it is
      # not supported
      if len(list(wrappee.parameters(recurse=False))) != 0:
          raise ValueError(
              f"PackNet does not support the module {wrappee.__class__.__name__}"
          )
  
      for submodule_name, submodule in wrappee.named_children():
          setattr(wrappee, submodule_name, PackNetModel.wrap(submodule))
      return wrappee
  
imports: import typing
  import torch
  import torch.nn
  from torch import Tensor
  from torch.nn import functional
  from abc import ABC, abstractmethod
  from enum import Enum
  from avalanche.core import BaseSGDPlugin, Template
  from avalanche.models.dynamic_modules import MultiTaskModule
  from avalanche.models.simple_mlp import SimpleMLP
  from typing import Union
  from avalanche.training.templates.base_sgd import BaseSGDTemplate
label_desc: Use image node_id 1 for calling the PackNetModel obj's underlying member method code with example usage: obj.wrap(wrappee) and returns: wrappee, wrappee, _PnLinear, _PnConvNd, _PnConvTransposedNd, wrappee