class_docstr: NLP Adapter Mixin that can augment any transformer-based model with Adapter module support.
  This mixin class should be used only with a top level ModelPT subclass, that includes either a `model` or an `enc_dec_model` submodule.
  This mixin class adds several utility methods to add, load and save adapters.
  
  An Adapter module is any Pytorch nn.Module that possess a few properties :
  
  - It's input and output dimension are the same, while the hidden dimension need not be the same.
  - The final layer of the Adapter module is zero-initialized, so that the residual connection to the adapter yields the original output.
  
  This mixin class aims to integrate with PEFT, which is one or more adapters modules.
  The two features of PEFT, layer selection and weight tying, are also supported in this mixin class.
class_docstr_tok: ["NLP","Adapter","Mixin","that","can","augment","any","transformer-based","model","with","Adapter","module","support",".","This","mixin","class","should","be","used","only","with","a","top","level","ModelPT","subclass",",","that","includes","either","a","`","model","`","or","an","`","enc_dec_model","`","submodule",".","This","mixin","class","adds","several","utility","methods","to","add",",","load","and","save","adapters",".","An","Adapter","module","is","any","Pytorch","nn.Module","that","possess","a","few","properties",":","-","It","'s","input","and","output","dimension","are","the","same",",","while","the","hidden","dimension","need","not","be","the","same",".","-","The","final","layer","of","the","Adapter","module","is","zero-initialized",",","so","that","the","residual","connection","to","the","adapter","yields","the","original","output",".","This","mixin","class","aims","to","integrate","with","PEFT",",","which","is","one","or","more","adapters","modules",".","The","two","features","of","PEFT",",","layer","selection","and","weight","tying",",","are","also","supported","in","this","mixin","class","."]
docstr: ModelPT override. Optimizer will get self._optimizer_param_groups.
  Makes two optimizer param groups, one for the frozen model params
  and one for the prompt-table/prompt-encoder params. The learning
  rate for the frozen model's params will always be zero effectively
  freezing the model's params but still allowing for the needed gradients
  to be passed around in pipeline parallel models. The prompt-encoder
  and/or prompt table will use the learning rate set by the user.
docstr_tok: ["ModelPT","override",".","Optimizer","will","get","self._optimizer_param_groups",".","Makes","two","optimizer","param","groups",",","one","for","the","frozen","model","params","and","one","for","the","prompt-table\/prompt-encoder","params",".","The","learning","rate","for","the","frozen","model","'s","params","will","always","be","zero","effectively","freezing","the","model","'s","params","but","still","allowing","for","the","needed","gradients","to","be","passed","around","in","pipeline","parallel","models",".","The","prompt-encoder","and\/or","prompt","table","will","use","the","learning","rate","set","by","the","user","."]
code: def setup_optimizer_param_groups(self):
      """
      ModelPT override. Optimizer will get self._optimizer_param_groups.
      Makes two optimizer param groups, one for the frozen model params
      and one for the prompt-table/prompt-encoder params. The learning
      rate for the frozen model's params will always be zero effectively
      freezing the model's params but still allowing for the needed gradients
      to be passed around in pipeline parallel models. The prompt-encoder
      and/or prompt table will use the learning rate set by the user.
      """
      if self.use_peft:
          self.freeze()  # Freeze the entire model
          opt_params = []
          for _, module in self.named_modules():
              if (
                  isinstance(module, AdapterModuleMixin)
                  and module.is_adapter_available()
              ):
                  module.set_enabled_adapters(enabled=True)
                  module.unfreeze_enabled_adapters()  # selectively unfreeze the adapter modules.
                  opt_params += [
                      p for p in module.parameters() if p.requires_grad
                  ]
  
          for name, param in self.named_parameters():
              if name in self.tunable_base_param_keys:
                  param.requires_grad = True
                  opt_params += [param]
  
          self._optimizer_param_groups = ({"params": opt_params},)
          logging.info(f"Optimizer groups set:\n{self.summarize()}")
      else:
          super().setup_optimizer_param_groups()
  
imports: import os
  import tempfile
  from typing import List, Optional, Union
  import torch
  from omegaconf import DictConfig, OmegaConf, open_dict
  from nemo.utils.model_utils import inject_model_parallel_rank
  from nemo.collections.nlp.modules.common.megatron.adapters.parallel_adapters import PromptEncoderAdapterConfig
  from nemo.collections.nlp.parts.peft_config import PEFT_CONFIG_MAP, CanonicalAdaptersPEFTConfig, LoraPEFTConfig, PEFTConfig, PtuningPEFTConfig
  from nemo.core.classes.mixins.adapter_mixins import AdapterModuleMixin
  from nemo.core.connectors.save_restore_connector import SaveRestoreConnector
  from nemo.utils import logging, model_utils
label_desc: Use image node_id 8 for calling the NLPAdapterModelMixin obj's underlying member method code with example usage: obj.setup_optimizer_param_groups() without return types