docstr: Loads a state_dict created by an earlier call to state_dict().
  If ``fp16_optimizer_instance`` was constructed from some ``init_optimizer``,
  whose parameters in turn came from ``model``, it is expected that the user
  will call ``model.load_state_dict()`` before
  ``fp16_optimizer_instance.load_state_dict()`` is called.
  
  Example::
  
      model = torch.nn.Linear(D_in, D_out).cuda().half()
      optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
      optimizer = FP16_Optimizer(optimizer, static_loss_scale = 128.0)
      ...
      checkpoint = torch.load("saved.pth")
      model.load_state_dict(checkpoint['model'])
      optimizer.load_state_dict(checkpoint['optimizer'])
docstr_tok: ["Loads","a","state_dict","created","by","an","earlier","call","to","state_dict","(",")",".","If","``","fp16_optimizer_instance","``","was","constructed","from","some","``","init_optimizer","``",",","whose","parameters","in","turn","came","from","``","model","``",",","it","is","expected","that","the","user","will","call","``","model.load_state_dict","(",")","``","before","``","fp16_optimizer_instance.load_state_dict","(",")","``","is","called",".","Example",":",":","model","=","torch.nn.Linear","(","D_in",",","D_out",")",".cuda","(",")",".half","(",")","optimizer","=","torch.optim.SGD","(","model.parameters","(",")",",","lr=1e-3",")","optimizer","=","FP16_Optimizer","(","optimizer",",","static_loss_scale","=","128.0",")","...","checkpoint","=","torch.load","(","``","saved.pth","''",")","model.load_state_dict","(","checkpoint","[","'model","'","]",")","optimizer.load_state_dict","(","checkpoint","[","'optimizer","'","]",")"]
code: def load_state_dict(self, state_dict):
      """
      Loads a state_dict created by an earlier call to state_dict().
      If ``fp16_optimizer_instance`` was constructed from some ``init_optimizer``,
      whose parameters in turn came from ``model``, it is expected that the user
      will call ``model.load_state_dict()`` before
      ``fp16_optimizer_instance.load_state_dict()`` is called.
  
      Example::
  
          model = torch.nn.Linear(D_in, D_out).cuda().half()
          optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
          optimizer = FP16_Optimizer(optimizer, static_loss_scale = 128.0)
          ...
          checkpoint = torch.load("saved.pth")
          model.load_state_dict(checkpoint['model'])
          optimizer.load_state_dict(checkpoint['optimizer'])
      """
      # I think it should actually be ok to reload the optimizer before the model.
      self.loss_scaler = state_dict["loss_scaler"]
      self.dynamic_loss_scale = state_dict["dynamic_loss_scale"]
      self.overflow = state_dict["overflow"]
      self.first_closure_call_this_step = state_dict[
          "first_closure_call_this_step"
      ]
      self.optimizer.load_state_dict(state_dict["optimizer_state_dict"])
      # At this point, the optimizer's references to the model's fp32 parameters are up to date.
      # The optimizer's hyperparameters and internal buffers are also up to date.
      # However, the fp32 master copies of the model's fp16 params stored by the optimizer are still
      # out of date.  There are two options.
      # 1:  Refresh the master params from the model's fp16 params.
      # This requires less storage but incurs precision loss.
      # 2:  Save and restore the fp32 master copies separately.
      # We choose option 2.
      #
      # Pytorch Optimizer.load_state_dict casts saved buffers (e.g. momentum) to the type and device
      # of their associated parameters, because it's possible those buffers might not exist yet in
      # the current optimizer instance.  In our case, as long as the current FP16_Optimizer has been
      # constructed in the same way as the one whose state_dict we are loading, the same master params
      # are guaranteed to exist, so we can just copy_() from the saved master params.
      for current_group, saved_group in zip(
          self.fp32_from_fp16_groups, state_dict["fp32_from_fp16"]
      ):
          for current, saved in zip(current_group, saved_group):
              current.data.copy_(saved.data)
  
imports: import torch
  from torch import nn
  from torch.autograd import Variable
  from torch.nn.parameter import Parameter
  from torch._utils import _flatten_dense_tensors, _unflatten_dense_tensors
  from ..amp._amp_state import _amp_state, maybe_print
  from ..amp.scaler import LossScaler
  from ..multi_tensor_apply import multi_tensor_applier
  from .fp16util import model_grads_to_master_grads, master_params_to_model_params, clip_grad_norm
label_desc: Use image node_id 9 for calling the FP16_Optimizer obj's underlying member method code with example usage: obj.load_state_dict(state_dict) without return types