class_docstr: Increase the learning rate of each parameter group from min lr to max lr
  over warmup_num_steps steps, and then fix at max lr.
  
  Args:
      optimizer (Optimizer): Wrapped optimizer.
      warmup_min_lr (float or list): minimum learning rate. Default: 0
      warmup_max_lr (float or list): maximum learning rate. Default: 0.001
      warmup_num_steps (int): number of steps to warm up from min_lr to max_lr. Default: 1000
      warmup_type {‘log’, ‘linear’}: increasing function from min_lr to max_lr during warmup. Default: log
      last_batch_iteration (int): The index of the last batch. Default: -1.
  Example:
      >>> optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
      >>> scheduler = WarmupLR(optimizer)
      >>> data_loader = torch.utils.data.DataLoader(...)
      >>> for epoch in range(10):
      >>>     for batch in data_loader:
      >>>         train_batch(...)
      >>>         scheduler.step()
class_docstr_tok: ["Increase","the","learning","rate","of","each","parameter","group","from","min","lr","to","max","lr","over","warmup_num_steps","steps",",","and","then","fix","at","max","lr",".","Args",":","optimizer","(","Optimizer",")",":","Wrapped","optimizer",".","warmup_min_lr","(","float","or","list",")",":","minimum","learning","rate",".","Default",":","0","warmup_max_lr","(","float","or","list",")",":","maximum","learning","rate",".","Default",":","0.001","warmup_num_steps","(","int",")",":","number","of","steps","to","warm","up","from","min_lr","to","max_lr",".","Default",":","1000","warmup_type","{","\u2018","log","\u2019",",","\u2018","linear","\u2019","}",":","increasing","function","from","min_lr","to","max_lr","during","warmup",".","Default",":","log","last_batch_iteration","(","int",")",":","The","index","of","the","last","batch",".","Default",":","-1",".","Example",":",">",">",">","optimizer","=","torch.optim.SGD","(","model.parameters","(",")",",","lr=0.1",",","momentum=0.9",")",">",">",">","scheduler","=","WarmupLR","(","optimizer",")",">",">",">","data_loader","=","torch.utils.data.DataLoader","(","...",")",">",">",">","for","epoch","in","range","(","10",")",":",">",">",">","for","batch","in","data_loader",":",">",">",">","train_batch","(","...",")",">",">",">","scheduler.step","(",")"]
code: def get_lr(self):
      if self.last_batch_iteration < 0:
          logger.warning(
              "Attempting to get learning rate from scheduler before it has started"
          )
          return [0.0]
      gamma = self._get_gamma()
      return [
          min_lr + (delta_lr * gamma)
          for min_lr, delta_lr in zip(self.min_lrs, self.delta_lrs)
      ]
  
imports: import argparse
  from torch.optim import Optimizer
  import math
  from deepspeed.utils import logger
label_desc: Use image node_id 2 for calling the WarmupLR obj's underlying member method code with example usage: obj.get_lr() and returns: unknown, list