class_docstr: Represents a scaled version of :class:`NoamLR` that preserves the base
  learning rate of the associated optimizer.
  
  .. math::
      \eta_t = \eta_{base} \min(\sqrt{\frac{T_{warmup}}{t}}, \frac{t}{T_{warmup}})
  
  Essentially, this is Noam learning rate schedule scaled by the square root
  of the number of warmup steps. It was originally proposed and implemented by
  Myle Ott in fairseq under the name ``InverseSquareRootLR``.
  
  It corresponds to increasing the learning rate linearly for the first
  :math:`T_{warmup}` training steps to the base learning rate, and decreasing
  it thereafter proportionally to the inverse square root of the step number.
  
  .. note::
      This scheduler is not chainable.
class_docstr_tok: ["Represents","a","scaled","version","of",":","class",":","`","NoamLR","`","that","preserves","the","base","learning","rate","of","the","associated","optimizer",".","..","math",":",":","\\eta_t","=","\\eta_","{","base","}","\\min","(","\\sqrt","{","\\frac","{","T_","{","warmup","}","}","{","t","}","}",",","\\frac","{","t","}","{","T_","{","warmup","}","}",")","Essentially",",","this","is","Noam","learning","rate","schedule","scaled","by","the","square","root","of","the","number","of","warmup","steps",".","It","was","originally","proposed","and","implemented","by","Myle","Ott","in","fairseq","under","the","name","``","InverseSquareRootLR","``",".","It","corresponds","to","increasing","the","learning","rate","linearly","for","the","first",":","math",":","`","T_","{","warmup","}","`","training","steps","to","the","base","learning","rate",",","and","decreasing","it","thereafter","proportionally","to","the","inverse","square","root","of","the","step","number",".","..","note",":",":","This","scheduler","is","not","chainable","."]
docstr: :param optimizer:
      The associated optimizer.
  :param num_warmup_steps:
      The number of warmup steps.
  :param start_lr:
      The initial warmup learning rate of all parameter groups, or of each
      parameter group respectively.
  :param last_epoch:
      The index of the last epoch.
  :param verbose:
      If ``True``, prints a message to stdout for each update.
docstr_tok: [":","param","optimizer",":","The","associated","optimizer",".",":","param","num_warmup_steps",":","The","number","of","warmup","steps",".",":","param","start_lr",":","The","initial","warmup","learning","rate","of","all","parameter","groups",",","or","of","each","parameter","group","respectively",".",":","param","last_epoch",":","The","index","of","the","last","epoch",".",":","param","verbose",":","If","``","True","``",",","prints","a","message","to","stdout","for","each","update","."]
code: def __init__(
      self,
      optimizer: Optimizer,
      num_warmup_steps: int,
      *,
      start_lr: Union[float, Sequence[float]] = 0.0,
      last_epoch: int = -1,
      verbose: bool = False,
  ) -> None:
      """
      :param optimizer:
          The associated optimizer.
      :param num_warmup_steps:
          The number of warmup steps.
      :param start_lr:
          The initial warmup learning rate of all parameter groups, or of each
          parameter group respectively.
      :param last_epoch:
          The index of the last epoch.
      :param verbose:
          If ``True``, prints a message to stdout for each update.
      """
      if num_warmup_steps == 0:
          raise ValueError("`num_warmup_steps` must be greater than 0.")
  
      self.num_warmup_steps = num_warmup_steps
  
      self.start_lrs = _get_per_param_group(
          optimizer, "start_lr", start_lr
      )
  
      super().__init__(optimizer, last_epoch, verbose)
  
imports: import math
  import warnings
  from abc import ABC, abstractmethod
  from typing import List, Sequence, Union, final
  from torch.optim import Optimizer
  from torch.optim.lr_scheduler import _LRScheduler
  from typing_extensions import TypeAlias
  from fairseq2.typing import finaloverride
label_desc: Use image node_id 1 to create a new MyleLR object from inherited base classes: LRSchedulerBase with example: obj = MyleLR(optimizer, num_warmup_steps)