class_docstr: The search space proposed by `ProxylessNAS <https://arxiv.org/abs/1812.00332>`__.
  
  Following the official implementation, the inverted residual with kernel size / expand ratio variations in each layer
  is implemented with a :class:`~nni.retiarii.nn.pytorch.LayerChoice` with all-combination candidates. That means,
  when used in weight sharing, these candidates will be treated as separate layers, and won't be fine-grained shared.
  We note that :class:`MobileNetV3Space` is different in this perspective.
  
  This space can be implemented as part of :class:`MobileNetV3Space`, but we separate those following conventions.
  
  Parameters
  ----------
  num_labels
      The number of labels for classification.
  base_widths
      Widths of each stage, from stem, to body, to head. Length should be 9.
  dropout_rate
      Dropout rate for the final classification layer.
  width_mult
      Width multiplier for the model.
  bn_eps
      Epsilon for batch normalization.
  bn_momentum
      Momentum for batch normalization.
class_docstr_tok: ["The","search","space","proposed","by","`","ProxylessNAS","<","https",":","\/\/arxiv.org\/abs\/1812.00332",">","`","__",".","Following","the","official","implementation",",","the","inverted","residual","with","kernel","size","\/","expand","ratio","variations","in","each","layer","is","implemented","with","a",":","class",":","`","~nni.retiarii.nn.pytorch.LayerChoice","`","with","all-combination","candidates",".","That","means",",","when","used","in","weight","sharing",",","these","candidates","will","be","treated","as","separate","layers",",","and","wo","n't","be","fine-grained","shared",".","We","note","that",":","class",":","`","MobileNetV3Space","`","is","different","in","this","perspective",".","This","space","can","be","implemented","as","part","of",":","class",":","`","MobileNetV3Space","`",",","but","we","separate","those","following","conventions",".","Parameters","--","--","--","--","--","num_labels","The","number","of","labels","for","classification",".","base_widths","Widths","of","each","stage",",","from","stem",",","to","body",",","to","head",".","Length","should","be","9.","dropout_rate","Dropout","rate","for","the","final","classification","layer",".","width_mult","Width","multiplier","for","the","model",".","bn_eps","Epsilon","for","batch","normalization",".","bn_momentum","Momentum","for","batch","normalization","."]
code: def __init__(
      self,
      num_labels: int = 1000,
      base_widths: Tuple[int, ...] = (
          32,
          16,
          32,
          40,
          80,
          96,
          192,
          320,
          1280,
      ),
      dropout_rate: float = 0.0,
      width_mult: float = 1.0,
      bn_eps: float = 1e-3,
      bn_momentum: float = 0.1,
  ):
      super().__init__()
  
      assert len(base_widths) == 9
      # include the last stage info widths here
      widths = [
          make_divisible(width * width_mult, 8) for width in base_widths
      ]
      downsamples = [True, False, True, True, True, False, True, False]
  
      self.num_labels = num_labels
      self.dropout_rate = dropout_rate
      self.bn_eps = bn_eps
      self.bn_momentum = bn_momentum
  
      self.stem = ConvBNReLU(
          3, widths[0], stride=2, norm_layer=MutableBatchNorm2d
      )
  
      blocks: List[nn.Module] = [
          # first stage is fixed
          DepthwiseSeparableConv(
              widths[0], widths[1], kernel_size=3, stride=1
          )
      ]
  
      # https://github.com/ultmaster/AceNAS/blob/46c8895fd8a05ffbc61a6b44f1e813f64b4f66b7/searchspace/proxylessnas/__init__.py#L21
      for stage in range(2, 8):
          # Rather than returning a fixed module here,
          # we return a builder that dynamically creates module for different `repeat_idx`.
          builder = inverted_residual_choice_builder(
              [3, 6],
              [3, 5, 7],
              downsamples[stage],
              widths[stage - 1],
              widths[stage],
              f"s{stage}",
          )
          if stage < 7:
              blocks.append(
                  Repeat(builder, (1, 4), label=f"s{stage}_depth")
              )
          else:
              # No mutation for depth in the last stage.
              # Directly call builder to initiate one block
              blocks.append(builder(0))
  
      self.blocks = nn.Sequential(*blocks)
  
      # final layers
      self.feature_mix_layer = ConvBNReLU(
          widths[7],
          widths[8],
          kernel_size=1,
          norm_layer=MutableBatchNorm2d,
      )
      self.global_avg_pooling = nn.AdaptiveAvgPool2d(1)
      self.dropout_layer = nn.Dropout(dropout_rate)
      self.classifier = MutableLinear(widths[-1], num_labels)
  
      reset_parameters(self, bn_momentum=bn_momentum, bn_eps=bn_eps)
  
imports: import math
  from typing import Optional, Callable, List, Tuple, Iterator, Union, cast, overload
  import torch
  from torch import nn
  from nni.mutable import MutableExpression
  from nni.nas.nn.pytorch import ModelSpace, LayerChoice, Repeat, MutableConv2d, MutableLinear, MutableBatchNorm2d
  from .utils.pretrained import load_pretrained_weight
label_desc: Use image node_id 1 to create a new ProxylessNAS object from inherited base classes: ModelSpace with example: obj = ProxylessNAS(num_labels, base_widths, dropout_rate, width_mult, bn_eps, bn_momentum)