class_docstr: An Inverted Residual Block, sometimes called an MBConv Block, is a type of residual block used for image models
  that uses an inverted structure for efficiency reasons.
  
  It was originally proposed for the `MobileNetV2 <https://arxiv.org/abs/1801.04381>`__ CNN architecture.
  It has since been reused for several mobile-optimized CNNs.
  It follows a narrow -> wide -> narrow approach, hence the inversion.
  It first widens with a 1x1 convolution, then uses a 3x3 depthwise convolution (which greatly reduces the number of parameters),
  then a 1x1 convolution is used to reduce the number of channels so input and output can be added.
  
  This implementation is sort of a mixture between:
  
  - https://github.com/google-research/google-research/blob/20736344/tunas/rematlib/mobile_model_v3.py#L453
  - https://github.com/rwightman/pytorch-image-models/blob/b7cb8d03/timm/models/efficientnet_blocks.py#L134
  
  Parameters
  ----------
  in_channels
      The number of input channels. Can be a value choice.
  out_channels
      The number of output channels. Can be a value choice.
  expand_ratio
      The ratio of intermediate channels with respect to input channels. Can be a value choice.
  kernel_size
      The kernel size of the depthwise convolution. Can be a value choice.
  stride
      The stride of the depthwise convolution.
  squeeze_excite
      Callable to create squeeze and excitation layer. Take hidden channels and input channels as arguments.
  norm_layer
      Callable to create normalization layer. Take input channels as argument.
  activation_layer
      Callable to create activation layer. No input arguments.
class_docstr_tok: ["An","Inverted","Residual","Block",",","sometimes","called","an","MBConv","Block",",","is","a","type","of","residual","block","used","for","image","models","that","uses","an","inverted","structure","for","efficiency","reasons",".","It","was","originally","proposed","for","the","`","MobileNetV2","<","https",":","\/\/arxiv.org\/abs\/1801.04381",">","`","__","CNN","architecture",".","It","has","since","been","reused","for","several","mobile-optimized","CNNs",".","It","follows","a","narrow","-",">","wide","-",">","narrow","approach",",","hence","the","inversion",".","It","first","widens","with","a","1x1","convolution",",","then","uses","a","3x3","depthwise","convolution","(","which","greatly","reduces","the","number","of","parameters",")",",","then","a","1x1","convolution","is","used","to","reduce","the","number","of","channels","so","input","and","output","can","be","added",".","This","implementation","is","sort","of","a","mixture","between",":","-","https",":","\/\/github.com\/google-research\/google-research\/blob\/20736344\/tunas\/rematlib\/mobile_model_v3.py","#","L453","-","https",":","\/\/github.com\/rwightman\/pytorch-image-models\/blob\/b7cb8d03\/timm\/models\/efficientnet_blocks.py","#","L134","Parameters","--","--","--","--","--","in_channels","The","number","of","input","channels",".","Can","be","a","value","choice",".","out_channels","The","number","of","output","channels",".","Can","be","a","value","choice",".","expand_ratio","The","ratio","of","intermediate","channels","with","respect","to","input","channels",".","Can","be","a","value","choice",".","kernel_size","The","kernel","size","of","the","depthwise","convolution",".","Can","be","a","value","choice",".","stride","The","stride","of","the","depthwise","convolution",".","squeeze_excite","Callable","to","create","squeeze","and","excitation","layer",".","Take","hidden","channels","and","input","channels","as","arguments",".","norm_layer","Callable","to","create","normalization","layer",".","Take","input","channels","as","argument",".","activation_layer","Callable","to","create","activation","layer",".","No","input","arguments","."]
code: def __init__(
      self,
      in_channels: MaybeIntChoice,
      out_channels: MaybeIntChoice,
      expand_ratio: Union[float, MutableExpression[float]],
      kernel_size: MaybeIntChoice = 3,
      stride: int = 1,
      squeeze_excite: Optional[
          Callable[[MaybeIntChoice, MaybeIntChoice], nn.Module]
      ] = None,
      norm_layer: Optional[Callable[[int], nn.Module]] = None,
      activation_layer: Optional[Callable[..., nn.Module]] = None,
  ) -> None:
      super().__init__()
      self.stride = stride
      self.out_channels = out_channels
      assert stride in [1, 2]
  
      hidden_ch = cast(
          int, make_divisible(in_channels * expand_ratio, 8)
      )
  
      # NOTE: this equivalence check (==) does NOT work for ValueChoice, need to use "is"
      self.has_skip = stride == 1 and in_channels is out_channels
  
      layers: List[nn.Module] = [
          # point-wise convolution
          # NOTE: some paper omit this point-wise convolution when stride = 1.
          # In our implementation, if this pw convolution is intended to be omitted,
          # please use SepConv instead.
          ConvBNReLU(
              in_channels,
              hidden_ch,
              kernel_size=1,
              norm_layer=norm_layer,
              activation_layer=activation_layer,
          ),
          # depth-wise
          ConvBNReLU(
              hidden_ch,
              hidden_ch,
              stride=stride,
              kernel_size=kernel_size,
              groups=hidden_ch,
              norm_layer=norm_layer,
              activation_layer=activation_layer,
          ),
          # SE
          squeeze_excite(cast(int, hidden_ch), cast(int, in_channels))
          if squeeze_excite is not None
          else nn.Identity(),
          # pw-linear
          ConvBNReLU(
              hidden_ch,
              out_channels,
              kernel_size=1,
              norm_layer=norm_layer,
              activation_layer=nn.Identity,
          ),
      ]
  
      super().__init__(*simplify_sequential(layers))
  
imports: import math
  from typing import Optional, Callable, List, Tuple, Iterator, Union, cast, overload
  import torch
  from torch import nn
  from nni.mutable import MutableExpression
  from nni.nas.nn.pytorch import ModelSpace, LayerChoice, Repeat, MutableConv2d, MutableLinear, MutableBatchNorm2d
  from .utils.pretrained import load_pretrained_weight
label_desc: Use image node_id 1 to create a new InvertedResidual object from inherited base classes: nn with example: obj = InvertedResidual(in_channels, out_channels, expand_ratio, kernel_size, stride, squeeze_excite, norm_layer, activation_layer)