class_docstr: Multiscale Vision Transformers
  Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, Christoph Feichtenhofer
  https://arxiv.org/abs/2104.11227
class_docstr_tok: ["Multiscale","Vision","Transformers","Haoqi","Fan",",","Bo","Xiong",",","Karttikeya","Mangalam",",","Yanghao","Li",",","Zhicheng","Yan",",","Jitendra","Malik",",","Christoph","Feichtenhofer","https",":","\/\/arxiv.org\/abs\/2104.11227"]
code: def forward(self, x):
      x = self.patch_embed(x)
  
      t = self.num_frames // self.patch_stride[0]
      h = self.train_crop_size // self.patch_stride[1]
      w = self.train_crop_size // self.patch_stride[2]
      b, _, _ = x.shape
  
      if self.cls_embed_on:
          cls_tokens = self.cls_token.expand(b, -1, -1)
          x = torch.cat((cls_tokens, x), dim=1)
  
      if self.sep_pos_embed:
          pos_embed = self.pos_embed_spatial.repeat(
              1, self.patch_dims[0], 1
          ) + torch.repeat_interleave(
              self.pos_embed_temporal,
              self.patch_dims[1] * self.patch_dims[2],
              dim=1,
          )
          pos_embed_cls = torch.cat(
              [self.pos_embed_class, pos_embed], 1
          )
          x = self.add([x, pos_embed_cls])
      else:
          x = self.add([x, self.pos_embed])
  
      if self.drop_rate:
          x = self.pos_drop(x)
  
      if self.norm_stem:
          x = self.norm_stem(x)
  
      thw = [t, h, w]
      for blk in self.blocks:
          x, thw = blk(x, thw)
      self.out_thw = thw
      x = self.norm(x)
      if self.cls_embed_on:
          x = self.index_select(
              x, dim=1, indices=torch.tensor(0, device=x.device)
          )
          x = x.squeeze(1)
      else:
          x = x.mean(1)
  
      x = self.head(x)
      return x
  
imports: import math
  from functools import partial
  from typing import Tuple, List
  import numpy
  import torch
  from torch import nn
  from torch.nn.init import trunc_normal_
  from einops import rearrange
  from towhee.models.layers.layers_with_relprop import GELU, Linear, Dropout, Conv2d, Conv3d, Softmax, Sigmoid, LayerNorm, Einsum, Add, Clone, MaxPool3d, IndexSelect
label_desc: Use image node_id 3 for calling the MViT obj's underlying member method code with example usage: obj.forward(x) and returns: x