class_docstr: 3D tensor parallel initialization among input.
  
  Args:
      num_group (int): The number of all tensor groups.
      depth (int): Depth of 3D parallelism.
      rank (int): The rank of current process.
      world_size (int): Size of whole communication world.
      config (Config): Running configuration.
      data_parallel_size (int): Size of data parallel.
      pipeline_parallel_size (int): Size of pipeline parallel.
      tensor_parallel_size (int): Size of tensor parallel.
class_docstr_tok: ["3D","tensor","parallel","initialization","among","input",".","Args",":","num_group","(","int",")",":","The","number","of","all","tensor","groups",".","depth","(","int",")",":","Depth","of","3D","parallelism",".","rank","(","int",")",":","The","rank","of","current","process",".","world_size","(","int",")",":","Size","of","whole","communication","world",".","config","(","Config",")",":","Running","configuration",".","data_parallel_size","(","int",")",":","Size","of","data","parallel",".","pipeline_parallel_size","(","int",")",":","Size","of","pipeline","parallel",".","tensor_parallel_size","(","int",")",":","Size","of","tensor","parallel","."]
code: def __init__(self, num_group: int, depth: int, *args):
      super().__init__(*args)
      self.num_group = num_group
      self.depth = depth
  
imports: import math
  import torch.distributed
  from colossalai.legacy.global_variables import tensor_parallel_env
  from colossalai.legacy.registry import DIST_GROUP_INITIALIZER
  from ..parallel_mode import ParallelMode
  from .process_group_initializer import ProcessGroupInitializer
label_desc: Use image node_id 1 to create a new Initializer_3D_Input object from inherited base classes: ProcessGroupInitializer with example: obj = Initializer_3D_Input(num_group, depth)