class_docstr: Serve as the single entry point to 3D parallel initialization.
  
  Args:
      rank (int): The rank of current process.
      world_size (int): Size of whole communication world.
      config (Config): Running configuration.
      data_parallel_size (int): Size of data parallel.
      pipeline_parallel_size (int): Size of pipeline parallel.
      tensor_parallel_size (int): Size of tensor parallel.
class_docstr_tok: ["Serve","as","the","single","entry","point","to","3D","parallel","initialization",".","Args",":","rank","(","int",")",":","The","rank","of","current","process",".","world_size","(","int",")",":","Size","of","whole","communication","world",".","config","(","Config",")",":","Running","configuration",".","data_parallel_size","(","int",")",":","Size","of","data","parallel",".","pipeline_parallel_size","(","int",")",":","Size","of","pipeline","parallel",".","tensor_parallel_size","(","int",")",":","Size","of","tensor","parallel","."]
docstr: Initialize 3D tensor parallel groups, and assign local_ranks and groups to each gpu.
  
  Returns:
      List[Tuple (local_rank, group_world_size, process_group, ranks_in_group, mode)]:
          Whole 3D tensor parallelism's information in a list of tuples.
docstr_tok: ["Initialize","3D","tensor","parallel","groups",",","and","assign","local_ranks","and","groups","to","each","gpu",".","Returns",":","List","[","Tuple","(","local_rank",",","group_world_size",",","process_group",",","ranks_in_group",",","mode",")","]",":","Whole","3D","tensor","parallelism","'s","information","in","a","list","of","tuples","."]
code: def init_dist_group(self):
      """Initialize 3D tensor parallel groups, and assign local_ranks and groups to each gpu.
  
      Returns:
          List[Tuple (local_rank, group_world_size, process_group, ranks_in_group, mode)]:
              Whole 3D tensor parallelism's information in a list of tuples.
      """
      parallel_setting = [
          self.input_initializer.init_dist_group(),
          self.weight_initializer.init_dist_group(),
          self.output_initializer.init_dist_group(),
          self.input_x_weight_initializer.init_dist_group(),
          self.output_x_weight_initializer.init_dist_group(),
      ]
      return parallel_setting
  
imports: import math
  import torch.distributed
  from colossalai.legacy.global_variables import tensor_parallel_env
  from colossalai.legacy.registry import DIST_GROUP_INITIALIZER
  from ..parallel_mode import ParallelMode
  from .process_group_initializer import ProcessGroupInitializer
label_desc: Use image node_id 2 for calling the Initializer_3D obj's underlying member method code with example usage: obj.init_dist_group() and returns: parallel_setting