class_docstr: Implementation of the distributed helper.
  
  Methods of this class can be used as an high-level wrappers
  aroung the torch.ddistributed API to allow for a simpler
  distributed communication.
  
  Only a single object of this class is instantiated
  as the "DistributedHelper" singleton.
  
  
  Note: differently from the original Pytorch API, which requires
  that input tensor(s) to be moved to the default device (forced to
  CUDA if using NCCL), these functions usually also manage input tensors
  residing on a different devices. The returned elements will
  be moved to the same device of the input tensor. Consider looking at
  the documentation of each method for more details.
class_docstr_tok: ["Implementation","of","the","distributed","helper",".","Methods","of","this","class","can","be","used","as","an","high-level","wrappers","aroung","the","torch.ddistributed","API","to","allow","for","a","simpler","distributed","communication",".","Only","a","single","object","of","this","class","is","instantiated","as","the","``","DistributedHelper","''","singleton",".","Note",":","differently","from","the","original","Pytorch","API",",","which","requires","that","input","tensor","(","s",")","to","be","moved","to","the","default","device","(","forced","to","CUDA","if","using","NCCL",")",",","these","functions","usually","also","manage","input","tensors","residing","on","a","different","devices",".","The","returned","elements","will","be","moved","to","the","same","device","of","the","input","tensor",".","Consider","looking","at","the","documentation","of","each","method","for","more","details","."]
docstr: Gather all for objects. This will also take care of moving cuda tensors
  (even the ones nested inside objects) to the correct default device.
  
  Same as torch.distributed:
      - Tensors nested inside the input object must reside in the
          default device. Future versions of Avalanche may adopt
          solutions to circumvent the limitations of
          orch.distributed.
  
  Differences with torch.distributed:
      - The input object will be transmitted using the current backend.
          However, the resulting tensors nested inside of it
          will be moved to the default device before returning them,
          no matter the backend in use.
      - No-op if not running a distributed training.
  
  :param obj: The object to be sent from the current process.
  :return: A list of objects, one from each process (in rank order).
docstr_tok: ["Gather","all","for","objects",".","This","will","also","take","care","of","moving","cuda","tensors","(","even","the","ones","nested","inside","objects",")","to","the","correct","default","device",".","Same","as","torch.distributed",":","-","Tensors","nested","inside","the","input","object","must","reside","in","the","default","device",".","Future","versions","of","Avalanche","may","adopt","solutions","to","circumvent","the","limitations","of","orch.distributed",".","Differences","with","torch.distributed",":","-","The","input","object","will","be","transmitted","using","the","current","backend",".","However",",","the","resulting","tensors","nested","inside","of","it","will","be","moved","to","the","default","device","before","returning","them",",","no","matter","the","backend","in","use",".","-","No-op","if","not","running","a","distributed","training",".",":","param","obj",":","The","object","to","be","sent","from","the","current","process",".",":","return",":","A","list","of","objects",",","one","from","each","process","(","in","rank","order",")","."]
code: def gather_all_objects(self, obj: BroadcastT) -> List[BroadcastT]:
      """
      Gather all for objects. This will also take care of moving cuda tensors
      (even the ones nested inside objects) to the correct default device.
  
      Same as torch.distributed:
          - Tensors nested inside the input object must reside in the
              default device. Future versions of Avalanche may adopt
              solutions to circumvent the limitations of
              orch.distributed.
  
      Differences with torch.distributed:
          - The input object will be transmitted using the current backend.
              However, the resulting tensors nested inside of it
              will be moved to the default device before returning them,
              no matter the backend in use.
          - No-op if not running a distributed training.
  
      :param obj: The object to be sent from the current process.
      :return: A list of objects, one from each process (in rank order).
      """
      out_list = [None for _ in range(self.world_size)]
      torch.distributed.all_gather_object(out_list, obj)
      return out_list
  
imports: import os
  import pickle
  import warnings
  from io import BytesIO
  from typing import ContextManager, Optional, List, Any, Iterable, Dict, TypeVar
  import torch
  from torch import Tensor
  from torch.nn.modules import Module
  from torch.nn.parallel import DistributedDataParallel
  from typing_extensions import Literal
  from torch.distributed import init_process_group, broadcast_object_list
  from avalanche.distributed.distributed_consistency_verification import hash_tensor
label_desc: Use image node_id 16 for calling the _DistributedHelperCls obj's underlying member method code with example usage: obj.gather_all_objects(obj) and returns: out_list