class_docstr: Interface that wraps a distributed training framework (Horovod, DDP).
  
  Distributed strategies modify the model and/or optimizer to coordinate gradient updates among multiple workers
  running in parallel. In most cases, these are using collective communication libraries pass messages between
  processes.
class_docstr_tok: ["Interface","that","wraps","a","distributed","training","framework","(","Horovod",",","DDP",")",".","Distributed","strategies","modify","the","model","and\/or","optimizer","to","coordinate","gradient","updates","among","multiple","workers","running","in","parallel",".","In","most","cases",",","these","are","using","collective","communication","libraries","pass","messages","between","processes","."]
code: def allreduce(self, t: torch.Tensor) -> torch.Tensor:
      pass
  
imports: from __future__ import annotations
  import contextlib
  from abc import ABC, abstractmethod
  from typing import Any, Callable, TYPE_CHECKING
  import torch
  from torch import nn
  from torch.optim import Optimizer
  from ludwig.modules.optimization_modules import create_optimizer
  from ludwig.utils.torch_utils import get_torch_device
label_desc: Use image node_id 14 for calling the DistributedStrategy obj's underlying member method code with example usage: obj.allreduce(t) without return types