class_docstr: BytePS trainer. Supports both multi-GPU and distributed training.
  It achieves better scalability than horovod in distributed training, if the model is communication
  intensive and you have properly set up the machines following its
  `best practices <https://github.com/bytedance/byteps/blob/master/docs/best-practice.md>`_
  which requires a few extra bandwidth servers than horovod.
  
  To use it, switch the trainer, and refer to BytePS documentation on how to
  launch server/scheduler/workers.
  
  Attributes:
      hvd (module): the byteps module that contains horovod-compatible APIs
          like `rank(),size()`.
          This attribute exists so that downstream code that uses these APIs
          does not need to worry about which library is being used under the hood.
class_docstr_tok: ["BytePS","trainer",".","Supports","both","multi-GPU","and","distributed","training",".","It","achieves","better","scalability","than","horovod","in","distributed","training",",","if","the","model","is","communication","intensive","and","you","have","properly","set","up","the","machines","following","its","`","best","practices","<","https",":","\/\/github.com\/bytedance\/byteps\/blob\/master\/docs\/best-practice.md",">","`","_","which","requires","a","few","extra","bandwidth","servers","than","horovod",".","To","use","it",",","switch","the","trainer",",","and","refer","to","BytePS","documentation","on","how","to","launch","server\/scheduler\/workers",".","Attributes",":","hvd","(","module",")",":","the","byteps","module","that","contains","horovod-compatible","APIs","like","`","rank","(",")",",","size","(",")","`",".","This","attribute","exists","so","that","downstream","code","that","uses","these","APIs","does","not","need","to","worry","about","which","library","is","being","used","under","the","hood","."]
docstr: Returns:
      bool: whether hvd is currently running under MPI
docstr_tok: ["Returns",":","bool",":","whether","hvd","is","currently","running","under","MPI"]
code: def mpi_enabled(self):
      """
      Returns:
          bool: whether hvd is currently running under MPI
      """
      return False
  
imports: import multiprocessing
  import os
  import sys
  import tensorflow
  from tensorpack.compat import tfv1
  from ..callbacks import CallbackFactory, RunOp
  from ..graph_builder.distributed import DistributedParameterServerBuilder, DistributedReplicatedBuilder
  from ..graph_builder.training import AsyncMultiGPUBuilder, SyncMultiGPUParameterServerBuilder, SyncMultiGPUReplicatedBuilder
  from ..graph_builder.utils import override_to_local_variable
  from ..input_source import FeedfreeInput, QueueInput
  from ..tfutils import get_global_step_var
  from ..tfutils.common import get_tf_version_tuple
  from ..tfutils.distributed import get_distributed_session_creator
  from ..tfutils.sesscreate import NewSessionCreator
  from ..tfutils.tower import TrainTowerContext
  from ..utils import logger
  from ..utils.argtools import map_arg
  from ..utils.develop import HIDE_DOC, deprecated
  from .tower import SingleCostTrainer
label_desc: Use image node_id 2 for calling the BytePSTrainer obj's underlying member method code with example usage: obj.mpi_enabled() and returns: False