docstr: A test to check is dispatching a model on cpu & gpu works correctly using a random `device_map`.
docstr_tok: ["A","test","to","check","is","dispatching","a","model","on","cpu","&","gpu","works","correctly","using","a","random","`","device_map","`","."]
code: def test_cpu_gpu_loading_random_device_map(self):
      r"""
      A test to check is dispatching a model on cpu & gpu works correctly using a random `device_map`.
      """
      device_map = {
          "transformer.word_embeddings": 0,
          "transformer.word_embeddings_layernorm": 0,
          "lm_head": 0,
          "transformer.h.0": "cpu",
          "transformer.h.1": "cpu",
          "transformer.h.2": 0,
          "transformer.h.3": 0,
          "transformer.h.4": 0,
          "transformer.h.5": 0,
          "transformer.h.6": 0,
          "transformer.h.7": 0,
          "transformer.h.8": 0,
          "transformer.h.9": 1,
          "transformer.h.10": 0,
          "transformer.h.11": 1,
          "transformer.h.12": 0,
          "transformer.h.13": 0,
          "transformer.h.14": 1,
          "transformer.h.15": 0,
          "transformer.h.16": 0,
          "transformer.h.17": 1,
          "transformer.h.18": 1,
          "transformer.h.19": 0,
          "transformer.h.20": 1,
          "transformer.h.21": 1,
          "transformer.h.22": 0,
          "transformer.h.23": 0,
          "transformer.ln_f": 1,
      }
  
      bnb_config = BitsAndBytesConfig(
          llm_int8_enable_fp32_cpu_offload=True
      )
  
      model_8bit = AutoModelForCausalLM.from_pretrained(
          self.model_name,
          device_map=device_map,
          quantization_config=bnb_config,
      )
  
      # Check that the model has been correctly set on device 0, 1, and `cpu`.
      self.assertEqual(
          set(model_8bit.hf_device_map.values()), {0, 1, "cpu"}
      )
  
      self.check_inference_correctness(model_8bit)
  
imports: import gc
  import importlib.metadata
  import tempfile
  import unittest
  from packaging import version
  from transformers import AutoConfig, AutoModel, AutoModelForCausalLM, AutoModelForSeq2SeqLM, AutoModelForSequenceClassification, AutoTokenizer, BitsAndBytesConfig, pipeline
  from transformers.testing_utils import is_accelerate_available, is_torch_available, require_accelerate, require_bitsandbytes, require_torch, require_torch_gpu, require_torch_multi_gpu, slow
label_desc: Use image node_id 3 for calling the MixedInt8TestCpuGpu obj's underlying member method code with example usage: obj.test_cpu_gpu_loading_random_device_map() without return types