docstr: A simple test to check if 4-bit RWKV inference works as expected.
docstr_tok: ["A","simple","test","to","check","if","4-bit","RWKV","inference","works","as","expected","."]
code: def test_rwkv_4bit(self):
      r"""
      A simple test to check if 4-bit RWKV inference works as expected.
      """
      model_id = "RWKV/rwkv-4-169m-pile"
  
      quantization_config = BitsAndBytesConfig(
          load_in_4bit=True, bnb_4bit_use_double_quant=True
      )
  
      model = AutoModelForCausalLM.from_pretrained(
          model_id, quantization_config=quantization_config
      )
      tok = AutoTokenizer.from_pretrained(model_id)
  
      text = "Hello my name is"
      input_ids = tok.encode(text, return_tensors="pt").to(0)
  
      _ = model.generate(input_ids, max_new_tokens=30)
  
imports: import gc
  import importlib.metadata
  import tempfile
  import unittest
  from packaging import version
  from transformers import AutoConfig, AutoModel, AutoModelForCausalLM, AutoModelForSeq2SeqLM, AutoModelForSequenceClassification, AutoTokenizer, BitsAndBytesConfig, pipeline
  from transformers.testing_utils import is_bitsandbytes_available, is_torch_available, require_accelerate, require_bitsandbytes, require_torch, require_torch_gpu, require_torch_multi_gpu, slow, torch_device
label_desc: Use image node_id 8 for calling the Bnb4BitTest obj's underlying member method code with example usage: obj.test_rwkv_4bit() without return types