docstr: Load a model from a directory or the HF Hub. Use load_in_4bit or load_in_low_bit parameter
  the weight of model's linears can be loaded to low-bit format, like int4, int5 and int8.
  
  Three new arguments are added to extend Hugging Face's from_pretrained method as follows:
  
  :param load_in_4bit: boolean value, True means loading linear's weight to symmetric int 4 if
                       the model is a regular fp16/bf16/fp32 model, and to asymmetric int 4
                       if the model is GPTQ model.
                       Default to be ``False``.
  :param load_in_low_bit: str value, options are ``'sym_int4'``, ``'asym_int4'``,
                          ``'sym_int5'``, ``'asym_int5'``, ``'sym_int8'``, ``'nf3'``,
                          ``'nf4'``, ``'fp4'``, ``'fp8'``, ``'fp8_e4m3'``, ``'fp8_e5m2'``,
                          ``'fp16'`` or ``'bf16'``, ``'sym_int4'`` means symmetric int 4,
                          ``'asym_int4'`` means asymmetric int 4, ``'nf4'`` means 4-bit
                          NormalFloat, etc. Relevant low bit optimizations will be applied
                          to the model.
  :param optimize_model: boolean value, Whether to further optimize the low_bit llm model.
                         Default to be ``True``.
  :param modules_to_not_convert: list of str value, modules (nn.Module) that are skipped when
                                 conducting model optimizations. Default to be ``None``.
  :param cpu_embedding: Whether to replace the Embedding layer, may need to set it
      to ``True`` when running BigDL-LLM on GPU on Windows. Default to be ``False``.
  :param lightweight_bmm: Whether to replace the torch.bmm ops, may need to set it
      to ``True`` when running BigDL-LLM on GPU on Windows. Default to be ``False``.
  :return: a model instance
docstr_tok: ["Load","a","model","from","a","directory","or","the","HF","Hub",".","Use","load_in_4bit","or","load_in_low_bit","parameter","the","weight","of","model","'s","linears","can","be","loaded","to","low-bit","format",",","like","int4",",","int5","and","int8",".","Three","new","arguments","are","added","to","extend","Hugging","Face","'s","from_pretrained","method","as","follows",":",":","param","load_in_4bit",":","boolean","value",",","True","means","loading","linear","'s","weight","to","symmetric","int","4","if","the","model","is","a","regular","fp16\/bf16\/fp32","model",",","and","to","asymmetric","int","4","if","the","model","is","GPTQ","model",".","Default","to","be","``","False","``",".",":","param","load_in_low_bit",":","str","value",",","options","are","``","'sym_int4","'","``",",","``","'asym_int4","'","``",",","``","'sym_int5","'","``",",","``","'asym_int5","'","``",",","``","'sym_int8","'","``",",","``","'nf3","'","``",",","``","'nf4","'","``",",","``","'fp4","'","``",",","``","'fp8","'","``",",","``","'fp8_e4m3","'","``",",","``","'fp8_e5m2","'","``",",","``","'fp16","'","``","or","``","'bf16","'","``",",","``","'sym_int4","'","``","means","symmetric","int","4",",","``","'asym_int4","'","``","means","asymmetric","int","4",",","``","'nf4","'","``","means","4-bit","NormalFloat",",","etc",".","Relevant","low","bit","optimizations","will","be","applied","to","the","model",".",":","param","optimize_model",":","boolean","value",",","Whether","to","further","optimize","the","low_bit","llm","model",".","Default","to","be","``","True","``",".",":","param","modules_to_not_convert",":","list","of","str","value",",","modules","(","nn.Module",")","that","are","skipped","when","conducting","model","optimizations",".","Default","to","be","``","None","``",".",":","param","cpu_embedding",":","Whether","to","replace","the","Embedding","layer",",","may","need","to","set","it","to","``","True","``","when","running","BigDL-LLM","on","GPU","on","Windows",".","Default","to","be","``","False","``",".",":","param","lightweight_bmm",":","Whether","to","replace","the","torch.bmm","ops",",","may","need","to","set","it","to","``","True","``","when","running","BigDL-LLM","on","GPU","on","Windows",".","Default","to","be","``","False","``",".",":","return",":","a","model","instance"]
code: def from_pretrained(cls, *args, **kwargs):
      """
      Load a model from a directory or the HF Hub. Use load_in_4bit or load_in_low_bit parameter
      the weight of model's linears can be loaded to low-bit format, like int4, int5 and int8.
  
      Three new arguments are added to extend Hugging Face's from_pretrained method as follows:
  
      :param load_in_4bit: boolean value, True means loading linear's weight to symmetric int 4 if
                           the model is a regular fp16/bf16/fp32 model, and to asymmetric int 4
                           if the model is GPTQ model.
                           Default to be ``False``.
      :param load_in_low_bit: str value, options are ``'sym_int4'``, ``'asym_int4'``,
                              ``'sym_int5'``, ``'asym_int5'``, ``'sym_int8'``, ``'nf3'``,
                              ``'nf4'``, ``'fp4'``, ``'fp8'``, ``'fp8_e4m3'``, ``'fp8_e5m2'``,
                              ``'fp16'`` or ``'bf16'``, ``'sym_int4'`` means symmetric int 4,
                              ``'asym_int4'`` means asymmetric int 4, ``'nf4'`` means 4-bit
                              NormalFloat, etc. Relevant low bit optimizations will be applied
                              to the model.
      :param optimize_model: boolean value, Whether to further optimize the low_bit llm model.
                             Default to be ``True``.
      :param modules_to_not_convert: list of str value, modules (nn.Module) that are skipped when
                                     conducting model optimizations. Default to be ``None``.
      :param cpu_embedding: Whether to replace the Embedding layer, may need to set it
          to ``True`` when running BigDL-LLM on GPU on Windows. Default to be ``False``.
      :param lightweight_bmm: Whether to replace the torch.bmm ops, may need to set it
          to ``True`` when running BigDL-LLM on GPU on Windows. Default to be ``False``.
      :return: a model instance
      """
      pretrained_model_name_or_path = (
          kwargs.get("pretrained_model_name_or_path", None)
          if len(args) == 0
          else args[0]
      )
      config_dict, _ = PretrainedConfig.get_config_dict(
          pretrained_model_name_or_path
      )
      bigdl_transformers_low_bit = config_dict.pop(
          "bigdl_transformers_low_bit", False
      )
      invalidInputError(
          not bigdl_transformers_low_bit,
          f"Detected model is a low-bit({bigdl_transformers_low_bit}) model, "
          f"Please use load_low_bit to load this model.",
      )
  
      # For huggingface transformers cls.HF_Model.from_pretrained could only restore the model
      # in the original format, which is not quantized,
      # we can convert the model to quantized later.
      load_in_4bit = kwargs.pop("load_in_4bit", False)
      load_in_low_bit = kwargs.pop("load_in_low_bit", None)
      optimize_model = kwargs.pop("optimize_model", True)
      user_quantization_config = kwargs.pop("quantization_config", None)
  
      if (
          user_quantization_config is not None
          and "BitsAndBytesConfig"
          in str(user_quantization_config.__class__)
      ):
          if user_quantization_config.bnb_4bit_quant_type is not None:
              bnb_4bit_type = (
                  user_quantization_config.bnb_4bit_quant_type
              )
              if bnb_4bit_type == "nf4":
                  load_in_low_bit = "nf4"
              elif bnb_4bit_type == "fp4":
                  warnings.warn(
                      "BigDL LLM QLoRA does not support fp4 now, use default nf4",
                      FutureWarning,
                  )
                  load_in_low_bit = "nf4"
              elif bnb_4bit_type == "int4":
                  load_in_low_bit = "sym_int4"
              elif bnb_4bit_type == "bf16":
                  load_in_low_bit = "bf16"
              else:
                  invalidInputError(
                      False,
                      "Only nf4 or int4 is supported for bnb_4bit_quant_type",
                  )
          else:
              warnings.warn(
                  "bnb_4bit_quant_type is None, use default int4",
                  FutureWarning,
              )
              load_in_low_bit = "sym_int4"
          if user_quantization_config.bnb_4bit_use_double_quant is True:
              warnings.warn(
                  "BigDL LLM QLoRA does not support double quant now, set to False",
                  FutureWarning,
              )
          if (
              user_quantization_config.bnb_4bit_compute_dtype
              is not None
          ):
              bnb_dtype = (
                  user_quantization_config.bnb_4bit_compute_dtype
              )
              if bnb_dtype == torch.float32:
                  kwargs["torch_dtype"] = bnb_dtype
              elif bnb_dtype == torch.bfloat16:
                  kwargs["torch_dtype"] = bnb_dtype
              else:
                  invalidInputError(
                      False,
                      "Only float32 or bfloat16"
                      " is supported for bnb_4bit_compute_dtype",
                  )
          else:
              warnings.warn(
                  "torch_dtype is None, use default float32",
                  FutureWarning,
              )
              kwargs["torch_dtype"] = torch.float32
          optimize_model = False
          kwargs["modules_to_not_convert"] = ["lm_head"]
  
      if load_in_4bit or load_in_low_bit:
          if config_dict.get("quantization_config", None) is not None:
              from bigdl.llm.transformers.low_bit_linear import (
                  get_block_size,
              )
  
              q_config = config_dict["quantization_config"]
              if q_config["quant_method"] == "gptq":
                  invalidInputError(
                      q_config["bits"] == 4,
                      "Only 4-bit gptq is supported in bigdl-llm.",
                  )
                  invalidInputError(
                      q_config["desc_act"] is False,
                      "Only desc_act=False is supported in bigdl-llm.",
                  )
                  if load_in_low_bit is not None:
                      invalidInputError(
                          load_in_low_bit == "asym_int4",
                          "You can only load gptq model as aysm_int4 low bit type.",
                      )
  
                  load_in_low_bit = "asym_int4"
                  if (
                      int(q_config["group_size"])
                      % get_block_size(load_in_low_bit)
                      != 0
                  ):
                      invalidInputError(
                          False,
                          (
                              f"group_size must be divisible by "
                              f"{get_block_size(load_in_low_bit)}."
                          ),
                      )
                  if user_quantization_config is not None:
                      invalidInputError(
                          user_quantization_config.bits == 4,
                          "Only 4-bit gptq is supported in bigdl-llm.",
                      )
                      invalidInputError(
                          user_quantization_config.use_exllama is False,
                          "Only use_exllama=False is supported in bigdl-llm.",
                      )
                  else:
                      from transformers import GPTQConfig
  
                      user_quantization_config = GPTQConfig(
                          bits=4, use_exllama=False
                      )
                  kwargs[
                      "quantization_config"
                  ] = user_quantization_config
              elif q_config["quant_method"] == "awq":
                  from bigdl.llm.transformers.awq.awq_config import (
                      AwqConfig,
                  )
  
                  awq_config = AwqConfig.from_dict(q_config)
                  invalidInputError(
                      awq_config.bits == 4,
                      "Only 4-bit awq is supported in bigdl-llm.",
                  )
                  invalidInputError(
                      awq_config.version == "gemm",
                      "Only gemm version is supported in bigdl-llm.",
                  )
                  invalidInputError(
                      awq_config.zero_point,
                      "Only awq zero_point = True is supported in bigdl-llm.",
                  )
                  if load_in_low_bit is not None:
                      invalidInputError(
                          load_in_low_bit == "asym_int4",
                          "You can only load awq model as aysm_int4 low bit type.",
                      )
  
                  load_in_low_bit = "asym_int4"
  
                  if (
                      int(awq_config.group_size)
                      % get_block_size(load_in_low_bit)
                      != 0
                  ):
                      invalidInputError(
                          False,
                          (
                              f"group_size must be divisible by "
                              f"{get_block_size(load_in_low_bit)}."
                          ),
                      )
  
                  kwargs["quantization_config"] = awq_config
  
          # load int x-bit
          kwargs["low_cpu_mem_usage"] = True
          # set default torch_dtype='auto'
          kwargs["torch_dtype"] = kwargs.get("torch_dtype", "auto")
          # Avoid tensor parallel F.Linear Operations
          if "pretraining_tp" in config_dict:
              if "config" in kwargs:
                  setattr(kwargs["config"], "pretraining_tp", 1)
              else:
                  kwargs["pretraining_tp"] = 1
          q_k = load_in_low_bit if load_in_low_bit else "sym_int4"
          model = cls.load_convert(q_k, optimize_model, *args, **kwargs)
      else:
          # load default
          model = cls.HF_Model.from_pretrained(*args, **kwargs)
  
      return model
  
imports: import transformers
  from transformers.configuration_utils import PretrainedConfig
  from .utils import extract_local_archive_file, load_state_dict, get_local_shard_files
  from bigdl.llm.ggml.quantize import ggml_tensor_qtype
  from bigdl.llm.utils.common import invalidInputError
  from bigdl.llm.transformers.gguf.api import load_gguf_model
  import torch
  import warnings
  import copy
  from .utils import logger
label_desc: Use image node_id 1 for calling the _BaseAutoModelClass obj's underlying member method code with example usage: obj.from_pretrained(cls) and returns: model