class_docstr: Construct a GPT Tokenizer. Based on Byte-Pair-Encoding with the following peculiarities:
  
  - lowercases all inputs,
  - uses `SpaCy` tokenizer and `ftfy` for pre-BPE tokenization if they are installed, fallback to BERT's
    `BasicTokenizer` if not.
  
  This tokenizer inherits from [`PreTrainedTokenizer`] which contains most of the main methods. Users should refer to
  this superclass for more information regarding those methods.
  
  Args:
      vocab_file (`str`):
          Path to the vocabulary file.
      merges_file (`str`):
          Path to the merges file.
      unk_token (`str`, *optional*, defaults to `"<unk>"`):
          The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
          token instead.
class_docstr_tok: ["Construct","a","GPT","Tokenizer",".","Based","on","Byte-Pair-Encoding","with","the","following","peculiarities",":","-","lowercases","all","inputs",",","-","uses","`","SpaCy","`","tokenizer","and","`","ftfy","`","for","pre-BPE","tokenization","if","they","are","installed",",","fallback","to","BERT's","`","BasicTokenizer","`","if","not",".","This","tokenizer","inherits","from","[","`","PreTrainedTokenizer","`","]","which","contains","most","of","the","main","methods",".","Users","should","refer","to","this","superclass","for","more","information","regarding","those","methods",".","Args",":","vocab_file","(","`","str","`",")",":","Path","to","the","vocabulary","file",".","merges_file","(","`","str","`",")",":","Path","to","the","merges","file",".","unk_token","(","`","str","`",",","*","optional","*",",","defaults","to","`","``","<","unk",">","''","`",")",":","The","unknown","token",".","A","token","that","is","not","in","the","vocabulary","can","not","be","converted","to","an","ID","and","is","set","to","be","this","token","instead","."]
code: def __init__(
      self, vocab_file, merges_file, unk_token="<unk>", **kwargs
  ):
      try:
          import ftfy
          from spacy.lang.en import English
  
          _nlp = English()
          self.nlp = _nlp.tokenizer
          self.fix_text = ftfy.fix_text
      except ImportError:
          logger.warning(
              "ftfy or spacy is not installed using BERT BasicTokenizer instead of SpaCy & ftfy."
          )
          self.nlp = BasicTokenizer(do_lower_case=True)
          self.fix_text = None
  
      with open(vocab_file, encoding="utf-8") as vocab_handle:
          self.encoder = json.load(vocab_handle)
      self.decoder = {v: k for k, v in self.encoder.items()}
      with open(merges_file, encoding="utf-8") as merges_handle:
          merges = merges_handle.read().split("\n")[1:-1]
      merges = [tuple(merge.split()) for merge in merges]
      self.bpe_ranks = dict(zip(merges, range(len(merges))))
      self.cache = {}
  
      super().__init__(unk_token=unk_token, **kwargs)
  
imports: import json
  import os
  import re
  import unicodedata
  from typing import Optional, Tuple
  from ...tokenization_utils import PreTrainedTokenizer, _is_control, _is_punctuation, _is_whitespace
  from ...utils import logging
label_desc: Use image node_id 1 to create a new OpenAIGPTTokenizer object from inherited base classes: PreTrainedTokenizer with example: obj = OpenAIGPTTokenizer(vocab_file, merges_file, unk_token)