class_docstr: This tokenizer allows us to tokenize using both spm and word token boundaries. It works as below:
  1) Splits the sentence into candidate tokens delimited by space
  2) Checks if the given token is part of the word based ontology vocab. If yes, it tokenizes it as a
     full word token
  3) If no, it breaks it down using spm voabulary.
class_docstr_tok: ["This","tokenizer","allows","us","to","tokenize","using","both","spm","and","word","token","boundaries",".","It","works","as","below",":","1",")","Splits","the","sentence","into","candidate","tokens","delimited","by","space","2",")","Checks","if","the","given","token","is","part","of","the","word","based","ontology","vocab",".","If","yes",",","it","tokenizes","it","as","a","full","word","token","3",")","If","no",",","it","breaks","it","down","using","spm","voabulary","."]
code: def __init__(
      self,
      ontology_vocab_path: str = "",
      ontology_vocab_lowercase: bool = True,
      *args,
      **kwargs,
  ):
      super().__init__(
          *args,
          **kwargs,
      )
      self.ontology_vocab_path = ontology_vocab_path
      self.ontology_vocab_lowercase = ontology_vocab_lowercase
      self.ontology_vocab = self.load_ontology_vocab(
          self.ontology_vocab_path, self.ontology_vocab_lowercase
      )
      log_class_usage(__class__)
  
imports: import copy
  import json
  import re
  from collections import OrderedDict
  from typing import List, NamedTuple, Optional, Union
  import torch
  from fairseq.data.encoders.gpt2_bpe import get_encoder
  from fairseq.data.encoders.gpt2_bpe_utils import Encoder
  from pytext.config import ConfigBase
  from pytext.config.component import Component, ComponentType, create_component
  from pytext.data.pickleable_gpt2bpe_encoder import PickleableGPT2BPEEncoder
  from pytext.torchscript.tokenizer import ScriptDoNothingTokenizer, ScriptWordTokenizer
  from pytext.utils.file_io import PathManager
  from pytext.utils.usage import log_class_usage
  from transformers.tokenization_bert import BasicTokenizer, WordpieceTokenizer
label_desc: Use image node_id 2 to create a new SPEandWordTokenizer object from inherited base classes: SentencePieceTokenizer with example: obj = SPEandWordTokenizer(ontology_vocab_path, ontology_vocab_lowercase)