class_docstr: This tokenizer allows us to tokenize using both spm and word token boundaries. It works as below:
  1) Splits the sentence into candidate tokens delimited by space
  2) Checks if the given token is part of the word based ontology vocab. If yes, it tokenizes it as a
     full word token
  3) If no, it breaks it down using spm voabulary.
class_docstr_tok: ["This","tokenizer","allows","us","to","tokenize","using","both","spm","and","word","token","boundaries",".","It","works","as","below",":","1",")","Splits","the","sentence","into","candidate","tokens","delimited","by","space","2",")","Checks","if","the","given","token","is","part","of","the","word","based","ontology","vocab",".","If","yes",",","it","tokenizes","it","as","a","full","word","token","3",")","If","no",",","it","breaks","it","down","using","spm","voabulary","."]
docstr: Creates the string format of the given list of tokens parsed by SPEandWordTokenizer
  Args:
      token_strs: List of string tokens accepted by SPEandWordTokenizer
  Returns:
      Final string format
docstr_tok: ["Creates","the","string","format","of","the","given","list","of","tokens","parsed","by","SPEandWordTokenizer","Args",":","token_strs",":","List","of","string","tokens","accepted","by","SPEandWordTokenizer","Returns",":","Final","string","format"]
code: def decode(self, sentence: str):
      """Creates the string format of the given list of tokens parsed by SPEandWordTokenizer
      Args:
          token_strs: List of string tokens accepted by SPEandWordTokenizer
      Returns:
          Final string format
      """
      token_list = []
      spm_cont = ""
      # Processes all tokens according to below logic
      # 1) All ontology vocab tokens are treated as separate tokens to be joined by space
      # 2) All other tokens are spm tokens which are conatenated together as one, and then
      #    separated by space "_" is replaced by space
      for token in sentence.split():
          if token.lower() in self.ontology_vocab:
              if spm_cont:
                  token_list.append(spm_cont)
                  spm_cont = ""
              token_list.append(token.lower())
          else:
              spm_cont += token.lower()
      if spm_cont:
          token_list.append(spm_cont)
  
      token_str = " ".join(token_list).replace("▁", " ")
      token_str = " ".join(token_str.split())
  
      return token_str
  
imports: import copy
  import json
  import re
  from collections import OrderedDict
  from typing import List, NamedTuple, Optional, Union
  import torch
  from fairseq.data.encoders.gpt2_bpe import get_encoder
  from fairseq.data.encoders.gpt2_bpe_utils import Encoder
  from pytext.config import ConfigBase
  from pytext.config.component import Component, ComponentType, create_component
  from pytext.data.pickleable_gpt2bpe_encoder import PickleableGPT2BPEEncoder
  from pytext.torchscript.tokenizer import ScriptDoNothingTokenizer, ScriptWordTokenizer
  from pytext.utils.file_io import PathManager
  from pytext.utils.usage import log_class_usage
  from transformers.tokenization_bert import BasicTokenizer, WordpieceTokenizer
label_desc: Use image node_id 5 for calling the SPEandWordTokenizer obj's underlying member method code with example usage: obj.decode(sentence) and returns: token_str