docstr: Encodes a string into tokens.
  
  Special tokens are artificial tokens used to unlock capabilities from a model,
  such as fill-in-the-middle. So we want to be careful about accidentally encoding special
  tokens, since they can be used to trick a model into doing something we don't want it to do.
  
  Hence, by default, encode will raise an error if it encounters text that corresponds
  to a special token. This can be controlled on a per-token level using the `allowed_special`
  and `disallowed_special` parameters. In particular:
  - Setting `disallowed_special` to () will prevent this function from raising errors and
    cause all text corresponding to special tokens to be encoded as natural text.
  - Setting `allowed_special` to "all" will cause this function to treat all text
    corresponding to special tokens to be encoded as special tokens.
  
  ```
  >>> enc.encode("hello world")
  [31373, 995]
  >>> enc.encode("<|endoftext|>", allowed_special={"<|endoftext|>"})
  [50256]
  >>> enc.encode("<|endoftext|>", allowed_special="all")
  [50256]
  >>> enc.encode("<|endoftext|>")
  # Raises ValueError
  >>> enc.encode("<|endoftext|>", disallowed_special=())
  [27, 91, 437, 1659, 5239, 91, 29]
  ```
docstr_tok: ["Encodes","a","string","into","tokens",".","Special","tokens","are","artificial","tokens","used","to","unlock","capabilities","from","a","model",",","such","as","fill-in-the-middle",".","So","we","want","to","be","careful","about","accidentally","encoding","special","tokens",",","since","they","can","be","used","to","trick","a","model","into","doing","something","we","do","n't","want","it","to","do",".","Hence",",","by","default",",","encode","will","raise","an","error","if","it","encounters","text","that","corresponds","to","a","special","token",".","This","can","be","controlled","on","a","per-token","level","using","the","`","allowed_special","`","and","`","disallowed_special","`","parameters",".","In","particular",":","-","Setting","`","disallowed_special","`","to","(",")","will","prevent","this","function","from","raising","errors","and","cause","all","text","corresponding","to","special","tokens","to","be","encoded","as","natural","text",".","-","Setting","`","allowed_special","`","to","``","all","''","will","cause","this","function","to","treat","all","text","corresponding","to","special","tokens","to","be","encoded","as","special","tokens",".","``","`",">",">",">","enc.encode","(","``","hello","world","''",")","[","31373",",","995","]",">",">",">","enc.encode","(","``","<","|endoftext|",">","''",",","allowed_special=","{","``","<","|endoftext|",">","''","}",")","[","50256","]",">",">",">","enc.encode","(","``","<","|endoftext|",">","''",",","allowed_special=","''","all","''",")","[","50256","]",">",">",">","enc.encode","(","``","<","|endoftext|",">","''",")","#","Raises","ValueError",">",">",">","enc.encode","(","``","<","|endoftext|",">","''",",","disallowed_special=","(",")",")","[","27",",","91",",","437",",","1659",",","5239",",","91",",","29","]","``","`"]
code: def encode(
      self,
      text: str,
      *,
      allowed_special: Union[
          Literal["all"], AbstractSet[str]
      ] = set(),  # noqa: B006
      disallowed_special: Union[
          Literal["all"], Collection[str]
      ] = "all",
  ) -> list[int]:
      """Encodes a string into tokens.
  
      Special tokens are artificial tokens used to unlock capabilities from a model,
      such as fill-in-the-middle. So we want to be careful about accidentally encoding special
      tokens, since they can be used to trick a model into doing something we don't want it to do.
  
      Hence, by default, encode will raise an error if it encounters text that corresponds
      to a special token. This can be controlled on a per-token level using the `allowed_special`
      and `disallowed_special` parameters. In particular:
      - Setting `disallowed_special` to () will prevent this function from raising errors and
        cause all text corresponding to special tokens to be encoded as natural text.
      - Setting `allowed_special` to "all" will cause this function to treat all text
        corresponding to special tokens to be encoded as special tokens.
  
      ```
      >>> enc.encode("hello world")
      [31373, 995]
      >>> enc.encode("<|endoftext|>", allowed_special={"<|endoftext|>"})
      [50256]
      >>> enc.encode("<|endoftext|>", allowed_special="all")
      [50256]
      >>> enc.encode("<|endoftext|>")
      # Raises ValueError
      >>> enc.encode("<|endoftext|>", disallowed_special=())
      [27, 91, 437, 1659, 5239, 91, 29]
      ```
      """
      if allowed_special == "all":
          allowed_special = self.special_tokens_set
      if disallowed_special == "all":
          disallowed_special = self.special_tokens_set - allowed_special
      if disallowed_special:
          if not isinstance(disallowed_special, frozenset):
              disallowed_special = frozenset(disallowed_special)
          if match := _special_token_regex(disallowed_special).search(
              text
          ):
              raise_disallowed_special_token(match.group())
  
      try:
          return self._core_bpe.encode(text, allowed_special)
      except UnicodeEncodeError:
          # BPE operates on bytes, but the regex operates on unicode. If we pass a str that is
          # invalid UTF-8 to Rust, it will rightfully complain. Here we do a quick and dirty
          # fixup for any surrogate pairs that may have sneaked their way into the text.
          # Technically, this introduces a place where encode + decode doesn't roundtrip a Python
          # string, but given that this is input we want to support, maybe that's okay.
          # Also we use errors="replace" to handle weird things like lone surrogates.
          text = text.encode("utf-16", "surrogatepass").decode(
              "utf-16", "replace"
          )
          return self._core_bpe.encode(text, allowed_special)
  
imports: from __future__ import annotations
  import functools
  from concurrent.futures import ThreadPoolExecutor
  from typing import AbstractSet, Collection, Literal, NoReturn, Optional, Union
  import regex
  from tiktoken import _tiktoken
label_desc: Use image node_id 4 for calling the Encoding obj's underlying member method code with example usage: obj.encode(text) and returns: self, self