class_docstr: Tokenizes raw text in document type columns into TokenizedSentence .
  
  This class represents a non fitted tokenizer. Fitting it will cause the
  internal RuleFactory to construct the rules for tokenizing from the input
  configuration.
  
  Identifies tokens with tokenization open standards. A few rules will help
  customizing it if defaults do not fit user needs.
  
  For extended examples of usage see the `Examples
  <https://github.com/JohnSnowLabs/spark-nlp/blob/master/examples/python/annotation/text/english/document-normalizer/document_normalizer_notebook.ipynb>`__.
  
  ====================== ======================
  Input Annotation types Output Annotation type
  ====================== ======================
  ``DOCUMENT``           ``TOKEN``
  ====================== ======================
  
  Parameters
  ----------
  targetPattern
      Pattern to grab from text as token candidates, by default ``\S+``
  prefixPattern
      Regex with groups and begins with ``\A`` to match target prefix, by
      default ``\A([^\s\w\$\.]*)``
  suffixPattern
      Regex with groups and ends with ``\z`` to match target suffix, by
      default ``([^\s\w]?)([^\s\w]*)\z``
  infixPatterns
      Regex patterns that match tokens within a single target. groups identify
      different sub-tokens. multiple defaults
  exceptions
      Words that won't be affected by tokenization rules
  exceptionsPath
      Path to file containing list of exceptions
  caseSensitiveExceptions
      Whether to care for case sensitiveness in exceptions, by default True
  contextChars
      Character list used to separate from token boundaries, by default ['.',
      ',', ';', ':', '!', '?', '*', '-', '(', ')', '"', "'"]
  splitPattern
      Pattern to separate from the inside of tokens. Takes priority over
      splitChars.
  splitChars
      Character list used to separate from the inside of tokens
  minLength
      Set the minimum allowed length for each token, by default 0
  maxLength
      Set the maximum allowed length for each token, by default 99999
  
  Examples
  --------
  >>> import sparknlp
  >>> from sparknlp.base import *
  >>> from sparknlp.annotator import *
  >>> from pyspark.ml import Pipeline
  >>> data = spark.createDataFrame([["I'd like to say we didn't expect that. Jane's boyfriend."]]).toDF("text")
  >>> documentAssembler = DocumentAssembler().setInputCol("text").setOutputCol("document")
  >>> tokenizer = Tokenizer().setInputCols(["document"]).setOutputCol("token").fit(data)
  >>> pipeline = Pipeline().setStages([documentAssembler, tokenizer]).fit(data)
  >>> result = pipeline.transform(data)
  >>> result.selectExpr("token.result").show(truncate=False)
  +-----------------------------------------------------------------------+
  |output                                                                 |
  +-----------------------------------------------------------------------+
  |[I'd, like, to, say, we, didn't, expect, that, ., Jane's, boyfriend, .]|
  +-----------------------------------------------------------------------+
class_docstr_tok: ["Tokenizes","raw","text","in","document","type","columns","into","TokenizedSentence",".","This","class","represents","a","non","fitted","tokenizer",".","Fitting","it","will","cause","the","internal","RuleFactory","to","construct","the","rules","for","tokenizing","from","the","input","configuration",".","Identifies","tokens","with","tokenization","open","standards",".","A","few","rules","will","help","customizing","it","if","defaults","do","not","fit","user","needs",".","For","extended","examples","of","usage","see","the","`","Examples","<","https",":","\/\/github.com\/JohnSnowLabs\/spark-nlp\/blob\/master\/examples\/python\/annotation\/text\/english\/document-normalizer\/document_normalizer_notebook.ipynb",">","`","__",".","======================","======================","Input","Annotation","types","Output","Annotation","type","======================","======================","``","DOCUMENT","``","``","TOKEN","``","======================","======================","Parameters","--","--","--","--","--","targetPattern","Pattern","to","grab","from","text","as","token","candidates",",","by","default","``","\\S+","``","prefixPattern","Regex","with","groups","and","begins","with","``","\\A","``","to","match","target","prefix",",","by","default","``","\\A","(","[","^\\s\\w\\","$","\\",".","]","*",")","``","suffixPattern","Regex","with","groups","and","ends","with","``","\\z","``","to","match","target","suffix",",","by","default","``","(","[","^\\s\\w","]","?",")","(","[","^\\s\\w","]","*",")","\\z","``","infixPatterns","Regex","patterns","that","match","tokens","within","a","single","target",".","groups","identify","different","sub-tokens",".","multiple","defaults","exceptions","Words","that","wo","n't","be","affected","by","tokenization","rules","exceptionsPath","Path","to","file","containing","list","of","exceptions","caseSensitiveExceptions","Whether","to","care","for","case","sensitiveness","in","exceptions",",","by","default","True","contextChars","Character","list","used","to","separate","from","token","boundaries",",","by","default","[","'",".","'",",","'",",","'",",","'",";","'",",","'",":","'",",","'","!","'",",","'","?","'",",","'","*","'",",","'-","'",",","'","(","'",",","'",")","'",",","'","''","'",",","``","'","''","]","splitPattern","Pattern","to","separate","from","the","inside","of","tokens",".","Takes","priority","over","splitChars",".","splitChars","Character","list","used","to","separate","from","the","inside","of","tokens","minLength","Set","the","minimum","allowed","length","for","each","token",",","by","default","0","maxLength","Set","the","maximum","allowed","length","for","each","token",",","by","default","99999","Examples","--","--","--","--",">",">",">","import","sparknlp",">",">",">","from","sparknlp.base","import","*",">",">",">","from","sparknlp.annotator","import","*",">",">",">","from","pyspark.ml","import","Pipeline",">",">",">","data","=","spark.createDataFrame","(","[","[","``","I","'d","like","to","say","we","did","n't","expect","that",".","Jane","'s","boyfriend",".","``","]","]",")",".toDF","(","``","text","''",")",">",">",">","documentAssembler","=","DocumentAssembler","(",")",".setInputCol","(","``","text","''",")",".setOutputCol","(","``","document","''",")",">",">",">","tokenizer","=","Tokenizer","(",")",".setInputCols","(","[","``","document","''","]",")",".setOutputCol","(","``","token","''",")",".fit","(","data",")",">",">",">","pipeline","=","Pipeline","(",")",".setStages","(","[","documentAssembler",",","tokenizer","]",")",".fit","(","data",")",">",">",">","result","=","pipeline.transform","(","data",")",">",">",">","result.selectExpr","(","``","token.result","''",")",".show","(","truncate=False",")","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","-+","|output","|","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","-+","|","[","I","'d",",","like",",","to",",","say",",","we",",","did","n't",",","expect",",","that",",",".",",","Jane","'s",",","boyfriend",",",".","]","|","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","-+"]
docstr: Sets regex patterns that match tokens within a single target. Groups
  identify different sub-tokens.
  
  Parameters
  ----------
  value : List[str]
      Regex patterns that match tokens within a single target
docstr_tok: ["Sets","regex","patterns","that","match","tokens","within","a","single","target",".","Groups","identify","different","sub-tokens",".","Parameters","--","--","--","--","--","value",":","List","[","str","]","Regex","patterns","that","match","tokens","within","a","single","target"]
code: def setInfixPatterns(self, value):
      """Sets regex patterns that match tokens within a single target. Groups
      identify different sub-tokens.
  
      Parameters
      ----------
      value : List[str]
          Regex patterns that match tokens within a single target
      """
      return self._set(infixPatterns=value)
  
imports: from sparknlp.common import *
label_desc: Use image node_id 10 for calling the Tokenizer obj's underlying member method code with example usage: obj.setInfixPatterns(value) and returns: self