class_docstr: Dictionary encapsulates the mapping between normalized words and their integer ids.
  
  Notable instance attributes:
  
  Attributes
  ----------
  token2id : dict of (str, int)
      token -> token_id. I.e. the reverse mapping to `self[token_id]`.
  cfs : dict of (int, int)
      Collection frequencies: token_id -> how many instances of this token are contained in the documents.
  dfs : dict of (int, int)
      Document frequencies: token_id -> how many documents contain this token.
  num_docs : int
      Number of documents processed.
  num_pos : int
      Total number of corpus positions (number of processed words).
  num_nnz : int
      Total number of non-zeroes in the BOW matrix (sum of the number of unique
      words per document over the entire corpus).
class_docstr_tok: ["Dictionary","encapsulates","the","mapping","between","normalized","words","and","their","integer","ids",".","Notable","instance","attributes",":","Attributes","--","--","--","--","--","token2id",":","dict","of","(","str",",","int",")","token","-",">","token_id",".","I.e",".","the","reverse","mapping","to","`","self","[","token_id","]","`",".","cfs",":","dict","of","(","int",",","int",")","Collection","frequencies",":","token_id","-",">","how","many","instances","of","this","token","are","contained","in","the","documents",".","dfs",":","dict","of","(","int",",","int",")","Document","frequencies",":","token_id","-",">","how","many","documents","contain","this","token",".","num_docs",":","int","Number","of","documents","processed",".","num_pos",":","int","Total","number","of","corpus","positions","(","number","of","processed","words",")",".","num_nnz",":","int","Total","number","of","non-zeroes","in","the","BOW","matrix","(","sum","of","the","number","of","unique","words","per","document","over","the","entire","corpus",")","."]
docstr: Filter out tokens in the dictionary by their frequency.
  
  Parameters
  ----------
  no_below : int, optional
      Keep tokens which are contained in at least `no_below` documents.
  no_above : float, optional
      Keep tokens which are contained in no more than `no_above` documents
      (fraction of total corpus size, not an absolute number).
  keep_n : int, optional
      Keep only the first `keep_n` most frequent tokens.
  keep_tokens : iterable of str
      Iterable of tokens that **must** stay in dictionary after filtering.
  
  Notes
  -----
  This removes all tokens in the dictionary that are:
  
  #. Less frequent than `no_below` documents (absolute number, e.g. `5`) or 
  
  #. More frequent than `no_above` documents (fraction of the total corpus size, e.g. `0.3`).
  #. After (1) and (2), keep only the first `keep_n` most frequent tokens (or keep all if `keep_n=None`).
  
  After the pruning, resulting gaps in word ids are shrunk.
  Due to this gap shrinking, **the same word may have a different word id before and after the call
  to this function!** See :class:`gensim.models.VocabTransform` and the
  `dedicated FAQ entry <https://github.com/RaRe-Technologies/gensim/wiki/Recipes-&-FAQ#q8-how-can-i-filter-a-saved-corpus-and-its-corresponding-dictionary>`_ on how  # noqa
  to transform a corpus built with a dictionary before pruning.
  
  Examples
  --------
  .. sourcecode:: pycon
  
      >>> from gensim.corpora import Dictionary
      >>>
      >>> corpus = [["máma", "mele", "maso"], ["ema", "má", "máma"]]
      >>> dct = Dictionary(corpus)
      >>> len(dct)
      5
      >>> dct.filter_extremes(no_below=1, no_above=0.5, keep_n=1)
      >>> len(dct)
      1
docstr_tok: ["Filter","out","tokens","in","the","dictionary","by","their","frequency",".","Parameters","--","--","--","--","--","no_below",":","int",",","optional","Keep","tokens","which","are","contained","in","at","least","`","no_below","`","documents",".","no_above",":","float",",","optional","Keep","tokens","which","are","contained","in","no","more","than","`","no_above","`","documents","(","fraction","of","total","corpus","size",",","not","an","absolute","number",")",".","keep_n",":","int",",","optional","Keep","only","the","first","`","keep_n","`","most","frequent","tokens",".","keep_tokens",":","iterable","of","str","Iterable","of","tokens","that","*","*","must","*","*","stay","in","dictionary","after","filtering",".","Notes","--","--","-","This","removes","all","tokens","in","the","dictionary","that","are",":","#",".","Less","frequent","than","`","no_below","`","documents","(","absolute","number",",","e.g",".","`","5","`",")","or","#",".","More","frequent","than","`","no_above","`","documents","(","fraction","of","the","total","corpus","size",",","e.g",".","`","0.3","`",")",".","#",".","After","(","1",")","and","(","2",")",",","keep","only","the","first","`","keep_n","`","most","frequent","tokens","(","or","keep","all","if","`","keep_n=None","`",")",".","After","the","pruning",",","resulting","gaps","in","word","ids","are","shrunk",".","Due","to","this","gap","shrinking",",","*","*","the","same","word","may","have","a","different","word","id","before","and","after","the","call","to","this","function","!","*","*","See",":","class",":","`","gensim.models.VocabTransform","`","and","the","`","dedicated","FAQ","entry","<","https",":","\/\/github.com\/RaRe-Technologies\/gensim\/wiki\/Recipes-","&","-FAQ","#","q8-how-can-i-filter-a-saved-corpus-and-its-corresponding-dictionary",">","`","_","on","how","#","noqa","to","transform","a","corpus","built","with","a","dictionary","before","pruning",".","Examples","--","--","--","--","..","sourcecode",":",":","pycon",">",">",">","from","gensim.corpora","import","Dictionary",">",">",">",">",">",">","corpus","=","[","[","``","m\u00e1ma","''",",","``","mele","''",",","``","maso","''","]",",","[","``","ema","''",",","``","m\u00e1","''",",","``","m\u00e1ma","''","]","]",">",">",">","dct","=","Dictionary","(","corpus",")",">",">",">","len","(","dct",")","5",">",">",">","dct.filter_extremes","(","no_below=1",",","no_above=0.5",",","keep_n=1",")",">",">",">","len","(","dct",")","1"]
code: def filter_extremes(
      self, no_below=5, no_above=0.5, keep_n=100000, keep_tokens=None
  ):
      """Filter out tokens in the dictionary by their frequency.
  
      Parameters
      ----------
      no_below : int, optional
          Keep tokens which are contained in at least `no_below` documents.
      no_above : float, optional
          Keep tokens which are contained in no more than `no_above` documents
          (fraction of total corpus size, not an absolute number).
      keep_n : int, optional
          Keep only the first `keep_n` most frequent tokens.
      keep_tokens : iterable of str
          Iterable of tokens that **must** stay in dictionary after filtering.
  
      Notes
      -----
      This removes all tokens in the dictionary that are:
  
      #. Less frequent than `no_below` documents (absolute number, e.g. `5`) or \n
      #. More frequent than `no_above` documents (fraction of the total corpus size, e.g. `0.3`).
      #. After (1) and (2), keep only the first `keep_n` most frequent tokens (or keep all if `keep_n=None`).
  
      After the pruning, resulting gaps in word ids are shrunk.
      Due to this gap shrinking, **the same word may have a different word id before and after the call
      to this function!** See :class:`gensim.models.VocabTransform` and the
      `dedicated FAQ entry <https://github.com/RaRe-Technologies/gensim/wiki/Recipes-&-FAQ#q8-how-can-i-filter-a-saved-corpus-and-its-corresponding-dictionary>`_ on how  # noqa
      to transform a corpus built with a dictionary before pruning.
  
      Examples
      --------
      .. sourcecode:: pycon
  
          >>> from gensim.corpora import Dictionary
          >>>
          >>> corpus = [["máma", "mele", "maso"], ["ema", "má", "máma"]]
          >>> dct = Dictionary(corpus)
          >>> len(dct)
          5
          >>> dct.filter_extremes(no_below=1, no_above=0.5, keep_n=1)
          >>> len(dct)
          1
  
      """
      no_above_abs = int(
          no_above * self.num_docs
      )  # convert fractional threshold to absolute threshold
  
      # determine which tokens to keep
      if keep_tokens:
          keep_ids = {
              self.token2id[v]
              for v in keep_tokens
              if v in self.token2id
          }
          good_ids = [
              v
              for v in self.token2id.values()
              if no_below <= self.dfs.get(v, 0) <= no_above_abs
              or v in keep_ids
          ]
          good_ids.sort(
              key=lambda x: self.num_docs
              if x in keep_ids
              else self.dfs.get(x, 0),
              reverse=True,
          )
      else:
          good_ids = [
              v
              for v in self.token2id.values()
              if no_below <= self.dfs.get(v, 0) <= no_above_abs
          ]
          good_ids.sort(key=self.dfs.get, reverse=True)
      if keep_n is not None:
          good_ids = good_ids[:keep_n]
      bad_words = [
          (self[idx], self.dfs.get(idx, 0))
          for idx in set(self).difference(good_ids)
      ]
      logger.info(
          "discarding %i tokens: %s...",
          len(self) - len(good_ids),
          bad_words[:10],
      )
      logger.info(
          "keeping %i tokens which were in no less than %i and no more than %i (=%.1f%%) documents",
          len(good_ids),
          no_below,
          no_above_abs,
          100.0 * no_above,
      )
  
      # do the actual filtering, then rebuild dictionary to remove gaps in ids
      self.filter_tokens(good_ids=good_ids)
      logger.info("resulting dictionary: %s", self)
  
imports: from collections import defaultdict
  from collections.abc import Mapping
  import logging
  import itertools
  from typing import Optional, List, Tuple
  from gensim import utils
label_desc: Use image node_id 13 for calling the Dictionary obj's underlying member method code with example usage: obj.filter_extremes(no_below, no_above, keep_n, keep_tokens) without return types