docstr: Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original. 
docstr_tok: ["Prepare","masked","tokens","inputs\/labels","for","masked","language","modeling",":","80","%","MASK",",","10","%","random",",","10","%","original","."]
code: def mask_tokens(self, inputs, mlm_probability=0.15):
      """Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original."""
      labels = inputs.clone()
      # We sample a few tokens in each sequence for masked-LM training (with probability mlm_probability defaults to 0.15 in Bert/RoBERTa)
  
      masked_indices = torch.bernoulli(
          torch.full(labels.shape, mlm_probability)
      ).bool()
      # do not mask special tokens
      masked_indices[:, 0] = False
      masked_indices[:, -1] = False
  
      labels[
          ~masked_indices
      ] = -100  # We only compute loss on masked tokens
  
      # 80% of the time, we replace masked input tokens with tokenizer.mask_token ([MASK])
      indices_replaced = (
          torch.bernoulli(torch.full(labels.shape, 0.8)).bool()
          & masked_indices
      )
      inputs[indices_replaced] = self.tokenizer.convert_tokens_to_ids(
          self.tokenizer.mask_token
      )
  
      # 10% of the time, we replace masked input tokens with random word
      indices_random = (
          torch.bernoulli(torch.full(labels.shape, 0.5)).bool()
          & masked_indices
          & ~indices_replaced
      )
      random_words = torch.randint(
          len(self.tokenizer), labels.shape, dtype=torch.long
      )
      inputs[indices_random] = random_words[indices_random]
  
      # The rest of the time (10% of the time) we keep the masked input tokens unchanged
      return inputs, labels
  
imports: import re
  import html
  import logging
  import pandas
  import os
  import random
  import torch
  from pathlib import Path
  import pickle
  import shutil
  import itertools
  import more_itertools
  from sklearn.model_selection import train_test_split
  from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler, Dataset
  from torch.utils.data.distributed import DistributedSampler
  import spacy
  from tqdm import tqdm, trange
  from fastprogress.fastprogress import master_bar, progress_bar
  from transformers import WEIGHTS_NAME, BertConfig, BertForSequenceClassification, BertTokenizer, XLMConfig, XLMForSequenceClassification, XLMTokenizer, XLNetConfig, XLNetForSequenceClassification, XLNetTokenizer, RobertaConfig, RobertaForSequenceClassification, RobertaTokenizer, DistilBertConfig, DistilBertForSequenceClassification, DistilBertTokenizer, CamembertConfig, CamembertForSequenceClassification, CamembertTokenizer, ElectraConfig, ElectraForSequenceClassification, ElectraTokenizer
label_desc: Use image node_id 3 for calling the BertLMDataBunch obj's underlying member method code with example usage: obj.mask_tokens(inputs, mlm_probability) and returns: inputs,  labels