docstr: Return:
      :obj:`tuple(torch.FloatTensor)` comprising various elements depending on the configuration (:class:`~transformers.BertConfig`) and inputs:
      last_hidden_state (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, sequence_length, hidden_size)`):
          Sequence of hidden-states at the output of the last layer of the model.
      pooler_output (:obj:`torch.FloatTensor`: of shape :obj:`(batch_size, hidden_size)`):
          Last layer hidden-state of the first token of the sequence (classification token)
          further processed by a Linear layer and a Tanh activation function. The Linear
          layer weights are trained from the next sentence prediction (classification)
          objective during pre-training.
  
          This output is usually *not* a good summary
          of the semantic content of the input, you're often better with averaging or pooling
          the sequence of hidden-states for the whole input sequence.
      hidden_states (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_hidden_states=True`` is passed or when ``config.output_hidden_states=True``):
          Tuple of :obj:`torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer)
          of shape :obj:`(batch_size, sequence_length, hidden_size)`.
  
          Hidden-states of the model at the output of each layer plus the initial embedding outputs.
      attentions (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_attentions=True`` is passed or when ``config.output_attentions=True``):
          Tuple of :obj:`torch.FloatTensor` (one for each layer) of shape
          :obj:`(batch_size, num_heads, sequence_length, sequence_length)`.
  
          Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
          heads.
      highway_exits (:obj:`tuple(tuple(torch.Tensor))`:
          Tuple of each early exit's results (total length: number of layers)
          Each tuple is again, a tuple of length 2 - the first entry is logits and the second entry is hidden states.
docstr_tok: ["Return",":",":","obj",":","`","tuple","(","torch.FloatTensor",")","`","comprising","various","elements","depending","on","the","configuration","(",":","class",":","`","~transformers.BertConfig","`",")","and","inputs",":","last_hidden_state","(",":","obj",":","`","torch.FloatTensor","`","of","shape",":","obj",":","`","(","batch_size",",","sequence_length",",","hidden_size",")","`",")",":","Sequence","of","hidden-states","at","the","output","of","the","last","layer","of","the","model",".","pooler_output","(",":","obj",":","`","torch.FloatTensor","`",":","of","shape",":","obj",":","`","(","batch_size",",","hidden_size",")","`",")",":","Last","layer","hidden-state","of","the","first","token","of","the","sequence","(","classification","token",")","further","processed","by","a","Linear","layer","and","a","Tanh","activation","function",".","The","Linear","layer","weights","are","trained","from","the","next","sentence","prediction","(","classification",")","objective","during","pre-training",".","This","output","is","usually","*","not","*","a","good","summary","of","the","semantic","content","of","the","input",",","you","'re","often","better","with","averaging","or","pooling","the","sequence","of","hidden-states","for","the","whole","input","sequence",".","hidden_states","(",":","obj",":","`","tuple","(","torch.FloatTensor",")","`",",","`","optional","`",",","returned","when","``","output_hidden_states=True","``","is","passed","or","when","``","config.output_hidden_states=True","``",")",":","Tuple","of",":","obj",":","`","torch.FloatTensor","`","(","one","for","the","output","of","the","embeddings","+","one","for","the","output","of","each","layer",")","of","shape",":","obj",":","`","(","batch_size",",","sequence_length",",","hidden_size",")","`",".","Hidden-states","of","the","model","at","the","output","of","each","layer","plus","the","initial","embedding","outputs",".","attentions","(",":","obj",":","`","tuple","(","torch.FloatTensor",")","`",",","`","optional","`",",","returned","when","``","output_attentions=True","``","is","passed","or","when","``","config.output_attentions=True","``",")",":","Tuple","of",":","obj",":","`","torch.FloatTensor","`","(","one","for","each","layer",")","of","shape",":","obj",":","`","(","batch_size",",","num_heads",",","sequence_length",",","sequence_length",")","`",".","Attentions","weights","after","the","attention","softmax",",","used","to","compute","the","weighted","average","in","the","self-attention","heads",".","highway_exits","(",":","obj",":","`","tuple","(","tuple","(","torch.Tensor",")",")","`",":","Tuple","of","each","early","exit","'s","results","(","total","length",":","number","of","layers",")","Each","tuple","is","again",",","a","tuple","of","length","2","-","the","first","entry","is","logits","and","the","second","entry","is","hidden","states","."]
code: def forward(
      self,
      input_ids=None,
      attention_mask=None,
      token_type_ids=None,
      position_ids=None,
      head_mask=None,
      inputs_embeds=None,
      encoder_hidden_states=None,
      encoder_attention_mask=None,
  ):
      r"""
      Return:
          :obj:`tuple(torch.FloatTensor)` comprising various elements depending on the configuration (:class:`~transformers.BertConfig`) and inputs:
          last_hidden_state (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, sequence_length, hidden_size)`):
              Sequence of hidden-states at the output of the last layer of the model.
          pooler_output (:obj:`torch.FloatTensor`: of shape :obj:`(batch_size, hidden_size)`):
              Last layer hidden-state of the first token of the sequence (classification token)
              further processed by a Linear layer and a Tanh activation function. The Linear
              layer weights are trained from the next sentence prediction (classification)
              objective during pre-training.
  
              This output is usually *not* a good summary
              of the semantic content of the input, you're often better with averaging or pooling
              the sequence of hidden-states for the whole input sequence.
          hidden_states (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_hidden_states=True`` is passed or when ``config.output_hidden_states=True``):
              Tuple of :obj:`torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer)
              of shape :obj:`(batch_size, sequence_length, hidden_size)`.
  
              Hidden-states of the model at the output of each layer plus the initial embedding outputs.
          attentions (:obj:`tuple(torch.FloatTensor)`, `optional`, returned when ``output_attentions=True`` is passed or when ``config.output_attentions=True``):
              Tuple of :obj:`torch.FloatTensor` (one for each layer) of shape
              :obj:`(batch_size, num_heads, sequence_length, sequence_length)`.
  
              Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
              heads.
          highway_exits (:obj:`tuple(tuple(torch.Tensor))`:
              Tuple of each early exit's results (total length: number of layers)
              Each tuple is again, a tuple of length 2 - the first entry is logits and the second entry is hidden states.
      """
      if input_ids is not None and inputs_embeds is not None:
          raise ValueError(
              "You cannot specify both input_ids and inputs_embeds at the same time"
          )
      elif input_ids is not None:
          input_shape = input_ids.size()
      elif inputs_embeds is not None:
          input_shape = inputs_embeds.size()[:-1]
      else:
          raise ValueError(
              "You have to specify either input_ids or inputs_embeds"
          )
  
      device = (
          input_ids.device
          if input_ids is not None
          else inputs_embeds.device
      )
  
      if attention_mask is None:
          attention_mask = torch.ones(input_shape, device=device)
      if encoder_attention_mask is None:
          encoder_attention_mask = torch.ones(
              input_shape, device=device
          )
      if token_type_ids is None:
          token_type_ids = torch.zeros(
              input_shape, dtype=torch.long, device=device
          )
  
      # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
      # ourselves in which case we just need to make it broadcastable to all heads.
      extended_attention_mask: torch.Tensor = (
          self.get_extended_attention_mask(
              attention_mask, input_shape, device
          )
      )
  
      # If a 2D ou 3D attention mask is provided for the cross-attention
      # we need to make broadcastable to [batch_size, num_heads, seq_length, seq_length]
      if encoder_attention_mask.dim() == 3:
          encoder_extended_attention_mask = encoder_attention_mask[
              :, None, :, :
          ]
      if encoder_attention_mask.dim() == 2:
          encoder_extended_attention_mask = encoder_attention_mask[
              :, None, None, :
          ]
  
      encoder_extended_attention_mask = (
          encoder_extended_attention_mask.to(
              dtype=next(self.parameters()).dtype
          )
      )  # fp16 compatibility
      encoder_extended_attention_mask = (
          1.0 - encoder_extended_attention_mask
      ) * -10000.0
  
      # Prepare head mask if needed
      # 1.0 in head_mask indicate we keep the head
      # attention_probs has shape bsz x n_heads x N x N
      # input head_mask has shape [num_heads] or [num_hidden_layers x num_heads]
      # and head_mask is converted to shape [num_hidden_layers x batch x num_heads x seq_length x seq_length]
      head_mask = self.get_head_mask(
          head_mask, self.config.num_hidden_layers
      )
  
      embedding_output = self.embeddings(
          input_ids=input_ids,
          position_ids=position_ids,
          token_type_ids=token_type_ids,
          inputs_embeds=inputs_embeds,
      )
      encoder_outputs = self.encoder(
          embedding_output,
          attention_mask=extended_attention_mask,
          head_mask=head_mask,
          encoder_hidden_states=encoder_hidden_states,
          encoder_attention_mask=encoder_extended_attention_mask,
      )
      sequence_output = encoder_outputs[0]
      pooled_output = self.pooler(sequence_output)
  
      outputs = (
          sequence_output,
          pooled_output,
      ) + encoder_outputs[
          1:
      ]  # add hidden_states and attentions if they are here
      return outputs
  
imports: import torch
  from torch import nn
  from torch.nn import CrossEntropyLoss, MSELoss
  from transformers.file_utils import add_start_docstrings, add_start_docstrings_to_model_forward
  from transformers.models.bert.modeling_bert import BERT_INPUTS_DOCSTRING, BERT_START_DOCSTRING, BertEmbeddings, BertLayer, BertPooler, BertPreTrainedModel
label_desc: Use image node_id 6 for calling the DeeBertModel obj's underlying member method code with example usage: obj.forward(input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask) and returns: outputs