class_docstr: Dual transformer wrapper that combines two `Transformer2DModel`s for mixed inference.
  
  Parameters:
      num_attention_heads (`int`, *optional*, defaults to 16): The number of heads to use for multi-head attention.
      attention_head_dim (`int`, *optional*, defaults to 88): The number of channels in each head.
      in_channels (`int`, *optional*):
          Pass if the input is continuous. The number of channels in the input and output.
      num_layers (`int`, *optional*, defaults to 1): The number of layers of Transformer blocks to use.
      dropout (`float`, *optional*, defaults to 0.1): The dropout probability to use.
      cross_attention_dim (`int`, *optional*): The number of encoder_hidden_states dimensions to use.
      sample_size (`int`, *optional*): Pass if the input is discrete. The width of the latent images.
          Note that this is fixed at training time as it is used for learning a number of position embeddings. See
          `ImagePositionalEmbeddings`.
      num_vector_embeds (`int`, *optional*):
          Pass if the input is discrete. The number of classes of the vector embeddings of the latent pixels.
          Includes the class for the masked latent pixel.
      activation_fn (`str`, *optional*, defaults to `"geglu"`): Activation function to be used in feed-forward.
      num_embeds_ada_norm ( `int`, *optional*): Pass if at least one of the norm_layers is `AdaLayerNorm`.
          The number of diffusion steps used during training. Note that this is fixed at training time as it is used
          to learn a number of embeddings that are added to the hidden states. During inference, you can denoise for
          up to but not more than steps than `num_embeds_ada_norm`.
      attention_bias (`bool`, *optional*):
          Configure if the TransformerBlocks' attention should contain a bias parameter.
class_docstr_tok: ["Dual","transformer","wrapper","that","combines","two","`","Transformer2DModel","`","s","for","mixed","inference",".","Parameters",":","num_attention_heads","(","`","int","`",",","*","optional","*",",","defaults","to","16",")",":","The","number","of","heads","to","use","for","multi-head","attention",".","attention_head_dim","(","`","int","`",",","*","optional","*",",","defaults","to","88",")",":","The","number","of","channels","in","each","head",".","in_channels","(","`","int","`",",","*","optional","*",")",":","Pass","if","the","input","is","continuous",".","The","number","of","channels","in","the","input","and","output",".","num_layers","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","number","of","layers","of","Transformer","blocks","to","use",".","dropout","(","`","float","`",",","*","optional","*",",","defaults","to","0.1",")",":","The","dropout","probability","to","use",".","cross_attention_dim","(","`","int","`",",","*","optional","*",")",":","The","number","of","encoder_hidden_states","dimensions","to","use",".","sample_size","(","`","int","`",",","*","optional","*",")",":","Pass","if","the","input","is","discrete",".","The","width","of","the","latent","images",".","Note","that","this","is","fixed","at","training","time","as","it","is","used","for","learning","a","number","of","position","embeddings",".","See","`","ImagePositionalEmbeddings","`",".","num_vector_embeds","(","`","int","`",",","*","optional","*",")",":","Pass","if","the","input","is","discrete",".","The","number","of","classes","of","the","vector","embeddings","of","the","latent","pixels",".","Includes","the","class","for","the","masked","latent","pixel",".","activation_fn","(","`","str","`",",","*","optional","*",",","defaults","to","`","``","geglu","''","`",")",":","Activation","function","to","be","used","in","feed-forward",".","num_embeds_ada_norm","(","`","int","`",",","*","optional","*",")",":","Pass","if","at","least","one","of","the","norm_layers","is","`","AdaLayerNorm","`",".","The","number","of","diffusion","steps","used","during","training",".","Note","that","this","is","fixed","at","training","time","as","it","is","used","to","learn","a","number","of","embeddings","that","are","added","to","the","hidden","states",".","During","inference",",","you","can","denoise","for","up","to","but","not","more","than","steps","than","`","num_embeds_ada_norm","`",".","attention_bias","(","`","bool","`",",","*","optional","*",")",":","Configure","if","the","TransformerBlocks","'","attention","should","contain","a","bias","parameter","."]
docstr: Args:
      hidden_states ( When discrete, `torch.LongTensor` of shape `(batch size, num latent pixels)`.
          When continuous, `torch.FloatTensor` of shape `(batch size, channel, height, width)`): Input
          hidden_states.
      encoder_hidden_states ( `torch.LongTensor` of shape `(batch size, encoder_hidden_states dim)`, *optional*):
          Conditional embeddings for cross attention layer. If not given, cross-attention defaults to
          self-attention.
      timestep ( `torch.long`, *optional*):
          Optional timestep to be applied as an embedding in AdaLayerNorm's. Used to indicate denoising step.
      attention_mask (`torch.FloatTensor`, *optional*):
          Optional attention mask to be applied in Attention.
      cross_attention_kwargs (`dict`, *optional*):
          A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under
          `self.processor` in
          [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py).
      return_dict (`bool`, *optional*, defaults to `True`):
          Whether or not to return a [`models.unet_2d_condition.UNet2DConditionOutput`] instead of a plain tuple.
  
  Returns:
      [`~models.transformer_2d.Transformer2DModelOutput`] or `tuple`:
      [`~models.transformer_2d.Transformer2DModelOutput`] if `return_dict` is True, otherwise a `tuple`. When
      returning a tuple, the first element is the sample tensor.
docstr_tok: ["Args",":","hidden_states","(","When","discrete",",","`","torch.LongTensor","`","of","shape","`","(","batch","size",",","num","latent","pixels",")","`",".","When","continuous",",","`","torch.FloatTensor","`","of","shape","`","(","batch","size",",","channel",",","height",",","width",")","`",")",":","Input","hidden_states",".","encoder_hidden_states","(","`","torch.LongTensor","`","of","shape","`","(","batch","size",",","encoder_hidden_states","dim",")","`",",","*","optional","*",")",":","Conditional","embeddings","for","cross","attention","layer",".","If","not","given",",","cross-attention","defaults","to","self-attention",".","timestep","(","`","torch.long","`",",","*","optional","*",")",":","Optional","timestep","to","be","applied","as","an","embedding","in","AdaLayerNorm","'s",".","Used","to","indicate","denoising","step",".","attention_mask","(","`","torch.FloatTensor","`",",","*","optional","*",")",":","Optional","attention","mask","to","be","applied","in","Attention",".","cross_attention_kwargs","(","`","dict","`",",","*","optional","*",")",":","A","kwargs","dictionary","that","if","specified","is","passed","along","to","the","`","AttentionProcessor","`","as","defined","under","`","self.processor","`","in","[","diffusers.models.attention_processor","]","(","https",":","\/\/github.com\/huggingface\/diffusers\/blob\/main\/src\/diffusers\/models\/attention_processor.py",")",".","return_dict","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","or","not","to","return","a","[","`","models.unet_2d_condition.UNet2DConditionOutput","`","]","instead","of","a","plain","tuple",".","Returns",":","[","`","~models.transformer_2d.Transformer2DModelOutput","`","]","or","`","tuple","`",":","[","`","~models.transformer_2d.Transformer2DModelOutput","`","]","if","`","return_dict","`","is","True",",","otherwise","a","`","tuple","`",".","When","returning","a","tuple",",","the","first","element","is","the","sample","tensor","."]
code: def forward(
      self,
      hidden_states,
      encoder_hidden_states,
      timestep=None,
      attention_mask=None,
      cross_attention_kwargs=None,
      return_dict: bool = True,
  ):
      """
      Args:
          hidden_states ( When discrete, `torch.LongTensor` of shape `(batch size, num latent pixels)`.
              When continuous, `torch.FloatTensor` of shape `(batch size, channel, height, width)`): Input
              hidden_states.
          encoder_hidden_states ( `torch.LongTensor` of shape `(batch size, encoder_hidden_states dim)`, *optional*):
              Conditional embeddings for cross attention layer. If not given, cross-attention defaults to
              self-attention.
          timestep ( `torch.long`, *optional*):
              Optional timestep to be applied as an embedding in AdaLayerNorm's. Used to indicate denoising step.
          attention_mask (`torch.FloatTensor`, *optional*):
              Optional attention mask to be applied in Attention.
          cross_attention_kwargs (`dict`, *optional*):
              A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under
              `self.processor` in
              [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py).
          return_dict (`bool`, *optional*, defaults to `True`):
              Whether or not to return a [`models.unet_2d_condition.UNet2DConditionOutput`] instead of a plain tuple.
  
      Returns:
          [`~models.transformer_2d.Transformer2DModelOutput`] or `tuple`:
          [`~models.transformer_2d.Transformer2DModelOutput`] if `return_dict` is True, otherwise a `tuple`. When
          returning a tuple, the first element is the sample tensor.
      """
      input_states = hidden_states
  
      encoded_states = []
      tokens_start = 0
      # attention_mask is not used yet
      for i in range(2):
          # for each of the two transformers, pass the corresponding condition tokens
          condition_state = encoder_hidden_states[
              :, tokens_start : tokens_start + self.condition_lengths[i]
          ]
          transformer_index = self.transformer_index_for_condition[i]
          encoded_state = self.transformers[transformer_index](
              input_states,
              encoder_hidden_states=condition_state,
              timestep=timestep,
              cross_attention_kwargs=cross_attention_kwargs,
              return_dict=False,
          )[0]
          encoded_states.append(encoded_state - input_states)
          tokens_start += self.condition_lengths[i]
  
      output_states = encoded_states[
          0
      ] * self.mix_ratio + encoded_states[1] * (1 - self.mix_ratio)
      output_states = output_states + input_states
  
      if not return_dict:
          return (output_states,)
  
      return Transformer2DModelOutput(sample=output_states)
  
imports: from typing import Optional
  from torch import nn
  from .transformer_2d import Transformer2DModel, Transformer2DModelOutput
label_desc: Use image node_id 2 for calling the DualTransformer2DModel obj's underlying member method code with example usage: obj.forward(hidden_states, encoder_hidden_states, timestep, attention_mask, cross_attention_kwargs, return_dict) and returns: Transformer2DModelOutput, output_states