class_docstr: Abstract Configuration class to store `sparsity configuration of a self attention layer`.
  It contains shared property of different block-sparse sparsity patterns. However, each class
  needs to extend it based on required property and functionality.
class_docstr_tok: ["Abstract","Configuration","class","to","store","`","sparsity","configuration","of","a","self","attention","layer","`",".","It","contains","shared","property","of","different","block-sparse","sparsity","patterns",".","However",",","each","class","needs","to","extend","it","based","on","required","property","and","functionality","."]
docstr: Initialize the Sparsity Pattern Config.
  Arguments:
       num_heads: required: an integer determining number of attention heads of the layer.
       block_size: optional: an integer determining the block size. Current implementation of
       sparse self-attention is based on blocked sparse matrices. In which this parameter
       defines size of such blocks, `Block X Block`.
       different_layout_per_head: optional: a boolean determining if each head should be
       assigned a different sparsity layout; default is false and this will be satisfied
       based on availability.
docstr_tok: ["Initialize","the","Sparsity","Pattern","Config",".","Arguments",":","num_heads",":","required",":","an","integer","determining","number","of","attention","heads","of","the","layer",".","block_size",":","optional",":","an","integer","determining","the","block","size",".","Current","implementation","of","sparse","self-attention","is","based","on","blocked","sparse","matrices",".","In","which","this","parameter","defines","size","of","such","blocks",",","`","Block","X","Block","`",".","different_layout_per_head",":","optional",":","a","boolean","determining","if","each","head","should","be","assigned","a","different","sparsity","layout",";","default","is","false","and","this","will","be","satisfied","based","on","availability","."]
code: def __init__(
      self, num_heads, block_size=16, different_layout_per_head=False
  ):
      """Initialize the Sparsity Pattern Config.
      Arguments:
           num_heads: required: an integer determining number of attention heads of the layer.
           block_size: optional: an integer determining the block size. Current implementation of
           sparse self-attention is based on blocked sparse matrices. In which this parameter
           defines size of such blocks, `Block X Block`.
           different_layout_per_head: optional: a boolean determining if each head should be
           assigned a different sparsity layout; default is false and this will be satisfied
           based on availability.
      """
  
      self.num_heads = num_heads
      self.block_size = block_size
      self.different_layout_per_head = different_layout_per_head
      self.num_layout_heads = (
          num_heads if different_layout_per_head else 1
      )
  
imports: import random
  import torch
label_desc: Use image node_id 1 to create a new SparsityConfig object  with example: obj = SparsityConfig(num_heads, block_size, different_layout_per_head)