class_docstr: Manage Embedding Weights on CPU and CUDA memory uses a software cache.
  CPU maintains the entire original weight.
  CUDA maintains a fraction of the weights used in the upcoming computation. The row number in CUDA is controlled by `cuda_row_num`.
  During training, GPU needs to transmit embedding rows between CPU and GPU.
  Args:
      weight (torch.Tensor): the weight of the Embedding layer.
      cuda_row_num (int, optional): the number of rows cached in CUDA memory. Defaults to 0.
      buffer_size (int, optional): the number of rows in a data transmitter buffer. Defaults to 50_000.
      pin_weight (bool, optional): use pin memory to store the cpu weight. If set `True`, the cpu memory usage will increase largely. Defaults to False.
      evict_strategy (EvictionStrategy, optional): the eviction strategy. There are two options.
      `EvictionStrategy.LFU`: use the least frequently used cache.
      `EvictionStrategy.DATASET`: use the stats collected from the target dataset. It usually leads to less cpu-gpu communication volume.
      Defaults to EvictionStrategy.DATASET.
class_docstr_tok: ["Manage","Embedding","Weights","on","CPU","and","CUDA","memory","uses","a","software","cache",".","CPU","maintains","the","entire","original","weight",".","CUDA","maintains","a","fraction","of","the","weights","used","in","the","upcoming","computation",".","The","row","number","in","CUDA","is","controlled","by","`","cuda_row_num","`",".","During","training",",","GPU","needs","to","transmit","embedding","rows","between","CPU","and","GPU",".","Args",":","weight","(","torch.Tensor",")",":","the","weight","of","the","Embedding","layer",".","cuda_row_num","(","int",",","optional",")",":","the","number","of","rows","cached","in","CUDA","memory",".","Defaults","to","0.","buffer_size","(","int",",","optional",")",":","the","number","of","rows","in","a","data","transmitter","buffer",".","Defaults","to","50_000",".","pin_weight","(","bool",",","optional",")",":","use","pin","memory","to","store","the","cpu","weight",".","If","set","`","True","`",",","the","cpu","memory","usage","will","increase","largely",".","Defaults","to","False",".","evict_strategy","(","EvictionStrategy",",","optional",")",":","the","eviction","strategy",".","There","are","two","options",".","`","EvictionStrategy.LFU","`",":","use","the","least","frequently","used","cache",".","`","EvictionStrategy.DATASET","`",":","use","the","stats","collected","from","the","target","dataset",".","It","usually","leads","to","less","cpu-gpu","communication","volume",".","Defaults","to","EvictionStrategy.DATASET","."]
code: def _find_free_cuda_row(self) -> int:
      if self._cuda_available_row_num == 0:
          return -1
      candidates = torch.nonzero(self.cached_idx_map == -1).squeeze(1)
      return candidates[0].item()
  
imports: import sys
  from contextlib import contextmanager
  from enum import Enum
  from typing import List, Optional
  import numpy
  import torch
  from contexttimer import Timer
  from torch.profiler import record_function
  from .copyer import LimitBuffIndexCopyer
label_desc: Use image node_id 15 for calling the CachedParamMgr obj's underlying member method code with example usage: obj._find_free_cuda_row() and returns: candidates, unknown