metadata
base_model:
- nvidia/audio-flamingo-2
pipeline_tag: feature-extraction
AFClap - Audio Flamingo CLAP
Audio-Flamingo-2のCLAPモデル（epoch_16.pt）。
Model Info
- Base: HTSAT + T5
- Audio Embed Dim: 2048
- Sample Rate: 16000 Hz
- Original: nvidia/audio-flamingo-2
Installation
pip install laion-clap librosa soundfile
Usage
import torch
from laion_clap import CLAP_Module
def load_afclap(ckpt_path):
model = CLAP_Module(
enable_fusion=True,
amodel='HTSAT-afclap',
tmodel='t5'
).cuda()
model.load_afclap_ckpt(ckpt=ckpt_path, verbose=True)
return model
# Load model
model = load_afclap("epoch_16.pt")
# Get audio embedding
audio_embed = model.get_audio_embedding_from_filelist(
["audio1.wav", "audio2.wav"],
sr=16000,
use_tensor=True
)
# Get text embedding
text_embed = model.get_text_embedding(
["This is a classical song.", "This is a rock song."],
use_tensor=True
)
# Compute similarity
similarities = torch.tensor(audio_embed) @ torch.tensor(text_embed).t()
print(similarities)