audio-embeddings

Shazam-inspired music recognition system — FAISS vector search + spectral fingerprinting, with CLAP/MuQ embeddings, FastAPI backend and React frontend.

Self-supervised learning (SSL) audio embedding framework with PyTorch Lightning + Hydra supporting Audio-JEPA, BEST-RQ (ViT based), and the BEST-RQ-2 model family.

Song identification combining landmark audio fingerprinting with CNN/AST/MERT embeddings (contrastive InfoNCE), benchmarked on Jamendo with strong noise robustness.

Self-hosted audio API in one Docker container. Stem separation, mastering, MIR (BPM/key/chords), audio→MIDI, text-to-music + SFX (MusicGen/Stable Audio/Riffusion/AudioLDM2), diarization, speech enhancement, restoration, embeddings + zero-shot tagging, workflows, async + webhooks. REST + MCP. CPU + CUDA.

Unfinished research into musical taste: audio embeddings, candidate-conditioned recommendation, and what we learned from the experiments.

Audio-similarity discovery across 272,853 tracks, with strict Spotify language filtering, native Spicetify playback, anonymous feedback, and a blinded evaluator.

Music mood embedding, clustering & zero-shot labeling engine (MERT + CLAP) — a pure, stateless Python MIR toolbox.

Transfer sonic characteristics from reference sounds to a source audio using Music2Latent embeddings

Music-emotion alignment via contrastive learning. Pairs frozen MERT audio encoder with BGE text encoder in a shared 512D space for emotion-based music retrieval.

Universal audio embeddings + tagging + similarity search with Streamlit demo and FastAPI; PANNs/YAMNet, FAISS/Qdrant, Grad-CAM.

Machine learning song recommender: cultural retrieval + CLAP audio embeddings + LLM rationales, over a two-gate async FastAPI pipeline on Postgres/pgvector, with a static Next.js showcase.

Self-hosted semantic music search and audio tagging over a catalog on Backblaze B2. Essentia extracts BPM, key, genre, and mood; CLAP embeddings power find-similar and plain-English text-to-audio search. All local OSS models — B2 is the only credential.

Multimodal extractors for video, image, audio, text & PDF — turn any file into searchable vector embeddings (SigLIP, Gemini, E5, CLAP, ArcFace).

Local adaptive music player using CLAP audio embeddings, listening history, and a logistic ranker to personalize playback with session context and exploration.

CLI and Python library for embedding human health acoustics using Google HeAR.

Music analysis for DJs — interpretable DSP features (BPM, four-on-floor, spectral balance, key) combined with MERT audio embeddings, UMAP genre clustering, Engine DJ playlist import and similar-track suggestions.

Cante Style Embeddings - Cantaor Identification

End-to-end bioacoustic search for animal vocalizations in Amazonian forest recordings using SurfPerch, Qdrant, and an interactive review UI.

Explainable music search with optional CLAP audio enrichment

Perch 2.0 bird and wildlife sound embeddings, quantised to INT8 (12 MB) for NPUs like the STM32N6