Summary

audio.cpp is a high-performance, all-in-one C++ inference engine designed for modern local audio models. It supports a variety of audio-related AI tasks such as text-to-speech, speech-to-text, voice activity detection, voice conversion, and music generation, delivering optimized and portable performance without Python dependencies.

Key Features

• Built on ggml, providing a portable native stack with backends for CUDA, HIP/ROCm, Vulkan, Metal, and CPU.
• Supports Windows, Linux, and macOS on various hardware including NVIDIA, AMD, Apple Silicon, and CPU-only machines.
• Offers significant performance improvements over Python reference implementations, with speedups ranging from 1.8x to over 8x in CUDA environments, and reductions in end-to-end latency by 45% to 85%.
• Supports loading models in GGUF format with improved speed and reduced VRAM usage.
• Includes pipelines for multi-step workflows using experimental JSON support.
• Contains built-in audio utilities such as denoising, enhancement, resampling, and Short-Time Fourier Transform (STFT/ISTFT) functions for practical production tasks.
• Designed for real end-to-end execution powering TTS, voice cloning, ASR, diarization, VAD, source separation, alignment, codec-style models, and more within a unified framework.
• Community-driven with an open development model encouraging contributions particularly in UI, API server, and pipeline/workflow subsystems.

Intended Users

This framework is aimed at developers and researchers who need a fast, portable environment to run and integrate various state-of-the-art audio AI models locally without the complexity and overhead of Python environments and dependency management.