Summary

AudioCraft is a PyTorch library developed for deep learning research focused on audio processing and generation. It incorporates state-of-the-art models for high-quality audio generation, including MusicGen for controllable text-to-music synthesis.

Features

• Includes state-of-the-art models such as MusicGen (controllable text-to-music model), AudioGen (text-to-sound), and EnCodec (high fidelity audio codec).
• Provides both inference and training code to enable development and reproduction of audio generative models.
• Supports additional models like Multi Band Diffusion, MAGNeT for non-autoregressive text-to-music/sound, AudioSeal for audio watermarking, and JASCO for chord-, melody-, and drum-conditioned text-to-music.
• Installation requires Python 3.9 and PyTorch 2.1.0, with instructions provided for package setup and dependencies like ffmpeg.
• Models and training pipelines are open source and distributed under MIT and CC-BY-NC 4.0 licenses respectively.
• API documentation and detailed FAQs are available to assist users.

Intended Audience

Researchers and developers interested in AI-driven audio generation, including those who want both to use pretrained models for text-to-music/sound generation and to train or extend such models.