Summary

NeuralNote is an audio plugin that converts audio into MIDI using deep learning. It supports polyphonic transcription and pitch bend detection for any tonal instrument, including voice, and can be used within a Digital Audio Workstation (DAW) or as a standalone application.

Features

• Audio to MIDI transcription with support for polyphonic instruments and pitch bend detection
• Lightweight and fast processing allowing parameter adjustments during playback
• Direct MIDI export via drag and drop to a MIDI track
• Compatible with various audio formats: WAV, AIFF, FLAC, MP3, and OGG (Vorbis)
• Available as VST3, AU (macOS only), and standalone versions for Windows, macOS (Universal), and Linux
• Uses Spotify's basic-pitch model internally, running with RTNeural and ONNXRuntime frameworks

Usage

Users apply NeuralNote as an AudioFX plugin on the track they want to transcribe. Transcription can occur in real-time while recording or during playback. Parameters for transcription can be adjusted while listening, and the resulting MIDI can be exported easily.

Technical Details and Build

NeuralNote is open source with Apache-2.0 license. It requires git, cmake, and a suitable compiler for building from source. The project supports IDE usage and contributions. The plugin is based on and extends the work of Spotify's basic-pitch and integrates third-party libraries such as JUCE, RTNeural, ONNXRuntime, and others.

Limitations

Real-time transcription is not currently feasible due to latency requirements of the Constant-Q Transform and model processing.