Gradium open-sources ultra-small speech model: weights only 64MB, can speak on phones even offline

Beating AI News Flash: AI voice company Gradium has open-sourced its speech generation model Phonon. It began private testing in April this year with approximately 100 million parameters, and this time has released 31 million and 40 million parameter versions, publicly disclosing the model weights for the first time. Among them, the Q8 weights of the 31 million parameter English version are only 64MB, and it can run on an ordinary CPU. The 40 million parameter version supports five languages: English, French, German, Spanish, and Portuguese, offering both single-language and five-in-one models.

Phonon was trained by distilling a larger model. It reads text directly without first converting it to phonemes, then uses a Transformer to progressively predict acoustic features and generates audio through the Mimi decoder. Each output corresponds to 80 milliseconds of speech, allowing synthesis and playback simultaneously without waiting for the entire sentence to be generated.

In terms of speed, Gradium disclosed that Phonon takes only 7.5 milliseconds to generate the first audio segment on the Snapdragon 8 Elite Gen 5, achieving a synthesis speed 24 times real-time. On the CPU of a MacBook Air M5, the first audio takes approximately 7 milliseconds.

In terms of audio quality, Gradium organized 1,189 listeners who completed over 17,000 blind listening votes. The 31 million parameter English version scored 2098 Elo, higher than the 2056 of the 100 million parameter Pocket TTS. In another 500-sentence English test containing complex text such as phone numbers and email addresses, Phonon achieved a word error rate of 1.7%, lower than the 3.5% of another 82 million parameter open-source speech model, Kokoro. These results all come from tests published by Gradium.

Phonon has been integrated into tools such as Ollama and MLX-Audio. Partner Xybrid also ran an 8B Agent, Whisper, and Phonon simultaneously on a 16GB M1 Pro MacBook, enabling a 3D game character to converse in real time with a completely offline connection.

Currently, only the 31 million parameter English version uses the Apache 2.0 license, allowing commercial use. The 40 million parameter and multilingual versions use the CC BY-NC license, and commercial use requires separate authorization. No Chinese model has been released yet.