Microsoft completes the entire voice agent link: transcribe while speaking, speech generation as low as 45ms

Dongcha Beating AI News, Microsoft AI launched three audio models in one go, namely MAI-Transcribe-2-Streaming, MAI-Voice-2.1 and MAI-Voice-2.1-Flash. The former is responsible for converting real-time speech into text, and the latter two are responsible for generating text into speech, mainly for low-latency voice agents.

Among them, MAI-Transcribe-2-Streaming can continue to output text before the person has finished speaking, and supports 60 languages ​​and automatic language detection. Artificial Analysis' streaming speech transcription rankings show that it ranks first in both final and first partial transcription accuracy, with a word error rate of 2.5%. Final text was returned approximately 0.13 seconds after the end of speech was detected, compared to 2.7% and 0.49 seconds for Grok Voice Transcribe 2.0.

MAI-Voice-2.1 supports 23 languages, allowing the same voice to switch between different languages. The Flash version is designed to reduce latency, with model inference time of about 45ms and a price of $15 per million characters; the standard version is about 550ms and a price of $22. Both models support vocal matching with a short piece of reference audio.

This combination solves the waiting problem that most affects the voice agent experience. The system can start transcribing and reasoning in advance when the user is speaking, and then uses Flash to quickly generate a voice reply, without having to wait for a complete sentence to finish before starting work.