Mistral AI Unveils Open-Source Text-to-Speech Model Competing With Big Names

Mistral AI, the French company known for its innovative approaches in artificial intelligence (AI), has just released Voxtral TTS, an open-source text-to-speech model that is set to shake up the market. This new technology competes directly with established players such as ElevenLabs and Deepgram.

Breaking Down the Technology

Voxtral TTS supports nine languages including English, French, German, Spanish, Dutch, Portuguese, Italian, Hindi, and Arabic. The model is designed to be versatile enough for use in a wide range of applications, from voice assistants on smartphones or smartwatches to enterprise solutions like customer support systems.

According to Pierre Stock, VP of Science Operations at Mistral AI, the company built this small-sized speech model specifically with edge devices in mind. "Our customers have been asking for a speech model that can fit into their existing tech stack," he explained during an interview with TechCrunch. The cost is significantly lower than other market offerings while still delivering state-of-the-art performance.

Customization and Adaptability

A key feature of Voxtral TTS is its ability to adapt a custom voice using just under five seconds of sample audio. This capability allows for the capture of subtle accents, inflections, intonations, and irregularities in speech patterns. Stock highlighted that this makes it particularly useful for use cases such as dubbing or real-time translation.

Moreover, the model can switch between languages seamlessly without losing voice characteristics. "This is crucial," said Stock, emphasizing how easy language switching could be a game-changer for businesses operating across multiple markets with diverse linguistic needs."

The Competitive Landscape

Mistral AI's entry into the text-to-speech market comes at an interesting time. The industry is already crowded, but companies like ElevenLabs and Deepgram have established themselves as leaders in this space. However, Mistral aims to carve out its own niche by offering a cost-effective solution that still delivers high-quality results.

Stock noted that the company's approach focuses on making AI more accessible for smaller businesses or individuals who might not be able to afford premium solutions from larger players. "We believe in democratizing access to advanced technologies," he said, underscoring Mistral’s commitment to open-source innovation and inclusivity within the tech community.