OpenAI Unveils Advanced Speech & Voice Models for Developers

OpenAI has announced the release of next-generation speech-to-text and text-to-speech models in its API, aiming to elevate the capabilities of voice-driven applications. These models promise improved accuracy, customization, and usability, expanding the potential of intelligent voice agents.

Key Highlights:

• Speech-to-Text Upgrades: Two new models—gpt-4o-transcribe and gpt-4o-mini-transcribe—set a new benchmark for transcription accuracy, particularly in complex scenarios involving accents, background noise, and varying speech speeds. gpt-4o-transcribe delivers lower Word Error Rates (WER) than Whisper models across key benchmarks, including the multilingual FLEURS benchmark, thanks to reinforcement learning innovations and training on diverse, high-quality audio datasets. These improvements result in more reliable transcription across over 30 languages. The new speech-to-text models are now available via the Speech-to-Text API.
  

• Text-to-Speech Enhancements: The gpt-4o-mini-tts model introduces steerability, allowing developers to instruct not just what the model says, but how it says it—whether as a sympathetic customer service agent or a bedtime storyteller. This unlocks greater customization for creative and business use cases. Voices are preset and artificial, with safeguards in place to ensure consistency and safety. The text-to-speech model is available through the Text-to-Speech API.

Technical Innovations Behind the Models

• Advanced Pretraining: The models are built on GPT-4o architectures and trained on specialized, high-quality audio datasets. This focused approach enhances the model’s understanding of speech nuances, leading to exceptional performance across audio tasks.

• Enhanced Distillation Techniques: New distillation methods improve efficiency, enabling smaller models to retain the quality of larger counterparts while delivering responsive, realistic, natural-sounding conversations. This enables our smaller models to maintain high conversational quality and responsiveness.

• Reinforcement Learning Approach: For speech-to-text tasks, a reinforcement learning-heavy methodology improves transcription precision and reduces recognition errors. This positions our speech-to-text solutions as highly competitive in challenging speech recognition scenarios.

Available Now

These new audio models are accessible to all developers through OpenAI’s API with more details on how to integrate audio here. Integration with the Agents SDK simplifies the development process for those building conversational or speech-driven applications. For developers seeking low-latency experiences, OpenAI also recommends its Realtime API speech-to-speech models for low-latency speech-to-speech experiences.

Looking Ahead

OpenAI plans to further refine its audio models, exploring opportunities for developers to incorporate custom voices for personalized experiences while maintaining strong safety standards. The company is also engaging with policymakers, developers, and researchers to address the evolving landscape of synthetic voices. Future investments include expanding into other modalities, such as video, to support the development of rich, multimodal agent experiences.

What This Means

With these upgrades, OpenAI strengthens its position in the rapidly evolving voice AI market, where competitors like ElevenLabs and Synthesia continue to push innovation in synthetic voice and video tools. Developers now have access to more accurate, customizable speech models directly through OpenAI's API, simplifying the process of building high-quality voice agents. For those already using ChatGPT or text-based models, integrating the new voice APIs offers an easy path to creating personalized, natural-sounding experiences without starting from scratch. This progress not only raises the bar for transcription accuracy and voice personalization but also intensifies competition across industries—from customer service and content creation to real-time communication platforms.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.