Last week, internet sleuths discovered that OpenAI had trademarked the term "Voice Engine" and rumors began swirling that the company was about to release a Siri and Alexa competitor. Today, OpenAI released a preview of Voice Engine, a new model that generates natural-sounding speech resembling the original speaker using a single 15-second audio sample.
The technology, which has been under development since late 2022, has already been used to power preset voices in OpenAI's text-to-speech API, ChatGPT Voice, and Read Aloud. Despite these promising applications, the company has not yet announced a date for public availability, and is taking a cautious approach to its broader release due to potential misuse.
To better understand the potential uses and implications of Voice Engine, OpenAI has been privately testing the technology with a small group of trusted partners. These early adopters have developed impressive applications, such as providing reading assistance with natural-sounding voices, translating content to reach global audiences, improving essential service delivery in remote settings, and supporting individuals with speech impairments or disabilities.
Examples shared by OpenAI of Voice Engine in action:
Providing reading assistance to non-readers and children through natural-sounding, emotive voices representing a wider range of speakers than what's possible with preset voices. Age of Learning, an education technology company dedicated to the academic success of children, has been using this to generate pre-scripted voice-over content
Translating content, like videos and podcasts, so creators and businesses can reach more people around the world, fluently and in their own voices. One early adopter of this is HeyGen, an AI visual storytelling platform that works with their enterprise customers to create custom, human-like avatars for a variety of content, from product marketing to sales demos.
Reaching global communities, by improving essential service delivery in remote settings. Dimagi is building tools for community health workers to provide a variety of essential services, such as counseling for breastfeeding mothers. To help these workers develop their skills, Dimagi uses Voice Engine and GPT-4 to give interactive feedback in each worker's primary language including Swahili or more informal languages like Sheng, a code-mixed language popular in Kenya.
Check out the OpenAI blog for even more examples.
While these use cases demonstrate the positive potential of Voice Engine, OpenAI recognizes the serious risks associated with generating speech that closely resembles people's voices, particularly in the context of an election year. The company is actively engaging with partners from various sectors to incorporate their feedback and ensure responsible development and deployment of the technology.
OpenAI's approach to building Voice Engine safely includes requiring partners to adhere to usage policies prohibiting impersonation without consent, obtaining explicit and informed consent from the original speaker, and clearly disclosing the use of AI-generated voices. The company has also implemented safety measures such as watermarking to trace the origin of generated audio and proactive monitoring of how the technology is being used.
OpenAI is not alone in advancing synthetic voice technology. Other players in the space, such as ElevenLabs, provide state-of-the-art AI voice solutions for various products and services, including professional voice cloning, dubbing, and translations.
Chris McKay
This week also saw the debut of Hume AI's Empathetic Voice Interface, which utilizes an empathic large language model to adjust its language and tone of voice based on context and the user's emotional expressions. These developments underscore the rapid progress and growing interest in AI-powered voice technologies across industries.
Chris McKay
Looking ahead, OpenAI encourages societal resilience against the challenges posed by increasingly convincing generative models. This includes phasing out voice-based authentication for sensitive information, exploring policies to protect individuals' voices in AI, educating the public about AI capabilities and limitations, and accelerating the development and adoption of techniques for tracking the origin of audiovisual content.
As the debate surrounding synthetic voice technology continues, OpenAI's preview of Voice Engine underscores both the potential benefits and the need for responsible deployment. The company's cautious approach and ongoing engagement with stakeholders is critical to understanding and mitigating the risks associated with this powerful technology.