Skip to content
K = 0.731
← All topics

Voice and audio

AI voice and audio: speech synthesis, real-time conversation, music generation and audio understanding.

27 picksLast 30 days 9 itemsCollected 69 items

New

Picks about Voice and audio

10/16Fri
  1. Hugging Face Blog74

    Granite Switch 4.1 Combines Chat, Decisions, and Speech in One Endpoint

    IBM released Granite-Switch-4.1-8B and -30B, combining chat, a general-purpose Decision aLoRA, a specialized Guardian aLoRA, and Granite-Speech-5.0-Turbo-CTC transcription through one hosted endpoint. IBM says the combined model uses approximately the same GPU VRAM as its base model, with adapters reusing the base model’s KV cache. Users can also compose custom Switch models from Granite Libraries adapters; an Adapter SDK for training adapters in Mellea is planned.

    Why it matters: Shared KV caches and modular adapters offer a concrete approach to combining general and specialized decisions with transcription while keeping GPU memory use close to the base model.

10/14Wed
  1. Artificial Analysis · X78

    Fish Audio’s Drama 3 Preview leads open-weights models in Artificial Analysis TTS rankings

    Artificial Analysis reports that Fish Audio’s Drama 3 Preview ranks #1 among open-weights models on all three of its Text to Speech leaderboards and leads all models in Pronunciation Robustness. It scores 93.2% on pronunciation robustness, ahead of Eleven v4 at 91.7% and Eleven v4 Turbo at 90.1%, while ranking #10 in Provider Voice and #14 in Controlled Voice. Its weights are available on Hugging Face under the Fish Audio Community License, and Fish Audio API pricing is $15/1M characters.

    Why it matters: The comparison separates pronunciation robustness from voice preference rankings, showing how Drama 3 Preview’s lead on one measure differs from its placement on others.

10/2Fri
  1. Mustafa Suleyman · X66

    Microsoft AI launches MAI-Transcribe-2-Streaming, topping AA-WER Streaming accuracy rankings

    Microsoft AI is launching MAI-Transcribe-2-Streaming, which Mustafa Suleyman calls the world's most accurate real-time transcription model. He claims it is 55% faster and 60% cheaper than ElevenLabs and invites developers to build agents on Microsoft's platform. In the quoted post, Artificial Analysis reports #1 rankings for final and first partial transcript accuracy on AA-WER Streaming, with final transcript WER of 2.5% at 0.13s after speech ends and streaming pricing of $0.54 per hour of audio.

    Why it matters: The quoted benchmark comparison pairs transcription accuracy with latency and pricing, giving readers concrete tradeoffs to consider when choosing a streaming speech model for agents.

9/30Wed
  1. Hugging Face Blog60

    Hugging Face releases Open TTS Leaderboard for objective evaluation of multilingual TTS and voice cloning

    Hugging Face released Open TTS Leaderboard, which uses objective metrics including WER/CER, RTFx, TTFA, and speaker similarity to evaluate open-source TTS models, reducing evaluation cycles from weeks of voting to hours.

    Why it matters: Hugging Face replaces vote-based evaluation with objective metrics, showing how to compare open-source TTS models across languages and voice cloning.

9/25Fri
  1. Google DeepMind71

    Google DeepMind releases Gemini 3.8 Live with Live Avatar

    Google DeepMind has released Gemini 3.8 Live with Live Avatar, combining real-time video generation and voice to give Gemini's conversational model a near-real-time visual avatar, with accurate lip-sync, natural expressions, and smooth turn-taking.

    Why it matters: Gemini 3.8 Live adds real-time video avatars, giving readers a view of how multimodal conversation is being applied in enterprise settings and where the feature is available.

9/24Thu
  1. Greg Brockman · X82

    ChatGPT Voice upgrade adds app integrations and ChatGPT Work

    ChatGPT Voice has been upgraded to work with integrations such as email, calendars, and Slack, and to connect to ChatGPT Work. On web and mobile, users can create documents, presentations, websites, and spreadsheets by voice, or handle complex tasks in the browser. Powered by GPT-6 Astra, Sol, and Luna, the feature begins rolling out globally today with the latest app release.

    Why it matters: With email, calendar, and Slack integrations and access to Work, ChatGPT Voice gives readers a way to assess which workflows can be handled through voice.

9/23Wed
  1. Logan Kilpatrick · X71

    Google releases Gemini 3.8 Flash and Flash-Lite TTS voice models

    Google has released the Gemini 3.8 Flash and Flash-Lite text-to-speech models, describing them as new state-of-the-art voice models. They offer a new voice-design experience, more than 2,000 production-ready voices, voice cloning, and support for 100 languages; voice mixing is coming soon. Google says they rank first on Hume AI's voice benchmark.

    Why it matters: Google's Gemini 3.8 Flash and Flash-Lite TTS release lists more than 2,000 voices, support for 100 languages, voice cloning, and other capabilities.

  2. Google DeepMind71

    Google DeepMind releases Gemini 3.8 Flash TTS and Flash-Lite TTS

    Google DeepMind has released two speech-generation models: Gemini 3.8 Flash TTS for character design and fine-grained creative control, and Gemini 3.8 Flash-Lite TTS for high-volume voiceovers and voice agents.

    Why it matters: The article outlines the models' voice cloning, line-by-line direction, and benchmark results, helping readers assess their suitability for dubbing and podcast production.

  3. Qwen · X71

    Alibaba releases Qwen-Audio-3.1 family, including TTS-Next and ASR-Next

    Alibaba has released Qwen-Audio-3.1 with upgrades to ASR, TTS, and Realtime, plus two new models, TTS-Next and ASR-Next. Five models now cover audio understanding, generation, interaction, and creation.

    Why it matters: The article lists the capabilities of all five audio models and their price reductions, helping readers assess voice-stack choices and cost changes.

9/16Wed
  1. Logan Kilpatrick · X72

    Google releases Gemini 3.8 Live and 3.8 Live Extended Thinking real-time audio models

    Google has released Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, new state-of-the-art real-time audio models positioned for leading price and performance. Gemini 3.8 Live supports 97 languages with seamless switching, as well as asynchronous tool calls and other features.

    Why it matters: Google's new real-time audio models offer language coverage and asynchronous tool calls, helping readers assess the practical limits of live voice interaction.

  2. Google DeepMind82

    Google DeepMind releases Gemini 3.8 Live and 3.8 Live Extended Thinking

    Google DeepMind has released two real-time voice-conversation models: Gemini 3.8 Live for scale and cost efficiency, and Gemini 3.8 Live Extended Thinking for complex tasks and multi-step reasoning.

    Why it matters: The two live voice models support parallel reasoning and tool calls and report results on several speech benchmarks, offering a way to assess the limits of voice agents.

  3. Google DeepMind · X77

    Google DeepMind releases Gemini 3.8 Live and 3.8 Live Extended Thinking

    Google DeepMind has released Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, calling them its best conversational AI yet. The company says the models can converse and think while handling tasks in the background without interrupting the user's workflow.

    Why it matters: The new conversational models give readers an update on Google DeepMind's positioning for real-time dialogue and background task handling.

9/10Thu
  1. OpenAI News65

    OpenAI introduces GPT-Live-1 voice model in its API

    OpenAI has introduced GPT-Live-1 in its API, giving developers natural full-duplex voice conversations. The model follows instructions more reliably and supports custom voices and telephony integration.

    Why it matters: GPT-Live-1 brings full-duplex voice conversation to the API, helping readers assess how to integrate voice apps and the model's capabilities.

8/28Fri
  1. Hugging Face Blog60

    Open ASR Leaderboard adds Monsoon, its first benchmark set for Global South languages

    Voice Arena and Hugging Face have added Monsoon en-IN and Monsoon hi-IN to the Open ASR Leaderboard, making Hindi the first Indian language represented on the multilingual leaderboard.

    Why it matters: The Open ASR Leaderboard has added its first Hindi and Indian English benchmarks, showing how speaker attributes are being incorporated into public evaluations.

8/27Thu
  1. Google DeepMind71

    Google DeepMind releases Gemini 3.5 Transcribe speech-to-text model

    Google DeepMind has released Gemini 3.5 Transcribe, which it calls its most accurate speech-to-text model yet. It converts raw audio directly into accurate, formatted text.

    Why it matters: The official post reports WER for streaming and non-streaming APIs and compares the model with the previous-generation Chirp 3, helping readers assess speech transcription in practical workflows.

8/21Fri
  1. Hugging Face Blog60

    Hugging Face research measures benchmark optimization in speech recognition

    Hugging Face has published research measuring benchmark optimization in speech recognition through three tests: consensus disagreement, masked-entity retrieval, and spelling switches. The study evaluates 11 open-source ASR models.

    Why it matters: The study presents three reproducible probes and quantifies how often ASR models reproduce incorrect benchmark transcripts, with direct implications for evaluation design.

8/12Wed
  1. Google Research72

    Google researchers advance AMIE to expert-level clinical interviews with audio and video

    Google Research has released AMIE (Video), which uses Gemini and Project Astra for real-time video clinical interviews. It can perceive nonverbal cues, guide virtual physical examinations, and reason about diagnoses in real time.

    Why it matters: AMIE extends from text to live video consultations, showing how an asynchronous multi-agent architecture can balance conversational latency with clinical reasoning.

8/11Tue
  1. Hugging Face Blog62

    NVIDIA releases open-weights Magpie TTS multilingual model, adding Arabic, Korean, and Brazilian Portuguese

    NVIDIA has released Magpie TTS Multilingual, a 364M-parameter open-weights speech-synthesis model supporting 12 languages. The latest release adds Modern Standard Arabic, Korean, and Brazilian Portuguese, and improves quality in several existing languages through updated training data.

    Why it matters: The article reports the 364M open TTS model's language expansion and measured TTFA on different GPUs, helping readers estimate latency budgets for self-hosted voice pipelines.

7/15Wed
  1. Hugging Face Blog62

    Hume releases Real World VoiceEQ to measure human quality in voice AI

    Hume has released Real World VoiceEQ, a benchmark for human quality in voice interaction. It covers more than 40 proprietary and open-source voice models, over 15 evaluation dimensions, and more than 60 metrics across ASR, TTS, speech-to-speech, and voice understanding.

    Why it matters: Hume built a voice-AI benchmark from more than a million human ratings, showing why current metrics can overestimate real conversational performance.

7/1Wed
  1. Hugging Face Blog62

    Hugging Face and Cerebras bring Gemma 4 to real-time voice AI

    Hugging Face and Cerebras demonstrated a real-time speech-to-speech pipeline using Cerebras to accelerate Gemma 4 31B inference, bringing voice-interaction latency closer to natural conversation.

    Why it matters: The source presents a replaceable open-source speech-to-speech pipeline, helping readers assess engineering combinations for low-latency voice interaction.