Judge the state of AI by X or Hacker News in early 2026, and you’d think it’s all visual. The feed is full of “Nano Banana” image-generation experiments and breathless coverage of Seedance 2.0, a video model that finally pushed time coherence past the one-minute mark.

Meanwhile, audio AI quietly took an interesting turn.

The part of the stack dealing with Speech-to-Text (STT), Text-to-Speech (TTS), and dialog-based voice agents has largely broken free from the scaling logic that drives everything else. Video generation still requires centralized GPU farms and nine-figure compute budgets. Audio doesn’t, and that gap is widening.

As of February 2026, the audio stack is faster, cheaper, and more fragmented than any other AI modality. Two distinct markets are forming: proprietary, enterprise-grade platforms built for compliance and broadcast quality, and a scrappy open-source ecosystem meant to run on whatever hardware you have in your pocket. Here’s where things actually stand.

Enterprise audio infrastructure: ElevenLabs and proprietary TTS

To understand why open source has momentum, start with who they’re running against.

On February 4, 2026, ElevenLabs announced a $500 million Series D led by Sequoia, valuing the company at $11 billion. They closed 2025 with ARR over $330 million. That’s a real business.

ElevenLabs isn’t just a TTS API anymore. It has three main products: ElevenAgents, a low-code platform for deploying conversational bots used by Revolut and Deutsche Telekom to replace legacy contact centers; ElevenCreative, a dubbing and localization engine supporting 70+ languages used by Nvidia and mainstream media; and a Licensed Voice Marketplace that solved the legal problem of commercial voice cloning by doing equity-backed deals with voice talent.

That last piece is the actual moat. Any open-source system can clone a voice. Very few organizations can legally deploy that clone commercially without IP exposure. ElevenLabs sells legal clearance as much as it sells audio quality.

For enterprises that need strict SLAs, broadcast-quality output, and zero legal vagueness, ElevenLabs is the default. For everyone else, the picture looks different.

Architectural shifts: replacing cascaded systems with native audio models

The more interesting technical development of the past two years is what’s happening to the underlying architecture.

Traditional conversational AI worked in three steps: transcribe audio to text (STT), run the said text through an LLM, then synthesize the response back to speech (TTS). This pipeline introduced a latency of 1 to 3 seconds per exchange and lost all nonverbal information along the way. Tone, hesitation, urgency—stripped out at the first step, never recovered.

The French AI lab Kyutai released Moshi to address this. Moshi is a full-duplex spoken LLM, meaning it processes and generates audio without first converting to text. Its “Inner Monologue” architecture jointly models time-aligned text tokens as prefixes of acoustic tokens—it thinks and speaks simultaneously rather than sequentially. Paired with the Helium 7B language backbone and a highly compressed 1.1 kbps Mimi audio codec, Moshi runs at 160ms glass-to-glass latency. That’s about 40ms faster than average human conversational response times.

It also handles interruptions through modeling the user and AI on separate audio streams, which fixes the most obvious UX problem with cascaded systems: the bot couldn’t process your interruption while it was still speaking.

Open-source multilingual TTS: Alibaba’s Qwen3-TTS

If Moshi proved the architecture could work, Qwen3-TTS is what most developers are actually using.

Released in January 2026 under the Apache 2.0 license, the Qwen3-TTS family (0.6B to 1.7B parameters) has become the most widely adopted open-source TTS solution. Three things make it competitive with closed-source alternatives: zero-shot voice cloning from three seconds of reference audio; a 12Hz multi-codebook speech encoder with dual-track streaming that hits time-to-first-audio of 97ms; and localization that actually works—10 major languages plus 9 regional dialects, including Hokkien, Cantonese, and Sichuanese, capturing prosody that Western-trained models tend to flatten.

In WER and voice similarity benchmarks, Qwen3-TTS matches or beats closed-source competitors. The gap between open and proprietary audio models has effectively closed for most use cases.

As SOTA audio artificial intelligence models sprout from the largest labs, eventually open-source ventures catch up, pushing the frontier and accessibility. Providing the technological capabilities in the hands of the masses.

Edge AI and local inference: audio micro-models

The real signal of where things are going comes from what developers are doing on nights and weekends.

There’s a growing contingent on Hacker News and GitHub that has stopped using metered cloud APIs for audio entirely. They’re running micro-models locally. The math is simple: if a model fits on your MacBook’s Neural Engine and produces output in under 150ms, why are you paying per API call?

Three models define this space right now:

Kokoro (82M parameters) runs natively on a MacBook’s Neural Engine or a smartphone NPU, with near-zero latency, and produces speech quality competitive with that of server-side models. It’s the obvious pick for offline applications where cost is the main constraint.

CosyVoice2-0.5B uses finite scalar quantization (FSQ) to compress its speech token codebook. At 0.5B parameters, it streams at 150ms while supporting frame-level control over emotion and dialect, without the compute overhead of larger models.

FishAudio (fish-speech-1.5) is a compact model trained with large-scale RLHF that reduces hallucination artifacts common in semantic-only TTS architectures. These artifacts—where the model generates plausible-sounding but phonemically incorrect output—are a known failure mode at this model size.

The shift isn’t only about cost. Local inference bypasses API rate limits, corporate privacy policies, and safety filters. When a smartphone can run a sub-150ms voice agent locally, the argument for routing audio through a cloud API gets harder to make. And if it works just good enough for use cases, why opt for stronger models at the expense of latency?

Long-form audio generation: VibeVoice and low-frame-rate tokenization

Conversational agents and long-form audio generation are different problems. Generating 60 minutes of podcast audio requires the model to keep a consistent speaker profile, emotional arc, and acoustic quality across a context window that most models weren’t designed to handle.

VibeVoice, developed by Microsoft researchers, tackles this using extremely low-frame-rate tokenizers running at 7.5 Hz for both acoustic and semantic encodings. The lower frame rate lowers computational overhead enough that the model can process contexts up to 64,000 tokens. In practice, VibeVoice-1.5B can generate up to 90 minutes of continuous multi-speaker dialogue in a single pass without speaker accent drift or acoustic degradation.

That’s a legitimately hard problem to solve. The frame-rate approach is less flashy than attention mechanism improvements, but it works.

Speech-to-text benchmarks: speed and accuracy

In a voice pipeline with a total latency budget under one second, transcription can’t be the slow part. OpenAI’s Whisper set the floor in 2022. The 2026 field looks different.

Daily.co published a benchmark recently using the open-source pipecat-ai/stt-benchmark across 10 cloud providers. The STT market has quietly compressed.

For years, Deepgram—a San Francisco company that rebuilt ASR from scratch using end-to-end deep learning rather than adapting legacy approaches—was in a category of its own on speed. That gap has closed. Soniox, a newer entrant that prices by token rather than by minute and has quietly accumulated strong accuracy numbers, now sits within milliseconds of Deepgram on latency while beating it on accuracy. Speechmatics, a Cambridge-founded company with one of the longest track records in enterprise ASR, leads the group on accuracy at the cost of slightly higher latency.

Picking an STT provider is now less about finding the best one and more about avoiding the wrong fit. Legacy cloud providers score reasonably on accuracy, but return transcripts in over a second. That’s slow enough to break real-time conversation—not feel slow, actually break it. Providers built specifically for voice agents hit sub-250ms. An agent at 250ms can handle interruptions. One at 1100ms cannot.

The benchmark also changed how accuracy is measured. Instead of raw WER, it tracks Semantic WER: only errors that would actually change how an LLM interprets the transcript. Filler words, punctuation, and minor rephrasing—all ignored. By that standard, the top providers are already good enough that transcription accuracy no longer becomes the bottleneck. The constraint is latency, not correctness.

It’s the same compression happening everywhere in audio AI right now. “Good enough” used to be a consolation; now it’s the target. The useful models are the ones that hit it without wasting compute getting there.

Conclusion

Video generation is stuck on centralized infrastructure. The computing demands are too high to distribute efficiently, and that situation isn’t likely to change soon.

Audio took a different path. The efficiency of audio tokenization enables high-quality speech processing to run at the edge on consumer hardware with a delay that beats human reaction times. That’s a different economics than anything else in AI right now.

The market is splitting. High-value, legally complex production tasks—broadcast dubbing, licensed celebrity voices, enterprise call centers—will consolidate via platforms such as ElevenLabs. Consumer devices and developer environments will run Qwen3-TTS, Moshi, and Kokoro: private, zero-marginal-cost, sub-100ms.

Mainstream tech coverage is fixated on AI video because it’s visually spectacular. But the barrier to entry for audio AI has already collapsed. If you’re building something that needs to talk to people, the local edge model is probably already good enough. You have to pay attention.

References

• Benchmarking STT for Voice Agents
• ElevenLabs raises $500M Series D at $11B valuation
• Qwen3-TTS