Back to list
Meituan LongCat Team Unveils LongCat-AudioDiT: Advancing Zero-Shot TTS Voice Cloning via Waveform Latent Space Diffusion
Research BreakthroughTTSVoice CloningMeituan

Meituan LongCat Team Unveils LongCat-AudioDiT: Advancing Zero-Shot TTS Voice Cloning via Waveform Latent Space Diffusion

The Meituan LongCat team has officially announced the release of LongCat-AudioDiT, a specialized model designed to push the boundaries of zero-shot Text-to-Speech (TTS) voice cloning. By fundamentally rethinking the audio synthesis pipeline, the team has moved away from traditional intermediate representations such as Mel-spectrograms. Instead, LongCat-AudioDiT operates directly within the waveform latent space using a diffusion-based framework. This strategic shift is intended to eliminate the cascade errors that typically arise during multi-stage data conversion processes in conventional TTS systems. By allowing the AI to learn the inherent patterns of sound directly, the model aims to achieve a higher level of fidelity and accuracy in voice cloning, representing a significant technical breakthrough in the field of generative audio.

美团技术团队
Share

Key Takeaways

  • Breakthrough in Zero-Shot Cloning: Meituan's LongCat team has launched LongCat-AudioDiT to overcome existing limitations in zero-shot voice cloning technology.
  • Elimination of Intermediate Steps: The model completely abandons the use of Mel-spectrograms and other intermediate representations in the synthesis process.
  • Waveform Latent Space Diffusion: LongCat-AudioDiT performs text-to-speech generation directly within the waveform latent space using diffusion models.
  • Reduction of Cascade Errors: By bypassing traditional conversion stages, the architecture prevents the accumulation of errors that often degrade audio quality.
  • Direct Pattern Learning: The system is designed to help AI learn the underlying laws of sound directly, rather than relying on proxy representations.

In-Depth Analysis

Overcoming the Bottlenecks of Traditional TTS

In the evolution of Text-to-Speech (TTS) technology, achieving high-quality zero-shot voice cloning—where a model replicates a voice based on a very short sample without prior training on that specific speaker—has remained a significant challenge. The Meituan LongCat team identified that a primary technical bottleneck lies in the reliance on intermediate representations. Traditionally, TTS systems convert text into a Mel-spectrogram before a separate vocoder transforms that spectrogram into an audible waveform.

LongCat-AudioDiT addresses this by "skipping the middleman." According to the Meituan technical team, the model is designed to let the AI directly learn the inherent laws and patterns of sound itself. By removing the intermediate stages, the team aims to break the current performance ceiling of zero-shot voice cloning, providing a more seamless and integrated approach to audio generation.

The Shift to Waveform Latent Space Diffusion

The core innovation of LongCat-AudioDiT lies in its use of the waveform latent space. Most contemporary diffusion-based TTS models operate on Mel-spectrograms, which are compressed visual representations of audio frequencies. While effective, the conversion between text, Mel-spectrograms, and final waveforms often introduces "cascade errors"—small inaccuracies at each stage that compound to reduce the final output's clarity and resemblance to the target voice.

By implementing a diffusion model (AudioDiT) directly in the waveform latent space, Meituan's approach ensures that the generation process remains closer to the raw audio data. This method blocks the source of data conversion errors at the root. The model focuses on the latent characteristics of the waveform, allowing for a more precise reconstruction of the target voice's unique timbre and prosody. This direct-to-waveform approach represents a fundamental shift in how generative AI handles the complexities of human speech.

Industry Impact

The release of LongCat-AudioDiT marks a pivotal moment for the AI audio industry, particularly in the realm of personalized voice synthesis. By demonstrating that intermediate representations like Mel-spectrograms can be successfully bypassed, Meituan is setting a new architectural standard for high-fidelity voice cloning.

For the broader AI industry, this research highlights the importance of reducing architectural complexity to minimize error propagation. As zero-shot TTS becomes more accurate and easier to deploy, we can expect significant advancements in areas such as digital assistants, content creation, and real-time translation, where the ability to clone a voice accurately and instantly is paramount. LongCat-AudioDiT proves that moving closer to the raw data source—the waveform itself—is a viable and superior path for the next generation of audio AI.

Frequently Asked Questions

Question: What is the main difference between LongCat-AudioDiT and traditional TTS models?

Traditional TTS models usually rely on intermediate representations like Mel-spectrograms to bridge the gap between text and audio. LongCat-AudioDiT abandons these intermediate steps, performing diffusion-based generation directly in the waveform latent space to avoid data conversion errors.

Question: How does LongCat-AudioDiT improve the quality of voice cloning?

By operating directly in the waveform latent space, the model eliminates "cascade errors"—the cumulative inaccuracies that occur when moving between different data formats. This allows the AI to capture the natural laws of sound more accurately, resulting in higher-fidelity zero-shot voice clones.

Question: Who developed LongCat-AudioDiT and what is its primary goal?

LongCat-AudioDiT was developed by the Meituan LongCat technical team. Its primary goal is to break the current technical limits of zero-shot voice cloning and provide a more direct, error-resistant method for high-quality speech synthesis.

Read original article

Related News

Optimizing High-Concurrency Generative SLM Inference Engines Under Strict Latency Constraints for Search and Recommendation
Research Breakthrough

Optimizing High-Concurrency Generative SLM Inference Engines Under Strict Latency Constraints for Search and Recommendation

Meituan's technical team has unveiled an advanced high-concurrency inference engine tailored for Small Language Models (SLMs) with 0.5B to 1B parameters deployed in online search and recommendation systems. While traditional large language model serving architectures are tailored for open-ended text generation, search and recommendation platforms operate under extreme low-latency boundaries where standard systems falter. By categorizing tasks into bounded generation workloads—such as prefill-only classification and fixed-step beam search—the engineering team restructured the execution path using a static Directed Acyclic Graph (DAG) abstraction. Integrated with PyTorch AOTInductor compilation, packed prefill, intra-request KV cache sharing, and selective FP8 quantization, the engine keeps end-to-end latency strictly below 20 milliseconds. The system achieves a prefill-only effective Model FLOPs Utilization (MFU) of 67% and delivers a tenfold throughput improvement over SGLang, establishing a scalable paradigm for enterprise AI deployment.

Research Breakthrough

How Third Moment Research Deploys 10,000 Autonomous Agents for Over 36 Hours of Continuous Operation

AI laboratory Third Moment Research has demonstrated an operational architecture capable of running 10,000 agents continuously for 36 or more hours. Featured on the OpenAI Blog, the initiative highlights systems designed to maintain sustained focus for days while actively improving with continued use. To achieve this scale and long-duration execution, Third Moment Research leverages a suite of foundational models and specialized tools, including GPT-6 Astra, GPT-6.1 Sol, Codex, and Daybreak for cybersecurity operations. This milestone underscores substantial advancements in multi-agent orchestration, agentic endurance, autonomous software engineering, and defensive cybersecurity workflows, pointing toward scalable, self-improving multi-agent environments.

Mathematicians Face Years of Analysis as OpenAI Abruptly Drops Unprecedented Flood of Mathematical Results
Research Breakthrough

Mathematicians Face Years of Analysis as OpenAI Abruptly Drops Unprecedented Flood of Mathematical Results

OpenAI has abruptly published a massive volume of mathematical findings, leaving the global academic community in a state of profound shock and contemplation. In discussions with The Verge, more than three dozen leading mathematicians characterized the unexpected release with terms ranging from 'staggering' and 'overwhelming' to 'unprecedented,' 'surreal,' and 'pure insanity.' The sudden drop has triggered widespread awe and excitement alongside deep uncertainty regarding how researchers will process and verify the sheer scale of the output. Academic experts indicate that sorting through and making sense of this deluge of AI-generated mathematics will demand years of dedicated effort, marking a transformative moment in the relationship between commercial frontier artificial intelligence labs and pure academic sciences.