Back to List
Meituan LongCat Team Unveils LongCat-AudioDiT: Advancing Zero-Shot TTS Voice Cloning via Waveform Latent Space Diffusion
Research BreakthroughTTSVoice CloningMeituan

Meituan LongCat Team Unveils LongCat-AudioDiT: Advancing Zero-Shot TTS Voice Cloning via Waveform Latent Space Diffusion

The Meituan LongCat team has officially announced the release of LongCat-AudioDiT, a specialized model designed to push the boundaries of zero-shot Text-to-Speech (TTS) voice cloning. By fundamentally rethinking the audio synthesis pipeline, the team has moved away from traditional intermediate representations such as Mel-spectrograms. Instead, LongCat-AudioDiT operates directly within the waveform latent space using a diffusion-based framework. This strategic shift is intended to eliminate the cascade errors that typically arise during multi-stage data conversion processes in conventional TTS systems. By allowing the AI to learn the inherent patterns of sound directly, the model aims to achieve a higher level of fidelity and accuracy in voice cloning, representing a significant technical breakthrough in the field of generative audio.

美团技术团队
Share

Key Takeaways

  • Breakthrough in Zero-Shot Cloning: Meituan's LongCat team has launched LongCat-AudioDiT to overcome existing limitations in zero-shot voice cloning technology.
  • Elimination of Intermediate Steps: The model completely abandons the use of Mel-spectrograms and other intermediate representations in the synthesis process.
  • Waveform Latent Space Diffusion: LongCat-AudioDiT performs text-to-speech generation directly within the waveform latent space using diffusion models.
  • Reduction of Cascade Errors: By bypassing traditional conversion stages, the architecture prevents the accumulation of errors that often degrade audio quality.
  • Direct Pattern Learning: The system is designed to help AI learn the underlying laws of sound directly, rather than relying on proxy representations.

In-Depth Analysis

Overcoming the Bottlenecks of Traditional TTS

In the evolution of Text-to-Speech (TTS) technology, achieving high-quality zero-shot voice cloning—where a model replicates a voice based on a very short sample without prior training on that specific speaker—has remained a significant challenge. The Meituan LongCat team identified that a primary technical bottleneck lies in the reliance on intermediate representations. Traditionally, TTS systems convert text into a Mel-spectrogram before a separate vocoder transforms that spectrogram into an audible waveform.

LongCat-AudioDiT addresses this by "skipping the middleman." According to the Meituan technical team, the model is designed to let the AI directly learn the inherent laws and patterns of sound itself. By removing the intermediate stages, the team aims to break the current performance ceiling of zero-shot voice cloning, providing a more seamless and integrated approach to audio generation.

The Shift to Waveform Latent Space Diffusion

The core innovation of LongCat-AudioDiT lies in its use of the waveform latent space. Most contemporary diffusion-based TTS models operate on Mel-spectrograms, which are compressed visual representations of audio frequencies. While effective, the conversion between text, Mel-spectrograms, and final waveforms often introduces "cascade errors"—small inaccuracies at each stage that compound to reduce the final output's clarity and resemblance to the target voice.

By implementing a diffusion model (AudioDiT) directly in the waveform latent space, Meituan's approach ensures that the generation process remains closer to the raw audio data. This method blocks the source of data conversion errors at the root. The model focuses on the latent characteristics of the waveform, allowing for a more precise reconstruction of the target voice's unique timbre and prosody. This direct-to-waveform approach represents a fundamental shift in how generative AI handles the complexities of human speech.

Industry Impact

The release of LongCat-AudioDiT marks a pivotal moment for the AI audio industry, particularly in the realm of personalized voice synthesis. By demonstrating that intermediate representations like Mel-spectrograms can be successfully bypassed, Meituan is setting a new architectural standard for high-fidelity voice cloning.

For the broader AI industry, this research highlights the importance of reducing architectural complexity to minimize error propagation. As zero-shot TTS becomes more accurate and easier to deploy, we can expect significant advancements in areas such as digital assistants, content creation, and real-time translation, where the ability to clone a voice accurately and instantly is paramount. LongCat-AudioDiT proves that moving closer to the raw data source—the waveform itself—is a viable and superior path for the next generation of audio AI.

Frequently Asked Questions

Question: What is the main difference between LongCat-AudioDiT and traditional TTS models?

Traditional TTS models usually rely on intermediate representations like Mel-spectrograms to bridge the gap between text and audio. LongCat-AudioDiT abandons these intermediate steps, performing diffusion-based generation directly in the waveform latent space to avoid data conversion errors.

Question: How does LongCat-AudioDiT improve the quality of voice cloning?

By operating directly in the waveform latent space, the model eliminates "cascade errors"—the cumulative inaccuracies that occur when moving between different data formats. This allows the AI to capture the natural laws of sound more accurately, resulting in higher-fidelity zero-shot voice clones.

Question: Who developed LongCat-AudioDiT and what is its primary goal?

LongCat-AudioDiT was developed by the Meituan LongCat technical team. Its primary goal is to break the current technical limits of zero-shot voice cloning and provide a more direct, error-resistant method for high-quality speech synthesis.

Read Original Article

Related News

Meituan LongCat Team Launches General 365: A Rigorous New Benchmark for AI Reasoning
Research Breakthrough

Meituan LongCat Team Launches General 365: A Rigorous New Benchmark for AI Reasoning

The Meituan LongCat team has officially released General 365, a sophisticated evaluation benchmark designed to measure the reasoning capabilities of large language models (LLMs). In an initial assessment of 26 mainstream models, the benchmark revealed a significant performance gap across the industry. Gemini 3 Pro, currently regarded as one of the most capable models, achieved an accuracy rate of only 62.8%. More strikingly, the vast majority of the models tested failed to reach the 60% threshold, which is considered a basic passing grade. This release by Meituan sets a new, more challenging standard for AI evaluation, highlighting that complex reasoning remains a major hurdle for even the most advanced artificial intelligence systems today.

LARYBench: Defining the ImageNet for Embodied Action Representation and Generalization
Research Breakthrough

LARYBench: Defining the ImageNet for Embodied Action Representation and Generalization

The Meituan Technical Team has introduced LARYBench (Latent Action Representation Yielding Benchmark), a systematic evaluation framework designed to measure general latent action representations derived from large-scale visual data. This benchmark marks a significant milestone in embodied AI, often compared to the 'ImageNet' moment for action representation. Experimental findings reveal that general vision models significantly outperform specialized embodied AI expert models in both action generalization and control precision. Crucially, the research demonstrates that embodied action representations can effectively emerge from large-scale human video data, suggesting a new paradigm for training AI to understand and execute physical movements without relying solely on specialized robotic datasets.

Challenging Anthropomorphism: Why Age of Empires II Might Have Human-Like Attributes if LLMs Do
Research Breakthrough

Challenging Anthropomorphism: Why Age of Empires II Might Have Human-Like Attributes if LLMs Do

A provocative research paper by Adrian de Wynter, titled 'If LLMs Have Human-Like Attributes, Then So Does Age of Empires II,' challenges the prevailing tendency in AI research to ascribe anthropomorphic qualities to Large Language Models (LLMs). The study argues that attributes such as morality or natural language understanding, often assumed to emerge in LLMs, are empirically non-unique. By training a simple neural network on the classic videogame Age of Empires II, de Wynter demonstrates that if these attributes are granted to LLMs, they could logically be attributed to any entity within a sufficiently powerful substrate, including LEGO or even the Greater Boston Area. The paper calls for explicit measurement criteria in AI evaluation and proposes a 'null assumption' of non-uniqueness to prevent circular or uninformative conclusions in the field of computation and language.