Back to list
Meituan LongCat Team Unveils WBench: The First Systematic Multi-Round Benchmark for Interactive Video World Models
Research BreakthroughMeituanWorld ModelsOpen Source

Meituan LongCat Team Unveils WBench: The First Systematic Multi-Round Benchmark for Interactive Video World Models

The Meituan LongCat team has announced the open-sourcing of WBench, a groundbreaking evaluation framework designed to measure the performance of interactive video world models. As the first systematic multi-round benchmark in this field, WBench serves as a diagnostic tool—likened to a 'CT scanner'—to identify the technical bottlenecks encountered when AI transitions from passive video generation to active, multi-turn interaction. By testing models across diverse scenarios ranging from lunar environments to futuristic urban settings, WBench aims to define the current boundaries of world models and provide a clear roadmap for future development in interactive artificial intelligence.

美团技术团队
Share

Key Takeaways

  • Introduction of WBench: Meituan's LongCat team has developed and open-sourced WBench, the first systematic benchmark for evaluating interactive video world models.
  • Focus on Interaction: Unlike previous benchmarks that focus on passive viewing, WBench evaluates the model's ability to handle active, multi-round interactions.
  • Diagnostic Capabilities: The tool is described as a 'CT scanner' for AI, capable of precisely locating where models fail during the transition from observation to interaction.
  • Open-Source Contribution: By making WBench public, the Meituan LongCat team provides the industry with a standardized method to measure the boundaries of world model capabilities.

In-Depth Analysis

Bridging the Gap Between Observation and Interaction

The development of world models has traditionally focused on the generation of realistic video content, often resulting in systems that are excellent at 'passive viewing' but struggle when required to engage in 'active interaction.' The Meituan LongCat team identifies this transition as a critical frontier in AI research. WBench is specifically engineered to address this gap. By implementing a systematic multi-round evaluation process, the benchmark tests how a model maintains consistency, logic, and responsiveness over several layers of interaction. This shift is essential for moving toward AI that can not only depict a world but also function within it dynamically.

The 'CT Scanner' Approach to AI Evaluation

One of the most significant aspects of WBench is its diagnostic nature. The LongCat team utilizes the metaphor of a 'CT scanner' to describe the benchmark's functionality. In the context of complex world models—which may involve intricate physics, spatial reasoning, and temporal consistency—identifying the exact point of failure is often difficult. WBench provides the granularity needed to pinpoint these 'stuck' points. Whether the model fails in maintaining environmental persistence during a 'moonwalk' scenario or loses architectural coherence in a 'cyber city' setting, WBench offers a structured way to visualize and analyze these limitations. This precision allows developers to move beyond general performance metrics and focus on specific algorithmic improvements.

Defining the Boundaries of Simulated Worlds

The scope of WBench is broad, covering a variety of simulated environments to ensure a comprehensive evaluation. The mention of scenarios ranging from 'moonwalks' to 'cyber cities' suggests that the benchmark tests models against both low-gravity physical simulations and complex, high-density urban environments. By testing the boundaries of these world models, WBench establishes a baseline for what current technology can achieve and highlights the remaining hurdles in creating truly interactive, high-fidelity digital universes. This systematic approach ensures that as world models evolve, their progress can be measured against a consistent and rigorous set of interactive standards.

Industry Impact

The release of WBench by the Meituan LongCat team is poised to have a significant impact on the AI research community. By providing the first systematic multi-round evaluation benchmark, Meituan is filling a critical void in the current AI ecosystem. Standardized evaluation is often a precursor to rapid innovation; just as ImageNet accelerated computer vision, WBench could accelerate the development of interactive world models by providing a common language for performance. Furthermore, the decision to open-source the tool encourages collaborative improvement and transparency, allowing researchers worldwide to test their models against the same 'CT scanner' and collectively push the boundaries of what interactive AI can achieve.

Frequently Asked Questions

Question: What makes WBench different from existing video evaluation benchmarks?

Unlike traditional benchmarks that primarily evaluate the visual quality or coherence of a single generated video (passive viewing), WBench is the first to systematically measure multi-round interactions. It focuses on how a model responds to active inputs over time, making it a unique tool for evaluating 'interactive' world models.

Question: Why does the Meituan LongCat team refer to WBench as a 'CT scanner'?

The term 'CT scanner' is used to highlight the benchmark's ability to perform a deep, diagnostic analysis of a model. It doesn't just give a pass/fail grade; it identifies the specific technical areas where a world model 'gets stuck' or fails to maintain interaction logic, allowing for more targeted research and development.

Question: Is WBench available for public use?

Yes, the Meituan LongCat team has open-sourced WBench, making it available for the global AI research community to use, evaluate, and build upon to advance the field of interactive world models.

Read original article

Related News

Optimizing High-Concurrency Generative SLM Inference Engines Under Strict Latency Constraints for Search and Recommendation
Research Breakthrough

Optimizing High-Concurrency Generative SLM Inference Engines Under Strict Latency Constraints for Search and Recommendation

Meituan's technical team has unveiled an advanced high-concurrency inference engine tailored for Small Language Models (SLMs) with 0.5B to 1B parameters deployed in online search and recommendation systems. While traditional large language model serving architectures are tailored for open-ended text generation, search and recommendation platforms operate under extreme low-latency boundaries where standard systems falter. By categorizing tasks into bounded generation workloads—such as prefill-only classification and fixed-step beam search—the engineering team restructured the execution path using a static Directed Acyclic Graph (DAG) abstraction. Integrated with PyTorch AOTInductor compilation, packed prefill, intra-request KV cache sharing, and selective FP8 quantization, the engine keeps end-to-end latency strictly below 20 milliseconds. The system achieves a prefill-only effective Model FLOPs Utilization (MFU) of 67% and delivers a tenfold throughput improvement over SGLang, establishing a scalable paradigm for enterprise AI deployment.

Research Breakthrough

How Third Moment Research Deploys 10,000 Autonomous Agents for Over 36 Hours of Continuous Operation

AI laboratory Third Moment Research has demonstrated an operational architecture capable of running 10,000 agents continuously for 36 or more hours. Featured on the OpenAI Blog, the initiative highlights systems designed to maintain sustained focus for days while actively improving with continued use. To achieve this scale and long-duration execution, Third Moment Research leverages a suite of foundational models and specialized tools, including GPT-6 Astra, GPT-6.1 Sol, Codex, and Daybreak for cybersecurity operations. This milestone underscores substantial advancements in multi-agent orchestration, agentic endurance, autonomous software engineering, and defensive cybersecurity workflows, pointing toward scalable, self-improving multi-agent environments.

Mathematicians Face Years of Analysis as OpenAI Abruptly Drops Unprecedented Flood of Mathematical Results
Research Breakthrough

Mathematicians Face Years of Analysis as OpenAI Abruptly Drops Unprecedented Flood of Mathematical Results

OpenAI has abruptly published a massive volume of mathematical findings, leaving the global academic community in a state of profound shock and contemplation. In discussions with The Verge, more than three dozen leading mathematicians characterized the unexpected release with terms ranging from 'staggering' and 'overwhelming' to 'unprecedented,' 'surreal,' and 'pure insanity.' The sudden drop has triggered widespread awe and excitement alongside deep uncertainty regarding how researchers will process and verify the sheer scale of the output. Academic experts indicate that sorting through and making sense of this deluge of AI-generated mathematics will demand years of dedicated effort, marking a transformative moment in the relationship between commercial frontier artificial intelligence labs and pure academic sciences.