Meituan LongCat Launches General 365: A New Benchmark Highlighting AI Reasoning Gaps

Key Takeaways

• New Evaluation Standard: Meituan's LongCat team has open-sourced General 365, a benchmark specifically designed to measure reasoning in AI models.
• Performance Benchmark: In a test of 26 mainstream models, Gemini 3 Pro emerged as the leader but only achieved a 62.8% accuracy rate.
• Widespread Failure: The majority of tested models failed to reach the 60% accuracy threshold, indicating a significant gap in current reasoning capabilities.
• Industry Benchmark: General 365 aims to set a new, more challenging standard for evaluating the logical depth of Large Language Models (LLMs).

In-Depth Analysis

The Introduction of General 365 by Meituan LongCat

The Meituan LongCat team has officially introduced General 365, an open-source evaluation framework that focuses on the reasoning performance of AI models. As the AI industry moves beyond basic conversational tasks, the ability to perform complex reasoning has become a critical differentiator for Large Language Models. By releasing General 365, the LongCat team provides a new tool for the developer community to rigorously assess how well models can handle logical challenges. This benchmark is positioned as a "new yardstick" for the industry, suggesting that existing metrics may not be sufficiently challenging for the current generation of AI.

Analyzing the Performance of Mainstream Models

The initial results released alongside General 365 offer a sobering look at the current state of AI reasoning. The LongCat team conducted practical tests on 26 mainstream models to establish a baseline for the benchmark. The findings reveal that even the most advanced models currently available struggle with the tasks presented in General 365.

Gemini 3 Pro, which is widely regarded as one of the most capable models in the world, recorded an accuracy rate of 62.8%. While this was the highest score among the 26 models tested, it highlights a significant ceiling in current reasoning technology. The fact that the "strongest" model is only slightly above a 60% accuracy level suggests that there is substantial room for improvement in how AI handles complex logical structures.

The 60% Accuracy Threshold

One of the most critical findings from the LongCat team's report is the performance of the broader field of AI models. According to the data, the vast majority of the 26 models tested failed to reach the 60% accuracy mark. In many academic and professional contexts, 60% is considered the minimum threshold for a passing grade.

This widespread inability to reach a basic level of proficiency on the General 365 benchmark indicates that reasoning remains a fundamental weakness for most LLMs. The results suggest that while models may be proficient at generating text or following simple instructions, their ability to navigate intricate reasoning paths is still underdeveloped. This gap between general performance and reasoning accuracy is what General 365 seeks to highlight and eventually help bridge.

Industry Impact

The release of General 365 by Meituan's LongCat team is likely to have a significant impact on how AI models are developed and marketed. By providing a benchmark where even the top-performing models struggle, Meituan is pushing the industry toward more rigorous self-assessment.

For the AI research community, these results serve as a call to action. The 62.8% accuracy of Gemini 3 Pro and the sub-60% performance of most other models provide a clear metric for future iterations. Developers now have a specific target to aim for as they work to improve the logical consistency and problem-solving depth of their systems. Furthermore, as an open-source tool, General 365 allows for transparent and standardized comparisons across different organizations, fostering a more competitive and data-driven environment for AI advancement.

Frequently Asked Questions

Question: What is the primary purpose of the General 365 benchmark?

General 365 was created by the Meituan LongCat team to serve as a new standard for evaluating the reasoning capabilities of Large Language Models. It aims to provide a more difficult and accurate measure of logical performance than previous benchmarks.

Question: Which model performed the best on the General 365 test?

Based on the initial testing of 26 mainstream models, Gemini 3 Pro achieved the highest accuracy rate at 62.8%. However, this score also highlights the significant challenges that remain in AI reasoning.

Question: Why is the 60% accuracy mark significant in this report?

The 60% mark is often viewed as a passing threshold. The fact that most mainstream models failed to reach this level on General 365 underscores the current limitations of AI when it comes to complex reasoning tasks.