Cerebras Inference Platform Achieves Record Speeds with Qwen 3.8 27B and OpenAI GPT OSS 120B

Key Takeaways

• Exceptional Inference Speeds: Qwen 3.8 27B achieves ~1500 tokens/s, and OpenAI GPT OSS 120B reaches ~3000 tokens/s on Cerebras infrastructure.
• Unpruned Model Integrity: Cerebras hosts original, unpruned versions of open-source models to ensure maximum output quality and architectural transparency.
• Advanced Context Support: Context windows are optimized for both free and paid tiers, ranging from 64k to 131k depending on the model and subscription level.
• Sophisticated Quantization Strategy: The platform employs selective weight-only quantization (4-bit to 16-bit) for storage while maintaining high-precision execution for sensitive layers.
• Flexible Access Tiers: Users can access these high-speed models via public endpoints (Free/Pay-as-you-go) or through Dedicated Endpoints for production-grade SLAs.

In-Depth Analysis

Breakthrough Performance in LLM Inference

The latest documentation from Cerebras Inference reveals a significant leap in processing speeds for large language models (LLMs). The Qwen 3.8 27B model, a 27-billion parameter powerhouse, is now clocked at approximately 1500 tokens per second. Even more striking is the performance of the OpenAI GPT OSS 120B model, which, despite its massive 120-billion parameter size, achieves a throughput of ~3000 tokens per second. These speeds represent a critical benchmark for real-time AI applications where latency and throughput are paramount.

To support these high speeds across various use cases, Cerebras has defined specific context window limits. For the Qwen 3.8 27B model, the context window is set at 64k for the free tier and 128k for the paid tier. The GPT OSS 120B model offers a slightly larger range, with 65k for free users and 131k for paid subscribers. This tiered approach allows developers to scale their applications from initial testing to large-scale document processing without switching platforms.

Model Quality and Compression Transparency

A central theme in the Cerebras documentation is the preservation of model quality through rigorous architectural standards. Unlike many platforms that utilize pruned models to save on computational costs, Cerebras explicitly states that it does not host pruned models on its public endpoints. All models available are the original, unpruned versions. While the company continues to conduct research into pruning techniques—such as REAP (Router-weighted Expert Activation Pruning)—these experimental models are reserved for the research community on Hugging Face and are not part of the standard API offering.

To balance the needs of storage efficiency and model fidelity, Cerebras utilizes selective weight-only quantization. This process involves storing weights in partial 16-bit, 8-bit, or 4-bit formats, which aligns with current industry standards. However, the platform distinguishes itself by ensuring that quality-sensitive layers are stored at full precision. By performing dequantization on the fly, Cerebras ensures that the actual operations—including activations and attention mechanisms—are conducted in high precision. This technical choice is designed to preserve the maximal quality of the original model while optimizing the underlying hardware storage.

Infrastructure and Deployment Options

Cerebras provides a structured path for developers to transition from experimentation to production. The public endpoints are designed for accessibility, offering a free trial and a pay-as-you-go tier. These are subject to rate limits and standard pricing models. For enterprises requiring more robust solutions, Cerebras offers Dedicated Endpoints. These reserved capacities provide higher throughput, additional model families, and production-level Service Level Agreements (SLAs).

The documentation also highlights a user-friendly onboarding process, including a Quickstart guide for making initial API calls and a model selection guide to help users choose the appropriate model based on their specific use case. This infrastructure is built to handle the complexities of modern LLMs while providing the transparency needed for developers to understand exactly how their models are being stored and executed.

Industry Impact

The availability of Qwen 3.8 27B and GPT OSS 120B at these speeds significantly lowers the barrier for high-performance AI integration. By delivering thousands of tokens per second, Cerebras is enabling a new class of responsive AI applications that were previously hindered by the slow inference speeds of large-scale models. Furthermore, the commitment to unpruned models and high-precision operations sets a high standard for quality in the inference-as-a-service market. This approach ensures that the intelligence of the original open-source models is not compromised for the sake of speed, providing a reliable foundation for developers who prioritize accuracy and architectural integrity.

Frequently Asked Questions

Question: What are the specific inference speeds for the new models on Cerebras?

Answer: The Qwen 3.8 27B model achieves approximately 1500 tokens per second, while the OpenAI GPT OSS 120B model reaches approximately 3000 tokens per second.

Question: Does Cerebras use pruned models for its public API endpoints?

Answer: No. Cerebras hosts only the original, unpruned versions of models on its public endpoints to preserve maximal quality. While they research pruning techniques like REAP, those models are not available through the shared API.

Question: How does Cerebras handle model quantization without losing quality?

Answer: Cerebras uses selective weight-only quantization during storage (16-bit, 8-bit, or 4-bit). Sensitive layers are kept at full precision, and dequantization is performed on the fly so that operations are executed in high precision.