BaseRT

BaseRT: The Fastest High-Performance AI Inference Runtime for Apple Silicon

BaseRT is the premier high-performance inference engine designed specifically for the Apple Silicon ecosystem. As the industry's fastest runtime on Apple Silicon, BaseRT provides developers and engineers with an optimized platform for running large language models (LLMs) and other open-source models with unprecedented efficiency and speed.

Built to leverage the unique architecture of Apple’s M-series chips, BaseRT eliminates the latency typically associated with cloud-based AI while ensuring that data privacy remains a top priority. Whether you are working on complex research projects or building production-ready enterprise applications, BaseRT offers the speed and reliability required for modern on-device AI workloads.

What's BaseRT?

BaseRT is a specialized runtime environment engineered to maximize the throughput and minimize the latency of AI models running on Apple hardware. In a landscape where speed is critical, BaseRT positions itself as the fastest solution available for Apple Silicon users, consistently outperforming other popular frameworks like MLX and llama.cpp.

At its core, BaseRT is about performance and local control. By using BaseRT, users can serve powerful models directly on their machines without the need for external API keys or cloud subscriptions. This local-first approach ensures that no data leaves the device, providing a secure environment for sensitive coding, research, and data processing tasks.

Key Features of BaseRT

BaseRT stands out due to its technical superiority and developer-centric design. Here are the core features that make BaseRT the leading choice for Apple Silicon:

• Industry-Leading Speed: BaseRT is significantly faster than competing runtimes, offering massive gains in both Prefill and Decode stages.
• Optimized for Apple Silicon: Specifically tuned for the M-series architecture, including the latest Apple M5 Pro benchmarks.
• Data Privacy and Security: Since BaseRT runs locally, there is no need for data to leave your device, ensuring total privacy.
• No API Keys Required: Users can run open-source models without managing tokens, credits, or third-party service agreements.
• Comprehensive Model Support: BaseRT supports a wide array of state-of-the-art models, including the latest versions of Llama, Qwen, and Gemma.
• Local Coding Agent Integration: Seamlessly integrate with local coding tools to keep your development environment private and fast.

BaseRT Performance Benchmarks

When it comes to raw performance, BaseRT demonstrates a clear lead over alternatives like MLX and llama.cpp. Benchmarks conducted on an Apple M5 Pro show that BaseRT excels in two critical areas of LLM performance: Decode and Prefill.

Decode Performance (Tokens / Sec)

In the Decode phase (TG128), BaseRT consistently provides higher throughput compared to its competitors:

• Qwen3 0.6B (Q4): BaseRT achieves 531 tokens/sec, which is 33% faster than MLX (398) and 37% faster than llama.cpp (386).
• Llama 3.2 1B (Q4): BaseRT reaches 342 tokens/sec, outperforming MLX by 15% (298) and llama.cpp by 28% (267).
• Llama 3.2 3B (Q4): BaseRT delivers 137 tokens/sec, maintaining a 5% lead over MLX and a 14% lead over llama.cpp.

Prefill Performance (Tokens / Sec)

The Prefill stage highlights the true power of BaseRT, showing massive performance multipliers:

• Qwen3 30B-A3B (PP128, Q4): BaseRT records a staggering 2,478 tokens/sec. This is 288% faster than MLX (639) and 94% faster than llama.cpp (1,280).
• Gemma 4 E2B (PP2048, Q8): BaseRT hits an incredible 16,264 tokens/sec, which is 32% faster than MLX (12,355) and a massive 539% faster than llama.cpp (2,547).

Supported Models in BaseRT

BaseRT is designed to be versatile, supporting a wide range of popular open-source architectures. Currently supported models include:

• Qwen Family: Qwen3, Qwen3.5, and Qwen3.6.
• Llama Family: Llama 3.1 and Llama 3.2.
• Gemma Family: Gemma 3 and Gemma 4.
• Mistral: High-performance execution for Mistral models.
• Phi-3: Efficient runtime for Microsoft's Phi-3 series.
• Nomic BERT: Optimized support for BERT-based embedding and classification tasks.

Use Case: Local Coding Agents with BaseRT

One of the most powerful applications for BaseRT is powering local coding agents. By serving a model through BaseRT, developers can point their coding agents to the local instance, ensuring that all code analysis and generation happen entirely on the local machine.

Using BaseRT for coding agents offers several advantages:

• No Data Leakage: Your proprietary source code never leaves your device to be processed by a third-party API.
• Zero Latency: Local execution via BaseRT removes the network lag associated with remote server requests.
• Offline Capability: You can continue to use high-powered AI assistance even without an internet connection.

How to Use BaseRT

Getting started with BaseRT is straightforward and can be accomplished via the terminal on your Apple Silicon device.

1. Installation

To install BaseRT, run the following command in your terminal:

$ curl -LsSf https://basecompute.co/install.sh | sh

2. Serving a Model

Once installed, you can serve a model using the basert serve command. For example, to serve Gemma 4:

# Serve a model with BaseRT
basert serve basecompute/gemma-4-E4B-it

3. Setting Up a Coding Agent

You can then install the coding agent plugin and run your local environment:

# Install the coding-agent plugin
pi install git:github.com/basecompute/pi-basert

# Run it — everything is set
pi

FAQ

Q: Is BaseRT really the fastest runtime for Apple Silicon? A: Yes. Benchmarks show that BaseRT is up to 6.4x faster than llama.cpp and up to 3.9x faster than MLX in prefill tasks on Apple Silicon hardware.

Q: Do I need an internet connection to use BaseRT? A: Once the models are downloaded, BaseRT runs entirely locally on your machine. No data leaves your device, and no active internet connection is required for inference.

Q: Which Apple chips are supported? A: BaseRT is optimized for Apple Silicon, including the M1, M2, M3, M4, and the latest M5 series, such as the M5 Pro.

Q: Can I use BaseRT for commercial development? A: BaseRT is ideal for engineers building on-device AI and working with open-source models, providing an enterprise-ready environment for local AI execution.

Q: Where can I find community support for BaseRT? A: You can join the BaseRT community on Discord to connect with other engineers and developers building on-device AI solutions.