Lumi / Benchmarks
Benchmarks · Lumi Bare Metal
Measured. Not estimated.
Every number Lumi publishes, with the machine it ran on and how it was measured. Where another engine is as fast, we say so.
The short version
| Result | What was measured | Machine |
|---|---|---|
| 200+ tok/s | Qwen 35B MoE, text generation | RTX 5080 Laptop GPU (16 GB), Core Ultra 9 275HX, 32 GB RAM |
| 8.8× | 200 coding agents at once vs Ollama: 232.9 s vs 2,049.3 s, all 200 finished on both | RTX 3080 (10 GB), 32 GB RAM |
| 100 agents | at full 16K-token context each, 100/100 recall | RTX 3080 (10 GB), 32 GB RAM |
| 20.7–62.6 ms | for a priority request to cut in while 8 agents run, models that fit on the card | RTX 3080 (10 GB), 32 GB RAM |
Lumi's lead is largest when more than one thing is running, and on large Mixture-of-Experts models that don't fit entirely on the graphics card. On small models with a single chat, Lumi is about as fast as Ollama, and sometimes a little slower. Both are below.
Test machines
- Laptop: NVIDIA GeForce RTX 5080 Laptop GPU (16 GB), Intel Core Ultra 9 275HX, 32 GB RAM, Windows 11.
- Desktop: NVIDIA GeForce RTX 3080 (10 GB), 32 GB RAM, Windows. Measured October–November 2026.
Every engine loaded the same model files (GGUF) at the same 16,384-token context length. Comparisons are against stock Ollama, llama.cpp's server and LM Studio, each as published.
200 coding agents vs Ollama
We ran 200 copies of unmodified Aider v0.86.2 at the same time against Qwen3 8B, each with a 16,384-token context. Each agent wrote a different C++20 data structure, then a second round of 200 agents reviewed the code, and every result was compiled with MSVC (cl.exe /std:c++20 /O2) and its test binary run. Lumi was reached through its Ollama-compatible endpoint, so Aider didn't change. Thinking was switched off on both engines, so both produced the same kind of answer.
| Engine | 200 builds | 200 reviews | Total wall time | Compiled and passed |
|---|---|---|---|---|
| Lumi | 173.4 s | 59.5 s | 232.9 s | 200 / 200 |
| Ollama (thinking off) | 1,633.5 s | 415.8 s | 2,049.3 s | 200 / 200 |
Eight agents at once, six models
Total generation speed with eight agents working at the same time, and how long a new priority request waited before it was answered. RTX 3080 (10 GB); larger models run partly from system RAM on every engine.
| Model | Lumi | Ollama | llama.cpp server | LM Studio | Cut-in: Lumi | Cut-in: Ollama |
|---|---|---|---|---|---|---|
| Qwen3 8B | 287.8 | 96.0 | 48.3 | 41.9 | 66 ms | 2.6 s |
| Qwen2.5 Coder 7B | 321.6 | 107.0 | 57.7 | 48.9 | 56 ms | 2.3 s |
| gpt-oss 20B | 105.5 | 33.8 | 21.9 | 21.0 | 254 ms | 7.4 s |
| Qwen3 30B-A3B (MoE) | 145.2 | 18.8 | 11.7 | 11.2 | 127 ms | 13.4 s |
| Qwen3 Coder 30B-A3B (MoE) | 149.8 | 18.2 | 11.4 | 10.8 | 119 ms | 13.8 s |
| Qwen 35B-A3B (MoE) | 62.5 | 10.3 | 5.7 | 6.0 | 443 ms | 4.7 s |
Tokens per second, all eight agents combined. Lumi figures are with its adaptive generation on.
A single chat
One conversation, one stream of text. This is where most local engines are already fast, and where the differences are small.
| Model | Lumi | Ollama |
|---|---|---|
| Qwen3 8B | 84.9–99.1 | 103.0 |
| Qwen2.5 Coder 7B | 95.1–111.9 | 112.9 |
| gpt-oss 20B | 50.3–53.6 | 47.4 |
| Qwen3 30B-A3B (MoE) | 46.3–51.6 | 33.7 |
| Qwen 35B-A3B (MoE) | 51.2–59.7 | 27.7 |
Tokens per second, RTX 3080 (10 GB). The Lumi range covers its two generation settings.
Where Ollama is as fast: on small models that sit entirely on the graphics card, a single chat runs at about the same speed on Ollama, and in our runs slightly faster. If that's all you need, Ollama is a fine choice. Lumi is built for the moment a second thing starts: another agent, a background job, your own question while the swarm works.
100 agents at full context
A hundred agents ran at once on one RTX 3080, each given a 15,386-token code header (94% of its 16,384-token window) with three values hidden at the start, the middle and the end. Every agent had to find all three to answer correctly. All 100 did: 1,538,592 prompt tokens processed in 16.85–19.43 seconds of wall time, with no agent's context shortened.
Your request first
With eight agents generating in the background, a new priority request was answered after 20.7–62.6 ms on models that fit entirely on the RTX 3080, and 119–444 ms on the 20–35B MoE models that run partly from system RAM. The same request waited 0.6–13.8 seconds on Ollama, depending on the model.
What we haven't published yet
The full method for the RTX 5080 laptop run (prompt and output lengths), the exact Ollama version and settings, and the raw logs and test scripts for every table on this page will be published with the beta, so you can run them on your own machine.
Changelog
13 November 2026: first publication.
Private beta
The private beta opens in waves.
Windows PCs with NVIDIA RTX graphics first. The core is free for local use.