A home computer proposes tokens
An idle MacBook, Windows PC or gaming desktop runs a small model and guesses the next few tokens for a request that is being served in the cloud.
Turn idle home computers into inference capacity.
GridWing runs a small draft model on computers that would otherwise sit idle. A datacenter GPU checks the drafts and keeps the tokens it agrees with, so it serves more tokens per second. The output is the same as running the large model alone. Any home computer can take part: MacBooks, Windows PCs, gaming desktops.
Research prototype · 1.63× throughput on one H800
1Demo
The idea in under a minute.
Why running models locally is the wrong use of home computers, how drafting and verification work, and what we measured.
2How it works
Speculative decoding, with the drafts made outside the datacenter.
The GPU stays in charge of every token. Home computers only make proposals.
01 / Draft
02 / Verify
The GPU checks them in one pass
The large model scores all proposed tokens in a single forward pass. It keeps the ones it would have produced anyway and replaces the first one it disagrees with.
03 / Fall back
Bad drafts cost nothing
If a device is slow, drops offline, or sends wrong guesses, the GPU just decodes as usual. Drafts can speed serving up, but they can't change what the model outputs.
3Results
More tokens per second from the same GPU. Measured, not projected.
Throughput vs. no drafts
1.63×
Tokens / s on one H800
3,399
Helper computers
7
| Helper computers | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|
| Tokens / s | 1,985 | 2,667 | 2,941 | 3,172 | 3,287 | 3,251 | 3,399 |
Lab experiment. The target model runs on one NVIDIA H800, and each helper computer (modeled on a MacBook) is emulated on its own GPU. Without drafts the same setup serves 2,085 tokens/s. Next we're testing with real MacBooks and Windows PCs over home internet connections.
4Why
Serving AI is limited by datacenter GPUs. Most of the world's computers aren't in datacenters, and most of the time they're doing nothing.
We want inference to be able to use them.
5Team
Shuibai Zhang
Founder
CS PhD student at UW–Madison, on leave to work on GridWing. Research on efficient LLM inference and post-training (NeurIPS 2026, COLM 2026). Before this, tech lead at an AI startup building local LLMs and agents. Wrote Molten, a Rust + Metal inference engine for Apple Silicon.