Turn idle home computers into inference capacity.

GridWing runs a small draft model on computers that would otherwise sit idle. A datacenter GPU checks the drafts and keeps the tokens it agrees with, so it serves more tokens per second. The output is the same as running the large model alone. Any home computer can take part: MacBooks, Windows PCs, gaming desktops.

Watch the demo › See results

Research prototype · 1.63× throughput in lab tests

1Demo

The idea in under a minute.

Why running models locally is the wrong use of home computers, how drafting and verification work, and what we measured.

gridwing_demo.mp41:09 · no audio

2How it works

Speculative decoding, with the drafts made outside the datacenter.

Lossless by construction: the GPU stays in charge of every token, and home computers only make proposals.

01 / Draft

A home computer proposes tokens

An idle MacBook, Windows PC or gaming desktop runs a small model and guesses the next few tokens for a request that is being served in the cloud.

02 / Verify

The GPU checks them in one pass

The large model scores all proposed tokens in a single forward pass. It keeps the ones it would have produced anyway and replaces the first one it disagrees with.

03 / Fall back

Bad drafts cost nothing

If a device is slow, drops offline, or sends wrong guesses, the GPU just decodes as usual. Drafts can speed serving up, but they can't change what the model outputs.

Speculative
sampling

\tilde{x} \sim q : accept with prob. min (1, \frac{p ( x ~ )}{q ( x ~ )}), else resample from (p - q)_{+}

\tilde{x} \sim q : accept w.p. min (1, \frac{p ( x ~ )}{q ( x ~ )}) else resample from (p - q)_{+}

Lossless

Pr [x] = min (p (x), q (x)) + (p (x) - q (x))_{+} = p (x)

The output is exactly the target model's distribution, whatever the draft model is. A weak or buggy draft model only lowers the acceptance rate. For untrusted devices, the GPU checks drafts against its own samples instead of trusting their reported $q$ , so a malicious device can slow things down but never change the answer.

$\tilde{x}$ draft token · $q$ draft model on a home computer · $p$ target model on the GPU · $(\cdot)_{+} = max (\cdot, 0)$

3Results

More tokens per second from the same GPU. Measured, not projected.

Throughput vs. no drafts

1.63×

Tokens / s on one H800

3,399

Helper computers

Helper computers	1	2	3	4	5	6	7
Tokens / s	1,985	2,667	2,941	3,172	3,287	3,251	3,399

Lab experiment. The target model runs on one NVIDIA H800, and each helper computer (modeled on a MacBook) is emulated on its own GPU. Without drafts the same setup serves 2,085 tokens/s. Next we're testing with real MacBooks and Windows PCs over home internet connections.

4Why

Serving AI is limited by datacenter GPUs. Most of the world's computers aren't in datacenters, and most of the time they're doing nothing.

We want inference to be able to use them.

5Team

Shuibai Zhang

Founder

Shuibai is a third-year CS PhD student at UW–Madison, on leave to build GridWing full-time. His research is on making large language models faster and smarter: diffusion language models, parallel decoding, mixture-of-experts and RL post-training. His work has been published at NeurIPS, ICML, ICLR and COLM, with oral presentations at ICLR 2026, ICML 2025 and ICLR 2025.

He built Open-dCoder, the first fully open diffusion LLM for code, and Molten, a Rust + Metal engine that runs a 24B coding model at 300+ tokens/s on a Mac, up to 1.6× faster than mlx-lm. Before GridWing he was tech lead at an AI startup building local LLMs and agents. That work convinced him home computers are better used to help the cloud than to replace it.

Homepage Email GitHub X