Guides / Computing costs and local AI

We split one AI model across two home computers. In our tests, it got slower

We split Llama 3.1 8B and 3.3 70B across a desktop and 2017 iMac over Wi-Fi. All splits generated text more slowly; bottleneck causes need more tests.

Engineering note

By Steven Michael Barber · Published October 22, 2026 · Updated November 3, 2026 · 6 min read

In part 1 we benchmarked a Dell desktop with no dedicated graphics card. In part 2 we turned a 2017 iMac into an Ubuntu server that reached about 70% of the Dell's speed. This part records our October 2026 test of the question the series started with: can two household computers share one model, pooling their memory to run something bigger than either could alone?

What this part shows: with llama.cpp's RPC mode over home Wi-Fi, every split we measured generated text more slowly than the Dell alone, from 18% slower on a 70B model to more than half slower on an 8B model. Along the way we found that the Dell's 32 GB sits in one memory slot, so it runs on a single memory channel. We suspect that limit matters more than anything the iMac can add, but we have not yet tested a second module. We have not tested a wired connection, more than two machines, or the phones.

How splitting works

Ollama runs each model on one machine, so we used llama.cpp, the open-source engine behind the GGUF model files Ollama downloads. Its RPC mode lets one computer run a model and hand some of the model's layers to a server on another computer. The model has to be loaded only on the main machine; the other one receives its share of the weights over the network.

We used the same official llama.cpp release on both machines (build b11819) and the same model file Ollama had already downloaded, so the weights match parts 1 and 2. On Ubuntu Server the release also needed one small package, libgomp1, before it would start.

Security first. llama.cpp's own documentation calls the RPC server experimental and insecure and says never to run it on an open network. It has no password. So before starting it we turned on the iMac's firewall and allowed only the Dell's address to reach SSH and the RPC port. Nothing was opened on the router. The firewall rule is not the whole protection: we ran the RPC server only while a test was running and stopped it straight afterwards, because anything at the Dell's address can use it while it runs. If you copy this, also reserve the main machine's address in your router, so another device can't inherit it later.

The network we had

Both machines are on Wi-Fi. We measured the link before splitting anything: three 512 MiB transfers each way, using a small Python server on the iMac and Windows' built-in curl.exe.

The Dell-to-iMac direction matters most, because that is how the iMac receives its share of the model.

Each machine alone

We first measured each machine alone with llama.cpp's own benchmark tool on Llama 3.1 8B (4.58 GiB):

Writing text, the part you wait on in a chat, the iMac managed 65% of the Dell's speed. Reading the prompt, it managed only 15%.

The 8B model split across both

We then told llama.cpp to place 8 or 16 of the model's 32 layers on the iMac:

Both splits were slower than either machine on its own. The extra time stayed at roughly 110 to 120 milliseconds per token whether the iMac held 8 layers or 16, which points to the cost of talking over Wi-Fi rather than the iMac's processor. We did not capture the network traffic, so that is a strong suspicion, not a measurement.

A model the Dell should not hold

The real test was a model too big for the Dell. Windows reported about 19 GiB available after a restart, and Llama 3.3 70B at its smallest Ollama quantization (q2_K) is 24.6 GiB. We expected the Dell to fail alone. It didn't: the model ran. Windows gave up most of its file cache to make room, and it may also have trimmed or paged out other programs' memory; we did not measure which.

The split read the prompt 27% faster, which we have not explained, and wrote 18% more slowly. Each was a single short run, so treat both as indicative. q2_K is also the lowest-quality version of the model; it buys size at the cost of answer quality, which we did not test.

Worked example: where 855 milliseconds went

Generating text on a CPU mostly means reading the whole model from memory once per token. Using that idea, we can estimate what the split should have cost.

The Dell alone took 704 ms per token. With 61 of 80 layers, its share is about 704 × 61 ÷ 80 = 537 ms. On the 8B model the iMac read weights at about 26 GB per second, and its 19 layers of this 26.4 GB model are about 6.3 GB, so its share is about 6.3 ÷ 26 = 241 ms. Together that is about 778 ms.

The measured time was 855 ms, leaving about 77 ms per token unexplained. That is consistent with the cost of passing work over Wi-Fi and through the RPC layer, but this simple model also assumes every layer costs the same and reuses the iMac's speed from the 8B test, so part of the gap may be estimation error. On the 8B model the unexplained part was about 115 ms. Either way, the measured result stands: the split was slower than the Dell alone.

What we think limits the Dell

While the 70B test ran, Windows' Task Manager showed something we had missed in part 1: one of two memory slots in use. The Dell's 32 GB is a single module, so its memory works on one channel instead of two. One DDR5-5600 channel can deliver at most 44.8 GB per second. The Dell's 8B result works out to about 41 GB per second, close to that ceiling. A second module would double the channels, and if the pattern holds, roughly double the speed. That 41 GB per second is itself inferred, from the model's size times its token rate, so this is a hypothesis until we test a second module.

If it holds, the bottleneck was never only the Dell's memory size. It was memory speed, plus about 12 GB in use by Windows and a few background apps even at idle.

What comes next

Splitting a model over Wi-Fi did not pay off with these two machines. At the end of this test we identified two alternatives to price and measure:

1. A second memory module for the Dell, which would give it 64 GB on two channels.
2. A used workstation running only Ubuntu Server, with 64 GB of older DDR4 memory across four channels, dedicated to models with no desktop apps competing for memory.

Neither alternative was measured in this experiment. A follow-up needs the same benchmarks, along with a wired-network comparison, before we can say whether either improves the result.