Measured results

Every number on this page comes from a run artifact in the repository. Nothing here is projected. Where a result is unflattering, it is on the page anyway — a benchmark you cannot reproduce is marketing.

Environment

Date	2026-08-03
Workload	500 synthetic sentences, ~26 tokens each
Model	nomic-embed-text (768-d) via Ollama
Hardware	MacBook Air (Apple Silicon) × 2
Transport	Worker A on localhost, worker B over a Cloudflare quick tunnel

Throughput

Embedding throughput500 items · items per hour · higher is better

Every run completed 500/500 items with zero failures and zero lease expirations.

Configuration	Wall	items/hr	Run id
1 Mac, shard_size=1	20.16s	89,296	20260803T072921Z_941104a5
2 Macs, shard_size=1	17.11s	105,200	20260803T080944Z_6038c355
1 Mac, shard_size=32	7.12s	252,686	20260803T090720Z_bc8d40f5

The headline result is batching, not distribution

Packing 32 items per task made the same job on the same machine 2.8× faster. Almost all of the original wall time was HTTP round trips, not embedding.

Adding a second Mac produced 1.18×. One well-configured machine currently beats two badly-configured ones by a wide margin.

These two results are not directly comparable

The two-Mac run was measured before sharding landed, at shard_size=1. There is no multi-machine run at shard_size=32 yet, so this page cannot tell you what a sharded two-Mac pool does. That run is the obvious next measurement.

Work distribution

The two-machine run split work 91% / 9%.

Work split, 2-Mac runrun 20260803T080944Z_6038c355 · clean slate

Both workers started at zero completed tasks, so this is genuine per-job assignment, not leftover counters from an earlier run.

The local worker averaged 31.2 ms per task; the tunnelled worker averaged 56.7 ms — about 1.8× slower, entirely explainable by round-trip time. But that 1.8× latency difference produced a 10× difference in work received.

That gap is the scheduler, not the network. Leasing is greedy and first-come: the local worker's poll returns instantly, so it takes the next batch before the remote worker's request has finished crossing the tunnel. With 31 ms of work per task, the queue is drained before the remote machine can meaningfully participate.

What would fix it

• Larger tasks. Raising shard_size makes each lease worth more, so round-trip overhead stops dominating. This is available today.
• Larger jobs. At 500 items the whole run is over in seconds. A job of 50,000 items keeps the remote worker pulling long enough to contribute.
• A fairness policy. Round-robin assignment, or a cap on in-flight tasks per worker. Not implemented.

What has not been measured

Being explicit, since absence of a number is easy to mistake for a good one:

• No multi-machine run at shard_size=32
• No run above 500 items
• No OCR, classify, or generate benchmarks at all
• No worker-death or reassignment test — every published run had zero lease expirations, so the recovery path is exercised by unit behaviour, not by measurement
• No retrieval-quality comparison between nomic-embed-text and any cloud model
• No cloud cost baseline measured on identical inputs

Reproducing

macbatch control start
macbatch serve                      # in a second terminal
macbatch bench embed --n 500 --mode solo --shard-size 32

Artifacts land in benchmarks/runs/<run_id>/. See Methodology for what the harness measures and how.