What macbatch is

macbatch runs latency-tolerant AI batch work across Macs you already own.

The unit of work is a job: a list of items and a model. The control plane packs those items into shards, workers lease shards, run the model locally through Ollama, and post results back. If a worker disappears mid-shard, its lease expires and another worker picks the shard up.

for item in items:
    result = f(model, item)     # one whole model, one worker, one shard

Every worker runs a complete model. Nothing is split across machines — no tensor parallelism, no layer sharding, no low-latency requirement between nodes. That is what makes a friend's Mac on a different continent a usable worker.

When this is the right tool

Good fit	Bad fit
Re-embedding a corpus after switching models	Chat completions behind a user-facing request
OCR over a backlog of scanned pages	Anything with a per-request latency SLA
Nightly enrichment or classification passes	Training or fine-tuning
Work measured in hours, not milliseconds	Jobs needing one model larger than a single Mac's memory

If a job can wait an hour, macbatch can probably run it. If it cannot wait a second, macbatch is the wrong shape entirely.

The three processes

Control plane — one FastAPI process holding a SQLite queue. It hands out leases and records results. It never calls workers.

Worker — macbatch serve, a loop that heartbeats, leases shards, runs them, and bulk-posts completions. Runs on any Mac with an outbound internet connection.

Ollama — the actual runtime, listening on 127.0.0.1:11434 on each worker. macbatch does not implement inference; it schedules it.

Job to resultmap-reduce shape of a single job

shard_size is the main tuning knob. It trades control-plane chatter against reassignment granularity — a bigger shard means fewer HTTP round trips but more lost work when a worker dies mid-shard.

Failure is the default assumption

Consumer laptops close their lids, lose Wi-Fi, and get carried out of the house. The scheduler treats every worker as temporary:

• Delivery is at-least-once. A task can run twice; make your consumers idempotent.
• Leases expire after 600 seconds and the task returns to pending.
• A task that errors is requeued until it has been attempted three times, then marked failed.
• A job finishes as completed if nothing failed, partial if something did.

See Leases and failure for the exact state machine.

Next

• Install — npm for workers, pip for hacking on it
• Run your first job — end to end in three terminals