Troubleshooting

Start with:

macbatch doctor

It checks Ollama, Homebrew, cloudflared, the config file, control-plane reachability, and the Ollama API in one pass.

bench fails with "No alive workers"

The control plane has no worker that heartbeated in the last 60 seconds.

macbatch control workers    # is anything listed, and is alive=yes?

Common causes: serve is not running, it is pointed at a different control plane, or its tunnel URL is stale. Check what the worker actually thinks it is talking to with macbatch status.

Worker logs "ollama not reachable yet"

The worker starts anyway and fails on the first shard. Verify Ollama:

curl http://127.0.0.1:11434/api/tags
ollama serve            # if the service is not running
ollama pull nomic-embed-text

macbatch setup normally handles all of this. Re-run it if the machine was set up with --skip-ollama.

Port 8000 already in use

lsof -nP -iTCP:8000 -sTCP:LISTEN
kill <PID>

Or just use another port — macbatch control start --port 8001. If the process already listening is a healthy macbatch control plane, control start says so and exits 0 rather than starting a duplicate.

Tunnel URL stopped working

Cloudflare quick tunnels are ephemeral. The hostname changes every restart, and public_url is cleared from config when the tunnel exits.

Re-point workers at the new URL:

macbatch serve --control-url https://new-url.trycloudflare.com

Note that effective_control_url() prefers the explicitly configured control_url over public_url. This is deliberate — a dead tunnel used to break workers running on the control-plane machine itself.

Tasks keep getting requeued

A shard that errors goes back to pending and is retried until three attempts, then becomes failed. Find out why:

curl <url>/v1/jobs/<job_id>/results | python3 -m json.tool

Each entry carries an error string from the worker that tried it. The usual causes are a missing model on that worker (ollama pull it), an item with an empty text field, or Ollama timing out on an oversized item.

Job stuck with leased tasks and no progress

A worker took shards and vanished. Leases expire after 600 seconds and reclaim automatically on the next request. To force it:

curl -X POST <url>/v1/admin/reclaim

Count how often this has happened for a job — lease_expired_events in GET /v1/jobs/<job_id> is a direct measure of worker flakiness.

Two Macs, but one does almost all the work

Expected with the current scheduler. Leasing is greedy and first-come — a fast local worker drains the queue before a remote one can pull. The measured split on a two-machine run was 91% / 9%.

Mitigations that help today: raise --shard-size so each task is worth more, and use a larger --n. A fairness policy in the scheduler is the real fix and is not implemented.

npm i -g macbatch fails in postinstall

The installer needs Python ≥ 3.11 on PATH. Point it at one explicitly:

MACBATCH_PYTHON=/opt/homebrew/bin/python3.12 npm i -g macbatch

Reading the logs

Everything is JSON lines:

tail -f ~/.macbatch/logs/macbatch.jsonl
macbatch -v serve      # DEBUG level