Runtimes and models

macbatch schedules work. It does not implement inference.

Ollama is the runtime

Every worker talks to a local Ollama instance at http://127.0.0.1:11434. The worker knows exactly two calls:

Call	Used by	Shape
POST /api/embeddings	embed	{model, prompt} → {embedding: [...]}
POST /api/generate	ocr, classify, generate	{model, prompt, stream: false, images?} → {response}

That is the entire runtime surface. Supporting a different engine means implementing those two shapes; nothing above the worker changes.

Ollama was chosen for install friction, not performance. brew install ollama and a model pull is something a non-technical friend can complete, and macbatch setup automates even that.

Models

The default is nomic-embed-text — 768 dimensions, small, fast, good enough for retrieval on ordinary prose.

Any model in the Ollama registry works, as long as every worker that will accept the job has already pulled it. There is no model distribution mechanism. A task routed to a worker missing the model errors, requeues, and burns attempts.

macbatch setup --model nomic-embed-text --jobs embed --control-url <url>
ollama pull mxbai-embed-large     # add more by hand

Dimensions are not interchangeable

nomic-embed-text produces 768-dimensional vectors. OpenAI's text-embedding-3-small produces 1536 and 3-large produces 3072. You cannot mix vectors from different models in one index, and retrieval quality differs. Evaluate recall on your own corpus before switching a production index — the cost model prices compute, not quality.

Job types

Type	Status	Notes
embed	Working, benchmarked	The only type with published numbers
ocr	Implemented, unbenchmarked	Sends base64 images to a vision model via /api/generate
classify	Implemented, unbenchmarked	Shard-aware; falls back to single item
generate	Implemented, unbenchmarked	Same path as classify

Only embed has measured results on this site. The others exercise code paths that work but have not been run at scale — treat them as unproven.

OCR

The worker accepts either image_b64 inline or an image_path local to that worker. Since workers do not share a filesystem, image_path is only useful for single-machine runs; distributed OCR should send base64 or, better, gain a URL-fetch path.

Default prompt when none is supplied:

Extract all text from this image. Preserve reading order.

What is not supported

• No model splitting. Every worker runs a whole model. A model that does not fit in one Mac's memory cannot run at all.
• No tensor or pipeline parallelism. Deliberate — round-trip latency between consumer machines over the internet makes it pointless.
• No fine-tuning or training.
• No arbitrary code. The job-type allowlist is the security boundary between a control plane and the machines volunteering for it.

MLX

Apple's MLX would be the natural way to get more out of Apple Silicon, and the worker interface is deliberately narrow enough to make it a drop-in. It is not implemented. Ollama's one-command install currently outweighs the throughput left on the table.