Local models on Apple Silicon are better than you think.

I keep meeting people who tried a local model once in 2023, watched it hallucinate through a simple question, and wrote the whole category off. I understand the reflex. I also think it's badly out of date. If you own an Apple Silicon Mac, you are carrying around some of the best consumer inference hardware ever shipped, and the models that fit on it crossed the usefulness line a while ago.

This is the post I wish someone had handed me before I started building OMEGA's local stack: which runtime to pick, what your RAM actually buys, where local models genuinely earn a place in daily work, and where the cloud is still clearly better. No hedging on that last part. The gap is real, and pretending otherwise is how local-AI advocates lose people.

Unified memory is the whole story

On a PC with a discrete GPU, model weights have to live in VRAM, and consumer VRAM is scarce. A 12GB card holds a 7B model at a reasonable quantization and not much more; anything bigger spills across a comparatively slow bus and performance falls off a cliff. The usual escape hatch is workstation hardware most people will never buy.

Apple Silicon has no cliff. CPU and GPU share one pool of unified memory, so a 32GB Mac has most of 32GB addressable by the GPU with no copying across a bus, and on the Pro and Max chips that pool sits on very wide memory. Once a model fits, bandwidth is what decides tokens per second. One caveat: macOS reserves a share of the pool for itself, so you never get to hand the model every byte you paid for. The practical result stands anyway. Apple built the best mainstream local-inference laptop as a side effect of designing for video editors, and it costs the same whether or not you ever use it for this.

Picking a runtime

Ollama

Ollama is the easiest on-ramp and the one I recommend to anyone who just wants to try this today. After one install and one pull command, a model is answering on a local port with an OpenAI-compatible API. It sits on top of llama.cpp, so it runs the enormous GGUF model library and behaves the same on a Mac, a Linux server, or a gaming PC. That portability is a genuine strength, and so is the community around it. The trade is that llama.cpp treats Apple Silicon as one backend among many. Its Metal support is good, but it wasn't designed around unified memory, and in my own use the difference shows up most on long prompts, where the wait for the first token gets noticeable.

LM Studio

LM Studio is the best front door if you want a GUI. Model search is built in, the download manager flags models that won't fit your RAM, there's a chat window for quick experiments, and a local server mode for pointing other tools at it. On Apple Silicon it can run models through an MLX engine as well as llama.cpp, which is a quietly excellent feature: MLX performance without touching Python. If OMEGA didn't exist and a non-programmer friend asked me for a local setup, LM Studio is what I'd install on their machine.

MLX

MLX is different in kind. It isn't an app; it's Apple's open-source array framework, built by Apple's machine learning research group and designed from the first commit around unified memory. The mlx-lm package adds model loading, generation, and quantization on top, and the mlx-community organization on Hugging Face maintains a deep catalog of pre-quantized models. Because MLX is a real framework rather than an inference wrapper, you can also fine-tune with LoRA on the same laptop that runs your inference. The cost is that MLX by itself is a toolkit: no interface, no model manager, and if you want a server you stand it up yourself. It's also Apple Silicon only, which is either disqualifying or completely irrelevant depending on what's on your desk.

OMEGA is a native Apple Silicon app, so for us the choice was easy: we build on MLX and ship the missing product layer around it. More on that below.

What your RAM actually buys

The arithmetic fits on a napkin. At 4-bit quantization, weights cost roughly half a gigabyte per billion parameters: an 8B model lands around 4.5GB, a 32B model around 18GB. On top of the weights you pay for the KV cache, which grows with every token of context you hold in play, and you pay for macOS plus everything else you have open, which is rarely under several gigabytes. With that math in hand, the size classes sort themselves out.

• 8GB: a cramped local-model machine, and I won't pretend otherwise. A 3B model at 4-bit runs and is genuinely useful for classification and short drafting, but an 8B forces macOS to start evicting everything else you're doing. Treat local as a bonus here, not a plan.
• 16GB: the practical floor, and where most MacBook Airs live. A 7B or 8B model at 4-bit runs comfortably with a browser open. This size class is where local models stopped being toys.
• 32GB: 14B-class models all day, and 30B-class models at 4-bit if you keep context modest. Somewhere around here the output quality starts surprising people who haven't looked since 2023.
• 48GB and up: 30B-class models with real context headroom, and at 64GB a 4-bit 70B becomes an option. The bigger chips also ship faster memory, so these machines gain speed along with room.

Note what's absent from that list: nothing on it matches a frontier model. That's fine. The point of local is different: stop paying frontier prices for work that never needed frontier capability.

Where local models genuinely earn their keep

After months of routing real work through both sides, here is where the local model wins on merit, not ideology.

Drafting. First passes at emails, rewrites, outlines, summaries of documents you were going to read anyway. Anything a human reviews before it ships. Intuition says the quality bar disqualifies small models here; in practice a good 8B produces a first draft you edit rather than discard, and the review step was always in the loop.

Classification and extraction. Route this message, tag these files, pull the total and date off a receipt, decide whether a turn needs a tool. Short inputs, short structured outputs. Small models are startlingly good at this, and it quietly makes up an enormous share of what an assistant actually does all day. Paying cloud rates for it is burning money.

Private data. Contracts under NDA, client files, medical records, a journal. Some data isn't yours to upload, and some you simply don't want to. A local model reads it without the data ever leaving the machine, and no privacy policy needs to be trusted, because no third party is involved.

Offline. A plane, or a hotel with Wi-Fi in name only. An assistant that goes inert without a network connection is a thin client for someone else's computer. Local weights keep working.

There's also a quieter advantage: latency. A short local turn starts streaming before a cloud request would have finished its network round trip. For quick classification and one-line answers, the local path often feels faster even when the cloud model is smarter, because on those turns intelligence was never the bottleneck.

Where the cloud still wins, clearly

Deep reasoning is not close. Ask an 8B model to refactor a module with a subtle ownership bug and it will produce plausible code that is wrong in ways it cannot see. At that scale the model doesn't know what it doesn't know. Frontier models fail too, but they fail less, and they're far better at noticing their own uncertainty. For multi-step planning, hard debugging, and novel synthesis, I route to the cloud without hesitation.

Long context is a memory problem before it's an intelligence problem. Every token you hold in play grows the KV cache, and on a 16GB machine a big document plus an 8B model plus macOS does not fit gracefully. Cloud models hold hundreds of thousands of tokens because someone else bought the hardware. If the task is "read this entire contract and answer questions about how the clauses interact," local is the wrong tool on most Macs today, and I'd rather say so than have you find out mid-task.

Reliable tool use over long chains is also still a bigger-model skill. A small model can call one tool correctly. Ten dependent calls, with recovery when the third one fails, is a different sport.

How OMEGA routes between them

OMEGA ships a bundled local server we call Omega-MLX. It runs beside the app in standby and loads whatever model the runtime asks it to. You never install Python or babysit a server process. If you've used the raw MLX toolchain, Omega-MLX is the product wrapper it was missing; if you haven't, it's invisible, which is the point.

Routing runs on four slots in ascending cost and capability: an ultra-light slot for trivial turns, a light slot for fast simple queries, a medium slot for balanced reasoning with tools, and a heavy slot for deep multi-step work. Every slot is provider-agnostic and every assignment is yours. In a hybrid setup, the ultra-light slot is typically your local MLX model, which drives the marginal cost of a large share of everyday turns to zero, while the heavy slot is whatever frontier model you've connected, through your own API keys with no token markup or through a Claude or Codex subscription you already pay for.

This split works because of how OMEGA's Neural-Fractal Agentic AI™ (NFA) engine operates: it decomposes tasks into small, scoped units, and small scoped units are exactly what local models are good at. The frontier model plans the difficult step; the local model handles the tagging and drafting around it for free.

And one rule we hold absolutely: OMEGA never pretends a 7B model is a frontier model, and it never silently swaps your model out. The model you assigned to a slot is the model that runs the turn. If your chosen local model can't emit tool calls and the turn needs one, OMEGA doesn't escalate to a bigger model behind your back; it runs the tool itself and hands the result to your model in the prompt. Your selection is the authority, full stop. In the other direction, memory is provider-portable, so you can start a conversation on the local model and escalate to a cloud model mid-chat for the hard part without losing any context. When the hard part is done, nothing stops you dropping back down. Per-workspace budgets cap what the cloud side can spend while the local side stays free.

The mental model: your Mac is a very good junior assistant, and the cloud is your expensive senior consultant. Use the junior for volume and privacy and the senior for judgment. The job of a router is to make that split automatic without ever overriding you. OMEGA is that router, plus the memory that survives the handoff. It runs natively on macOS 14 or later on Apple Silicon; details are on the pricing page.