Why Apple Silicon punches above its weight on local LLMs

A 16-inch MacBook Pro with 64GB unified memory will run a model that a $1600 RTX 4090 cannot. If you built PCs in the discrete-GPU era, that sounds backwards. It comes down to one design choice: unified memory.

This is post 4 of 7 in the AI Running series. Last post made the case that VRAM is the hard ceiling. True for discrete GPUs. Apple Silicon bends that ceiling, and it bends it in exactly the way local LLMs care about.

The discrete-GPU model

A normal PC keeps two separate pools of memory:

• System RAM. What the CPU sees. 16, 32, 64 GB on a typical desktop.
• VRAM. What the GPU sees. Soldered to the GPU board. 12, 16, 24 GB on consumer cards.

To run inference on the GPU, the model file has to sit in VRAM. If it doesn't fit, you've got two options. Offload some layers to the CPU and crawl, or just don't run it. Your 64GB of system RAM is no help to a 24GB GPU.

That's why a 4090 tops out around a 32B Q4 model. Anything bigger can't land on the chip.

The Apple Silicon model

M-series chips put the CPU, GPU, and Neural Engine on one piece of silicon, all sharing a single pool of memory. There's no separate "VRAM" at all. The GPU can address every byte the CPU can.

Here's what that buys you. If your Mac has 64GB unified memory, the GPU can use up to about 75% of it for model weights. Apple reserves the rest for the OS. So a 64GB MacBook Pro hands the GPU roughly 48GB of usable VRAM. A 4090 gives you 24GB. The Mac doubles it for about the same money.

Bigger Macs tilt the scales harder. A Mac Studio M3 Ultra with 192GB unified memory gives the GPU around 144GB to play with. That's enough for a 405B model at Q4. Nothing in the consumer discrete-GPU market gets close.

What about raw speed?

Apple Silicon is slower than a 4090 on small models, no point pretending otherwise. A 4090 will do 100+ tokens per second on a 7B model. An M3 Max does 60 to 90. The 4090 has more memory bandwidth (1 TB/s against roughly 400 GB/s on M3 Max) and more raw compute.

But on a large model, the 4090 is doing 0 tok/s, because the model never fits in the first place. The Mac is doing 10 to 15 tok/s on a 70B Q4. Slow beats nothing.

The crossover lands around 32B parameters. Below that, the 4090 wins. Above it, a 64GB+ Mac wins.

What fits in each tier

Working numbers, mid-2026, for an M3-class Mac at Q4 quantization:

• 16 GB. 7B and 8B models. Maybe a 13B if you close everything else. Tight, but workable for chat and small tasks.
• 32 GB. Comfortable on 7B to 13B. 32B is reachable, but the OS gets cranky.
• 64 GB. The first happy tier. 32B Q8 or 70B Q4 with room left for VS Code, a browser, Slack.
• 128 GB. 70B Q8 or a low-quant 200B-class MoE. Real workstation territory.
• 192 GB (Mac Studio Ultra only). 405B at Q4. Top of the consumer ladder.

The 64GB tier is where Apple Silicon earns its name. It's the cheapest "I can actually run real models" machine you can buy in 2026.

What this looks like in practice

On my M3 Max 64GB:

# install ollama via homebrew

brew install ollama

# pull and run a 70B model

ollama run llama3.1:70b "explain TLS handshake in three sentences"

The 70B Q4 file is around 40GB. It downloads once, then maps into unified memory in a few seconds. From the second prompt on, latency is just the GPU doing matrix multiplies on weights that already live in RAM.

I get roughly 12 tokens per second on that setup. Slow next to a Claude API call. Instant next to "this model does not run on your machine."

The Metal runtime

Apple Silicon doesn't run CUDA. The GPU programming model is Metal, Apple's own API. Two things follow from that.

Anything written specifically for CUDA won't run on a Mac. PyTorch, llama.cpp, and the major runtimes all ship Metal backends now, but some niche library a researcher wrote last year might not.

The one that matters for local LLMs is llama.cpp's Metal kernels. Ollama and LM Studio both wrap llama.cpp, so pick either and you get Metal acceleration for free.

The old line was "Macs are slow on AI." That was true in 2022. By 2026 the major local-LLM runtimes are first-class on Apple Silicon, often tuned for it before any other platform.

What Apple Silicon is bad at

Three honest weaknesses:

• Training, not inference. Unified memory is great for inference because you can fit the weights. Training wants far more memory (gradients, optimizer state), and the bandwidth cost of unified memory starts to bite. Train on H100s, infer on Macs.
• Multi-GPU. A Mac is one chip. You can't tensor-parallel across two M3 Maxes the way you can across two 4090s. Your ceiling per machine is the chip you bought.
• Power-curve scaling. A Mac Studio Ultra at 192GB pulls under 400W at full load. Genuinely impressive, but a server room full of H100s doesn't care about watts. For pure throughput at scale, datacenter GPUs still win.

For one developer running models on a laptop, none of that bites. Inference is the whole job.

Should you buy one for this?

If you're already on Apple, the only question is how much RAM. The honest tiers:

• 24 to 32 GB: enough for 7B to 13B. Decent starting point.
• 64 GB: the sweet spot. A real local LLM machine without Pro Studio money.
• 96 to 128 GB: only if you're running 70B Q8 or larger every day.
• 192 GB Studio: niche. You either need 405B at home or you don't.

If you're not on Apple and you want a local LLM box, the call is harder. A used 4090 plus a good PC will be faster on small models for less money. A Mac Studio M2 Ultra 64GB gets you bigger models for similar money. Neither is wrong. They just optimize for different work.

What's next

Hardware is half the story. The other half is the runtime: the software that takes a GGUF file, loads it into memory, and turns prompts into tokens. Three of them run the local-LLM scene right now, llama.cpp, Ollama, and LM Studio. That's the next post.