The pitch for local LLMs in 2026
The case for running an LLM on the machine you already own. Privacy, no per-call cost, faster first token, no rate limits, and it works on a flight.
Blog
Local Llms
Posts on AI engineering, LLM systems, and software development.
The local-LLM vocabulary
Parameters, B, dense vs MoE, base vs instruct, tokens, context windows, chat templates, GGUF, and quant suffixes. Read it once and any HuggingFace model card stops being scary.
Quantization, distillation, pruning: how a 140GB model fits on your laptop
Three ways to shrink an LLM, and why one of them does almost all the work. What Q4_K_M actually means and what each shortcut costs you.
Streaming, throughput, and the KV cache
Why TTFT and tok/s are different numbers, why streaming feels faster than it is, and the KV cache that makes the 1000th token cost about the same as the first.
Picking a local model by task
The 2026 open leaders, sorted by what you actually want to do: coding, chat, the small-model crowd, structured output, vision, embeddings, and audio.
System requirements by OS for local LLMs
What macOS, Linux, and Windows each need before you run a local LLM in 2026. Mac is the smoothest, Linux gives you the most knobs, and native Windows finally just works.
Every machine can run a local LLM (here's what fits)
A per-tier guide to running local LLMs in 2026, from 8GB integrated graphics to a 192GB Mac Studio. Specific models, specific speeds, specific configs.
Your first local LLM, start to finish
Install Ollama, pull Llama 3.2 3B, chat with it, hit its API, and fix the five things that break on a first install. You finish with a working local LLM.
Wiring a local LLM into the tools you already use
How to point VS Code (Continue, Cline), web chat UIs (Open WebUI, LibreChat, Page Assist), and your own code at a local model using the OpenAI-compatible API. Swap cloud for local without rewriting anything.
Local Llms#10
Local RAG and embeddings
Build a working local RAG pipeline in about 30 lines using nomic-embed-text, Chroma, and Llama 3.2. And why running it on your own machine beats the cloud for personal notes.
Local Llms#11
Local agents and tool use
Function calling on open models in 2026. Which ones actually work, why local agents break when they break, and the scaffolding that keeps them upright.
Local Llms#12
Fine-tuning a model locally
When fine-tuning is actually the right call (it usually isn't) and how to pull off a LoRA run on a 16GB Mac, with a worked Llama 3.2 3B example.