Loss

Posts that use this term

• Multi-agent systems: orchestration and handoffs

  Adding a second agent doesn't halve your problems, it multiplies them. Supervisor patterns, handoffs, and the real cost of splitting one agent into several.

• Fine-tuning a model locally

  When fine-tuning is actually the right call (it usually isn't) and how to pull off a LoRA run on a 16GB Mac, with a worked Llama 3.2 3B example.

• Quantization, distillation, pruning: how a 140GB model fits on your laptop

  Three ways to shrink an LLM, and why one of them does almost all the work. What Q4_K_M actually means and what each shortcut costs you.

• The local-LLM vocabulary

  Parameters, B, dense vs MoE, base vs instruct, tokens, context windows, chat templates, GGUF, and quant suffixes. Read it once and any HuggingFace model card stops being scary.

• What it takes to run a model on your own machine

  Why VRAM is the one number that decides whether a local LLM runs, what quantization really does to a model file, and the hardware ladder from an 8GB laptop to a 192GB workstation.

• How a model learns: training and inference

  Training is the expensive one-time event where a model's numbers get tuned. Inference is the cheap repeated use afterwards. The gap in cost is enormous, and it shapes the whole industry.

• Install llama.cpp

  Build llama.cpp from source with Metal or CUDA, then run a GGUF model with llama-cli. The closest thing to bare-metal local inference.