RAG

Posts that use this term

• Observability: what a production AI system actually logs

  Closing the AI Engineering series. Tracing prompts and tool calls, tracking spend and latency, and catching failures that don't throw an error.

• Agent memory: scratchpad vs long-term store

  An agent forgets everything the moment its context window ends, unless something is built to save it. Short-term scratchpad vs a real long-term store, and what's actually worth keeping.

• Beyond vector search: hybrid, rerank, and graph RAG

  When plain vector search hits its ceiling, three upgrades actually help: hybrid search, reranking, and graph RAG. When each one earns its added complexity.

• RAG done right: chunking and retrieval mechanics

  Naive RAG breaks in practice long before the model does. Chunking strategy and embedding choice decide most of it, and here's where top-k search quietly fails.

• Prompting as an engineering discipline

  A prompt that runs a thousand times a day needs to be treated like code, not a note you typed once. Few-shot, chain-of-thought, and structured output, with the trade-offs.

• From running models to building systems

  Running a model and building with one are different skills. This post opens the AI Engineering series and lays out why the next nine posts exist.

• Troubleshooting local LLMs (and how to keep up after this series)

  The full catalog of local-LLM failures: OOM, slow tok/s, garbage output, instruction drift, bad RAG hits, tool-call hallucination. Plus where to follow the field once you're on your own.

• Fine-tuning a model locally

  When fine-tuning is actually the right call (it usually isn't) and how to pull off a LoRA run on a 16GB Mac, with a worked Llama 3.2 3B example.

• Local agents and tool use

  Function calling on open models in 2026. Which ones actually work, why local agents break when they break, and the scaffolding that keeps them upright.

• Local RAG and embeddings

  Build a working local RAG pipeline in about 30 lines using nomic-embed-text, Chroma, and Llama 3.2. And why running it on your own machine beats the cloud for personal notes.

• Wiring a local LLM into the tools you already use

  How to point VS Code (Continue, Cline), web chat UIs (Open WebUI, LibreChat, Page Assist), and your own code at a local model using the OpenAI-compatible API. Swap cloud for local without rewriting anything.

• Every machine can run a local LLM (here's what fits)

  A per-tier guide to running local LLMs in 2026, from 8GB integrated graphics to a 192GB Mac Studio. Specific models, specific speeds, specific configs.

• Picking a local model by task

  The 2026 open leaders, sorted by what you actually want to do: coding, chat, the small-model crowd, structured output, vision, embeddings, and audio.

• The local-LLM vocabulary

  Parameters, B, dense vs MoE, base vs instruct, tokens, context windows, chat templates, GGUF, and quant suffixes. Read it once and any HuggingFace model card stops being scary.

• The pitch for local LLMs in 2026

  The case for running an LLM on the machine you already own. Privacy, no per-call cost, faster first token, no rate limits, and it works on a flight.

• Prompt, RAG, fine-tune: three ways to shape a model

  Three levers for shaping what an LLM does: prompting (ask better), RAG (give it the right context), fine-tuning (change the weights). What each costs, what each fixes, and how to pick.

• RAG: giving a model memory it doesn't have

  RAG is the pattern of fetching relevant text from a search system and putting it in the LLM's context window before asking your question. Not magic, not fine-tuning, just better prompts.

• The context window, and why models hallucinate

  An LLM only sees a fixed-size slice of text at a time. When it doesn't know something, it predicts anyway. That's a hallucination, not a bug.