Observability: what a production AI system actually logs
Closing the AI Engineering series. Tracing prompts and tool calls, tracking spend and latency, and catching failures that don't throw an error.
Blog
#ai-engineering
Posts on AI engineering, LLM systems, and software development.
Evals: how you actually know it got better
Feeling better isn't the same as measurably better. Offline eval sets, LLM-as-judge, human eval, and regression testing for prompts and RAG pipelines.
Guardrails: schema validation, filtering, and trust
A model's output isn't trusted just because it came from your own system. JSON-mode reliability, schema validation, content filtering, and basic jailbreak resistance.
Multi-agent systems: orchestration and handoffs
Adding a second agent doesn't halve your problems, it multiplies them. Supervisor patterns, handoffs, and the real cost of splitting one agent into several.
Agent memory: scratchpad vs long-term store
An agent forgets everything the moment its context window ends, unless something is built to save it. Short-term scratchpad vs a real long-term store, and what's actually worth keeping.
The agent loop: plan, act, observe
Most agent failures look the same: stuck calling the same tool forever. The plan-act-observe loop, why it gets stuck, and the fixes that actually work.
Beyond vector search: hybrid, rerank, and graph RAG
When plain vector search hits its ceiling, three upgrades actually help: hybrid search, reranking, and graph RAG. When each one earns its added complexity.
RAG done right: chunking and retrieval mechanics
Naive RAG breaks in practice long before the model does. Chunking strategy and embedding choice decide most of it, and here's where top-k search quietly fails.
Prompting as an engineering discipline
A prompt that runs a thousand times a day needs to be treated like code, not a note you typed once. Few-shot, chain-of-thought, and structured output, with the trade-offs.
From running models to building systems
Running a model and building with one are different skills. This post opens the AI Engineering series and lays out why the next nine posts exist.