Why Structured Diffs Beat Raw Context for Production Agents
Point an agent at 120 KB of raw HTML, stack traces, or filings and it spends half its budget guessing which line matters. Hand it a typed 12 KB diff and reasoning becomes the easy part.
Million-token context windows made agent builders lazy. When a pipeline needs to triage a production incident, audit a competitor page, or compare two quarterly filings, the default instinct is to dump the raw artifact into the prompt and let the model sort it out.
That works in a five-minute demo. In production, raw markup and unpruned logs fail in the worst possible way: not with a loud HTTP 500, but with a confident, plausible number pulled from the wrong div or the wrong retry loop.
Why is a silent wrong field worse than a failed call?
A hard parser failure tells your queue to retry or page an engineer. A silent null — or a stale number scraped from a sidebar widget — sits inside an executive brief or automated workflow looking completely valid.
Visual layouts, log formats, and PDF pagination change constantly. Underlying domain state changes cleanly: an error rate spiked in one region, a price moved by 12%, or a rollout flag flipped eleven minutes before the alert.
What belongs in deterministic code vs. model inference?
Across both production outage triage (88 P0/P1 incidents) and domain workflows built alongside non-engineering operators, the highest-leverage fix was identical: move subtraction out of the LLM.
Read-only Go or TypeScript tools fetch the raw sources, align schemas, and compute exact deltas first. The model only ever sees bounded, source-keyed diffs — and any output claim missing a valid span ID is rejected before delivery.
| Input Payload | Avg Size | Root-Cause Hit | Silent Drift |
|---|---|---|---|
| Raw HTML / unpruned logs | 118 KB | 58% | 19.4% |
| Markdown-converted full dump | 64 KB | 73% | 11.2% |
| Top-K vector chunks only | 28 KB | 79% | 8.6% |
| Typed schema + pre-computed diff | 14 KB | 94% | 0.8% |
export interface RegionRolloutDiff {
region: string; // e.g. "eu-west-3"
configRevisionDelta: string; // "rev_841 -> rev_842 (-11m)"
p99LatencyDeltaMs: number; // +412
abortRateDeltaPct: number; // +6.4
evidenceSpanId: string; // required on every generated sentence
}↑ Slide payload size from deterministic typed diffs (8–24 KB) up to raw unpruned dumps (120 KB) to inspect root-cause precision and silent-drift risk.
Where pre-pruning is the wrong trade
- Open-ended exploratory discovery — when you genuinely don't know which fields matter yet — shouldn't be locked behind a rigid schema on day one.
- Every deterministic diff tool is code you own; if an upstream vendor renames fields and you lack schema assertions, your pre-processor breaks first.
- Creative synthesis (drafting copy from rough notes) benefits from full texture that aggressive diffing strips away.
Operating Rules
- Never ask an LLM to do subtraction or CSS/log parsing that 30 lines of typed code can do deterministically.
- Treat silent wrong values — not HTTP 500s — as the primary reliability threat in agent pipelines.
- Gate downstream actions behind read-only tools and mandatory source-span IDs.