Stop Mocking Your Own Boundaries: Hermetic Tests and Personal APIs
Function mocks only prove your code calls your mocks; shared staging clusters make you wait 25 minutes to learn a queue flaked. Keeping real handlers intact behind deterministic state boundaries solves both — at work and on this site.
A 25-minute pre-commit suite quietly ruins how engineers write software. Instead of verifying one small invariant at a time, people batch four unrelated changes into one PR, cross their fingers against shared staging queues, and hit 'Re-run failed jobs' until green.
When we audited where those 25 minutes actually went across a multi-service Go/C++ backend, core business assertions took under 50 seconds. Everything else was pod spin-up, image pushes, cross-run state collisions, and real wall-clock `sleep()` retries.
Why not just mock internal functions instead?
Mocking internal methods is fast, and mostly useless for distributed state machines. It skips wire serialization, auth interceptors, deadline propagation, and idempotency keys — the exact seams where production bugs hide.
The sweet spot is in-process hermeticism: boot the real RPC servers and real handlers inside one OS process over in-memory buffers, backed by a stateful relational double and a controllable monotonic clock that advances a two-hour lease expiry in two milliseconds.
How does the same boundary rule power this website's live `NOW` bay?
Look at the `Listening` and `Health` rows on `gulati.sh`. Both are published globally while bound strictly to my own Spotify and Google accounts: Google Fit's legacy REST API sunset in favor of `health.googleapis.com/v4` (`dailyRollUp` with restricted OAuth scopes), and public traffic spikes will quickly `429` a personal Spotify developer app if left uncached.
Both `/api/now-playing` and `/api/health` follow the exact same boundary rule: server-side owner OAuth refresh tokens behind CDN/memory rate-shields (`30s` for Spotify, `10m` for Google Health `v4`), zero account PII in responses, and a typed `HealthSnapshot` / `EnrichedTrackMeta` fallback so an expired token never breaks the UI.
| Approach | Wall Clock | Flake Rate | Catches Wire/State Bugs? |
|---|---|---|---|
| Unit function mocks | 12s | 0.0% | No — skips RPC & tx seams |
| Shared remote staging | 25m 00s | 14.2% | Yes, buried under noise |
| Docker-compose per PR | 08m 40s | 3.8% | Yes, slow real-clock sleeps |
| In-process hermetic + fake clock | 01m 30s | 0.0% | Yes — full handler fidelity |
// 1. Try cloud Google Health v4 dailyRollUp (OAuth2 refresh token)
// 2. Fall back to persisted POST /api/health webhook snapshot (.next/cache)
// 3. Guarantee valid HealthSnapshot shape in <2ms even when unconfigured
export async function GET(): Promise<NextResponse<HealthSnapshot>> {
const live = await fetchGoogleHealthLive();
if (live) return NextResponse.json(live);
const stored = await readStoredSnapshot();
return NextResponse.json(stored ?? defaultHealthSnapshot);
}↑ Compare function mocks, shared staging (25m), and in-process hermetic suites (90s) across wall-clock stages and flake rate.
Where in-memory doubles stop matching reality
- In-memory SQL/Spanner doubles won't catch query-planner regressions, lock-contention cliffs, or missing production indexes — run nightly load suites against real engines for that.
- Google Health API `v4` scopes are restricted; on a 1-user personal site in GCP 'Testing' mode, OAuth refresh tokens expire every 7 days unless paired with a persistent webhook cache.
- Kernel-level TCP stalls, MTU blackholes, and regional DNS hiccups never appear over memory buffers.
Operating Rules
- Keep real service binaries intact; swap only the transport buffer, storage double, and monotonic clock.
- Design personal APIs (`/api/health`, `/api/now-playing`) with tiered fallbacks so expired tokens never break UI invariants.
- Reserve shared staging environments for load and planner profiling, not pre-commit logic checks.