I deliver internal systems for companies, and I run the infrastructure they sit on. This is where the parts worth writing down end up — usually the ones that cost two hours to work out and take three sentences to explain.
An identity gate only executes when the identity is wrong, and nobody walks that path. Running it wrong on purpose surfaced two problems in two minutes: comparing the account id lets root through (root shares your account), and the error message itself crashed under set -u. The two acceptance criteria for a negative test, and what else only ever runs on failure.
The directory key trap from the first entry traces back to SPA error handling applied to a site with no client-side routing. Three costs of calling a 404 a 200 (soft 404s, blind monitoring, deceiving yourself), what the error response should actually say, and why a private S3 bucket answers 403 rather than 404.
Multi-page bundlers want every HTML entry point enumerated. That list is hand-maintained and missing an entry raises no error — the build succeeds one page short, while the dev server still shows it, so only production output reveals the gap. Scanning the filesystem instead, three implementation details, and why drafts belong behind a directory convention rather than an exclusion list.
set -e assumes stopping beats continuing, and a deploy script is where that fails — half-deployed is broken and silent. It caught me once: sync finished, the session expired midway through rewriting directory keys, and both the invalidation and the whole-site verification were skipped. Where to draw the line at the first write to production, and why -u and pipefail stay.
One max-age for everything ends two ways: too long and changes never show, too short and the CDN buys nothing. The test — does the filename change with the content — splits a static site into exactly three groups. What immutable adds beyond a long max-age, why max-age=0 is not "do not cache", and why invalidation clears the CDN but never the browser.
Lighthouse measured one Google Fonts stylesheet costing close to 800ms of first paint. Where that time goes (two origins, two rounds of DNS and TLS, with a sequential dependency between them), how the preload-then-swap-rel trick works and what it costs, and why the tens of thousands of characters in a Chinese face make webfonts a bad trade from the start.
The script I gave three sentences in the first entry, written out properly. Both common static-site failures — a missing directory key and a deploy that never landed — return 200, so only content comparison catches them. Why <title> makes the right fingerprint, why set -e is deliberately absent, and why it also belongs on a daily schedule.
Pointing the English page's canonical at the Chinese one tells search engines the English page need not be indexed — and it duly disappears. On self-referencing canonicals, reciprocal hreflang sets, changing <html lang> too, why feeds come in pairs, and scripting a check across the seven places one article touches.
An access line is emitted by middleware after the handler returns, so all it sees is the response — the error detail died at the return. Pairing by timestamp works at low traffic and collapses under load. On threading a request id through everything and handing it back to the client, how far to structure logs, and the three categories that count as leaked the moment they are written.
Container creation and publishing are separate calls with a wait between them, and reply_to_id takes a published post id — three constraints that make a ten-post thread strictly serial. The real problem is that the chain has no transaction: when the sixth fails, the first five are already public. How to make it resumable.
Nothing found, no error, so you conclude it does not exist — when in fact an export from half an hour ago is still live and you queried a different account. Credential precedence, the aws sts get-caller-identity gate, and why a read against the wrong account is harder to catch than a write.
Putting several clients' backends on one machine is a sound call for a small team; the risk is that everyone then assumes they cannot affect each other. Vhosts, containers and log groups are isolated. Disk, the Docker daemon, nginx reload, the kernel and the egress IP are not — those five are your blast radius. With the disk-incident commands and four reasons to split a project out.
You fixed it, you pushed it, the problem persists — and "the fix is wrong" looks exactly like "the fix is not running". Baking the git sha in with -ldflags -X, why the frontend needs this even more, and how to turn a deploy script's last step from hope into verification.
Delegate an implementation and the completion report describes what the agent meant to do — producing the report and executing the work are separate paths. Five ways report and diff diverge, a sign-off procedure (read the diff yourself, rerun tests with -count=1, inspect the test diff for false greens), and why review cost is fixed the moment you write the brief.
The POST your payment provider sends is public, unauthenticated, forgeable and repeated — and what it changes is a money field. Working through ECPay's AIO: recomputing CheckMacValue, the two callbacks async payments produce, idempotency on MerchantTradeNo, and why the wrong response body makes the provider keep calling.
Six hops in the chain and the phone always says the same thing. Three unrelated failures: a client library's if (secure) undoing your WebSocket config, EMQX reading an empty 200 as a denial, and Docker's default bridge not resolving service names. What they share is that the broken hop reports itself healthy.
The log says column "invoice_no" does not exist. The code is right and the migration is in the repo — prod just never ran it. Detecting drift left by hand-edited production databases, why prod is the baseline when cleaning up, and why the fix is permissions rather than discipline.
The problem with a .env file is not that it is insecure — it is that it gets copied, into backups, build contexts and downloads folders. A workflow that keeps credentials off disk, why zone ids should not be stored either, and when to skip the judgement call and just rotate.
The browser blames MIME types; the real cause is a deploy script deleting the chunk a user was about to fetch. Load the page after the deploy and all is well — have it open during the deploy and you get a white screen. Plus why the fix is dropping --delete.
The backend returned a 500 with a real message; the client received {"status":"ok","message":"Request processed"}. Right status, right JSON shape, wrong content — courtesy of proxy_intercept_errors and one error_page line. Plus how to peel the layers to find the rewrite.
An eight-week incident study of a production LLM agent runtime. What 4,286 unit tests and 827 governance checks did not catch was the model rewriting a tool error into a fluent, plausible completion report.
A safety cap on a paginated fetcher never errored and never warned. It quietly grew a trend that did not exist, pointing the opposite way from reality. Plus why cat: matches more than you think.
With a REST origin, CloudFront will not resolve /blog/ to blog/index.html — and an SPA fallback turns that 404 into a 200 serving the homepage. Plus why sync --delete keeps undoing the fix.