Sleeping well after shipping code you didn't read
Test copy of the proposal on semantic-mediawiki.org, used to test Template:AgendaDay26. Time slot and status are test values from the draft program.
Sleeping well after shipping code you didn't read (Discussion, 45 min, In-person, Accepted)
Speakers: Lex · Start: 2026/11/13 11:00:00 · Conference: MediaWiki Users and Developers Conference Fall 2026
AI coding agents make application development cheap and fast, but introduce new risks. This session examines the rules we rely on when shipping agent-written code and how consistently we actually follow them. This will be a discussion led by User:Lex with User:Cicalese.
AI coding agents have made it cheap to turn an idea into a working application. But agents work probabilistically, not deterministically, so the new freedom comes with new risk: unknown unknowns, potentially born of both incompetence and complacency.
Most of us have shipped code we didn't review line-by-line. We all have rules that are supposed to make that safe. This session shall be both about what those rules are and how meticulously we actually follow them.
I'd like to suggest discussing questions like these:
(How) can we ensure the outcome on the right while using the practice on the left to full capacity?
- What do you still read yourself: the spec, the interfaces, the types, the schema changes, the tests? Do you write any of them by hand?
- Do you keep no-vibe zones — auth, payments, permissions, crypto, migrations, anything with DELETE or concurrency — and do you actually respect them?
- What does your CI enforce without exception: type checks, linters, security scans, dependency audits, coverage thresholds?
- Do you have your agent build in observability — alerts on user-facing symptoms, structured logs — and hold it to an error budget?
- How do you decide which PRs get a real review, and how small do you keep them?
- How do you fight bloat: error-swallowing try/catch blocks, reflexive null guards, stubbed methods? Do you schedule refactors?
- Please add more!
To prepare some exemplary ground, please check out these two thoughts of mine:
