Your Build, Their Lever: When Vendors Turn AI Loose and Send You the Bill

This week a flagship vendor switched on an AI inside customer build systems. It scans code and pushes patches on its own. Oversight came after the flip, after the risk already moved in. Offense moved first and reset the baseline; defenders are backstopping a live grid while someone else plays with the breaker.

Set the order. Over the summer, an AI shop bragged about cutting through more than a thousand fresh vulnerabilities in weeks and treated it like a leaderboard. A government challenge team showed it could surface dozens more in hours of compute. Criminal crews didn’t wait for a memo. Ukrainian responders logged Russian operators offloading reconnaissance and theft to large models in real time. One actor used an off-the-shelf model to design and sell turnkey ransomware with built-in evasion and recovery sabotage. Defenders are moving, but they’re chasing a tempo set by the people who tore out the guardrail.

Three drivers, three costs.

First, profit and leverage. The platform pitch is simple: rebrand “find and fix,” wrap it in acronyms, and claim the right to judge every release. Wire an AI scanner into the build pipeline and the vendor becomes the checkpoint on risk. Let it both scan and generate patches and it sits on top of your change control. The money matters, but leverage is the point. Once you rely on them for discovery, they decide what gets named and when. That’s a choke point in a lab coat. It’s also a backdoor into governance: the bouncer now mixes your drink.

Second, liability pushed downhill. When models can assemble exploit chains at machine speed, patch windows collapse. Enterprises won’t wait for upstream fixes; they’ll be told to auto-generate local patches for third-party code. Some of those will collide with official updates, brick devices, or corrupt data. Watch blame slide: “misconfiguration,” “unsupported environment,” “unauthorized modification.” The cost lands on hospitals delaying treatment, schools missing payroll, small utilities printing bills by hand. Vendors will call it “shared responsibility.” In practice, you carry the outage while they book the fucking growth.

Third, precedent and drift. Normalize customers altering closed products without the maker’s approval and you’ve quietly rewritten the contract. Today it’s emergency mitigation; tomorrow it’s routine. Upstreams learn to ship faster and test less because someone else will mop up the fallout. Normalize autonomous agents inside production networks and your baseline becomes opaque decisions no one can reconstruct under pressure. Once “that’s just how it’s done,” try forcing a change to be explainable before it rolls. Good luck.

Language does cleanup here. “Self-healing networks” sounds clever until you ask for an audit trail that survives discovery and a rollback plan tested under live load. “Autonomous red teaming” sounds bold until you ask who signs for it when the tool crosses a legal line. The adjectives are cover for costs pushed onto people who didn’t consent to carry them.

Yes, defenders can use acceleration. They should. But the public-interest baseline is proof before scale. Show audit logs that hold up in court, not a glossy demo. Show rollback exercised during normal traffic, not at 2 a.m. in a sandbox. Budget maintenance, not just launch confetti. Put independent oversight in the loop with a clear veto. Make deactivation simple, fast, and proven. Tie procurement to these guarantees: no logs, no rollback, no deal. Tie liability to whoever controls the decision path: if the vendor holds the gate, the vendor holds the risk. Insurers and regulators should treat opaque autonomy as an extra hazard, not a perk.

If you sit inside a build system today, assume you’ll be asked to trust an invisible editor. Don’t. Demand a kill switch you can reach without opening a ticket. Require human-readable diffs for every machine-made change. Separate detection from remediation so one product can’t grade its own homework. Stage releases, limit how many systems any change is allowed to touch at once, and rehearse rollback the way you rehearse backups. Put it in the contract that responsibility can’t be washed away with a status page and a shrug.

I’ve sat through enough breaches to know the first hours are mud, guesses, and omissions. don’t trust the coffee. The pattern holds: automation opened the door, money walked through it, and accountability waited outside. If there’s a path forward, it’s the plain one, slow the escalation, document every change, and force the people selling speed to answer for its consequences. Offense rewrote the rules. Defense does not have to accept them.

© Elias Rourke, 2025. Use it if it helps. Credit if you’ve got a conscience.