# Building Checks That Can't Fail: Why You Need To Direct Your AI Agents To Test, Debug And Fail Loudly *By James Hills — Updated November 2026* Written by: [James Hills](https://flowmediamarketing.com/james-hills.html) Published: 09 November 2026 Last Updated: 09 November 2026 Hits: 20Reading time: 04:36 An AI agent that says "tested and passing" has told you almost nothing until you know whether its test could ever have failed. After a year of building publishing software with AI coding agents, my team has a core rule: each check must catch a broken build, processes must fail loudly when something goes wrong, and synthetic users must walk the app before I ever see it. Here is why that discipline matters and how we apply it. **Don't Blame AI - This Isn't****Article Index** The testing discipline behind every tool we build with AI agents starts with the core understanding that this is human-directed agentic programming. If you don't stand on the shoulders of decades of good software development and project management discipline then you are destined to fail. - AI-built software usually works on the first try and breaks later, in the places nobody looked. - A check that can't fail isn't evidence, so prove each one catches a known-broken version before you trust it. - Processes should stop and report when their input is wrong, instead of finishing the job with corrupted data. - Synthetic users in Playwright and Claude in Chrome find broken buttons, typos, accessibility gaps and wasted steps before a human reviews anything. - This discipline is the difference between vibe coding and software you can run a business on. 1. [Why Vibe-Coded Tools Often Work on Day One and Break Once You Use Them In Production](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html#why-vibe-coded-tools-often-work-on-day-one-and-break-once-you-use-them-in-production) 2. [A Check That Can't Fail Isn't Evidence](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html#a-check-that-cant-fail-isnt-evidence) 3. [Make It Fail Loudly When the Input Is Wrong](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html#make-it-fail-loudly-when-the-input-is-wrong) 4. [My Synthetic User Testing Finds What Code Reviews Can't See](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html#my-synthetic-user-testing-finds-what-code-reviews-cant-see) 5. [How To Build This Discipline Into Your Own AI Projects](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html#how-to-build-this-discipline-into-your-own-ai-projects) 6. [Human Intuition Is Still Critical To The Testing Process - We're Illogical and Like To Break Stuff!](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html#human-intuition-is-still-critical-to-the-testing-process-were-illogical-and-like-to-break-stuff)   ## Why Vibe-Coded Tools Often Work on Day One and Break Once You Use Them In Production When I started, my projects were closer to what people think of as vibe coding: "Hey ChatGPT, create me a plugin that builds an FAQ box at the bottom of every article, pulling data from the post it appears on." It would do that, and do it nicely, and it would almost always have a problem we didn't discover until we started using it more. That sucked. As the tools got more complicated, the failures got harder to see. When software sends thank-you emails to contributors, I'm not going to know whether that worked. I'd just assume it did since the debugging round said everything was ready to ship. We hit the same thing in FlowAds, our ad and affiliate placement tool, when links weren't rendering the affiliate code correctly, and in a dozen other situations. Each one looked fine from the outside. ### Overcoming The Challenge Of Transforming Your Blog From Writing About Others To Writing About You Why publishers who feature their own experts and community outlast blogs that chase search terms, plus how to find experts, run intake interviews and build relationships. FlowMediaMarketing [Read More](https://ads.flowmediamarketing.com/go/shm5OamrlR8TGhcg6xx_kw.2488) ### Great Lakes Cruising Is Booming - And the Industry Experts Who Dismissed It Were Wrong Great Lakes cruising doubled since 2015 with $300M projected for 2026. First-hand Viking Octantis experience plus analysis on why skeptics who dismissed Milwaukee got it completely wrong. FlowMediaMarketing [Read More](https://ads.flowmediamarketing.com/go/shm5OamrlR8TGhcg6xx_kw.652) ### What Do Gen Z Travelers Think About Cruising And Where Do They Want To Go Next? Gen Z cruisers seek affordable, sustainable voyages with digital connectivity, favoring Caribbean and European destinations while prioritizing unique experiences. FlowMediaMarketing [Read More](https://ads.flowmediamarketing.com/go/shm5OamrlR8TGhcg6xx_kw.654) Promoted by FlowAds This is one reason most vibe-coded software sucks, and why I resist calling the work I do vibe coding. It's also why software engineers and project managers still have jobs and COOs need to spend time cultivating a culture of excellence that focuses on maintaining the right processes, not just outsourcing everything to teams of agents while laying off expensive senior staff. ## A Check That Can't Fail Isn't Evidence The core rule is simple to say and easy to skip. Before a test, probe or deploy check counts as evidence, it has to be shown to fail when something is wrong. The cleanest way is to run it against a known-broken version, such as an older release that had the bug. If the check passes the broken version too, the check is the defect. A test that only confirms a page loaded, a script that hides its own errors or a comparison against the wrong baseline will all report success every time. Now we require the design to tell us when something fails, and to be robust enough to anticipate the failure in the first place. ## Make It Fail Loudly When the Input Is Wrong The same thinking applies inside the tools. Sometimes, when I'm testing and something doesn't feel right, the fix is as easy as adding debug output so we can see what's happening. Other times it means requiring that a process doesn't run at all if it didn't read its source document properly. A tool that stops and says "I couldn't read the source, nothing was written" costs you a minute. A tool that pushes ahead with a corrupted payload can publish a broken article, email the wrong people or quietly drop affiliate codes for weeks. Most of this design language lives in the standing instructions every agent on my team works from. I still remind the agents during development, and again while debugging. ## My Synthetic User Testing Finds What Code Reviews Can't See The newest layer in my testing process is using synthetic users. We developed Playwright scripts and Claude in Chrome prompts to walk through an entire app the way a person would, trying to accomplish real goals like building a meme or creating a roundup. Then the agent reports back with findings. Sometimes the findings are basic: a broken button, a language file issue, a typo or colors that fail the WCAG contrast guidelines (AA calls for a 4.5:1 ratio for normal text) that keep pages readable for people with low vision. Other times they are fundamental. On one walk, the agent found that a task took 24 steps and came back with a plan to cut it to 14. That's a MASSIVE improvement. These walks have caught real bugs that automated tests and code review missed. One audit of our promotion tools found confirmation dialogs that didn't work on the Dispatch, Post Now and Delete buttons, and an Instagram dispatch that went live on a single click. We fixed the broken dialogs in a follow-up release. ## How To Build This Discipline Into Your Own AI Projects You don't need a custom framework to start. Add three rules to the instructions you give your coding agent: - **Prove every test can fail** - show it catching a deliberately broken version before it counts. - **Stop on bad input** - if the process can't read what it needs, it halts and reports instead of guessing. - **Walk it like a user** - before you review a feature, have an agent complete a real task in a browser and report every step, error and slowdown. Then keep the human at the end of the line, reviewing the agent's report before anything ships. ## Human Intuition Is Still Critical To The Testing Process - We're Illogical and Like To Break Stuff! The checks catch most of what's broken, but the best signal is still a person using the software and sensing that something doesn't feel right. When you get that feeling, don't accept "tested and passing." Ask the agent to show you the test failing on purpose. If it can't, you've found your next bug. --- ***Thanks for reading. We hope this was helpful!*** --- Written by: [James Hills](https://flowmediamarketing.com/james-hills.html) Pro-Blogger James Hills is the founder of FlowMediaMarketing.com and runs a network of travel and lifestyle publications including ManTripping.com and CruiseWestCoast.com on a stack of publishing and site-management tools he built with AI coding agents. For media inquiries on AI tooling and process design for small businesses, contact James at FlowMediaMarketing.com. ### Related Articles #### Not Just "Intelligence" - Why Building Tools To Unlock The Power Of AI Is Essential To Success [Not Just "Intelligence" - Why Building Tools To Unlock The Power Of AI Is Essential To Success](https://flowmediamarketing.com/blog/building-tools-to-unlock-the-power-of-ai.html) #### FlowGate: The Release Process That Turns AI-Built Apps Into Real Products [FlowGate: The Release Process That Turns AI-Built Apps Into Real Products](https://flowmediamarketing.com/blog/flowgate-release-process-for-ai-built-apps.html) #### I Turned My NAS Into a Development and Testing Server, Here's What I Learned [I Turned My NAS Into a Development and Testing Server, Here's What I Learned](https://flowmediamarketing.com/blog/nas-development-testing-server-lessons-learned.html) ### Latest News Updates: #### How I'm Transforming My Blogging Strategy To Develop, Currate and Publish Original High-Quality Content [How I'm Transforming My Blogging Strategy To Develop, Currate and Publish Original High-Quality Content](https://flowmediamarketing.com/blog/gathering-original-high-quality-content.html) #### Not Just "Intelligence" - Why Building Tools To Unlock The Power Of AI Is Essential To Success [Not Just "Intelligence" - Why Building Tools To Unlock The Power Of AI Is Essential To Success](https://flowmediamarketing.com/blog/building-tools-to-unlock-the-power-of-ai.html) #### FlowGate: The Release Process That Turns AI-Built Apps Into Real Products [FlowGate: The Release Process That Turns AI-Built Apps Into Real Products](https://flowmediamarketing.com/blog/flowgate-release-process-for-ai-built-apps.html) #### Overcoming The Challenge Of Transforming Your Blog From Writing About Others To Writing About You [Overcoming The Challenge Of Transforming Your Blog From Writing About Others To Writing About You](https://flowmediamarketing.com/blog/writing-about-others-to-writing-about-you.html) - - Flow Media Marketing is based in Southwest Michigan. Learn More About Our People and Brands: - [James Hills](https://flowmediamarketing.com/james-hills.html) - [Heather Hills](https://flowmediamarketing.com/heather-hills.html) - [Flow Media Marketing](https://flowmediamarketing.com/flow-media-marketing.html) - [Flow Voyages](https://flowmediamarketing.com/flow-voyages.html) - [Cruise West Coast](https://flowmediamarketing.com/cruise-west-coast.html) - [Couple Places](https://flowmediamarketing.com/couple-places.html) - [MenWhoBlog](https://flowmediamarketing.com/menwhoblog.html) - [ManTripping](https://flowmediamarketing.com/mantripping.html) - [The Grape Stomp](https://flowmediamarketing.com/the-grape-stomp.html) --- *Source: [Building Checks That Can't Fail: Why You Need To Direct Your AI Agents To Test, Debug And Fail Loudly](https://flowmediamarketing.com/blog/checks-that-cant-fail-ai-agent-testing.html)* **Tags:** AI Content, Agentic Coding