Test, Maintenance, Score

How many of those hours actually required a human?

Most test failures aren't bugs. A label changes, a button moves, an environment responds incorrectly, or game data has expired. The test stops for a valid reason, but it didn't find anything.

Until this breakdown is measured, the figure above remains an overall cost. You know how much you're paying. You don't know what you're paying for.

The table below will help you find out. Record your next twenty failures as they happen, without changing anything about the way you work.

• A real application bug. The test found an actual defect.
• UI: A legitimate interface change. The product has changed; the test must follow suit.
• Flaky instability or flaky test. Same code, different result.
• Outdated data or selector. The test set or selector has become outdated.

Resource from the talk given on September 15, 2026, at the Software Testing Night in Sophia Antipolis. Mr Suricate, a French provider of a no-code automated testing platform.