A/B testing
A comparison of two versions of a page or user flow across two user samples to determine which one converts better. This is not a software quality test: it measures behavior, not defects.
71 definitions to ensure everyone—product, QA, and engineering—is on the same page. From test cases to load testing, including false positives and QA debt.
A comparison of two versions of a page or user flow across two user samples to determine which one converts better. This is not a software quality test: it measures behavior, not defects.
The ability of an online service to be used by everyone, including people with disabilities. It is assessed against standards such as the RGAA in France or the WCAG internationally. Learn more
The difference between expected behavior and observed behavior. A broader term than “bug”: an anomaly can stem from the code, the data, the environment, or the specification itself.
Direct verification of data exchanges between systems, without going through the interface. Faster and more stable than an interface test, it detects regressions at the service contract level. Learn more
A condition checked at a specific point in a scenario. If it is false, the execution fails. A scenario without an assertion does not test anything—it simply runs through the steps.
Running test scenarios using a program rather than a person. This does not eliminate manual testing; rather, it frees up human time for exploration and judgment. Learn more
An ordered list of test cases to write or revise, prioritized by business risk rather than ease of automation.
A software defect that causes unexpected behavior. The cost of fixing it increases depending on when it is discovered: a few minutes during development, several days in production.
A compiled and linked version of an application at a given point in time. This is the object that is tested before deployment to production.
A set of scenarios run together in a given environment on a given date. It generates a single execution report.
Description of a specific test: initial state, actions, expected result. A scenario consists of several test cases.
Continuous integration and continuous delivery. An automated pipeline that builds, tests, and deploys an application every time the code is changed. Learn more
The portion of the product actually verified by testing. It is measured by business workflow rather than as a percentage of lines of code: covering 80% of the code without covering the order processing flow provides no protection. Learn more
Criteria that must be met for a feature to be considered complete. These criteria serve as the basis for test scenarios.
Test the same page on multiple browsers and versions. Just because it displays correctly in Chrome doesn't mean it will in Safari.
A data layer that the site exposes to tracking tools such as Google Tag Manager. A broken data layer doesn't break the site—it silently skews your statistics. Learn more
A list of criteria a team sets for itself to declare a task complete. When these criteria include testing, test debt stops accumulating.
A buildup of unstable, obsolete, disabled, or never-written tests. Like technical debt, this comes at the cost of maintenance time and undetected regression issues. Learn more
A tree-based representation of a web page, as rendered by the browser. UI tests rely on this to identify the elements to be manipulated.
A test that walks through the entire workflow, from the interface to the back-end systems, just as a real user would. Learn more
An environment dedicated to running tests: acceptance testing, pre-production, and production. A test scenario that runs successfully in acceptance testing may fail in other environments due to a simple difference in data.
Launching a scenario at a specific time, in a specific environment. It generates a status report, duration, screenshots, and logs.
A test conducted without a predefined script, in which a person actively searches for vulnerabilities based on their experience. It is the natural complement to automation, not its competitor.
A test that passes even though the defect exists. This is the more dangerous of the two: it gives rise to unwarranted confidence.
A test that fails even though the product works. When false positives occur repeatedly, they erode confidence in subsequent tests, to the point where no one reads the reports anymore.
Test instability, where a test passes or fails randomly even though the code has not changed. This is often caused by poorly managed expectations, shared data, or a slow environment.
Verifying that a feature produces the expected result from the user's perspective. This is the type of testing most closely aligned with business needs. Learn more
Syntax in structured natural language, of the "Given, When, Then" type, which describes a scenario in a way that is readable by both business teams and developers.
A visual comparison of two versions of a page, section by section. They detect what no functional test can: a missing legal notice, a button that has become invisible. Learn more
Digital eco-design process. Tested automatically, it checks page size, the number of requests, and the efficiency of loaded resources. Learn more
Running a browser without a graphical interface. Faster and more resource-efficient, this mode is suitable for large-scale runs in continuous integration.
A fix delivered urgently directly to production. This is when a rapid regression test suite is most valuable.
Service interruption or degradation in a production environment. An incident detected by an automated test costs less than one detected by a customer. Learn more
Verification that multiple components work together properly, whereas unit tests verify them individually. Learn more
A practice that involves merging and reviewing code changes several times a day, with tests running automatically each time. Learn more
Data prepared for running a scenario: accounts, products, orders. A poorly isolated test suite is one of the leading causes of unstable tests. Learn more
An expression that identifies an element on the page, using CSS or XPath. A locator that relies too heavily on the structure will break at the first redesign.
Logs generated during a run. They help explain why a scenario failed when a screenshot isn't enough.
Work to adapt scenarios to product changes. This is the cost item that kills most automation projects, and the first one to be scaled up for industrial production. Learn more
Test performed by a person. Indispensable for exploration, usability, and judgment. Costly when the same verification must be repeated with every release.
A simulated component that replaces a real system during a test, to overcome limitations related to its availability or cost. While useful, a mock never proves that the actual integration works.
Regularly run test scenarios on the production environment to detect a failure before users do. Learn more
Average time to detect and average time to resolve an incident. Two performance metrics that provide more insight than the raw number of bugs. Learn more
An approach that allows you to build scenarios without writing code, using blocks and actions. It makes automation accessible to people in various business roles and reduces reliance on a single person.
Verifying that a change hasn't broken anything that was working before. This is the primary use of automation—and the most cost-effective one. Learn more
The ability to understand a system's internal state based on what it exposes: logs, metrics, and traces. It complements testing; it does not replace it.
User paths whose unavailability has a direct impact on revenue or compliance. A testing strategy begins with these paths.
Measuring response times and resource usage under a given load. A functional but slow website is effectively down for some users. Learn more
A series of automated steps triggered whenever the code is modified: compilation, testing, deployment. Learn more
A model that recommends a lot of unit tests, fewer integration tests, and few end-to-end tests. Useful as a guideline, but should be adjusted when the risk is concentrated in the workflows.
Quality Assurance. The set of practices aimed at ensuring the quality of a software product, from requirements definition through production. QA is not limited to testing.
Outsourcing all or part of the testing function to a service provider, with coverage objectives and contractual oversight. Learn more
Campaign summary: past scenarios, failed attempts, durations, captures, logs. This is the feature that lets you analyze a failure rather than simply endure it. Learn more
The validation phase for a release prior to its deployment into production. It may be technical, functional, or user-led.
A bug has appeared in a feature that previously worked. This is almost always a side effect of a change made elsewhere. Learn more
European General Data Protection Regulation. The test checks for consent, cookie banners, retention periods, and the trackers that are actually set. Learn more
A sequence of actions and checks that replicates a complete user journey. This is the basic unit of automation.
Verification of an application's security measures: authentication, authorizations, injection, and data exposure. This is complementary to a security audit, but not equivalent to it. Learn more
An expression that identifies an element on the page based on its structure or attributes. The choice of selector determines the robustness of the scenario.
The ability of a tool to automatically correct a scenario in which an element has changed its position or identifier, without human intervention. Learn more
Automated monitoring of factors that affect search engine optimization: tags, HTTP statuses, canonical tags, structured data, and load times. Learn more
Moving testing activities to earlier stages of the development cycle. The earlier a defect is found, the less it costs.
A short series of quick checks performed immediately after a delivery to confirm that the essentials are working before proceeding further.
Test-Driven Development: You write the test before the code it verifies. A development practice distinct from the automation of interface tests.
Testing of user flows on mobile browsers and native iOS and Android apps, performed on real devices. Learn more
Gradual load increase up to the breaking point, to determine the actual limits of an infrastructure before commercial operation. Learn more
User acceptance testing. Final validation by future users, who verify that the product meets their actual needs and not just the specifications.
Testing of an isolated function or component, written and executed by developers. The foundation of the testing pyramid.
Verifying that we've built the right product, whereas verification ensures that we've built it correctly. These are two different questions that are often confused.
Web Content Accessibility Guidelines, an international web accessibility standard with which the RGAA aligns. Three levels of compliance: A, AA, AAA. Learn more
An automatic call sent by one system to another when an event occurs. Used to push a test result to a tracking tool or a messaging system. Learn more
Tell us which one. We're constantly expanding the glossary based on questions we receive from our customers and prospects.