A/B testing
A comparison of two versions of a page or user flow across two user samples to determine which one converts better. This is not a software quality test: it measures behavior, not defects.
95 definitions to ensure everyone—product, QA, and engineering—is on the same page. From test cases to load testing, including false positives and QA debt.
A comparison of two versions of a page or user flow across two user samples to determine which one converts better. This is not a software quality test: it measures behavior, not defects.
The ability of an online service to be used by everyone, including people with disabilities. It is assessed against standards such as the RGAA in France or the WCAG internationally. Learn more
The difference between expected behavior and observed behavior. A broader term than “bug”: an anomaly can stem from the code, the data, the environment, or the specification itself.
Direct verification of data exchanges between systems, without going through the interface. Faster and more stable than an interface test, it detects regressions at the service contract level. Learn more
A condition checked at a specific point in a scenario. If it is false, the execution fails. A scenario without an assertion does not test anything—it simply runs through the steps.
Acceptance-Test-Driven Development: Business, development, and testing teams collaborate to define acceptance criteria before development begins. These criteria serve as both specifications and acceptance criteria.
Running test scenarios using a program rather than a person. This does not eliminate manual testing; rather, it frees up human time for exploration and judgment. Learn more
An ordered list of test cases to write or revise, prioritized by business risk rather than ease of automation.
Behavior-Driven Development: The expected behavior is described in natural language—often in Gherkin format (given, when, then)—before the code is written. Goal: Business, development, and testing teams share the same definition of what is expected.
A software defect that causes unexpected behavior. The cost of fixing it increases depending on when it is discovered: a few minutes during development, several days in production.
A compiled and linked version of an application at a given point in time. This is the object that is tested before deployment to production.
A document that lists the test cases to be verified, their steps, and the expected results. It serves as a shared reference between the business and QA teams and is the starting point for automation. It used to be called an acceptance test specification.
A set of scenarios run together in a given environment on a given date. It generates a single execution report.
Description of a specific test: initial state, actions, expected result. A scenario consists of several test cases.
Continuous integration and continuous delivery. An automated pipeline that builds, tests, and deploys an application every time the code is changed. Learn more
A consent management platform—the banner that obtains the user’s consent before trackers are set. It determines what should happen, but it does not guarantee that the scripts will comply with it: that is precisely what a test verifies.
French Data Protection Authority. It issues recommendations on cookies and trackers and imposes penalties for violations. Its guidelines serve as the de facto standard for consent checks.
Explicit user consent must be obtained before non-essential trackers are set. A refusal must block the trackers, not just hide them: the verification process involves refusing the request and then ensuring that no trackers are sent.
The portion of the product actually verified by testing. It is measured by business workflow rather than as a percentage of lines of code: covering 80% of the code without covering the order processing flow provides no protection. Learn more
Criteria that must be met for a feature to be considered complete. These criteria serve as the basis for test scenarios.
The severity level assigned to an incident, which determines the urgency and the alert channel. A broken payment flow and a missing image do not have the same criticality, even though both result in a failure.
Test the same page on multiple browsers and versions. Just because it displays correctly in Chrome doesn't mean it will in Safari.
European Directive on Non-Financial Reporting by Companies. It requires companies to document the environmental impact of their services—including digital services—using specific, date-stamped metrics rather than annual estimates.
A data layer that the site exposes to tracking tools such as Google Tag Manager. A broken data layer doesn't break the site—it silently skews your statistics. Learn more
A list of criteria a team sets for itself to declare a task complete. When these criteria include testing, test debt stops accumulating.
The measured deviation between a run and previous runs on the same stage of a route. A slow drift does not trigger any technical alerts; it can only be detected by comparing measurements over time.
A buildup of unstable, obsolete, disabled, or never-written tests. Like technical debt, this comes at the cost of maintenance time and undetected regression issues. Learn more
A tree-based representation of a web page, as rendered by the browser. UI tests rely on this to identify the elements to be manipulated.
European Regulation on Digital Operational Resilience in the Financial Sector, in effect since January 2025. It requires that the continuity of critical services be tested and documented.
The European Accessibility Act, the European directive on accessibility. It has applied to a large portion of private-sector online services since June 28, 2025. The RGAA, for its part, is the technical standard used to verify compliance.
A French method for assessing the environmental footprint of a web page, which assigns a score and a grade ranging from A to G. It is based on the transferred weight, the number of requests, and the complexity of the document.
A test that walks through the entire workflow, from the interface to the back-end systems, just as a real user would. Learn more
An environment dedicated to running tests: acceptance testing, pre-production, and production. A test scenario that runs successfully in acceptance testing may fail in other environments due to a simple difference in data.
Launching a scenario at a specific time, in a specific environment. It generates a status report, duration, screenshots, and logs.
A test conducted without a predefined script, in which a person actively searches for vulnerabilities based on their experience. It is the natural complement to automation, not its competitor.
A test that passes even though the defect exists. This is the more dangerous of the two: it gives rise to unwarranted confidence.
A test that fails even though the product works. When false positives occur repeatedly, they erode confidence in subsequent tests, to the point where no one reads the reports anymore.
Test instability, where a test passes or fails randomly even though the code has not changed. This is often caused by poorly managed expectations, shared data, or a slow environment.
Verifying that a feature produces the expected result from the user's perspective. This is the type of testing most closely aligned with business needs. Learn more
Syntax in structured natural language, of the "Given, When, Then" type, which describes a scenario in a way that is readable by both business teams and developers.
A visual comparison of two versions of a page, section by section. They detect what no functional test can: a missing legal notice, a button that has become invisible. Learn more
Digital eco-design process. Tested automatically, it checks page size, the number of requests, and the efficiency of loaded resources. Learn more
Running a browser without a graphical interface. Faster and more resource-efficient, this mode is suitable for large-scale runs in continuous integration.
A fix delivered urgently directly to production. This is when a rapid regression test suite is most valuable.
Encrypted version of the web protocol. Ensuring that a session remains end-to-end HTTPS is one of the simplest—and most often overlooked—security checks, especially after adding a third-party script.
Service interruption or degradation in a production environment. An incident detected by an automated test costs less than one detected by a customer. Learn more
Verification that multiple components work together properly, whereas unit tests verify them individually. Learn more
A practice that involves merging and reviewing code changes several times a day, with tests running automatically each time. Learn more
A token returned to the user after authentication, or attached to a one-time link. Its lifetime is a security measure: attempting to use an expired token should fail, and this can be verified from a browser.
Data prepared for running a scenario: accounts, products, orders. A poorly isolated test suite is one of the leading causes of unstable tests. Learn more
An expression that identifies an element on the page, using CSS or XPath. A locator that relies too heavily on the structure will break at the first redesign.
Logs generated during a run. They help explain why a scenario failed when a screenshot isn't enough.
Work to adapt scenarios to product changes. This is the cost item that kills most automation projects, and the first one to be scaled up for industrial production. Learn more
Test performed by a person. Indispensable for exploration, usability, and judgment. Costly when the same verification must be repeated with every release.
Reusable action block—written once and called in multiple scenarios. Closing a consent banner or logging in are typical examples: when the rule changes, you update it in just one place.
A simulated component that replaces a real system during a test, to overcome limitations related to its availability or cost. While useful, a mock never proves that the actual integration works.
Regularly run test scenarios on the production environment to detect a failure before users do. Learn more
Average time to detect and average time to resolve an incident. Two performance metrics that provide more insight than the raw number of bugs. Learn more
An approach that allows you to build scenarios without writing code, using blocks and actions. It makes automation accessible to people in various business roles and reduces reliance on a single person.
Verifying that a change hasn't broken anything that was working before. This is the primary use of automation—and the most cost-effective one. Learn more
The ability to understand a system's internal state based on what it exposes: logs, metrics, and traces. It complements testing; it does not replace it.
A one-time code sent via text message or email to validate a sensitive action. Testing a workflow that relies on this requires actually receiving the message—not bypassing this step.
User paths whose unavailability has a direct impact on revenue or compliance. A testing strategy begins with these paths.
Measuring response times and resource usage under a given load. A functional but slow website is effectively down for some users. Learn more
A series of automated steps triggered whenever the code is modified: compilation, testing, deployment. Learn more
A document that defines the expected measurement events, their names, and their parameters. It serves as a reference for the data team, but there is no guarantee that the site will implement it: a data layer test measures the gap between the plan and reality.
An open-source browser automation tool that is widely used in development. It also serves as an output format: a no-code script exported to Playwright prevents you from being locked into a single tool.
An environment that closely simulates production, serving as the final step before going live. This is where most non-regression campaigns are conducted, because the data there closely resembles the actual data.
A model that recommends a lot of unit tests, fewer integration tests, and few end-to-end tests. Useful as a guideline, but should be adjusted when the risk is concentrated in the workflows.
Quality Assurance. The set of practices aimed at ensuring the quality of a software product, from requirements definition through production. QA is not limited to testing.
Outsourcing all or part of the testing function to a service provider, with coverage objectives and contractual oversight. Learn more
Campaign summary: past scenarios, failed attempts, durations, captures, logs. This is the feature that lets you analyze a failure rather than simply endure it. Learn more
The validation phase for a release prior to its deployment into production. It may be technical, functional, or user-led.
A bug has appeared in a feature that previously worked. This is almost always a side effect of a change made elsewhere. Learn more
A general framework for improving accessibility; the French adaptation of the WCAG used to verify compliance. It describes the criteria to be checked; the requirement itself stems from the law and the EAA.
General Eco-Design Framework for Digital Services, published by Arcep and Arcom in 2024. It outlines the eco-design process for a website or application, while the EcoIndex provides a metric for it.
European General Data Protection Regulation. The test checks for consent, cookie banners, retention periods, and the trackers that are actually set. Learn more
Return on investment. When applied to test automation, it is measured primarily in terms of campaign time saved and regression issues caught before production, not just in terms of licenses saved.
A sequence of actions and checks that replicates a complete user journey. This is the basic unit of automation.
Verification of an application's security measures: authentication, authorizations, injection, and data exposure. This is complementary to a security audit, but not equivalent to it. Learn more
An expression that identifies an element on the page based on its structure or attributes. The choice of selector determines the robustness of the scenario.
The ability of a tool to automatically correct a scenario in which an element has changed its position or identifier, without human intervention. Learn more
Automated monitoring of factors that affect search engine optimization: tags, HTTP statuses, canonical tags, structured data, and load times. Learn more
The value above which an execution is considered a failure: response time, page weight, or visual deviation. The threshold is set by the team, not by the tool.
Moving testing activities to earlier stages of the development cycle. The earlier a defect is found, the less it costs.
A short series of quick checks performed immediately after a delivery to confirm that the essentials are working before proceeding further.
Test-Driven Development: You write the test before the code it verifies. A development practice distinct from automated unit testing. This should not be confused with BDD and ATDD, which apply the same principle to expected behavior and acceptance criteria.
The duration of a specific action in a user flow, from the click to when the screen is actually usable. This is the unit of measurement for a performance test, distinct from server response time.
Testing of user flows on mobile browsers and native iOS and Android apps, performed on real devices. Learn more
Gradual load increase up to the breaking point, to determine the actual limits of an infrastructure before commercial operation. Learn more
User acceptance testing. Final validation by future users, who verify that the product meets their actual needs and not just the specifications.
Testing of an isolated function or component, written and executed by developers. The foundation of the testing pyramid.
Verifying that we've built the right product, whereas verification ensures that we've built it correctly. These are two different questions that are often confused.
Web Content Accessibility Guidelines, an international web accessibility standard with which the RGAA aligns. Three levels of compliance: A, AA, AAA. Learn more
An automatic call sent by one system to another when an event occurs. Used to push a test result to a tracking tool or a messaging system. Learn more
Tell us which one. We're constantly expanding the glossary based on questions we receive from our customers and prospects.