In brief: A test campaign involves the structured execution of a series of tests to validate the quality of a website or application prior to its production release. Such campaigns can be conducted manually or through automation, adhering to distinct phases: preparation, execution, and analysis. This guide elucidates the concept of a test campaign, its various classifications, the benefits conferred by automation, and the requisite steps for its successful implementation.
To ensure the quality of a mobile app or website, it’s important to conduct testing campaigns. But what exactly are they? What are the different types of testing campaigns? And what steps are involved in a testing campaign? Mr Suricate explains Mr Suricate all.
In military parlance, a campaign signifies the entirety of operations executed by an armed force within a defined location and timeframe. This definition, in fact, precisely mirrors that of a test campaign, which may explain the nomenclature. Fundamentally, it corresponds to the execution of a comprehensive suite of tests over a specified duration.
What is its utility? A test campaign enables the most efficient possible evaluation of an application or website, either holistically or partially. Its objectives are multifaceted: to confirm consistent application behavior following software updates, to ascertain that new feature implementations have not introduced regressions into existing functionalities (regression tests), and to validate the behavior of prospective features (validation tests), among others. Nevertheless, the overarching objective consistently remains the same: to ensure the correct operation of the software, mobile application, or website.
Types of Test Campaigns
Given the existence of various categories and types of tests (e.g., functional, non-functional, regression, validation, acceptance, exploratory), multiple campaign types are employed, contingent upon the specific testing objectives.
– The Validation Campaign:
A validation campaign aims to verify that a new feature or an entire new product (e.g., mobile application, website) performs in accordance with predefined expectations.
This campaign may encompass numerous test cases, thereby facilitating a comprehensive and in-depth evaluation of the new functionality or product.
– The Regression Campaign:
A regression campaign, ideally comprising automated regression tests, is conducted to ensure that the introduction of new functionalities has not induced defects or behavioral alterations in existing features.
While typically encompassing the entire product (software, mobile application, website), it requires fewer test cases than a validation campaign. Crucially, it mandates regular execution.
– The Acceptance Campaign:
An acceptance campaign ensures that a code delivery has not introduced critical bugs or regressions into the application, website, or software.
It covers core functionalities, allowing for relatively quick execution (5 to 30 minutes) and requiring a limited number of test cases. Each code delivery must be accompanied by an acceptance campaign comprising vital tests.
Suri-tip: The selection of a test campaign depends on your needs, resources, and available time. However, it is crucial to prioritize the different types of campaigns. To optimize time, it is important to launch and validate an acceptance campaign before initiating a regression campaign. This provides a rapid and efficient status of the platform's condition before deploying more extensive testing.
What about automation?
While validation campaigns are difficult to automate, or at least more complex and less beneficial, regression campaigns should be automated as much as possible, as they require regular execution and thus represent a significant time drain when performed manually. As for acceptance campaigns, they MUST be automated, and this should be a priority (if a choice must be made), as automation allows for much faster detection of bugs, enabling prompt correction. This is crucial for the quality of your application!
What are the stages of a test campaign?
The success of test campaigns hinges on test planning (test plans). These plans are essential for defining what will be tested (which functionality, all or part of the product), why, how, when, and by whom.
According to the ISTQB definition, a test plan is « a document describing the scope, approach, resources, and schedule of planned test activities. It identifies, among other things, the items and features to be tested, who will perform each task, the degree of independence of the testers, the test environment, and the test design and measurement techniques to be used. »
Furthermore, test plans must also include a risk analysis to outline the limitations of these choices and their potential impacts on the product.
What's next? Following this, there are 5 additional stages that constitute a test campaign:
1. Design:
Based on the test plans, this stage involves selecting the tests to be executed: unit, integration, system, acceptance, validation, regression, etc. It also involves determining their priority and the conditions under which these tests will be performed. This phase is critical, as it dictates how product quality will be assessed.
2. Setup:
Before the campaign can be executed, it must first be created within the chosen test management tool. This includes setting up the test environment with its data, as well as preparing the individual tests. Note: This stage also involves writing tests for new feature validation campaigns.
3. Execution:
This is the execution phase of the test campaign. Here, test cases are run according to the test plan and the configurations defined with the business stakeholders. It is during this stage that tests are executed, analyzed, detected anomalies are validated, and decisions are made to suspend the campaign if necessary.
4. Review:
This stage is also highly significant, as it allows for studying the test results and verifying detected anomalies to subsequently proceed with maintenance and correction. It also provides an opportunity to review encountered difficulties, identify test cases requiring updates, and determine necessary changes for upcoming campaigns.
5. Closure:
Once the test campaign has been executed and analyzed, only one task remains: archiving it. In the context of a regression or acceptance campaign, this involves performing maintenance by implementing new test cases and deleting or modifying existing ones. Without this, the test campaign will not achieve its intended effectiveness.
In Conclusion
As the Tester's Tavern states, a test campaign is not merely about executing test cases. It encompasses an entire process that requires proactive planning and implementation. There are choices to be made, priorities to define, tests to determine, and results to analyze. Crucially, these decisions must align with one's needs, as well as available human, material, and financial resources. Whether manual or automated, thorough preparation is paramount.
FAQ
What is a Test Campaign?
It is the planned execution of a set of test cases on an application to verify its quality before production deployment. It encompasses preparation, execution, and results analysis.
What are the Types of Test Campaigns?
Specifically, these campaigns are categorized as functional, non-regression, performance, or accessibility, depending on their objective. They can be executed as one-off events prior to a release or on a recurring basis.
Should test campaigns be automated?
Yes, particularly when they are repetitive. Automation enables the re-execution of a campaign with each update, achieving faster results and requiring no team mobilization, while simultaneously enhancing test coverage reliability.
