In a nutshell: A testing campaign is the organized execution of a set of tests to validate the quality of a website or application before it goes live. It can be manual or automated and follows specific steps: preparation, execution, and analysis. This guide explains what a testing campaign is, the different types of testing campaigns, the benefits of automation, and the steps to carry one out successfully.
To ensure the quality of a mobile app or website, it’s important to conduct testing campaigns. But what exactly are they? What are the different types of testing campaigns? And what steps are involved in a testing campaign? Mr Suricate explains Mr Suricate all.
In military jargon, a campaign refers to all operations conducted by an armed force in a specific location over a limited period of time. And honestly, we could use exactly the same definition for a testing campaign! Perhaps that’s why it’s called a testing campaign, after all. Because it simply refers to runninga series of tests over a specific period of time.
What is it for? A testing campaign allows you to test the application or website—either in its entirety or in part—as effectively as possible. It can serve multiple purposes: verifying that the application continues to behave as expected after software updates, ensuring that the addition of new features hasn’t caused regression in existing ones (regression testing), verifying the behavior of a new feature to be implemented (validation testing), and so on. But the ultimate goal remains the same: to ensure that the software, mobile app, or website functions correctly.
The Different Types of Testing Campaigns
Since there are several families and types of tests (functional, non-functional, regression, validation, acceptance, exploratory, etc.), there are several types of testing campaigns, depending on what you want to test.
– The validation campaign:
The validation campaign involves verifying that a new feature or a new product as a whole (mobile app, website) behaves as expected.
It can consist of numerous test cases, which allows for a truly thorough testing of the new feature or product in question.
– The regression campaign:
Consisting of regression tests—preferably automated—the regression testing campaign is used to verify that the addition of new features has not caused any bugs or changes in the behavior of existing features.
It generally covers the entire product (software, mobile app, website), but requires fewer test cases than the validation campaign. Most importantly, it must be performed regularly.
– The acceptance campaign:
The acceptance testing campaign ensures that a code release has not caused any critical bugs or regressions in the application, website, or software.
It covers the main features, which means it runs fairly quickly (5 to 30 minutes) and requires a relatively small number of test cases. Each code release must be accompanied by an acceptance test suite consisting of critical tests.
Suri-tip: The choice of testing campaign depends on your needs, resources, and available time . But keep in mind that it’s important to prioritize the different types of campaigns. To optimize your time, it’s important to launch and validate an acceptance testing campaign before launching a regression testing campaign, which provides a quick and effective snapshot of the platform’s status before bringing out the big guns.
And what about automation?
While validation campaigns are difficult to automate—or at least more complex and less worthwhile— regression campaigns should be automated as much as possible, since they must be performed regularly and are therefore time-consuming when done manually. As for acceptance testing campaigns, they MUST be automated as a top priority (if a choice must be made), because this allows bugs to be detected much more quickly and thus corrected as soon as possible. The quality of your application depends on it!
What are the steps involved in a testing campaign?
The success of test campaigns depends on test planning (test plans). These plans help define what will be tested (which feature, all or part of the product), why, how, when, and by whom.
To quote the ISTQB definition, a test plan is “ a document that describes the scope, approach, resources, and schedule of planned test activities. Among other things, it identifies the elements and characteristics to be tested, who will perform each task, the degree of tester independence, the test environment, the test design techniques, and the test measurement techniques to be used .”
However, test plans must also include a risk analysis to identify the limitations of these choices and their potential impact on the product.
What's next? Next come the other five steps that make up a test campaign:
1. Design:
According to the test plans, this is the stage at which the tests to be run are selected: unit, integration, system, acceptance, validation, regression, etc. This is also where their priority and the conditions under which these tests will be run are determined. It is essential not to skip this phase, as this is where decisions are made about how to assess the product’s quality.
2. Implementation:
Before you can run the campaign, you must first create it in your chosen test management tool. This involves setting up the test environment, including its data, as well as preparing the tests. Note: This step is also used to write tests for campaigns that validate new features.
3. Implementation:
This is the execution phase of the testing campaign. During this phase, test cases are run according to the test plan and the configurations defined in collaboration with the business team. It is, of course, at this stage that the tests are executed, but it is also when they are analyzed, identified issues are validated, and a decision is made to suspend the campaign if necessary.
4. The results:
This step is also very important because it is here that we can analyze the test results and verify the detected anomalies in order to then proceed with maintenance and corrections. It also allows us to take stock of the challenges encountered, the test cases that need to be updated, the changes to be made for future campaigns, and so on.
5. The conclusion:
Once the test campaign has been run and analyzed, there is only one thing left to do: archive it. In the context of a regression or acceptance test campaign, this involves maintenance work—adding new test cases and deleting or modifying old ones. Without this, the test campaign will not be as effective as it is supposed to be.
In conclusion
As mentioned on *La Taverne du Testeur*, a testing campaign isn’t just about running test cases. It’s an entire process that requires careful planning and implementation. There are choices to be made, priorities to be set, tests to be selected, and, of course, results to be analyzed. Above all, these choices must be made based not only on your needs but also on your human, material, and financial resources. Whether manual or automated, it’s all about preparation.
FAQ
What is a testing campaign?
It is the planned execution of a set of test cases on an application to verify its quality before it goes live. It involves preparation, execution, and analysis of the results.
What are the different types of testing campaigns?
Depending on the objective, we distinguish between functional, non-regression, performance, and accessibility campaigns, among others. These campaigns can be one-time efforts prior to a release or recurring.
Should You Automate Your Testing Campaigns?
Yes, as long as they're repetitive. Automation makes it possible to rerun a campaign with every update—faster and without involving a team—while ensuring more reliable coverage.