End-to-end (E2E) testing is a way to find defects in web applications by simulating the behavior of real users.
The main weakness of this method is that the tests do not always reflect the behavior of actual users, which completely defeats their purpose.
To address this issue, we collaborated with the Computer Science Research Laboratory (LaBRI) at the University of Bordeaux to find a way to improve E2E tests by identifying exactly when, where, and how the tests differ from actual user behavior so that we could correct them.
A New Method Put to the Test
The method tested in this study consisted of two fundamental components:
1. An automatic recording tool
Initially, a non-intrusive recording tool was used to record test execution traces, as well as traces of real users interacting with the application.
To make the recordings easier to understand, screenshots were included with each interaction ( made possible by the Chrome extension API).
These clear visual representations of the traces provided a detailed understanding of the targets and any specific interactions.
2. Diff Algorithm (Diffing Traces)
The purpose of a diff algorithm/interface was to identify and highlight the differences between test runs, even when the same scenario was run by one or more users.
A trace captures interactions performed through the web interface (clicks and keystrokes) and can be recorded while a test is running or when a user is navigating the web application.
Comparing a test trace generated by a script with a user trace highlights their differences in terms of interaction and helps identify:
- Interactions not detected by the test.
- Actions performed by the testing tool but not by the user, or vice versa.
- Any other type of discrepancy.
Evaluation of the Method
In the end, we found that the algorithm was indeed able to identify:
- New interactions that are more relevant to a real user's experience than those used in the test.
- Poorly designed test cases, which help testers understand where to improve their tests.
These results led us to conclude that this process perfectly complements all E2E testing toolkits by improving the quality of tests implemented using the most popular frameworks (Playwright, Cypress, etc.)
We were able to pinpoint exactly where the tests were either out of sync with the application or too far off from a real user's behavior.
Going forward, we would like to build on this real-world experience by including more users in each scenario to reduce the subjectivity of the results and continue working with the university to further refine this method.
Mr Suricate Dedicated to the continuous improvement of your automated tests
Even the best software must be made even more useful and relevant in a digital world that is constantly evolving, just as the needs of modern users are.
Mr Suricate low-code solutions Mr Suricate helped countless companies accelerate their digital transformation, but we also prioritize innovation to better meet the needs of businesses today and tomorrow.
