This article is for everyone that asks, “why would you want to keep testing the features that are already live? Shouldn’t we focus on testing new features?” Or simply, “what is regression testing”? Those of you who already know the importance of testing regressions for both can probably focus on other content instead.
Regression testing does what it sounds like – it tests for regressions. Regressions change an application in a way that is worse – the classic example is the deployment of a new feature breaking some other already existing and otherwise functioning part of the codebase. https://prodperfect.com/wp-content/uploads/2021/09/breaking-2-min.jpegs need to make sure that new code they push does not introduce issues into existing production code, hence the need for regression tests. Remember:
“An ounce of prevention is worth a pound of cure.” -Benjamin Franklin
Types of Regression Testing
Regression tests can consist of a very large number of tests in sufficiently mature apps, which can negatively impact test runtime. Teams must balance testing new code for correctness while also ensuring no new bugs are introduced into existing code. At the same time, they also need to keep pace with short software development life cycles and deliver results on a regular basis. As tests are constantly added to the regression test suite, without intelligent test suite management, over time test suites often become bloated and long test runtimes can become a serious issue. Therefore, there are several types of regression testing that aim to solve the issue of bloat.
These types of regression tests generally advocate for a strategy that involves testing a limited subset of your regression test suite based on the nature of the changes made. These types include corrective regression testing, selective regression testing, progressive regression testing, and partial or complete regression testing.
Regression Testing & Unit Testing
Regression testing for unit testing is arguably one of the most important uses of regression tests. Breaking down your software architecture into the fundamental building blocks and running regression tests against every code change helps ensure that the core functionality of your app is not impacted by new code changes.
Once apps reach a certain level of complexity, it becomes extremely difficult or even impossible to reason about how a particular change may affect different parts of the app. Concepts like idempotence and the single-responsibility principle in theory help reduce the likelihood of such situations, but if developers could write perfect code every time, there would be no need for software testing at all. Thus, regression testing your unit tests helps protect against unforeseen errors, as well as potential bugs in new and refactored code.
Automated Regression Testing
Pruning aside, regression tests necessitate running an ever-growing list of tests against new commits. This can represent a significant workload for large apps, which makes it unfeasible for manual testing. Thus, many teams instead choose to utilize automated regression testing. Not only does automating your regression test suite free up time for your QA engineers and help reduce burnout associated with manual testing, but it also enables you to run regression tests as part of a continuous testing process.
Regression Testing Tools
When you do decide to automate: automated browser regression test suites are built in a number of ways. The most basic is scripting them in Selenium, Cypress, Capybara, with Javascript in TestCafe, or using other such frameworks. They can also be built using record-and-play tools such as Selenium / TestCafe IDE. Machine Learning is making record-and-play stronger and less time-intensive, and will eventually allow record-and-play to drive itself using web traffic data.
Advantages & Disadvantages
Regression tests serve an important function when it comes to avoiding the introduction of new bugs into existing code. However, it comes at a cost that should be weighed up before committing as a team to use it or not. Below are some of the advantages and disadvantages of implementing this type of testing into a project.
Advantages
- Helps provide assurance and security that code changes do not break existing features, either in directly modified code or in seemingly unrelated code
- A lot of the tedium associated with constantly running the same tests over and over can be alleviated with automated regression testing, providing excellent test coverage without the manpower requirement of manual testing
- Regression tests allows you to catch bugs faster, as it is much easier to tie down bugs to a single commit or test run when regression testing is run on a consistent schedule
- Regression test suites also allows you to intervene and fix bugs faster, rather than wait for them to be found by your users post-deployment
Disadvantages
- Regression test suites can very quickly become bloated if they are not maintained, as new tests are constantly appended to the existing test suite to account for new commits
- Scalability also impacts test runtime, as larger test suites also take longer to run, which delays testing turnaround and the time it takes to get feedback to developers
- Manual regression tests can become a very time-consuming and expensive process, typically requiring dedicated QA engineers to handle the workload
The Future of Testing
Very few developers would argue against the importance of regression tests. It’s very hard to reason about the impact of code changes in large codebases, even if the change itself is small. Unwanted side-effects can be avoided in some very specific ways, such as by only using functional programming languages, but for many projects, this kind of change isn’t feasible, so regression testing helps developers find and catch these bugs before they make their way to production.
At ProdPerfect we’re looking for new ways to make regression tests even easier and more seamless. By leveraging the power of AI to deploy genetic algorithms on historical regression test data, we can help pinpoint which tests are providing value to your test suite and which are not. This helps cut down on test bloat, a pain point of the regression test process, and cut down on test runtime without impacting test suite quality.
If you’d like to see how ProdPerfect can help optimize regression testing for your project, get in touch with a member of our staff today and see how we can help you.