Fancy techniques and advanced strategy are all well and good sometimes, but they don’t do you a lick of good if you don’t have the fundamentals down, which is why we think it’s always worth revisiting our core philosophy on software testing itself.
We don’t want to be too down on the industry, everyone is under more pressure than ever, but it still seems like the core competencies underlying software testing often get overlooked in the rush to get a product or feature live. A lot of teams start out with limited staff, and have no time to establish a formal regression testing program, which leads to serious problems down the road. If you’re just trying to keep your head above water, “later” becomes a promise you can’t keep, and the maintenance load grows extremely fast once you start adding tests — you’ll run out of resources faster than you think.
A lot of this is just the nature of startup work, but where possible, you want to look ahead, and try to avoid ending up with a serious mess later. This is a form of technical debt that is particularly challenging to pay off, and transitioning from an ad hoc testing system into something consistent and manageable is a huge challenge, and sometimes nearly impossible. Organizations without a clear plan for testing — especially fast-growing ones — often struggle to stick to their plans, no matter their theoretical commitment to the process and how well-intentioned they are.
To make matters worse, the legacy tech behind QA means that the process has substantial up-front costs, can be fraught, and is a challenging task even under the best circumstances. Burnout is real and overworked employees don’t excel in any areas, much less the ones you may have unintentionally de-prioritized.
So how do you avoid falling into this extremely common trap? We’ve got some suggestions for how to think about and implement software testing that are based on our experience and knowledge gleaned from on-the-ground implementation. Take a few minutes to read through and consider the who, what, when and how of software testing. A small time investment up front can help you reap massive benefits in the future — good testing creates a flywheel effect, reducing chaos and helping you implement a virtuous cycle of software development.
The “Why” of Software Testing
Again, this is going to seem obvious, but it’s worth going back to first principles sometimes. Everyone knows you need to test, but it can be easy to overlook exactly why. Here’s the key concerns:
- Find bugs: By rooting these out early, you prevent problems and maintain the quality of your product. Maintaining functional quality is essential for your users.
- Identify problems early: It’s not terribly helpful to discover bugs once your code is already out there, and it can be catastrophic. An ounce of prevention is worth a pound of cure, after all. The earlier bugs are found, the cheaper and faster they are to fix.
- User experience: By testing early and adequately, you produce a higher quality product, and you are more agile in responding to user requests because your bandwidth isn’t eaten up by trying to fix problems. Your CRO will also thank you.
- Smoother rollouts: Better to have a sound testing regimen in place before releasing your app to the public. Your CMO will also thank you.
The “Who” of Software Testing
Someone needs to own the software testing process, so assign this responsibility to a test lead or manager with the right competencies. The testing lead should develop a system that balances human input, tools, and services to execute your QA plan.
This leader will be responsible for:
- Planning tests and assigning test engineers
- Ensuring that the engineers have the tools and resources they need
- Aligning testing with development progress
- Maintain reporting in order to measure and drive quality improvement over time — both in testing and in development
The test engineers will be responsible for:
- Developing test cases
- Executing testing and reporting defects where necessary
- Maintaining consistent and continuous regression testing, including tests for new featured products, as well as maintaining automated test scripts
The “When” of Software Testing
Test early, test often. The sooner you can catch a bug, the more time and resources you will save, and the sooner you can take products to market. Without a lean and effective test suite, this can be an incredible challenge — an unwieldy testing apparatus will struggle to keep up with the demands of a preemptive testing regime.
Here’s some key moments when you should be testing:
- When a new feature or functionality is added
- When a new requirement is added due to functionality or features being modified
- When your source code is optimized
- After a bug fix implementation
- After a major site redesign
The “How” of Software Testing
Even if you are aware of the need for a robust and consistent software testing program, it may be out of your reach due to time or resource constraints. Rather than put off testing entirely, you should approach the challenge with prioritization in mind.
- Yes, the up front cost can be considerable, but you don’t have to tackle everything at once. Feel free to “chunk” — some tests are better than none
- Set your rules based on priority; determine your priorities by focusing on what it is that your users want, and make sure you’re building off of good data here from your product analytics toolset or your UX team
- Make sure that your testing suite, once established, is lean and well-prioritized, so that you can implement continuous testing, early in the SDLC, and with short run times, if at all possible
- Make sure that you can run the full testing suite without putting a burden on the development team
Prioritizing testing isn’t always easy — a great QA leader will use a mix of discipline and good data to know when not to write and maintain more tests!
Some Final Thoughts on the “Why” of Software Testing
It’s not always at the top of any developer’s priority list, but quality, well-implemented software testing is the foundation from which your product is built. If you get your testing right, and if you can establish a functioning test automation program, the rewards will be incredible.
To reiterate: a study by the Systems Sciences Institute at IBM found that fixing bugs in the design phase costs six times less than fixing them during implementation, fifteen times less than addressing them during beta testing, and one-hundred times less than the cost of maintenance once the update has been released.
The fundamental reason why software testing and regression testing are important is this: it’s just plain good for business.