Browser automation testing causes a lot of justified dread in most engineering leaders. Abandoning it entirely seems beyond the pale, but maintaining a test suite can take upwards of 25% of an engineering team’s time, and the E2E QA cycle can consume the vast majority of deployment time, inhibiting teams from moving to true continuous development.
It’s no surprise that most engineering leaders grind their teeth when they have to deal with their E2E testing apparatus. Not Jim Castillo and Arpan Dalal.
Jim is the VP Engineering and Arpan the Sr. Director of Engineering at RepairPal, responsible for scaling their technology and team while rapidly expanding the RepairPal featureset to capture a greater share of the market. This team doesn’t have the luxury of throwing money and time at a bloated and clunky QA apparatus.
How Can You Get Great QA in a CI/CD Pipeline?
After RepairPal’s latest venture round, Jim knew his team needed to find the next gear, but knew that status quo testing would get in the way. “The amount of time we were spending testing was holding us back. We were releasing 1-2 times/week. We wanted to move toward releasing daily and even on demand.”
Arpan knew having a dedicated QA team would slow him down, but software developers aren’t well-positioned to develop and maintain browser automation, either. “That’s the problem we were trying to solve: Engineers can write good unit tests, but how can we reliably move from staging to production, much more quickly, while still guaranteeing that what we’re releasing is quality? That’s hard.“ Quality is key at RepairPal — “Every transaction matters. Our business model depends on customers coming back again and again. So we can’t have major issues.”
“I’ve been in organizations where I’ve had dedicated QA teams. And the results are never there. My experience taught me that there has to be a better way.”
Plug In and Accelerate
So Jim started looking for a better solution. He knew that better automation has been developed all along the CI pipeline, and believed that with new technology and a CI/CD philosophy behind him, he might be able to make E2E testing work in the high-speed posture he needed. “You look at these organizations like Atlassian and you see where they invest: they can ship code constantly. There are some processes that humans do that can be fully automated, and we can leverage that to grow even faster–to be our best that an engineering organization can be.”
As Jim and Arpan developed their deployment stack, Jim found ProdPerfect, and knew he could plug it into the pipeline he was already creating. They set clear criteria for piloting ProdPerfect as part of their deployment: RepairPal needed ProdPerfect to catch the right bugs, run quickly, and get rapid, clear feedback to his developers so they could debug in situ rather than as part of a separate QAS cycle.
“We knew ProdPerfect was the right choice when we saw that ProdPerfect was able to really discover what’s important to test by watching what our users cared about. I had this moment in the sales cycle: if ProdPerfect was able to do what it seemed to, then I knew ProdPerfect could help us take the next step.
“I saw the results. I saw that automated tests were built autonomously. I could understand what the tests did. I could interpret the results. So I could align my engineering team to understand where the issues are and quickly react.”
Jim and Arpan decisively pivoted their team to a many-times-per-day deployment posture, and have been able to reap the rewards in happier customers, faster feature releases, and product roadmap consistency. ProdPerfect plays its part, by flagging key bugs as they’re merged, and otherwise getting out of the way.“ProdPerfect helps us move very quickly: it’s a fundamental part of our development process. It helps us solve testing in an efficient, automated, zero-time-consuming way. It runs on its own. It’s going to be reliable, it’s going to be fast, I know it’s a holistic solution, and I know I can trust it. So it’s a key part of our CI/CD pipeline.”