Preventing QA Burnout Means Choosing the Right Values

“Transparency, Humility, Enthusiasm, Impact-Orientation, and Ownership. These are very admirable values, but they seem much more about how to live than how to run an organization. And they seem to have very little to do with your product. Why did you pick them?”

An investor recently asked me this question.

At ProdPerfect, we’ve chosen these company values because they fight burnout within our organization, within the QA teams we serve, and in the world around us.

Our Company Mission: Fighting Burnout

We designed our company values explicitly around our core mission of fighting burnout. We want to fight burnout for both our customers and our own team. When organizations neglect these values, eventual burnout becomes inevitable:

  • Without organizational transparency, teams lack understanding of the facts and reasoning behind big decisions, and may not fully trust leadership, creating an atmosphere that breeds anxiety and resentment, contributing to employee burnout.
  • Without humility, employees are likely to spend emotional energy bickering, fighting, and playing politics, leading also to burnout.
  • Without enthusiasm, work becomes a grind or drudgery, and employees spend time at work preferring to be elsewhere, contributing to burnout.
  • Without connecting the impact of their work to the mission of the organization or the welfare of others, individuals default to believing that their role does not have intrinsic value, contributing to burnout.
  • Without clear ownership of KPIs or problems, people easily become confused, problems persist, and frustration arises. When employees don’t feel that their area of ownership is respected, they feel helpless, and they may develop resentment. Each of these experiences contribute to burnout.

We continually strive internally to fight burnout by individually cultivating, and collectively teaching, reinforcing, and celebrating transparency, humility, enthusiasm, impact-orientation, and ownership. Holding to these values makes for a much more delightful, fulfilling, and productive workplace.

But, beyond our internal organization, how does ProdPerfect’s QA automation service fight burnout? How does it help other businesses adopt and reap the benefits of our values?

ProdPerfect QA Automation as Organizational Transformer

When ambitious, forward-thinking organizations partner with us, they are able to break through barriers to excellence that are a direct result of the core dysfunctions of legacy software quality assurance. ProdPerfect helps our clients reach further towards each of these values, decreasing burnout common to the conventional state of a software organization.

Transparency: In conventional QA testing, there is little clarity to the software organization as to which tests have been developed or why. Teams of similar sizes can have a few dozen end-to-end tests or thousands of them. What ultimately drives this? It’s unclear. This creates frustration for the developers whose code is being tested, and a lack of alignment between developers and QA engineers. Cross-silo fighting often results.

With ProdPerfect, data exclusively drives what the machine identifies as important to test. It’s clear that each test case reflects a pattern that users perform frequently, and teams can see exactly how frequently users interact with patterns that do and don’t have test cases in the ProdPerfect suite. Testing decisions become extremely transparent and there is less room for argument.

Humility: In conventional testing, intuition drives the decisions around what to test. The industry has for decades asked QA automation engineers to get in a room with some excel sheets and come up with what’s important to test. Sometimes the product team helps, sometimes developers help, and sometimes product analytics helps. But ultimately these decisions are made by people’s intuitions and they become attached to them. When bugs are missed, egos come out to play. “Why wasn’t this covered?” is the conversation every QA lead dreads, but must frequently face. When this question is asked, people get defensive, justifying their decisions. By its design, this system fails the people involved.

Because ProdPerfect’s test suite is objectively built off of data, there are no egos involved. The tests exist because users told us that these workflows are important to them. If a bug is caught, great. If a bug makes it into production, we can know that the facts told us it wasn’t worth the maintenance resources or test runtime (see: testing pyramid) to build a test for it. No room for ego.

Enthusiasm: Who loves maintaining end-to-end test suites? Who loves chasing down and diagnosing a flaky or unstable test? That’s right, nobody. ProdPerfect means machines take that burden away from humans, and so they can move on to work that deserves more enthusiastic attention.

Impact: Because of the intuitive nature of conventional test case development, it’s impossible to know if the test you’re writing has a large impact, a small impact, or zero impact. You might be writing tests that cover a pattern of behavior that nobody will ever use on your application, and you know that.

ProdPerfect’s data focus means every test is impactful. No longer are engineers spending time writing tests that have little or no value to the software organization. That waste has been removed.

Ownership: Who owns quality? Is it QA, or developers? Is it both? How often do these two departments fight back and forth over who is responsible for which area of quality, and how often do they fight over who has the power to make decisions about quality? When a bug is found in an end-of-sprint regression cycle, who is responsible for hunting it and fixing it? In many organizations, the relationship between software developers and QA engineers is quite toxic, and it’s because their ownership seems to overlap, with neither being clear what they have full ownership of and control over.

With ProdPerfect deployed, we can get part of the way towards clarifying that question. Developers should own their own unit and integration tests over the parts of the application they write. ProdPerfect will own testing the application as a whole: with each build, developers will get objective feedback about whether they broke the application. If they did, they own fixing their code, and have the power, tools, and context to do so. QA can continue supporting in many other realms: from other forms of testing to serving as analysts and advisers to developers to help them write quality code from square one.

Ultimately, software development and QA testers or engineers are set up for a frustrating relationship in the old model. Most experienced engineering leaders reading this will recognize elements of the above from their past, and most are likely familiar with toxic or dysfunctional developer/QA relations. By breaking that wheel of ineffective, unclear, intuition-based testing and mutual antipathy between these two groups, we enable a transformation towards a much healthier, more productive, and less burnt-out organization.

We exist as an organization because we wanted to bring something into the world that would help us and others further what we find most important. With our team, our investors, and our partners, we will strive to help all of us to avoid burnout, and thereby to thrive.

ProdProfile: Josh Pevner


1. What’s your background that’s led you to your current position at ProdPerfect?

I came from a completely weird gig doing natural language processing, machine learning, and healthcare-related work. And then Co-founders Erik and Dan told me: “Hey, we’re working on this project, it’s really interesting.”  And then they pitched it to me. And I was like: “Oh, that’s absolutely boring. That’s a brilliant business.” A lot of people go after these hyper-local, social, global kind of businesses. This is the exact opposite of that. It’s not particularly interesting, which is exactly why we want to automate it. I’ve built some of these regression tests before. It’s not a fun gig. It’s tedious, painstaking work, and that’s why we are making it go away.  


2. Can you paint for me a picture of what life is like in your role?

My day-to-day is an adventure. My role straddles a lot of lines between doing the grunt work of coding and the people management side of making sure my team is happy and knows what they’re doing. In QA, so much of your job is being this relaxing force for the customer that you in turn want to make yourself a source of calm. But the company is changing so fast that the delivery team is constantly on fire. That’s the heart of opportunity right now: making it so that we live by our philosophy of avoiding burnout and automating menial labor. So what we’re working on is building up internal tools to make it easier for the delivery team to address what happens when a client’s test fails. Getting as much as we can into a process that’s at least scalable, if not truly automated.

3. What are some elements of a “typical” interaction with customers and/or product?

Our team tends not to have direct interactions with customers.  As far as product: we’re the “secret sauce” of the company. We have this thing called the analysis engine.  It’s where all the automation happens. The analysis engine takes in all the client’s data, slurps it all up, and says: “Okay, here are all the tests you actually want to run.” And it does this by performing this analysis saying: “Let me map out your website, and then look at the most popular flows through it.” It’s like looking at where the most congestion is on a roadmap. And saying: “This is what we need to be watching.” That’s what gives us our coverage. Currently, we generate what’s called test cases that get turned into tests. The goal over the next couple quarters is to move directly to generating. Generation and automation is a big part of what we’re working on. Also, improving the quality of the results: making them robust and scalable and all those magical things.

That’s the heart of opportunity right now: making it so that we live by our philosophy of avoiding burnout…”

4. What intrigues you about engineering at ProdPerfect?

My role is very multi-faceted. The job has two different sorts of challenges. The technical challenges are pretty straightforward. The human relationships and management side is more butt-kicking—making sure everyone feels like the company is following its values, knows where they’re supposed to be, and turns into this harmonious hive. Whereas everything is already built out in a bigger company, here we’re growing together as a family. We’re learning that what you focus on is everything. This is definitely a values-first company. So making sure what we focus on aligns with our values. From building culture to building internal processes. And making sure our product reflects those values. 

5. How do you explain to people the value of ProdPerfect?

You don’t need to be technical to appreciate ProdPerfect’s value. The value of ProdPerfect is that it keeps peoples’ websites up. We find bugs. That’s what we do. Our job is to make sure that when people make changes to their websites, they don’t break them. We find problems with peoples’ websites before they do. That’s what our tooling is about: analyzing a website’s traffic, building up a set of test suites so we understand what works and what doesn’t. And then quickly informing the customer when it doesn’t. The fact that it’s such a straightforward value proposition is why I joined ProdPerfect.

6. In your own words, how would you differentiate ProdPerfect from competitors?

A lot of folks who generate tests tend to put a lot of the labor on the customer – they are product companies. Whereas we are a service company. As much as we are converting into a product company over time, because we’ve had a background as a service company, that’s going to shape the DNA of the product to be much friendlier. We will continue to provide support. Because being there for the customer matters. There’s also a difference culturally. We’re in a different phase than our competitors. They’re a little bit more mature, and that’s why they’re going to need to figure out where they’re going next. We’re going to wake up to that same problem one day. But I believe that because we’re self-aware, when we enter that world we’ll be prepared.   

ProdProfile: Kyle McCullough

1. What’s your background that’s led you to your current position at ProdPerfect?

My background is mostly in back-end software engineering, site reliability and operations. I’m familiar with standard QA practices, but it’s never been my job.

I knew ProdPerfect was a good fit because, in my prior job, I built a smaller-scale version of what ProdPerfect does. I built an environment that allowed our internal teams to test any branch of their software in a scalable, dynamic QA environment. That project gave me an idea of what it would look like to do this kind of work at scale, which is one of ProdPerfect’s primary goals—to make testing easy and scalable.

2. Can you paint for me a picture of what life is like in your role?

At this stage, we’re figuring out how to scale, which is one of my favorite problems to solve. I like taking a technical solution and figuring out how to expand it and roll it out.

One project my team is working on now is what we call our remote harness environment. Until recently, customers ran tests in their own Continuous Integration (CI) systems. They would maintain their own servers to do that. The remote harness allows customers to run tests on demand, on infrastructure that ProdPerfect controls. They don’t have to worry about running servers or doing anything that would make integrating with ProdPerfect difficult. My job is building that and then scaling it. I.e.: How do we support 10 times as many customers as we have now?

3. What are some elements of a “typical” interaction with customers considering our product?

The role of my team is to maintain infrastructure, build APIs, and provide a nice, uniform experience for running workloads in our environment. So my interaction with customers is through tooling—publishing and releasing tooling customers can use to run tests. Our goal is to provide consistent, reliable tooling that behaves predictably and is a joy to use.

Internally, it’s a support-based interaction. If customers have questions like “How does this work?” or “Why is this not working?”, we’ll investigate, support, and troubleshoot. We’ll figure out why the environment is failing to behave the way we want it to. For other teams, we develop best practices and make sure that, across the organization, we’re doing things consistently in a way that’s scalable and maintainable.

“I saw how powerful my work was in enabling other teams.”

4. What intrigues you about engineering at ProdPerfect?

The thing that got me hooked was when I saw how powerful my work was in enabling other teams. I realized the work I’d done—even though it was difficult—was worth the investment because it saved so many other teams so many hours. The idea of doing that at scale was really interesting. It’s a really hard and finicky problem to solve even for a single company. So it’s super ambitious to solve that broadly for lots of companies.

I also find it interesting because we’re a start-up. You don’t know how we’ll grow. So it’s the excitement of building something out that doesn’t exist yet and learning how to scale it as the business grows.

5. How do you explain to people the value of ProdPerfect?

The value we provide is in making the process of developing software better and in making better quality software. We cut out friction by making it easier to test, maintain and develop software, and deploy things with confidence. Of course, there’s also value in money saved in not shipping bugs.

Another thing I think about for anyone involved in the development lifecycle is: How frustrating is it for them to test software and get a release out? If we make testing easy and quick, everyone will use it. But if it’s tedious and frustrating, people are going to skip steps. And they’re not going to use systems that are there.

One of the other things that got me hooked was our CEO and Cofounder Dan’s vision and enthusiasm. The company has a set of values and everything revolves around that.

6. In your own words, how would you differentiate ProdPerfect from competitors?

We are multiple products in one. There’s the analytics side: capturing analytics from customer sites. Then there’s machine-learning automation: a lot of companies do testing, but our efforts are to remove humans from that process and still make testing useful. Then, there’s what my team does: serve as an infrastructure provider. We’re not just handing off tests and saying: “Here, go run these.” We’ll even run the tests for our customers.

We collect the data, develop the tests in an automated fashion, and we even provide the infrastructure to run those tests. It’s as low-friction as we can make it. The unique combination of how we’re solving this problem is definitely what sets us apart.

ProdProfile: Hao Chen

1. What’s your background that’s led you to your current position at ProdPerfect?

Before ProdPerfect, I was working for one of ProdPerfect’s first customers. At the time, I was doing QA myself. Other engineers were doing QA. Everyone I could tap on the shoulder was doing QA. There was a lot of value in removing the QA burden. So when ProdPerfect CEO and Co-founder Dan was pitching me his idea, I was like: “Wow, this is actually useful.”

Fast-forward a year-and-a half later, I’m here at ProdPerfect.

2. Can you paint for me a picture of what life is like in your role?

I’m one of the engineering team leads. Our team works on both internal tooling and customer-facing User Interfaces (UIs). It’s a mix of making sure everyone on the team understands why we’re working on certain products/features, team members have everything they need and are unblocked, our team has the right resources and full-stack skill set to design, develop, and support new products/features, and others in the organization have good visibility into our product roadmap and current progress. Especially when you’re working on user-focused interfaces, it’s important to be product-minded and have that customer-centric way of thinking.

3. What are some elements of a “typical” interaction with customers considering our product?

Sometimes we’re providing support to the end-user or customer. Other times, it’s doing research asking customers: “What are your needs? Does this product or feature we’re building solve your needs?” Questions from customers come through many channels. In the spirit of transparency, we’ve been collecting questions that come in about the product into an internal quiz and ask team members: Which team is best suited to answer this question?” At this stage, there’s multiple stakeholders that collaborate to answer these questions.

Cross-team collaboration is one of the hardest things at any company. One thing that helps us here is our stated company values, which include transparency, humility, enthusiasm, impact, and ownership. Having open and honest discussions about our successes and failures helps us grow as a team. It’s about figuring it out together.

“It’s about figuring it out together.”

4. What intrigues you about engineering at ProdPerfect?

I like coming to work every day because of the people. Our CEO and Co-founder Dan has done a great job building a team of smart, ego-less people who share a similar mindset of wanting to make an impact and do good, who are kind and fun. I want to keep nurturing that company culture as we grow.

An unwritten value here is learning. A lot of people are just lifelong learners here—whether it’s learning new languages, taking classes, or just expanding their knowledge. There’s lots of people doing book club together—reading books on different topics that interest them and then having discussions on them. Those are all great ways to learn and grow.

5. How do you explain to people the value of ProdPerfect?

ProdPerfect is automating QA as a service. The company’s mission is to prevent burnout. We prevent burnout by helping customers prevent fires by catching significant, actionable bugs before they hit production. We catch bugs by running regression tests for customers, typically against their staging environments.These regression tests are machine-assisted in their creation by analyzing the most heavily-trafficked user flows on the customers’ web applications collected by our JavaScript snippet.  At the end of the day, ProdPerfect reduces the QA burden by automating away the painful parts of software development.

6. In your own words, how would you differentiate ProdPerfect from competitors?

We always have competitors. From companies doing no QA to having their own internal QA team to outsourcing parts of their QA. But no one is doing exactly what we’re doing. We have the data-science portion—analyzing traffic—but we also have that human in the loop. Test suite creation is machine-assisted, and it’ll be moving more toward that direction to help scale the company. But at the same time, context is hard to know without having a human in the loop: someone that’s knowledgeable about the customer’s web application. That’s the value of our delivery team. ProdPerfect perfectly combines extra QA manpower with data-based decision-making to give our customers peace of mind.

ProdProfile: Stephen DeFlumeri

1. What’s your background that’s led you to your current position at ProdPerfect?

With software, everyone thinks of actual development. Nobody was thinking about: Quality Assurance (QA) is super important. So I started researching and reading about it.

The company I was at before solved a massively important problem in QA. But I saw this problem of regression testing. Nobody could figure it out. It was this pain point when Erik approached me. Then it was like a lightbulb went off, like: “That makes so much sense. Why has nobody done this before? So I said, “let’s do it.” I was under the 10th employee. I’ve been here from the start and don’t plan on going anywhere.

2. Can you paint for me a picture of what life is like in your role?

Every day is different. I’m not sitting behind a desk having these dreary boring 30-minute calls where nobody wants to talk to me. We have this really interesting culture. Everybody has their role, but everyone wants to help out with everybody’s else’s role. That’s why we have such fascinating work. Everybody cares. Everybody wants to solve problems, whether it’s their problems or somebody else’s.

3. What are some elements of a “typical” interaction with customers considering our product?

Zoom meetings are awesome. But there is no replacing having a conversation in person. You get to know someone, not just like: Hey, the weather outside is rainy today. There’s a big problem where salespeople aren’t humanized, they’re just this being on the other side of the line. You want to let people know you care about more than what we’re selling. It’s about relationships.

We start off with a conversation. There’s nothing worse than getting to the demo where people still have conceptual questions. I want to the demo to be: “I get it and now I want to dig deep.” That’s where it gets fun. It’s also hard to show what our data analysis looks like in a black box. So we give companies a way to see what their testing suite is and try it without committing to a long-term contract first. Customers should know us front-and-back like if they were interviewing a person to go do that job.

“You want to let people know you care about more than what we’re selling. It’s about relationships.”

4. What intrigues you about sales at ProdPerfect?

I really like selling to engineers. I get excited talking to leaders with these really cool ideas. I get to learn about new companies who are doing fascinating things. And I get to help make that fascinating thing even better. To say: This is awesome. Now we can make it where your development team is more efficient. You can spread your brand out there and people are going to love it even more.

I was never really good at development. So my philosophy was: If I can’t make it, why don’t I help sell it? It still gives me the ability to be tight with engineering, understand product and the whole Software Development Life Cycle (SDLC). QA touches so many different parts of the business. That gives me a really cool advantage because I like learning. I like being a part of it.

5. How do you explain to people the value of ProdPerfect?

I talk about three pain points: 1) It’s really hard to figure out what to test. 2) Building those tests is an art form. 3) Because we’re so agile in the way we develop software, the maintenance and evolution of test suites has become this daunting task no one wants to do. So why don’t we let machines do that part? I love hitting home: “Hey, let’s solve this in a way more unique way that makes it way less painful.”

Regular people? I say: When’s the last time you’ve been on an e-commerce site and you click the check-out and it doesn’t work? How angry did that make you? People go: Oh, it’s the worst. I go: We help prevent those issues from happening. We let users tell us those workflows so we know exactly what they’re doing. And make sure those flows are working at all times. When I explain it like that, even my grandparents get it. And my grandfather got a cell phone for the first time 6 months ago.

6. What are the top few frequently asked questions people have about us? About our product?

The biggest one we get is Personally Identifiable Information (PII) and data privacy. If you’ve seen The Karate Kid, Mr. Miyagi has a really good quote: “Best block: not be there.” We want to block all of that personal data. Questions also come up around that fact that we are machine-led and human-assisted. The goal is to be fully close-cycled, where machines do everything. Our CEO Dan Widing puts it the right way: we are a self-driving car that still has a driver in the driver’s seat. How much human involvement is needed? It’s continuing to trend downward. It’s much less resource-intensive than it would be to manually build, maintain, and update. We let machines do the heavy-lifting.

Ultimately, as much as our job is teaching about ProdPerfect, even more of our job is evangelizing the QA infrastructure outside of what we do. ProdPerfect can work all day, but if we don’t have all the testing components to make the ecosystem complete, one tire works but the rest are flat. The car won’t drive.

7. In your own words, how would you differentiate ProdPerfect from competitors?

We’re the only company I’ve seen that’s using data in this way to build tests. Traditional tooling has been reactive: companies still need to build their own requirements and test cases. We’re one of the first proactive solutions. Not only are we telling you what should be tested, we’re continually building it, maintaining it, updating it, without the customer having to tell us what to do. They’re not managing us. They can focus elsewhere, which I think is really cool.

Everyone’s trying to make QA more agile. There’s no company telling you to slow down. Mark Zuckerburg said: “move fast and break stuff.” The philosophy now in QA is: “let’s move fast and not break stuff.” So “let’s move faster and let’s use data to do the heavy lift” is ProdPerfect’s unique approach.