1. What led you to ProdPerfect?
I’ve been working in technology sales for the past 13 years and previously was at a company where I successfully sold to ProdPerfect cofounders Erik and Dan. Erik and Dan ended up in NYC last summer because they were accepted into the Entrepreneurs Roundtable Accelerator (ERA) program. We’d hang out, get drinks, eat, and really got to know each other. And because I had sold to them and they enjoyed working with me, they said: “We want you to come sell for us.”
Working at ProdPerfect has been extremely exciting. I accepted not only because it was an awesome opportunity for career growth and learning where I knew my opinions and feedback would have a real impact, but also because ProdPerfect is doing something absolutely revolutionary in the industry that shines through in the work that I do every day.
2. Can you paint for me a picture of what life is like in your role?
I meet with CTOs, heads of engineering teams, directors, and heads of QA regularly and speak with them about how we can improve their regression testing efforts, increase efficiencies, save their team time, and better accurately test. I often hear healthy skepticism from them: “This sounds like magic/Does it actually work?” and then I get to show them it’s not magic and it does work. It keeps me on my toes and is exciting to demonstrate that our solution can provide them a bunch of benefits they never thought they could have. It’s also fun to hear people get elated and say things like: “How has no one thought of this before?!”
3. What are some elements of a typical interaction with customers?
Organizations don’t want to feel like we’re adding in more work or hindering their processes, but that we’re taking this work off their plates and helping them. So we make it clear that we’re integrating right into their Continuous Integration (CI) tool so our tests run with every build automatically, as if they had built them in-house. And because we’re driven by data, we’re able to take on the maintenance of the code as well: we fix broken tests build-to-build and add new tests as the web app evolves over time. Our model alleviates concerns around how much work they’re going to have to do.
Typical reactions after the first call are: “Wow, this definitely seems valuable.” “This seems like magic.” “Everyone should have this!” And of course: “I like the promise that you’re making, but I need to learn more.” Sometimes, clients are skeptical that there’s a particular area of their platform that we’ll be able to support. I’m a big advocate of being honest and upfront. Having a demo of the client’s platform as a step in our process to assess what we can and cannot handle and being transparent about that helps to 1) level expectations with the client, and 2) instill confidence in working with us.
4. What intrigues you about your role in sales at ProdPerfect?
I get to challenge the way people think about testing. In regression testing, coverage has always been: “What percentage of the application is covered?” We’re asking people to philosophically change the way they think to consider: “What if we could guarantee that the majority of observed user behavior from production is being covered by our testing suite?” Challenging people way more technical than me, who’ve been doing engineering for 15, 20+ years.
We also challenge perspectives that existing test suites encompass the “right kinds of tests.” One company had built 1,000 of their own test cases, which took hours to run, and they couldn’t run a regression test with every build as a result, which meant they’d see bugs in production even though they had many tests. Our response to this dilemma was: “How do you know those 1,000 test cases you’ve created are the right tests?” And to help them see it as: “we’re going to build a lean test suite based on user behavior that runs with every build and catches more bugs” because we’re testing accurately for how users are using the platform.
5. How do you explain to people the value of ProdPerfect?
For someone with no tech experience: when you build software, you need to test it. A lot of companies test their software manually or build automation. But automation needs to be maintained. That takes time and money. Figuring out what to test is one of the hardest jobs in the testing world. Humans are guessing what to test and then spending time building and maintaining tests as the application changes over time. Yet, they’re not always testing accurately. ProdPerfect’s solution automatically builds and maintains test suites based on real user behavior so companies don’t have to do it or maintain it themselves.
For someone technical, I’d ask them questions like: “Hey you have a web app, how are you testing it? What’s a pain in the butt about it? How much time are you spending manually testing or building/maintaining automation? Let’s talk about how we can automate the whole process so your team doesn’t have to spend effort or mindshare on it.” Just higher-level, taking it off team plates so they don’t have to bother with it. I’ve never heard one person say in the last year that they enjoy doing regression testing. Customers know we’ve got them covered.
6. What are the top few frequently asked questions people have about us? About our product?
One common question is about things that won’t get picked up from data, which is what’s used to build our testing suites: “We’re worried you’re not going to get enough traffic around what we need to be covered. You won’t pick up on this really important edge case or mission-critical feature.” We solve for this by allocating a certain amount of wildcard test cases for each client: custom test cases they can hand off to us. And we’ll build out those tests, incorporate them into the suite, and maintain them alongside other test scripts that were generated from the data.
Some people are wary about letting us control maintaining and updating the code. Recently, we’ve come out with new functionality in our test results summary web portal that allows clients to request to skip a task if they know a big User Interface (UI) change is coming. This new functionality removes that test out of the suite and adds it to a queue to be fixed by the delivery engineer (proactively rather than reactively) so that when the client deploys, that test failing doesn’t block the deploy.
Privacy and security is a concern as well, as we’re asking companies to put a data tracking snippet onto production so we can collect clickstream metadata. Once we explain and/or prove that our snippet avoids collecting PII (by not collecting any keystrokes or info about the dom) and that we don’t have a performance impact on production, we can normally circumvent lengthy infosec processes, but we also have a ton of experience going through them as necessary.
7. In your own words, how would you differentiate ProdPerfect from competitors?
We’re the only tool/service in the industry that’s solving the most critical challenge of testing: knowing what is important to test. With other tools and services, you still have to define test cases or user stories and hand them off for crowdtesting, or build and maintain automation. Humans approximate what they think should be the test cases, then build a bunch of tests, some of which aren’t important. This increases deploy time and sometimes prevents us from running a regression on every build. Bugs still make it to production because we’re not accurately testing for how users are using the application.
ProdPerfect uses live user traffic to both 1) build an automation suite and in the process catch more bugs before production. And 2) maintain the suite. Teams put dozens to hundreds of engineering hours into fixing broken tests and maintaining automation suites. Our model allows us to automatically evolve the suite and keep the code maintained for clients, all the while providing real-time feedback to QA/dev teams on every single build.