Continuous Testing Management Made Easy

Almost every company I know is currently going through some sort of digital transformation. The speed with which QA is getting builds from development teams is increasing. But how do you know that the changes introduced by a new build are actually covered by your regression tests? In this episode Eran Sher, the CEO and co-founder of SeaLights.io, shares some ways in which to deal with these issues (and more!) to greatly improve the quality of your unit and regression tests.

About Eran Sher

Eran has over 20 years experience as an entrepreneur, building emerging and high growth enterprise software companies. He is currently the Co-Founder and CEO of Sealights, the first cloud-based, continuous testing platform. Prior to founding SeaLights, Eran co-founded Nolio (One of the first Application Release Automation platform) which was acquired by CA Technologies. Following which, Eran lead the DevOps Business Unit as VP Strategy.
Prior to Nolio, Eran held senior management positions at PortAuthority Technologies ( acquired by Websense), Mercury Interactive ( acquired by HP) and Conduct ( acquired by Mercury Interactive).

SeaLights is the first and the most innovative cloud-based Code↔Test Quality Management Platform. Thier mission is to bring together QA teams and R&D, engineers, and managers in one single cohesive & actionable platform that enables high quality while increasing release speed.

To do this they built the next generation Test, Code & Quality management platform that makes quality measurable across all tests, tools, technologies, and environments!

Quotes & Insights from this Test Talk

The problem today, and I’ll explain the importance of this metric, is that at least what we’ve seen, 95% of the companies take a decision if the release is good or bad only according to the fact if the test passed or failed. What happened if you are running all your regression tests, all the tests are passing, but actually you don’t know what they are covering. They’re actually didn’t test the code changes that were introduced in that build and you’re going to release it. That’s one key metric that companies need to look at today. You need to make sure that the content of the build is tested. It’s hard because with continuous integration, you’re getting lots of builds. You’re getting incremental code changes. In many cases, the feature is not even ready as a whole. You’re getting part of the feature until it’s ready with ready with one of the last builds
The way we do it is that we are scanning every build, and we know what changed in each of the builds. We know in a dynamic … The code coverage of all your tests. Not just the unit test, but every test you might run from the browser to the back end, multiple technologies. From that we can identify those qualities also. It’s done automatically. It’s a plug in that you connect to your CI, and immediately every build that is generated with your CI is being tracked and reported. You don’t need to do anything. It’s a one time simple configuration.
When companies are moving to continuous integration, you have lots of builds. If the test cycle is getting longer, it means that the developers would need to wait after each build. They will need to wait more time to get the test results, then if they can continue with their development. The testing time of each build is becoming very critical because now it has a direct impact on the efficiency of your development team. If your build time, which also includes the test time, is increasing over time you need to be aware of that. It means that you’re going to hurt the efficiency of your developers because after each build … Let’s say they need to wait 40 minutes, that might be too long. If you can work in a smarter way and complete the build in ten minutes, you will save a lot of time for every developer.
If you don’t know how to measure the code coverage of your test, then you might be wasting lots of time and lots of money writing the same test, duplicate the test, duplicated with regard to the code coverage and not really understanding what your tests are not covering. That’s where you need to focus with your next test. These efforts today about automating tests, this is … Again, we talked about the transformation that QA engineers, testers, test automaton engineers need to do. They need to start working like developers. If you’re developing a test automation, that means that you are writing code. If you’re writing code, you need to understand what your tests are covering and what they’re not covering so you can improve your test on the code level. I think it’s a must tool for everyone who is developing test automation. It will make you a better test automation developer so you can develop fewer tests faster with higher quality.
The three things: The dashboard, quality holes, and you will know exactly what your tests are covering and what they’re not covering. All your tests, not just the unit test.
“If there is one advice, it’s put the quality dashboard. Everything today is within your CI CD … It’s actually running behind the scenes. You need this visibility. Start with measuring the important quality metrics, putting a quality dashboard that everyone will see. For every build you need to understand what was tested, what the status, what’s the coverage, is it improving, is it decreasing, do you have quality holes or not? Make sure you have a quality dashboard for every build and it’s essential as one so everyone can share it, developers, QA, DevOps, management, that will be my main advice.

Resources

Sealights interactive product demo

Connect with Eran Sher

May I Ask You For a Favor?

Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.

Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!