One of the most common challenges for quality engineers is making the mental shift from merely pursuing pass or fail test results to understanding the characteristics and trends of tests. That’s what we’ll be test talking about today with Greg Paskal. Greg will share how to discover the personality of your application using dashboards created with ELK Stack. Listen up to discover how to create tests that are accurate and actionable.

About Greg Paskal

Greg is currently the Director of Quality Assurance – Automation at Ramsey Solutions, A Dave Ramsey Company. He is also the author of multiple white papers on test automation and testing in general.

Greg recently published his first book, “Test Automation in the Real World”, sharing insights from over 30 years of automated testing development. He has spoken at many conferences including StarEast, StarWest, QAI and QA TrailBlazers.Tutorial.

Quotes & Insights from this Test Talk with Greg Paskal

• Elastic Stack is a group of opensource tools. And one of them is like a data warehouse type of concept that gathers information that we would inject into it. Then there is an additional tool that allows searching on that I believe the term that the ElasticSearch and then the Kaban toolset which allows the visualization of that these are all bundled together. Some of them are open source; some of them are a bit proprietary to the elastic toolset. But that’s basically what the Elastic Stack is it comes with this bundle of things.
• It’s an excellent opportunity to get to go to a conference. They come to the automation guild let’s say or are any of the other types of test conferences that are available. You know they get there, and they’re meeting folks, and they’re there maybe attending different sessions and whatever. And then they go back, and it’s as if all that effort and the cost and everything had no impact on them. I think that that hurts us as automation engineers it hurts us as technologists. That we don’t go into a conference with the intent to learn and meet as many folks as we can. And from that you know come away even if it’s one good takeaway or two takeaways that we bring back to the office and apply for our employer.
• So when you start to look at data over time, you begin to look at past just a test pass or fail. You start to look at one other parameter that’s very helpful which is how long did that test take to run. It’s something we overlook. And yet almost all of our tests produce that sort of data in the reporting. If you can record that one attribute and say hey my login test let’s say it took three seconds to run and you could start to record that information over the next let’s say the next 90 days. And you knew there was going to be some development in your sign on process may be your log in process and you started to notice hey you know 30 days ago we bumped up from three seconds to four seconds and then about another month after that and we’re now taking five seconds to log into our app consistently. That’s something that could creep up over time, and as a tester, you might not even recognize that slight difference of 1 or 2 seconds.
• So I worked on visualization to identify flaky tests, and it was pretty easy to do. All I did is I log into the data lake every time we began a test set and every time we exited the test set, and I gave it a unique ID and said For every start there should be a finish. And if there’s not, we’re going to call it off like a test. So we have a visualization. Overall the entire automation that shows us that and down to the specific product is pretty cool.
• The tool I predominantly build everything in is called visual builder and that’s the one I would encourage people if you’re gonna put time into anything that’s the one to look at and try to get turns around because it provides the most flexibility of all the visualization tools within the Kabana tool set from my perspective.
• I would suggest folks go out to Elastics web site and check out maybe a free demo on the elastic cloud product or consider standing perhaps an instance of this up on a Linux box you have an experiment with it. Be patient and start taking those initial baby steps of how do I get data in there. What data should I get in there and then how do I make something meaningful out of it by creating your first visualization?

Connect with Greg Paskal

• Twitter: @GregPaskal
• Blog:gregpaskal.com
• Book Site: realworldtestautomation.com
• Greg’s awesome book Test Automation in the Real World

Rate and Review TestTalks

Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

Powered By SauceLabs

Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!