Continuous Delivery Automation

How can we deliver what we’ve made to our customers as early as possible, as well as get faster feedback from our product users? To achieve this, more and more companies are moving toward a continuous delivery process.

How does automation fit into this paradigm shift? In this episode Sahas, shares tons of real world experience with us about how to make our CI/ automation testing efforts awesome.

About Sahas

Sahas is a developer and general technologist with lots of passion for producing high-quality working and usable software. An individual with entrepreneurial drive, he has more than ten years experience in the IT industry, DevOps, quality engineering, consulting, leading and coaching Agile teams.

Along the way, Sahas has lent his expertise to all kinds of companies and organizations: some large, multinational organizations collaborate across the globe, successful startups, and many consulting engagements.

Quotes & Insights from this Test Talk

• You shouldn’t be thinking about quality as an after-the-fact, which unfortunately has been the way how we have worked so far. We worked first to get the requirements out and we’ll design, I’ll do some coding, and finally we will say, “Okay. I’m ready to bring in some testers. Let’s start doing testing.” Instead of trying to verify our understanding and quality at the end of the cycle, continuous delivery … If we want to deliver product continuously, we have to continuously evaluate the quality at every stage. It’s, testing throughout the cycle is important, very critical piece of building quality, fitting quality into continuous delivery.
• Over time, one of the conventional complaints that I heard over and over with many teams is, “We invested in automation. We invested in GUI automation. We created 700 test cases. After six months, everything started failing. We don’t know why.” Of course, it would because you did not maintain it. Your product moved on. Your code moved on. You maintained your product code, your production code, but you did not maintain your test cases. Why wouldn’t it fail? c It’s a concern if it’s not failing. It’s implementing our energy in a much more constructive, thoughtful way, is important. That’s the fifth aspect. I call it rationalized automation.
• I follow specific coding practices to design your test cases, all your tests should be independent. Even a GUI test, it should be independent. You shouldn’t be depending on external data, meaning you shouldn’t be assuming that I have a particular item in my inventory and I’m trying to change and edit that thing. If you’re trying to change and edit something, you should better populate that item. You know that item exists. You just go and edit and update. 
• For our automation dashboards we use ElasticSearch, Logstash, and Kibanastack, ELKstack. It’s completely open source, basically that’s used for a few different purposes. One is collect, log collection and analysis areas. It’s used in analytics. Lincoln is a big player in that. They kind of use a BI and combination of ElasticSearch to index their data. The whole backbone behind it is ElasticSearch indexing and the whole technology behind it works really faster to give you the data or whatever you want. They index it. There are some best practices, to index the data in a specific way so it gives you benefits.
• We kind of established something called continuous delivery maturity model that includes your practices, all of it from product owner, how do they write the story? How do you write acceptance criteria? How do you verify acceptance criteria at the end of the story before you close that? All the way from there. Some of your coding practices, some of your development practices, code quality, your quality practices, that includes test automation, and we try to bring in … When I say quality, it’s not only functional, it is functional, it is load and performance, it is security, it is operations and everything.
• We’ve been doing quality in a particular way for so long, constantly wherever I go, I would hear, “We’re automated. We’re running it every night.” What I would say is, it’s great what we’ve been doing so far, but in the future, it’s not gonna encourage that way of doing it. We have to bring quality into the team. We have to take the team along with us. There are some miscalculations. When somebody talks about Agile, people start thinking about there are no more, there is no room for quality engineers anymore. There are no QAs. Everybody should do everything kind of bandwagon. What I mean is, there’s definitely a lot of room for quality experts, quality evangelists. Somebody has to look at it from a quality lens.

Resources

• Elasticsearch
• Blazemeter Taurus
• Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

Connect with Sahas

• Twitter:@Sahaswaranamam
• Blog: CdInsight

May I Ask You For a Favor?

Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.

Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.