Today we’ll be test talking with Paul Grossman, AKA the “Dark Arts Automation Engineer” about his five secrets of test automation. I’ve been following Paul for a long time on Linkedin, Facebook and other automation boards. He’s always posting useful test automation tips and tricks to help out the testing and automation community, so it was great to finally speak with him — and he did not disappoint. I think you’re really going to enjoy this episode. Check it out!
About Paul
Paul is a well-known test automation engineer with over 15 years experience who has designed numerous WinRunner and QTP/UFT automation frameworks for companies like Redbox, NORC, Abbott Labs and Grainger.
He has spoken at HP Discover, QAAM, PSQT and the Dallas User’s group, and is one of the top contributors to Facebook’s largest automation group, Advance Test Automation.
Paul can also be found teaching others about testing at the TestingInstitute.
Quotes & Insights from this Test Talk
- When I’m looking at a framework I’m looking at a whole collection of really good ideas.
- Exploratory testing is done better with manual testers. Also automation is better at doing stuff that are repetitive and us humans we’re bored by doing the same thing over and over and over again. Sometimes we get tired, sometimes we skip steps and then we look and go, “Oh my gosh I just found a defect.” How did you find it? I skipped this 1 step here that’s what it was I can go back. I didn’t actually find a defect.
- Some defect are low level. I call it the Pony type of … the Pony level of defects basically you’re going to get a pony for Christmas before that defect gets fixed.
- We were tracking our metrics in Excel sheets. One metric was just the number of hours our tests ran, which was 35,000 hours from four machines over three years, including an endurance test that ran 24/7. Multiply that by maybe $40 per hour to pay a manual tester to do it and they would have spent $1.4 million dollars. We also found 7 show-stopper defects, including a memory leak that would have been difficult for manual testers to detect. Any one of those defects could have resulted in $100k in lost sales for the company. All told we saved the company at least $2.1 million with test automation over manual testing.
- Sometimes I think of this as the George Bailey approach. You have to imagine well if we didn’t have automation what would things be like? Then of course there’s the other question that comes out when we talk a little bit about money is our test automation tools. I’m going to exclude Selenium out of this, are they expensive? The first thing we think is, “Yeah there’s a lot of money for those tools.” Let’s say there’s about $10,000 a pop for a license of those and that can sound really expensive. Then again you think about a manual tester and for $10,000 is going to show up for a whole year and do manual testing? Probably not, it’s just pretty … That’s below poverty level.
- I’d see LeanFT as a stepping stone between the UFT and full blown Selenium. I see that as a really great opportunity from HP and I can’t wait to start actually working on project with LeanFT.
Resources
- Alan Richardson‘s Selenium Java Course
- Boyd Patterson and his Test Design Studio tool for static code analysis of QTP/UFT code
- Linda Hayes: Automation Testing Handbook
- Adam Gensler: Simple Automation Framework for Remarkable Obvious Notes (SAFFRON)
- Wilson Mar: Automation and everything else in life
- Tarun Lalwani: KnowledgeInbox.com and three awesome books on QTP UFT.
- Dani Vainstien: www.advancedqtp.com Wrote “Fresher” Courses on QTP.
- Marcus Merrell & Will Roden: SoftwareInquisition.com – You can only find it in the Wayback Machine (aka InternetArchive.org
- Bryan Harrington: Code library at psacake.com.
- Lee Barnes: Conference speaker. Has five Rants on LinkedIn regarding test automation.
Connect with Paul
- LinkedIn: https://www.linkedin.com/in/pmgrossman
- Email: qtpmgrossman [@] gmail
- Twitter: @qtpmgrossman
- Training: http://www.testinginstitute.com/faculty.php
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!