144: Automated Responsive Design Testing, Galen

With all the different mobile platforms available today, responsive design testing has become critical. Your website needs to look awesome — not only on a luscious, 27’” Retina iMac, but also on netbooks, iPads, iPhones, smart watches & Blackberries. Your apps also need to scale to all those different screen resolutions. So how do we tests these UI based scenarios in an automated fashion? That’s what we’ll be test talking about today with the creator of the Galen Framework, Ivan Shubin. Galen Framework is an open-source tool for responsive and cross-browser layout testing. Listen up and discover how to automate your UI-based test cases.

About Ivan Shubin

Ivan started hist career in 2007 as a Test Automation Engineer at Lohika in Ukraine. In 2010 he moved to Netherlands and started working as a Software Engineer in Test at eBay/Marktplaats. Throughout hist career he has worked on test automation, CI pipelines and deployments. Currently, he is contributing to the open-source project Galen Framework in his spare time.

Quotes & Insights from this Test Talk

At that time, I didn’t really think about image comparison, because a lot of people were telling me that they tried image comparison and that didn’t work. It was really hard thing. I decided not even try it. When I looked at the website, I noticed that some of the common things like the header is on top, it has a specific height all the time. Then if you resize a browser to a mobile device size, then suddenly menu, instead of being horizontally lined, becomes, how you say it, rendered as a table. Two column table, just one of the examples. Some elements go away, some elements stay. Some elements stretch, some elements are located on the website on the right side. Then I realized that actually if I write all that down, it might become already a good explanation of layout. For instance, I also thought about this idea. What if I want to tell my friend how my website looks like and my friend is blind? For instance, he cannot see the website however I tell him. Then I started writing a little bit those sentences, and eventually those became just Galen statements and became the Galen language. Of course before that, I actually tried to do it all in Java instead of inventing this language. I tried to just write all has hamcrest matchers, but that didn’t look nice at all. It became very, very … It was actually hard to write. I realized that actually inventing a language will make it easier.
The easiest way to start writing a test in Galen is actually just writing a simple text file. Galen doesn’t really care about extension, but I use GSpec extension because I have my own syntax so works for me. You can just start with a single file. You can declare objects. If you want to start testing layout with Galen, first of all, you need to declare objects on your page. The objects are a pair of names and locations. By default, it uses CSS locator or you can use what ever. Once you have declared those objects, you can start testing those. You can say that this object should be left of that object at a specific arrange pixels or an exact pixel, it should be an exact pixel distance. Actually that’s it.
Actually Galen is completely based in Selenium. There is a bit of obstruction, of course, so at some point, I was planning to make it possible to use Galen without Selenium. I didn’t have enough time for that. Galen uses Selenium to open a browser, to identify all the elements of the page, to get their locations.
Currently, I’m working on another feature on version two, in release 2.4. It’s imitation testing Actually I already implemented it, just need to beautify the report. Imitation testing, what would do is it would run your layout check and if the layout check succeeds, then it knows that your layout test passes. Now it can find a way to see if your layout test doesn’t catch a specific issue. Tries to randomly displace some elements on a page, drag them, resize them, tests your tests, your tests. That’s another idea that I’m currently working on.
There are two things in Galen framework. There is an image comparison, of course, but that is explicit. You have to say that this element should be verified using an image comparison against this specific image file. Then only the area of that element will be checked. Galen was originally built without image comparison. It just uses the location of the element. When Galen tests the layout, it first fetches the locations of elements, like left and top. It fetches their size. From there, it already knows the boundary box of every element. What it does, it just checks the boundary box and that’s it. That is why Galen framework is not, how you say it, is not like proper visual testing tool. If you don’t use image comparison, you don’t really check the pixels and you don’t check what is rendered, it’s more like a tool for developers to quickly verify that their layout is not broken. In this case, this is not.
The only advice I can give you so far is try not to write a complete page spec. Try to minimize your specs. If you write 300 lines of Galen specs code, or 400 lines, and so on, later it will become a mess. This will be really hard to maintain. I see a lot of people do this, and later they just throw the tests away. It’s hard to maintain. Then they say, “Oh Galen doesn’t work. Galen sucks.” What is nice about Galen is that you can actually make it more flexible and you can condense your specs. For instance, if you think properly about your layout, what sentences would you use to describe your website, you can actually put it down using custom rules. You can condense your tests, sorry, you can actually describe, hold your layout in custom rules in simple sentences. Then those sentences would generate some specs. Then, in the end, your spec file would be smaller. Then, of course, here, you need to have some practice. It’s hard to do it right off.

Resources

Connect with Ivan Shubin

May I Ask You For a Favor?

Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.

Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!