Automation Tools for Mainframe Testing

Let me guess.

You’re thinking, “Mainframe testing? Really?”

Believe it or not, mainframes are still the systems of choice for companies that need high-transaction throughput with guaranteed reliability and consistent data. Insurance companies, banks, and the retail, and transportation industries, for example.

It’s human nature for a tester to always be on the lookout for the latest shiny testing tools that only support Web technologies. But what if you work for a company that still uses mainframes, and you need to automate some of your tests?

Terminal EmulatorMainframe Shift-left Testing

Remember: just because mainframes have been around forever doesn’t mean you can’t follow modern testing practices.

According to Rosalind, a distinguished engineer at IBM, mainframes run all modern languages (Java, Swift, Node, etc.). Whatever you need to write your application; you can write it in that environment and then run it on the system.

It also supports Git and CI/CD tools like Jenkins, so from a development process and practice, and it works the same any other development environment.

Terminal Emulator

Green Screen Tools for Automation Testing Mainframes

I used to work for a large healthcare company that still used a mainframe application behind the scenes. Even though they had a browser-based application version, most customers refused to upgrade and use the new application because they were so proficient with data entry using the green screen application.

This situation may sound like a nightmare, but it actually turned into a blessing. Because the green screen application was no longer being modified, the UI was very stable. And since it was just a matter of keyboard manipulation to test the app, it was relatively easy to automate.

If you find yourself in the same situation, here are some green screen tools you should check out:

Chris Haggan a Rational Test Product Manager, mentioned in the comments that “these days we would suggest Rational Integration Tester for mainframe service testing as that supports all the IBM Generic Service Client technologies as well as CICS, DPL, IMS, PL1, Cobol Copybook, and DB2/Z. As with the Generic Service Client, these tests can all be run at scale with Rational Performance Tester (which is where most testers would encounter Generic Service Client).

This is pretty cool, but make sure not all your test cases are written just for green screen terminal emulator testing.

Once companies that have older, back-end systems start getting into more modern development, they usually start creating interfaces to interact with the backend.

That interface could be a Rest service, a Web service, a clicks transaction, an MQ call, or even a file that’s being dropped into the environment then picked up and processed.

This is important because many other tools are available to help you with interface testing as well.

API Testing Tools

Tools for Mainframe Interface and Performance Testing

As I mentioned earlier, many mainframe system services can be tested by running tools outside of the mainframe.

Since many of these functions are exposed by an API and are as easy to access and test as any other API services, you might be familiar with using the same sort of testing tools for functional automation, performance, and monitoring.

Also, as in traditional, Web-based software development, the developer should be creating unit tests as well.

Mainframe development is no different.

If your mainframe team is just getting started with automation, unit testing is where you should start.

Unit Tests for Mainframes

Rosalind mentioned that zUnit is relatively new; even though it’s been around for a while; it’s been significantly improved and continues to grow to allow you to do much more including true unit testing. Well, with the caveat that in a COBOL program, you don’t have the same concept of classes and methods. You have a large COBOL program. So you’re testing a COBOL program with a unit test, but it’s still at the program level, which is a whole lot better than the transaction on the application level.

The takeaway is that if you are testing a COBOL application, you can still create unit tests for your program!

Overall Mainframe Testing Approach

Generally, your testing strategy for mainframe applications should be the same as any other development efforts.

The only difference is that whole mainframe systems provide you with much more data. So you can get more info about how that application is performing, which is nice, but it can also be overwhelming if you don’t know what to do with it.

The other thing is that it depends on the application’s language. So if a newer language like Java or Node was used to develop your software, then all of the processing you do anywhere else is precisely the same. It’s simply running on a different set of hardware.

If it’s COBOL, then the application was probably written differently, which means you will be testing it differently. For example, you can’t do a method test.

You have to test at a program or transaction level. There are some differences from that standpoint.

The tools are going to be somewhat different from the same point of debugging the system if you want to debug something in the environment.

But most testing tools that are going to hit an API are going to hit an interface. They work with a mainframe application precisely the same.

So it depends on the tool you’re using, and it depends on what you’re testing as far as the differences you might see when planning your testing process.