Before we jump right to the internet of things (IoT) testing tools, I think it will help to do a quick overview of the two most common protocols used for IoT devices: MQTT and CoAp.  

Luckily I was able to glean most of this information from Scott Moore, founder of SimpleAPM, who has worked for over 27 years with some of the largest applications and infrastructures in the world.

Read on to learn more about:

• MQ Telemetry Transport or MQTT
• Constrained Application Protocol or CoAP
• IoT Performance Testing Tools that support MQTT or CoAP

First, What is MQTT? 

MQ Telemetry Transport (MQTT) is probably the most popular protocol you’ll deal with when it comes to your IoT performance testing. MQTT is a transport communication protocol, which generally means it’s going to send information.

MQTT was originally developed by IBM to monitor an oil pipeline. It’s based on TCP and has been around since 1999, so it’s not new.

Typically, it uses a publish-subscribe-based messaging model by default. In this setup standard, you have what’s called a broker.

The broker will house topics, and there can be a one-to-one or one-to-many relationship between the publishers of data.

In most cases, an IoT sensor is giving this information to the broker, and a subscriber to the broker. So the subscriber would be the recipient of this information.

It follows that as a sensor publishes data—for instance, the current temperature in a smart home or a break in an oil pipeline, we know that something has ruptured and there’s a problem. For example, it may send a 1 or a 0 to topic one, which could be all about damages, topic two could be temperature values, etc.

The sensor is publishing to these topics, and once it’s been updated, the subscriber is waiting on new information to be received. Once it’s been received, that subscriber does something to it with that data.

MQTT protocol was designed for high volume, very low bandwidth, and low energy utilization, so it could theoretically last out in the field or on a desert pipeline for quite a long time.

It has a very light footprint because it had to be transmitted to a satellite connection (instead of having hardwired internet connectivity) and then back down to a central headquarters area. It is also event-oriented and avoids polling, so you won’t be draining batteries.

The Constrained Application Protocol (CoAP)

is binary and is very much like an HTTP2 in the sense that it’s not sending across a bunch of text; it’s just sending across ones and zeros, which makes it much more efficient. It’s asynchronous, which means it can send data anytime it wants and doesn’t have to wait on something else, yet it has built-in discovery.

It acts very much like a rest service and has four main verbs, similar to HTTP, so it has a GET, a POST, a PUSH, and Delete, supports various mime types, and has HTTP response codes.

CoAP has its own response codes as well. It also supports UDP binding with multicast support, and it has a caching model as well.

Instead of a broker who is sort of that in-between gateway product, with MQTT, you have what’s called a proxy instead.

This proxy takes CoAP traffic and converts it to HTTP so that it can talk to a Cloud-based application on the Web that supports standard HTTP conversation.  

Just like the broker is the middleman for MQTT, the proxy is the middleman for CoAP.

There are some differences between the two, but both protocols are attempting to achieve the same result.

Now that you know a little bit about the protocols, let’s take a look at some common automation tools you can use to test IoT devices.

Which tools support MQTT or CoAP for Performance Testing?

• Iotify.io enables the simulation of thousands of IoT endpoints on the Cloud and the measurement of your system performance under load. It also helps you fix scaling issues before your customer discovers them.
• Mosquitto is a very popular open-source broker people use in their test lab. They’ll usually put Mosquitto on a raspberry pi or some other small server that mimics a broker. Mosquitto is lightweight and suitable for use on every IoT device, from low-power, single-board computers to full servers.
• Malaria is a set of tools to help with testing the scalability and load behavior of MQTT environments.
• Copper is a light-level testing Firefox plugin. Copper allows you to use the browser to go to a CoAp server. So instead of you typing in the URL http://, you would type in CoAp://, and it’s able to traverse that server and perform these requests in response as if it were a CoAP browser rather than a standard Firefox browser. It does the translation for you. Unfortunately, since Mozilla changed their add-in model, this is not supported in the newer versions of Firefox. (As an alternative, check out the Chrome version.)  
• MQTTBox enables you to create MQTT clients to publish or subscript topics, create MQTT virtual device networks, load test MQTT devices or brokers, and much more. 
• MQTT JMeter is a plugin that extends your JMeter’s capability to test against MQTT protocol as easily as ordinary HTTP protocol.
• Gatling-MQTT-Protocol is an unofficial plugin for the Gatling load testing framework. It enables usage and the measurement of service performance using MQTT pub/sub protocol.
• NeoLoad 6.8 (MQTT, AMQP) is the latest release for its performance testing platform. With this release, NeoLoad continues to provide performance testing automation for a range of Web, mobile, and SAP applications, as well as APIs.
• SmartBear LoadUI NG (MQTT, CoAP) is a load testing tool for REST and SOAP APIs, databases, and microservices.
• Micro Focus StormRunner (TRUEAPI) is an SaaS-based, Cloud computing load and performance testing solution.
• Micro Focus LoadRunner (MQTT/CoAP) is arguably the best known and industry-leading load testing software tool. It helps you analyze and prevent application performance problems and detects them.
• Locust Paho Testing Utilities are a collection of Python code and tools to help test MQTT clients and brokers.
• Check-MQTT is a Nagios/Icinga plugin for checking connectivity to an MQTT broker Or with –readonly monitor an MQTT application. It can also check the status of MQTT clients, maintaining the status on an MQTT broker. 
• MQTT-stresser is a load testing tool to stress test a MQTT message broker.
• MQTT-bot for broker testing on CLI
• MQTT-PWN intends to be a one-stop-shop for IoT broker penetration testing and security assessment operations.

Discover a Real-World Example of Testing IoT

A great place to see a hands-on demonstration of testing IoT systems and applications is performance testing expert Scott Moore’s upcoming PerfGuild session on IoT performance testing.  

He’ll share the lessons he’s learned about IoT testing as well as walk us through a real-world example of something he’s recently worked on.

After Scott’s session, you’ll have the information you need to build an end-to-end IoT testing strategy.

So shortcut your learning curve to testing an IoT smart city and register NOW to view this and 15+ other awesome sessions that will accelerate your performance and site reliability testing efforts.