Beginners Guide To The MQTT Protocol

What is MQTT?

MQTT is a lightweight publish/subscribe messaging protocol designed for M2M (machine to machine) telemetry in low bandwidth environments.

It was designed by Andy Stanford-Clark (IBM) and Arlen Nipper in 1999 for connecting Oil Pipeline telemetry systems over satellite.

Although it started as a proprietary protocol it was released Royalty free in 2010 and became an OASIS standard in 2014.

MQTT stands for MQ Telemetry Transport but previously was known as Message Queuing Telemetry Transport.

MQTT is fast becoming one of the main protocols for IOT (internet of things) deployments. See MQTT vs HTTP at bottom of the page.

MQTT Versions

There are two different variants of MQTT and several versions.

• MQTT v3.1.0 –
• MQTT v3.1.1 – In Common Use
• MQTT v5 – Currently Limited use
• MQTT-SN – See notes later

The original MQTT which was designed in 1999 and has been in use for many years and is designed for TCP/IP networks.

MQTTv3.1.1 is version in common use.

There is very little difference between v3.10 and 3.1.1. Here is a Github page detailing the main differences

Here is the actual Specification MQTT V3.1 and here is a more detailed overview of the MQTT protocol packet structure,.

The latest MQTT version(v5) ,has now been approved (Jan 2018).. You can download the specification here.

If you are wondering what happened to 4 then see here.

An overview of the key ideas in version 5 is here.

Here is a Github page detailing the main differences between MQTT v3.1.1 and MQTT v5.

MQTT Version 5 Support Notes:

As of release 1.6 the mosquitto broker supports MQTT v5 in addition to MQTT v3.11.

You can continue to use older version 3.11 client with the latest broker.

Currently there is limited client support for v5. See later.

MQTT-SN Notes

MQTT-SN which was specified in around 2013, and designed to work over UDP, ZigBee and other transports.

MQTT-SN doesn’t currently appear to be very popular. and the specification hasn’t changed for several years, but I expect that to change as IOT deployments start. See MQTT-SN working Notes. for more details on MQTT-SN.

Getting Started- MQTT Course

This course is for complete beginners and in it we will be covering the following :

1. How MQTT Works -Client connections
2. MQTT Topics and Topic Structure
3. MQTT Publish and Subscribe
4. Understaning Persistent Connections- Python Examples
5. Last will and testament explained with examples
6. Username and Password Authentication-with usage examples
7. Understanding MQTT keep alives with example

MQTT Clients

Because MQTT clients don’t have addresses like email addresses, phone numbers etc. you don’t need to assign addresses to clients like you do with most messaging systems.

For MQTTv3.1.1 there is client software available in almost all programming languages and for the main operating systems Linux, Windows, Mac from the Eclipse Paho project.

•  Paho Python client.
•  Node.js MQTT Client-Starting Guide

For MQTTv5.0 there is limited support for client software from Eclipse, Currently there is only a C client available. Here is a link to the client comparison chart and download page .

Currently I’m using a Python client from this github page that support v5.

MQTT Brokers or Servers

Note: The original term was broker but it has now been standardized as Server. You will see Both terms used.

There are many MQTT brokers available that you can use for testing and for real applications.

There are free self hosted brokers , the most popular being Mosquitto and commercial ones like HiveMQ.

If you don’t want to install and manage your own broker you can use a cloud based broker from Cloud service providers like IBM, Microsoft (Azure) etc

Eclipse has a free public MQTT broker and COAP server that you can also use for testing. The address is iot.eclipse.org and the port is 1883 or 8883(SSL).

See the MQTT Brokers and Servers article for a list of  hosting options. and the Mosquitto page for more details on the mosquitto broker

You can set up your own test environment with you own free MQTT broker.see the Mosquitto page for more details on the mosquitto broker.

There is also a broker available for node-red but it isn’t being actively developed see How to Install and Run The Mosca MQTT Broker on Node-Red

MQTT Security

MQTT supports various authentications and data security mechanisms.

It is important to note that these security mechanisms are configured on the MQTT broker, and it is up to the client to comply with the mechanisms in place.

See An Introduction to MQTT security mechanisms

MQTT Over WebSockets

Websockets allows you to receive MQTT data directly into a web browser.

This is important as the web browser may become the DE-facto interface for displaying MQTT data.

MQTT websocket support for web browsers is provided by the Javascript MQTT Client.

See –Using MQTT Over WebSockets

Common Questions

If you are familiar with the web and email then you will probably find, as I did, that MQTT is very different. These are some of the questions I had, and saw on other sites and forums that may clear things up a little.

Q, Can multiple client publish to the same topic?

A- Yes

Q- Is is possible to know the identity of the client that published a message?

A- No not unless the client includes that information in the topic or payload.

Q- What happens to messages that get published to topics that no one subscribes to?

A- They are discarded by the broker.

Q-How can I find out what topics have been published?

A- You can’t do this easily as the broker doesn’t seem to keep a list of published topics as they aren’t permanent.

Q- Can I subscribe to a topic that no one is publishing to?

A- Yes

Q- Are messages stored on the broker?

A- Yes but only temporarily. Once they have been sent to all subscribers they are then discarded. But see next question.

Q- What are retained messages?

A- When you publish a message you can have the broker store the last published message. This message will be the first message that new subscribers see when they subscribe to that topic. MQTT only retains 1 message.

MQTT and Python

Many of the example scripts on this site use python. Python is a good choice for learning MQTT as it is easy to use and readily available.

• Using the Paho Python MQTT Client.
• Publishing Messages Using The Paho Python Client
• Subscribing to Topics Messages Using The Paho Python Client

MQTT and Node-Red

If you don’t want to program then Node-Red is a Flow based tool that makes it easy to create MQTT projects. See Beginners guide to node-red to get started.

Other MQTT Tutorials

• Understanding the MQTT Protocol Packet Structure
• Understanding Topic Naming and Design Notes
• Understanding MQTT QOS Levels- Part 1

• Understanding MQTT QOS Levels- Part 2

MQTT Quiz

Test your MQTT knowledge with the MQTT basics quiz

MQTT vs HTTP

If you are wondering if MQTT is the best choice for your project then here are  collection of articles comparing MQTT with HTTP.

• Internet of Things: Battle of The Protocols (HTTP vs. Websockets vs. MQTT)
• MQTT vs. HTTP: which one is the best for IoT?
• HTTP vs MQTT performance tests
• MQTT and HTTP : Comparison between two IoT Protocols – contains errors but chart useful

MQTT-SN vs Coap for robotics

Real World MQTT Example Deployments

It’s often useful and interesting to see how a particular technology is actually being used. Here are some examples I’ve come across:

• Freight Farms-MQTT
• IP pinger swarm using MQTT
• How do you use MQTT- Mosquitto.org
• Owntracks -Location tracking

Resources

• Really good introductory series- Recommended reading MQTT essentials.
• Very good MQTT pdf with detailed examples.
• Installing and setting up The Mosquitto broker.