Introduction to MQTT-SN (MQTT for Sensor Networks)

Due to the large number of sensors in IOT Sensor networks these sensors will be mainly wireless.

The main characteristics of these networks that drove the design are:

• Low Power battery operated sensors with very limited processing power and storage.
• Limited payload size
• Not always on (sleeping)

MQTT-SN (MQTT for Sensor networks) was designed specifically to work on wireless networks, and , as far as possible, to work in the same way as MQTT.

It uses the same publish/subscribe model and can be considered as a version of MQTT.

MQTT-SN vs MQTT

The main differences involve:

• Reducing the size of the message payload
• Removing the need for a permanent connection by using UDP as the transport protocol.

The MQTT-SN specification lists these differences.

1. Connect message split into three messages two are optional and are used for the will message
2. Topic id’s used in place of topic names.
3. Short Topic names
4. Pre-defined topics.
5. Discovery process to let clients discover the Gateway
6. Will Topic and messages can be changed during the session
7. Off line keep alive procedure for sleeping clients.

Architecture and Components

The specification lists three components:

1. MQTT-SN client
2. MQTT-SN Gateway
3. MQTT-SN forwarder.

What seems to be missing is a broker/server as in the MQTT sense.

However the architectural diagram in the specification does appear to show one and the RSMB server does implement one.

The diagram below is taken from the MQTT-SN specification:

Ref: MQTT-SN Specification -pdf

When client 1 talks to client 2 does it require the MQTT broker to do so? Specifically in the diagram above for clients 3 and 4. Is it

MQTT-SN >MQTT-SN -In this case the gateway is also a MQTT-SN broker.

or is it

MQTT-SN >MQTT>MQTT-SN -In this case the gateway is an MQTT-SN gateway and an MQTT broker.

GateWay Types

The specification defines two gateway types.

A transparent gateway were each MQTT-SN connection has a corresponding MQTT connection. This is the easiest type to implement.

An aggregating gateway were multiple MQTT-SN connections share a single MQTT connection

MQTT-SN Protocol Guide

• MQTT-SN Operation
• MQTT-SN Connections
• Will Topic and Messages
• QOS Levels and QOS -1
• MQTT-SN Topics
• Subscribing to MQTT-SN Topics
• Topic Registration
• Publishing Messages
• Publishing Messages Without a Connection- QOS -1
• Gateway Discovery

MQTT-SN Operation

Although MQTT-SN uses UDP as the transport protocol and not TCP it is designed, as far as possible., to work in the same way as MQTT.

In that regard MQTT-SN usually requires a connection to the broker before it can send and receive messages.

This connection is, in effect, a virtual connection.

However there is a mode of operation that doesn’t require a connection, but this doesn’t work with pure Gateways. (see below).

MQTT-SN Connections

The first message sent from an MQTT-SN client to a broker/gateway is the connection message which is acknowledged by a CONNACK message as per MQTT.

Once a connection has been established the client can subscribe and publish messages.

Just as in MQTT a connection can be persistent or non persistent and this is indicated in the clean session flag.

With a persistent connection (clean session=false) the broker stores connection details of the client.

However in MQTT-SN The Will topic and will message is also included in the clean session.

The connections uses the same ping request and ping response type messages found in MQTT to verify the connection is still present.

Connection Packet

The connection packet is much simpler than the MQTT connection packet as it has no authentication or will message.

The packet is show below:

The client id (up to 23 characters) must be unique as per MQTT and the duration is the keep alive interval which is a 2 byte integer.

The only flags used are the clean session and the will flag.

Will Topic and Messages

With MQTT the will topic and message is sent with the connection message.

However with MQTT-SN the Will topic and Will messages are sent separately.

The client must indicate to the server (using a flag) that it has a will topic and message.

If the will flag isn’t set the server responds with an Connection acknowledgement

However, if the Will flag is set then the server will prompt the client for a will topic and message before sending the Connection acknowledgement message.

The procedure is shown in the diagram below( taken from the official specification).

The following two screen shots show the results of setting the will flag with clean session set to true.

Python client screen

 

RSMB Console

 

Clean Session Flag And Last Will

The following table summarises the Clean Session and Will Connection Settings (taken from the specification):

Clean Session Flag	Will Flag	Result
True	True	The GW (gateway) will delete all subscriptions and Will data related to the client, and starts prompting for new Will topic and Will message.
True	False	The GW will delete all subscriptions and Will data related to the client, and returns CONNACK (no prompting for Will topic and Will message).
False	True	The GW keeps all stored client’s data, but prompts for new Will topic and Will message. The newly received Will data will overwrite the stored Will data
False	False	The GWkeeps all stored client’s data and returns CONNACK (no prompting for Will topic and Will message).

Will Message and Topic Updates

Unlike MQTTv3 the MQTT-SN protocol provides a mechanism for updating the will topic and will message during a connection.

QOS Levels and QOS -1

MQTT-SN supports QOS 0,1,2 as per MQTT, but it also supports a special publish QOS of 3 or -1.

Note: it is known as QOS -1 but the QOS flag in the message is set to 11 or decimal 3.

Publishing messages with a QOS of -1 or 3 doesn’t require an initial connection to have been set up, and requires the use of short topic names or pre-defined topic ids.

MQTT-SN Topics

In MQTT there is only a single topic format. In MQTT-SM this format is known as the long topic name. MQTT-SN has 3 other topic formats and these are described in the Topic names and identifiers tutorial.

Subscribing to MQTT-SN Topics

You can subscribe to a topics using 3 different formats:

• A long topic name as per MQTT e.g. house/sensor1
• A short topic name of 2 characters only e.g. s1
• A pre-defined topic id (integer) e.g. 1

Wildcards can be used as per MQTT, but they only make sense for long topic names.

Note: Predefined topics are defined on the Gateway and client using a list.

Using a Topic Name

As per MQTT 3.1 a topic name is a UTF8-encoded string that can include wild card characters + and #.

When you subscribe to a topic using a topic name the subscribe functions returns a topic ID for this topic name.

The topic ID can then be used to publish messages on this topic.

Using a Topic ID

Although you can use a topic ID to subscribe it is not normally done as you must first find the topic id using the registration process.

Using a Short Topic Name

This is a two character name e.g. “AA” . It is very useful as you can also publish on this topic without requiring registration.

Using a Predefined Topic Name

Topics can be predefined on both the broker and client. Predefined topic names also don’t require topic registration, but they do require a topic list.

For a client to subscribe to a predefined topic -topic1 then this topic must be defined on the broker in a predefined topic list.

Predefined topics aren’t currently supported on RSMB but they are supported on the paho Gateway.

Topic Registration

A client can register a topic with a broker and a broker can also register a topic with a client.

Client Registration

The client registers a long topic name with the broker and the broker returns a topic ID that the client uses to refers to that topic name when it publishes messages.

Broker or Gateway Registration

If a client subscribes to a wildcard topic e.g. house/#

What happens when a broker receives a publish for topic house/bulb2, as the client doesn’t have a topic ID for house/bulb2.

In the case the broker assigns a topic ID and notifies the client using topic registration.

Publishing Messages

Publishing Messages With an Established Connection

You can publish a message using:

• A topic ID
• A short topic name- 2 characters

You can a get a topic ID by either:

• Subscribing to the long topic name.
• Registering the Long topic name.
• Using a pre-defined topic-id

The Subscribe and Register functions both return a topic ID that you use in place of the long topic name when publishing.

Topic ids are assigned to each client, and they are not broker wide. For example:

A client may subscribe to topic house/bulb1 and get a topic ID of 1.

A second client may subscribe to topic house/bulb2 and get a topic ID of 1.

Topic id 1 for client 2 refers to house/bulb2, and for client1 topic id 1 refers to house/bulb1

Note: When using QOS of 1 or 2 you need to wait for a PUBACK message before you publish a new message.

So the format is

1. PUB
2. Wait PUBACK
3. PUB
4. etc

In MQTT_SN you can get a PUBACK even to a QOS level 0 message.

This is because it is used to return a error if the publish was rejected.

Using Long Topic Names

In order to use long topic names you need to register the topic and then publish using the returned topicId.

This is shown clearly in the screen shot below where I use the mqtt-sn pub & sub tools to publish using a long topic name.

Notice that you have a connection topic registration followed by a publish.

Publishing Messages Without a Connection- QOS -1

You can publish messages without first creating a connection by using QOS of -1 or 3.

The reason it is also known as qos3 is because the qos uses 2 bits so

00=qos 0
01=qos 1
10=qos 2
11=qos -1 or 3

When publishing with qos -1 no connection message is sent.

You can publish a message without registering a topic or subscribing to a topic using:

• A pre configured topic ID
• A short topic name – 2 characters.

This is shown below using the MQTT-SN client tools

Published messages aren’t acknowledged.

This mode of publish is ideal for simple sensors.

Gateway Discovery

MQTT-SN clients have the ability to discover brokers/gateways.

There are two mechanisms used:

• Advertising by a broker or Gateway
• A Search by the client

Both methods use a multicast packet.

Currently there is no standardized multicast packet address.

I did come across a proposal on Github. For testing you will have to use a known free multicast address.

How Gateway Discovery Works

The broker of gateway advertises on a multicast address.

In the packet is the name of the advertising Gateway the advertising duration and the Gateway number.

The client maintains a list of active gateways and can use the duration to determine if a gateway is still active.

The client must be listening on the multicast address.

Alternatively the client can search for a gateway by sending a search packet on a multicast address.

This can be answered by either another client or a Gateway.

However for some reason the Gateway response doesn’t contain the Gateway address, but a response from another client does.

You can find more details in the MQTT-SN Gateway Advertisement and Discovery tutorial.

MQTT-SN Brokers,Gateways,Tools and Clients

Brokers and Gateways

To test MQTT-SN you will need a broker. There currently aren’t many MQTT-SN brokers available.

The RSMB broker developed by Ian Craggs of IBM was the first and was the basis for the Mosquitto MQTT broker.

This broker hasn’t been actively developed for many years but I’m hopeful it will be soon.

At the moment predefined topics and sleeping clients aren’t implemented.

The broker functions as a MQTT-SN broker and MQTT-SN to MQTT Gateway.

Here are the download  install and configuration instructions.

Paho Eclipse Gateway

This started I believe as a fork of RSMB and works as a pure Gateway.

This git page has the download and install instructions.

Note:

your might need to use:

git clone  https://github.com/eclipse/paho.mqtt-sn.embedded-c

and not

git clone -b experiment https://github.com/eclipse/paho.mqtt-sn.embedded-c

MQTT-SN Python Client

The RSMB src files also have a Python client. The client is written in python2.7

I have ported it to python3 and have modified it to work similar to the Paho  MQTT client. The files have lots of print statements that I have used to try and understand how it works.

Here are my current usage notes

C Client

Because MQTT-SN is for low level hardware this will be the most popular client used.

You can download this client here.

MQTT-SN Pub and Sub Tools

These are the MQTT-SN equivalent to the mosquitto_pub and sub tools.

Download here

• MQTT-SN Publish and Subscribe Test Tools

News and Views (Dec 2019)

MQTT-SN has stalled in development and there aren’t the variety of brokers and clients that you have with MQTT.

However having said that it is likely to become more popular and there is a move now to have it developed as an OASIS standard.
https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt-sn

Personally I think MQTT-SN will eventually be the main protocol for connecting sensors to MQTT networks and MQTT will become more of a backbone protocol.

Video-MQTT-SN Overview Video for Beginners

Useful Links

• Understanding IP multicasting
• Aquila 2 – MQTT-SN bridge hackaday project
• MQTT-SN Specification -pdf
• Eclipse MQTT-SN Gateway

Related Tutorials

• Using The Paho MQTT-SN Gateway
• MQTT-SN QOS 3 to MQTT Gateway Using Node-Red
• MQTT-SN Publish and Subscribe Test Tools
• Publish and Subscribe Using the Python MQTT-SN Client
• MQTT-SN Gateway Advertisement and Discovery
• MQTT-SN Topic Names and Identifiers
• MQTT-SN RSMB Broker Overview,Install and Configuration
• Using The Python MQTT-SN Client