Introduction to MQTT-SN (MQTT for Sensor Networks)

Due to the large number of sensors in IOT Sensor networks these sensors will be mainly wireless.

The main characteristics of these networks that drove the design are:

  • Low Power battery operated sensors with very limited processing power and storage.
  • Limited payload size
  • Not always on (sleeping)

MQTT-SN (MQTT for Sensor networks) was designed specifically to work on wireless networks, and , as far as possible, to work in the same way as MQTT.

It uses the same publish/subscribe model and can be considered as a version of MQTT.

MQTT-SN vs MQTT

The main differences involve:

  • Reducing the size of the message payload
  • Removing the need for a permanent connection by using UDP as the transport protocol.

The MQTT-SN specification lists these differences.

  1. Connect message split into three messages two are optional and are used for the will message
  2. Topic id’s used in place of topic names.
  3. Short Topic names
  4. Pre-defined topics.
  5. Discovery process to let clients discover the Gateway
  6. Will Topic and messages can be changed during the session
  7. Off line keep alive procedure for sleeping clients.

Architecture and Components

The specification lists three components:

  1. MQTT-SN client
  2. MQTT-SN Gateway
  3. MQTT-SN forwarder.

What seems to be missing is a broker/server as in the MQTT sense.

However the architectural diagram in the specification does appear to show one and the RSMB server does implement one.

The diagram below is taken from the MQTT-SN specification:

MQTT-SN architecture-diagram
Ref: MQTT-SN Specification -pdf

When client 1 talks to client 2 does it require the MQTT broker to do so? Specifically in the diagram above for clients 3 and 4. Is it

MQTT-SN >MQTT-SN -In this case the gateway is also a MQTT-SN broker.

or is it

MQTT-SN >MQTT>MQTT-SN -In this case the gateway is an MQTT-SN gateway and an MQTT broker.

GateWay Types

The specification defines two gateway types.

A transparent gateway were each MQTT-SN connection has a corresponding MQTT connection. This is the easiest type to implement.

An aggregating gateway were multiple MQTT-SN connections share a single MQTT connection

MQTT-SN-GateWay-Types
MQTT-SN Protocol Guide

MQTT-SN Operation

Although MQTT-SN uses UDP as the transport protocol and not TCP it is designed, as far as possible., to work in the same way as MQTT.

In that regard MQTT-SN usually requires a connection to the broker before it can send and receive messages.

This connection is, in effect, a virtual connection.

However there is a mode of operation that doesn’t require a connection, but this doesn’t work with pure Gateways. (see below).

MQTT-SN Connections

The first message sent from an MQTT-SN client to a broker/gateway is the connection message which is acknowledged by a CONNACK message as per MQTT.

Once a connection has been established the client can subscribe and publish messages.

Just as in MQTT a connection can be persistent or non persistent and this is indicated in the clean session flag.

With a persistent connection (clean session=false) the broker stores connection details of the client.

However in MQTT-SN The Will topic and will message is also included in the clean session.

The connections uses the same ping request and ping response type messages found in MQTT to verify the connection is still present.

Connection Packet

The connection packet is much simpler than the MQTT connection packet as it has no authentication or will message.

The packet is show below:

connect-message-mqtt-sn

The client id (up to 23 characters) must be unique as per MQTT and the duration is the keep alive interval which is a 2 byte integer.

The only flags used are the clean session and the will flag.

Will Topic and Messages

With MQTT the will topic and message is sent with the connection message.

However with MQTT-SN the Will topic and Will messages are sent separately.

The client must indicate to the server (using a flag) that it has a will topic and message.

If the will flag isn’t set the server responds with an Connection acknowledgement

However, if the Will flag is set then the server will prompt the client for a will topic and message before sending the Connection acknowledgement message.

The procedure is shown in the diagram below( taken from the official specification).

Will-flag-diagram-mqtt-sn

The following two screen shots show the results of setting the will flag with clean session set to true.

Python client screen

will-request-mqtt-sn

 

RSMB Console

 

will-request-mqtt-sn-rsmb

Clean Session Flag And Last Will

The following table summarises the Clean Session and Will Connection Settings (taken from the specification):

Clean Session Flag Will Flag Result
True True The GW (gateway) will delete all subscriptions and Will data related to the client,
and starts prompting for new Will topic and Will message.
True False The GW will delete all subscriptions and Will data related to the client,
and returns CONNACK (no prompting for Will topic and Will message).
False True The GW keeps all stored client’s data, but prompts for new Will topic and
Will message. The newly received Will data will overwrite the stored Will data
False False The GWkeeps all stored client’s data and returns CONNACK (no prompting
for Will topic and Will message).

Will Message and Topic Updates

Unlike MQTTv3 the MQTT-SN protocol provides a mechanism for updating the will topic and will message during a connection.

QOS Levels and QOS -1

MQTT-SN supports QOS 0,1,2 as per MQTT, but it also supports a special publish QOS of 3 or -1.

Note: it is known as QOS -1 but the QOS flag in the message is set to 11 or decimal 3.

Publishing messages with a QOS of -1 or 3 doesn’t require an initial connection to have been set up, and requires the use of short topic names or pre-defined topic ids.

MQTT-SN Topics

In MQTT there is only a single topic format. In MQTT-SM this format is known as the long topic name. MQTT-SN has 3 other topic formats and these are described in the Topic names and identifiers tutorial.

Subscribing to MQTT-SN Topics

You can subscribe to a topics using 3 different formats:

  • A long topic name as per MQTT e.g. house/sensor1
  • A short topic name of 2 characters only e.g. s1
  • A pre-defined topic id (integer) e.g. 1

Wildcards can be used as per MQTT, but they only make sense for long topic names.

Note: Predefined topics are defined on the Gateway and client using a list.

Using a Topic Name

As per MQTT 3.1 a topic name is a UTF8-encoded string that can include wild card characters + and #.

When you subscribe to a topic using a topic name the subscribe functions returns a topic ID for this topic name.

The topic ID can then be used to publish messages on this topic.

Using a Topic ID

Although you can use a topic ID to subscribe it is not normally done as you must first find the topic id using the registration process.

Using a Short Topic Name

This is a two character name e.g. “AA” . It is very useful as you can also publish on this topic without requiring registration.

Using a Predefined Topic Name

Topics can be predefined on both the broker and client. Predefined topic names also don’t require topic registration, but they do require a topic list.

For a client to subscribe to a predefined topic -topic1 then this topic must be defined on the broker in a predefined topic list.

Predefined topics aren’t currently supported on RSMB but they are supported on the paho Gateway.

Topic Registration

A client can register a topic with a broker and a broker can also register a topic with a client.

Client Registration

The client registers a long topic name with the broker and the broker returns a topic ID that the client uses to refers to that topic name when it publishes messages.

Broker or Gateway Registration

If a client subscribes to a wildcard topic e.g. house/#

What happens when a broker receives a publish for topic house/bulb2, as the client doesn’t have a topic ID for house/bulb2.

In the case the broker assigns a topic ID and notifies the client using topic registration.

Publishing Messages

Publishing Messages With an Established Connection

You can publish a message using:

  • A topic ID
  • A short topic name- 2 characters

You can a get a topic ID by either:

  • Subscribing to the long topic name.
  • Registering the Long topic name.
  • Using a pre-defined topic-id

The Subscribe and Register functions both return a topic ID that you use in place of the long topic name when publishing.

Topic ids are assigned to each client, and they are not broker wide. For example:

A client may subscribe to topic house/bulb1 and get a topic ID of 1.

A second client may subscribe to topic house/bulb2 and get a topic ID of 1.

Topic id 1 for client 2 refers to house/bulb2, and for client1 topic id 1 refers to house/bulb1

Note: When using QOS of 1 or 2 you need to wait for a PUBACK message before you publish a new message.

So the format is

  1. PUB
  2. Wait PUBACK
  3. PUB
  4. etc

In MQTT_SN you can get a PUBACK even to a QOS level 0 message.

This is because it is used to return a error if the publish was rejected.

Using Long Topic Names

In order to use long topic names you need to register the topic and then publish using the returned topicId.

This is shown clearly in the screen shot below where I use the mqtt-sn pub & sub tools to publish using a long topic name.

mqtt-sn-publish

Notice that you have a connection topic registration followed by a publish.

Publishing Messages Without a Connection- QOS -1

You can publish messages without first creating a connection by using QOS of -1 or 3.

The reason it is also known as qos3 is because the qos uses 2 bits so

00=qos 0
01=qos 1
10=qos 2
11=qos -1 or 3

When publishing with qos -1 no connection message is sent.

You can publish a message without registering a topic or subscribing to a topic using:

  • A pre configured topic ID
  • A short topic name – 2 characters.

This is shown below using the MQTT-SN client tools

mqtt-sn-qos-1-publish

Published messages aren’t acknowledged.

This mode of publish is ideal for simple sensors.

Gateway Discovery

MQTT-SN clients have the ability to discover brokers/gateways.

There are two mechanisms used:

  • Advertising by a broker or Gateway
  • A Search by the client

Both methods use a multicast packet.

Currently there is no standardized multicast packet address.

I did come across a proposal on Github. For testing you will have to use a known free multicast address.

How Gateway Discovery Works

The broker of gateway advertises on a multicast address.

In the packet is the name of the advertising Gateway the advertising duration and the Gateway number.

The client maintains a list of active gateways and can use the duration to determine if a gateway is still active.

The client must be listening on the multicast address.

Alternatively the client can search for a gateway by sending a search packet on a multicast address.

This can be answered by either another client or a Gateway.

However for some reason the Gateway response doesn’t contain the Gateway address, but a response from another client does.

You can find more details in the MQTT-SN Gateway Advertisement and Discovery tutorial.

MQTT-SN Brokers,Gateways,Tools and Clients

Brokers and Gateways

To test MQTT-SN you will need a broker. There currently aren’t many MQTT-SN brokers available.

The RSMB broker developed by Ian Craggs of IBM was the first and was the basis for the Mosquitto MQTT broker.

This broker hasn’t been actively developed for many years but I’m hopeful it will be soon.

At the moment predefined topics and sleeping clients aren’t implemented.

The broker functions as a MQTT-SN broker and MQTT-SN to MQTT Gateway.

Here are the download  install and configuration instructions.

Paho Eclipse Gateway

This started I believe as a fork of RSMB and works as a pure Gateway.

This git page has the download and install instructions.

Note:

your might need to use:

git clone  https://github.com/eclipse/paho.mqtt-sn.embedded-c

and not

git clone -b experiment https://github.com/eclipse/paho.mqtt-sn.embedded-c

MQTT-SN Python Client

The RSMB src files also have a Python client. The client is written in python2.7

MQTT-SN-python-client-files

I have ported it to python3 and have modified it to work similar to the Paho  MQTT client. The files have lots of print statements that I have used to try and understand how it works.

Here are my current usage notes

C Client

Because MQTT-SN is for low level hardware this will be the most popular client used.

You can download this client here.

MQTT-SN Pub and Sub Tools

These are the MQTT-SN equivalent to the mosquitto_pub and sub tools.

Download here

News and Views (Dec 2019)

MQTT-SN has stalled in development and there aren’t the variety of brokers and clients that you have with MQTT.

However having said that it is likely to become more popular and there is a move now to have it developed as an OASIS standard.
https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt-sn

Personally I think MQTT-SN will eventually be the main protocol for connecting sensors to MQTT networks and MQTT will become more of a backbone protocol.

Video-MQTT-SN Overview Video for Beginners

Useful Links

Related Tutorials

 

Please rate? And use Comments to let me know more

36 comments

    1. I haven’t tested but It is something I will do and report on. However I did read that there is little difference between them provided there is an MQTT connection in place. If there isn’t and one needs to be set up then MQTT would be much slower.
      Rgds
      Steve

      1. Thanks for the reply. If there is difference between them, what is based on?
        Is it the difference on transport protocol, or something else?

        1. The only thing that comes to min is that TCP needs acknowledgement messages whereas a UDP message doesn’t but I’m yet to actually test it.
          Rgds
          Steve

          1. sorry don’t understand the question.
            As a matter of Interest are you evaluating MQTT-SN for a project or just out of interest?
            Rgds
            Steve

    1. Yes but you need an MQTT-SN broker or gateway in the cloud to connect it to.. Does that make sense?
      rgds
      steve

      1. If we need to compare several protocols (HTTP, MQTT, MQTT-SN) for data delay, transmitted to cloud, we need to find cloud, that have such Gateway ?

  1. Hi, how the GATEWAY convert the MQTT-SN message into MQTT?
    And after this conversion the message are MQTT but carryout via UDP ?

  2. HI Steve, Thanks for all the information and knowledge sharing that you have been providing.

    Please correct me on my understanding below:
    1. For all wireless related communication technologies (like WiFi), choosing MQTT-SN can be the best suitable option and For all TCP-IP related communication technologies choosing MQTT is the best suitable option?
    2. Does MQTT also support wireless related communication technologies (like WiFi)?

    Thanks in advance

    1. Hi
      Both protocols work over Wi-Fi and Ethernet.MQTT-SN is a better choice for low power sensors that may sleep and wake up to transit data.
      MQTT-SN works over UDP which is connectionless protocol and lighter than TCP.
      MQTT-SN will also work over serial lines and ZigBee but I have never used it on either.
      In essence I would say that if you are sending data more or less continuously then MQTT is the better option.If you are sending data occasionally then MQTT-SN would be the better choice.
      However MQTT-SN is currently not well supported and so MQTT is almost always used.
      Hope that helps
      Rgds
      Steve

  3. Hi,
    I have a query, does anyone knows any consumer end-products that use MQTT-SN to communicate?

    I am looking for some reverse engineering.

    Thanks and Regards,
    Hemant

  4. Hi, thanks for your post on MQTT-SN. Please I wanted to clarify. Does MQTT-SN have the ability to find devices without registering?

  5. Hello I am using the eclipse paho gateway and I am having problems with the “Callback” class on my MQTT-SN client.
    Publish can publish something, but Subscribe doesn’t get it. Is this problem common for this gateway?

    1. Hi
      I haven’t used it but I think the gateway is just a gateway and not a broker like the RSMB. So you would need to send from an MQTT client which sends it to the gateway and to your client. Have your tried the RSMB to make sure.
      If you don’t get it to work send me your script.
      Rgds
      Steve

      1. I actually tried RSMB last year and I couldn’t even install it!
        I installed the mosquitto broker on my ubuntu and I’m using the MQTT-SN gateway as an intermediary between MQTT and MQTT-SN. My project should only use MQTT-SN clients.

          1. Yes, I want to please, did you use RSMB?
            will it run with the Eclipse Paho Gateway?

          2. I will email it you. You don’t need the eclipse gateway as it is a broker and gateway
            Rgds
            Steve

    1. Currently I am unaware of a mqtt-sn node and so to use it you would need to create your own. It is something I’ve had on my list for a while but haven’t time to pursue it further.
      Rgds
      Steve

Leave a Reply

Your email address will not be published. Required fields are marked *