MQTT v 5.0 New Features Overview

mqttv5MQTT v5 introduced many new features and changed how some existing features work.

In this tutorial I will be covering all of these features with brief examples, and also in many case, links to detailed examples and videos.

Note: Examples used in the tutorial were created using the Mosquitto broker v1.6.2 and the Python client available on Github here.

I’ve divided the features into sections to make it easier to determine where they apply.

Connection Features

Publish Features

Subscribe Features

General

Video Overview

Connection Features

Clean Sessions Or Clean Start

This has been renamed to clean start and works basically the same as before except that we now have a session expiry timer.

This tutorial covers clean sessions in MQTT3.11 and is still applicable to MQTTv5.

However in MQTTv5 if you set the clean start to false you need to provide a session expiry value. If you don’t then it is set to 0 which makes it basically the same as clean sessions True.

If you set the expiry time to 300 seconds.

This means that if a client disconnects and reconnects within  5 minutes with clean start= False,QOS>1 then session state data ( e.g subscribed topics, queued messages) are retained.

However if the client disconnects and reconnects after 5 minutes with clean start= False,QOS>1 then session state data ( e.g subscribed topics, queued messages) are lost, and the client must resubscribe to topics, and messages sent while the client was disconnected are lost.

See the tutorial with demo code on Clean start and session expiry for more details

Why Use it? – It frees up server space as the server can clear all stored data for that client.

Client Limitations/Restrictions

A client can now indicate to a server various client restrictions. It can tell the server to:

  • Restrict size of messages to send to the client
  • Restrict the number of QOS 1 and 2 messages

Message Size Restrictions

The client tells the server the maximum message size it is prepared to receive and any messages larger than this are dropped by the server.

Uses: Used by low powered clients that lack the memory to store the messages. See MQTT and Mosquitto Message size restrictions

Receive Maximum

The client tells the server to limit the number of QOS 1 and 2 messages that can be sent concurrently.

Uses:  Again used by low powered clients as QOS 1 and 2 messages mean that the client needs to send additional acknowledgement messages.

Server Limitations/Restrictions

Providing certain functionality can strain server resources and so the server may not support them.

It is now possible for the server to indicate this is the message exchange using the properties field.

Example: The screen shot below shows the CONNACK message properties field From the Mosquitto Server.

connection-propeties

There are many other limitations that the server can indicate.

By default if the server doesn’t indicate that it doesn’t support a particular feature then it is supported.

For example it is possible for the server not to support retain messages using the Retain Available property, but in the screen shot above it is not there, and so the server supports retain messages.

Will Delay Interval

This is used to specify a time delay in sending the will messages.

This prevents the will message being sent for a short connection interruption.

User Properties (Connection)

User properties are defined by the user and are a collection of key value pairs.

They can be used to send connection related properties from the Client to the Server.

They are not forwarded to the destination client. See user properties on publish below. and understanding MQTTv5 User Properties

Server Reference or Server Redirect

This is a very interesting feature as it allows the server to redirect the client to another broker/server. From the specification

The Server uses a Server Reference in either a CONNACK or DISCONNECT packet with Reason code of 0x9C (Use another server) or Reason Code 0x9D (Server moved) as described in section 4.13.

Publish

Message Expiry

When a message is published with a QOS of 1 or 2 and the receiving client has:

  1. Connected with clean session of False
  2. Subscribed with a QOS or 1 or 2

Then the broker will hold published messages for the client if it is currently disconnected.

On MQTTv3 this was indefinite, but on v5 you can set the period that the server will hold the message before discarding.

This is done in the publish message below is a sample Python code snippet:

client_pub.publish('test', msg_out1,qos=1,message_expiry_interval=20)

With the above setting then if the client doesn’t reconnect within 20 seconds the message is discarded.

Payload Format Indicator

Set on the Publish Message. There are only two possibilities.

  • Binary – this is the same as v3.1
  • utf-8 – new to v5

Uses: Useful at the receiver as is knows if it needs to decode the message payload.

User Properties

These are defined by the user and are a collection of key value pairs.

In Javascript =object

Python = dictionary

Example:

{msg_number:1,time:100}

keys=msg_number and time

values= 1 and 100

These are useful for sending information to the destination outside of the payload.

In v3.1.1 this information was usually sent as JSON encoded data as part of the payload.

So in v3.1.1 the payload would be:

{msg_number:1,time:100,payload:payload_data}

This isn’t a problem if the payload is text data but if the payload is binary data then it will need to be converted using Base64 encoding.

User properties can be set in all packet types:

Publish user properties are sent to the destination.

Topic Aliases

Topic aliases were introduced in MQTT-SN and are mechanism for reducing the size of published packets by reducing the size of the topic field.

A topic alias once established can be used in place of a topic string.

So if the topic houses/house1/upstairs/light1 had an alias of 1 then this could be used in place of the topic string when publishing messages.

Before you can use topic aliases the client must indicate the topic alias maximum number to the server.

By default if this isn’t set then it is 0 which means topic aliases aren’t allowed by the client.

See MQTTv5 Topic aliases for more details.

Response Topic and Request /Response Pattern

In Web Applications the client(browser) makes a request and the server responds.

MQTT uses a publish and subscribe pattern where there is no direct communication between the sending client and the destination client/server.

Using the response topic in the publish message allows you to implement the request/response pattern that is common in Web applications.

Tutorial- Understanding And Using MQTT v5 Request Response

Subscribe

Non Local Publishing

In MQTTv3.1.1 if you subscribe to the same topic as you publish on then you will receive all of the messages that you publish.

By Using the non local option the broker will not send you any messages that you have published.

This is a subscription option.

Python Example:

client.subscribe('org/common',no_local=True)

Retained Message Control

Retained messages still work as in 3.1.1 but subscribe options have been added to control what the client receives.:

0  -Send retained messages when client subscribes. Same as in MQTT 3.1.1
1   -Send retained messages when client subscribes. if the subscription does not currently exist.
2 – Don’t send retained messages when client subscribes.

Uses: Likely to help in implementing MQTT bridges.

Subscription Identifier

This is optional and if used it is an integer associated with the subscription and used to filter incoming messages to correct message handler.

However this could also be done by analysing the topic itself as in MQTTv3.1.1

Python Example:

client.subscribe('test', qos=0, subscription_identifier=1)
client.subscribe('test/#', qos=0, subscription_identifier=2)

Shared Subscriptions

The idea is to provide for client load balancing.

It is accomplished by using the reserved topic $SHARE

The format is:

$SHARE/Sharename/topic

The sharename cannot be a wildcard character and must be at least 1 character.

See MQTTv5 Shared Subscriptions for more details

General

Reason Codes on All ACK Messages

In MQTTv3.1.1 there were very few indications from the server as to what went wrong during the various stages of :

  • Establishing a connection
  • Publishing a Message
  • Subscribing to topics

All Acknowledgement packets (except PINGRESP)  carry a reason code and there are 128 codes available.

Uses and example : Makes it much easier for an application to understand what has gone wrong and take the appropriate action.

A publish ACK with reason code 16 means that there are no matching subscribers. Therefore in MQTTv5 it is possible to detect if a topic is in use to some degree.

Server Disconnect

In MQTTv3.1.1 only the client could send the disconnect message.

If the server encountered problems it would simply terminate the TCP/IP session.

In MQTTv5 the server can send the client a disconnect message along with a reason code.

The screen shot below is taken from the MQTTv5 specification.

disconnect-reason=codes

Resources:

Related Tutorials

 

Please rate? And use Comments to let me know more

14 comments

  1. Hi Steve,
    Thanks for the wonderful sessions.
    Can I request a On-Demand Read (requesting read at a particular time) from an MQTT Client (say a energy monitoring MQTT supported device) client? May I know which section specifies this in the MQTT specification document (any version that supports)?

  2. Hi Steve,
    Thank you for all of your webpages. They’re very informative. Do you have any tutorial for Sparkplug B? What is the difference between Sparkplug B and Mqtt? Do I need special software to do Sparkplug B?

    Thank you again in advanced.

  3. Hi Steve,
    Thanks for wonderful sessions, we are trying to save battery power for IOT device in sending data to GIOT server, the data is sent once in a hour, so does it make sense to use Keep_alive ? Or just start connect session and transfer data only during that 1 or 2 minute, ?
    Staying connected will consume power OR reconnecting only when required is recomended, we are using MQTT v3.11

  4. In addition,
    So my target board that I am working on is wired ethernet connected to my router.
    I am trying to connect to my wireless laptop with mosquitto installed.
    So I am debugging through my laptop as well with atollic ide.

    So when i am stepping through the code, after stepping over the connect() function – the next code suppose to publish to the mosquitto server… but there is a bad status returned by it…

    Thanks again,

    neiz

  5. Hi,

    When using MQTTv5 Java client library, do you know how to set the version? I need to set the version to 3.1.1 (aka 4) but setMqttVersion of MqttConnectOptions was not ported to MqttConnectionOptions class.

    Thanks
    Roger

  6. hi Steve, thank you as always, another useful and informative topic.
    can you tell me please does mosquitto 5 have or are there any plans you know of to build in a beacon function.
    At the moment beacon is sent from my local web pages, it would be really nice if mosquitto could be configured to do it
    cheers
    Dave

      1. hi again Steve
        its just a way of sending the local time to a node, similar to getting time via UDP, it was just a thought that if Mosquitto could do it then things like lights switching on / off at a given time could in fact be working in real time rather than a time on the node which may not necessarily be real time
        cheers
        Dave

        1. Dave
          See what you mean. Mosquitto doesn’t send anything itself (apart from pings and will messages) it just moves messages between publishers and subscribers.
          Rgds
          Steve

Leave a Reply

Your email address will not be published. Required fields are marked *