MQTT Topic and Payload Design Notes

Designing a topic naming and payload scheme will be an important part of any MQTT deployment.

In this tutorial we look at possible design schemes for MQTT networks.



MQTT Network Devices

An MQTT network will consist of devices like sensors that can.

  1. Send unsolicited data
  2. Send requested data
  3. Receive command or control data

So if we take a simple light switch as an example device then at a minimum it needs to:

  1. Publish it’s current state i.e is it on or off.
  2. Respond to a command to change state.

To accomplish this we can use the topic name and the message payload.

The topic name can include not only the topic name, but also information describing the message payload. e.g.

we could use a topic name of:

  • sensor1 – Topic has sensor name only
  • or sensor1/status -Topic has sensor name and information about the type of data e.g status information.

So a simple naming scheme for our light switch would be:

The light switch publishes status information on topic

light-switch/status or just light-switch

and receives commands on

light-switch/set

The message payload would be a simple value i.e. ON or OFF.

However we could adopt a different approach and place extra information in the message payload.

So our light switch could publish on the topic:

light-switch

and use a JSON encoded message payload like

{“status”:”ON”} or {“status”:”OFF”}  – Data

{“set”:”ON”} or {“set”:”OFF”}        – –Commands

The important thing to note is that there is always a choice of putting information in the topic field or in the message payload.

Example 1 -Sensors in a Building

If we imagine a single building with say 10 devices e.g Temperature and humidity sensors. Light switches, door locks etc.

We could adopt a topic naming scheme where each sensor publishes on its own topic like:

house/sensor1, house/sensor2 ……….   house/sensor10

If the sensor only published a single value e.g ON,OFF,Open Closed,Temperature etc then our payload could just contain that value.

Thus our living room light switch would publish.

on topichouse/living-room-light

With message payload of – ON or OFF

We could also choose to publish on a single topic and include the sensor ID in the payload as follows:

on topichouse

With message payload Json encoded – {“living-room-light”:”ON”} or {“living-room-light”:”OFF”}

Now how do we control our light switch? The light switch would need to receive commands.

So we need to create a topic to receive commands.

We could use a topic structure of:

  • house/living-room-light/cmd
  • house/living-room-light/cmd

with the message payload of  – {“SET”:”ON”} or {“SET”:”OFF”}

or

  • house/living-room-light/Set
  • house/living-room-light/Set

with the message payload of ON or OFF

Another alternative is use the payload for the device and also to encode the command:

topic – house and

Payload of – {“living-room-light”:{“SET”:”ON”}} or  {“living-room-light”:{“SET”:”OFF”}}

Approaches Creating a Topic Naming Scheme

When creating the topic naming scheme the approaches are:

  1. Put as much information as possible in the topic field or
  2. Put as much information as possible in the message payload

A topic hierarchy can consists of:

  1. High level topic grouping devices.- optional
  2. Assigned Sensor name – optional
  3. function e.g. status,set,get,cmd –optional

The message payload can consist of

  • Payload data
  • Sensor ID- optional
  • Function – optional

Firstly it is important to note that there currently is no standard topic naming scheme or message format.

Here is a very useful proposal on Github which I would recommend you read.

There is also no right or wrong way.

However you should be very aware that you have far more control when publishing to a topic

Let’s take an example scenario described above and examine two schemes in more detail for our 10 devices/sensors.

Using the sensor name in the topic. i.e

We use the topic scheme house/sensor01……. house/sensor10 and only send data in the payload.

If the broker receives a message published to house/sensor01 it send it only the devices subscribed on house/sensor01. This would be sensor01.

However using the topic naming scheme were we group all 10 sensors under the topic house. and use the payload to determining the sensors e.g.

Payload of {“sensor01″:”on”}

In this scenario all sensors/devices would need to subscribe to the house topic.

A message to sensor01 would be sent to all clients subscribed to the house topic which means all 10 sensors. So now we have 10 messages being sent.

Each sensor would need to examine the message to see if is for itself.

Therefore my approach is to

  • Use a topic name for an individual device or small group of devices/sensors.
  • Use a separate topic name for data and commands.
  • Data in the message payload should be device specific were possible.
  • Data in the message payload that relates to multiple attributes of the device is JSON encoded. See packaging data below.

Packaging Topic data

In the case of a complex machine the choice will need to be made whether to publish on multiple topics or use a single topic with packaged JSON data.

Consider this topic tree which I found on a public broker

Test/AppSpin/read false
Test/Position3/read false
Test/TankEmpty/read true
Test/AppOpen/read false
Test/Position7/read false

It could easily be replaced by publishing on a single topic (test) and using a dictionary for the data as follows

{AppSpin_read:False,Position3/read:False,TankEmpty_read:True,AppOpen_read:False, Position7_read:False}

The data is easily read on the other side and it involves 1 publish and not multiple publishes.

I did some tests on network traffic based on packaged data see MQTT Sensors and Network Traffic Observations.

You should note that current IOT/MQTT dashboards take data normally as JSON encoded data, and it is likely to become standard.

See video Publishing and Receiving JSON data with Python.

Message Encryption

An advantage of using the payload and a single topic is that the payload data can be easily encrypted which would protect the topic information as it wouldn’t be public. See Encrypting message payloads

The Homie Convention

This is a Github proposal that attempts to standardise how MQTT devices make themselves discoverable on an MQTT network.

It contains a detailed description the of the data that a device should publish when it connects to an MQTT broker.

This document should serve as a good guideline for you design.

Public Topics

Currently MQTT implementations are mainly private and the topic naming scheme is chosen by the implementation team.

In the future MQTT is likely to be deployed in a public environment e.g for:

  • Publishing traffic information and updates
  • Arrivals and departure information at airports,train stations etc
  • Beacons in shops
  • etc

In this case there will need to be some kind on naming convention to make it easy for people to locate the service and subscribe.

As far as I am aware there is currently no proposal for this.

Naming Scheme Examples:

Owntracks is a mobile phone app that reports geographic location the topic hierarchy design is here and the message format design is here

If you have any examples of naming schemes on actual MQTT deployments I would be grateful if you would share them in the comments.

Resources

Related Tutorials:

Facebooktwittergoogle_plusredditpinterestlinkedinmail

5 comments

  1. Steve, a helpful guide to stetting topics /payloads. Regarding the observations I guess the real benefit of using JSON must lie in the reliability of encoding / decoding the packed JSON payloads. A single topic/payload pair should be by definition more fool proof than a multiple packaged payload. The Chatty versus non chatty is very interesting.
    My 2 cents anyway.

    1. Glad You found it helpful. Yes Using JSON makes it very easy to pull the data into a program. You can also encrypt it easily as it is in the payload which might be useful.
      I think there will be a lot more discussion on this going forward and especially public topics.

Leave a Reply

Your email address will not be published. Required fields are marked *