MQTT is primarily a M2M protocol. It was originally designed for send sensor data from a remote oil field.
As the number of sensors increases the amount of network traffic generated by sensors will increase dramatically and depending on the transport it could prove expensive.
It will be important to understand this sensor traffic and how it can be reduced.
In this tutorial I will be looking at sensor data and sensor types under various conditions, and I will analyze the amount of network data that is generated.
This you may find useful when planning a sensor deployment.
Sensor Basics and Types and Characteristics
Although there will be very many sensors recording lots of different types of data. e.g we could have pressure sensors,temperature sensors, door sensors etc.
Generally they fall into two basic types:
Binary sensors e.g. open, closed or on and off
Multi value type sensors that can have multiple values e.g a temperature sensor.
Sensors will report changes on a predetermined schedule.
For example a temperature sensor could report the temperature every second,every 10 seconds every minute etc.
How often the sensor will need to report its data will depend very much on the application.
In addition the sensor data may change or may remain the same. For example:
- if you were measuring the temperature of a room it might not change at all for several hours.
- If you were monitoring a door (open/close) it might remain closed all day. However it might be important that you knew within seconds if the door status has changed.
With this in mind the two important things to consider are.
- The sensor scanning interval
- The need to report all data or just changes
Examples:
- A security sensor on a door/window etc would have a sensor scanning interval of perhaps every second but only need to report changes.
- A sensor monitoring pressure that is constantly changing would have a sensor scanning interval of perhaps every second and report continuously.
I call these sensors “not chatty” or “chatty” .
Sensor Data Analysis Example
I’ve created a simple python script that simulates multiple simple on/off type sensors and I measured the traffic generated under the following conditions:
- The sensors are chatty
- The sensors are not chatty
- The sensor data is published on multiple topics.
- The sensor data is combined on a single topic using JSON data
Test 1
Below is a screen shot example of a Python script output showing a simple single sensor sending an on/off message.
In one run I configured the sensor to be chatty, and then in the second run to only send changes (non chatty).
I’ve configured the script to count the bytes sent on the network.
Notice that the chatty sensor send over 43KBytes of data in 30 minutes whereas the non-chatty sensor only sends 142 bytes.
Combining Sensor Channels
If you have for example several sensors e.g.20 then do you publish sensor data for each sensor individually or do you combine them.
So for example we could publish sensor data for each sensor on their own topic.
i.e.
sensor1 would publish on topic-base/sensor1 and sensor2 would publish on topic-base/sensor2
or
do we combine the sensors and publish on a singe topic e.g. topic-base/sensors.
In this case we would need to analyze the data to separate the data for each sensor.
Example publishing a simple on message for five sensors:
Method 1
topic-base/sensor1 ON
topic-base/sensor2 ON
topic-base/sensor3 ON
topic-base/sensor4 ON
topic-base/sensor5 ON
Method 2
topic Data
topic-base/sensors ON,ON,ON,ON,ON
The preferred method of combining data would be to use a JSON encoded object. So instead of the data looking like this:
ON,ON,ON,ON,ON it would look more like this:
In which case the size of each data packet would now be increased considerably.
The screenshots below show the results of sending data on multiple sensors. Again we look at chatty and non-chatty.
The screen shot below shows the same tests but with the sensors configured as non-Chatty
Observation Notes:
- It is much more efficient to only send changes rather than send data at regular intervals.
- Combining sensor data doesn’t reduce network traffic but can slightly increase it. However current IOT/MQTT dashboards take data normally as JSON encoded data, and it is likely to become standard.
Important Note:
When running a sensor in non-chatty mode it is important to set the retain message flag so that a new subscriber receives the last sensor reading. See retained messages.
In addition if you want to know if the sensor is active even though it is not publishing data. See Checking Active MQTT Client Connections
Related Tutorials and Resources:
- Beginners Guide to MQTT
- Understanding MQTT Topic Naming and Design Notes
- Sending and receiving JSON data over Python
- Logging Sensor Data
- MQTT Connections – Should You Stay Connected?