It is possible to send any type of data using MQTT including files,images and video.
However MQTT wasn’t designed for file sending but for sending small data packets from remote sensors.
Having said that the actual maximum message size is very large (256MB).
However not all MQTT brokers will support the maximum packet size and all public brokers have message size restrictions often between 128KB and 256kB.
In addition clients will also have message size restrictions.
Therefore a common practise is to split large files into chunks for sending.
File Data and Meta Data
Generally we will want to send the actual file data along with the file name.
When reading a file from disk a text file can be converted to text on reading or read directly as bytes.
For image/video files then they will be read as bytes.
To avoid complications I prefer to read all files as binary data so that they can be place directly in the message payload as MQTT is a binary protocol.
However sending binary data makes it more difficult to add meta data to the file data.
For the file name which is most important bit of meta data the easiest approach is to include it as part of the topic.
MQTTv5 introduced some important additions which allows you to add the file meta data outside of the payload using User properties.
Indicating Start and end of File
To tell the receiver that the file transfer is starting we can use a start and end message.
These messages would not contain file data but would contain file meta data.
Verifying Successful transfer
There are a number of ways this can be done e.g.
- Use the file length
- Use a file hash
- length + file hash.
To avoid the the problem of buffering the entire file then the file length and digest is send in the end of file message.
Broker/Sever Extensions
Some commercial brokers have developed server extensions aimed at making file transfer over MQTT easier. See hiveMQ and EMQX
Common Questions and Answers
Q1- MQTT wasn’t designed for file transfer so why use it?
A1- For convenience. HTTP wasn’t designed for email or file downloading but it used for both.
Q2 – Is MQTTv5 better than MQTT v3.1.1 for File transfer.
A2- MQTTv5 has some useful features that make file sending easier as data can be sent outside the message payload.
Q3 – Can I use QOS of 0?
A3 -Yes but many programs will use the puback to control message flow
Common Use Cases
- Uploading diagnostic logs or telemetry data from vehicles or sensors.
- Sending firmware updates (OTA) to IoT devices.
- Transferring images or video clips from security cameras to a server or to different locations.
Summary
Although MQTT was intended as a file transport protocol it can be used for sending files across a network.
Although it may be frowned upon by some it will probably become a common use case in the future due the ease of doing it and the convenience.
File Transfer Examples