MQTT topics are a form of addressing that allows MQTT clients to share information.
MQTT Topics are structured in a hierarchy similar to folders and files in a file system using the forward slash ( / )as a delimiter.
Using this system you can create a user friendly and self descriptive naming structures of you own choosing.
Topic names are:
- Case sensitive
- use UTF-8 strings.
- Must consist of at least one character to be valid.
Except for the $SYS topic there is no default or standard topic structure.
That is there are no topics created on a broker by default, except for the $SYS topic.
All topics are created by a subscribing or publishing client, and they are not permanent.
A topic only exists if a client has subscribed to it, or a broker has a retained or last will messages stored for that topic.
The $SYS topic
This is a reserved topic and is used by most MQTT brokers to publish information about the broker.
They are read-only topics for the MQTT clients. There is no standard for this topic structure but there is a guideline here that most broker implementations seem to follow.
I have created a node-red dashboard that monitors the $SYS topic and produces a display as shown below:
Here is the flow:
Subscribing to Topics
A client can subscribe to individual or multiple topics.
When subscribing to multiple topics two wildcard characters can be used. They are:
- # (hash character) – multi level wildcard
- + (plus character) -single level wildcard
Wildcards can only be used to denote a level or multi-levels i.e /house/# and not as part of the name to denote multiple characters e.g. hou# is not valid
Topic naming Examples:
Valid Topic subscriptions
Single topic subscriptions
Using Topic Wildcards
Subscribing to topic house/#
Subscribing to topic house/+/main-light
but doesn’t cover
Invalid Topic Subscriptions
- house+ – Reason- no topic level
- house# – Reason- no topic level
Publishing to Topics
A client can only publish to an individual topic. That is, using wildcards when publishing is not allowed.
E.G- To publish a message to two topics you need to publish the message twice
When are Topics Created
Topics are created dynamically when:
- Someone subscribes to a topic
- Someone publishes a message to a topic with the retained message set to True.
When are Topics Removed from a Broker
- When the last client that is subscribing to that broker disconnects, and clean session is true.
- When a client connects with clean session set to True.
Republishing Topic Data
This is likely to be done when changing or combining naming schemes.
The idea is that a client would subscribe to a topic, e.g.
hub1/sensor1 and republish the data using a new topic naming of house1/main-light.
Video A Beginners Guide to MQTT Topics
Simple Topic Guidelines
- Because topics are case sensitive use lower case only.
- Separate command and response topics using a prefix e.g command/ and response/
- Don’t use topics with spaces.
- Use letters numbers and dashes only
- Include routing information in the topic structure.
Common Questions and Answers
Q- How do I subscribe to all topics?
A-Subscribe to #
Q- How Do I subscribe to all $SYS topics?
A-Subscribe to $SYS/#
Q- Should I start my Topic hierarchy with a /.
A- It is not necessary and just adds another level to the structure.
Q- Can I get list of all topics on a broker?
A- Not unless you subscribe to all topics and scan them.
Q- Can I tell who is subscribed to a topic?
Q- How do I discover topics?
A- There is currently no mechanism for that except as described in list all topics.
How MQTT Works << Prev ..Next .>>MQTT Publishing,Subscribing and Message Exchange –
Related Tutorials and Resources:
- MQTT Topic and Payload Design Notes
- MQTT Sensors and Network Traffic Observations
- MQTT topic naming convention
- Gitihub architectural proposal for smart home
Hi Steve, great content. I would like to publish to different topics. can I publish using wildcards? for example: Area1/sensor1 and Area1/sensor2… publish method: Area1/#
Or the wildcards only work for subscribers?
The main objective is that I need to send information to one subscriber but also to all subscribers. how do I do it?
Wildcards only work for subscriptions.
I have a question that I’m struggling to find a direct answer to and was hoping you could point me in the right direction. I have the mosquitto broker installed on my server and a handful of esp32s publishing to topics and everything is going well. I’d like to create a website(locked behind auth of course) that on connect grabs the latest values for device/+ topics using websockets to sync the statuses. I’m using the mqtt.js library and connecting is fine, but it will only receive messages on publish but not on connect. Is what I’m trying to do possible with the mqtt protocol/websockets or is this a fools errand?
I have my mostquitto broker set up to retain messages and when esp32s publish they’re passing the retain flag as well.
If I understood correctly it looks like all you need to do is to get the clients to publish with the retain flag set.
What a great website, thank you.
The question I have relates to topic structure.
Suppose I have a topic structure such as:-
In my callback function I want to do different actions according to the message received.
Can I use a wildcard to test the callback received ‘topic’ such as (not real code)
If (strcmp(#/#/light/status) ==’on’)) ……
Or do I need to manually test each topic branch condition separately? (which would detract from having a structure in the first place)
No as the topic will never contain #. However as the topic always ends in status you can look for that. better to look for /status if looking at the entire topic.
What I normally do is to split the string in python
last element is
now we can use if
Is it possible to use the + and the # in the same subscription request?
to match all of :
I have a system set up like this and it’s not responding, though it reports that subscriptions are successful. If I try hardcoding the name instead of “+” it works.
That should work ok. I just tried it and it works.
test it on the test broker test.mosquitto.org
can we pass a list of topics ?like this
[(topic1,qos),(topic2,qos)] for publish multiple topics?
I want to publish multiple topics at a time. What should I need to do ?
Please reply as soon as possible
Yes It is a list of tuples exacly as you have shown
how can we manage multiple topic subscription in MQTT without using #?
subscribe without #
because device1 and device3 level can have 100 topics
do we need to use any async calls for the same ?
so that if we are subscribing 100 topics then every topic should have one thread working for that topic itself
please provide solution as early as possible.
Not quite sure or what you mean can you explain further.
Using Topic Wildcards
Subscribing to topic house/#
I think it’s important to say that it covers
without any sub-topic too!!
Btw. it covers
too but its no good idea to use blank level.
(Tested with mosquitto, mosquitto_sub and mosquitto_pub)
Tks for that I will include it.
I find it odd that it is not possible to get a list of (subscribed) topics from a broker.
The broker has to know which topic to send to which client, so this list should exist somewhere but is not accessible ?
Or does the broker just sends off every message to every client and let the client decide if they want to accept it or not ? (which would be rather inefficient)
Some brokers may do that but currently mosquitto doesn’t. However as you aaid there is no reason why not.
hello, my project consist to comunicate mosquitto broker with an Api (made using django).
my esp32 is sendig sensors data to the mosquitto using Mqtt, and i would send this data to the Api . Any idea can help me please !
Seems that the Api would need to request the data from broker by subscribing but without more details I cannot really say.
I am making an iot application where I have to connect to topics of multiple pharmacies with multiple refrigerators each containing 7 switches(show on and off status) and one temperature sensor and one humidity sensor.
My topics look like this:
For refrigerator 1 in pharmacy 1: Pharmacy1/Refrigerator1/Switch1 …. (7 topics for switchers), Pharmacy1/Refrigerator1/sensor1, Pharmacy1/Refrigerator1/sensor2
For refrigerator 2 in pharmacy 1: Pharmacy2/Refrigerator2/Switch1 ………. and so on.
1. How do I input the topics in my python? So that I can subscribe to all the topics. and then work on payload coming from each topic.
MY action for switches is different, action for humidity is different and for temperature is different.
I tried using wildcard for subscribing, but then, I cannot use the action separately for switches and sensors.
2. I want to count the on switches in each pharmacy. So I have put a callback function on the payload. But, my code works only for Refrigerator1 in pharmacy1. I want to count switches (which are on) for all the refrigerators in each pharmacy.
How do I find only switch topics/only sensors from the long list? As I want to make it scalable for a large number of pharmacies and refrigerators.
IF I use wildcard for subscribing: subscribe to topic #/Switch1 , #/Switch2 ….. , #/sensor1, #/sensor2 and then write :
if topic == Pharmacy1/+/Switch1, it gives errors that the topics are not in the list
I would look at a structure like
and subscribe to iot/#
you cannot use wildcards in the == statement
I would use the split to get an array from the topic like
Hope that helps
Thanks for the quick response steve, but can you please tell
1. Does that mean I have to write down all the topics and pass them to subscriber as topic_list?
2. OR in subscriber, i have to write to write only iot/#, because in this case, i can’t use split write?
I dont understand fully, that if i i have to split, is it compulsory to use a topic_list of all topics, or using iot/# in subscriber would work?
just subscribe to iot/#
and that will get all topics
I want to subscribe multiple topics at a time. What should I need to do ?
in python you can pass a list of topics like this
Other clients work much the same
I have a structure like this:
Is there any way to subscribe to get all the temperature and humidity but only device1 and device3? (avoiding receive topics under devide2 or other possible devices)
It would be something like this:
Being the pipe a logical or operator
Pipe is note allowed. You would need to subscribe to
to get what you want.
I have a case where it would be great if the broker could communicate whether there are any subscriptions for a particular topic. Based on what I have read, I believe the answer is no, but I’m hoping I am wrong.
In MQTTv3.1.1 that is correct but v5 has a no matching subscriber in the PUBACK message.So you would need to publish on that topic with QOS of 1 or 2.
Hello Steve, I have a MQTT gateway for my controllers, and I set up its publish topic. When I subscribe to it, the messages that are shown (that I receive) have a lot of replacement characters, they look like �J�o����
Is the problem the protocol that my gateway is sending the data in, or is it something else?
Thank you in advance,
It is probably because you aren’t using Unicode for the topic
I’ve got a question about $SYS topics. Something is wrong: I subscribe to “$SYS/#” and also to “house/bulb1”. Then I publish to “house/bulb1”. I never receive “house” messages, only get $SYS messages. Eliminate the subscribe(“$SYS/#”) and house messages are received.
I am using Python, paho-mqtt. The broker is test.mosquitto.org, currently version 2.0.3.
Oh, I just tried “broker.hivemq.com” and did get back both $SYS and house messages.
And “mqtt.eclipse.org” is not responding.
Is there anything I can do to “fix” the problem I see with test.mosquitto.org?
If you get them from hivemq then you should get them from test.mosquitto unless they have disabled publishing to it. Try another topic like test/house/house1 and see if it makes a difference
I have a subscription using a wildcard and want to make sure I have received all the available topics before my program moves on.
Is there a way to do this other than just using a timer to wait for a set time and hoping that everything has been received?
Great information here and extremely useful!
There is no way of knowing how many sub topics there are and also more could be added as the topic tree is dynamic. However even if your program moves on a message to a new topic will still be received.
In Python I store the topics in a dictionary using the topic as the key.
Hope that helps
I have a query.
Currently I am working on a system with the following functionality
1. Mulitple sensors are sending data to the HTTP service (on my server)
2. The service processes the data and stores the data into a table.
3. This data is further processed by other services that run continuously
4. On my browser in node js, the front end application sends request to another service on my server which then responds back with the final processed data and other information related to each of these sensors and it is then displayed on the front end
Can you suggest how I can do this functionality in MQTT. I am looking at your tutorials and so far just installed Mosquitto.
Can you guide me with the next steps. How do I trigger these functionalities for processing the data received from each of these sensors. I want to even use subscription on front end so that the broker pushes data to the front-end only when there is change and not vice-versa.
Basically I want to know
1. How to create topics in Mosquiito
2. when a data is received for a topic, it should be processed by my applications on the back-end (how to trigger them)
3. Then how to tell Mosquitto that the data is modified, so it should push it to the browser (/android and IOS apps) front end if at all they are open / logged in
If you can provide me with any links to your tutorials that you have already shown how to do these it would very helpful.
I am complete newbie to MQTT. I am good with linux, C, C++, JAVA, Python. My current services are written in JAVA.
Thanks in advance 🙂
Suggest you go to the course page
and work your way through it to get a better understanding.
For mosquitto or any MQTT broker it does message transfer and that is all. It doesn’t process the message so if you want to only send change then you need to do that on the sensor.
This tutorial may help
On the server side for display then the cloud based server must be capable of receiving mqtt which unless it is a virtual server it probably can’t, So you might need to design it so the sensor data is collected and processed locally and sent for display via http to the web server.
Let me know how you get on
I am following your documentation, I am new born baby in this Mosquitto MQTT app installation and configuration.
Got struck on my Cluster and Scale configuration.
Steps I followed.
1. Edited “/etc/mosquitto/mosquitto.conf”
## maximum connection to mqtt allowed
* soft nofile 20000
* hard nofile 20000
3. Based troubleshooting guidance I followed the links and created new file under
/etc/init/mosquitto.conf, added the lines
limit nofile 16384 16384
With all these settings, I didn’t successes still I am getting the same error
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 14679 14679 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 14679 14679 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Thank you and appreciate, if you can guide me, what is the missed part here.
You may find these two resources useful
assume we have 100 apartments and each apartment has 10 sensors. propose a topics scheme (how will you design your topic names) for the subscribers which are.
1- tenants (a tenant is a person who is renting the apartment and should have access to all the sensors in his apartment)
2- Maintenance company (the company should have access to all the sensors in all the apartments)
How it works ?
Use a topic base of apartments. The maintenance company will have access to this.
for each apartment use:
each tenant subscribes to his own topic e.g tenant 1 will subscribe and publish on
You can restrict access using ACLs (access control Lists)
I publish data to 2 topics from 2 different clients and later I want to subscribe to these topics and save the data in 2 tables named gnss and acc in a database.
Client 1 will publish data to topics: my/device1/gnss and my/device1/acc
Client 2 will publish data to topics: my/device2/gnss and my/device2/acc
Now I subscribe to all these 4 topics by using multilevel wildcard: my/#
But how can I save the data of all clients into 2 tables gnss and acc irrespective of the client.
if msg.topic == “my/device1/gnss”:
sql = “””INSERT INTO gnss (topic, gnssdata) VALUES (%s,%s)”””
tuple = (msg.topic, msg.payload)
if msg.topic == “my/device1/acc”:
sql1 = “””INSERT INTO acc (topic, accdata) VALUES (%s,%s)”””
tuple = (msg.topic, msg.payload)
The above code will just save the data from Client1. How to do this for all the clients if I am having various clients which are publishing data to the broker.
I tried using: if msg.topic == “my/+/gnss”:
sql = “””INSERT INTO gnss (topic, gnssdata) VALUES (%s,%s)”””
tuple = (msg.topic, msg.payload)
But this doesn’t work.
Just do a find on the topic for “acc” and also for “gnss” and store accordingly.
Does that make sense.
Hi Steve, good intro, thanks
is there access control of who can subscribe the topic and publish msg on the topic?
I think I find the answer at http://www.steves-internet-guide.com/topic-restriction-mosquitto-configuration/
Good site, Thanks
Do all publishers share the same namespace on the broker? What happens if two different publishers happen to chose the same name for different items?
Yes and you can get conflicts if they decided to use the same topic names. However most brokers let you restrict access to topic trees and so you can easily stop this on a managed system.
I was wondering if you have any idea why publishing a single message to multiple topics is not currently supported?
No idea and it isn’t in v5 so it is unlikely ever to be there.
when I clear all topics by connecting a client with clear session = true, do other clients get any notification about this? And also do the clients have to resubscribe after a clear (I assume yes)?
The clean session flags only applies to that client and not to the clients subscribed to a particular topic.
The client will need to subscribe to the topics when it connects again but not other clients.
Does that make sense?
I am using the Eclipse Paho gateway between mqtt and mqtt-sn. The Paho gateway forwards to the Eclipse mqtt broker.
When I add a subscribe client to the Eclipse broker with a subscribe to cmd/+ , everything works as expected. When I publish a command from my mqtt-sn app like cmd/A, The subscribed client recieves it with topicname cmd/A.
But the other way around does not work. When my mqtt-sn app subscribes to data/+, it registers a topic with e.g. topic-id 1. When my mqtt broker client publishes something on data/A, my mqtt-sn app receives a message on a new topic-id 2. And I don’t have or did recieve the topic-name of this topic-id (no way to find the A).
From the mqtt-sn-1.2 specs:
“A GW sends a REGISTER message to a client if it wants to inform that client about the topic name and the assigned topic id that it will use later on when sending PUBLISH messages of the corresponding topic name. This happens for example when the client re-connects without having set the “CleanSession” flag or the client has subscribed to topic names that contain wildcard characters such as # or +”
For some reason these REGISTER messages were not sent, or my app missed them.
Anyone encountered a similar problem?
Sorry for the late reply. If you enable debugging on the broker you should wee the messages being send to your client.
Thanks for your help!
The sn-client is a nRF52840 DK. The dump is created with a client created with the NordicThread sdk version 2.0.0. Today I tried with 3.1.0. Behaviour is the same.
I annotated with ///
* MQTT-SN Transparent Gateway
* Part of Project Paho in Eclipse
* Author : Tomoaki YAMAGUCHI
* Version: 1.0.0
20190707 110523.239 PahoGateway-01 has been started.
SensorN/W: Gateway Port: 47193 Broadcast Address: ff03::1 Interface: wpan0
Broker: 22.214.171.124 : 1883, 8883
20190707 110525.830 CONNECT MySensGw 10 14 00 04 4D 51 54 54 04 02 00 3C 00 08 4D 79 53 65 6E 73 47 77
20190707 110526.241 CONNACK MySensGw 05 00
/// sn-client registers MySensTx/9 for publishing. Not part of the problem
20190707 110526.267 REGISTER 0001 MySensGw 0B 00 01 00 01 00
/// sn-client registers MySensRx/+ fore receiving
20190707 110530.000 REGISTER 0002 MySensGw 0B 00 02 00 02 00
/// sn-client subscribes to MySensRx/+
20190707 110530.673 SUBSCRIBE 0003 MySensGw 82 0F 00 03 00 0A 4D 79 53 65 6E 73 52 78 2F 2B 01
20190707 110530.835 SUBACK 0003 MySensGw 13 20 00 02 00 03 00
/// sn-client publishes MySensRx/9 and receives a Register. How does the client knows about this MySensRx/9 ?
/// This was sent to this client with a publish from a different client on a different system
/// I am also missing Ack
20190707 110547.824 PUBLISH 0000 MySensGw 0A 00 03 00 01 9C 04 C0 73 01
20190707 110547.825 PUBLISH 0000 —> MySensGw 0C 00 00 03 00 00 30
20190707 110550.280 REGACK 0001 <— MySensGw 0B 00 03 00 01 00
20190707 110626.264 PINGREQ MySensGw C0 00
I think it may be because you are using wild cards for the subscribe. The register applies to MySensRX/+
If you still have problems I will need to copy your setup and give it a try. Let me know.
I solved the problem.
May be I was not clear on that, but I am trying to use the OpenThread implementation with target Nordic nRF52840 using the Nordic Thread and Zigbee SDK.
Actually, I found three problems:
– In the syslog of Paho, entries with the same timestamp are in random order.
– My test application did not handle MQTTSN_EVENT_REGISTER_RECEIVED properly.
– The register_handle method of Nordic in their file mqttsn_packet_receiver.c tries to get the topic_name from topic.cstring while it actually can be found in topic.lenstring.data. I don’t know if this is a Paho or Nordic bug.
It is working now. I can subscribe on my nRF52840 MQTT-SN client with a wildcard and send messages from a MQTT client on the internet to my client in an OpenThread mesh network. And the topic name is registered on the client, so,when I am subscribed to cmd/+, I can find out by the topic_id whether the received message was published to cmd/test1 or cmd/test2.
Glad its working
Great site. Can I have multiple wildcards? For example, is this valid:
Also you said there is no wildcards within topics, is there any work around for this? Anyw
You can use wildcards at multiple levels.
There is no workaround for wildcards in topic names as far as I know.
Hello steve and anybody, who can tell me the topic ‘house/room1’ and ‘house/room1/ are the same?
They are different. It is effectively house/room1?. just as house/room1 and /house/room1 are different
thank you very much. I’m get.
Hello, this was a great intro to MQTT.
Can Brokers/Servers have licensing server associated to each /topic? Does it have this facility, lend easily to this method of billing or does it need to architecting?
I haven’t seen anything like that but it may come.
this article helps me a lot
Thank you for this information! I also had a question : If a subscribe to a wildcard topic in Node-Red is there a way to separate the different topics coming through the subscribe node? For example if I am subscribe to sensor/#. How could I split it into sensor/temperature and sensor/humidity? Thanks!
The easiest way is to use the switch node and use topic not payload in the property field.
If you have problems use the ask steve link and I’ll email you a screenshot and flow.
hello steve ,how i execute my java code for subscriber and publish in linux terminal??
Sorry but I don’t use the Java client