MQTT Clean Sessions and QOS Examples

mqtt-clean-session-iconWhen a client connects to a broker it can connect using either

  • a non persistent connection (clean session) or
  • a persistent connection..

With a non persistent connection the broker doesn’t store any subscription information or undelivered messages for the client.

This mode is ideal when the client only publishes messages.

It can also connect as a durable client using a persistent connection.



In this mode the broker will store subscription information, and undelivered messages for the client.

In order for the broker to store session information for a client a client id must be used.

Note: Messages that have been delivered to the client are deleted from the broker. They are not stored like email messages are stored.

When a client connects to broker it uses a flag (clean_session flag) to indicate whether or not a persistent connection is required.

However it is important to realise that not all messages will be stored for delivery, as the quality of service, of the subscriber and publisher has an effect.

In this example we look at the effects of the clean_session flag and of the quality of service settings on message delivery.

Example Outline and Objectives

In this Example I will be using a Python script that creates two client connections.

One connection subscribes to a topic and the other connection publishes on that topic.

The objective is to see what happens to published messages when a subscriber is disconnected for any reason ,and how the clean_session flag, and QOS settings affect message delivery.

Basic Process

  1. Initialise the client with clean_session flag set appropriately
  2. Subscribe to a topic with QOS set
  3. Disconnect.
  4. Publish to the topic that the client subscribed to with QOS set.
  5. Reconnect client
  6. Make note of any messages received

Python Client Configuration

The clean session flag is set in the client constructor which takes 4 optional parameters. The default values are shown below:

Client(client_id=””, clean_session=True, userdata=None, protocol=MQTTv311, transport=”tcp”)

in your script create a client instance using:

client= paho.Client("Python1",False) #create client object persistent connection

or

client= paho.Client("Python1",True) #create client object clean seesion

Test 1

clean_session =True,

publish and subscribe default QOS  (quality of service) of 0

Expected result- No messages received as the broker shouldn’t store them.

Actual Result -No messages received when reconnected as the broker didn’t store them.

Screen shots

clean-session-1

Test 2

clean_session =False,

publish and subscribe default QOS (quality of service) of 0

Expected result- Messages to be received.

Actual Result -No messages received when reconnected

Comment– It is important to realise that not only do we need to set the clean_session flag to False, but we also need to publish and subscribe with a QOS of 1 or above.

However the subscription information is remembered by the broker, and so the client doesn’t need to subscribe again.

Screen shots

clean-session-2

Test 3

clean_session =True,

publish and subscribe qos (quality of service) = 1

Expected result- No messages received when reconnected..

Actual Result -No messages received when reconnected as the clean_session flag was true.

Screen shots

clean-session-3

Test 4

clean_session =False,

publish and subscribe qos (quality of service) = 1

Expected result- Messages to be received when reconnected.

Actual Result – Messages received.when reconnected

Comment-Clean session flag is false and QOS on publish and subscribe are greater than 0.

clean-session-4

Test 5

clean_session =False,

publish QOS (quality of service) =0 and subscribe QOS(quality of service) = 1

Expected result- ?

Actual Result – No Messages received when reconnected

Comment-Clean session flag is false but QOS on publish isn’t>0

clean-session-5

The table below is a summary showing how QOS, cleaned,session flag, and retained message flag affects what messages are received.

mqtt-qos-retain-clean-session-table

Clean Sessions Video

If you prefer videos then I’ve created a video that covers the main points. You can watch it on YouTube here .

Demo Script

Common Questions and Answers

Q- How many messages will the broker store?

A- This is a broker setting. The mosquitto broker defaults to 100 – Setting is max_queued_messages count

Q- How do I reset the client connection to be non-Persistent?

A- Connect again will clean session set to True

Summary

Although MQTT doesn’t store messages in the same fashion as services like email, it does, in certain circumstances, store them.

Whether or not they are stored depends on the QOS of both the publisher and the subscriber.

However once they have been received and acknowledged by the subscriber, the broker deletes them immediately, regardless of the Clean session and QOS settings.



MQTT by Example Series

Related tutorials and Resources:

Please rate? And use Comments to let me know more

13 comments

  1. Steve,
    Your tutorials have been invaluable!! Thank you.

    I have a question. I am programming an X10 security system that uses MQTT as a means to communicate with Home Assistant, using X10 MS14A motion sensors. The Home Assistant system is running on a Raspberry Pi 3B+. On another machine, Ubuntu 20.04, I am running the mqtt broker (Mosquitto v3..1.1), the heyu (v 2-11-r3 X10 software) and the python scripts to glue it all together.

    I have all the security switches working fine with Home Assistant from a python script on the ubuntu server. I have written a script to monitor the heyu log file for motion detection. When I detect motion I parse the message to get the heyu ALIAS name for the device. Once I have the ALIAS name, I call anther python script that takes the ALIAS name as input and from a python dictionary using the ALIAS name as input, I grab the topic matching the ALIAS name. Once I have the topic, I subscribe to that topic to find out the state of security, i.e whether the security is on or off.

    The problem I have is that motion get triggered multiple times as long an there is motion.. If I check to state of the topic, retain_flag = True, QOS = 1, clean_flag = False, it works the first time I check, but then the message of being set is being removed so I can’t check it the next iteration of motion.

    Is there a way for the subscriber to check the state of a topic and not have the state message removed?

    Thanks for your feedback
    Carlos

    1. Hi
      Glad you find the site useful. I’m confused by this paragraph.

      The problem I have is that motion get triggered multiple times as long an there is motion.. If I check to state of the topic, retain_flag = True, QOS = 1, clean_flag = False, it works the first time I check, but then the message of being set is being removed so I can’t check it the next iteration of motion.

      With a retained message it is not removed only replaced by another one that has been published.
      So it does not matter how many clients subscribe and how often they will get the retained message.
      I have missed the point?
      Steve

      1. If I understand this correctly:
        “Note: Messages that have been delivered to the client are deleted from the broker. They are not stored like email messages are stored.”

        The message form the publisher is deleted once the subscriber has received it. My security..py app publishes the setting and may not change state for a period of time, and the state.py script has to be able to check the state multiple times.

        If the state.py app checks the state multiple time, won’t the first time it checks the state the message and it is removed? So the next time it goes to check the state it won’t be able to because the publisher has not changed the state

        1. Carlos
          Not if you set the retain flag on the published message then it will stay on the broker until replaced.
          Does that make sense

          1. Steve,

            If that is the case, when does this apply:
            “Note: Messages that have been delivered to the client are deleted from the broker. They are not stored like email messages are stored.”

            Thank you very much for your input!!

            Carlos

          2. When the message is not retained. The broker will only store 1 retain message per topic. So the next message will overwrite the retained message.

        2. Steve,
          Ok, I get it. I have another strange problem, related to the previous issue.
          1. When I run the program, I’m not getting any output. But when I run it with pdb, I’m ggetting output, as I would expect, problem is, I get 2 messages.
          The publisher, broker clean flag False, qos=1, retain=True
          The client, subscribe, broker clean flag False, qos=1, retain=True
          and tried with qos=0, same result, 2 messages every time.

          Unfortunately, the 2 messages have the oposite states, The first message has the correct state but the second has the incorrect state,

          Any thoughts? Mainly why it works with pdb?

          thanks Carlos

  2. Hi Steve. Thank you for this useful tutorial for beginners. I really appreciate if you could publish the content of the python script you mentioned before, just to understand how do you collect the information and show it

    Thank you in advance

Leave a Reply

Your email address will not be published. Required fields are marked *