Paho Python MQTT Client – Understanding Callbacks

Callbacks are functions that are called in response to an event.

The events and callbacks for the Paho MQTT client are as follows:

  • Event Connection acknowledged Triggers the on_connect callback
  • Event Disconnection acknowledged Triggers the on_disconnect callback
  • Event Subscription acknowledged Triggers the  on_subscribe callback
  • Event Un-subscription acknowledged Triggers the  on_unsubscribe callback
  • Event Publish acknowledged Triggers the on_publish callback
  • Event Message Received Triggers the on_message callback
  • Event Log information available Triggers the on_log callback





The Paho client is designed to only use the callback functions if they exist, and doesn’t provide any default callback functions.

However; it does provide a mechanism to set them.

To use a callback you need to do two things

  1. Create the callback function
  2. Assign the function to the callback.

Callback Function Example

As an example we will use the on_connect callback.

First I create my function. I’ve called it on_connect but I could have called it anything I wanted.

Here is the function.

def on_connect(client, userdata, flags, rc):
     logging.info("Connected flags"+str(flags)+"result code "\
     +str(rc)+"client1_id ")
     client.connected_flag=True

The function simply logs a message, and sets a flag.

Now to associate my function with the On_connect callback I use the following:


client.on_connect= on_connect

Note: If my function had been called myfunction then I would use the following:


client.on_connect= myfunction

Callbacks and the Client Loop

Callbacks are dependent on the client loop as without the loop the callbacks aren’t triggered.

Below is the code of a simple python script that creates a connection and then waits in a loop for the on_connect callback to be triggered.

The on_connect callback then sets the flag that terminates the loop and the script ends.

This means that if the callback isn’t processed then the while loop never ends as we shall see later.

callback-on_connect-example

Here is the output generated by the script.

Notice how the loop executes waiting for the callback and stops once the callback gets processed.callback-on_connect-result-example

In the script above the client..loop_start() method starts a loop to process callbacks.

So let’s run the above script again, but this time I won’t start the loop. To do that I simply comment out the client.start_loop() line.

callback-loop-comment-out

Here is what happens when I run the script.

callback-loop stopped

You can see that I had to manually terminate the script as it never processed the callback, and the callback is what triggers the callback that terminates the while loop.

How Callbacks Work

If you dig into the Client code you will find the code for the Connect acknowledge message.

When the client receives a CONNACK message the callback is triggered if it exists.

Here is the relevant section of the code. (around line 276)

client-callbacks

Notice the if on_connect statement. This is what checks for the callback, and if it exists it calls the callback- self.on_connect().

The self.on_connect function is the function we assigned earlier using.

client.on_connect= on_connect

You should see from the code that the function call passes several arguments, and so the function we create needs to be able to handle those arguments.

If you look back at the function I created you can see that it handles 4 arguments. The documentation has a description of those arguments

callback-connect

Asynchronous or Synchronous?

If you use the loop_start() or loop_forever functions then the loop runs in a separate thread, and it is the loop that processes the incoming and outgoing messages.

In this case the callbacks can occur anytime in your script and are asynchronous.

However if you call the loop() function manually in the script then the callbacks are synchronous with your script as they can only occur when you call the loop() function.

See Understanding the Loop Function

Video- The Loop and Callbacks.

Returning Values from Callbacks

Due to the way they operate It is not possible to return values from a callback using the standard return command.

Therefore if you need to get status information from a callback you need to use some form of global variable in the callback.

If you refer back to the on_connect call back example I used the connected_flag which is a property of the client object.

Because the client object is available throughout the script it can e accessed anywhere in the script.

In my scripts the my general approach is extend the main mqtt client class with flags and other variable I need.

When I create client object it has these flags already assigned.

This is what it looks like:

import paho.mqtt.client as mqtt #import mqqt client
### extend main class to include flags etc
mqtt.Client.bad_connection_flag=False
mqtt.Client.connected_flag=False
mqtt.Client.disconnect_flag=False
mqtt.Client.disconnect_time=0.0
mqtt.Client.pub_msg_count=0
##
client=mqtt.Client("myclient") #create client object
###
client.client.connected_flag=True #set flag to true somewhere in script
#####

Common Questions and Answers

Q- MY callback is not being triggered?

A- Have you assigned the function to the Callback? Are you running a loop for that client?

Q-My callback returns a value but I don’t see it?

A- Because of the asynchronous nature of callbacks it doesn’t make send to return a value from them

Related Articles and Resources:

Facebooktwittergoogle_plusredditpinterestlinkedinmail

12 comments

  1. Hi Steve,

    great job for this tuto, i just have one question, some one give me a project and to connect to the broker he uses :

    def mqttCallbacks(self):
    subscribe.callback(self.message_callback, ‘#’, port = self.port, hostname=self.broker, auth=self.authentication, protocol=mqtt.MQTTv311, tls=self.TLS)

    def message_callback(self, client, userdata, msg):

    but i can’t find a way to get the return code, i need to store it to know in real time the connection status

  2. Hi Steve
    Thanks for your tutorial. I see you use a loop_flag which get a value within the on_connect function and then the value is used on the main program. I have tried to do the same. i.e. use a value changed within the on_message callback function. But it doesn’t work. From within the function, I can print the received value and it is ok. But when I print the value from other part of the program it does not have any value. I even used global to define the variables to be used within the callback function. The result is the same. Please help.

    1. Jimmy
      I use the client object for the flag. This makes it available across the script.
      import paho.mqtt.client as mqtt
      mqtt.Client.connected_flag=False # assign property to the Client object
      client= mqtt.Client(cname,cleansession) #create a new client

      The inside the on_connect I use:
      if rc==0:
      client.connected_flag=True
      else:
      client.bad_connection_flag=True

      Note: bad connection flag aslo need to be set not shown above.

      I have two general functions that I call to set the flags and callbacks. They look like this
      #################

      def Initialise_client_object():
      mqtt.Client.last_pub_time=time.time()
      mqtt.Client.topic_ack=[]
      mqtt.Client.run_flag=True
      mqtt.Client.subscribe_flag=False
      mqtt.Client.bad_connection_flag=False
      mqtt.Client.connected_flag=False
      mqtt.Client.disconnect_flag=False
      mqtt.Client.disconnect_time=0.0
      mqtt.Client.pub_msg_count=0

      def Initialise_clients(cname,mqttclient_log,cleansession=True):
      #flags set
      client= mqtt.Client(cname,cleansession)
      if mqttclient_log: #enable mqqt client logging
      client.on_log=on_log
      client.on_connect= on_connect #attach function to callback
      client.on_message=on_message #attach function to callback
      client.on_disconnect=on_disconnect
      client.on_subscribe=on_subscribe
      return client

      ##################

      1. Steve.
        Thanks for your explanation.
        To clarify. I am trying not to use flags or internally generated values like the clock.
        But to use values received from the outside via MQTT message within the script. An external clock for example, since the RPI will not be connected to the internet and therefore does not have a way of knowing the actual time.
        Currently I am receiving a value, lets call it external_value1. When receive this value, I add (or subtract ) a numerical value to it.
        #####
        import paho.mqtt.client as mqtt
        import time
        import sys
        import datetime
        ####
        external_value1 = 0
        added_value2 = 0.0
        topic = [“home”, “”]
        #####
        def on_message(client, userdata, message):
        global external_value1, added_value2
        topic = message.topic.split(“/”)
        if topic[1] == “mytopic”:
        external_value1 = str(message.payload.decode(“utf-8”))
        added_value2 = float(external_value1) – time.time()
        print(external_value1, added_value2) # this prints the actual received external_value1 and the resulting added_value2
        if topic[1] == “othertopic”
        …… do something else
        ####
        client = mqtt.Client(client_id=”machine”)
        client.on_connect = on_connect
        client.on_message = on_message
        client.on_log = on_log
        ####
        client.connect(broker_address, 1883, 60)
        ####
        client.suscribe(“home/mytopic”, 1)
        client.suscribe(“home/othertopic”, 1)
        ####
        client.loop.start()
        ####
        while true:
        try:
        print(external_value1, added_value2) #this prints 0, 0.0 values
        time.sleep(10)
        except (KeyboardInterrupt, SystemExit):
        client.disconnect()
        client.loop_stop()
        sys.exit()
        #####
        My question is why I cannot see the values received within the script. Only inside the call back function.
        Thanks
        Jimmy

        1. You are using global variables so you should be able to see them.
          However the general way of dealing with the messages is to drop them in a list or queue.
          For a list you just do define the list in the main script
          myarray.extend(message).
          You can then get the value anywhere in the script.

          The problem you might get with arrays is if they change while you are looping through them.

          queues are usually better. Here is the relevant code taken from one of my scripts
          ############
          from queue import Queue
          q=Queue()
          ########
          def on_message(client,userdata, msg):
          topic=msg.topic
          m_decode=str(msg.payload.decode(“utf-8″,”ignore”))
          q.put( m_decode)
          ############
          while not q.empty():
          results = q.get()
          if results is None:
          continue
          #######
          See this tutorial for more about the queue class
          https://pymotw.com/3/queue/index.html

          1. Steve.
            Thanks. Firt, You help me made sure my procedure is as expected. I will go through it again. Hopefully I misspelled the name of a variable. – Something that has happened to me before, more times that I would like to admit. – Second, I would examine the usage of queues. It seems that is just what I need now. Thanks for you time, your kindness and your help. May you receive blessings.

  3. Hi Steve,
    Thanks for the tutorial. I am trying to understand how the process of acknowledgement works in MQTT. Say, a publisher publishes a message with qos 1 or 2 and the subscriber receives the message. How will the publisher understand if the following message with msg ID is recieved by the subscriber? Is there any method to realize this functionality using on_publish callback method?

  4. Hi Steve,

    Can I define my on_message callback with only message argument?
    mqttc.on_message = on_message
    def on_message(self, message): #client, userdata are not used in my callback implementation

Leave a Reply

Your email address will not be published. Required fields are marked *