The Paho MQTT Python Client-Beginners Guide

mqtt-python-clientThe paho MQTT python client from Eclipse supports MQTT v 3.1 and 3,1.1, and works with Python 2.7 and 3.x.

Installing The Client

You can Install the client using PIP with the command:

  • pip install paho-mqtt

You will find the online client documentation here. and also the install files if you need them.

Tutorial Outline

In this tutorial we look at the main client object, and it’s methods.

We will then create a simple Python example script that subscribes to a topic and publishes messages on that topic.

If all goes well we should see the published messages.

The example scripts are kept simple, and I don’t include any error checking. I use my own locally installed broker, but you will probably find it easier when starting to use a free online broker like:

  • test.mosquitto.org
  • broker.hivemq.com
  • iot.eclipse.org

The Python MQTT Client

The core of the client library is the client class which provides all of the functions to publish messages and subscribe to topics.

If you want to look at the code for this class you should find the code in the client.py file in the mqtt directory. (windows machine)

This directory is located in python34\Lib\site-packages\paho\mqtt

Where python34 is the root of my python install.

mqtt-client-files

Main Client Methods

The paho mqtt client class has several methods.The main ones are:

  • connect() and disconnect()
  • subscribe() and unsubscribe()
  • publish()

Each of these methods is associated with a callback. See Later.

Importing The Client Class

To use the client class you need to import it. Use the following:

Import paho.mqtt.client as mqtt

Creating a Client Instance

The client constructor takes 4 optional parameters, as shown below .but only the client_id is necessary, and should be unique.

Client(client_id=””, clean_session=True, userdata=None, protocol=MQTTv311, transport=”tcp”)

To create a instance use:

client =mqtt.Client(client_name)

See Working with Client objects for more details

Connecting To a Broker or Server

Before you can publish messages or subscribe to topics you need to establish a connection to a broker.

To do this use the connect method of the Python mqtt client.

The method can be called with 4 parameters. The connect method declaration is shown below with the default parameters.

connect(host, port=1883, keepalive=60, bind_address="")

Note: You only need to supply the broker name/IP address.

The general syntax is

client.connect(host_name)

See Working with Client Connections for more details.

Publishing Messages

Once you have a connection you can start to publish messages.

To do this we use the publish method.

The publish method accepts 4 parameters. The parameters are shown below with their default values.

publish(topic, payload=None, qos=0, retain=False)

The only parameters you must supply are the topic, and the payload.

The payload is the message you want to publish.

The general syntax is:

client.publish("house/light","ON")

Example Python Script:

We are now in a position to create our first Python Script to Publish a message.

The script below publishes the message OFF to topic house/main-light


import paho.mqtt.client as mqtt #import the client1
broker_address="192.168.1.184" 
#broker_address="iot.eclipse.org" #use external broker
client = mqtt.Client("P1") #create new instance
client.connect(broker_address) #connect to broker
client.publish("house/main-light","OFF")#publish

Note: I am using my own local broker but you can use an online broker like the one at iot.eclipse.org.

Subscribing To Topics

To subscribe to a topic you use the subscribe method of the Paho MQTT Class object.

The subscribe method accepts 2 parameters – A topic or topics and a QOS (quality of Service) as shown below with their default values.

subscribe(topic, qos=0)

We will now subscribe to topics and in this example we will subscribe to the topic house/bulb1 which is also the same topic that I’m publishing on.

Doing this lets us see the messages we are publishing but we will need to subscribe before we publish.

So our script outline becomes.

  1. Create new client instance
  2. Connect to broker
  3. Subscribe to topic
  4. Publish message

Our new example script is shown below, and I have inserted some print statements to keep track of what is being done.


import paho.mqtt.client as mqtt #import the client1
broker_address="192.168.1.184" 
#broker_address="iot.eclipse.org"
print("creating new instance")
client = mqtt.Client("P1") #create new instance
print("connecting to broker")
client.connect(broker_address) #connect to broker
print("Subscribing to topic","house/bulbs/bulb1")
client.subscribe("house/bulbs/bulb1")
print("Publishing message to topic","house/bulbs/bulb1")
client.publish("house/bulbs/bulb1","OFF")

If we run the script this is what we see:

intro-mqtt-python-script

So where is the message that I published?

When a client subscribes to a topic it is basically telling the broker to send messages to it that are sent to the broker on that topic.

The broker is ,in effect, publishing messages on that topic.

When the client receives messages it generate the on_message callback.

To view those messages we need to activate and process the on_message callback.

Aside: Callbacks are an important part of the Python Client and are covered in more detail in Understanding Callbacks.

Callbacks also depend on the client loop which is covered in Understanding the Client Loop.

However at this stage it may be better to just except them and proceed with the script.

To process callbacks you need to:

  1. Create callback functions to Process any Messages
  2. Start a loop to check for callback messages.

The client docs describe the on_message callback and the parameters it excepts.

Here is my callback function, which basically just prints the received messages:

def on_message(client, userdata, message):
    print("message received " ,str(message.payload.decode("utf-8")))
    print("message topic=",message.topic)
    print("message qos=",message.qos)
    print("message retain flag=",message.retain)

Note the message parameter is a message class with members topic, qos, payload, retain.

I.e message.topic will give you the topic.

Now we need to attach our callback function to our client object as follows:

client.on_message=on_message        #attach function to callback

and finally we need to run a loop otherwise we won’t see the callbacks. The simplest method is to use loop_start() as follows.

client.loop_start()    #start the loop

We also need to stop the loop at the end of the script (loop_stop()), and in addition wait a little to give the script time to process the callback, which we accomplish using the time.sleep(4) function.

This what our completed example script now looks like:


import paho.mqtt.client as mqtt #import the client1
import time
############
def on_message(client, userdata, message):
    print("message received " ,str(message.payload.decode("utf-8")))
    print("message topic=",message.topic)
    print("message qos=",message.qos)
    print("message retain flag=",message.retain)
########################################
broker_address="192.168.1.184"
#broker_address="iot.eclipse.org"
print("creating new instance")
client = mqtt.Client("P1") #create new instance
client.on_message=on_message #attach function to callback
print("connecting to broker")
client.connect(broker_address) #connect to broker
client.loop_start() #start the loop
print("Subscribing to topic","house/bulbs/bulb1")
client.subscribe("house/bulbs/bulb1")
print("Publishing message to topic","house/bulbs/bulb1")
client.publish("house/bulbs/bulb1","OFF")
time.sleep(4) # wait
client.loop_stop() #stop the loop

If you run the script you should see the following

intro-mqtt-python-script-1

Note: logically you should be able to start the loop before you create a client connection, but it you do then you get unexpected results.

Useful Exercises

You should try commenting out, one by one, the lines:

  • client.on_message=on_message
  • client.loop_start()
  • client.Loop_stop()

and run the script to see the results.

Troubleshoot using Logging

To help troubleshoot your applications you can use the built in client logging callback.

To use it you create a function to process the logging callback. My function is shown below and it simply prints the log message.

def on_log(client, userdata, level, buf):
    print("log: ",buf)

and then attach it to the callback:

client1.on_log=on_log

You should then see details of connections,publish and subscribe messages like that shown below:

mqtt-client-logging

The above is a quick overview to get started you can find out more details in the tutorials below:

Video

Here is a video that covers the main points above- Using the Paho Python MQTT client.

Common Problems

1. Not seeing any messages or not seeing all expected messages.

Possible causes

  1. You haven’t started a network loop or called the loop() function. Or you haven’t registered or created the callback functions.
  2. You haven’t subscribed to the correct topics or subscription has failed.
  3. Access restrictions are in place.

2.- My messages don’t appear in the order I expected?

Possible causes

  1. The callback functions are async functions which can be called at any time. Use a queue to store the messages and print in one place. I use the Python logging module.

Related Tutorials and Resources

Facebooktwittergoogle_plusredditpinterestlinkedinmail

60 comments

  1. Hi Steve,

    Great article! Very helpful. In the comments section you wrote:

    The on_message callback is what triggers the storing. I wrote a logging script but I used a different thread to store the data so as not to block the main script.
    The on_message callback just drops the message in a queue that gets processed by the storage thread.
    If you want I will send it to you so you can take a look.
    as for the 1000 messages I think it is too many as it could be several hours or days depending on how active the topics are.

    Can you send me that code?

    Thanks,
    Rich

  2. Hi Steve.

    I have garden lights and internal lights that have sonoff smart switches connected but they have been flashed with tasmota. This gives off mqtt results. I have set up a broker with paho-mqtt and your script above in python works and I can turn on and off the lights with a python script. I would love to add the status of each light on a webpage and even be able to turn it on and off from my webpage.

    I have a pi nginx webserver which runs my paho-mqtt/mosquitto on too.

    Do you know how to get this info on to a webpage ?

    Thanks in advance

    1. There are many ways of doing this but I assume you want a python solution running locally.
      If so then I have script that was developed for something else but will do what you want. Use the ask steve page and contact me and I’ll send it to you.
      You might also want to consider node-red as it makes it very easy to display data. You could also use it for control.
      If you need to do this across the Internet then thingsboard will also do it but I’ve only used that for display.
      You might find these videos useful
      Node-red
      https://youtu.be/e70ta8jI_nM
      https://youtu.be/Gu0Vq2kVNzw
      Thingsboard
      https://youtu.be/eqbTNXf2m7s

  3. Hi
    Is there any way i can get the list of all clients and their IPs , connected to broker, by altering the same code. Please mention if there is any other way of doing this.
    thanks.

  4. Using the on_message callback in paho, when i’m subscribed to a topic. i’m trying to display a message if the last message received was over 15 seconds earlier. I’m trying to control a relay based and if i get flooded with messages, the relays opens and closes nonstop.

    Any idea on how to do this?

  5. Hi Steve,

    In my callback function, I want to store the messages in the database. I want to store the messages in two databases parallely. Is it okay to create and run threads in my callback? Suppose the threads which I created take more time, will it affect client.loop_forever()

    client.on_message= onMessage #attach function to callback
    client.loop_forever()

    def onMessage(client, userdata, message):
    t1 = Thread(target = saveToDatabase ,args=(msg))
    t2 = Thread(target = saveToDatabase ,args=(msg))
    t1.start()
    t2.start()

    1. It is probably better to start a thread for the databases outside the on_message callback and keep it running in the background.
      In the on_message callback drop the message in a queue and let the database thread process the queue.
      I’ve got code that does it. It’s not tidy but you are welcome to it.Just use the ask steve page to send me your email address

  6. I am quite new to MQTT. I was wondering if we could do a video conferencing using MQTT. If yes, could you please explain in detail how abouts of the process?

    Thanks in advance

    1. Probably but MQTT isn’t designed for it and so I doubt if the quality would be any good. I can imagine that one day security cameras might use it but I haven’t really given much thought to it.
      Sorry I can’t be of more help.
      rgds
      steve

  7. Hi Steve,

    I have written a script for the python client which works perfectly. Now I need to extend my application wherein I need to pass the client object within multiple python scripts to subscribe and publish to different topics.

    Would appreciate if you could help on how can this be done.

    TIA

  8. Hi steve,

    I tried with the script given in this tutorial to connect to broker and I get this error “socket.error: [Errno 111] Connection refused” whenever I try to connect to broker. I am using python version 2.7.12

    import paho.mqtt.client as mqtt #import the client1
    import time
    ############
    def on_message(client, userdata, message):
    print(“message received ” ,str(message.payload.decode(“utf-8”)))
    print(“message topic=”,message.topic)
    print(“message qos=”,message.qos)
    print(“message retain flag=”,message.retain)
    ########################################
    broker_address=”iot.eclipse.org”
    print(“creating new instance”)
    client = mqtt.Client(“P1”) #create new instance
    client.on_message=on_message #attach function to callback
    print(“connecting to broker”)
    client.connect(broker_address) #connect to broker
    client.loop_start() #start the loop
    print(“Subscribing to topic”,”house/bulbs/bulb1″)
    client.subscribe(“house/bulbs/bulb1”)
    print(“Publishing message to topic”,”house/bulbs/bulb1″)
    client.publish(“house/bulbs/bulb1″,”OFF”)
    time.sleep(4) # wait
    client.loop_stop() #stop the loop

    Output:
    creating new instance
    connecting to broker
    Traceback (most recent call last):
    File “mqttclient.py”, line 16, in
    client.connect(broker_address) #connect to broker
    File “/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py”, line 768, in connect
    return self.reconnect()
    File “/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py”, line 895, in reconnect
    sock = socket.create_connection((self._host, self._port), source_address=(self._bind_address, 0))
    File “/usr/lib/python2.7/socket.py”, line 575, in create_connection
    raise err
    socket.error: [Errno 111] Connection refused

  9. Greetings Steve.
    Great tutorial steve
    I want to create a sensor network using 6 esp8266( in which 4 esp8266 are for sensor and 2 esp8266 for actuators) to send sensor data to mosquito broker hosted in raspberry pi.So l want to create python script which can store all sensor data in mysql database and the python script should compare the received sensor values with the threshold value so that it publish control messages to actuators esp8266.
    Can you help me with how best can i do it..if you have any python scriptd related to this ,can you help. I need a starting point.i am a newbi to python and mqtt.
    Stay well

  10. I have subscribed to multiple topics using paho mqtt client. On receiving the messages from broker, I want to store the messages into mysql database. I want to gather the messages altogether before inserting into DB.I have set the threshold say 1000 messages. Only when the threshold is reached the messages has to be inserted into DB altogether at once.For each topic, there is corresponding table in the database. Which callback function Should I use on_message() callback or message_callback_add()? Which is better in such scenario? Is there a chance to lose the message because I am collecting the messages in bulk and after some time delay storing it in the database.?

    //MySQLDBClass.py

    def __init__(self):
    self.insertcounter = 0
    self.insertStatement = ”
    self.bulkpayload = ”
    self.maxInsert = 1000

    def onMessage(self, client, userdata, msg):
    if msg.topic.startswith(“topic1/”):
    self.bulkpayload += “(” + msg.payload.decode(“utf-8”) + “,” + datetime + “),”
    elif msg.topic.startswith(“topic2/”):
    self.insertStatement += “INSERT INTO mydatabase.table1 VALUES (” + msg.payload.decode(“utf-8”) + “,” + datetime + “);”
    elif msg.topic.startswith(“topic3/”)
    self.insertStatement += “INSERT INTO mydatabase.table2 VALUES (” +msg.payload.decode(“utf-8”) + “,” + datetime + “);”
    elif msg.topic.startswith(“messages”):
    self.insertStatement += “INSERT INTO mydatabase.table3 VALUES (‘” + msg.topic + “‘,” + msg.payload.decode(“utf-8”) + “,” + datetime + “);”
    else:
    return # do not store in DB

    self.insertcounter += 1

    if ( self.insertcounter > self.maxInsert ):
    if ( self.bulkpayload != ” ):
    self.insertStatement += “INSERT INTO mydatabase.table4 VALUES” + self.bulkpayload + “;”
    self.bulkpayload = ”
    for result in cursor.execute(insertStatement, multi=True):
    print result.row_count

    1. The on_message callback is what triggers the storing. I wrote a logging script but I used a different thread to store the data so as not to block the main script.
      The on_message callback just drops the message in a queue that gets processed by the storage thread.
      If you want I will send it to you so you can take a look.
      as for the 1000 messages I think it is too many as it could be several hours or days depending on how active the topics are.

          1. Okay Thank you so much. Suppose, I want to log the number of messages received by on_message callback during a time span of 60 seconds. For this should I create a separate own thread outside the on_message callback to log every minute? or I can add time.sleep(60) inside on_message callback? I want to log the number of messages received in time span before sending to the database and log number of messages inserted into database during that time span.

          2. Don’t use time.sleep(). Just keep track of time in the database thread and do write every 60 seconds.
            e.g if lastimewrite-time.time()>60
            In the on_message callback just drop the messages into a list or use the queue module.

  11. I want to ask is it necessary to use client.loop_start() and client.loop_stop()? Can I use while True? If I use client.loop_start(), can I exclude the while True loop that is running my Raspberry Pi client program? Thanks.

  12. Hi Steve,
    I am trying my first steps with Python and MQTT and maybe you can give me a hint for my beginner problem.

    How can i access to message.payload from outside the “def” part?
    Concrete i want to use an “if query”, which reacts on a special keyword published by a second client.

    Thank you!

    1. I assume you mean that you want to access it outside the on_message callback.
      def on_message(client, userdata, msg):
      topic=msg.topic
      m_decode=str(msg.payload.decode(“utf-8″,”ignore”))

      You could use a global variable to hold the message. Using globals is frowned upon but easy to implement
      def on_message(client, userdata, msg):
      global m_decode
      topic=msg.topic
      m_decode=str(msg.payload.decode(“utf-8″,”ignore”))

      This could be overwritten before you get to process it. A better option is to
      append it to a list and pop the message of the list anywhere in the script.
      def on_message(client, userdata, msg):
      topic=msg.topic
      m_decode=str(msg.payload.decode(“utf-8″,”ignore”))
      message_list.append( m_decode)

      You could also use the queue class which is probably the better solution especially if you are using threads.

      1. I too have some problems processing the incoming payload.
        If I try:
        def on_message(mosq, obj, msg):
        global m_payload
        print “Topic: ” + str(msg.topic)
        print “QoS: ” + str(msg.qos)
        print “Payload: ” + str(msg.payload)
        m_payload=str(msg.payload)
        if m_payload==’ON’:
        GPIO.output(LED1,True)
        if m_payload==’OFF’:
        GPIO.output(LED1,False)

        I get the error: “NameError: global name ḿ_payload’ is not defined ”
        that error is triggered by my “if m_payload==’ON’: statement.
        if I exactly follow your definition with “m_payload=str(msg.payload.decode(“utf-8″,”ignore”))”
        then I get an even bigger error:
        “SyntaxError: Non-ASCII character ‘\xe2’ in file mqttpublish.py on line 32, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details”

        That link however doesnot make anything clear.
        So my question remains: how to process the payload message??
        Any help is appreciated

        1. You are using python2 by the looks of your code.
          Maybe the payload is already a string so try
          m_payload=msg.payload and see if it works.
          If not try to get the payload type and let me know what it is.
          rgds
          steve

        2. OK solved like this:

          def on_message(mosq, obj, msg):
          if (msg.payload==’ON’):
          GPIO.output(LED1,True)
          print ‘lamp aan’
          print “Topic: ” + str(msg.topic)
          print “QoS: ” + str(msg.qos)
          if (msg.payload==’OFF’):
          GPIO.output(LED1,False)
          print ‘lamp uit’
          print “Topic: ” + str(msg.topic)
          print “QoS: ” + str(msg.qos)

          1. Sadly in my solution the indents have disappeared and indents being are an important part of the code, so I will try to clarify:
            The two “if” blocks need to be indented from the “def on_message()” statement
            and the GPIO.outlet print “QoS” blocks need to be indented from the “if” statements

  13. Thanks, this is really simplified.
    How do I split the publisher and subscriber entities into two separate files?
    Publishing from one end and subscribing on the other receiving published info on it?

  14. hey , thanks for the tutorial. I would like to post an issue which I came across in mqtt.
    I am running a client which connects to the broker running in the same PC using connect function. Then am running loop_start() function and then subscribing to the topic and getting the message . so far so good. Now I want to get disconnected from the broker , so am using disconnect() function. But the Disconnect function returns the value ‘4 ‘ and it is not calling any on_disconnect function. Can you look into this and resolve my issue .
    Thanking You

  15. I faced problem I cant cope with. Executing a script:

    #!/usr/bin/python
    import paho.mqtt.client as mqtt
    server = “192.168.88.20”
    port = 1883
    user = “jXXXX”
    passw = “XXXX”
    client = mqtt.Client(client_id=”Rpi3″, clean_session=True, userdata=None, protocol=’MQTTv311′)
    client.username_pw_set(user,passw)
    client.connect(server)
    client.publish(“home/rpi3″,”Hello”, qos=0, retain=False)

    I got:
    Traceback (most recent call last):
    File “mqqt.py”, line 10, in
    client.connect(server)
    File “/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py”, line 760, in connect
    return self.reconnect()
    File “/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py”, line 932, in reconnect
    return self._send_connect(self._keepalive, self._clean_session)
    File “/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py”, line 2114, in _send_connect
    keepalive))
    struct.error: cannot convert argument to integer

    Somebody can help?

    1. The broker address is the address of the machine running the NQTT broker. You can use public brokers like “test.mosquitto.org”,”broker.hivemq.com”,”iot.eclipse.org”.

      1. If i’m using my PC as a localhost for the broker then the address should be my pc ip address, am i right? Or can i just type in localhost when i assign the value into the broker_address variable?

  16. Hi Steve,
    I have a few question, hope you could help me out.
    Q1. Which party generates the Callback? regarding the “on_connect()”. It looks like the client generates after it gets ACK message from broker. Is my understanding correct?
    How about “on_message()”

    Q2. I am not clear of the meaning of
    “def on_connect(Client, userdate, flag, rc):”
    “client.on_connect=on_connect”

    It looks On_Connect() has been defined in client module already. To me , it just need to check the value of “rc” to make judgment.

    Thanks very much!

    corydon

    1. Yes you are correct the CONNACK triggers the callback (on_connect).
      The client.on_connect() method is defined in the MQTT class.
      the statement
      client.on_connect=on_connect
      replaces the method defined in the class with the one you define in your script called on_connect().
      Does that Make Sense?
      Rgds
      Steve
      P.S I just did a video on client connections which you may find helpful
      https://youtu.be/vT4FTRgipOM

  17. This was a massive help to me, just starting with mqtt (and Python) and this really got things moving. Thank you.

  18. Hey! Amazing tutorial. Thank you.

    I am trying out some of the things that you have shown. I am able to subscribe to a topic, and have the message show up in my Python console. So the loops, and the call back configuration seems to be all set.

    I am trying to compare the message that I am receiving to a known string. How do I do that?
    So basically, I need message.payload.decode(“utf-8”) to be equal to a string variable(that’s just variable for Python I guess) and then in an if-statement inside the loop, I am trying to get the code to compare it for every iteration of the loop to a string. Once it is equal, the if statement will run clientName.disconnect() and .loop_stop()

    I tried, under the def on_message, to do that by doing this:
    messCon = str(message.payload.decode(“utf-8”)) and also down inside the loop. At both places it is either “Invalid syntax” or that my variable ‘messCon’ is not defined.

    I am at a loss on how to get this to work.

    Any help is appreciated.

    -Sohum

    1. Hi
      that looks about right it may just be the if statement missing the ==

      if messConn==str(message.payload(“uft-8”):
      clientName.disconnect()
      loop_stop()

  19. Hello,
    i’m trying to write o code which gets tempreture value and write it to a file. But my code getting stuck if there is no broker and then when i try to execute code with broker code does not give any response. i’m using loop_start() func. And i understood that there is a thread on the backround working. So is there a way to stop loop properly when there is no broker or stop the code with “ctrl + c”. Thank you

    1. Hi
      When the broker disconnects you get a on_disconnect callback.
      Put the loop_stop() call in there.
      Here is a copy of my on_dicconect()
      def on_disconnect(client, userdata, rc):
      m=”disconnecting reason ” ,str(rc)
      logging.info(m)
      client.connected_flag=False
      client.disconnected_flag=True

      Note: I use a flag to stop the loop because I normally don’t use the loop_start.
      So yous would be:
      def on_disconnect(client, userdata, rc):
      loop_stop()

      rgds
      steve

Leave a Reply

Your email address will not be published. Required fields are marked *