Paho Python MQTT Client-Understanding The Loop

When writing code using the Paho Python client you would have had to use the loop() function .

In this tutorial we will look at what it does, and why it is necessary.

When new messages arrive at the Python MQTT client they are placed in a receive buffer.

The messages sit in this receive buffer waiting to be read by the client program.


You could program the client to manually read the receive buffers but this would be tedious.

The loop() function is a built in function that will read the receive and send buffers, and process any messages it finds.

On the receive side it looks at the messages, and depending on the message type, it will trigger the appropriate callback function. See Understanding callbacks.

For example if it sees a CONNACK message it triggers the on_connect() callback.

Now instead of manually reading the receive buffer you just need to process the callbacks.

Outgoing messages and message acknowledgements are placed in the send buffer.

The loop function will read this buffer and send any messages it finds.

Calling the Loop Function

See network loop in docs for function reference.

The Paho Python client provides three methods:

  • loop_start()
  • loop_forever() and
  • loop().

The loop_start() starts a new thread, that calls the loop method at regular intervals for you. It also handles re-connects automatically.

To stop the loop use the loop_stop() method.

The loop_forever()  method blocks the program, and is useful when the program must run indefinitely.

The loop_forever() function also handles automatic reconnects.

The loop can be stopped by calling loop.stop().

You should stop the loop before you exit the script.

You can also manually call the loop() method in your program.

If you do this you must remember to call it regularly.

That is it must be in a loop. e.g pseudo code below:


Some code

client.loop(.1) #blocks for 100ms

some code

Because the loop is a blocking function I call it with a timeout the default timeout is 1 second.

If you call the loop manually then you will need to create code to handle reconnects.

Important! If your client script has more than one client connection then you must call or start a loop for each client connection.

For example, if I create two clients client 1 and client2 in a script, then you would expect to see client1.loop() and client2.loop() in the script.

Implementation Note:

I have experienced strange behaviour when starting a loop before creating a connection . So

client= mqtt.Client(cname)

Works Ok but

client= mqtt.Client(cname)

sometimes gives strange results.

Stopping the loop automatically

If you are using the loop_start() function then you will probably need to stop the loop automatically if the connection fails.

The easiest way of doing this is using the on_disconnect callback.

def on_disconnect(client, userdata,rc=0):
    logging.debug("DisConnected result code "+str(rc))

However you should only stop the loop if you are completely finished, and are going to exit.

Stopping the loop will stop auto reconnects unless you take steps to call the loop manually.

Loop_start vs Loop_forever

Loop_start starts a loop in another thread and lets the main thread continue if you need to do other things in the main thread then it is important that it doesn’t end.

To accomplish this you need to use your own wait loop.

The loop_forever call blocks the main thread and so it will never terminate.

The loop_forever call must be placed at the very end of the main script code as it doesn’t progress beyond it.

To terminate the script you will need to use a callback function to disconnect the client.

Handling Multiple Clients

If your script connects using multiple clients then each client will need a loop.

Therefore if you are using the loop_start() method then you will need to call it for each client connection.

Therefore if you have 2 client connections you will need two loops which equals two additional threads.

For 2 clients it isn’t really a problem, but what if you have several hundred client connections then you will need several hundred additional threads.

In this situation it is better to manually call the loop for each client and use thread pooling.

I will cover this is another tutorial at a later date.

However for up to around 20 client connections then is is easier to use the inbuilt loop_start and stop functions.

To make it simpler add the clients to a list and loop through the list to start the loops and the same to stop e.g

import paho.mqtt.client as mqtt
#create clients
for i  in range(nclients):
   client= mqtt.Client(cname)
for client in clients:

Common Questions and Answers

Q- My callbacks aren’t being called. Why is that?

A- You need to start a loop or call the loop() function to process callbacks.

Q- My script has ended but the loop still appears to be running?

A- It probably is. You need to stop the loop before exiting the script.

Q- Why call the loop manually when you can use loop_start()?

A-The main reason is when the script has lots of client connections (>20) as if you use loop_start() then you need a loop for each client.


The built in loop functions will process the receive and send buffers to send and receive messages and message acknowledgements.

To make life easier use the loop_start and stop functions unless you have 100’s of client connections.

Course Links

  1. Introduction to the Paho Python MQTT Client
  2. Introduction to the Client Class
  3. Connecting to a Broker
  4. Publishing Using The Paho Python MQTT Client
  5. Subscribing using The Paho Python Client
  6.  Receiving Messages with the Paho MQTT Python Client
  7. Understanding The Loop
  8. Understanding Callbacks
  9. Handling Multiple Client Connections
Please rate? And use Comments to let me know more


  1. Hi Steve
    We are using a Pyhton script as a Middleware in between a Twincat SPS and a cloud-based Platform.
    From the Twincat SPS we are sending and receiving Data via OPC UA and sending it to the Platform with MQtt. We’ve managed to get this working quite well, but there is one problem we can’t solve.
    When the connection to the OPC UA Server is lost or closed, we are getting an Error in the Program but the loop for the MQtt Client is still running. Is there a way to stop the loop as soon as an error in the main program occurs? The part of the Error where the Error occurs is in a while true loop.

      1. The Python Script doesn’t exit as we are not getting an exit code. The Connection to the MQtt Broker happens bevor the error in the program occurs. Therefore, the Loop doesn’t stop, but the program does. It would be perfect if there would be an option to stop the loop when an error in the program occurs.

        1. Can you use the ask steve page and send me the script.

          I’m assuming you don’t want to reconnect on disconnect and then you will restart the script manually.

  2. Hi Steve, I want publish data to broker once connected. I would want to the data to be sent every second.

    However, I would also want to stop publishing if connection with broker suddenly failed. Then I want attempt to reconnect indefinitely. Once reconnect success, I would like to continue publishing the data.

    I understand that loop_forever() may be useful here, but how should I approach this ? How do I make publish every second ONLY if connected to broker?

    It will be very helpful if you can explain the flow or pseudocode for this one. Thanks!

    1. I usually set a flag in the on_connected callback and test it before publishing. In the latest version of the client there is an is_connected method that does the same so
      if client.is_connected():

      does that make sense?

      1. I have an issue with that. I tried shutting down my broker and the on_disconnect function does not seem to be called until the broker is restarted again.

        And due to that, the connected_flag is always True until the broker is restarted, then False for awhile and then True again (presumably because reconnected successfully).

        Therefore, I couldn’t publish based on the connected_flag.

        I’ve used loop_start() and a while True block in my code.

        P.S. Most of the code was modified based on your demo file.

        What could be the problem?

  3. Hi, Steve!
    I’m a beginner trying to learn MQTT. I’m so sorry if this is kind of a dumb question for you, but I was wondering what should I do to create pub-sub connection between 2 laptops? Should I run a script to subscribe in one laptop and publish in the other? Would the connection be established automatically if I use “” as my broker, or should I specify something to make sure they can recognize each other?

    Thank you for your helpful tutorial.

    1. Yes each laptop would run a pub_sub client. The choice of broker isn’t important as long as they use the same one. Running your own local broker means that you have access to the broker console which is very useful for debugging and learning.

  4. Hi Steve,
    i am using Mqtt for out IOT device , we have 2(A,B) servers, each one having the flask micro services,
    from server A i am sending the data you can consider as a producer and from server B micro service i am receiving the data , after 1-2 days if we not restarted services B micro service is not receiving the data , if we restart the service A , then all the data is publishing at once because of qos=2

    can you please let me know how to avoid this mqtt not responding state,
    we are using the 2 flask service , when ever our initializes at the same time we are initializing the mqtt
    def init_mqtt():
    “””MQTT client connections.”””
    broker = config.MQTT_BROKER_IP
    CLIENT.connect(broker, 1883, 60)
    CLIENT.subscribe(‘operations_ack’, 2)
    CLIENT.message_callback_add(“operations_ack”, delete_operations)
    except Exception as err:
    log.error(“Mqtt Error in init_mqtt-” + str(err))
    def create_app():
    “”” to create and configure the flask application.”””
    app = Flask(__name__, instance_relative_config=True)
    # Initializing the Mqtt client
    from .common import mqtt_client
    except Exception as err:
    log.error(“error at app creation”)

    1. Sorry
      I’m not really familiar with flask. Not sure about the mqtt not responding state. It seems that from your description it works ok and then stops sending after a few days is that correct.
      Did you check if the messages had been published and queued on the broker or if they are being queued on the sender?

      1. Hi Steve,
        The messages are queued on the broker only, they are not received to subscriber side , but when we restart flask services all messages are sending at once, and i have written a call back for disconnect but it is not calling.

        1. So the subscriber client is hanging? Not sure what you mean by written a callback for disconnect. When are you trying to disconnect? Don’t you want to receive all of the messages?

          1. If it is not being triggered then I guess the connection is still open or at least the client and server think it is.
            Do you see the ping requests?

  5. I am doing my tasks in threads, and using client.loop_forever().
    But when I close my GUI, the loop is still working, any way to stop it

  6. I have been having a lot of trouble with thie loop_forever function.
    I have a project like this:

    def function():
    “let a led blinking a bit”

    def on_message(client, userdata, message):
    if msg == 0:


    The problem is while the function let a led blinking i can’t receive new messages. I would like to check in the function() if theres a new message so that i would now if it shoud run the “blink function” again or should turn for example the led off.
    Are there any solutions.

    Sorry for my English i’m german an i’m still learning.

    1. Hi
      If the function runs forever which I assume it does then you need to test State to decide on the action the function should take.
      The on message callback would be used to set this state but you cannot call the function twice


      def function():
      global state
      if state
      “let a led blinking a bit”

      def on_message(client, userdata, message):
      global state
      if msg == 1:
      if msg == 0:

      Does that help?

  7. Hi,

    I appreciate your content and have created Python code that works perfectly based off of this and your other posts. However, I am running into a situation where MQTT is using 100% of CPU when I use loop_start() and virtually none when I use loop_forever(). Why would it be different? Visit the link below to see very basic sample code:

    A couple of observations:
    1. The code as written pegs the CPU at 100%
    2. If I comment out “loop_start()”, “While True” and “pass”, and uncomment “loop_forever()”, CPU utilization drops to about 2%.

    Why is the CPU usage so different between these two? Am I doing something wrong? Is there a better way to do this that I somehow missed?


    1. It is the while True loop that is causing your problem.
      while True:

      The loopforever() never gets executed while the while loop is in place.
      Use loopforever if the main thread doesn’t do anything and all you do is wait for incoming messages.
      However if you need to do something in hte main loop as well as receive new messages then use the loop_start() and hold the main thread using a while loop e.g.pseudo code
      Initiialize callbacks
      start loop
      start while loop
      code for main loop e.g
      sleep 1 second
      publish message
      #break from loop under a condition could be a received message or when x messages sent
      stop loop

      Hope it makes sense
      stop loop

  8. just a simply question.

    when I receive the payload with my script on_connect; this is sometimes the same as the previous one. I understand that other clients would still need to be able this last message, therefore it is not deleted. But the thing I am not sure, how to deal with duplicates. do I need to take care of that myself?

    1. Hi
      You can get duplicates but it is not common especially on a test network. I would take another look at your script to see if there s a problem with it.
      If you send it to me using the ask-steve page I’ll take a look

  9. Hello Steve,
    thank you for your great work, really helped me understand everything around mqtt.
    However one thing is still kind of blurry, how to keep client receiving data from brooker 24/7?

    Do I Simply create python script which has got
    while True:
    basically without end and run it by cron with @reboot prefix so it’s started immediatelly after machine reboot?
    or how you and you guys actually work with py scripts ? as they simply won’t receive anything when they ended.

    That’s pretty much last piece to my growing home-puzzle 😀


    1. Hi
      Yes that is basically what you do. You could also use the loop_forever() which does the same thing.

      1. thanks 😉
        is it good approach to have like publishing script and subscribing script separated? Am thinking about subscibing script will be basically looping 24/7 but publishing should run once per 2mins as it reads data on regular basis.
        Using homie 3.0 structure i can declare $state offline for publishing and $state ready for subscribing -> action part

        Or do you recommend everyting in one go?

        1. Hi
          If the publish relies on data it receives from the subscribe then it is easier to use a single script otherwise it makes no difference.

  10. hi steve! i have 2 clients connected in 1 broker but the thing is i want to have it to loop indefinitely.


    but it only shows one data of my esp

    1. Hi
      Once you call one loop_forever it basically stops there and so the second is never called.
      while True:

    1. Not really as it usually works but occasionally it doesn’t. Don’t know why. I just encountered the problem a few times and so I always do it that was now.It may be or have been a timing issue.

    2. Sorry sue but I tried it 10 minutes ago and it worked ok. I came across the problem over 18 months ago and all I remember was that the script didn’t work as expected I think it didn’t call the callbacks.
      I put it in the tutorial in case anyone else get the problem as it was difficult to troubleshoot and doesn’t really make sense.

  11. Hi Steve,

    thank you so much for all your tutorials! I find your blog more useful than anything else for working with mosquitto broker and paho clients, really.
    You say that for multiple clients we need multiple connections thus multiple loops, but does this mean that if I want a client to connect to N brokers then I have to create N client objects? Or maybe there’s a way to create N connections for one client. In my case I have a client that is expected to subscribe/pubilsh to different topics on different brokers remaining connected to them.

    Thank you again,

    1. Hi
      1 client 1 connection. A client cannot be connected to several brokers at the same time.
      However you don’t need multiple loops but you do need N client objects.
      What I do is
      create client object
      add client reference to an array
      The I use a loop to work trough the clients and manually call client.loop() within that loop.
      I do have some example code that I wrote a while ago that checks several brokers using that technique. If you contact me on the ask-steve page I’ll email it to you.

  12. How are you getting documentation on what exactly a loop does?
    I’m trying to read more on the official PAHO MQTT PYTHON and it’s very light. I’m having trouble understanding what exactly loop_forever() does and how loop() work in general.

    Does loop() or loop_forever() continually go through main, starting a new thread each time?
    or does it only go through on_connect and on_message? I don’t know why the official documentation not mention the specifics regarding this anywhere 🙁


    def main():
    client = mqtt.Client()
    client.on_connect = on_connect
    client.on_message = on_message
    client.connect(‘host.local’, 1883, 60)
    # Connect to the MQTT server and process messages in a background thread.
    # Main loop to listening
    print(‘Script is running, press Ctrl-C to quit…’)

    if __name__ == ‘__main__’:

    1. Hi
      I looked at the actual client code.
      The loop simply starts a new thread. The job of this thread is simply to look at the send receive buffers and process them if there is data. If there is data it calls the callback function to alert your program.
      Normally there is only one loop and hence 1 additional thread.
      Here is the code for loop start
      def loop_start(self):
      “””This is part of the threaded client interface. Call this once to
      start a new thread to process network traffic. This provides an
      alternative to repeatedly calling loop() yourself.
      if self._thread is not None:
      return MQTT_ERR_INVAL

      self._thread_terminate = False
      self._thread = threading.Thread(target=self._thread_main)
      self._thread.daemon = True

      You can see it starts a new thread and your main program continnues which is why you usually need a while loop in the main program to stop it ending.
      However the loop_forever has a while loop which stops the main program from ending. This is why you need to put it at the end of the main program.
      Once you call loop_forever the second thread will process the callbacks as before but nothing more is happening in the main thread.

      Here are the first few lines of loop_forever
      def loop_forever(self, timeout=1.0, max_packets=1, retry_first_connection=False):
      run = True

      while run:
      if self._thread_terminate is True:

      Does that make more sense?

  13. I’ve found that client.loop_stop() just waits forever. Fortunately in my program it’s acceptable to just terminate the process, otherwise I’d be concerned with why client.loop_stop() doesn’t work.

    1. Hi
      I’ve not experienced any problems with the loop_stop(). It maybe that it wasn’t called in the program or you had multiple loops.
      If you still have problems use the ask steve page and send me the code and I’ll take a look

  14. Hi Steve. Thanks very much for your great series on mqtt which has enabled me to get a nice start on my home automation using node-red. One issue I have as a newbee is that during the loop, paho seems to be catching and not reporting all Python run-time errors. It’s been a challenge to debug my code as things like calling the wrong function name or even a bad string constructor are hidden. Is there a solution for this? Thanks again.

  15. Helo Steve
    I posted a message on one of the topics asking for trouble shooting advise re: connect – disconnect -connect ; would not connect. I forgot where I posted the orginal message.

    I found the poblem and you allready had it documented on this page
    client= mqtt.Client(cname)
    sometimes gives strange results.
    ******** do not do this you will not be able to do the second connect -***
    make sure loop start happens after connect
    and I made sure loop_stop() is called before calling disconnect()
    Thanks for the web page very helpful

  16. I use mqtt paho to connect to ttn in order to receive an upling of two bytes
    It connect correctly and I receive correctly the first message and use loop_forever waiting for the next message scheduled 30sec later. However the script crashe imediatly after this first uplink.
    Any suggestion why ?
    thank in advance for your help,

    python 3.4 on raspberry jessie

    code + crash message :
    def on_connect(mqttc, mosq, obj,rc):
    print(“Connected with result code:”+str(rc))
    mqttc.subscribe(‘+/devices/#’) # subscribe for all devices of user

    # gives message from device
    def on_message(mqttc,obj,msg):
    x = json.loads(msg.payload.decode(‘utf-8’))
    device = x[“dev_id”]
    print (“line23”,device)
    payload_raw = x[“payload_raw”]
    payload_plain = base64.b64decode(payload_raw)
    print (“line26”,payload_plain)
    datetime = x[“metadata”][“time”]

    def on_publish(mosq, obj, mid):
    print(“mid: ” + str(mid))

    def on_subscribe(mosq, obj, mid, granted_qos):
    print(“Subscribed: ” + str(mid) + ” ” + str(granted_qos))

    def on_log(mqttc,obj,level,buf):
    print(“message:” + str(buf))
    print(“userdata:” + str(obj))

    mqttc= mqtt.Client()
    # Assign event callbacks

    mqttc.username_pw_set(APPID, PSW)
    print (“line36”)
    # and listen to server
    run = True
    while run:
    crash message :

    Exception in thread Thread-62:
    Traceback (most recent call last):
    File “/usr/lib/python3.4/”, line 920, in _bootstrap_inner
    File “/usr/lib/python3.4/”, line 868, in run
    self._target(*self._args, **self._kwargs)
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 2650, in _thread_main
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 1481, in loop_forever
    rc = self.loop(timeout, max_packets)
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 1003, in loop
    rc = self.loop_read(max_packets)
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 1284, in loop_read
    rc = self._packet_read()
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 1849, in _packet_read
    rc = self._packet_handle()
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 2305, in _packet_handle
    return self._handle_publish()
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 2500, in _handle_publish
    File “/usr/local/lib/python3.4/dist-packages/paho/mqtt/”, line 2647, in _handle_on_message
    self.on_message(self, self._userdata, message)
    File “”, line 23, in on_message
    device = x[“dev_id”]
    TypeError: string indices must be integers

    1. Hi
      Did you say you received the firstt message OK?
      Take the loop_start and stop outside the while loop and don’t use loop_forever as well as loop_start and see how it goes.
      Use the ask-steve page to send me the python console output if it crashes.

  17. Hi Steve!
    Thank you for this amazing tutorial!
    I have a script that is able to receive MQTT events (I use on_message callback). My client is connected with connect_async method. The problem for me is that the thread responsible for handling this connection waits for one message to be proceeded to be able to get another message from the incoming message buffer. Is it possible somehow to start the message processing as soon as it gets to the buffer without waiting for the previous message to be handled completely? E.g. creating new thread for message handling?

    1. Hi
      Each message will activate the callback and you can only have one callback active for that client connection at the same time so that callback has to complete before it can be triggered again.
      The message can be passed to the main thread using a queue and so it can be processed while the callbacks are active.
      In many of my scripts the on_message callback drops the message in a queue and I unpack the queue in another thread. I can send you an example if you want.

  18. Hi Steve,

    Thanks for the light about threading and multiple clients, there is not that much tutos.
    I have by the way some issues implementing it.

    I have a broker on a gateway where the sensors send messages, and a cloud broker one where the gateway send final results. I have to subscribe to both broker topics.

    There is no other way to thread (cloud base client, multiple sensors clients)? (as the cloud client use web sockets and sensors not)

    I followed your tuto with the thread but I can’t see my messages, and it seems the client.connected_flag=True doesn’t seem to execute ..

    Do you have an exemple to show for the base please ?

    Here is what I’m using for the moment

    def Connect(client, who):
    if who == 0:
    client.on_message = on_scp_message
    client.on_connect = on_scp_connect
    client.on_publish = on_scp_publish
    client.on_subscribe = on_scp_subscribe
    client.username_pw_set(USERNAME, PASSWORD)
    client.connect(ENDPOINT, 443)
    —– And some other settings if who = 1, without certificates etc …

    for i in range(nclients):
    cname = “Client-” + str(i)
    if i == 0: client = mqtt.Client(DEVICE_ID, transport=’websockets’)
    else: client = mqtt.Client(cname)
    ex = futures.ThreadPoolExecutor(max_workers=10)
    while True:
    for i in range(len(clients)):
    if client.connected_flag == False:
    f = ex.submit(Connect, client, i)

    Thanks a lot

    1. Hi
      With only two connections you might just want to start two client loops. However I will send you two scripts to your email address that use threading and futures. They are scripts for testing brokers that I’ve written and may be useful to you.
      I’ll try to put together a tutorial to over multiple client connections in the next few weeks.

      1. Hi Steve,

        I could manage to do multi client connexion thanks to your code you sent me! 🙂

        thanks and cheers

    1. No not in loop_start().There is no need as the 1 second is the maximum time it blocks before returning. As it runs in its own thread it doesn’t make a difference.
      If you call the loop manuallu using
      client.loop() then you can pass in a timeout e.g
      client.loop(.01) will only block for 1/100 of a second.

Leave a Reply

Your email address will not be published. Required fields are marked *