Python MQTT Client Connections- Working with Connections

python-mqtt-client-connectionsThe MQTT client uses a TCP/IP connection to the broker.

Once the connection is established the client can send data to the broker, and the broker can send data to the client as required.

You can consider a TCP/IP connection to be similar to a telephone connection.

Once a telephone connection is established you can talk over it until one party hangs up.

In this tutorial we will look at connecting to an MQTT broker using the paho python mqqt client.

The Connect Method

To establish a connection to an MQTT broker using the Python client you use the connect method of the client object.

The method can be called with 5 parameters (MQTTv3.1). The connect method declaration is shown below with the default parameters.

connect(host, port=1883, keepalive=60, bind_address=””,clean_session=True)

and for MQTTv5 it takes 6 parameters

connect(host, port=1883, keepalive=60, bind_address=””,clean_start=True,properties=None)

The only parameter you need to provide is the host name.

This can be the IP address or domain name of the broker.

Note: you may need to setup other settings like passwords, last will and testament etc before connecting. See Working with the Client object

The connect method is a blocking function which means that your script will stop while the connection is being established.

Note: the protocol version is set when creating the client.

Was The Connection Attempt Successful?

When a client issues a connect request to a broker that request should receive an acknowledgement.

mqtt-client-connection-diagram

The broker acknowledgement will generate a callback (on_connect) in the client.

If you want to be sure that the connection attempt was successful then you will need to setup a callback function to handle this callback before you create the connection.

The function should receive 4 parameters, and can be called anything you want.

I have called mine on_connect().

Here is an example function definition MQTTv3.1:

def on_connect(client, userdata, flags, rc):
    if rc==0:
        print("connected OK Returned code=",rc)
    else:
        print("Bad connection Returned code=",rc)

The client is a client object.

rc (return code) is used for checking that the connection was established. (see below).

Note: It is also common to subscribe in the on_connect callback

MQTTv5 has an additional properties parameter

on_connect(client, userdata, flags, reasonCode, properties=None)

Connection Return Codes

  • 0: Connection successful
  • 1: Connection refused – incorrect protocol version
  • 2: Connection refused – invalid client identifier
  • 3: Connection refused – server unavailable
  • 4: Connection refused – bad username or password
  • 5: Connection refused – not authorised
  • 6-255: Currently unused.

Flags and userdata aren’t normally used .The full documentation is here.

Note: MQTTv5 has more reason codes see MQTTv5 New features

Processing The On_connect Callback

To process the callback you will need to run a loop. (see understanding the network loop) .

Therefore the script generally looks like this.

  1. Create Client object.
  2. Create callback function on_connect()
  3. Bind callback to callback function (on_connect())
  4. Connect to Broker.
  5. Start a loop.

Because the callback function is asynchronous you don’t know when it will be triggered.

What is sure however is that there is a time delay between the connection being created, and the callback being triggered.

It is important that your script doesn’t proceed until the connection has been established.

For quick demo scripts I use time.sleep() to wait, and give the connection time to be established.

However for working scripts I process the callback and use it to flag a successful or unsuccessful connection.

So instead of this:

  1. Create connection
  2. Publish message

We have this:

  1. Create connection
  2. Verify successful connection or quit
  3. Publish message and or subscribe

Here is some example code that uses time.sleep() to wait, and give the connection setup time to complete:

import paho.mqtt.client as mqtt    #import client library
def on_connect(client, userdata, flags, rc):
   if rc==0
      print("connected ok")
client = mqtt.Client(“python1”)             #create new instance 
client.on_connect=onconnect  #bind call back function
client.connect(broker_address)               #connect to broker
client.loop_start()  #Start loop 
time.sleep(4) # Wait for connection setup to complete
Other code here
client.loop_stop()    #Stop loop 

Enhancing the Callback

To get better control of the connection I use a flag in the on_connect callback.

The flag I create as part of the client object so it is available throughout the script.

client.connected_flag=False

At the start of the script I set this flag (connected_flag) to False and toggle it to True when the Connection is successful, and back to False when we get a disconnect.

def on_connect(client, userdata, flags, rc):
    if rc==0:
        client.connected_flag=True #set flag
        print("connected OK Returned code=",rc)
        #client.subscribe(topic)
    else:
        print("Bad connection Returned code= ",rc)

We can now use this flag to create a wait loop.

client.connect(broker_address)      #connect to broker
while not client.connected_flag: #wait in loop
     time.sleep(1)

Note: from version 1.6 there is an is_connected() method

Example Client Connection Script

The following script is a basic client connection script


#!python3
import paho.mqtt.client as mqtt  #import the client1
import time

def on_connect(client, userdata, flags, rc):
    if rc==0:
        client.connected_flag=True #set flag
        print("connected OK")
    else:
        print("Bad connection Returned code=",rc)

mqtt.Client.connected_flag=False#create flag in class
broker="192.168.1.184"
client = mqtt.Client("python1")             #create new instance 
client.on_connect=on_connect  #bind call back function
client.loop_start()
print("Connecting to broker ",broker)
client.connect(broker)      #connect to broker
while not client.connected_flag: #wait in loop
    print("In wait loop")
    time.sleep(1)
print("in Main Loop")
client.loop_stop()    #Stop loop 
client.disconnect() # disconnect

If I run this script this is what I see:

basic-connection-script-example-run

Failed Connection Examples

There are various conditions were the connection can fail to complete. They are:

  • Incorrect client settings e.g. bad password..
  • No network connection
  • Bad Network Connection parameters e.g. bad port number

It is important that these are detected and handled by the connection script.

We are going to look at a few of these and modify our connection code to detect them.

Note: For these examples I will use the Paho MQTT client and the Mosquitto broker.

Connection Failures that Create an Exception

Trying to connect to a broker using a bad IP address or port number will generate a socket error, and raise an exception.

So the first screen shot shows the result of using bad port number.

mqtt-failed-connection-bad-port

This causes a Winsock error in Windows

In Python we can use a Try block to catch this so instead of

client.connect(broker,port) #connect to broker

We use


try:
    client1.connect(broker,port) #connect to broker
except:
    print(“connection failed”)
    exit(1) #Should quit or raise flag to quit or retry

When the connection attempt failed we would see:

mqtt-failed-connection

Connection Failures Detected Through Return Code

To determine if the connection was successful we need to examine the return code of the on_connect callback.

A return code of 0 is successful, whereas other values indicate a failure.

In the example below we will try to connect to a broker without providing the required authentication.

mqtt-failed-connection-example-authentication

Notice the connection fails and returns a return code of 5 which indicates authentication failure.

You should also notice that because I am using the loop_start() function the client will try to reconnect, but this is pointless as the result will be the same.

So our code should :

  1. Stop the loop
  2. Stop the script

We can stop the loop in the on_connect callback. However to stop the main script we need to set a flag that we can use to exit.

I prefer to use a flag and stop the loop as part of the main script.

Here is what the modified on_connect callback looks like:

def on_connect(client, userdata, flags, rc):
    if rc==0:
        client.connected_flag=True #set flag
        print("connected OK")
    else:
        print("Bad connection Returned code=",rc)
        client.bad_connection_flag=True

Here is the main script modifications to quit.

mqtt.Client.bad_connection_flag=False #
while not client.connected_flag and not client.bad_connection_flag: #wait in loop
    print("In wait loop")
    time.sleep(1)
if client.bad_connection_flag:
    client.loop_stop()    #Stop loop
    sys.exit()

When I run the script this is what I see:

 

bad_connection-example-auth
Using Authentication

If the broker requires username and password authentication (see Mosquitto username and password authentication ) then you need to set this before connecting.

This you do using the username_pw_set() helper function. e.g

client.username_pw_set(username="steve",password="password")
# now can connect

Connecting Using Websockets

Normally the python client will connect using MQTT but it can also connect using MQTT over websockets.

To tell the client to use websockets instead of MQTT use the command

client= paho.Client(“cname”,transport=’websockets’)

instead of simply

client= paho.Client(“cname”)

You will also need to change the port..Websockets generally uses port 9001.

See MQTT over Websockets for more details

Handling Disconnects and Reconnects

A client can disconnect gracefully, if it has no more data to send by sending a disconnect message.

The Paho client provides the disconnect method for this.

It can also get disconnected due to a bad network connection.

If the connection fails for some reason then you will need to decide whether or not you should try to reconnect.

mqtt-connection-Failure

A disconnect triggers the on_disconnect callback which you will need to examine.

This callback takes 3 parameters:

Client- Client object that disconnected

Userdata- user defined data not often used

Return Code (rc)- Indication of disconnect reason. 0 is normal all other values indicate abnormal disconnection

Here is the on_disconnect() code I use:

def on_disconnect(client, userdata, rc):
    logging.info("disconnecting reason  "  +str(rc))
    client.connected_flag=False
    client.disconnect_flag=True

You can see that I simply log it, and then set flags that can be used by the main program to detect the disconnect.

Note: You will need to be calling, or running a loop to trigger the callback.

Reconnecting

Generally you will need to reconnect as soon as possible.

If you run a network loop using loop_start() or loop_forever() then re-connections are automatically handled for you.

A new connection attempt is made automatically in the background every 3 to 6 seconds.

If you call the loop() function manually then you will need to handle the re-connection attempts yourself. See understanding the loop.

You can do this by using a connection flag that is toggled by the on_connect and on_disconnect callbacks.

Client Connection Summary

Taking into account the above our client connection code should.

  • Connect to broker
  • Examine connection status and proceed if good
  • If connection status is bad attempt retry and or quit.
  • Handle disconnects and reconnects

Common Problems

1 .Not seeing any messages or not seeing all expected messages.

Possible causes

  1. You haven’t started a network loop or called the loop() function. Or you haven’t registered or created the callback functions.
  2. You haven’t subscribed to the correct topics or subscription has failed.
  3. Access restrictions are in place. See ACLs

2.- My messages don’t appear in the order I expected?

Possible causes

  1. The callback functions are async functions which can be called at any time. Use a queue to store the messages and print in one place.

Video- MQTT client connections Using The Python MQTT Client

Grateful if you would provide feedback as it will help with subsequent videos.

Course Links

  1. Introduction to the Paho Python MQTT Client
  2. Introduction to the Client Class
  3. Connecting to a Broker
  4. Publishing Using The Paho Python MQTT Client
  5. Subscribing using The Paho Python Client
  6.  Receiving Messages with the Paho MQTT Python Client
  7. Understanding The Loop
  8. Understanding Callbacks
  9. Handling Multiple Client Connections

Related tutorials and Resources

Please rate? And use Comments to let me know more

84 comments

  1. Hi Steve, can you help with this error please:
    >>> import paho.mqtt.client as mqtt
    >>> import random
    >>> import time
    >>>
    >>> # define una función para conectarse al broker MQTT
    >>> def on_connect(client, userdata, flags, rc):
    … print(“Conectado al broker con código de resultado: ” + str(rc))

    >>> # crea un cliente MQTT y establece la función de conexión
    >>> client = mqtt.Client()
    >>> client.tls_set(ca_certs=”/etc/mosquitto/ca_certificates/ca.crt”,
    … certfile=”/etc/mosquitto/certs/server.crt”,
    … keyfile=”/etc/mosquitto/certs/server.key”)
    >>> client.on_connect = on_connect
    >>>
    >>> # conecta el cliente al broker MQTT
    >>> client.connect(“localhost”, 8883, 60)
    Traceback (most recent call last):
    File “”, line 1, in
    File “/usr/local/lib/python3.8/dist-packages/paho/mqtt/client.py”, line 914, in connect
    return self.reconnect()
    File “/usr/local/lib/python3.8/dist-packages/paho/mqtt/client.py”, line 1073, in reconnect
    sock.do_handshake()
    File “/home/teresa/ssl.py”, line 1309, in do_handshake
    self._sslobj.do_handshake()
    ConnectionResetError: [Errno 104] Connection reset by peer
    >>>
    >>> # publica 2 valores aleatorios de temperatura en el tema “temperatura/sensor1”
    >>> for i in range(2):
    … temperatura = random.uniform(20, 30)
    … client.publish(“temperatura/sensor1”, str(temperatura))
    … print(“Publicado valor de temperatura: ” + str(temperatura))
    … time.sleep(1) # pausa de 1 segundo

    Publicado valor de temperatura: 28.513090441345145

    Publicado valor de temperatura: 26.305160010960986
    >>> # desconecta el cliente del broker MQTT
    >>> client.disconnect()
    4
    >>>

    that is my script, to publish multiple messages in my local host broker, but it seems I’m missing som config or anythig to set the tls for my client and broker.
    Thank you for anathig you can do for me.

    1. Hi
      Have you checked the permissions for the cert file as this often causes problems on Linux.
      I recommend copying the cert files to you home folder and testing from there. Once it is working then move them into the etc/… folder.
      If that doesn’t work get back to me
      Rgds
      Steve

  2. Hello Steve i have been experiencing this error kindly help resolve the error
    PS C:\paho\mqqt> python pahoweb.py
    Traceback (most recent call last):
    File “C:\paho\mqqt\pahoweb.py”, line 81, in
    client = connect_mqtt()
    ^^^^^^^^^^^^^^
    File “C:\paho\mqqt\pahoweb.py”, line 62, in connect_mqtt
    client.connect(host, port, keepAlive)
    File “C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Lib\site-packages\paho_mqtt-1.6.1-py3.11.egg\paho\mqtt\client.py”, line 914, in connect
    return self.reconnect()
    ^^^^^^^^^^^^^^^^
    File “C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Lib\site-packages\paho_mqtt-1.6.1-py3.11.egg\paho\mqtt\client.py”, line 1045, in reconnect
    on_pre_connect = self.on_pre_connect
    ^^^^^^^^^^^^^^^^^^^
    File “C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Lib\site-packages\paho_mqtt-1.6.1-py3.11.egg\paho\mqtt\client.py”, line 1863, in on_pre_connect
    return self._on_pre_connect
    ^^^^^^^^^^^^^^^^^^^^
    AttributeError: ‘Client’ object has no attribute ‘_on_pre_connect’. Did you mean: ‘on_pre_connect’?

    1. Not sure where you got
      client = connect_mqtt()
      If you send me the script I will take a quick look. Use the ask steve page to contact me and then you can send it via email but with the extension changed.
      Rgds
      Steve

  3. Hi Steve,
    when i try to connect to client using via client.connect(broker,port) i get “socket.timeout: timed out” exception. I am using port 1883. Also the code is working localy on my computer, but while running the python code on jenkins i hit this issue.

      1. Hi Steve,

        Thanks for the prompt reply, I am not using any open source message broker, its a broker that is created by my internal project team. I am able to subscribe & get messages from the broker from my laptop, but when trying to execute the same code via jenkins pipeline I get socket connection time out.

  4. Hi,

    Thank you for your great tutorial! It is very useful. Unfortunately, I face a little issue…I do not understand why executing client.disconnect() before client.loop_stop() does not produce any result…the messages are not published and not even the on_disconnect() callback is called – I expected the on_disconnect() callback to be executed while the loop is still running.
    Thanks,
    Paul

      1. Hi, Thank you for your reply. I put a time.sleep(.5) between them. Also, what it seems peculiar is that on_disconnect() is called after the loop stopped (and not when I put the loop_stop() instruction after client is disconnected).

        1. Increase the time to 5 secs to give the disconnect a chance before stopping the loop and see if that resolves it.
          Rgds
          Steve

          1. It worked. Thank you. Also, could you explain me – in the previous case, where loop_stop() was called the first – please why the disconnect message was visible even if the loop was stopped? I expected the callbacks to be treated only whilst the loop is still up and running. Or the on_disconnect is a special one, thus, it is treated even after the loop is not running anymore?

          2. There is a timing issue as you have two threads involved. The on_disconnect callback in handled just like other callbacks in the loop.
            Rgds
            Steve

  5. Hi Steve, greetings from Chile, I’m telling you that I have a very particular problem, I send data packets through an EMQX broker, the publishers have no problem, however the computer that is subscribed sticks and throws me rc=0 but I don’t know connect in fact does not even appear in my broker, do you think what could be happening?

    1. Hi
      Do I understand that the subscriber doesn’t appear to be connected to the broker? Have you tried using other tools like mqttbox or mosquitto_sub or my python monitor?
      rgds
      steve

  6. Hello I get this error when i try to connect to mqtt cloud, but this exact code with exact credentials works in other system.

    File “C:\Users\HP\Desktop\python-paho-hivemq-cloud\mqtt_client.py”, line 61, in
    client.connect(“05a52813a07546ec88518a270db50483.s1.eu.hivemq.cloud”, 8883)
    File “C:\Users\HP\AppData\Local\Programs\Python\Python310\lib\site-packages\paho\mqtt\client.py”, line 914, in connect
    return self.reconnect()
    File “C:\Users\HP\AppData\Local\Programs\Python\Python310\lib\site-packages\paho\mqtt\client.py”, line 1073, in reconnect
    sock.do_handshake()
    File “C:\Users\HP\AppData\Local\Programs\Python\Python310\lib\ssl.py”, line 1341, in do_handshake
    self._sslobj.do_handshake()
    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

    I tried everything on internet and it did not work.
    Thank you

    1. The error message says it all . The certificate has expired. Not sure what you mean by works on other systems.
      Rgds
      Steve

  7. Hi Steve! I am new to MQTT and working on a sensor system for a carbon capture HVAC system. I just want to say that I really appreciate what you have set up on this site with regards to MQTT support. It has been invaluable to my work! Thank you so much!

    1. No problem and tks for the donation it is not the amount that matters but the thought. Good luck with your project.
      Rgds
      Steve

  8. Hi Steve,
    I’m new to mqtt. I’m getting live data from sensors and i’m published that data(json object) to mqtt with python.If connection lost the data is lost.
    when connection lost, I want to store the data in a file or database and when it reconnects, i want to publish that stored data to mqtt and delete that data from stored file or database after published to mqtt then i want to publish the live data to mqtt.
    please, give me a solution steve.

    1. Hi
      If you are thinking of short term connection loss then I would use a queue. You place incoming data in a queue and then empty the queue on the outbound side.
      If you are thinking longer term then use a file or database. I will look at the file solution and put some code together.
      Rgds
      Steve

  9. Thanks for the explanations. With your help, I managed to create a publisher and a subscriber relationship. I am using an Arduino Uno board which records values for temperature and humidity. A publisher is connected to the board and taking the sensor measurements, and a subscriber is used to get the data from the publisher.
    I am creating a Flask website to display the temperature and humidity measurements, which are received by the subscriber. How could I make the data accessible remotely?
    I need to access the data from any device, not just my laptop where the connection is made. I am using a free broker for mqtt in python.

    1. When you publish the values to the broker they can be received by any subscriber. So your Flask app needs to subscribe to the topic to receive them. Does that make sense?
      Rgds
      Steve

  10. Thanks for all your great tutorials.
    I use mqtt in my project an it work fine, now i want to know when it disconnect to the broker and I want to get it connection automatically again without run again from the beginning.
    how can i do it

    1. Hi
      If you use the loop_start() then it should connect automatically.To know when it disconnects use the on_disconnect callback
      Rgds
      Steve

  11. thank you for your post, but now I want to use mqtt in my project then it lost connection with broker in the middle of the program. I want to get it connected automatically ,because my project is always running. in addition to i want to know when it loss connection.
    thanks

  12. Hi Steve,

    For MQTT messages, how do we achieve request and response type messages? here’s an example

    If I am a client of one application and publishing 4 messages with the same topic, let’s say for parameters A, B, and C, and then A again, i will get 4 on_message callbacks.
    How do we map those responses back to my requests? How do we know that when on_message is fired in the client, that it is a response to the request for parameter A(1), or B, or C, or A(2)?
    what are the best practices for doing that type of request-response?

      1. I am trying to publish temperature values continuously to a topic with a python script and subscribing to it. I wanted to check 0%, 3%, 5%, and 10% packet loss introduced at the publisher side and check packet loss vs latency for each QoS level. Is it possible?

  13. Hi Steve, its nice tutorial. I am facing one problem with mqtt QoS 0 and payload is json. For large payloads disconnect callback is getting called with rc 1 and none of the payload is getting published. Though the on_message method is getting called and its printing the payload.

    1. Not quite sure as what is happening. When you publish a large message the client disconnects?
      Are you also trying to receive that message as you mention the on_message callback?
      Rgds
      Steve

  14. Steve, great Tutorials – really well described for me to understand!! Can you advise on MQTT broker disconnection exceptions? I have 3 RaspPi on a local network with the main RPI acting as the MQTT broker sending data to the other two MQTT clients (with loop forever) for simple slave LED displays. If a client is suddenly disconnected (lost power/wife unplugs) the broker Pi fails with an exception “OSError: [Errno 113] No route to host” . I’m not really concerned about the clients failing but the main Pi (& broker) is doing very much more so it’s a pain when that exits.
    Cheers

    1. Hi
      Nice to see I’m not the only one who has a wife that likes to unplug things.
      Strange the broker shouldn’t crash when a client disconnects for any reason. Can you use the ask steve page and send me a screen shot of the broker console.

  15. Hi, I still have a problem to handle a bad connection.
    Here’s my pseudocode:

    def MQTTinit(self):
    def on_connect(client,userdata,flags,rc):
    def on_disconnect(client,userdata,rc):
    self.client=mqtt.Client(CID,True,None)
    self.client.on_connect=on_connect
    self.client.on_disconnect=on_disconnect
    self.client.connect(SERVERURL)
    self.client.loop_start()

    def main(self):
    if (self.client.is_connected()):
    ret=self.client.publish(TOPIC,PAYLOAD,QOS,False)
    print(ret)

    At the beginning the on_connect callback executed and everything working just fine, is_connected return true and ret return 0. However when I cut the network (I disable network access via router), is_connected still return true, ret return 0 and on_disconnect callback not executed. Nothing happen until I re-enable network and the broker started receiving data again. Am I missing something?

    1. If you unplug your ethernet cable you will probably get what you want.
      I suspect that for what you are doing it doesn’t detect a broken connection as there is no requirement for tcp/ip to transmit messages.
      The mqtt ping should detect this so adjust the keepalive period to 60 secs and see what happens
      rgds
      steve

  16. Hi Everyone,
    I want to do same thing in c/c++.
    1) I need c/c++ mqtt library on my raspberry pi.
    2) My mqtt broker is active in my raspberry pi .

    please guide me.
    Thanks

  17. Hi, thank you for the nice tutorial. I’m looking forward to add some authentication attributes. is it possible to add them to the Connect packet or add parameters to the connect method? If it’s possible can you help me please?

  18. hi steve,
    I tried authenticating my client with username and password. (assuming I wrote both pub and sub program)
    I am trying to test this connection this way:
    1) Run pub with client id and username and password (pub1)
    2) Run sub
    3) Create another pub with same client id but with no username and pass(pub2)
    4) Check if broker allows pub1 or pub2

    According to me: The broker should not authenticate pub2 over pub1 with same client id

    But this isnt the case, sub is just taking all the message on the topic specified.

    Am I testing it correctly ?

    1. Hi
      Something is wrong with the testing as using two clients with the same client id will fail as there will be a constant disconnect/reconnect.
      Rgds
      Steve

  19. Hi, Steve i am getting connection failed but i am not getting the “rc” to know why it is failing
    Output that i am getting:(‘Connecting to broker ‘, ‘127.0.0.1’)
    connection failed
    Below is my code:
    import paho.mqtt.client as mqtt
    import time

    def on_connect(client, userdata, flags, rc):
    if rc==0:
    client.connected_flag=True
    print(“connected OK”,rc)
    else:
    print(“Bad connection Returned code=”,rc)
    client.bad_connection_flag=True

    mqtt.Client.connected_flag=False #create flag in class
    mqtt.Client.bad_connection_flag=False
    broker=”127.0.0.1″
    port=”1883″
    client = mqtt.Client(“python”)
    client.on_connect=on_connect
    client.loop_start()
    print(“Connecting to broker “,broker)
    try:
    client.connect(broker,port)
    except:
    print(“connection failed”)
    exit(1)
    while not client.connected_flag and not client.bad_connection_flag:
    print(“In wait loop”)
    time.sleep(1)
    if client.bad_connection_flag:
    client.loop_stop() #Stop loop
    sys.exit()

    1. It is probably failing early and not in the on_connect callback. This is usually because the broker isn’t running or running on a different port.
      This is were I think you are failing.try:
      client.connect(broker,port)
      except:
      print(“connection failed”)
      exit(1)

  20. I get an error when I run this as a script:
    Exception in thread Thread-1:
    Traceback (most recent call last):
    File “/usr/lib/python3.5/threading.py”, line 914, in _bootstrap_inner
    self.run()
    File “/usr/lib/python3.5/threading.py”, line 862, in run
    self._target(*self._args, **self._kwargs)
    File “/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py”, line 2913, in _thread_main
    self.loop_forever(retry_first_connection=True)
    File “/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py”, line 1604, in loop_forever
    self.reconnect()
    File “/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py”, line 926, in reconnect
    raise ValueError(‘Invalid host.’)
    ValueError: Invalid host.

    I get the “Connecting to Broker” output, followed by “connection “failed” (deliberate) then I see “in wait loop” then the above error, then back to “in wait loop”

    Is this an error with python or with paho?

    My overall goal is to publish a message to my broker (Mosquitto on Pi) that contains certain OS info, such as temps, usage, uptime etc.
    I have a python script that runs as a system service
    Im trying to figure out how to handle a failed mqtt publish, such as when the broker goes off line

    1. Did you put the connection in a try/except block?
      If you get a disconnect you also need to detect it and stop publishing.
      I use a connect flag which I set and unset tin the on_connect and on_disconnect callbacks.
      it looks like this
      mqtt.Client.connected_flag=False#create flag in mqtt class
      cname=”Timer-test”
      client = mqtt.Client(cname)

      in the callback
      client.connected_flag= True
      Rgds
      Steve

      1. Thanks for the reply, yes I did.
        My code is as below:
        import paho.mqtt.client as mqtt #import the client1
        import time

        def on_connect(client, userdata, flags, rc):
        if rc==0:
        client.connected_flag=True #set flag
        print(“connected OK”)
        else:
        print(“Bad connection Returned code=”,rc)
        client.bad_connection_flag=True

        mqtt.Client.connected_flag=False #create flag in class
        mqtt.Client.bad_connection_flag=False
        broker=”192.168.0.133″
        port=”1883″
        client = mqtt.Client(“python1”) #create new instance
        client.on_connect=on_connect #bind call back function
        client.loop_start()
        print(“Connecting to broker “,broker)

        try:
        client.connect(broker,port) #connect to broker
        except:
        print(“connection failed”)

        while not client.connected_flag and not client.bad_connection_flag: #wait in loop
        print(“In wait loop”)
        time.sleep(1)
        if client.bad_connection_flag:
        client.loop_stop() #Stop loop
        sys.exit()

        1. Hi
          The problem is the connection fails but you don’t terminate the script. Use
          try:
          client.connect(broker,port) #connect to broker
          except:
          print(“connection failed”)
          exit(1)
          Rgds
          Steve

          1. Thanks, that certainly seems to work better.
            But I do not see similar output to your guide, i.e. it outputs “Connecting to broker” and then “connection failed”
            I doesn’t return the rc as your screenshot shows?

    1. He can’t.The only way is to build it into the application. If you look at the analogy between MQTT and radio or TV the broadcaster has no idea who received the broadcast.
      Rgds
      Steve

      1. No. I want to check the message in while loop whether message come or not on subsciber side. is there anyway to do that. Thx

          1. Okay. Im working on automation project. I can publish start and finish messages. When start message is arrived while loop begin on the subcriber side. I want to stop while loop with finish message but I cannot check the finish message in while loop.Thx.

          2. Hi
            Use the ask steve page to conact me and send me the script you are currently using.
            Rgds
            Steve

  21. Hello Steve,
    i am very new to IoT and Programming.could you just tell me how can i print the client_id the way i printed result code.i am trying to do this as seen in the last line of code below:

    def on_connect(client, userdata, flags, rc):
    if rc==0
    print(“connected ok”)
    print(client.client_id)

    but i am getting n output.could you please tell me why.
    thanks

    1. should be
      def on_connect(client, userdata, flags, rc):
      if rc==0:
      print(“connected ok”)
      print(client.client_id)

      don’t forget the indents
      rgds
      steve

  22. Hello Steve
    Thank you for your website. I have a python script running that checks for a csv file, connects to the broker, then sends the file, then disconnects from the program. My problem is that when the next file comes in it detects the file, detected that the broker is disconnected, attempts to connect with the same connect line of code, but I never get an on_connect call back and so it does not publish.

    I think the problem is that even though I get an on_disconnect call back it is not fully disconnected and therefore it won’t make a second connection. If I stop and restart the script it works the first time and disconnects but wont reconnect and publish.

    Thanks for your help
    Mark

  23. Dear Steve,

    I am totally new to MQTT so please forgive my bad explanation of my problem:
    I tried to run your mqtt-data-logger Python script, but it tells me

    No module named ‘paho’.

    I searched in my python 3.7 directory and couldn’t find any trace of ‘paho’.

    I’ve read, that it’s a library, which may be imported or installed, but I don’t know how to acquire it.

    Sidenote:
    I want to export data from MQTT spy into excel on windows. I have a machine, which sends it’s data to the broker. From there I’d like to acquire data for example the current and gather it in an excel sheet for predictive maintenance usage.

    Do you have any tips for me – that’d be so awesome.

    Thanks in advance and best regards

  24. Hi Steve,

    My application had problem: many close_wait on gunicorn and python worker when i publish to mobile (i tried both method: client.publish or publish.single).

    What’s wrong?
    Note: my application sent many messages.

      1. Hi Steve,
        Thanks for your reply.

        1. i used client.loop_forever() for on worker file. That’s subcribe for a channel
        2. i used client.publish or try with publish.single for send message to channel.
        r = publish.single (
        topic=mqtt_topic,
        payload=message,
        qos=0,
        retain=False,
        hostname=self.server,
        port=self.port,
        client_id=””,
        keepalive=self.timeout,
        will=None,
        auth={
        ‘username’: self.user,
        ‘password’: self.password
        },
        tls=None,
        protocol=mqtt.MQTTv311,
        transport=”tcp”
        )
        when can not close the socket: CLOSE_WAIT. i try to use:
        self.client.publish(mqtt_topic, message)
        But it’s same. many close_wait on my application.

        Note: all messages were received but my system will be down soon when can not close the socket

        Like:
        python 3423 htk 8u IPv4 527485 0t0 TCP localhost:35266->localhost:1883 (ESTABLISHED)
        gunicorn 3467 htk 16u IPv4 529421 0t0 TCP localhost:54220->localhost:1883 (CLOSE_WAIT)
        gunicorn 3467 htk 20u IPv4 527133 0t0 TCP localhost:32810->localhost:1883 (CLOSE_WAIT)
        gunicorn 3467 htk 22u IPv4 527053 0t0 TCP localhost:57792->localhost:1883 (CLOSE_WAIT)
        gunicorn 3467 htk 25u IPv4 528742 0t0 TCP localhost:46667->localhost:1883 (CLOSE_WAIT)
        gunicorn 3467 htk 32u IPv4 529522 0t0 TCP localhost:58477->localhost:1883 (CLOSE_WAIT)
        gunicorn 3467 htk 38u IPv4 528894 0t0 TCP localhost:60896->localhost:1883 (CLOSE_WAIT)
        mosquitto 21027 htk 5u IPv4 358221 0t0 TCP localhost:34834->localhost:1883 (ESTABLISHED)

  25. Hi, I like the tutorial. I am struggling though with a publish quirk.

    I have a python script (running under piCore on a Raspberry) calling AWS IOT successfully all the time. However once the script has connected successfully and then I force a network error by taking down my router and the script publishes another message it just goes through as though all was well with a good return code. I do understand that QOS 0 fires and forgets but I am surprised that MQTT doesn’t check that the session is valid.

    Is that how it operates?

    THanks Len

    1. Len
      Yes the client hasn’t detected a problem and so it just tries to publish the message.
      If you use QOS 1 then it will resend the message because it didn’t get a PUBACK.
      Rgds
      Steve

      1. Hi Steve,

        Thanks for all your great tutorials, they are really useful.

        I am facing a similar issue. I am publishing with QOS 2, and when I turn off my laptop wifi and publish a message, I receive rc=0, but the message was not published. It is published when the wifi connection is restored, so that’s ok.
        What I need is to detect if the network is down when I try to publish. Is there a way different from checking network connection before calling the publish( ) method?

        Another question: are the meanings of the Publish return codes the same as the Connection return codes described in this article?

        Thanks,
        Laura

        1. You can use the on disconnect callback and set a flag when disconnected. It might not catch all problems so you could also check for the on_publish callback and raise an error if you don’t get the ack message after x secs.
          If you take a look at the sending file tutorial and script I wait for the puback before sending the next block of data.
          http://www.steves-internet-guide.com/send-file-mqtt/
          Rgds
          Steve

          1. Thanks for the reply!
            Like you said, the on_disconnect callback is not catching this problem. I will try your suggestion with the ack message.

            Regards,
            Laura

Leave a Reply

Your email address will not be published. Required fields are marked *