Python MQTT Client Connections- Working with Connections

The MQTT client uses a TCP/IP connection to the broker.

Once the connection is established the client can send data to the broker, and the broker can send data to the client as required.

You can consider a TCP/IP connection to be similar to a telephone connection.

Once a telephone connection is established you can talk over it until one party hangs up.

In this tutorial we will look at connecting to an MQTT broker using the paho python mqqt client.

The Connect Method

To establish a connection to an MQTT broker using the Python client you use the connect method of the client object.

The method can be called with 5 parameters (MQTTv3.1). The connect method declaration is shown below with the default parameters.

and for MQTTv5 it takes 6 parameters

The only parameter you need to provide is the host name.

This can be the IP address or domain name of the broker.

Note: you may need to setup other settings like passwords, last will and testament etc before connecting. See Working with the Client object

The connect method is a blocking function which means that your script will stop while the connection is being established.

Note: the protocol version is set when creating the client.

Was The Connection Attempt Successful?

When a client issues a connect request to a broker that request should receive an acknowledgement.

The broker acknowledgement will generate a callback (on_connect) in the client.

If you want to be sure that the connection attempt was successful then you will need to setup a callback function to handle this callback before you create the connection.

The function should receive 4 parameters, and can be called anything you want.

I have called mine on_connect().

Here is an example function definition MQTTv3.1:

def on_connect(client, userdata, flags, rc):
    if rc==0:
        print("connected OK Returned code=",rc)
    else:
        print("Bad connection Returned code=",rc)

The client is a client object.

rc (return code) is used for checking that the connection was established. (see below).

Note: It is also common to subscribe in the on_connect callback

MQTTv5 has an additional properties parameter

on_connect(client, userdata, flags, reasonCode, properties=None)

Connection Return Codes

• 0: Connection successful
• 1: Connection refused – incorrect protocol version
• 2: Connection refused – invalid client identifier
• 3: Connection refused – server unavailable
• 4: Connection refused – bad username or password
• 5: Connection refused – not authorised
• 6-255: Currently unused.

Flags and userdata aren’t normally used .The full documentation is here.

Note: MQTTv5 has more reason codes see MQTTv5 New features

Processing The On_connect Callback

To process the callback you will need to run a loop. (see understanding the network loop) .

Therefore the script generally looks like this.

1. Create Client object.
2. Create callback function on_connect()
3. Bind callback to callback function (on_connect())
4. Connect to Broker.
5. Start a loop.

Because the callback function is asynchronous you don’t know when it will be triggered.

What is sure however is that there is a time delay between the connection being created, and the callback being triggered.

It is important that your script doesn’t proceed until the connection has been established.

For quick demo scripts I use time.sleep() to wait, and give the connection time to be established.

However for working scripts I process the callback and use it to flag a successful or unsuccessful connection.

So instead of this:

1. Create connection
2. Publish message

We have this:

1. Create connection
2. Verify successful connection or quit
3. Publish message and or subscribe

Here is some example code that uses time.sleep() to wait, and give the connection setup time to complete:

import paho.mqtt.client as mqtt    #import client library
def on_connect(client, userdata, flags, rc):
   if rc==0
      print("connected ok")
client = mqtt.Client(“python1”)             #create new instance 
client.on_connect=onconnect  #bind call back function
client.connect(broker_address)               #connect to broker
client.loop_start()  #Start loop 
time.sleep(4) # Wait for connection setup to complete
Other code here
client.loop_stop()    #Stop loop 

Enhancing the Callback

To get better control of the connection I use a flag in the on_connect callback.

The flag I create as part of the client object so it is available throughout the script.

client.connected_flag=False

At the start of the script I set this flag (connected_flag) to False and toggle it to True when the Connection is successful, and back to False when we get a disconnect.

def on_connect(client, userdata, flags, rc):
    if rc==0:
        client.connected_flag=True #set flag
        print("connected OK Returned code=",rc)
        #client.subscribe(topic)
    else:
        print("Bad connection Returned code= ",rc)

We can now use this flag to create a wait loop.

client.connect(broker_address)      #connect to broker
while not client.connected_flag: #wait in loop
     time.sleep(1)

Note: from version 1.6 there is an is_connected() method

Example Client Connection Script

The following script is a basic client connection script

#!python3
import paho.mqtt.client as mqtt  #import the client1
import time

def on_connect(client, userdata, flags, rc):
    if rc==0:
        client.connected_flag=True #set flag
        print("connected OK")
    else:
        print("Bad connection Returned code=",rc)

mqtt.Client.connected_flag=False#create flag in class
broker="192.168.1.184"
client = mqtt.Client("python1")             #create new instance 
client.on_connect=on_connect  #bind call back function
client.loop_start()
print("Connecting to broker ",broker)
client.connect(broker)      #connect to broker
while not client.connected_flag: #wait in loop
    print("In wait loop")
    time.sleep(1)
print("in Main Loop")
client.loop_stop()    #Stop loop 
client.disconnect() # disconnect

If I run this script this is what I see:

Failed Connection Examples

There are various conditions were the connection can fail to complete. They are:

• Incorrect client settings e.g. bad password..
• No network connection
• Bad Network Connection parameters e.g. bad port number

It is important that these are detected and handled by the connection script.

We are going to look at a few of these and modify our connection code to detect them.

Note: For these examples I will use the Paho MQTT client and the Mosquitto broker.

Connection Failures that Create an Exception

Trying to connect to a broker using a bad IP address or port number will generate a socket error, and raise an exception.

So the first screen shot shows the result of using bad port number.

This causes a Winsock error in Windows

In Python we can use a Try block to catch this so instead of

client.connect(broker,port) #connect to broker

We use

try:
    client1.connect(broker,port) #connect to broker
except:
    print(“connection failed”)
    exit(1) #Should quit or raise flag to quit or retry

When the connection attempt failed we would see:

Connection Failures Detected Through Return Code

To determine if the connection was successful we need to examine the return code of the on_connect callback.

A return code of 0 is successful, whereas other values indicate a failure.

In the example below we will try to connect to a broker without providing the required authentication.

Notice the connection fails and returns a return code of 5 which indicates authentication failure.

You should also notice that because I am using the loop_start() function the client will try to reconnect, but this is pointless as the result will be the same.

So our code should :

1. Stop the loop
2. Stop the script

We can stop the loop in the on_connect callback. However to stop the main script we need to set a flag that we can use to exit.

I prefer to use a flag and stop the loop as part of the main script.

Here is what the modified on_connect callback looks like:

def on_connect(client, userdata, flags, rc):
    if rc==0:
        client.connected_flag=True #set flag
        print("connected OK")
    else:
        print("Bad connection Returned code=",rc)
        client.bad_connection_flag=True

Here is the main script modifications to quit.

mqtt.Client.bad_connection_flag=False #

while not client.connected_flag and not client.bad_connection_flag: #wait in loop
    print("In wait loop")
    time.sleep(1)
if client.bad_connection_flag:
    client.loop_stop()    #Stop loop
    sys.exit()

When I run the script this is what I see:

 

Using Authentication

If the broker requires username and password authentication (see Mosquitto username and password authentication ) then you need to set this before connecting.

This you do using the username_pw_set() helper function. e.g

client.username_pw_set(username="steve",password="password")
# now can connect

Connecting Using Websockets

Normally the python client will connect using MQTT but it can also connect using MQTT over websockets.

To tell the client to use websockets instead of MQTT use the command

client= paho.Client(“cname”,transport=’websockets’)

instead of simply

client= paho.Client(“cname”)

You will also need to change the port..Websockets generally uses port 9001.

See MQTT over Websockets for more details

Handling Disconnects and Reconnects

A client can disconnect gracefully, if it has no more data to send by sending a disconnect message.

The Paho client provides the disconnect method for this.

It can also get disconnected due to a bad network connection.

If the connection fails for some reason then you will need to decide whether or not you should try to reconnect.

A disconnect triggers the on_disconnect callback which you will need to examine.

This callback takes 3 parameters:

Client- Client object that disconnected

Userdata- user defined data not often used

Return Code (rc)- Indication of disconnect reason. 0 is normal all other values indicate abnormal disconnection

Here is the on_disconnect() code I use:

def on_disconnect(client, userdata, rc):
    logging.info("disconnecting reason  "  +str(rc))
    client.connected_flag=False
    client.disconnect_flag=True

You can see that I simply log it, and then set flags that can be used by the main program to detect the disconnect.

Note: You will need to be calling, or running a loop to trigger the callback.

Reconnecting

Generally you will need to reconnect as soon as possible.

If you run a network loop using loop_start() or loop_forever() then re-connections are automatically handled for you.

A new connection attempt is made automatically in the background every 3 to 6 seconds.

If you call the loop() function manually then you will need to handle the re-connection attempts yourself. See understanding the loop.

You can do this by using a connection flag that is toggled by the on_connect and on_disconnect callbacks.

Client Connection Summary

Taking into account the above our client connection code should.

• Connect to broker
• Examine connection status and proceed if good
• If connection status is bad attempt retry and or quit.
• Handle disconnects and reconnects

Common Problems

1 .Not seeing any messages or not seeing all expected messages.

Possible causes

1. You haven’t started a network loop or called the loop() function. Or you haven’t registered or created the callback functions.
2. You haven’t subscribed to the correct topics or subscription has failed.
3. Access restrictions are in place. See ACLs

2.- My messages don’t appear in the order I expected?

Possible causes

1. The callback functions are async functions which can be called at any time. Use a queue to store the messages and print in one place.

Video- MQTT client connections Using The Python MQTT Client

Grateful if you would provide feedback as it will help with subsequent videos.

Course Links

1. Introduction to the Paho Python MQTT Client
2. Introduction to the Client Class
3. Connecting to a Broker
4. Publishing Using The Paho Python MQTT Client
5. –Subscribing using The Paho Python Client
6.  Receiving Messages with the Paho MQTT Python Client
7. Understanding The Loop
8. Understanding Callbacks
9. Handling Multiple Client Connections

Related tutorials and Resources

• You can find the documentation for the connect method here
• The Hive MQTT essentials series especially part 3 and part 4 for this tutorial.
• Python MQTT Manual Reconnect
• Checking Active MQTT Client Connections