Using the Arduino PubSub MQTT Client

arduinoThere are several  MQTT clients available for Arduino and we are going to use the PubSub MQTT client.

Before you can use this client you need to install it into the Arduino Library.

Go to the Library manager and so a search for MQTT. You will find quite a few listings scroll down the select the PubSub client.



pubsubclient-arduino

The documentation for the client API is available here.

About This Tutorial

We will start by going through the basic functions available and how and in what order they are used.

We will see how you connect to a broker,subscribe to topics and publish messages and finally how we receive incoming messages. Two working Demo scripts are available for download at the end.

Client Basics

Like most most MQTT clients you follow 4 basic steps

  • Create a Client Object
  • Create a Connection to a Broker
  • Publish and/or subscribe
  • Process received Messages

The first step is to create an Instance of the client object the API lists several constructors they are:

  • PubSubClient ()
  • PubSubClient (client)
  • PubSubClient (server, port, [callback], client, [stream])

where

client= client transport object instance EthernetClient
server=broker name or IP address
port = broker port default is 1883
callback= a function to process received messages
stream= instance of a message stream to store received messages

The one I will Illustrate here is PubSubClient (client)

Example code Using Ethernet:

EthernetClient ethClient;
PubSubClient mqttClient(ethClient);

Now our client is called mqttClient

Now we need to create a connection to a broker.

First we need to use the setServer method which requires a broker address and port (optional)

setServer(server, port)

example Code:

mqttClient.setServer("192.168.1.68", 1883);

Now if we want to receive messages we also need to create a callback function and set it using setCallback(callback) method;.

Example Code:

mqttClient.setCallback(callback_function);

However we will skip that step here and continue.

So now we need to connect using the connect (clientID) function. Here clientID is the name of the client and must be unique:

Example code:

mqttClient.connect("arduino-1")

Now we are connected we can subscribe and publish.

To subscribe to a topic use the subscribe function

subscribe(topic,qos)

This requires a topic and an option QOS (1 or 0)  defaults to 0

Example code:

mqttClient.subscribe("test")

We can Publish using the publish method. .There are four options all return an Integer, 1 for success (true) and 0 for fail(false).

These two use a character array for the payload:

  • publish (topic, payload)
  • publish (topic, payload, retained)

These two use a Byte Array for the payload:

  • publish (topic, payload, length)
  • publish (topic, payload, length, retained)

Example code using character string:

boolean rc = mqttClient.publish("test", "This is a test message");

Example code using bytes:

byte outmsg[]={0xff,0xfe};
boolean rc = mqttClient.publish("test", outmsg,2);

Return Codes

Most functions return a value which indicates success or fail.

Important ones to be aware of are the connect function which provides a boolean true/false for success or fail.

So it is usual to test for a successful connection before proceeding as you cannot publish or subscribe until you are connected.

The code below tests the connection and if successful goes ahead and subscribes.

  if (mqttClient.connect("arduino-1")) {
    // connection succeeded
    Serial.println("Connected now subscribing");
    boolean r= mqttClient.subscribe("test");

  } 
  else {
    // connection failed
    // mqttClient.state() will provide more information
    // on why it failed.
    Serial.println("Connection failed ");
  }

You should also note the comment in the else statement regarding using the state() function to get more detail on the connection.

I recommend you look at the docs for detailed information regarding the return codes and their meaning.

The Loop Function

The loop function is responsible for processing the message queues and is required in the script and is placed inside a while loop so that it is called frequently.

The idea behind this function is basically the same as the one used in the Python client which I described in detail here.

Example Code:

The example code publishes a string and a byte message every second. It also subscribes to a topic but doesn’t have a callback function to process the received messages.download

Receiving Messages

We have already seen how to subscribe which we did as part of the setup and now we need to ensure that we add the callback using the setcallback() function.

The function takes the name of the function to call.

  mqttClient.setServer("192.168.1.68", 1883);
  mqttClient.setCallback(callback);

We now need to create the callback function to process the received message.

The callback function needs to accept various arguments. The structure is

void callback(const char[] topic, byte* payload, unsigned int length)

Note: length is the length of the payload.

Example code:
The code prints the received topic and message.

  void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i=0;i<length;i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

Moving Received Messages to the Main Loop

If you need to access the received messages elsewhere in the code which you normally do then you need to copy it into a buffer.

Below is a modified version of the callback.

  void callback(char* topic, byte* payload, unsigned int length) {
   //Payload=[];
   Topic=topic;
   Rflag=true; //will use in main loop
   r_len=length; //will use in main loop
   Serial.print("length message received in callback= ");
   Serial.println(length);
     for (int i=0;i<length;i++) {
    buffer[i]=payload[i];
    Serial.print((char)payload[i]);
       }

}

The buffer is a byte array, the r_len in an integer and the Rflag is a boolean and all are global variables.

byte* buffer;
boolean Rflag=false;
int r_len;

The code snippet below show them being used in the main loop.

if(Rflag)
{
   Serial.print("Message arrived in main loop[");
  Serial.print(Topic);
  Serial.print("] ");
   Serial.print("message length =");
    Serial.print(r_len);
  //Serial.print(Payload);
    for (int i=0;i<r_len;i++) {
    Serial.print((char)buffer[i]);
  }
  Serial.println();
  Rflag=false;
  }

Example Demo Code includes receiving Messages

.download

Example Demo Code includes moving Messages into buffer and using in main loop.

download

Question

If you run demo code for example2 you only receive messages for the string message as shown in the screen shot below. Why is that? Answer at the end.

arduino-test-1

Related Tutorials

Answer to question

This is because the bytes published on a different topic and so you need to subscribe to that topic as well.

Please rate? And use Comments to let me know more

33 comments

  1. Hi Steve,

    Thank you for the detailed explanation. That really helps.
    However, when I try to build the code I get an error at – client.setCallback(callback);

    It says – no suitable constructor exists to convert from “” to “std::function
    I am very new to this topic so excuse me if this is a really trivial question.
    Could you let me know where am I going wrong or what is the solution to this problem?

  2. Hello
    Have followed your guides with mqtt and node red etc. A year ago I knew nothing about this 🙂 Now trying to follow your guide on mqtt and Arduino. I have succeeded to some extent but thought to ask if you plan to make a film about this is so much easier when you talk about what happens and then you follow the text and test yourself similar to what you did on others with mqtt.
    Cheers from Sweden

  3. Hi Steve,
    I have an arduino+ethernet shield working with my MQTT Broker, but i want to replace the ethernet shield with the ESP8266. There are a lot of examples and tutorials over the internet but only connecting the ESP8266 to the MQTT broker and not using the arduino. Its always uploading sketch to the ESP8266 and not to the arduino. Do you know if there is a way to “replace” the ethernet shield with the ESP8266?

    Cheers.

    1. Hi
      Not an expert on this but I bought a board thinking it was a arduino hat but it turn out it was an ESP8266 arduino compatible and you use it like an arduino you just need to install the board driver in the ide. I found the instructions on the internet.
      rgds
      steve

      1. If you dont have the ESP support on Arduino IDE yet, follow these steps.
        https://randomnerdtutorials.com/installing-the-esp32-board-in-arduino-ide-windows-instructions/

        And then
        Below is what I’m working with now, please excuse the code as there will be a bit more than you need here.

        #include
        #include
        #include
        #include “SafeStringReader.h”

        const char* mqttSubscribeTopic = “subTopic”;
        const char* mqttPublishTopic = “pubTopic”;
        const char* mqttServer = “IP HERE”;
        int mqttPort = 1883;
        long lastReconnectAttempt = 0;
        unsigned long lastMillis = 0;
        uint32_t counter = 0;

        WiFiClient espClient;
        PubSubClient mqttClient(espClient);
        WiFiManager wm; // global wm instance

        WiFiManagerParameter custom_field; // global param ( for non blocking w params )

        createSafeStringReader(sfReader2, 500, “+”);

        void setup()
        {
        WiFi.mode(WIFI_STA); // explicitly set mode, esp defaults to STA+AP
        Serial.begin(9600);
        Serial2.begin(9600);

        Serial.setDebugOutput(true);
        delay(3000);
        Serial.println(“\n Starting”);

        pinMode(TRIGGER_PIN, INPUT);

        //wm.resetSettings(); // wipe settings

        wm.addParameter(&custom_field);
        wm.setSaveParamsCallback(saveParamCallback);

        std::vector menu = {“wifi”,”param”};
        wm.setMenu(menu);

        //wm.setClass(“invert”); // set dark theme

        WiFiManagerParameter custom_text(“Get the API key from your online account.”);
        wm.addParameter(&custom_text);

        int customFieldLength = 40;
        new (&custom_field) WiFiManagerParameter(“apikey”, “API Key”, “”, customFieldLength,”placeholder=\”Your API Key here\””);

        wm.setConfigPortalTimeout(240); // auto close configportal after n seconds

        bool res;
        res = wm.autoConnect(“Smart Home”,””); // password protected ap

        if (!res)
        {
        Serial.println(“Failed to connect or hit timeout”);
        ESP.restart();
        }
        else
        {
        Serial.println(“WiFi connected…”);
        }

        sfReader2.connect(Serial2); // where SafeStringReader will read from

        mqttClient.setServer(mqttServer, mqttPort);
        mqttClient.setCallback(mqttCallback);
        }

        void loop()
        {
        //Serial.println(custom_field.getValue());

        if (!mqttClient.connected())
        {
        reconnect(mqttSubscribeTopic);
        }

        if (sfReader2.read())
        {
        Serial.println(sfReader2);
        Serial2.write(“Received with thanks from ESP.+”);

        if (mqttClient.connected())
        {
        mqttClient.publish(mqttPublishTopic, sfReader2.c_str());
        }
        }

        mqttClient.loop();
        }

        void mqttCallback(char* topic, byte* payload, unsigned int length)
        {
        String payloadString = “”;

        for (int i = 0; i hasArg(name)) {
        value = wm.server->arg(name);
        }
        return value;
        }

        void saveParamCallback()
        {
        Serial.println(“[CALLBACK] saveParamCallback fired”);
        Serial.println(“PARAM customfieldid = ” + getParam(“customfieldid”));
        }

  4. is there a way to use the message received in client.subscribe instead of only showing in the serial monitor? i want to pick up that message and change an array to send another message . sorry for the bad english

    1. You need to put it in an array so you can access it outside the callback. I haven’t tried this with arduino but use it with python.
      I will try it myself in a few days.

      1. // Subscribe to “mytopic/test” and display received message to Serial
        client.subscribe(“m5coreTO”, [](const String & payload) {
        Serial.println(payload);
        });
        How do I save the payload to a buffer, instead of the Serial.printin

        1. Note: Added script as download on the tutorial page.

          You need to copy it into a buffer that you can access from the main loop. Here is part of hte code
          byte* buffer;
          boolean Rflag=false;
          int r_len;
          EthernetClient ethClient;
          PubSubClient mqttClient(ethClient);

          void callback(char* topic, byte* payload, unsigned int length) {
          //Payload=[];
          Topic=topic;
          Rflag=true; //will use in main loop
          r_len=length; //will use in main loop
          Serial.print(“length message received in callback= “);
          Serial.println(length);
          for (int i=0;iReply

  5. Hi Steve,

    I am a student who is doing a project of a robotic car controlled by MQTT, I have followed your tutorial but I have a problem, I do not receive all the MQTT messages sent from the broker (still using qos = 1). I have subscribed using qos = 1 but I am not receiving confirmation of receipt.

    For example if I send 4 messages, I only receive 1 in the car. I have a feeling that client.loop () blocks receiving messages.

    However, in an MQTT client installed on my android phone if I receive all the messages instantly.

    1. I assume arduino is the car? The loop shouldn’t block. I would need to see the code for the arduino.
      Rgds
      Steve

      1. Hi Steve, the car is OSOYOO Arduino car, I have the broker in a raspberry. I have tried with qos 0 but nothing change, how can I send you the code?

  6. Hi, Great tutorial. There one problem, I cannot seems to receive or subscribe for the message that has length more than 242. For example my string or byte array of message is bigger than 1000 so how would it be possible to receive it

  7. Hi Steve, Thanks a lot for your pages on MQTT – they have been helpful. I have a question about the frequency of calling mqtt.loop(). My HW is an Arduino ATMega2560 with a SIM800 using a GPRS connection. I have an application that is sensitive to delays in the loop and so I am trying to call mqtt.loop() as infrequently as I can get away with (obviously I only call the mqtt.loop() func after connecting, subscribing, etc). I have tried calling mqtt.loop() every 10s and I noticed that messages don’t get delivered to the client (or get delivered very infrequently). If I increase the frequency of calling mqtt.loop() to every 1s then messages get delivered reliably, but still, the messages only start getting delivered after about 5s (around the 5th time the mqtt.loop() func is called). I have checked the broker (mosquitto) and the client is connected in both scenarios and there seems to be no issue at the broker side. Are you able to share any insight into why the messages don’t come through immediately, with the 1st time mqtt.loop() is called?

    1. I can only think that there is another delay somewhere in the code. Are you calling the loop in a separate thread or in the main program? If you send me the relevant part of the code I will take a look.
      Rgds
      Steve

  8. Hello Steve,
    I am trying to develop a code very similar to the one described above. I cannot figure out where to find the necessary ip addresses used in your code here:
    IPAddress ip(192, 168, 1, 61);
    const char* server = “192.168.1.68”;
    I am assuming that the first one would be the ip address of the router that the ethernet shield is plugged into and the second would be the ip address of the device that the mqtt broker is running on?

  9. Hi,
    My node mcu/ esp8266 is not connecting to my local broker which is installed in my laptop. Esp8266 is connected to internet but it doesn’t connect to my local broker.

  10. hi, can you explain to me a little bit about MQTT as I’m stuck? I want to send a message from my laptop to node MCU or esp8266. I have an MQTT broker installed on my laptop, so I will use that broker which is on my laptop to send messages or receive messages. I want to control the bulb. I have written my IP address on the server line but nothing happens. the cade says connection failed. what should I do?

  11. I have a question about client.loop()
    My program runs the void loop() at a rate of 250Hz.
    The last line is client.loop()

    My question is how can my void loop() run this fast while client.loop() is a blocking call?
    I tell you that I’m happy it runs that fast but I don’t understand it.

    1. In the Python client you can call it with a timeout to stop it blocking too long.Can you point me to the C client docs that you are using.
      Rgds
      Steve

  12. Hi Steve ….
    Do you have any info on Arduino MQTT over TLS (8883) with a server key ….
    I see so much conflicting info out there. It would be nice to know if you have bee able to achieve it

    Cheers
    Sean

  13. Hi,

    the callback function is never called , im using the same code and subscription is shown in the server but I’m not able to receive the message when it comes to the topic, any idea ?

Leave a Reply

Your email address will not be published. Required fields are marked *