Node-Red provides both an MQTT subscribe (input) and publish (output) node.
The configuration for these nodes are almost Identical as the main part of the configuration concerns the actual client connection.
Because of this it is useful to think of the publish and subscribe nodes as consisting of two components as shown in the schematic below:
Before we look a the actual configuration we will look at MQTT client connections in general.
Connecting to an MQTT Broker or Server.
To connect to an MQTT broker or server you need to know the Broker IP address or name, and the port that it is using (default 1883).
In addition a client needs to provide a client name to identify itself. Each client connection requires a unique client name.
A single client connection can be used to both publish and subscribe.
An MQTT broker can enforce encryption and username/password authentication in which case the connecting client must comply.
There are various flags and parameters that a client can set. These are:
- Clean Session Flag
- Time to Live
- Last Will and Testament
Clean Session Flag
MQTT clients will usually by default establish a clean session with a broker.
A clean session is one in which the broker isn’t expected to remember anything about the client when it disconnects.
With a non clean session the broker will remember client subscriptions and may hold undelivered messages for the client.
See –Understanding Clean sessions Video
Time to Live
This is a mechanism used to determine if a connection is still present. The default interval is 60 seconds.
See –MQTT Keep Alive Interval Explained
Last Will and Testament
The idea of the last will message is to notify a subscriber that the publisher is unavailable due to network outage.
The last will message is set by the publishing client on a topic and is stored on the broker.
It Is sent to subscribers if the publisher disconnects abnormally.
If the publisher disconnects normally the last Will Message is not sent.
See –Last will and testament-Example
Publishing Using The Publish Node
When a client publishes a message to a broker it needs to send:
- The message topic
- The message QOS
- Whether the message should be retained.- Retain Flag
The retain Flag is normally set to False which means that the broker doesn’t keep the message.
If you set the retain flag to True then the last message received by the broker with the retained flag set will be kept.
The QOS of the published message has no effect on the retained message.
The retain flag setting is the only setting that is unique to the publish node. See MQTT Retained Messages Explained
To publish a Message to an MQTT broker you use the MQTT publish node.
To use a publish node drag the node from the node palette on the left into the main workspace.
Double click on the node to configure it. The main configuration screen is shown below:
.
The main setting is the broker or server setting.
You can select a previously configured server which you can edit by clicking on the edit button, or add a new broker by selecting the add new mqtt server option from the server list.
The server name that appears in the server list is a combination of the client ID and Broker name or IP address.
Note: The MQTT specification originally used the term broker but it has now been changed to server.
You can also configure the topic address,the QOS for the message and the retain flag.
The topic address,the QOS and the retain flag can also be provided by the preceding node as part of the message object using:
- msg.topic =mytopic
- msg.qos =0,1 or 2
- msg.retain =true or false
Publish Message
The message to be published can be either a JavaScript Object string or a binary buffer.
If you pass in a JavaScript object it is automatically converted to a JSON encoded string before being published.
If you pass in numeric data it is converted to a string and if you pass in Boolean data it is also converted into a string before being published.
Editing the Server Settings
If you click on the edit icon to the right of the server you can edit the server settings.
The server settings has 4 tabs.
- connection tab
- Security
- Birth Message
- Will Message
Connection Tab
The port defaults to 1883 which is the standard MQTT port.
You can use either the IP address or FQDN for the server address.
The client name needs to be unique on the broker. See client names and duplicate client ids.
You can configure the client to use SSL for the connection by enabling the TLS box- see later
The default keep alive is 60 secs and clean session is False. It is normal to enable clean sessions.
Security Tab
This lets you configure a use a username and password for the connection.
Whether you need a username/password for the connection is determined by the MQTT broker.
Birth Message
The birth message is published by the client when the MQTT node starts.
Will Message
The Will message is part f the MQTT specification and is stored on the broker and sent in the advent of a failed client connection.
The message payload will either be a message to indicate the connection has failed or a status indicator.
The retained message will usually be set so that new clients know the status.
The topic will be a per-agreed topic. See Checking Active MQTT Client Connections
Using SSL for Encryption
The client supports SSL encryption between the client and server as well as using certificate based authentication.
To configure a secure connection you will need enable TLS
and add the CA certificate to node-red by uploading it. In the screen shot below I used the same self generated certificate as I used on Mosquitto broker.
If the CA file is on your local computer you can use it by enabling the use key and certificates from local files option. as shown below and entering the path and file name:Alternatively you can upload the files using the upload buttons.
Note: the verify server certificate checks the the name on the certificate is correct as should be enable on live systems. Leaving it un-checked is like using the -insecure option of the mosquitto_pub /_sub tool. Entering a name in the box doesn’t seem to make any difference.
If you are using Client certificates then you will also need to upload or provide the pathname to the client key and certificate as shown below:
SSL Public Certificates
If you are using a public broker like cloudmqtt then you will need to use a public CA.
You will need to get a copy of ca-certificates.crt or ca-certificates.pem file.
This file contains a list of trusted CA certificates and is available from Mozilla.
If you are using Linux it is usually in /etc/SSL/certs, but you can download it for the curl site here.
You will then need to upload it, just like with the self generated certificate as shown in the screenshot below.:
Configuring The Subscribe Node
When a client subscribes to a broker it needs to provide a topic and QOS.
The subscribe node has no unique settings.
To use a subscribe node drag the node from the node palette on the left into the main workspace.
Double click on the node to configure it. The main configuration screen is shown below, and is almost identical to the publish node.
Note: You don’t currently appear to be able to subscribe to multiple topics using the same node.
Receive Payload
Prior to version .20 the received payload was either a string or buffer object (if detected by the node)
In version .20 you can choose the expected payload type using a new drop down box as shown below.
If you select JSON the payload is converted into a JavaScript object and so you don’t need the JSON node on the output.
Editing A subscribe or Publish Node Using the Same Client connection Name
If 2 or more nodes anywhere in the Workspace use the same connection name e.g.
A publish node uses the connection name
node-client@192.168.1.157:1883 and another node or nodes (publish or subscribe) use that connection name. Then changes to the connection properties e.g port number,clean sessions etc will affect all of the nodes using that connection name.
Video
I’ve created a video on how to Publish and Subscribe to an MQTT broker Using Node Red
Related Tutorials
- Understanding the Node-Red Message Object
- Storing Data in Node-Red Variables
- Securing Node-Red with SSL and Username Authentication
- Node-Red HTTP Request Node for Beginners
All very clear, but in my opinion a fundamental part is missing (or I didn’t understand it). The “topic” to be sent, how do I get it? I mean the exact string of my device, with its payload attributes. Thanks in advance
The topic is any string that you want. There is no formal naming structure and so it is anything that you want it to be provided it follows some basic rules see
http://www.steves-internet-guide.com/understanding-mqtt-topics/
Hey Steve,
thank you for this tutorial!
I want to send a Byte array via MQTT. But somehow node-red is always packing it as JSON, which results in a way larger payload. Do you know how to manipulate the message, which will be send from MQTT?
Regards,
Toni
Modbus uses bytes and I did a tutorial on it and several videos which should help
http://www.steves-internet-guide.com/node-red-modbus/
I also did one on buffers which is probably the best place to start
http://www.steves-internet-guide.com/understanding-buffers-node-red/
rgds
steve
Do you have a example for the ssl Security in node-red ?
Hi
It is covered for the client here
http://www.steves-internet-guide.com/configuring-the-mqtt-publish-node/
The sever setup is very similar.
Rgds
Steve
Hi Steve,
Need some guidance on how to catch success or error from MQTT publish node.
I have a stream of incoming JSON files the contents of which need to be sent by MQTT.
If the publish is a success, I need to delete the file and if the publish is a failure, I need to keep the file for later resend.
Is this possible to achieve using NodeRed MQTT pulisher??
Thanks
Ajay
You need an acknowledgement from the receiver to be sure. Are you in control of the receiver?
Hello Steve,
I am trying to get data from my MQTT server and save it to MySQL using NODE-RED. However I am struggling to pull payload_raw value from one module (I can see it on MQTT) below what I get on MQTT every10min:
rg186/devices/4b14020b/up {“app_id”:”rg186″,”dev_id”:”4b14020b”,”hardware_serial”:”70B3D5B020038B08″,”port”:24,”counter”:8,”confirmed”:true,”payload_raw”:”/4hj+zz/5AEA9qBGAgAAAAAXAABIQxiPwqlBEwB9Ykc=”,”payload_fields”:{“battery”:65416,”light”:25595,”temperature”:156.15},”metadata”:{“time”:”2020-01-20T12:13:48.852152095Z”,”frequency”:867.3,”modulation”:”LORA”,”data_rate”:”SF7BW125″,”airtime”:92416000,”coding_rate”:”4/5″,”gateways”:[{“gtw_id”:”eui-c0ee40ffff296d5b”,”timestamp”:1505321075,”time”:””,”channel”:4,”rssi”:-29,”snr”:10.2,”rf_chain”:0,”latitude”:51.320107,”longitude”:-0.55894303,”location_source”:”registry”}],”latitude”:51.320133,”longitude”:-0.5588848,”location_source”:”registry”}}
Any advises how I should configure mqtt in and function to pull payload_raw and payload_fields?
Thank you in advance!
Hi
assume I assign your data to s
s=your data
the payload_raw is
var payload_raw=s.payload_raw
var t= s.metadata.time
var latitude=s.metadata.latitude
gateways
is
s.metadata.gateways
Hope that helps
rgds
steve
Hi Steve,
Thank you for your answer, however my problem is to setup right topic in “mqtt in”? I have now “rg186/devices/4b14020b/#” I do not really know how to access the payload, to get readings from ttn I use this command: mosquitto_sub -h eu.thethings.network -t “+/devices/+/up” -u “rg186” -P “ttn-account-***…” -v .
When there is new reading in MQTT will “mqttin” and function automatically save the data to database?
flow 1:
mqttin -> function -> MySQL
function:
var Data = msg.payload;
s = Data;
var payload_raw = s.payload_raw;
var t = s.metadata.time;
msg.topic = “INSERT INTO LoraTest (Bettery) VALUES (‘”+payload_raw+”‘)”;
msg.payload = [payload_raw];
return msg;
Hi
You need to contact me via the ask steve page and send me the flow you are using.
Rgds
Steve
Hi, i need help , how i can send a data from arduino mkr gsm to node red (not local) by using mqtt and mosquitto , thnks
Hi
Are you currently able to send and receive messages using arduino and mqtt?
no i can’t , i can send data from arduino mkr to node red(localhost) but to other server i can’t send
Hi
If it works locally then there is something wrong either with the remote broker (i.e. may need username/password etc) of firewall setting.
Check on an open broker like test.mosquitto.org and let me know
Rgds
Steve
hi , locally it works with COM but in this situation i don’t use serial COMi in input i use mqtt nodes and broker is not localhost , but no result
Hi
Can you use the ask steve page and send me the flow
Rgds
Steve
Steve,
Appreciate your work. Also saw your separate post on multiple mqtt brokers on same host. I have 1 raspberry on 2 separate routers/LANs (wlan0 and etho). One is for IOT (no gateway) and the other is for home network (gateway). I am trying to configure 1 mqtt broker in node-red for the IOT, and the another mqtt broker for the home network. Would like to pass messages from IOT broker to home network broker so that Home Assistant on another Pi can can get the messages. IOT broker works (local host port 1883), but adding a second broker with the Pi’s other network address (192.168.1.62 on port 1884) is not working.
Thoughts?? Thanks
Bob
It is probably the network bindings. Here is a copy of what I think is the relevant part of the mosq conf file
General Options
bind_address address
Listen for incoming network connections on the specified IP address/hostname only. This is useful to restrict access to certain network interfaces. To restrict access to mosquitto to the local host only, use “bind_address localhost”. This only applies to the default listener. Use the listener variable to control other listeners.
Not reloaded on reload signal.
bind_interface device
Listen for incoming network connections only on the specified interface. This is similar to the bind_address option but is useful when an interface has multiple addresses or the address may change.
It is valid to use this option together with bind_address for the default listener, or the bind address/host part of the listener definition. Care should be taken to ensure that the address being bound to is on the interface being bound to. If you set the bind_interface to be eth0, and bind_address to 127.0.0.1, then the broker will start correctly but you will be unable to connect.
This option is currently only available on Linux, and requires elevated privileges.
Thank you – it helped me a lot.
I have a few mosuitto mqtt brokers running on raspberry pi at home for home automation. Used node-red for programming und dashboards.
I’m on the way to set up in my camper a local hotspot with wifi, raspberry pi zero wifi with node-red and mosquitto mqtt broker. Connected ESP-8266 Wifi to measure battery power und temperature. More to come. It works within my camper.
Now I want to connect it to my home broker over the hotspots G4. I have also cloudmqtt, but also with an Domain set to my Home-IP. Its running, but now the challenge is to secure the communication with certificates and tsl/ssl. This is not my best known area.
Bruno
Hi I want provoid ssl certificate path from the function node to http node how to do that ? like we can set url like msg.url=”http://google.com” in function and send “msg” to http node
Confused as the tutorial is on the mqtt node not http
rgds
steve
Hi, I’m working on a project which using MQTT-Node RED. I need to get data from a website: “https://intelliweb.mastervolt.com”. I followed this instruction but my flow isn’t worked yet: https://www.symcon.de/forum/attachment.php?attachmentid=34745&d=1467291401 . I think I have some problem with TLS Configuration because I don’t know how to configure it.
Can you use the ask-steve page and send me the flow you are using
rgds
steve
What do you do is a topic publishes the same value all the time. What’s the easiest way in node-red of subscribing to that topic, but only doing something when that topic payload changes.
I do this with the topic logger which you can download form the site.
The trick is to store the last payload in a context,flow or global variable so that you can compare the last value to the current value.
If there is no change you just ignore
http://www.steves-internet-guide.com/node-red-variables/
http://www.steves-internet-guide.com/node-red-mqtt-topic-logger/
Does that help?
Rgds Steve
Good job!
However, I believe the port should be 8883 for TLS.
Yes correct 8883 is generally used for ssl and mqtt
Very detailed tutorial.