Mosquitto has two different log types
- System status logs
- Informational and debugging logs
Broker System Status logs
These are always logged to the $SYS topic and can be accessed using an MQTT client subscribed to the $SYS/# topic.
System status logs are always enabled by default but you can disable them or change the refresh time using the setting:
sys_interval seconds
Setting this to zero will disable publishing of the hierarchy. The default interval is 10 seconds.
Different brokers will provide different $SYS log topics. This Github page has a list and description of the $SYS log topics split into optional and required.
Information and Debugging logs
These provide detailed information about clients, and client broker commands.
Using them you can see which clients are connected, subscribed etc.
Logging is configured in the Logging section of the mosquitto.conf file.
You can choose which event types to log, by default error, information, notice and warning events are logged.
For debugging I use log_type all.
This is essentially the same as starting mosquitto with the verbose switch – mosquitto -v
Here is a screen shot showing a client connect, subscribe and disconnect sequence and the console log messages that are generated.
Note: On Windows logging is disabled when starting mosquitto as a service. This is because the broker is running as a background process and has no console attached.
Logging Destinations
You can log messages to multiple destinations.
Options are:
- topic – sent on topic tree
- stdout
- file
- syslog
For example you could send log messages to the console, to the $SYS topic, and to a log file.
The screenshot below shows my mosquitto.conf file configured to log to three destinations. (console, logfile and topic)
You can also disable logging using the log_dest none option.
Viewing Topic logs in a Client
Normally you will view these logs on the Mosquitto broker console but you can also view them using an MQTT client if you have logged them to a topic.
If you send the log messages to topic they are sent to the $SYS/broker/log topic , and you need to subscribe the topic tree $SYS/broker/log/#.
Two common question are:
- Can I see all connected clients?
- Can I list all topics?
The broker doesn’t let you do this directly but by enabling logging to a topic and monitoring the topic with an MQTT client you can get a good idea.
However using syslog and remote logging you can view connection details. See below.
Using Syslog
On Linux systems you can also use syslog to log the messages.
Messages are sent to the syslog file in the /var/log folder.
There are 2 settings
log_dest syslog log_facility 5 #this is what I use
If you use syslog for logging it is possible to send all your logs to a central log server or to a log analyser.
Using the log_facility allows you to only send logs from mosquitto.
Coming– Centralised mosquitto logging and analytics using node-red.
Historical Logs
On Linux if you log using to the file /var/log/mosquitto.log then the log file will automatically be rotated as an entry is made in the logrotate.d folder by the install script.
The file is shown below:
If you look in the /var/log/mosquitto folder you will see the mosquitto.log file and zipped archives.
The default configuration archives the log file when it exceeds 100k.
Log rotation is handled by cron and runs once per day.
You can manually rotate the mosquitto logs by using the command
sudo logrotate -v /etc/logrotate.d/mosquitto
Viewing Mosquitto.log on Running Broker
When testing I start mosquitto manually using the verbose option to see the console.
However if you have a running server started automatically you don’t have access to the console.
You can still debug by using the tail command to view the log file with the -f option which will effectively show new entries as they are logged.
The command is
sudo tail -n20 -f /var/log/mosquitto/mosquitto.log
Viewing The $SYS logs
To view the system status logs you need to subscribe to the $SYS topic using a MQTT client.
The topic path is $SYS/#
You can use the mosquiito_sub client that comes with the broker as shown below.
In the screenshot below ws4 is the broker name.
You can also use the Paho MQTT Python client see subscribing to topics.
If you use node-red I have created a flow that you can download that displays them in browser.
Log Timestamps
By default you will see the Unix times stamp format which isn’t really very useful when debugging from the console so you may want to make it more user friendly by adding the entry
log_timestamp_format %Y-%m-%dT%H:%M:%S
The logs should then appear as :
Recommended Settings
Default log-types are :
error,warning, notice and
information
and they are sufficient.
On all systems I would log to a file.
/var/log/mosquitto/mosquitto.log
The install script automatically assigns permissions to the mosquitto user.
If you start mosquitto from the command line and try logging to the folder you will probably get an error as the user will not have permissions.
You can either change the permissions on the folder or alternatively change the log folder to a folder that you already have permissions to.
You could also start mosquitto as the mosquitto user using:
sudo -u mosquitto mosquitto -c mos2.conf
where mos2.conf is your test configuration file.
see run command as another user
Common Questions and Answers
Q- Are $SYS logs saved to disk?
A- No
Q– Why Do I get the message: error: unable to open log file /var/log/mosquitto/mosquitto.log for writing.
A- This is a permission problem and is caused when you start mosquitto manually. You can either start mosquitto as the mosquitto user using sudo (shown above) or when testing change the log file location to your home directory.
Q- How can I list All topics?
A_ Mosquitto doesn’t provide this functionality by default but you can use topic logging as discussed above.
Q- How can I list All connected clients?
A- Use topic logging.
Q- Can I use the $SYS logs for monitoring the broker status?
A- Yes but not probably the best method and some brokers don’t have them enabled. If the broker doesn’t have any other methods built in (mosquitto doesn’t) then you would probably be better using an MQTT client to send background data and monitor that. See this article- why you shouldn’t use the $SYS topic for monitoring.
See also monitoring MQTT brokers .
If you found this guide useful then perhaps you would like to Mosquitto Configuration Tutorials
- Installing and Testing the Mosquitto MQTT broker
- Quick Guide to The Mosquitto.conf File With Examples
- Configuring and Testing MQTT Topic Restrictions
- Configuring Username and Password Authentication
- Configuring TLS (SSL) on Mosquitto
- Understanding and Configuring Bridging on Mosquitto
Other Related Articles and Resources:
Hello.
I’m just starting to look into using MQTT, I can’t use Python and am afraid I’m getting too old to learn it. I’m using a Tasmota 4 relay as I thought it would be a simple learning tool. I’m on Win10 and Mosquitto, not sure which version but it is the latest.
I’m confused by some of the comments about logging with a Windows system. Can you please clarify.
You stated,
“Note: On Windows logging is disabled when starting mosquitto as a service. This is because the broker is running as a background process and has no console attached.”
But you have Standard Out and Log to File, among others, in you example. Is your example from a Windows computer or Linux? Is there a way to enable logging on a Windows computer? Is it that logging in Windows can only be achieved when not running as a Windows service? If that is the case how do you run Mosquitto not as a service? If I don’t have Mosquitto running as a service I can’t figure out how to start it to send mosquitto_pub messages.
When running as a service, I can use these lines at the command prompt, and a variation in my script, to toggle the relay channels. I’m running on a LAN that is isolated from the internet.
“cd “C:\Program Files\mosquitto\”
mosquitto_pub -h 192.168.2.2 -p 1883 -t cmnd/tas4relay/POWER1 -m TOGGLE -r -V 5 ”
And this line shows the status when using MQTT_Viewer_v0.4.1.
“mosquitto_rr -t stat/tas4relay/POWER1 -e cmnd/tas4relay/POWER1 -n -v -d”
However, I don’t know how to retrieve the status for my program, which is written with AutoHotKey. Since MQTT Viewer accesses the information when running Mosquitto as a service, do I need logging to get the info for my script? In this case it tells me if the relay is on or off.
If I need to use logs, I may be able to use standard out, if Mosquitto will use it on Windows, but I know I could use a file. First I need to know if Mosquitto will do logging on Windows and, if it can, then I could use a suggestion about what would be the best method.
I appreciate all the effort you put into these tutorials, I’ve learned quite a bit already. Even without basic Python knowledge I have learned to control the relays. That’s saying a lot. Thanks
Richardd tks for the coffee.
The screenshots are linux as that is what I use for mosquitto.
However you can run the broker as a service or from a command line . I recommend that for testing you use the command line so you can view the console.
For linux and windows you need a basic configuration file and it is just 2 lines and show at the top of this page
http://www.steves-internet-guide.com/mosquitto-broker/
also add a logging line so the file should look like this
log_type all
log_dest stdout # this you might need to remove if you get an error
listener 1883
allow_anonymous true
but them in a file called test.conf and start using
mosquitto -c test.conf
If you don’t want to stop the broker running as a service then adjust the conf file to use a different port.
You can have lots of brokers running on the same host as long as they use different listeners.
Get back to me if you still have problems and I’ll do a video.
I will try and set it up on my machine later today and post any new thoughts
rgds
steve
Thanks for the quick reply. I worked on this most of the night and figured out most everything except getting info back from the relay. I got Mosquitto working without running it as a service, although I have a Mosquitto console open but nothing every appears in it. Can’t type in it and it doesn’t show any messages so I just run it hidden. I think the problem I was having is it takes a minute for mosquitto.exe to connect to everything before it will accept a command.
I’ve got the log file working but I’m not sure it’s going to do me any good. It only shows the status of the relays when the log all configuration is set. There are other devices I use that were automatically found that I didn’t realize were set to use MQTT, and now it logs a ton of data, so it’s going to be difficult for me to find a way to pull what I need.
Even then I’m not sure the log files are where I need to get the returned information from. MQTT Viewer was receiving the info before I got logging working. So maybe there’s something I’m missing about how to get the data from the subscribed topics. So this is the only question I have left at the moment.
Current it’s just if relay channels are on or off but the main reason I’m wanting to learn MQTT is to control my 3d printer. I have all my machines on an isolated LAN and Bambu Lab doesn’t like that. As punishment for not sending everything through their cloud and spyware app they block most of the functionality of the machine. It was recommended to me to use the MQTT interface but I don’t know of anyone who has actually used it so I’m trying to figure it out for myself.
While researching last night I did find a bunch of Python scripts for using the printer’s api, so now I’m wondering if I should be learning Python instead. I would think that since the script are already written the hard part is done, but I have no idea.
The second reason I wanted to learn MQTT is the few Tasmota devices, like this relay, I have in my house. I thought the lag time would be improved using MQTT instead of the web interface but there is no difference or MQTT may be slower. Some devices respond immediate for every command, while some can occasionally take up to a minute and a half to respond. Since there is no difference in the response I don’t see any advantage to switch MQTT instead of staying on the HTTP interface.
Ok, enough droning on. If you can tell me the best way to get the information from a subscribed topic, and maybe point me to a video or instruct page, I’ll work on that. If I need to ask more questions I’ll get organized so I can show you what I’ve tried.
Thanks again.
Richard
When running mosquitto from the command line the console is read only. You cannot enter commands into it.
If you have lots of other devices on the network then run mosquitto on a different port (1884). If you aren’t see the 3d printer send commands on the console then you should check the 3d printer as it either isn’t sending them or for some reason can’t see the broker.
you can also use MQTT explorer and subscribe to the # topic but it doesn’t give you as much detail as the console.
regarding the tasmota devices then speed wise there is not really any difference between http and MQTT but MQTT is simpler as you don’t need to know the IP address of the device.
If you are happy to run python scripts I can send you some test scripts that should help you get familiar with MQTT.
If you have a link to the 3d printer manual I will take a quick look at the MQTT configuration .
Rgds
Steve
I use mosquitto_pub to send commands, (AutoHotKey script)
“RunWait, mosquitto_pub -r -d -h 192.168.2.2 -p 1883 -t cmnd/tas4relay/POWER1 -m TOGGLE, , Hide UseErrorLevel”
and mosquitto_rr to subscribe to the topics.
“RunWait, mosquitto_rr -t stat/tas4relay/POWER1 -e cmnd/tas4relay/POWER1 -n -v -d `r”
I am under the impression that mosquitto_rr is better for when you want to quickly get info from a subscribed topic, like from a program that only runs for short periods. For instance, start a program to see if a light is on or off. Is mosquitto_rr correct or should I be using mosquitto_sub?
I don’t get any output to the mosquitto.exe window and not sure why or if it matters.
“Run, mosquitto.exe -v -c mosquitto.conf, , Hide”.
However, MQTT explorer was getting the info even before I got logging to work, and it used plain English saying something like “Relay 1 On”. I see in the log where I believe the info is being logged in code, but I wonder how MQTT explorer was receiving the data… which may be due to my next question.
I was unsuccessful when trying to use log_type topic but, thought it wasn’t available on Windows. The line in the config file directly under logging says “If using syslog logging (not on Windows)”. Maybe I misunderstood that part? Do I need to subscribe “$SYS/#” using Windows? I’ll work some more on that tonight.
I haven’t gotten to the 3d printer yet as I thought learning using the relay would be easier. I do believe the printer was one of the other devices mosquitto was seeing and logging. I’m sure I’ll have plenty of questions once I get there but I’m not in a hurry. However the API and MQTT information can be found here.
https://mchrisgm.github.io/bambulabs_api/api/mqtt_client.html
Thanks again
Hi
You need to use mosquitto_sub
The command
mosquitto.exe -v -c mosquitto.conf
will not work unless you have edited the file as described previously as the default mosquitto.conf file is all commented out and so is effectively blank.
Do not edit the file but create a new one for testing.
I think the log_type topic will work will try it tomorrow.
The subscribe to $SYS/broker/log/#
Rgds
Steve
Thanks Steve
Currently I’m going through all the logs types, destinations, $Sys, ect. that I can find to get a idea of what each does. I haven’t found where to get the on/off status yet but I’ll stumble onto it eventually.
FYI, I don’t see log_type topic as an option, and when I try to use it mosquitto crashes at startup. There is a log_dest topic I have active and am learning what it does.
Richard
Lost_dest topic
is corret
rgds
ssteve
Ok, disregard my last post. Just use the correct program and it magically works. And now that it’s working I better understand the relationships of the three programs. pub.exe, the broker tells the client to do something. sub.exe the broker tells the client to send any messages generated. rr.exe, the broker tells the client to update and send another message. Makes more sense when using things they way they were mean to be used.
It’s now sending back the last command received instead of the actual power status (On or Off), but I think I’m to the point I can figure more of it out on my own.
However mosquitto.exe still crashes when I have log_type topic set. I wonder if that’s what I need to get the actual power status.
Richard
I will check the log_type topic and let you know
Rgds
Steve
So I got the Relay control working last night. All I ended up using was log_dest syslog and log_type information.
When the script runs it opens four sub hidden windows, one for each channel of the relay, and then I can send a publish command for a channel and the sub window for that channel writes either on or off. The commands can either be On, Off, or Toggle, or a status command, but they all return the same on or off response. This does work but would there be a better way than opening four hidden windows?
I finally figured out how to send the output of the hidden sub windows to a file that my script can read. Since it’s reading and writing to a file it’s still a little clunky, but will do until I learn more.
One reason I am just writing to a file is there seems to be a problem with adding log_dest stdout. When I un-comment it in the config file Mostquitto won’t start. With in three or four seconds of trying to start it up it crashes. log_dest stderr works fine so I’m confused… again. I did install Mosquitto as a Windows 10 sevice but I disabled it in the Services window, so I don’t think that’s where the problem is. In my research I did see others ways to get the info to a standard out but haven’t et looked very far into them yet.
I’m considering running Mostquitto on a Pi, thinking it may be a little easier that way.
If you have any thoughts as to what the stdout problem could be please let me know.
Thanks again for all our help.
Richard
will try it this weekend and let you know. Pi is a good option I use it a lot.
Rgds
Steve
Richard
used this conf file
log_type all
connection_messages true
log_dest file C:\mos\mos.log
log_timestamp true
log_timestamp_format %Y-%m-%dT%H:%M:%S
log_dest topic
log_dest stdout
listener 1883
protocol mqtt
allow_anonymous true
and it started and worked ok. Without the line
log_dest stdout
noting is displayed on the console.
Rgds
Steve
Hi Steve,
Thanks for your article, it helps me so much.
I have run the mosquitto on version 2.0.15 and on Linux,
and I have some questions.
This is my setting about the log file(mosquitto.conf) :
———————————————————————————–
log_dest file ./mosquitto.log
log_dest topic
log_type all
connection_messages true
log_timestamp true
log_timestamp_format %Y-%m-%dT%H:%M:%S
———————————————————————————–
I run the mosquitto by using the command:
./src/mosquitto -c mosquitto.conf -v
1. The log message printed on the log file successfully, but it wasn’t shown on the terminal screen.
I would like to know if there is a way to print log messages on the log file and on the terminal.
2. I am learning the way you write in this article to make a mosquiito_sub client subscribe to the topic of the log message, but it didn’t work. Except for the setting: log_dest topic, is there anything I need to set to run this function?
If I ran the command: ./client/mosquto_sub -t $SYS/#,
the subscriber got nothing.
And if I ran the command: ./client/mosquto_sub -t \$SYS/#, this subscriber got some message of number which like:
mosquitto version 2.0.15
22 seconds
0
0
0
0
0
0
1
7.00
1.71
I can’t read these numbers,
I wondered if I did anything wrong?
Thank you so much for reading my message, wish you have a nice day.
Hi
you need
log dest stdout
in the comfig
and try
mosquitto_sub -h localhost -t $SYS/# -d
and let me know
Rgds
Steve
It’s work!
Thank you so much~~
Good news. Well done
Hello, which is the timestamp that mosquitto uses for the logging? Can we configure the timestamp that it’s logged?
Thank you in advance!
Yes
Folloing is taken from the docs here
https://mosquitto.org/man/mosquitto-conf-5.html
log_timestamp_format format
Set the format of the log timestamp. If left unset, this is the number of seconds since the Unix epoch. This option is a free text string which will be passed to the strftime function as the format specifier. To get an ISO 8601 datetime, for example:
log_timestamp_format %Y-%m-%dT%H:%M:%S
Is it possible to have the format with miliseconds?
Is that the log time?
Rgds
Steve
When running mosquitto in Ubuntu from the command line, I can use the -V verbose option to display the output in real time. However when running it as a service and using “journalctl -f -u” to see the output in real time, it does not work. Setting “log_dest stdout” does not work because I believe the mosquitto service unit file redirects it to a log file.
Is there ANY way to view the mosquitto output in real time when running it as a service?
Thanks!
Hi
Did you try
sudo tail -n20 -f /var/log/mosquitto/mosquitto.log
rgds
Steve
Hi Steve, can I use mosquitto logging to print received messages too? I have some packet loss and I’m trying to understand where it’s coming from, if from the broker, the wireless connection or the publishers. I have only used logs to monitor clients and topics so I’m not 100% sure it’s possible.
The logs will show you the published and received messages if you use verbose logging. However depending on the clients connected it can be very difficult to troubleshoot.
rgds
Steve
Thank you!
Hi Steve
Thanks for the useful information.
I am working with Windows 10, I configured logging to a file: “log_dest file c:/mqtt/logs/mosquitto.log”
Now I have the problem that I can not open the mosquitto.log file. I get the error that I don’t have permission to do so. What can I do to be able to view the log file?
You need to open as administrator. Open notepad as administrator and located the log file through notepad. To open notepad as administrator locate the notepad icon using the search and then right click on it and select run as administrator.
see
https://www.quora.com/How-do-I-run-Notepad-as-an-administrator-in-Windows-10
Permissions on the log file are set to restrictive (ie user (mosquitto) only). I would prefer this to be accessible for read to members of the mosquitto group (me 😉 ) too. Is it possible to configure this?
This is a Linux setup – I have it logging to /var/log/mosquitto/mosquitto.log.
Thanks
Paul
Tried to change but didn’t succeed.Will keep trying if you do find out before me then let me know.
Rgds
Steve
Were you able to find any solution to this?
Hi steve, I am trying to get a the log file but I do not know where it is located, this is the configuration that I did:
config file:
——————————————————————————————————————————–
# Note that if the broker is running as a Windows service it will default to
# “log_dest none” and neither stdout nor stderr logging is available.
# Use “log_dest none” if you wish to disable logging.
#log_dest none
log_dest topic
log_dest syslog
log_dest file C:\Archivos”de”programa\mosquitto\mosquitto.log
————————————————————————————————————————–
am I doing something wrong?
log dest syslog doesn’t work on windows. Not sure about the quotes in the pathname other than that it seems ok.
I would start it from the command line and log to the local folder for testing
Rgds
Steve
Hi there,
I have a strange problem in my home
Sometime arrive an open/close topic command for the chicken coop door in the middle of the night.
There is 3 days that I found it opened in the morning.
What I ask is how can I understand “who” send this message topic.
So is there possibility to have for example a log with the IP of client topic please ?
In this case if I know the IP I can understand from where the message arrive.
Thanks a lot
Denis
If you look at the logs on the broker it will tell your the name of the client that sent it. You might also see the client connect with the IP address but the connection might have been a while ago as it might have been left open.
Hi steve , is there any way to push the connection logs of clients from broker to databases?
Hi
Not as far as I know you would have to use an external program to read the logs and put them in a database.
rgds
steve
I don’t know if I am kidding myself and not actually seeing the mosquitto log but….
If I start a terminal window and type:
tail -f /var/log/mosquitto/mosquitto.log | perl -pe ‘s/(\d+)/localtime($1)/e’
I will get a scrolling window that displays the last few log entries and will continue to add new lines as they are added to the log.
Further more, the time stamp is in human readable time.
Can this be what I think it is?
Hi
Not sure what you were expecting?
I’m having issues connecting my ESP8266 to a local Mosquitto broker. It works if I connect my ESP8266 to my external IP but does not work internal IP.
Is there anyway to log connections to see if the broker is seeing my ESP8266 trying to connect and rejecting/accepting the connection?
Use log_type all.
But you may find it better to stop mosquitto and start it manually using
mosquitto -v
this will show the connection attempts on the console
Rgds
Steve
thanks that did it.
Thanks Steve,
The mosquito logging explanation really helped. With this and other info I was able to get mqtt messages running (with a Raspberry Pi acting as the server) to and from a PC with a client running in Processing. Also a client running in Processing on the Rpi. Now moving on to interfacing multiple Arduinos to all this and probably more Rpi’s also . Mosquitto has run flawlessly – I just needed logging to see how to make the initial connections.
Thank you again – very helpful……
Hi Steve,
Do you happen to know if more admin-level functions are logged as well, such as changing files the broker would use for ACLs or usernames and password changes through mosquitto_passwd?
Patrick
No there aren’t as far as I know but it could change as I know that it is being upgraded for mqtt v5 and that might contain additional features.
Hi Steve,
Any chance to show the content of the message published on the websocket?.
Thanks in advance.
Hi
need to do more research on that but if you add the following to the conf file you will see lots of info on websockets
log_type websockets
websockets_log_level 9
I need to check the log levels but level 6 seems to give basic info
rgds
steve