Understanding and Configuring Logging – Mosquitto Broker Configuration

Mosquitto has two different log types

  • System status logs
  • Informational and debugging logs

Broker System Status logs

These are always logged to the $SYS topic and can be accessed using an MQTT client subscribed to the $SYS/# topic.

System status logs are always enabled by default but you can disable them or change the refresh time using the setting:

sys_interval seconds

Setting this to zero will disable publishing of the hierarchy. The default interval is 10 seconds.

Different brokers will provide different $SYS log topics. This Github page has a list and description of the $SYS log topics split into optional and required.

Information and Debugging logs

These provide detailed information about clients, and client broker commands.

Using them you can see which clients are connected, subscribed etc.

Logging is configured in the Logging section of the mosquitto.conf file.

You can choose which event types to log, by default error, information, notice and warning events are logged.

For debugging I use log_type all.

This is essentially the same as starting mosquitto with the verbose switch – mosquitto -v

Here is a screen shot showing a client connect, subscribe and disconnect sequence and the console log messages that are generated.

mosquitto-verbose-logging

Note: On Windows logging is disabled when starting mosquitto as a service. This is because the broker is running as a background process and has no console attached.

Logging Destinations

You can log messages to multiple destinations.

For example you could send log messages to the console, to the $SYS topic, and to a log file.

The screenshot below shows my mosquitto.conf file configured to log to three destinations. (console, logfile and topic)

mosquitto-logging

You can also disable logging using the log_dest none option.

Viewing Topic logs in a Client

Normally you will view these logs on the Mosquitto broker console but you can also view them using an MQTT client if you have logged them to a topic.

If you send the log messages to topic they are sent to the $SYS/broker/log topic ,and you need to subscribe the topic tree $SYS/broker/log/#.

topic-logs

Two common question are:

  • Can I see all connected clients?
  • Can I list all topics?

The broker doesn’t let you do this directly but by enabling logging to a topic and monitoring the topic with an MQTT client you can get a good idea.

However using syslog and remote logging you can view connection details. See below.

Using Syslog

On Linux systems you can also use syslog to log the messages.

Messages are sent to the syslog file in the /var/log folder.

There are 2 settings

log_dest syslog
log_facility 5 #this is what I use

If you use syslog for logging it is possible to send all your logs to a central log server or to a log analyser.

Using the log_facility  allows you to only send logs from mosquitto.

Coming– Centralised mosquitto logging and analytics using node-red.

Historical Logs

On Linux if you log using to the file /var/log/mosquitto.log then the log file will automatically be rotated as an entry is made in the logrotate.d folder by the install script.

The file is shown below:

mosquitto-logrotate-file
If you look in the /var/log/mosquitto folder you will see the mosquitto.log file and zipped archives.

The default configuration archives the log file when it exceeds 100k.

Log rotation is handled by cron and runs once per day.

You can manually rotate the mosquitto logs by using the command

sudo logrotate -v /etc/logrotate.d/mosquitto

Viewing Mosquitto.log on Running Broker

When testing I start mosquitto manually using the verbose option to see the console.

However if you have a running server started automatically you don’t have access to the console.

You can still debug by using the tail command to view the log file with the -f option which will effectively show new entries as they are logged.

The command is

sudo tail -n20 -f /var/log/mosquitto/mosquitto.log

Viewing The $SYS logs

To view the system status logs you need to subscribe to the $SYS topic using a MQTT client.

The topic path is $SYS/#

You can use the mosquiito_sub client that comes with the broker as shown below.

In the screenshot below ws4 is the broker name.

mosquitto-sys-log

You can also use the Paho MQTT Python client see subscribing to topics.

If you use node-red I have created a flow that you can download that displays them in browser.

download

Log Timestamps

By default you will see the Unix times stamp format which isn’t really very useful when debugging from the console so you may want to make it more user friendly by adding the entry

log_timestamp_format %Y-%m-%dT%H:%M:%S

The logs should then appear as :

mos-logs-timestamp

Recommended Settings

Default log-types are :

error,warning, notice
and information

and they are sufficient.

On all systems I would log to a file.

/var/log/mosquitto/mosquitto.log

The install script automatically assigns permissions to the mosquitto user.

If you start mosquitto from the command line and try logging to the folder you will probably get an error as the user will not have permissions.

You can either change the permissions on the folder or alternatively change the log folder to a folder that you already have permissions to.

You could also start mosquitto as the mosquitto user using:

sudo -u mosquitto mosquitto -c mos2.conf

where mos2.conf is your test configuration file.

see run command as another user

Common Questions and Answers

Q- Are $SYS logs saved to disk?

A- No

Q– Why Do I get the message:  error: unable to open log file /var/log/mosquitto/mosquitto.log for writing.

A- This is a permission problem and is caused when you start mosquitto manually. You can either start mosquitto as the mosquitto user using sudo (shown above) or when testing change the log file location to your home directory.

Q- How can I list All topics?

A_ Mosquitto doesn’t provide this functionality by default but you can use topic logging as discussed above.

Q- How can I list All connected clients?

A- Use topic logging.

Q- Can I use the $SYS logs for monitoring the broker status?

A- Yes but not probably the best method and some brokers don’t have them enabled. If the broker doesn’t have any other methods built in (mosquitto doesn’t) then you would probably be better using an MQTT client to send background data and monitor that. See this article- why you shouldn’t use the $SYS topic for monitoring.

See also monitoring MQTT brokers .

Was This Useful?

Mosquitto Configuration Tutorials

Other Related Articles and Resources:

Save

Save

Save

Save

Please rate? And use Comments to let me know more

37 comments

  1. Hi Steve,
    Thanks for your article, it helps me so much.

    I have run the mosquitto on version 2.0.15 and on Linux,
    and I have some questions.
    This is my setting about the log file(mosquitto.conf) :
    ———————————————————————————–
    log_dest file ./mosquitto.log
    log_dest topic
    log_type all
    connection_messages true
    log_timestamp true
    log_timestamp_format %Y-%m-%dT%H:%M:%S
    ———————————————————————————–
    I run the mosquitto by using the command:
    ./src/mosquitto -c mosquitto.conf -v

    1. The log message printed on the log file successfully, but it wasn’t shown on the terminal screen.
    I would like to know if there is a way to print log messages on the log file and on the terminal.

    2. I am learning the way you write in this article to make a mosquiito_sub client subscribe to the topic of the log message, but it didn’t work. Except for the setting: log_dest topic, is there anything I need to set to run this function?
    If I ran the command: ./client/mosquto_sub -t $SYS/#,
    the subscriber got nothing.
    And if I ran the command: ./client/mosquto_sub -t \$SYS/#, this subscriber got some message of number which like:
    mosquitto version 2.0.15
    22 seconds
    0
    0
    0
    0
    0
    0
    1
    7.00
    1.71
    I can’t read these numbers,
    I wondered if I did anything wrong?

    Thank you so much for reading my message, wish you have a nice day.

  2. Hello, which is the timestamp that mosquitto uses for the logging? Can we configure the timestamp that it’s logged?
    Thank you in advance!

    1. Yes
      Folloing is taken from the docs here
      https://mosquitto.org/man/mosquitto-conf-5.html

      log_timestamp_format format

      Set the format of the log timestamp. If left unset, this is the number of seconds since the Unix epoch. This option is a free text string which will be passed to the strftime function as the format specifier. To get an ISO 8601 datetime, for example:

      log_timestamp_format %Y-%m-%dT%H:%M:%S

  3. When running mosquitto in Ubuntu from the command line, I can use the -V verbose option to display the output in real time. However when running it as a service and using “journalctl -f -u” to see the output in real time, it does not work. Setting “log_dest stdout” does not work because I believe the mosquitto service unit file redirects it to a log file.
    Is there ANY way to view the mosquitto output in real time when running it as a service?
    Thanks!

  4. Hi Steve, can I use mosquitto logging to print received messages too? I have some packet loss and I’m trying to understand where it’s coming from, if from the broker, the wireless connection or the publishers. I have only used logs to monitor clients and topics so I’m not 100% sure it’s possible.

    1. The logs will show you the published and received messages if you use verbose logging. However depending on the clients connected it can be very difficult to troubleshoot.
      rgds
      Steve

  5. Hi Steve
    Thanks for the useful information.
    I am working with Windows 10, I configured logging to a file: “log_dest file c:/mqtt/logs/mosquitto.log”
    Now I have the problem that I can not open the mosquitto.log file. I get the error that I don’t have permission to do so. What can I do to be able to view the log file?

  6. Permissions on the log file are set to restrictive (ie user (mosquitto) only). I would prefer this to be accessible for read to members of the mosquitto group (me 😉 ) too. Is it possible to configure this?

    This is a Linux setup – I have it logging to /var/log/mosquitto/mosquitto.log.
    Thanks

    1. Paul
      Tried to change but didn’t succeed.Will keep trying if you do find out before me then let me know.
      Rgds
      Steve

  7. Hi steve, I am trying to get a the log file but I do not know where it is located, this is the configuration that I did:
    config file:
    ——————————————————————————————————————————–
    # Note that if the broker is running as a Windows service it will default to
    # “log_dest none” and neither stdout nor stderr logging is available.
    # Use “log_dest none” if you wish to disable logging.
    #log_dest none
    log_dest topic
    log_dest syslog
    log_dest file C:\Archivos”de”programa\mosquitto\mosquitto.log
    ————————————————————————————————————————–
    am I doing something wrong?

    1. log dest syslog doesn’t work on windows. Not sure about the quotes in the pathname other than that it seems ok.
      I would start it from the command line and log to the local folder for testing
      Rgds
      Steve

  8. Hi there,
    I have a strange problem in my home

    Sometime arrive an open/close topic command for the chicken coop door in the middle of the night.
    There is 3 days that I found it opened in the morning.

    What I ask is how can I understand “who” send this message topic.
    So is there possibility to have for example a log with the IP of client topic please ?
    In this case if I know the IP I can understand from where the message arrive.

    Thanks a lot
    Denis

    1. If you look at the logs on the broker it will tell your the name of the client that sent it. You might also see the client connect with the IP address but the connection might have been a while ago as it might have been left open.

    1. Hi
      Not as far as I know you would have to use an external program to read the logs and put them in a database.
      rgds
      steve

  9. I don’t know if I am kidding myself and not actually seeing the mosquitto log but….
    If I start a terminal window and type:
    tail -f /var/log/mosquitto/mosquitto.log | perl -pe ‘s/(\d+)/localtime($1)/e’
    I will get a scrolling window that displays the last few log entries and will continue to add new lines as they are added to the log.
    Further more, the time stamp is in human readable time.
    Can this be what I think it is?

  10. I’m having issues connecting my ESP8266 to a local Mosquitto broker. It works if I connect my ESP8266 to my external IP but does not work internal IP.

    Is there anyway to log connections to see if the broker is seeing my ESP8266 trying to connect and rejecting/accepting the connection?

    1. Use log_type all.
      But you may find it better to stop mosquitto and start it manually using
      mosquitto -v
      this will show the connection attempts on the console
      Rgds
      Steve

  11. Thanks Steve,
    The mosquito logging explanation really helped. With this and other info I was able to get mqtt messages running (with a Raspberry Pi acting as the server) to and from a PC with a client running in Processing. Also a client running in Processing on the Rpi. Now moving on to interfacing multiple Arduinos to all this and probably more Rpi’s also . Mosquitto has run flawlessly – I just needed logging to see how to make the initial connections.
    Thank you again – very helpful……

  12. Hi Steve,
    Do you happen to know if more admin-level functions are logged as well, such as changing files the broker would use for ACLs or usernames and password changes through mosquitto_passwd?

    1. Patrick
      No there aren’t as far as I know but it could change as I know that it is being upgraded for mqtt v5 and that might contain additional features.

    1. Hi
      need to do more research on that but if you add the following to the conf file you will see lots of info on websockets
      log_type websockets
      websockets_log_level 9

      I need to check the log levels but level 6 seems to give basic info
      rgds
      steve

Leave a Reply

Your email address will not be published. Required fields are marked *