Understanding and Using the Mosquitto Dynamic Security Plugin

The dynamic security plugin can be used instead of the password file and the ACL (Access Control list) .

The plugin is available in mosquitto v2 but it is not enabled by default.

If you decide to use it then the manual recommends you use it for all listeners and don’t have some using the old password file and other listeners using the security plugin. Use:

per_listener_settings false

To configure the broker to use it then you need to set the path to the plugin, and also the path to the security file.

The security file is a JSON file.

Linux:

plugin path/to/mosquitto_dynamic_security.so
plugin_opt_config_file path/to/dynamic-security.json
Example:

plugin /usr/lib/x86_64-linux-gnu/mosquitto_dynamic_security.so
plugin_opt_config_file /home/steve/mos/v5/test.json


Windows:

plugin path\to\mosquitto_dynamic_security.dll
plugin_opt_config_file path\to\dynamic-security.json


To locate the plugin on  Linux use

whereis mosquitto_dynamic_security.so

Using the Plugin

You first need to edit the config file as shown above, and also to create a dynamic-security.json file.

The file can have any name and a starter basic file is created using the mosquitto_ctrl utility that comes with mosquitto v2.
Use:

mosquitto_ctrl dynsec init path/to/dynamic-security.json admin-user

example:
mosquitto_ctrl dynsec init path/to/dynamic-security.json steve


Where steve is the user that I will use for configuring the security settings.

You must do this on the broker itself or you will need to copy the file to the broker.

The proceeding command is the only mosquitto_ctrl command that doesn’t require that you specify a broker and port.

The mosquitto_ctrl utility is basically a modified mosquitto_pub client and publishes messages to the $CONTROL topics on the broker. When you generate the default file the user steve (my example) will be given the required permissions in the file. You could manually edit the security file to modify the security settings but this is too complex. Instead the mosquitto_ctrl utility is then used to dynamically modify the security settings on a running broker and can be run from any machine that has the utility installed. It is probably best that you use SSL when running this utility on a production broker but when testing I wouldn’t use SSL. Key Concepts The plugin lets you configure security using three objects- client, group and role objects. The client is actually the user name you use to access the broker assuming that you are not allowing anonymous access. Client Object This has the following properties: • user name • password • client Id (optional) • groups • roles • text name – a friendly name • text description • enabled – a client can be disabled A client can belong to a group/groups and assigned to a role/roles it can also be enabled and disabled. If the client_id is not specified then the username/password can be used by multiple connections which is the case for mosquitto when using standard password files. note: Need -c option for client_id Groups Object This has the following properties: • Group name -Used when modifying group • Roles • Friendly name (optional) • Description (optional) Roles Object The Roles object contains the following properties • Role name • Access Control List (ACL) • Friendly name (optional) • Description (optional) The roles object is used to give clients access to topics. Clients,Groups and Roles Clients can be assigned directly to roles and given or denied access to topics, but doing so is not as flexible as using groups. For example if 10 clients are assigned to role1 and you decide that you need to assign different access to 5 clients and so need to assign them to role2 then you would need to make 5 changes. However if you had assign 5 clients to group1 and 5 to group2 and assigned both groups to role 1 then you would only need to make 1 change to the group. That is provided the clients had been assigned to the correct group to start with. So try and use client–>Group –>Role and not client –> Role. How it works To understand how security is applied we will start using the default security file generated using mosquitto_ctrl. A snapshot of the file is shown below: The plugin uses events that trigger an ACL check. So for example when a client subscribes an ACL lookup would be triggered. You can see from the file that I have marked it into three blocks. Default Access The default access permissions in the default configuration file are defined in the defaultACLAcess entry (bottom block) and are very restrictive, and only grant access to the admin user. All other users and anonymous users (if enabled) are not allowed to publish or subscribe to any topics, not even the$SYS topic

You can see from the defaultACLAcess entry that we have four broker events that can be used to configure ACL permissions.

We have the unsubscribe and subscribe events which are triggered when a client subscribes or unsubscribes.

We also have the publishClientSend event which is triggered when the client published a message to the broker, and the publishClientReceive event when the broker needs to send a message to the client.

As stated in the manual there is an overlap between the subscribe and publishClientReceive events and usually using the subscribe event is sufficient.

However the publishClientReceive event gives you more control as you can subscribe to a wild card topic but still restrict access to certain topics in the hierarchy.

E.g

topic test/# subscribe allow

would allow access to all topics in the test hierarchy except test/secret

If a client supplies a username and password then the logon credentials will be checked against the client entry if any.

If the client doesn’t have an assigned role or group then the defaultACLAcess permissions will apply.

If allow anonymous is disabled then the connection must supply a valid username and password, defined in the configuration file as a client entry.

It is possible to simulate the old passwords file just by simply creating client entries with no roles or groups and setting all entries in defaultACLAcess to true.

"defaultACLAccess": {
"publishClientSend": true,
"subscribe": true,
"unsubscribe": true
}


The entry for the admin user is only necessary if you want to manage the settings using mosquitto_ctrl.

Priorities

A priority can be applied to groups and roles. The default priority is -1.

If a client has multiple roles assigned or is a member of multiple groups then you need to understand the order in which ACL permissions are applied.

The first check is made on the role assigned to the client.

If a client(user) has two roles for example role1 and role2 and role role1 and a priority of -1 (default ) and role2 has a priority of 5 then the permission of role2 will take affect.

If a client is assigned to a role or group with ACL permissions set then these ACL permissions will override those in the defaultACLAcess entry.

The documentation gives a more detailed but complex overview of priorities.

I would suggest you keep them as simple as possible.

Anonymous Group

If you have allowed anonymous access then all anonymous users will be placed in the anonymous group.

The name of this group needs to be created and set using:

mosquitto_ctrl <options> dynsec createGroup <groupname>
mosquitto_ctrl <options> dynsec setAnonymousGroup <groupname>

example
mosquitto_ctrl -h 192.168.1.91 -u steve -P password dynsec createGroup anonymous
anonymous mosquitto_ctrl -h 192.168.1.91 -u steve -P password dynsec setAnonymousGroup anonymous

You can now assign the group to a role and ACL permissions to that role.

ACL Worked Example

Configuration – The default defaultACLAcess is set as follows:

"defaultACLAccess": {
"publishClientSend": false,
"subscribe": false,
"unsubscribe": true
}


Client john is assigned to role1 and role 2. Role 1 has all allow permissions to topic test with priority -1 and role 2 is configured with deny subscribe to topic test with priority of 5.

Q1 Will john be able to subscribe to topic test?

Q2 Will john be able to publish to topic test?

Q3 What will be the effect of changing the permissions for role1 to deny publish.

Q4 If You change permissions to role1 will you need to restart mosquitto.

Answers at the end of the tutorial

Making Changes

You can edit the security file manually and this is likely to be the case with the initial security file.

However after initial deployment changes are made dynamically using the new mosquitto_ctrl tool which we look at in the next tutorial.

Q1 _can I use the %u and % c syntax like in the old ACL file?

A- It doesn’t appear so

Q2 – Should I move to the new plugin?

A2- Unless your ACL permissions change frequently then I think you would be better with the old files.

Q1 -role2 has priority and there is a deny permission to topic test.. So  John cannot subscribe to test

Q2 role2 has no entry for publish for which means it will look in role1 which has an entry and so John can publish.

Q3-John cannot publish

Q4 – No it is dynamic.

Important Note: When using clientid then you need to use the -l option and not -c.

Related Resources and Tutorials

1. Mariano says:

Hi Steve!

Is there a way to create a single group with the same role that has the same restrictions on the topics but that is with the client id or name?

Example

I have the users test1, test2, test3
I want this role to have, for example, permission to subscribe to a topic that has its own name or id

* subscribeLiteral /test/(%c) or /test/(clientName)

So test1 can subscribe to /test/test1
test2 can subscribe to /test/test2
test3 can subscribe to /test/test3

I want to see if this can be possible somehow as the other option I have left would be to create a role and group with the constraints for each and if I have 100,000 clients it would be hell to create the client, group, role and all the constraints for topics for each

If what I have said before is possible, I would only have to create the client and assign the group to it

Do you know if there is or exists a way to do this?
If necessary I can provide other examples if it is not understood

1. steve says:

Is there a reason why you don’t use the old acl file as that will work as you want it.
Rgds
Steve

1. Mariano says:

Yes I have seen and tested that in the old acl file you can do that, but if I am correct with everything I have seen, if I use the old way of the acl file, the client when publishing or subscribing to any topic mosquitto connects and it pretends to publish or subscribe, but in reality no topics or messages arrive since in theory you have your restrictions in your acl file, but in the acl of mosquitto ctrl directly denies you and removes you from the sub or pub.

What I want to avoid is that if there is something misconfigured, Mosquitto will deny you, take you back and tell you no, but with the old way of acl it can be confusing to detect constraint failures since it always accepts you

I’m right?

1. steve says:

definitely true for the old ACL files. I didn’t really test for the new style but I will try it and see.
Rgds
Steve

1. steve says:

As far as I can tell the broker doesn’t tell the client that it has denied subscribe or publish with the security plugin unless you use MQTTv5 and then only the publish message with qos of 1 and above.
Rgds
Steve

2. Hi Steve!
Thank you for all the information about Mosquitto. I usually read your blog and I learn a lot everytime. I remember that one of your guides was essential for my introduction on the MQTT world. I have a question about the ACLs that maybe you can answer.
Suppose I have the next structure: /srv/servers/[server]/sites/[site]. In this case I could have the next complete topic structures:
– /srv/servers/server1/sites/site1
– /srv/servers/server1/sites/site2
– /srv/servers/server2/sites/site1
And then I have five users: one of them can read and write everything in the broker, but the another ones have specific permissions.
user test_user_server1
topic /srv/servers/server1/#
user test_user_server2
topic /srv/servers/server2/#
user test_user_site1
topic /srv/servers/server1/site1/#
user test_user_site2
topic /srv/servers/server1/site2/#
user test_user_site3
topic /srv/servers/server2/site1/#
Let’s suppose I am test_user_site2, can I read the content of the messages published on /srv/servers/server1 and the previous topics (/srv/servers and /srv)? Or I will see just the topics path but no the messages in them? This question would be the same for the rest of users I mentioned, so with an answer about one would be enough.
I hope I explained myself well! And, of course, thank you very much in advance!
Regards,
Iván

1. steve says:

The easiest way to configure it would be to give each user explicit permissions on the respective server.
so
user 1 would have readwrite to server1 and a general setting or read

user user1

you would do likewise on the other brokers but using the appropriate user

Note: I haven’t tested it.
Rgds
Steve

3. Very nice article, as usual reference for mosquitto user guide.

Anyway I though Dynamic Security was my solution but after reading twice I’m not so sure.

I need to create users and perms on fly like dynamic but I don’t want to specify topic on each, may be I need a mix between to method. Let my explain, my goal is very simple one topic for all users and then sub topic for each user (so users are like in jail)
MyTopic/user1/#
MyTopic/user2/#
MyTopic/user3/#
Today I’m using old password and ACL files but I don’t like the need to change files each time and need to send HUP to reload config.
Any chance with dynamic to do that with just few lines, I mean user / group / role and just assign permission on MyTopic/%username/# or even MyTopic/%groupname/# as in ACL files?
Not sure what’s the best (and simple) way to do that

1. steve says:

I will take a look and get back
Rgds
Steve

4. Luis says:

Hi Steve, thanks for your explanations. I’m giving a client ID using the -i argument to all new clients, but the client ID never gets persisted at the json file. Everything is persisted correctly, user names, passwords, roles, groups, ACLs, etc. But the client ID defined when creating the client is not persisting. Even if a consult the information for any client (using the getClient option) I see empty the Clientid field. Have you ever had this issue?

1. steve says:

Hi
Yes this cropped up before and someone found it and posted it in the comments. Here is the comment
Instead of using -i, we should use -c to create client with client id associated to it. The following is the complete command for creating client with client id associated to it.
Let me know if it works and I will update the tutorial.
Rgds
Steve

1. Luis says:

Hi Steve, I just made the test and it works perfect. It seems mosquitto should update the documentation to indicate “-c” instead “-i” for providing the client id of new user. Worth to mention that “-i” still is correct to provide the client id for the admin connection that is performing the dynsec operation. Best reagards,

1. steve says:

Good news.
Rgds
Steve

5. Rafael says:

Hi, I’ve been trying to use mosquitto security plugin, but I’ve been facing a couple of issues. I managed to create the file as it was showed on this tutorial , so that I tried to include roles and clients in the file to create the file I typed this line:
mosquitto_ctrl dynsec init path/to/dynamic-security.json user
Then, I’ve tried to include roles or clients in the file for that I typed the following line:
mosquitto_ctrl -u user -P user dynsec createRole rolepizza

After enter this line I got the Connection error:Not authorized. I was wondering if this has something to do with the auth file that my broker is using, because to connect to the broker the client needs to inform the user and password, I also tried that so I typed the following line:

mosquitto_ctrl -u brokerUser -P brokerPassword dynsec createRole rolepizza

But still getting the same error. Anyway, the questions are:
can I use both procedures, the authentication file and dynamic security plugin at the same time?
If this isn’t the problem what would be ?

1. steve says:

Hi
You need to create the admin user first and use that use to make the changes. Do you have access to the broker and the file.
Rgds
steve

Hi Steve,
is there a way to implement authorization&authentication, i.e. user-password, in which this info is not sent in clear text? apart from TLS and PSK. Thank you so much for your helpful guides. Regards

1. steve says:

Not as far as I am aware but MQTTv5 support other authorisation methods that may do it.
Tgds
Steve

7. Muhammed Hashim says:

Is there any way to view the details in \$CONTROL/dynamic-security/v1.
I could push values into it and observe changes at the broker, but I can’t receive any values for get methods.
Am using MQTT explorer as my client and running broker in a ubuntu VM.

1. steve says:

Hi what get methods are you using? Do you have a link?
Rgds
Steve

8. Alexander Thuo says:

Hi Steve,
I want to integrate an Auth API endpoint with my MQTT Broker, how would I do that? Should I use the dynamic plugin or just the password and ACL files

1. steve says:

Hi
Sorry but can’t give you an answer at the moment I will have to give it some thought and get back.
Rgds
Steve

9. María says:

Hi Steve,

I would like to know what are the advantages and disadvantages of using the dynamic security plugin instead of the ACL and the password file.

What would you recommend? Using the dynamic security plugin or the ACL and password file?

Thanks!!

1. steve says:

The dynamic security plugin is very useful if you have multiple brokers and/or need to make frequent security changes.
On small installations wit one or two brokers and infrequent changes then stay with the password file and ACL list as it is easier.
Rgds
Steve

1. María says:

So for example, in the case of an industrial production enterprise that has different headquarters in various countries but that is all connected to the same broker, which one would you recommend?
I ask it because my final degree project covers this area and I would like to know which path is more recommended to follow.

Regards,

María

1. steve says:

With a single broker then either would do. However if it needs remote management the the security plugin is the better option.
Rgds
Steve

10. Nat says:

I was wondering, is there a way for the client that connects to know which group/roles are assigned to them ?

1. steve says:

No there isn’t as far as I know as you need admin privileges to see this information. But I will look into it further.
Rgds
Steve

11. Dorian says:

hey !

I am a little bit confused here. Are these commands actually applied also dynamically on Mosquitto service? Or it needs to be restarted each time I set up/add new rules?!

The config file is also written in case the service/system is restarted/rebooted?!

I asked this because I’d like to develop an interface where I should add/generate/manage clients &topics & ACL … and all of these stuff, without actually stopping the service. It will be possible/applicable to do that?

thanks !

ps: nice article! 🙂

1. steve says:

Hi
There is no need for a restart as it is dynamic
rgds
steve

12. Mike Sandstrom says:

Hi Steve,

To take this further, if I want to modify the json file dynamically based off of MQTT messages from clients, what approach could I take? Could I use a python client on the broker that uses the ‘os’ module to interact with the mosquitto_ctrl tool?

1. steve says:

Mike
Yes that is what I would do. It is on my to do list along with lots of other things but I might do it in node-red as it is easier to build a web interface for it.
rgds
steve

1. Mike says:

got it, thanks for the reply Steve, I hope all is well.

13. Werner says:

Minor nitpicking: there are a few “Recieve” in the text. These should be “Receive”. (I did check the code, just in case 🙂

1. steve says:

Hi
Tks for pointing that out. I have edited the page and hopefully found them all.
Rgds
Steve

14. Lukas says:

Hi everyone,
just a quick side note. Don’t forget to restart the broker after setting everything up.
Otherwise the plugin will not give any response (no error either).

And one quick question. Shouldn’t I be able to a new client in the dynamic-security.json after “createClient”?
In which file would it be stored then?

Best and thanks for this Blog!

15. Mike says:

On windows I can find mosquitto_dynamic_security.dll, but on raspberry pi and a virtual private server running debian I can’t seem to find mosquitto_dynamic_security.so ?

I tried
>whereis mosquitto_dynamic_security.so
but it just returns
mosquitto_dynamic_security:

I tried looking and searching with slightly modified names but no luck, any ideas please?

Thanks,
Mike

1. steve says:

I didn’t find it either it is on my todo list to find out why. Presently I am using the windows one to do the testing as it runs remote anyway.
Rgds
Steve

1. Mike says:

Thanks for letting me know it’s same on your system, I downloaded mosquitto source and compiled the plugin (untested so far).
These are the steps I took..

1. Needed cJSON for linking, so from my home directory did:

mkdir cjson
cd cjson
git clone https://github.com/DaveGamble/cJSON.git ./
sudo make install

2. get source for mosquitto (but build won’t currently work for me because of websockets library error, but get source anyway for the plugin source code), so from my home directory did:

mkdir mosquitto2
cd mosquitto2
git clone https://github.com/eclipse/mosquitto.git ./

3. build and install the dynamic_security plugin

cd /home/pi/mosquitto2/plugins/dynamic-security
sudo make install
(The output from that gives: “/usr/local/lib/mosquitto_dynamic_security.so”)

I haven’t yet tested mosquitto_dynamic_security.so against a running mosquitto broker, I just wanted to know how to make the library file if I had to..

All the best,
Mike

1. steve says:

Tks for the info I will try it myself when I get a chance.
Rgds
Steve

16. Sam says:

Hi, I think for the question 2 under ACL worked example, the answer should be YES. The reason why I say so is because –
a) I have tried it practically, and
b) This has to work. Then only the point of using multiple roles with a single client would work. If we try to publish to the test topic, an ACL check will occur in the role role2 first (since it has the highest priority). Since, no matching ACL will be found, it will search in the role with lesser priority, i.e., role1. And since role1 has allow for all ACLs to the topic test, the client john will be able to publish.
Similarly I think the answer to the third question should also change to – client will not be able to publish.

1. steve says:

Sorry for the delay tks for the feedback I will try and check the results shortly and let you know.

1. steve says:

Yes I agree with you and have changed the answers.
Rgds
Steve