Understanding and Using the Mosquitto Dynamic Security Plugin

The dynamic security plugin can be used instead of the password file and the ACL (Access Control list) .

The plugin is available in mosquitto v2 but it is not enabled by default.

If you decide to use it then the manual recommends you use it for all listeners and don’t have some using the old password file and other listeners using the security plugin. Use:

per_listener_settings false

To configure the broker to use it then you need to set the path to the plugin, and also the path to the security file.

The security file is a JSON file.

Linux:

plugin path/to/mosquitto_dynamic_security.so
plugin_opt_config_file path/to/dynamic-security.json
Example:

plugin /usr/lib/x86_64-linux-gnu/mosquitto_dynamic_security.so
plugin_opt_config_file /home/steve/mos/v5/test.json

Windows:

plugin path\to\mosquitto_dynamic_security.dll
plugin_opt_config_file path\to\dynamic-security.json

To locate the plugin on  Linux use

whereis mosquitto_dynamic_security.so

Using the Plugin

You first need to edit the config file as shown above, and also to create a dynamic-security.json file.

The file can have any name and a starter basic file is created using the mosquitto_ctrl utility that comes with mosquitto v2.
Use:

mosquitto_ctrl dynsec init path/to/dynamic-security.json admin-user 

example:
mosquitto_ctrl dynsec init path/to/dynamic-security.json steve

Where steve is the user that I will use for configuring the security settings.

default-security-dynamic-plugin

You must do this on the broker itself or you will need to copy the file to the broker.

The proceeding command is the only mosquitto_ctrl command that doesn’t require that you specify a broker and port.

The mosquitto_ctrl utility is basically a modified mosquitto_pub client and publishes messages to the $CONTROL topics on the broker.

When you generate the default file the user steve (my example) will be given the required permissions in the file.

You could manually edit the security file to modify the security settings but this is too complex.

Instead the mosquitto_ctrl utility is then used to dynamically modify the security settings on a running broker and can be run from any machine that has the utility installed.

It is probably best that you use SSL when running this utility on a production broker but when testing I wouldn’t use SSL.

Key Concepts

The plugin lets you configure security using three objects- client, group and role objects.

The client is actually the user name you use to access the broker assuming that you are not allowing anonymous access.

Client Object

This has the following properties:

  • user name
  • password
  • client Id (optional)
  • groups
  • roles
  • text name – a friendly name
  • text description
  • enabled – a client can be disabled

A client can belong to a group/groups and assigned to a role/roles it can also be enabled and disabled.

If the client_id is not specified then the username/password can be used by multiple connections which is the case for mosquitto when using standard password files.

note: Need -c option for client_id

Groups Object

This has the following properties:

  • Group name -Used when modifying group
  • Roles
  • Friendly name (optional)
  • Description (optional)

Roles Object

The Roles object contains the following properties

  • Role name
  • Access Control List (ACL)
  • Friendly name (optional)
  • Description (optional)

The roles object is used to give clients access to topics.

Clients,Groups and Roles

Clients can be assigned directly to roles and given or denied access to topics, but doing so is not as flexible as using groups.

For example if 10 clients are assigned to role1 and you decide that you need to assign different access to 5 clients and so need to assign them to role2 then you would need to make 5 changes.

However if you had assign 5 clients to group1 and 5 to group2 and assigned both groups to role 1 then you would only need to make 1 change to the group.

That is provided the clients had been assigned to the correct group to start with.

So try and use client–>Group –>Role and not client –> Role.

How it works

To understand how security is applied we will start using the default security file generated using mosquitto_ctrl.

A snapshot of the file is shown below:

Default_Dynamic_Security_File

The plugin uses events that trigger an ACL check. So for example when a client subscribes an ACL lookup would be triggered.

You can see from the file that I have marked it into three blocks.

Default Access

The default access permissions in the default configuration file are defined in the defaultACLAcess entry (bottom block) and are very restrictive, and only grant access to the admin user.

All other users and anonymous users (if enabled) are not allowed to publish or subscribe to any topics, not even the $SYS topic

You can see from the defaultACLAcess entry that we have four broker events that can be used to configure ACL permissions.

We have the unsubscribe and subscribe events which are triggered when a client subscribes or unsubscribes.

We also have the publishClientSend event which is triggered when the client published a message to the broker, and the publishClientReceive event when the broker needs to send a message to the client.

As stated in the manual there is an overlap between the subscribe and publishClientReceive events and usually using the subscribe event is sufficient.

However the publishClientReceive event gives you more control as you can subscribe to a wild card topic but still restrict access to certain topics in the hierarchy.

E.g

topic test/# subscribe allow
topic test/secret publishClientReceive deny

would allow access to all topics in the test hierarchy except test/secret

If a client supplies a username and password then the logon credentials will be checked against the client entry if any.

If the client doesn’t have an assigned role or group then the defaultACLAcess permissions will apply.

If allow anonymous is disabled then the connection must supply a valid username and password, defined in the configuration file as a client entry.

It is possible to simulate the old passwords file just by simply creating client entries with no roles or groups and setting all entries in defaultACLAcess to true.

"defaultACLAccess": {
"publishClientSend": true,
"publishClientReceive": true,
"subscribe": true,
"unsubscribe": true
}

The entry for the admin user is only necessary if you want to manage the settings using mosquitto_ctrl.

Priorities

A priority can be applied to groups and roles. The default priority is -1.

If a client has multiple roles assigned or is a member of multiple groups then you need to understand the order in which ACL permissions are applied.

The first check is made on the role assigned to the client.

If a client(user) has two roles for example role1 and role2 and role role1 and a priority of -1 (default ) and role2 has a priority of 5 then the permission of role2 will take affect.

If a client is assigned to a role or group with ACL permissions set then these ACL permissions will override those in the defaultACLAcess entry.

The documentation gives a more detailed but complex overview of priorities.

I would suggest you keep them as simple as possible.

Anonymous Group

If you have allowed anonymous access then all anonymous users will be placed in the anonymous group.

The name of this group needs to be created and set using:

mosquitto_ctrl <options> dynsec createGroup <groupname>
mosquitto_ctrl <options> dynsec setAnonymousGroup <groupname> 

example
 mosquitto_ctrl -h 192.168.1.91 -u steve -P password dynsec createGroup anonymous
anonymous mosquitto_ctrl -h 192.168.1.91 -u steve -P password dynsec setAnonymousGroup anonymous

You can now assign the group to a role and ACL permissions to that role.

ACL Worked Example

Configuration – The default defaultACLAcess is set as follows:

"defaultACLAccess": {
"publishClientSend": false,
"publishClientReceive": true,
"subscribe": false,
"unsubscribe": true
}

Client john is assigned to role1 and role 2. Role 1 has all allow permissions to topic test with priority -1 and role 2 is configured with deny subscribe to topic test with priority of 5.

Q1 Will john be able to subscribe to topic test?

Q2 Will john be able to publish to topic test?

Q3 What will be the effect of changing the permissions for role1 to deny publish.

Q4 If You change permissions to role1 will you need to restart mosquitto.

Answers at the end of the tutorial

Making Changes

You can edit the security file manually and this is likely to be the case with the initial security file.

However after initial deployment changes are made dynamically using the new mosquitto_ctrl tool which we look at in the next tutorial.

Common Questions and Answers

Q1 _can I use the %u and % c syntax like in the old ACL file?

A- It doesn’t appear so

Q2 – Should I move to the new plugin?

A2- Unless your ACL permissions change frequently then I think you would be better with the old files.

Answers

Q1 -role2 has priority and there is a deny permission to topic test.. So  John cannot subscribe to test

Q2 role2 has no entry for publish for which means it will look in role1 which has an entry and so John can publish.

Q3-John cannot publish

Q4 – No it is dynamic.

Important Note: When using clientid then you need to use the -l option and not -c.

Related Resources and Tutorials

 

Please rate? And use Comments to let me know more

54 comments

  1. Dear Steve,

    1) Is it possible to use Redis as a backend to Dynamic Security plugin instead of a text configuration file?
    2) Do you have a tutorial for using the “mosquitto-go-auth” especially with Redis?
    Thank you.
    Regards,
    Ismet

  2. actully i am using the dynamic security plugin for my mosquitto server is there any way to add user with password by sending message to mosquitto server . Actully i am using the java code to try and add user via command terminal but getting some issues so can you help me ?

  3. Excellent website! Helped me a lot!

    Have we decided amongst the various scenarios, the best way to have a dynamic client/user that only has access to their own topic (like %c of old ACL)? I don’t want to allow anonymous, I want to specify a separate account for a client/user, but lock down to publish to their own topic and nothing else.

    Sounds like I want dynsec, but with a global “pattern write /topic/%c” type thing.

    1. Neil
      Glad you find the site useful.
      I haven’t found a solution as there isn’t %c or%u option in the dynamic acl file that I can see.
      The only recourse is to use the old method.
      rgds
      Steve

  4. Hello, is it possible for a user to be assigned multiple specific client IDs, and how can this be done

    1. My understanding is that a user can connect using multiple clients with different client_ids. However when you specify a client_id for the user you tie it to that client_id as as far as I am aware a users can only be assigned a single client_id.
      However I haven’t checked this but I will .
      If you can elaborate on what you are trying to achieve I may be able to help a bit more.
      Rgds
      Steve

      1. I want to create a user with access to only three simultaneous connections. The first option is to assign them three client IDs. The second option is to limit the number of connections for a single user using a parameter like ‘max_connections 3’. Unfortunately, I couldn’t find any alternatives or equivalents for such options. The documentation also doesn’t provide any information on this. Is there an alternative way to enforce a limited number of connections for a single user?

        1. Sorry for the late reply but I’m very busy at the moment and haven’t had chance to give it more thought. My feeling is that what you are trying to do isn’t possible.
          The max_connections setting is per listener and not user.
          You can limit the connections with a given username/password to 1 but not to any other number as far as I can see.
          Rgds
          Steve

  5. Hi Steve!

    Is there a way to create a single group with the same role that has the same restrictions on the topics but that is with the client id or name?

    Example

    I have the users test1, test2, test3
    I want this role to have, for example, permission to subscribe to a topic that has its own name or id

    * subscribeLiteral /test/(%c) or /test/(clientName)

    So test1 can subscribe to /test/test1
    test2 can subscribe to /test/test2
    test3 can subscribe to /test/test3

    I want to see if this can be possible somehow as the other option I have left would be to create a role and group with the constraints for each and if I have 100,000 clients it would be hell to create the client, group, role and all the constraints for topics for each

    If what I have said before is possible, I would only have to create the client and assign the group to it

    Do you know if there is or exists a way to do this?
    If necessary I can provide other examples if it is not understood

    Thanks in advance.

      1. Yes I have seen and tested that in the old acl file you can do that, but if I am correct with everything I have seen, if I use the old way of the acl file, the client when publishing or subscribing to any topic mosquitto connects and it pretends to publish or subscribe, but in reality no topics or messages arrive since in theory you have your restrictions in your acl file, but in the acl of mosquitto ctrl directly denies you and removes you from the sub or pub.

        What I want to avoid is that if there is something misconfigured, Mosquitto will deny you, take you back and tell you no, but with the old way of acl it can be confusing to detect constraint failures since it always accepts you

        I’m right?

        I await your answer, thank you

        1. definitely true for the old ACL files. I didn’t really test for the new style but I will try it and see.
          Rgds
          Steve

          1. As far as I can tell the broker doesn’t tell the client that it has denied subscribe or publish with the security plugin unless you use MQTTv5 and then only the publish message with qos of 1 and above.
            Rgds
            Steve

  6. Hi Steve!
    Thank you for all the information about Mosquitto. I usually read your blog and I learn a lot everytime. I remember that one of your guides was essential for my introduction on the MQTT world. I have a question about the ACLs that maybe you can answer.
    Suppose I have the next structure: /srv/servers/[server]/sites/[site]. In this case I could have the next complete topic structures:
    – /srv/servers/server1/sites/site1
    – /srv/servers/server1/sites/site2
    – /srv/servers/server2/sites/site1
    And then I have five users: one of them can read and write everything in the broker, but the another ones have specific permissions.
    user test_user_server1
    topic /srv/servers/server1/#
    user test_user_server2
    topic /srv/servers/server2/#
    user test_user_site1
    topic /srv/servers/server1/site1/#
    user test_user_site2
    topic /srv/servers/server1/site2/#
    user test_user_site3
    topic /srv/servers/server2/site1/#
    Let’s suppose I am test_user_site2, can I read the content of the messages published on /srv/servers/server1 and the previous topics (/srv/servers and /srv)? Or I will see just the topics path but no the messages in them? This question would be the same for the rest of users I mentioned, so with an answer about one would be enough.
    I hope I explained myself well! And, of course, thank you very much in advance!
    Regards,
    Iván

    1. The easiest way to configure it would be to give each user explicit permissions on the respective server.
      so
      user 1 would have readwrite to server1 and a general setting or read

      topic read #
      user user1
      topic readwrite #

      you would do likewise on the other brokers but using the appropriate user

      Note: I haven’t tested it.
      Rgds
      Steve

  7. Very nice article, as usual reference for mosquitto user guide.

    Anyway I though Dynamic Security was my solution but after reading twice I’m not so sure.

    I need to create users and perms on fly like dynamic but I don’t want to specify topic on each, may be I need a mix between to method. Let my explain, my goal is very simple one topic for all users and then sub topic for each user (so users are like in jail)
    MyTopic/user1/#
    MyTopic/user2/#
    MyTopic/user3/#
    Today I’m using old password and ACL files but I don’t like the need to change files each time and need to send HUP to reload config.
    Any chance with dynamic to do that with just few lines, I mean user / group / role and just assign permission on MyTopic/%username/# or even MyTopic/%groupname/# as in ACL files?
    Not sure what’s the best (and simple) way to do that

    1. Hi Charly!

      I am looking for the exact same use case; did you ever find this out or get this working? Otherwise I am thinking I will resort to a script to modify the files whenever I need to add a new user.

      Any advice is appreciated 🙂

      1. The problem with dynamic security is that you can’t have user as a variable so the old acl is the only way as I see it but I haven’t looked at it for a while. I will revisit it and try to remember to pass it on to the mosquitto team.
        rgds
        Steve

  8. Hi Steve, thanks for your explanations. I’m giving a client ID using the -i argument to all new clients, but the client ID never gets persisted at the json file. Everything is persisted correctly, user names, passwords, roles, groups, ACLs, etc. But the client ID defined when creating the client is not persisting. Even if a consult the information for any client (using the getClient option) I see empty the Clientid field. Have you ever had this issue?

    1. Hi
      Yes this cropped up before and someone found it and posted it in the comments. Here is the comment
      Instead of using -i, we should use -c to create client with client id associated to it. The following is the complete command for creating client with client id associated to it.
      Let me know if it works and I will update the tutorial.
      Rgds
      Steve

      1. Hi Steve, I just made the test and it works perfect. It seems mosquitto should update the documentation to indicate “-c” instead “-i” for providing the client id of new user. Worth to mention that “-i” still is correct to provide the client id for the admin connection that is performing the dynsec operation. Best reagards,

  9. Hi, I’ve been trying to use mosquitto security plugin, but I’ve been facing a couple of issues. I managed to create the file as it was showed on this tutorial , so that I tried to include roles and clients in the file to create the file I typed this line:
    mosquitto_ctrl dynsec init path/to/dynamic-security.json user
    password: user
    Then, I’ve tried to include roles or clients in the file for that I typed the following line:
    mosquitto_ctrl -u user -P user dynsec createRole rolepizza

    After enter this line I got the Connection error:Not authorized. I was wondering if this has something to do with the auth file that my broker is using, because to connect to the broker the client needs to inform the user and password, I also tried that so I typed the following line:

    mosquitto_ctrl -u brokerUser -P brokerPassword dynsec createRole rolepizza

    But still getting the same error. Anyway, the questions are:
    can I use both procedures, the authentication file and dynamic security plugin at the same time?
    If this isn’t the problem what would be ?

    Thanks in advance.

    1. Hi
      You need to create the admin user first and use that use to make the changes. Do you have access to the broker and the file.
      Rgds
      steve

  10. Hi Steve,
    is there a way to implement authorization&authentication, i.e. user-password, in which this info is not sent in clear text? apart from TLS and PSK. Thank you so much for your helpful guides. Regards

  11. Is there any way to view the details in $CONTROL/dynamic-security/v1.
    I could push values into it and observe changes at the broker, but I can’t receive any values for get methods.
    Am using MQTT explorer as my client and running broker in a ubuntu VM.

  12. Hi Steve,
    I want to integrate an Auth API endpoint with my MQTT Broker, how would I do that? Should I use the dynamic plugin or just the password and ACL files

    1. Hi
      Sorry but can’t give you an answer at the moment I will have to give it some thought and get back.
      Rgds
      Steve

  13. Hi Steve,

    I would like to know what are the advantages and disadvantages of using the dynamic security plugin instead of the ACL and the password file.

    What would you recommend? Using the dynamic security plugin or the ACL and password file?

    Thanks!!

    1. The dynamic security plugin is very useful if you have multiple brokers and/or need to make frequent security changes.
      On small installations wit one or two brokers and infrequent changes then stay with the password file and ACL list as it is easier.
      Rgds
      Steve

      1. So for example, in the case of an industrial production enterprise that has different headquarters in various countries but that is all connected to the same broker, which one would you recommend?
        I ask it because my final degree project covers this area and I would like to know which path is more recommended to follow.

        Thanks for your previous reply and for this amazing blog!!

        Regards,

        María

        1. With a single broker then either would do. However if it needs remote management the the security plugin is the better option.
          Rgds
          Steve

  14. I was wondering, is there a way for the client that connects to know which group/roles are assigned to them ?

    1. No there isn’t as far as I know as you need admin privileges to see this information. But I will look into it further.
      Rgds
      Steve

  15. hey !

    I am a little bit confused here. Are these commands actually applied also dynamically on Mosquitto service? Or it needs to be restarted each time I set up/add new rules?!

    The config file is also written in case the service/system is restarted/rebooted?!

    I asked this because I’d like to develop an interface where I should add/generate/manage clients &topics & ACL … and all of these stuff, without actually stopping the service. It will be possible/applicable to do that?

    thanks !

    ps: nice article! 🙂

  16. Hi Steve,

    To take this further, if I want to modify the json file dynamically based off of MQTT messages from clients, what approach could I take? Could I use a python client on the broker that uses the ‘os’ module to interact with the mosquitto_ctrl tool?

    1. Mike
      Yes that is what I would do. It is on my to do list along with lots of other things but I might do it in node-red as it is easier to build a web interface for it.
      rgds
      steve

  17. Minor nitpicking: there are a few “Recieve” in the text. These should be “Receive”. (I did check the code, just in case 🙂

  18. Hi everyone,
    just a quick side note. Don’t forget to restart the broker after setting everything up.
    Otherwise the plugin will not give any response (no error either).

    And one quick question. Shouldn’t I be able to a new client in the dynamic-security.json after “createClient”?
    In which file would it be stored then?

    Best and thanks for this Blog!

  19. On windows I can find mosquitto_dynamic_security.dll, but on raspberry pi and a virtual private server running debian I can’t seem to find mosquitto_dynamic_security.so ?

    I tried
    >whereis mosquitto_dynamic_security.so
    but it just returns
    mosquitto_dynamic_security:

    I tried looking and searching with slightly modified names but no luck, any ideas please?

    Thanks,
    Mike

    1. I didn’t find it either it is on my todo list to find out why. Presently I am using the windows one to do the testing as it runs remote anyway.
      Rgds
      Steve

      1. Thanks for letting me know it’s same on your system, I downloaded mosquitto source and compiled the plugin (untested so far).
        These are the steps I took..

        1. Needed cJSON for linking, so from my home directory did:

        mkdir cjson
        cd cjson
        git clone https://github.com/DaveGamble/cJSON.git ./
        sudo make install

        2. get source for mosquitto (but build won’t currently work for me because of websockets library error, but get source anyway for the plugin source code), so from my home directory did:

        mkdir mosquitto2
        cd mosquitto2
        git clone https://github.com/eclipse/mosquitto.git ./

        3. build and install the dynamic_security plugin

        cd /home/pi/mosquitto2/plugins/dynamic-security
        sudo make install
        (The output from that gives: “/usr/local/lib/mosquitto_dynamic_security.so”)

        I haven’t yet tested mosquitto_dynamic_security.so against a running mosquitto broker, I just wanted to know how to make the library file if I had to..

        All the best,
        Mike

  20. Hi, I think for the question 2 under ACL worked example, the answer should be YES. The reason why I say so is because –
    a) I have tried it practically, and
    b) This has to work. Then only the point of using multiple roles with a single client would work. If we try to publish to the test topic, an ACL check will occur in the role role2 first (since it has the highest priority). Since, no matching ACL will be found, it will search in the role with lesser priority, i.e., role1. And since role1 has allow for all ACLs to the topic test, the client john will be able to publish.
    Similarly I think the answer to the third question should also change to – client will not be able to publish.

Leave a Reply

Your email address will not be published. Required fields are marked *