S-MQTTT, or: secure-MQTT-over-Traefik

The last days I have been experimenting in different ways how I can secure a MQTT setup for my home automation. There’s an increasing use of IoT here at my home and most of the applications communicate over MQTT. You simply cannot control every device and how it gathers information. To prevent eavesdropping, it’s time to secure MQTT.

This post is written with the Troy Hunt IoT series in mind. Of course, you should patch your devices and put them in a separate VLAN. However, if you have an MQTT security system and an MQTT light bulb, did you consider the light bulb had access to the security system via MQTT? Or did you consider IoT devices that are inside the same VLAN, but don’t use MQTT themselves, could sniff all (security) messages communicated over your message broker? It all boils down to the principles of zero trust.

For my home automation I am an avid Home Assistant user. Since a long time I have a Home Assistant setup which controls a variety of lights, switches and appliances. When I started introducing MQTT to my setup, I used it without TLS and without authentication. Over time more applications communicate over MQTT and I was worrying about two things:

  1. Untrusted devices could find and connect to the MQTT server without any effort;
  2. Every message in every topic could be listened for anonymously.

That’s why I set three goals to tighten things up:

  1. Every MQTT client must authenticate via unique usernames/passwords. Every client gets separate credentials so there’s no reuse of passwords anywhere.
  2. Enable TLS encryption for communication. The MQTT protocol (including authentication) is plain text, meaning username and password could be sniffed if no encryption is used.
  3. Use Access Control to prevent devices reading/writing topics they should have no interest in. If a trusted (authenticated) client sniffs into topics for other applications, they must be blocked.

Mosquitto

The message broker I personally use is Mosquitto, as it’s lightweight and extremely easy to use. Out of the box, it does allow anonymous connections and no users are registered, so you need to take care of both.

In your mosquitto.conf file, make sure you have those two lines present and make sure the mosquitto.passwd file exists (just update the path of the password file based on your installation):

allow_anonymous false
password_file <path/to/mosquitto>/mosquitto.passwd

Then supply Mosquitto with the credentials you want to add:

mosquitto_passwd <path/to/mosquitto>/mosquitto.passwd <username>

Again, replace the path & your preferred username and complete the prompt with the password.

My installation resides inside docker, so in my case, the configuration files are located at /mosquitto/config/ and I add all my clients in bulk (via Ansible) using the following command (mqtt is the name of my container)

docker exec mqtt mosquitto_passwd -b /mosquitto/config/mosquitto.passwd <user> <password>
docker exec mqtt kill -SIGHUP 1

The SIGHUP signal is used in Mosquitto to reload the configuration without restarting Mosquitto (which otherwise will probably loose some messages along the way).

Well done: You completed the first part of your goal securing MQTT!

Traefik

My favourite reverse proxy for production apps and home installation is Traefik. It just integrates flawless with the tools I prefer: a dockerized setup and automated certification renewal via Let’s Encrypt. And it’s so lightweight you have little overhead for hosts like a Raspberry Pi.

All of my frontend web applications are routed via Traefik’s HTTP(S) proxy. The fun thing is it also supports TCP and UDP traffic (which I also utilize in my failover TCP+UDP setup for OpenVPN).

MQTT is plain TCP traffic and Traefik is able to create a TLS tunnel for TCP traffic, so this is a fairly straightforward thing to configure. To understand the label configuration below, make sure you read the Traefik documentation.

labels:
  - traefik.enable=true
  
  - traefik.tcp.routers.mqtt.rule=HostSNI(`mqtt.example.com`)
  - traefik.tcp.routers.mqtt.entrypoints=mqtt
  - traefik.tcp.routers.mqtt.tls=true
  - traefik.tcp.routers.mqtt.service=mqtt
  
  - traefik.tcp.services.mqtt.loadBalancer.server.port=1883

Traefik usually connects to a container’s port if there’s only one port exposed. To be explicit I define a mqtt service in this case, loadbalancing the only port in the mosquitto container. I make this explicit because the default (unencrypted) MQTT port is 1883 and the default TLS encrypted port is 8883. If you ever read back the configuration you should be able to trace things back.

Next, the docker container uses an entrypoint called mqtt defined in the static configuration. Most Traefik setups use at least a web and websecure entrypoint, I added mqtt at port 8333. This creates a setup where the docker container itself exposes an (unencrypted) port 1883 towards Traefik, this container is inaccessible from the outside. Traefik creates an accessible entrypoint, which will be encrypted, at port 8883. This technique is called SSL Termination.

[entryPoints]
  [entryPoints.web]
    address = ":80"
  [entryPoints.websecure]
    address = ":443"
  [entryPoints.mqtt]
    address = ":8883"

Finally, the rule label in the docker container gives a URL to use (like mqtt.example.com) and with tls=true you tell Traefik to handle it as a TLS connection.

This is great: You are more than halfway through securing MQTT!

Access control

Access control in an MQTT server is the final step in securing your messaging system for IoT. Access control defines access on a per-user basis, so above steps for authentication and encryption are required to go further down the security lane. Initiating access control is a principle of a whitelist, anything not specified means there is no access. You only need to state which clients have access to which topics, anything else is excluded.

Like the password file, the ACL file is referenced in the mosquitto.conf:

acl_file <path/to/mosquitto>/acl

Next, you need to fill your ACL file. Jaimyn Mayer has an excellent tutorial for composing an ACL file with the usage of Home Assistant in mind so I won’t elaborate too much on this.

The basic format of the file consists of sections per user, where every topic is listed to grant read and/or write access. Because of the nested structure of MQTT topics, you can use wildcards to group topics at a higher level.

From Jaimyn’s example, using Home Assistant, Sonoff (WiFi powered) lights and light sensors:

# Give Home Assitant full access to everything
user homeassist
topic readwrite #

# Allow the sonoffs to read/write to cmnd/# and stat/#
user sonoffswitch
topic readwrite cmnd/#
topic readwrite stat/#

# Allows the light sensor to read/write to the sensor topics
user lightsense
topic cmnd/sensor/#
topic stat/sensor/#

Tip: if you don’t know which topics are used by your devices, send Mosquitto the SIGUSR2 signal and it outputs a hierarchy of topics:

kill -SIGUSER2 <pid-of-mosquitto>

In my docker setup this translates to (the output is send to stdout, so you need to check the container logs):

docker exec mqtt kill -SIGUSR2 1
docker logs mqtt

Again, when finished composing your ACL file, make sure to reload Mosquitto:

// For normal installation
kill -SIGHUP <pid-of-mosquitto>

// For docker installation
docker exec mqtt kill -SIGHUP 1

You are awesome! You completed your goal in securing your MQTT message broker!

Final thoughts

This ended up as a much longer post than anticipated. I was reluctant to get going with this setup but it ended up pretty nice. The overhead (and delay in message delivery) with TLS encryption in comparison with unencrypted MQTT is unnoticable for me. In addition, it gives me a much safer feeling compartimentising all the variety of IoT devices. I simply don’t trust all the ‘things’, especially the cheap stuff from far abroad. Now I know that stuff just can’t sniff around the communication of other devices.

Client TLS capabilities

Just to make one thing clear if you go down this road, it may seem obvious but encrypting MQTT traffic means every client must connect over TLS only. Switching over to Traefik means you go over the configuration of every MQTT client (lights, switches, cameras and so on) to enable a security flag in their respective settings. Otherwise you end up only pretending being a security endboss.