MQTT Client IDs Are Only Guaranteed to 23 Bytes: ACL Planning for SCADA Edge
Duplicate client IDs, wildcard rules nobody owns, retained commands that re-fire, and denied publishes that make no sound. What the MQTT spec actually guarantees at the edge.
Four lines flapped at once
Four packaging lines. The edge PC image was built on one machine and copied to the other three. The MQTT client ID came along with it. The broker log showed the same client ID connecting and disconnecting every few seconds from different IP addresses. On the operator screens, lines went offline and online at random.
This is not a broker bug. It is the behavior [MQTT-3.1.4-2] in MQTT 3.1.1 §3.1.4 requires: if a new CONNECT arrives with a client ID that is already connected, the broker must disconnect the existing client. Under MQTT 5.0 the loser at least gets a DISCONNECT reason code 0x8E (Session taken over), so there is a reason in the log.
Everyone spends time on topic names and payload formats, then hands broker permissions and client IDs to IT as a configuration detail. That is backwards. The broker is not a message pipe. It is the enforcement point between PLC gateways, edge collectors, historians, MES, dashboards, and vendor laptops. Treat it like HMI tag write permissions.
Client IDs: the spec only guarantees 23 bytes
MQTT 3.1.1 §3.1.3.1, requirement [MQTT-3.1.3-5]: a broker is compliant if it allows client IDs of 1 to 23 UTF-8 encoded bytes containing only [0-9a-zA-Z]. Anything longer, or with ., -, or _ in it, is at the broker's discretion (MAY).
mosquitto, EMQX and HiveMQ all accept the longer form, which is why names like plant1.packaging.edgegw.01 (26 bytes) are everywhere in the field. What your current broker accepts and what the spec guarantees are two different things. Swap the broker, or meet the small MQTT stack embedded in a gateway, and this is where it breaks. On an MQTT 5.0 broker the refusal comes back as CONNACK reason code 0x85 (Client Identifier not valid).
Fitting site, area, function and instance into 23 bytes looks like this:
p1pkgEdgeGw01 edge gateway, packaging line 1
p1pkgScadaPri SCADA server, primary
p1pkgScadaStb SCADA server, standby
p1MesConnProd MES connector, production
If you keep the readable dotted names, that is a choice, not something the spec protects. Write one line in the handover saying the ID length depends on the current broker's discretion. Either way, avoid mqtt_client, edge01, and shared vendor defaults — you cannot tell who is who in the broker log.
An empty client ID is another trap. It is only usable with CleanSession=1. Connect with an empty ID and CleanSession=0 and the broker must respond with CONNACK return code 0x02 (Identifier rejected) and close the connection [MQTT-3.1.3-8]. An edge gateway should never run on a broker-assigned random ID: there is then nothing for an ACL rule to name.
Start with client roles, not user names
Before writing ACL rules, list the roles that connect to the broker.
| Role | Publishes | Subscribes | Notes |
|---|---|---|---|
| Edge gateway | Telemetry, birth, Last Will, device status | Commands, configuration | Usually one per line, cell, or skid |
| SCADA server | Commands, acknowledgements, operator actions | Telemetry and status | Must not receive unrelated plant areas |
| Historian collector | Usually none or health status | Telemetry topics | Read-only in most designs |
| MES connector | Work order, recipe, lot context | Equipment state, counts, events | Needs tight namespace boundaries |
| Engineering tool | Test topics only | Test topics and diagnostics | Should not use production credentials |
| Dashboard | None | Aggregated or read-only topics | Avoid wildcard access to raw control topics |
The list makes the ACL reviewable. It also flushes out hidden clients: vendor service laptops, temporary Node-RED flows, report jobs added during commissioning.
On mosquitto, the shortest way to turn that table into rules is pattern, where %u substitutes the username and %c the client ID:
pattern write plant1/%u/telemetry/#
pattern read plant1/%u/command/#
Add a gateway and you do not touch the ACL file again. The cost is that if the username and the topic structure ever drift apart, everything is silently denied — so fix the naming rule first.
Topic permissions are directional
Do not grant both publish and subscribe on every topic under an area just because the client lives there.
plant1/packaging/line1/telemetry/# edge publishes, SCADA/historian subscribes
plant1/packaging/line1/status/# edge publishes, SCADA/MES subscribes
plant1/packaging/line1/command/# SCADA or MES publishes, edge subscribes
plant1/packaging/line1/config/# controlled publisher, edge subscribes
plant1/packaging/line1/test/# engineering only
The namespace shape can vary by site. The direction cannot. A historian collector has no reason to publish to command/#.
If you use Sparkplug B, the namespace is not yours to design. Eclipse Sparkplug 3.0 fixes the form spBv1.0/{group_id}/{message_type}/{edge_node_id}/{device_id} and the message types (NBIRTH, NDEATH, DBIRTH, DDEATH, NDATA, DDATA, NCMD, DCMD, STATE). The ACL has to follow that shape. NCMD and DCMD are the command direction, and only the host application should publish there. Sparkplug also requires NDEATH to be registered as the MQTT Will message, with Will QoS 1 and Will Retain false. Turn retain on and you are out of spec — and a node that is alive keeps looking dead.
One plant1/# rule outlives the project
A dashboard or vendor tool subscribes to plant1/# because it is convenient during startup. Months later the rule is still live, exposing raw telemetry, alarm topics, and command responses for unrelated areas.
Review every rule containing # separately. Each one needs an owner and a reason.
It is also worth knowing exactly how far a wildcard reaches. MQTT 3.1.1 §4.7.2, requirement [MQTT-4.7.2-1], says the server must not match topic filters starting with # or + against topic names beginning with $. So a bare # does not leak the broker's $SYS/ statistics. The flip side: to see broker diagnostics you have to allow $SYS/# explicitly, and that deserves its own permission.
Some MQTT 5.0 brokers disable wildcard subscriptions outright. The SUBACK reason code for that is 0xA2 (Wildcard Subscriptions not supported). When a dashboard quietly shows an empty screen, check that code first.
Last Will does not arrive immediately
This is where the most time gets lost. People expect the Last Will the moment the cable is pulled. It does not work that way.
The broker cannot see a half-open TCP connection. It decides using Keep Alive. In MQTT 3.1.1 §3.1.2.10, Keep Alive is a 16-bit value in seconds, so at most 65535 s. If Keep Alive is non-zero, the broker must disconnect the connection when no control packet arrives within one and a half times that period [MQTT-3.1.2-24]. With Keep Alive at 60 s, the Will publish can be up to 90 s late. If the SCADA side declares a device offline after 30 s, the screen lies for the difference.
MQTT 5.0 adds one more delay on purpose: the Will Delay Interval (Will property 0x18), which holds the Will back so a short reconnect does not raise a false death. Useful, but if it is not lined up with the SCADA offline timeout, the screen just stays green longer.
Test it by dropping the network, not by stopping the client cleanly. Then time it with a stopwatch and write the number down.
Retained commands re-fire
A retained command is delivered again when the edge client reconnects. The command re-executes, or the startup logic gets confused.
MQTT 5.0 gives you tools. A Message Expiry Interval (PUBLISH property 0x02) lets the broker drop messages that got stale. The Retain Handling subscription option (§3.8.3.1) has three settings: 0 always sends retained messages on subscribe, 1 sends them only if the subscription did not already exist, and 2 never sends them. To stop a command coming back to life on gateway reconnect, 2 is the right one. Whether the broker supports retention at all shows up as Retain Available (CONNACK property 0x25).
An opinion: retain status and birth messages. Never retain operator commands. The exception is an application that handles command IDs and expiry itself, and few sites are built that way.
A denied publish makes no sound
This is the strongest reason to run MQTT 5.0 at the edge.
The MQTT 3.1.1 PUBACK has a variable header of nothing but a 2-byte Packet Identifier (§3.4). There is no field for failure. When the broker denies a publish on an ACL, it has no way to tell the publisher, so it silently discards the message or closes the connection. At QoS 0 there is no trace at all. Operators see stale data and no reason for it.
Subscribes are slightly better. The 3.1.1 SUBACK return codes include 0x80 (Failure) (§3.9.3). But it is one lumped code — you cannot tell a permission problem from a bad topic filter.
MQTT 5.0 fixed this part.
| Packet | Reason code | Meaning |
|---|---|---|
| CONNACK | 0x87 | Not authorized |
| CONNACK | 0x85 | Client Identifier not valid |
| PUBACK / PUBREC | 0x87 | Not authorized |
| SUBACK | 0x87 | Not authorized |
| SUBACK | 0x8F | Topic Filter invalid |
| DISCONNECT | 0x8E | Session taken over |
The difference is that the client can log the reason itself. Under 3.1.1 that information exists only in the broker log, and nobody reads the broker log.
So put broker ACL denial logs into the same operational review as driver faults and OPC UA connection errors. Add a diagnostic tag or a health page if the platform supports it.
Split credentials by role
Shared credentials make an ACL meaningless. IEC 62443-3-3 SR 1.2 requires identification and authentication of software processes and devices, not just human users. SR 2.1 requires authorization to be enforced against that identity. If the SCADA server, historian and MES connector share one username, you meet neither: the broker cannot enforce different permissions, and you cannot narrow down the source of a bad publish.
Record where each credential lives — service account, secret store, environment variable, container secret, or gateway configuration. Without that, password rotation turns into a search. Test rotation in a maintenance window and watch reconnect and session reuse behavior; on MQTT 5.0, check whether a Session Expiry Interval (property 0x11) is set.
Checks to run before go-live
ACL testing is a commissioning step, not a broker install detail. Write the expected result next to each check.
- Connect each production client with its real credential and client ID. Confirm CONNACK return code 0.
- Confirm allowed subscriptions receive the expected data.
- Attempt one forbidden subscription outside the client's area. Expect SUBACK
0x80(3.1.1) or0x87(5.0). - Attempt one forbidden publish to a command or telemetry topic. On 5.0, expect PUBACK
0x87. On 3.1.1, expect nothing at the client — confirm it in the broker log. - Confirm the broker log records the denial with enough detail to troubleshoot.
- In a safe window, connect a duplicate client ID and confirm the existing connection drops.
- Drop the network instead of stopping the client. Record how many seconds the Will took. It should be within 1.5× Keep Alive.
- Test retained behavior separately for status topics and command topics.
Record results by role: which credential, which topic, what the broker returned, and what the SCADA screen showed.
What belongs in the handover
- Client ID naming rule, the assigned IDs, and your decision about the 23-byte limit.
- Topic namespace with publish and subscribe direction.
- ACL rules or an exported broker policy file.
- Credential owner and rotation procedure.
- Retained policy per topic group, with Message Expiry Interval values.
- Last Will topic and payload per edge gateway, plus the measured Will delay.
- Broker log examples for denied publish, denied subscribe, duplicate client ID, and authentication failure.
Next thing to check: whether your broker is running 3.1.1 or 5.0, and whether the edge client library actually surfaces 5.0 reason codes in its log. If the library throws them away, a 5.0 broker buys you nothing.