Why Your MQTT Dashboard Jumps Backward: Timestamps, Sequence Numbers and Replay

The value jumped four minutes backward

The oven temperature trend hit 184.2 °C at 09:18, and the next point dropped to 181.6 °C stamped 09:14. Nothing had crashed. There had been a three-minute network drop, and the gateway flushed its buffered samples right after reconnecting. The live stream had already resumed.

The HMI was not at fault. It drew the points in the order they arrived. The payload simply never said when each value was measured.

MQTT is good at moving messages. It does not tell you which sample happened first, whether a message is late, or whether a replayed value should overwrite a newer one. It was never supposed to. That is the payload designer's job.

The DUP flag is not a duplicate filter

This is the most common misconception: the PUBLISH fixed header carries a DUP flag, so surely the broker screens duplicates out.

It does not. MQTT 5.0 §3.3.1.1 says the opposite. DUP marks a transport retransmission, and normative statement [MQTT-3.3.1-3] is explicit: the DUP value of an incoming PUBLISH is not propagated to subscribers, and the outgoing DUP "MUST be determined solely by whether the outgoing PUBLISH packet is a retransmission."

In practice:

• A QoS 1 retransmission of the same packet arrives with DUP=1. You can catch that one.
• A gateway republishing its store-and-forward buffer after reconnect is issuing a new PUBLISH. It arrives with DUP=0.
• Two redundant gateways publishing the same equipment state both send DUP=0.

The duplicates that actually hurt in the field are the last two, and neither is visible in the DUP flag. You have to carry your own identity at the application level.

The Packet Identifier is no better. §2.2.1 defines it as a 16-bit Two Byte Integer, reused once the QoS 1/2 handshake completes, and rewritten across broker hops. It is not a deduplication key.

Do not collapse timestamps into one field

A single field named timestamp cannot tell you whether it means PLC sample time, gateway receive time, broker receive time, or application insert time. Those are different facts, and during a post-mortem you almost always need the first one.

{
  "source": "line3/oven1",
  "tag": "PV_Temperature",
  "value": 184.2,
  "quality": "good",
  "sampleTime": "2026-06-28T09:14:22.315Z",
  "publishTime": "2026-06-28T09:14:22.612Z",
  "bootId": "a7f3-2026-06-28T06:02:11Z",
  "seq": 18445521
}

sampleTime is process time — the moment the controller or gateway acquired the value. publishTime is communication time. The 297 ms gap above is ordinary buffering, but when that gap widens to minutes you are looking at a store-and-forward replay. Expose publishTime - sampleTime as a diagnostic tag in its own right; it is the first thing worth looking at during an outage.

If the controller cannot provide a trustworthy timestamp, say so in the interface document. Do not dress gateway time up as PLC time. That single shortcut causes more grief than anything else once someone has to reconstruct an event sequence.

Keep clocks boring

Timestamp design collapses the moment clocks drift. A one-minute offset is survivable on a temperature overview. It is not survivable for batch release or event sequence analysis.

• Put UTC in payloads. Convert to local time in the HMI or reporting layer.
• Monitor NTP status on gateways and servers. On a plant LAN, NTP typically holds within a few to a few tens of milliseconds. If you need finer ordering than that, stop reasoning from wall-clock time and use sequence instead.
• Change quality or raise a diagnostic when clock sync exceeds the allowed tolerance. Do not let it pass silently.
• Record the timestamp source in the commissioning notes.

If two gateways publish related events, their clocks must be close enough for the analysis you intend to do. If they are not, ordering by wall-clock time is meaningless in the first place.

Sequence numbers need a scope and a documented wrap

A sequence number is only useful when everyone knows what it counts.

Scope	Example	Good for	Risk
Per tag	Temperature tag increments on each publish	Detecting missed samples for one point	Cannot order events across tags
Per device	Gateway keeps one counter for all messages	Ordering everything from one device	Wraps faster on high-rate traffic
Per event stream	Alarm/event topic has its own counter	MES event duplicate detection	Reset behaviour must be defined
Per boot session	Resets on restart, carries a boot ID	Simple embedded devices	Consumers must read the boot ID

Wrap is arithmetic, not an abstract worry. On a gateway publishing 10 messages per second, a 16-bit counter wraps every 65536 / 10 ≈ 6554 seconds — one hour and 49 minutes. An 8-bit counter wraps every 25.6 seconds. If your historian connector treats "the number went down" as a gap, every wrap manufactures a false alarm.

A counter that silently resets to zero after reboot is the same failure wearing a different hat. Without a boot identifier or session ID, consumers either discard new messages as old duplicates or miss a real gap.

This design is already standardised — Sparkplug B

Before defining all of that yourself, read Eclipse Sparkplug 3.0. It specifies exactly what you were about to hand-roll.

• seq increments by one per message and rolls over from 255 to 0. It is 8-bit, so the arithmetic above applies directly.
• The seq in an NBIRTH must be 0. That is where consumers anchor the start of a stream.
• bdSeq is a separate counter incremented on each connect, and NBIRTH and NDEATH carry the same value, so a host can match exactly which session ended. That is the bootId you were about to invent.
• The payload timestamp is milliseconds since the UTC epoch. No local-time argument to have.

Whether to adopt Sparkplug outright is a separate call. The 8-bit seq pushes a 25-second wrap onto every consumer, and the protobuf payload makes casual debugging tedious. But even if you keep your own JSON schema, copy the bdSeq + seq pairing verbatim. It is a design that has already been beaten on in the field for years. Topic structure is covered separately in MQTT Sparkplug topic design.

Decide the deduplication key up front

QoS reduces loss; it does not remove the need for duplicate handling. QoS 1 is at-least-once by definition. Consumers need a stable identity per message.

• Sampled values: source + tag + sampleTime + seq
• Alarms or production events: source + eventId
• Devices whose sequence resets on reboot: source + bootId + seq
• MES transactions: lotId + equipmentId + eventType + eventTime

Never deduplicate on arrival time alone. Arrival time shifts with broker, network, and consumer load. QoS and retained combinations are covered in more depth in MQTT QoS and retained messages.

Late data is handled differently per consumer

Late data is not automatically bad. A historian must accept an older sampleTime during backfill. A live HMI should ignore it when a newer value is already on screen. MES accepts it only while the event window is open.

Consumer	Suggested behaviour
Live HMI	Show the newest valid sample by sample time; mark old data as stale or replay
Historian	Store by sample time; reject or quarantine impossible time jumps
Alarm/event service	Preserve order by event time and sequence; flag late arrival
MES transaction handler	Use idempotent event IDs; never create duplicate lot moves
Analytics job	Allow backfill, but keep ingestion time separately

Several consumers read the same topic. The payload must carry enough context for each to decide on its own, without anyone having to come and ask you what the policy is.

Retained messages: use the tools MQTT 5.0 gave you

Retained messages suit current-state topics, but they can hand a new subscriber the last value from before a shutdown as if it were current. Under 3.1.1 the only option was to embed the age in the payload and hope consumers checked it. 5.0 moves part of the job into the broker.

• Message Expiry Interval (§3.3.2.3.3, property identifier 0x02, a four-byte integer in seconds). A retained message older than this is deleted by the broker, and a queued message being forwarded goes out with the remaining time rather than the original. Set five minutes on a status topic and the "hours-old value pretending to be current" problem ends at the broker.
• Retain Handling (§3.8.3.1, bits 5–4 of the SUBSCRIBE subscription options). 0 sends retained messages on every subscribe (the default). 1 sends them only when the subscription is newly created. 2 never sends them. For a consumer that reconnects often, 1 is usually the right answer.
• Zero-byte retained payload (§3.3.1.3). Publishing with RETAIN=1 and a zero-length payload removes the retained message for that topic. Clear status topics this way on a planned shutdown and a dead gateway stops claiming to be alive.
• Will Delay Interval (§3.1.3.2.2, property identifier 0x18). Delays the LWT publication by a set number of seconds, so a five-second reconnect does not fire a death message and make MES mark the equipment offline. Birth and LWT design is covered in MQTT birth and last will.
• Session Expiry Interval (§3.1.2.11.2, property identifier 0x11). Sets how long a gateway can stay disconnected before its session is discarded. Size it together with the store-and-forward buffer.

Do not use retained messages for one-shot commands, production transactions, or alarm acknowledgements. A newly connected subscriber executing an old command the broker remembered is a real incident, not a theoretical one.

Four failures you will see

Gateway replay overwrites newer values. The symptom this article opened with. Force a gateway outage, reconnect, and confirm consumers order by sampleTime rather than by arrival. Buffer behaviour is covered in edge store-and-forward.

A copied configuration publishes the same source twice. Someone cloned a gateway config and left the source name and client ID untouched. The broker disconnects one, or both alternate publishing to the same topic. Do not trust the topic name to prove which device is talking — log source identity, client ID, and certificate identity together.

A counter reset looks like mass data loss. After reboot seq returns to zero and the historian connector flags every subsequent message as out of order. Whether it is Sparkplug's bdSeq or your own bootId, always order on the (bootId, seq) pair.

Retained online=true hides a dead device. The broker hands the last retained payload to a new dashboard while the gateway is offline and the screen stays green. Use Message Expiry Interval, LWT, and the zero-byte clear together — and separate communication quality from process quality on the display.

Test before handover

Do not settle for the clean lab path. Reproduce the failures.

1. Cut the gateway network and confirm store-and-forward preserves sampleTime.
2. Reconnect and confirm the duplicate and out-of-order rules behave as documented. Take a packet capture and see for yourself that the replayed messages arrive with DUP=0.
3. Restart the gateway and confirm sequence reset handling is correct.
4. Force the counter to wrap. That is 25 seconds at 8 bits, two hours at 16.
5. Subscribe with a fresh client and confirm the retained age check and Retain Handling setting do what you intended.
6. Force clock sync loss and confirm quality or diagnostics change.
7. Compare historian records, HMI display, and broker capture side by side for the same test window.

A good MQTT interface is not a set of topic names and JSON fields. It is an agreement about time, order, identity, and replay behaviour. Write that agreement down before operations start, or the first production outage will write it for you.