Backfilling a Historian Gap with OPC UA HistoryRead Without Double-Counting
Repairing a historian gap with OPC UA HistoryRead: scoping the window, letting returnBounds handle edges, keeping Bad quality, and paging safely.
An operator opens a two-week energy trend and finds a flat gap on Tuesday night. The gateway dropped its OPC UA session during a switch firmware push, reconnected an hour later, and nobody noticed until the monthly report came up short. Now someone wants the data "put back." That request is where most historian corruption starts — because backfill done casually loads the same boundary sample twice, silently drops Bad quality, or pulls the last-known value across the whole outage as a straight line.
HistoryRead is not a bigger subscription. A live subscription keeps the HMI current, and you tune it with sampling interval, publishing interval, and queue size. HistoryRead lives in a different part of the standard — OPC UA Part 11, Historical Access — and it asks a different set of questions:
- Which server actually holds the missed samples as system of record?
- Are the timestamps source time (when the PLC sampled) or server time (when the gateway received the value)?
- Do Bad and Uncertain samples survive the round trip, or does an import filter quietly discard them?
- Can you bound the request so a 30-day read doesn't stall the production server for everyone else?
Treat backfill as a controlled repair — scoped, logged, reversible in principle — not a read that happens to cover a lot of time.
Scope the gap to the second, not the day
Do not backfill "yesterday" because a trend looks blank. Find the exact missing interval per tag group first. The evidence is usually already logged somewhere:
- SCADA driver disconnect/reconnect events, and OPC UA session-close or secure-channel-renewal-failure entries.
- The historian's own archive and interface-buffer logs.
- Tag quality transitions from Good to Bad or Uncertain — those bracket the outage more precisely than any human's memory of "around 2am."
- Any time-sync alarms in the same window, which change how you trust the timestamps you're about to read.
Write the window with an inclusive start and exclusive end, e.g. 2026-06-27T02:14:10Z <= t < 2026-06-27T02:43:00Z. It reads like pedantry until the third repair attempt, when a <= on both ends is why one sample keeps landing twice.
Let returnBounds find your boundary samples
Most backfill errors happen at the edges. You want the last sample before the outage and the first after reconnect for trend continuity, but they must not be reloaded as if they were inside the gap. People solve this by hand — read a context window before, the gap, a context window after, then eyeball what to keep — and get it wrong under pressure.
The spec already does it. ReadRawModifiedDetails carries the fields that matter:
| Field | Set it to |
|---|---|
startTime / endTime | Your scoped gap, exclusive-end in intent |
numValuesPerNode | A hard cap per call — 5000 is a sane start; 0 means "everything," which is how you page 30 days in one blocking request |
returnBounds | true, so the server returns the bounding values just outside the interval, flagged, instead of you guessing them |
isReadModified | false for raw values; true only when you're auditing prior corrections |
With returnBounds = true you get the boundary points labeled as bounds, so the historian can interpolate across the edge without you loading a duplicate sample into the archive. That's the difference between a clean repair and an audit trail full of phantom transitions.
Confirm the server stores what you think it does
A server with good live values does not necessarily serve useful history. Some historize only selected nodes, some keep a short rolling buffer, some return processed aggregates when you wanted raw samples. Check before you commit a procedure to paper:
| Check | What to verify |
|---|---|
Historizing attribute | The Variable node actually claims history is being collected |
| Oldest sample | Retention covers the outage — a server holding 6 hours won't help after a weekend trip |
| Raw vs. processed | You're calling ReadRawModifiedDetails, not pulling Part 13 aggregates by accident |
| AccessLevel | The HistoryRead bit (0x04) is set, and UserAccessLevel grants it to your session — live read access proves nothing about history access |
| StatusCode | Bad and Uncertain samples come back, not silently dropped |
| Continuation points | Large results page correctly instead of truncating at page one |
If retention is short, that limit goes in the procedure in bold. A backfill that assumes a seven-day window fails during the first real maintenance shutdown — which is exactly when someone will need it.
Don't turn a repair into a denial of service
A request for 2,000 tags over 30 days hits disk, decompression, security checks, and network transfer at once, and HistoryRead is far heavier than the live subscription that server also has to keep serving. Keep batches small enough to retry, cap each call with numValuesPerNode, and follow continuation points rather than inflating request size. Run repair jobs off-peak when you can, and watch server CPU, disk latency, and the rejected-request counter while they run.
A decent backfill tool pauses, resumes, and reports progress. A bad one retries the same oversized request until the production server is slow for every operator on the plant — and now you have two incidents.
Quality and timestamps are data, not metadata
Loading numeric values is not the whole job. Decide these before you touch production:
- Bad quality — write it as a Bad-quality value or skip it? For troubleshooting and compliance, preserving the Bad interval is more honest than a clean-looking dashboard that hides the outage. When an aggregate subinterval has nothing, expect the server to say so with a
Good_NoDatastatus rather than inventing a value. - Uncertain quality — acceptable in OEE, batch, or energy reports, or not?
- Duplicate timestamps — overwrite, ignore, or create a second event? Test this with a five-tag set before a broad run; if the historian accepts duplicates, a single state transition can get counted twice.
- Source vs. insertion time — does the destination store both? You'll want the separation the first time a gateway historizes on receipt instead of on PLC sample time.
For an analog tag, a flat line across the outage is worse than a visible gap. If the server returns the last-known value repeatedly at Good quality, challenge that before it lands in a report — a gap tells the truth; a straight line lies quietly.
The failures that actually bite
Empty history while live data works. Usually the node isn't historized, or your service account has CurrentRead but not the HistoryRead bit. Check the node's Historizing attribute and your UserAccessLevel before assuming the outage window is wrong.
Values come back on server timestamps. Some gateways stamp on receipt, not on PLC sample time, so events shift by the communication delay. Compare a few returned timestamps against PLC event logs before trusting the lot.
Continuation points dropped. A paged response looks like a success but only the first page loaded. If your numbers seem too clean after a big read, this is the first thing to check.
Bad quality vanished. An import script filtering for Good only makes the trend pretty and erases the evidence of the outage you were documenting.
One rehearsal before the real thing
Prove the procedure once, deliberately, before an unplanned outage forces you to improvise. Pick ten tags with different behavior — fast analog, slow analog, a digital state, an alarm state, and a batch or lot context tag. Stop the historian interface or block the client for a short controlled window, restore it, and confirm the gap is visible. Backfill with the documented steps, then compare trends, raw sample counts, quality, and timestamps before and after, and confirm no report double-counts the boundary samples.
Keep a short record per job — NodeId list, gap window in UTC, cause, endpoint and server identity, the account used, raw-or-processed mode, values read vs. loaded, quality and duplicate rules, and who reviewed it. It isn't paperwork for its own sake; it's how the next engineer understands why the archive changed after the fact. Skip the rehearsal and your first real backfill becomes a live experiment during the outage review.