A Straight Line in Your Historian Isn't Always Data

The line that lied

A plant manager once showed me a temperature trend from an overnight shift and asked why the process looked so stable when the batch had actually failed. The trend was a clean, gently sloped line. It was also completely fake: the OPC UA server had lost its connection to the PLC at 02:14, and the historian filled the four-hour gap with a straight line between the last good sample and the first one after the collector reconnected. The process wasn't stable. The historian was blind, and the default settings let it lie about it.

That is the core problem with historian gap handling. A missing sample is information — it tells you the network dropped, or the instrument faulted, or a timestamp got rejected. The moment you let a trend connect across it or a report average over it, you've thrown that information away and replaced it with a number that looks trustworthy and isn't.

Before a tag ever shows up in a report, I want an answer to one question: what does a gap on this tag mean, and who gets hurt if we fill it? The answers differ wildly:

Process was steady, only the network failed — filling is probably fine.
PLC stopped scanning the instrument — the value is unknown, don't invent it.
Instrument faulted — the historian should have Bad quality, not a held number.
Timestamp rejected as too old or too far in the future — the sample is missing and the clock is wrong; fix the clock first.

Get that decision made per tag, up front. Bury it in report code and you'll rediscover it during an audit, which is the worst possible time.

Store quality, or you're storing rumors

A sample is not a number. It's a number plus the context that tells you whether to believe it. At minimum I want these five fields per point:

Field	Why it matters
Value	The measured or calculated number
Source timestamp	When the controller says the value was true
Server timestamp	When the collector actually received it
Quality	Good / Uncertain / Bad plus a specific sub-status
Collection source	Which OPC UA server, MQTT node, PLC driver, or manual entry

The quality field is not a boolean, and treating it as one is where most systems go wrong. OPC UA (Part 8 of the spec) gives you a full StatusCode: the top two bits split Good / Uncertain / Bad, but the sub-code is where the diagnosis lives. Bad_NoCommunication (0x80310000) means the comm link is down — that's a network problem. Uncertain_LastUsableValue (0x40900000) means the server is handing you a stale sample it knows is old. Uncertain_SubstituteValue means someone forced it. Collapse all of those into "bad" and you've thrown away the exact byte that would have told you where to look. DNP3 (IEEE 1815) does the same job with its flag octet — COMM_LOST, RESTART, REMOTE_FORCED, LOCAL_FORCED — and it's worth mapping those into your historian's quality model rather than discarding them at the driver.

For troubleshooting, the source timestamp and the quality tell you more than the value ever will. A flat temperature line with Good quality is a stable process. A flat line with Uncertain_LastUsableValue is a dead PLC link. Same picture, opposite meaning.

One interpolation rule for every tag is malpractice

Here's the mistake I see most: someone sets the historian's default interpolation to "linear" globally and moves on. Now a discrete pump-run bit gets a half-state during a gap, a totalizer gets a phantom straight-line climb, and an alarm state gets averaged. None of those are physically real.

Interpolation has to follow the tag's behavior:

Tag type	What it should do in a gap
Continuous analog (slow process)	Linear (sloped) fill acceptable for short gaps only
Discrete state	Stepped — hold the last state, never a half value
Counter / totalizer	Delta logic with rollover handling; never blind linear fill
Lab or manual sample	Hold only if the business rule says it represents the period
Alarm / event	Preserve event timestamps; never smooth or average
Calculated KPI	Recompute from qualified inputs; downgrade output quality if any input was missing

OPC UA Historical Access (Part 11) actually models this: every historized variable has a Stepped flag in its HA configuration that decides whether a history read interpolates with a slope or holds the last value. Set it wrong on a discrete tag and the server itself will hand out invented intermediate values on an interpolated read. It's a one-boolean fix that people forget exists.

And keep the display rule separate from the reporting rule. A 30-second linear fill that makes an operator trend readable is fine. The same fill in an environmental compliance report is potentially a regulatory problem. Write them as two rules, because they are two rules.

Cap the fill, or a two-hour outage becomes a straight line

If you allow interpolation, you must set a maximum gap length. Without it, the four-hour outage from the opening story renders as one tidy line, because two valid endpoints are all linear fill needs.

What I actually configure:

Analog temperature: fill up to 30 s for operator trend readability, no more.
pH used for environmental reporting: fill nothing — mark the period invalid and let the report show a gap.
Discrete run state: hold until the next event only if PLC scan and event capture were healthy through the gap.
Utility meters: estimate short gaps from surrounding totals, but stamp the estimate in the daily report so nobody treats it as metered.

That maximum comes from process dynamics and reporting risk, not from whatever the historian shipped with. A slow thermal loop tolerates a longer fill than a flow that swings in seconds.

Make bad quality impossible to miss

Operators and engineers should see a quality problem on the trend without ever opening the raw archive. If the trend silently connects a bad-quality period, every future investigation turns from engineering into archaeology.

Non-negotiables for me:

Break the line when quality is Bad and no fill rule applies. A visible gap beats a confident lie.
Draw estimated or interpolated segments as a dotted line — different from measured data at a glance.
Show the quality sub-code and source timestamp under the cursor, not just the value.
Render collector-outage windows as background bands so the outage duration is obvious.
Let export carry value and quality. An export that drops quality is where good data goes to become anonymous.

Reports need a quality threshold, not just an average

A report that averages whatever samples happen to exist will happily average across an outage and hand you a number with a straight face. Every interval needs a coverage rule.

For a shift report I'd write something like:

Production count: accept only if counter quality is Good at both shift start and shift end.
Average temperature: require ≥95% Good-quality coverage, otherwise mark partial.
Maximum pressure: Good-quality samples only; flag if any single outage exceeded 60 s.
Runtime %: compute from state/event data only when state quality is known.
Energy usage: allow estimated short gaps, but record the estimation method on the report.

The output carries a status — valid, partial, estimated, or invalid — right next to the number. A bare figure with no quality result is where a "0.3% yield loss" turns into a two-hour argument in the operations review.

The four ways this actually breaks

Last value carried forever. Collector drops, dashboard keeps showing the last Good value, everything looks calm while the historian sees nothing. Fix: stale detection tied to each tag's expected update rate. Past the stale timeout, flip the value to Bad or Uncertain — don't let it sit there looking alive.

Linear fill across a shutdown. Temperature drops during a planned stop, the historian draws a straight line from last-running to restart, and the report shows thermal behavior that never happened. Fix: gate interpolation on equipment state or batch phase so it can't cross a shutdown boundary.

Bad quality sneaking into averages. The classic: quality lives in a separate column and the report query just forgets to filter on it. Fix: before go-live, feed the report deliberately Bad samples and confirm it excludes or flags them. If you didn't test it with bad data, you don't know what it does with bad data.

Manual correction that erases the evidence. Someone edits or backfills to make a report look complete, and the original Bad-quality window vanishes. Fix: every manual correction keeps reason, user, time, original value, replacement value, and resulting quality. A correction that isn't auditable is just tampering with a nicer UI.

Prove it during commissioning

Don't trust any of this until you've broken it on purpose. Before anyone relies on the reports:

Pull the collector or block the PLC route for a short window — confirm the trend shows a gap or estimated segment per the rule, not a clean line.
Force an instrument fault quality if the driver allows it — confirm the report excludes or flags the period.
Backfill late data with old source timestamps — confirm the historian files it at the correct time, not at arrival time.
Toggle a discrete state during a simulated outage — confirm the system doesn't invent an event timestamp.
Export a trend and check that quality and timestamps come with it.
Run the shift report below the required coverage and confirm it comes back partial or invalid.

Keep the rules next to the tag, not in report code

The last thing I'll insist on: don't bury interpolation and quality rules only in the reporting layer, where they're invisible to the engineer maintaining the tag. Put them where scan rate, units, deadband, and retention already live — on the tag or asset definition:

Expected update interval
Stale timeout
Gap fill method
Maximum fill duration
Report coverage requirement
Whether estimated data is allowed
Whether manual correction needs approval

If a future engineer can look at the tag and tell you why the trend has a gap, why the report came back partial, and whether that filled value is trustworthy — you've done it right. If they have to reverse-engineer a report query at 2am to find out, you haven't.