Skip to content

fix(dashboard): draw every temperature series as a continuous line in the day view - #1087

Merged
Starosdev merged 9 commits into
developfrom
fix/1076-temperature-series-gaps
Nov 3, 2026
Merged

Starosdev merged 9 commits into
developfrom
fix/1076-temperature-series-gaps

Conversation

@Starosdev

@Starosdev Starosdev commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

In the dashboard temperature chart's day view, a drive sampled less often than its
neighbours was drawn as isolated points instead of a line, and the shared tooltip
showed no value for it unless the cursor sat exactly on one of those points. This
fills the padded slots that alignment creates, per series, so every series is drawn
as a continuous line between its own readings while real outages stay visible as
gaps.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or infrastructure change

Related Issues

Related to #1076. The closing keyword belongs on the promotion PR into master,
because a closing keyword only fires on a merge into the default branch.

Introduced by the alignment added for #692.

Root cause

The day view is the only temperature range that returns raw points:
lookupResolution returns "" for DURATION_KEY_DAY, added in 58abbcc so
tooltip timestamps match real collection times. Every other range aggregates to one
hour, which puts all series on the same grid, so only the day view is affected.

A SATA drive with Retrieve SCT Temperature History enabled writes the whole SCT
table, one point per logging minute, plus its current temperature. An NVMe drive
writes only its current temperature, one point per collector run. Those two cadences
then meet in alignTemperatureChartSeries, whose snap tolerance is the median
interval across all series combined, halved:

return intervals.length > 0 ? Math.min(median(intervals) / 2, MAX_SNAP_TOLERANCE_MS) : 0;

The minute-resolution series dominates the interval list, so the tolerance becomes
about 30 seconds. Every one of its readings becomes its own cluster, each NVMe
reading joins exactly one of them, and the NVMe series is padded with null in all
the others. ApexCharts draws no line through null, and the shared tooltip reads
null at those indexes.

Feeding the shipped function two hours of synthetic data, one point per minute for
the SATA series and one per fifteen minutes for the NVMe series, reproduces the
report exactly:

Input Snap tolerance NVMe slots with a value
1 min + 15 min 30 s 9 of 121
15 min alone 450 s 9 of 9

Changes Made

  • Extract seriesIntervals, the gaps between one series' own consecutive readings.
    snapTolerance now pools those, which is what it already did inline.
  • After alignment, interpolate each series' padded slots linearly between that
    series' own readings. ApexCharts has no connectNulls-style option (8.6.0 here after the merged dependency bumps; the
    documented stroke properties are show, curve, lineCap, colors, width,
    dashArray and fill), so a padded slot has to carry a value for the line to
    join and the shared tooltip to report anything. An interpolated slot therefore
    reports a temperature the drive never sent, which is why the bound below is
    narrow. Note it is not the value the rendered line shows either: stroke.curve
    comes from the line_stroke setting, which defaults to smooth, so the drawn
    curve between two readings is a spline rather than that straight segment.
  • Bound the fill to gaps within one and a half of that series' own median sampling
    intervals. A healthy gap of one interval is filled, with half an interval of
    headroom for collector jitter, while a gap of two intervals, which is what one
    missed collector run leaves, stays a gap at every cadence. Wider gaps, and the
    slots before a series' first reading and after its last, stay null.
  • Fill only for a series holding at least two of its own intervals. A single
    interval cannot distinguish a cadence from an outage, so a series with two
    readings a day apart would otherwise be drawn as a straight line across the
    whole day view.
  • Treat a zero degree reading as a reading (point.y !== null && Number.isFinite),
    matching the temperature pipe guard in danger-zones.md.
  • Carry provenance rather than discarding it. TemperatureChartPoint gains an optional
    estimated flag that the interpolation pass sets, and temperatureSlotProvenance
    classifies every slot of every aligned series as a measurement, an interpolated value,
    or a null slot that names the series' nearest real reading and the time it was taken.
    An estimated value renders with a leading ~ and a title of "Estimated between
    readings"; a reading renders exactly as before. Markers are applied to the output of
    formatTemperature, which still only formats and never converts, so the Fahrenheit
    double-conversion failure mode in danger-zones.md is untouched.
  • Answer the tooltip with a real reading where a series has no value at all. Padding
    outside the fill bound keeps a null y, so the line still shows a gap, but the tooltip
    now renders that slot as the nearest reading and its timestamp, for example
    33 °C @ 00:00, with a title explaining that no reading exists at the hovered time.
    The nearest-reading search walks slots and readings together in one pass, since both
    are in time order, and its comparison is strict so a slot exactly between two readings
    keeps the earlier one: a hovered time is never answered with a measurement taken after
    it.
  • Pass provenance to the tooltip as a callback, not a captured array.
    _updateTemperatureChartSeries replaces the series without rebuilding the tooltip
    closure, so a snapshot taken when the chart was first prepared would describe series
    that are no longer displayed after the first refresh. The component records provenance
    inside _deviceDataTemperatureSeries, the one place aligned series are produced.

Alignment itself stays. ApexCharts resolves every series in a shared tooltip with
one data-point index, which is the defect #692 fixed; without a shared index the
hover markers for the other series land on the wrong points.

Base

Rebased onto develop by merge, so this branch carries #1083, #1088, #1089 and #1092.
Its own diff is frontend-only; the backend tree is identical to develop.

While this was open, develop briefly failed Test Backend on an interaction between
#1088 and #1089 that no CI run had covered, because CI triggers on pull_request and
neither merge was built against the other. #1092 fixed it on develop. This branch
carries that fix rather than a second copy of it.

CI also runs on a push to develop now

ci.yaml triggered on pull_request alone, and the comment above the trigger claimed
it ran on pushes too. A pull-request run builds the merge of one branch with the
develop of the moment, so two pull requests that are each green can break develop in
combination, and nothing then tested the result. Of the six workflows a push to
develop does start, Publish Testing Image, osv, gitleaks, Security Scan,
Dependency Audit and Docker, none runs either test suite; Docker builds the
webapp and would catch a compile error, but a failing test reached nobody.

That is how #1088 and #1089 left the Go suite red on develop, and why an unrelated
pull request was the first to see it. A push run would have gone red on the second of
the two merges and named it.

Pushes to master need no entry: SonarQube Analysis already runs both suites there,
and master is reached only through a promotion pull request. Lint Backend is
restricted to pull requests, because it reports findings new relative to a base branch
and a push has no base, leaving github.base_ref empty and --new-from-rev a bare
origin/; every line reaching develop was already linted in its own pull request. No
concurrency group, deliberately: cancelling a superseded push run would discard the
result for the commit it skipped, which is the result this adds.

A merge queue would prevent this rather than detect it, but merge_group has never
fired in this repository, so adding that trigger would be inert until a queue is
enabled in settings.

Review

Two independent reviews ran against this branch, and both ranked the same issue first:
an interpolated slot was indistinguishable from a reading in the shared tooltip. The
second review called it blocking. Both are addressed above, by the ~ marker and by
lowering the fill bound from two intervals to one and a half. The second review's other
blocking finding, that the default collector schedule left the bug unfixed, is addressed
by the nearest-reading tooltip rather than by widening the bound.

The second review also showed that one and a half intervals is a heuristic and not an
outage detector, in both directions: a healthy five minute cadence with one collection
delayed three minutes leaves an eight minute gap that exceeds the seven and a half
minute bound and breaks a healthy line, while a missed five minute run followed by a
reading three minutes early leaves a seven minute gap that fills. Two intervals is
worse, because it hides one missed run at every cadence by construction, so the bound
stays at one and a half until there is measured jitter evidence to set it from.

Behavior worth naming for release notes

The day view previously showed only integer temperatures, because the raw points
are stored through toInt(). An interpolated slot can be fractional, and
formatTemperature renders through Angular's default 1.0-3, so a day-view tooltip
may now read 33.067. The other ranges already show fractional values from their
one-hour mean aggregation, so this is new to the day view rather than new to the
dashboard.

Rejected alternatives

  • A per-series snap tolerance alone. A series can only fill one slot per cluster, so
    the lines stay broken.
  • Carrying the last known value forward. It turns the sparser line into a staircase
    the drive never traced, and the chart's stroke curve is user-configurable.
  • Aggregating the day view server-side. The window would have to be at least the
    collector interval, which is user-configurable and not known to the backend, and
    it reverts 58abbcc.
  • Clustering at the coarsest per-series cadence and aggregating each denser series
    within a cluster with mean, which is what every other range already does. This
    fabricates nothing, and is the strongest alternative. It is not free either: it
    gives up the sub-cluster detail and the exact tooltip timestamps that 58abbcc
    bought for the day view, for every drive, to fix the rendering of the sparsest
    one. Worth revisiting if interpolation proves unpopular.

The default collector schedule

0 0 * * * gives a drive without an SCT temperature history table one or two readings
inside the one-day window, so it holds at most one sampling interval and no line can
honestly be drawn: a straight line across a whole day from two readings a day apart
carries no information, which is what the two-interval minimum exists to prevent.

The two halves of the reported symptom are therefore separated. The line genuinely
cannot be drawn and still is not, so the gap stays visible. The tooltip is answered,
at every cadence including the daily default, with the nearest real reading and its
timestamp instead of a bare --. Issue #1076 itself reports a */15 schedule, which
gives 96 readings a day and a fully continuous line.

Testing

  • I have tested this locally
  • I have added/updated tests that prove my fix/feature works
  • All existing tests pass

Eight specs added to temperature-chart-series.spec.ts: a continuous line for a
series sampled less often than its neighbour, linear interpolation of padded slots,
a collector outage preserved as a gap, no padding before a series' first or after
its last reading, a zero degree reading treated as a reading, the fill bound at its
boundary and just past it, a missed run at an eight-hourly cadence, and
interpolation driven by elapsed time rather than by a slot's position in the
irregular shared index.

One spec added to temperature-chart-apex.spec.ts, which previously only asserted
that a padded slot renders as --. Its own fixture still does, because that series
holds two readings; the new one holds three, so the shared tooltip reports the
interpolated value and the spec pins that text.

The implementation before and after this change was run against all 29 assertions
in the series suite, the 8 pre-existing ones and the 21 new ones. The old
implementation fails 10; the new one fails none. All four pre-existing specs are
unchanged in both. Every asserted interpolation value is exact in IEEE 754, so
toBe cannot flake.

Provenance is covered case by case: a measured slot renders with no marker, an
interpolated slot renders with one, a null slot names the nearest reading, a collector
outage keeps naming the reading before it, a series with no readings at all says
nothing, a zero degree reading is a valid nearest reading, and a slot exactly between
two readings keeps the earlier one. Two tooltip specs render through ApexCharts: one
asserting an estimated row and a measured row in the same hover, one asserting a
nearest-reading row. They assert the value and the presence of a time rather than an
absolute time string, because the runtime's own zone formats it. The pre-existing
tooltip spec still asserts -- on its own two-reading fixture, through the callback's
default, so the unannotated path stays pinned.

The Jasmine suite runs in CI through the Test Frontend job, which calls
make binary-frontend-test-coverage (ci.yaml), so these specs are gated there
alongside the existing #692 specs.

Checklist

  • My code follows the project's code style
  • I have performed a self-review of my code
  • I have commented my code where necessary (particularly complex areas)
  • I have updated the documentation if needed
  • My changes generate no new warnings
  • No console.log, debug statements, or commented-out code

Additional Notes

Review: not required (frontend rendering change, no security or access boundary, no
shared cross-repository contract, and it takes effect only through the normal
develop image and release path).

Once ghcr.io/starosdev/scrutiny:develop is republished, the reporter on #1076 can
confirm against their own Synology data with Retrieve SCT Temperature History left
enabled.

… the day view

The day view is the only temperature range that returns raw points, so a SATA
drive reporting an SCT temperature history table contributes one reading per
logging minute while an NVMe drive contributes one per collector run. The shared
snap tolerance is derived from the median interval across all series combined,
so the minute-resolution series sets it to about 30 seconds, every one of its
readings becomes its own cluster, and the NVMe series is padded with null in
almost every slot. ApexCharts draws no line through null and the shared tooltip
reports no value, which is issue #1076: isolated points instead of a line, and
an empty tooltip entry unless the cursor sits on one of them.

Alignment itself has to stay, because ApexCharts resolves every series in a
shared tooltip with one data-point index. Interpolate the padded slots instead,
per series, between that series' own readings: the straight segment the chart
already draws between two readings is what the interpolated value reports, so
no number appears that the rendered line does not already show.

Fill only across gaps within two of the series' own median sampling intervals,
and only when the series has at least two intervals, since a median over one
gap says nothing about cadence. A collector outage is wider than that and stays
visible as a gap, as do the slots before a series' first reading and after its
last. A zero degree reading is treated as a reading, matching the temperature
pipe guard.
…ure cadence

Review found that a fill bound of two of a series' own median sampling intervals
admits a gap of exactly two intervals, which is what one missed collector run
leaves. The bound therefore scaled with the cadence: the default collector cron
is `0 0 * * *`, so at an eight-hourly schedule a single missed run was drawn as
a straight sixteen-hour line with a fabricated temperature in every slot.

Lower the bound to one and a half intervals. A healthy gap of one interval is
still filled, with half an interval of headroom for collector jitter, while a
gap of two intervals stays a gap at every cadence rather than only at short
ones. This needs no absolute ceiling, because the rule is now expressed in the
only unit that matters, the series' own cadence.

Also correct two comments. The minimum-interval rule is not about a median
needing more than one sample, which it does not; it is that a single interval
cannot distinguish a cadence from an outage, so a series holding two readings a
day apart would be drawn as a straight line across the whole day view. And an
interpolated value is not the value the rendered line already shows: the stroke
curve is a user setting that defaults to 'smooth', so the drawn curve between
two readings is a spline, not this straight segment. The narrow bound, not that
equivalence, is what keeps the fill honest.

Add the tests the review found missing: the fill bound at its boundary and just
past it, a missed run at a long cadence, interpolation driven by elapsed time
rather than by a slot's position in the irregular shared index, and a shared
tooltip reporting an interpolated value where it previously reported nothing.
…measurement

Two independent reviews ranked this first: an interpolated slot was an ordinary
point, and the shared tooltip formatted it exactly like a reading the drive
sent. On a disk health dashboard that matters, because an operator reading the
chart during a thermal event could not separate measured evidence from assumed
gradual heating.

Carry provenance instead of discarding it. `TemperatureChartPoint` gains an
optional `estimated` flag that the interpolation pass sets, and
`estimatedTemperaturePoints` projects it into the shape the shared tooltip
needs, by series index then data-point index. An estimated value renders with a
leading `~` and a title of "Estimated between readings"; a reading renders
exactly as before.

The flag reaches the tooltip as a callback rather than a captured array on
purpose. `_updateTemperatureChartSeries` replaces the series without rebuilding
the tooltip closure, so a snapshot taken when the chart was first prepared
would describe a set of series that is no longer displayed after the first
refresh. The component records provenance inside
`_deviceDataTemperatureSeries`, the single place aligned series are produced, so
the two cannot disagree.

The prefix is applied to the output of `formatTemperature`, which still only
formats and never converts, so the Fahrenheit double-conversion failure mode in
danger-zones.md is untouched.

Tests cover both halves separately, as the review asked: a measured slot renders
with no marker, a padded slot renders with one, a slot left null is not reported
as an estimate, and the flag projection is asserted against the aligned series.
… series has none

The second review found that the interpolation bound leaves the reported bug
unfixed on the shipped default collector schedule. `0 0 * * *` gives a drive
without an SCT temperature history table one or two readings inside the one-day
window, so it holds at most one sampling interval, falls below the
two-interval minimum, and keeps isolated points and an empty tooltip.

Widening the bound is not the answer. A line drawn across a whole day from two
readings a day apart is exactly the fabrication the minimum exists to prevent,
and the reviews were right that inventing continuity from one observation is
worse than showing none.

Separate the two halves of the symptom instead. The line genuinely cannot be
drawn, and still is not: padding outside the bound keeps a null y, so the gap
stays visible. The tooltip can be answered honestly, and now is.
`temperatureSlotProvenance` classifies every slot of every aligned series as a
measurement, an interpolated value, or a null slot that names the series'
nearest real reading and the time it was taken. The tooltip renders that third
case as the reading and its timestamp, for example `33 °C @ 00:00`, with a title
explaining that no reading exists at the hovered time. It is a true statement
about a real measurement at every cadence, including the daily default, and it
replaces a bare `--`.

This also improves a collector outage, where every slot inside the gap now names
the last reading before it while the line still shows the gap.

The nearest-reading search walks slots and readings together in one pass, since
both are in time order. Its comparison is strict, so a slot exactly between two
readings keeps the earlier one: a hovered time is never answered with a
measurement taken after it.

`estimatedTemperaturePoints` is replaced by `temperatureSlotProvenance`, which
carries the same interpolation flag plus the nearest-reading case, so the
tooltip reads one projection rather than two parallel ones.
…er threshold

`AtaReplacementRiskWeights` derives a residual weight for every critical ATA
attribute the curated table omits, and required the attribute to carry observed
thresholds. That requirement was correct when it was written: ValidateThreshold
warned on any critical attribute whose value matched no bucket, so a critical
attribute with no buckets at all was WARN on every drive that reported it, and
weighting that warning would have charged healthy drives for a metadata gap.

#1088 removed the unconditional warning. A bucket-less attribute now leaves the
status untouched and the manufacturer threshold decides it, which
TestValidateThreshold_Attr177_NoBucketWarning_Eliminated pins. The same change
stripped the vendor-scaled buckets from attribute 184, so the surviving
requirement silently dropped it, and attribute 201 with it, out of replacement
risk altogether.

Drop the observed-threshold requirement. Both attributes are critical and both
are decided by a manufacturer threshold, so each contributes only when a drive
actually fails that threshold, which is a drive signal rather than a metadata
gap.

Behavior change: a drive whose End-to-End error (184) or Soft Read Error Rate
(201) fails its manufacturer threshold now carries up to 3 risk points for each,
at a trend multiplier of 1.0, and shows the attribute in its contributions
table. Those are residual weights above the curated table, which still sums to
100 and is unchanged, and the totals stay under the existing ceiling. A drive
that passes both thresholds scores exactly as before, because a bucket-less
attribute no longer produces a status of its own.

Both tests that encoded the old premise are updated. Coverage no longer filters
on observed thresholds, so it demands a weight for every critical attribute, and
the test that asserted 184 and 201 must not be scored now asserts that they are,
with the reason recorded.

#1088 and #1089 were each green on their own merge commit and conflict only in
combination, and no CI run has executed on the develop tip since they landed, so
nothing caught it there.
…nufacturer threshold"

This reverts commit a864377.

#1092 landed the same fix on develop independently, with the same diagnosis and
the same removal of the observed-threshold gate, verified against the deployed
develop image. Carrying a second copy here only conflicts with it, so this
branch takes develop's version instead and keeps its own change frontend-only.
CI triggered on pull_request alone, and the comment above the trigger claimed it
ran on pushes as well. A pull-request run builds the merge of one branch with
the develop of the moment, so two pull requests that are each green can still
break develop in combination, and nothing then tests the result: of the six
workflows a push to develop does start, Publish Testing Image, osv, gitleaks,
Security Scan, Dependency Audit and Docker, none runs either test suite. Docker
builds the webapp and would catch a compile error, but a failing test reaches
nobody.

#1088 and #1089 landed exactly that way. Each was green on its own merge commit,
together they left the Go suite failing on develop, and the first thing to notice
was an unrelated pull request opened afterwards, which is where #1091 came from.
A push run would have gone red on the second of the two merges, naming the
commit that broke it.

Pushes to master need no entry: SonarQube Analysis already runs both suites
there, and master is reached only through a promotion pull request, which is a
pull_request event.

Lint Backend is restricted to pull requests. It reports findings new relative to
the base branch, and a push has no base to be new against, so github.base_ref is
empty and --new-from-rev would be left as a bare "origin/". Every line reaching
develop was already linted in the pull request that brought it, and two green
changes interacting produce no new findings in changed lines.

No concurrency group on purpose. Cancelling a superseded push run would discard
the result for the commit that was skipped, which is the result this adds.
@Starosdev
Starosdev merged commit e276646 into develop Nov 3, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant