Repository navigation
fix(dashboard): draw every temperature series as a continuous line in the day view - #1087
Merged
Merged
Conversation
… the day view The day view is the only temperature range that returns raw points, so a SATA drive reporting an SCT temperature history table contributes one reading per logging minute while an NVMe drive contributes one per collector run. The shared snap tolerance is derived from the median interval across all series combined, so the minute-resolution series sets it to about 30 seconds, every one of its readings becomes its own cluster, and the NVMe series is padded with null in almost every slot. ApexCharts draws no line through null and the shared tooltip reports no value, which is issue #1076: isolated points instead of a line, and an empty tooltip entry unless the cursor sits on one of them. Alignment itself has to stay, because ApexCharts resolves every series in a shared tooltip with one data-point index. Interpolate the padded slots instead, per series, between that series' own readings: the straight segment the chart already draws between two readings is what the interpolated value reports, so no number appears that the rendered line does not already show. Fill only across gaps within two of the series' own median sampling intervals, and only when the series has at least two intervals, since a median over one gap says nothing about cadence. A collector outage is wider than that and stays visible as a gap, as do the slots before a series' first reading and after its last. A zero degree reading is treated as a reading, matching the temperature pipe guard.
…ure cadence Review found that a fill bound of two of a series' own median sampling intervals admits a gap of exactly two intervals, which is what one missed collector run leaves. The bound therefore scaled with the cadence: the default collector cron is `0 0 * * *`, so at an eight-hourly schedule a single missed run was drawn as a straight sixteen-hour line with a fabricated temperature in every slot. Lower the bound to one and a half intervals. A healthy gap of one interval is still filled, with half an interval of headroom for collector jitter, while a gap of two intervals stays a gap at every cadence rather than only at short ones. This needs no absolute ceiling, because the rule is now expressed in the only unit that matters, the series' own cadence. Also correct two comments. The minimum-interval rule is not about a median needing more than one sample, which it does not; it is that a single interval cannot distinguish a cadence from an outage, so a series holding two readings a day apart would be drawn as a straight line across the whole day view. And an interpolated value is not the value the rendered line already shows: the stroke curve is a user setting that defaults to 'smooth', so the drawn curve between two readings is a spline, not this straight segment. The narrow bound, not that equivalence, is what keeps the fill honest. Add the tests the review found missing: the fill bound at its boundary and just past it, a missed run at a long cadence, interpolation driven by elapsed time rather than by a slot's position in the irregular shared index, and a shared tooltip reporting an interpolated value where it previously reported nothing.
…measurement Two independent reviews ranked this first: an interpolated slot was an ordinary point, and the shared tooltip formatted it exactly like a reading the drive sent. On a disk health dashboard that matters, because an operator reading the chart during a thermal event could not separate measured evidence from assumed gradual heating. Carry provenance instead of discarding it. `TemperatureChartPoint` gains an optional `estimated` flag that the interpolation pass sets, and `estimatedTemperaturePoints` projects it into the shape the shared tooltip needs, by series index then data-point index. An estimated value renders with a leading `~` and a title of "Estimated between readings"; a reading renders exactly as before. The flag reaches the tooltip as a callback rather than a captured array on purpose. `_updateTemperatureChartSeries` replaces the series without rebuilding the tooltip closure, so a snapshot taken when the chart was first prepared would describe a set of series that is no longer displayed after the first refresh. The component records provenance inside `_deviceDataTemperatureSeries`, the single place aligned series are produced, so the two cannot disagree. The prefix is applied to the output of `formatTemperature`, which still only formats and never converts, so the Fahrenheit double-conversion failure mode in danger-zones.md is untouched. Tests cover both halves separately, as the review asked: a measured slot renders with no marker, a padded slot renders with one, a slot left null is not reported as an estimate, and the flag projection is asserted against the aligned series.
… series has none The second review found that the interpolation bound leaves the reported bug unfixed on the shipped default collector schedule. `0 0 * * *` gives a drive without an SCT temperature history table one or two readings inside the one-day window, so it holds at most one sampling interval, falls below the two-interval minimum, and keeps isolated points and an empty tooltip. Widening the bound is not the answer. A line drawn across a whole day from two readings a day apart is exactly the fabrication the minimum exists to prevent, and the reviews were right that inventing continuity from one observation is worse than showing none. Separate the two halves of the symptom instead. The line genuinely cannot be drawn, and still is not: padding outside the bound keeps a null y, so the gap stays visible. The tooltip can be answered honestly, and now is. `temperatureSlotProvenance` classifies every slot of every aligned series as a measurement, an interpolated value, or a null slot that names the series' nearest real reading and the time it was taken. The tooltip renders that third case as the reading and its timestamp, for example `33 °C @ 00:00`, with a title explaining that no reading exists at the hovered time. It is a true statement about a real measurement at every cadence, including the daily default, and it replaces a bare `--`. This also improves a collector outage, where every slot inside the gap now names the last reading before it while the line still shows the gap. The nearest-reading search walks slots and readings together in one pass, since both are in time order. Its comparison is strict, so a slot exactly between two readings keeps the earlier one: a hovered time is never answered with a measurement taken after it. `estimatedTemperaturePoints` is replaced by `temperatureSlotProvenance`, which carries the same interpolation flag plus the nearest-reading case, so the tooltip reads one projection rather than two parallel ones.
…er threshold `AtaReplacementRiskWeights` derives a residual weight for every critical ATA attribute the curated table omits, and required the attribute to carry observed thresholds. That requirement was correct when it was written: ValidateThreshold warned on any critical attribute whose value matched no bucket, so a critical attribute with no buckets at all was WARN on every drive that reported it, and weighting that warning would have charged healthy drives for a metadata gap. #1088 removed the unconditional warning. A bucket-less attribute now leaves the status untouched and the manufacturer threshold decides it, which TestValidateThreshold_Attr177_NoBucketWarning_Eliminated pins. The same change stripped the vendor-scaled buckets from attribute 184, so the surviving requirement silently dropped it, and attribute 201 with it, out of replacement risk altogether. Drop the observed-threshold requirement. Both attributes are critical and both are decided by a manufacturer threshold, so each contributes only when a drive actually fails that threshold, which is a drive signal rather than a metadata gap. Behavior change: a drive whose End-to-End error (184) or Soft Read Error Rate (201) fails its manufacturer threshold now carries up to 3 risk points for each, at a trend multiplier of 1.0, and shows the attribute in its contributions table. Those are residual weights above the curated table, which still sums to 100 and is unchanged, and the totals stay under the existing ceiling. A drive that passes both thresholds scores exactly as before, because a bucket-less attribute no longer produces a status of its own. Both tests that encoded the old premise are updated. Coverage no longer filters on observed thresholds, so it demands a weight for every critical attribute, and the test that asserted 184 and 201 must not be scored now asserts that they are, with the reason recorded. #1088 and #1089 were each green on their own merge commit and conflict only in combination, and no CI run has executed on the develop tip since they landed, so nothing caught it there.
…nufacturer threshold" This reverts commit a864377. #1092 landed the same fix on develop independently, with the same diagnosis and the same removal of the observed-threshold gate, verified against the deployed develop image. Carrying a second copy here only conflicts with it, so this branch takes develop's version instead and keeps its own change frontend-only.
CI triggered on pull_request alone, and the comment above the trigger claimed it ran on pushes as well. A pull-request run builds the merge of one branch with the develop of the moment, so two pull requests that are each green can still break develop in combination, and nothing then tests the result: of the six workflows a push to develop does start, Publish Testing Image, osv, gitleaks, Security Scan, Dependency Audit and Docker, none runs either test suite. Docker builds the webapp and would catch a compile error, but a failing test reaches nobody. #1088 and #1089 landed exactly that way. Each was green on its own merge commit, together they left the Go suite failing on develop, and the first thing to notice was an unrelated pull request opened afterwards, which is where #1091 came from. A push run would have gone red on the second of the two merges, naming the commit that broke it. Pushes to master need no entry: SonarQube Analysis already runs both suites there, and master is reached only through a promotion pull request, which is a pull_request event. Lint Backend is restricted to pull requests. It reports findings new relative to the base branch, and a push has no base to be new against, so github.base_ref is empty and --new-from-rev would be left as a bare "origin/". Every line reaching develop was already linted in the pull request that brought it, and two green changes interacting produce no new findings in changed lines. No concurrency group on purpose. Cancelling a superseded push run would discard the result for the commit that was skipped, which is the result this adds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In the dashboard temperature chart's day view, a drive sampled less often than its
neighbours was drawn as isolated points instead of a line, and the shared tooltip
showed no value for it unless the cursor sat exactly on one of those points. This
fills the padded slots that alignment creates, per series, so every series is drawn
as a continuous line between its own readings while real outages stay visible as
gaps.
Type of Change
Related Issues
Related to #1076. The closing keyword belongs on the promotion PR into
master,because a closing keyword only fires on a merge into the default branch.
Introduced by the alignment added for #692.
Root cause
The day view is the only temperature range that returns raw points:
lookupResolutionreturns""forDURATION_KEY_DAY, added in 58abbcc sotooltip timestamps match real collection times. Every other range aggregates to one
hour, which puts all series on the same grid, so only the day view is affected.
A SATA drive with
Retrieve SCT Temperature Historyenabled writes the whole SCTtable, one point per logging minute, plus its current temperature. An NVMe drive
writes only its current temperature, one point per collector run. Those two cadences
then meet in
alignTemperatureChartSeries, whose snap tolerance is the medianinterval across all series combined, halved:
The minute-resolution series dominates the interval list, so the tolerance becomes
about 30 seconds. Every one of its readings becomes its own cluster, each NVMe
reading joins exactly one of them, and the NVMe series is padded with
nullin allthe others. ApexCharts draws no line through
null, and the shared tooltip readsnullat those indexes.Feeding the shipped function two hours of synthetic data, one point per minute for
the SATA series and one per fifteen minutes for the NVMe series, reproduces the
report exactly:
Changes Made
seriesIntervals, the gaps between one series' own consecutive readings.snapTolerancenow pools those, which is what it already did inline.series' own readings. ApexCharts has no
connectNulls-style option (8.6.0 here after the merged dependency bumps; thedocumented
strokeproperties areshow,curve,lineCap,colors,width,dashArrayandfill), so a padded slot has to carry a value for the line tojoin and the shared tooltip to report anything. An interpolated slot therefore
reports a temperature the drive never sent, which is why the bound below is
narrow. Note it is not the value the rendered line shows either:
stroke.curvecomes from the
line_strokesetting, which defaults tosmooth, so the drawncurve between two readings is a spline rather than that straight segment.
intervals. A healthy gap of one interval is filled, with half an interval of
headroom for collector jitter, while a gap of two intervals, which is what one
missed collector run leaves, stays a gap at every cadence. Wider gaps, and the
slots before a series' first reading and after its last, stay
null.interval cannot distinguish a cadence from an outage, so a series with two
readings a day apart would otherwise be drawn as a straight line across the
whole day view.
point.y !== null && Number.isFinite),matching the temperature pipe guard in danger-zones.md.
TemperatureChartPointgains an optionalestimatedflag that the interpolation pass sets, andtemperatureSlotProvenanceclassifies every slot of every aligned series as a measurement, an interpolated value,
or a null slot that names the series' nearest real reading and the time it was taken.
An estimated value renders with a leading
~and a title of "Estimated betweenreadings"; a reading renders exactly as before. Markers are applied to the output of
formatTemperature, which still only formats and never converts, so the Fahrenheitdouble-conversion failure mode in danger-zones.md is untouched.
outside the fill bound keeps a null y, so the line still shows a gap, but the tooltip
now renders that slot as the nearest reading and its timestamp, for example
33 °C @ 00:00, with a title explaining that no reading exists at the hovered time.The nearest-reading search walks slots and readings together in one pass, since both
are in time order, and its comparison is strict so a slot exactly between two readings
keeps the earlier one: a hovered time is never answered with a measurement taken after
it.
_updateTemperatureChartSeriesreplaces the series without rebuilding the tooltipclosure, so a snapshot taken when the chart was first prepared would describe series
that are no longer displayed after the first refresh. The component records provenance
inside
_deviceDataTemperatureSeries, the one place aligned series are produced.Alignment itself stays. ApexCharts resolves every series in a shared tooltip with
one data-point index, which is the defect #692 fixed; without a shared index the
hover markers for the other series land on the wrong points.
Base
Rebased onto
developby merge, so this branch carries #1083, #1088, #1089 and #1092.Its own diff is frontend-only; the backend tree is identical to
develop.While this was open,
developbriefly failedTest Backendon an interaction between#1088 and #1089 that no CI run had covered, because CI triggers on
pull_requestandneither merge was built against the other. #1092 fixed it on
develop. This branchcarries that fix rather than a second copy of it.
CI also runs on a push to develop now
ci.yamltriggered onpull_requestalone, and the comment above the trigger claimedit ran on pushes too. A pull-request run builds the merge of one branch with the
develop of the moment, so two pull requests that are each green can break develop in
combination, and nothing then tested the result. Of the six workflows a push to
develop does start,
Publish Testing Image,osv,gitleaks,Security Scan,Dependency AuditandDocker, none runs either test suite;Dockerbuilds thewebapp and would catch a compile error, but a failing test reached nobody.
That is how #1088 and #1089 left the Go suite red on develop, and why an unrelated
pull request was the first to see it. A push run would have gone red on the second of
the two merges and named it.
Pushes to master need no entry:
SonarQube Analysisalready runs both suites there,and master is reached only through a promotion pull request.
Lint Backendisrestricted to pull requests, because it reports findings new relative to a base branch
and a push has no base, leaving
github.base_refempty and--new-from-reva bareorigin/; every line reaching develop was already linted in its own pull request. Noconcurrency group, deliberately: cancelling a superseded push run would discard the
result for the commit it skipped, which is the result this adds.
A merge queue would prevent this rather than detect it, but
merge_grouphas neverfired in this repository, so adding that trigger would be inert until a queue is
enabled in settings.
Review
Two independent reviews ran against this branch, and both ranked the same issue first:
an interpolated slot was indistinguishable from a reading in the shared tooltip. The
second review called it blocking. Both are addressed above, by the
~marker and bylowering the fill bound from two intervals to one and a half. The second review's other
blocking finding, that the default collector schedule left the bug unfixed, is addressed
by the nearest-reading tooltip rather than by widening the bound.
The second review also showed that one and a half intervals is a heuristic and not an
outage detector, in both directions: a healthy five minute cadence with one collection
delayed three minutes leaves an eight minute gap that exceeds the seven and a half
minute bound and breaks a healthy line, while a missed five minute run followed by a
reading three minutes early leaves a seven minute gap that fills. Two intervals is
worse, because it hides one missed run at every cadence by construction, so the bound
stays at one and a half until there is measured jitter evidence to set it from.
Behavior worth naming for release notes
The day view previously showed only integer temperatures, because the raw points
are stored through
toInt(). An interpolated slot can be fractional, andformatTemperaturerenders through Angular's default1.0-3, so a day-view tooltipmay now read
33.067. The other ranges already show fractional values from theirone-hour
meanaggregation, so this is new to the day view rather than new to thedashboard.
Rejected alternatives
the lines stay broken.
the drive never traced, and the chart's stroke curve is user-configurable.
collector interval, which is user-configurable and not known to the backend, and
it reverts 58abbcc.
within a cluster with
mean, which is what every other range already does. Thisfabricates nothing, and is the strongest alternative. It is not free either: it
gives up the sub-cluster detail and the exact tooltip timestamps that 58abbcc
bought for the day view, for every drive, to fix the rendering of the sparsest
one. Worth revisiting if interpolation proves unpopular.
The default collector schedule
0 0 * * *gives a drive without an SCT temperature history table one or two readingsinside the one-day window, so it holds at most one sampling interval and no line can
honestly be drawn: a straight line across a whole day from two readings a day apart
carries no information, which is what the two-interval minimum exists to prevent.
The two halves of the reported symptom are therefore separated. The line genuinely
cannot be drawn and still is not, so the gap stays visible. The tooltip is answered,
at every cadence including the daily default, with the nearest real reading and its
timestamp instead of a bare
--. Issue #1076 itself reports a*/15schedule, whichgives 96 readings a day and a fully continuous line.
Testing
Eight specs added to
temperature-chart-series.spec.ts: a continuous line for aseries sampled less often than its neighbour, linear interpolation of padded slots,
a collector outage preserved as a gap, no padding before a series' first or after
its last reading, a zero degree reading treated as a reading, the fill bound at its
boundary and just past it, a missed run at an eight-hourly cadence, and
interpolation driven by elapsed time rather than by a slot's position in the
irregular shared index.
One spec added to
temperature-chart-apex.spec.ts, which previously only assertedthat a padded slot renders as
--. Its own fixture still does, because that seriesholds two readings; the new one holds three, so the shared tooltip reports the
interpolated value and the spec pins that text.
The implementation before and after this change was run against all 29 assertions
in the series suite, the 8 pre-existing ones and the 21 new ones. The old
implementation fails 10; the new one fails none. All four pre-existing specs are
unchanged in both. Every asserted interpolation value is exact in IEEE 754, so
toBecannot flake.Provenance is covered case by case: a measured slot renders with no marker, an
interpolated slot renders with one, a null slot names the nearest reading, a collector
outage keeps naming the reading before it, a series with no readings at all says
nothing, a zero degree reading is a valid nearest reading, and a slot exactly between
two readings keeps the earlier one. Two tooltip specs render through ApexCharts: one
asserting an estimated row and a measured row in the same hover, one asserting a
nearest-reading row. They assert the value and the presence of a time rather than an
absolute time string, because the runtime's own zone formats it. The pre-existing
tooltip spec still asserts
--on its own two-reading fixture, through the callback'sdefault, so the unannotated path stays pinned.
The Jasmine suite runs in CI through the
Test Frontendjob, which callsmake binary-frontend-test-coverage(ci.yaml), so these specs are gated therealongside the existing #692 specs.
Checklist
Additional Notes
Review: not required (frontend rendering change, no security or access boundary, no
shared cross-repository contract, and it takes effect only through the normal
develop image and release path).
Once
ghcr.io/starosdev/scrutiny:developis republished, the reporter on #1076 canconfirm against their own Synology data with
Retrieve SCT Temperature Historyleftenabled.