Skip to content

textfile: fix panic on a metric without HELP after inconsistent help texts - #3866

Open
vignesh2027 wants to merge 3 commits into
prometheus:masterfrom
vignesh2027:textfile-missing-help-panic
Open

vignesh2027 wants to merge 3 commits into
prometheus:masterfrom
vignesh2027:textfile-missing-help-panic

Conversation

@vignesh2027

@vignesh2027 vignesh2027 commented Oct 22, 2026 •

Copy link
Copy Markdown

The textfile collector panics with a nil pointer dereference when the same metric appears in three or more .prom files, two of them with different HELP texts and a later one with no HELP line at all.

After the first mismatch, the help text check treats every later family with that name as inconsistent because of the || helpTexts[1] != "" branch. That branch is also taken when the family has no HELP, and the code then reads *mf.Help, which is nil. The panic happens inside the per-collector goroutine started by NodeCollector.Collect, so nothing recovers it and node_exporter exits.

This change logs the help text through mf.GetHelp() and only overwrites the recorded help text when the new family has one. A family with no HELP is now dropped and reported through node_textfile_scrape_error, the same way other inconsistent families are.

TestTextfileCollectorInconsistentHelpThenNoHelp writes four files: HELP "First", HELP "Second", no HELP, and HELP "First" again. On main it fails with the recovered nil pointer panic. With this change only the first file's metric is exported and the scrape error is set. The fourth file checks that a family with no HELP does not clear the inconsistency marker.

…texts

Once a metric has been seen with two different help texts, every later
family with the same name is treated as inconsistent. If such a later
family has no HELP line, the collector dereferenced its nil help text
and panicked. The panic happens in the per-collector goroutine, so it
takes down node_exporter.

Use the getter for the log line and keep the recorded help text when
the new family has none.

Signed-off-by: vigneshwar <applemacbook6sep2004@gmail.com>
@SuperQ

SuperQ commented Oct 28, 2026

Copy link
Copy Markdown
Member

Please do not @user mention maintainers directly. We have a number of maintainers and review as soon as we can. Direct pings are unnecessary and considered rude.

@vignesh2027

Copy link
Copy Markdown
Author

Sorry about that, I've removed the mention from the description.

Comment thread collector/textfile.go Outdated
Comment on lines +222 to +223
// A family without a help text can still land here once
// the metric has been marked as inconsistent.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// A family without a help text can still land here once
// the metric has been marked as inconsistent.

Comment thread collector/textfile_test.go Outdated
Comment on lines +197 to +203
files := map[string]string{
// Files are read in lexical order.
"a.prom": "# HELP foo First help text.\n# TYPE foo gauge\nfoo{file=\"a\"} 1\n",
"b.prom": "# HELP foo Second help text.\n# TYPE foo gauge\nfoo{file=\"b\"} 2\n",
"c.prom": "# TYPE foo gauge\nfoo{file=\"c\"} 3\n",
"d.prom": "# HELP foo First help text.\n# TYPE foo gauge\nfoo{file=\"d\"} 4\n",
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be added to the fixtures, not inlined with a WriteFile in the test.

@SuperQ

SuperQ commented Oct 28, 2026

Copy link
Copy Markdown
Member

Any reason to not extend the lists of tests in TestTextfileCollector()?

Cover the inconsistent help case with a fixture in TestTextfileCollector
instead of a separate test that writes files, and drop the extra comment.

Signed-off-by: vigneshwar <applemacbook6sep2004@gmail.com>
@vignesh2027

Copy link
Copy Markdown
Author

No good reason, thanks. I moved the case into a new metrics_merge_inconsistent_then_no_help fixture in TestTextfileCollector, removed the separate test, and applied your suggestion. The fixture still panics without the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants