Skip to content

feat(database): add a database page with the DB Doctor checks - #1123

Merged
kesselb merged 4 commits into
masterfrom
feat/database-checks
Sep 9, 2026
Merged

kesselb merged 4 commits into
masterfrom
feat/database-checks

Conversation

@karlitschek

Copy link
Copy Markdown
Member

Stacked on #1119. The base branch is feat/settings-tabs, because this adds a third page to the navigation that PR introduces — so the diff here is only the database work. GitHub will retarget this to master automatically when #1119 merges.

Why

The Monitoring app tells an admin what their database is — type, version, size — but never whether it is configured well. Nextcloud's DB Doctor app already answers that, so this ports its check engine into a third page sitting between Status and Background jobs.

On a stock Ubuntu MariaDB it surfaces 57 checks — 12 failing, 19 passing, 26 skipped — grouped by category.

What came across

  • the rule engine (Rule, RuleResult, RuleSet, the expression evaluator, the advisor) and the probe that snapshots the server
  • the MySQL/MariaDB rule set, which DB Doctor ported from phpMyAdmin's advisor. Its SPDX-FileCopyrightText for phpMyAdmin and the GPL-2.0-or-later provenance note travel with the file
  • the PostgreSQL and Nextcloud schema rule sets
  • the settings dialog for pointing the checks at a separate connection, so they can run as a user with rights Nextcloud itself does not need. Stored under serverinfo's own app config, password in the credentials manager, host validated against serverinfo.allowed_override_hosts
  • the live connections, throughput and latency boxes, in one row above the checks

What deliberately did not

The score, grade, mascot, history and trend charts. This is a page about what is wrong, not a dashboard to score.

Everything that writes. Failing checks offer the config snippet to paste into my.cnf or postgresql.conf, but nothing applies a value. A bundled monitoring app has no business holding write access to the database it is monitoring, so ApplyService, ApplyController and the audit trail stayed behind.

The "Run check now" button. The checks run once when the page is first opened, and the results survive switching tabs and coming back.

Two departures from how DB Doctor does it

One live endpoint, not two. Its dashboard polls /insights/metrics every 3 s and /check/ping at 1 Hz. Measuring on a live instance, a poll spends about 1.3 ms collecting data inside a request that takes ~450 ms — the request cycle costs two orders of magnitude more than the work. So /database/live returns the metrics and times its own round trip in a single call, and the boxes poll only while this page is on screen and the tab is visible.

Both live endpoints close the PHP session first. This one bit for real: holding the session for the length of a ~5 s check made the Status page's 2-second poll queue up behind it, far enough to hit AH00161: server reached MaxRequestWorkers and hang the page indefinitely. $this->session->close() before the work fixes it; neither endpoint writes to the session.

The charts reuse the chart.js setup the CPU and memory graphs already use rather than porting DB Doctor's own SVG sparkline and its 481-line latency chart — less new code, and visually consistent with the Status page.

Verification

Driven against a live NC 35 instance with Playwright and measured, not eyeballed:

  • /database/check returns 57 rules across 11 categories in ~1.2 s; the page renders 12 failing cards grouped by category
  • live boxes render at identical size on one row (334×129 measured) with real values — 1 / 151 connections, 14 queries/s, 0.6 ms latency
  • polling gating: 0 /database/live requests while on Status, ~7 per 30 s once on Database (median 418 ms), stopped when the tab is hidden
  • the settings dialog round-trips: Test connection dials without saving, Save persists

eslint, php-cs-fixer and psalm are all clean, and the committed js//css/ reproduce from a fresh build. The psalm baseline grew for the DI-constructed classes and for the private OC\DB\* classes the Nextcloud schema rules need; Doctrine DBAL is stubbed in tests/stub.phpstub, following the pattern already there.

Worth a maintainer's opinion

This duplicates DB Doctor's rule engine inside a bundled app, so the two copies will drift. I kept the ported files structurally identical and preserved the upstream rule ids so a future re-sync is mechanical, but if you would rather serverinfo deferred to DB Doctor when it is installed — or that this lived somewhere else entirely — I would rather hear it now than after the copies diverge.

One known wart came across with the port: on MariaDB, Concurrent_insert_off logs "Identifier 'concurrent_insert' is not numeric" because MariaDB reports that variable as AUTO. The advisor catches it and marks the rule skipped, so it is cosmetic, but it is a DB Doctor bug worth fixing at the source rather than diverging the copy here.

@karlitschek

Copy link
Copy Markdown
Member Author
Screenshot 2026-09-09 at 18 46 31

@karlitschek

Copy link
Copy Markdown
Member Author

Frank Karlitschek and others added 4 commits September 9, 2026 18:08
The Monitoring app reports what the database is, never whether it is
configured well, and the answer to that already exists in Nextcloud's DB
Doctor app. This ports its check engine into a third page, between Status
and Background jobs.

What came across:

- the rule engine (Rule, RuleResult, RuleSet, the expression evaluator and
  the advisor) and the probe that snapshots the server
- the MySQL/MariaDB rule set, ported in DB Doctor from phpMyAdmin's advisor
  and carrying its GPL-2.0-or-later attribution, plus the PostgreSQL and
  Nextcloud schema rules
- the dialog for pointing the checks at a separate connection, so they can
  run as a user with rights Nextcloud itself does not need. It stores under
  serverinfo's own app config, with the password in the credentials manager
- the live connections, throughput and latency boxes

What deliberately did not:

- the score, grade, mascot, history and trend charts. This is a page about
  what is wrong, not a gamified dashboard
- everything that writes. Failing checks offer the config snippet to paste
  into my.cnf or postgresql.conf, but nothing applies a value: a bundled
  monitoring app has no business holding write access to the database

The checks run once when the page is first opened rather than on a button,
and the page keeps its results when you switch tabs and come back.

Two departures from how DB Doctor does it. Its dashboard polls
/insights/metrics every three seconds and /check/ping once a second; here
both are one endpoint that returns the metrics and times its own round
trip, because the request cycle costs orders of magnitude more than the
queries inside it. And both live endpoints close the PHP session before
doing their work: holding it for the length of a five-second check made the
Status page's two-second poll queue up behind it, far enough to exhaust the
web server's workers.

The charts reuse the chart.js setup the CPU and memory graphs already use,
rather than porting DB Doctor's own sparkline and latency chart, and they
poll only while the database page is the one on screen and the tab is
visible.

Signed-off-by: Frank Karlitschek <frank.karlitschek@nextcloud.com>
Assisted-by: Claude Code:claude-opus-5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
The database page reports what is wrong with the database configuration,
but only to an admin who knows to open it. Settings > Overview is where
they already look for things that need attention, so a failing check now
says so there, linking straight to the findings.

Running fifty-odd rules against a live server on every Overview load is not
reasonable, so the outcome of a run is recorded in app config and the setup
check reads that. Opening the database page records a fresh result, which
means acting on a finding clears the warning without waiting for anything;
a server nobody has looked at for a day re-runs the checks once and records
that instead.

The link is passed as a rich object parameter rather than through
SetupResult's linkToDoc: the Overview does not render linkToDoc at all --
of the twenty-two checks on a stock install, the only one showing a link
builds it the way this now does.

Following that link needs the page to be addressable, so the admin settings
page now reads its view from the URL fragment (#status, #database,
leaves the back button alone while switching between them.

Signed-off-by: Frank Karlitschek <frank.karlitschek@nextcloud.com>
Assisted-by: Claude Code:claude-opus-5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
Two problems the database page introduced.

A rule whose formula cannot be evaluated was logged as a warning. That is
not an exceptional condition: a variable that is not numeric on this
particular server is ordinary — MariaDB reports concurrent_insert as AUTO
where MySQL reports an integer — and it happens on every run. The effect
was a warning written to nextcloud.log every time the page was opened,
which then inflated the "Errors in the log" setup check on the same
Overview page the database warning was added to. Verified on a MariaDB
instance: three runs used to add three lines, and now add none.

Nothing is lost by dropping the level. The rule already reports itself as
skipped with the reason attached, which is where an admin should read it;
the log line was only ever diagnostic detail. The apply-descriptor failure
next to it stays a warning, because that one means a rule is written wrong
rather than that the server is different.

The other problem was two event listeners registered with anonymous
handlers and never removed: visibilitychange in the live charts and
hashchange on the settings page. Both are now named and taken off on
unmount, which is what useLiveData already does.

Signed-off-by: Frank Karlitschek <frank.karlitschek@nextcloud.com>
Assisted-by: Claude Code:claude-opus-5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
Twenty of the fifty-seven ported checks are removed, leaving thirty-seven.

Nextcloud already reports missing indices, columns and primary keys on
Settings > Overview, so repeating them here only gave an admin two places
to read the same finding, and the transaction isolation check went with
them. The query cache and MyISAM groups describe engines and features that
a current Nextcloud does not use — the query cache was removed outright in
MySQL 8.0 — so on a supported server they either skip or advise tuning
something nothing touches. The rest are checks whose finding is not
actionable: how long the server has been up, whether it has served enough
queries to be worth analysing, slow_launch_time, skip_name_resolve, and the
two binary-log ones that only matter with replication configured.

Removing the schema-integrity rules pays for itself twice over. They were
the only consumers of the code that dispatches AddMissingIndicesEvent and
friends to every installed app and diffs the result against the live schema
through SchemaWrapper, so that work no longer runs on a page load, and the
app no longer reaches into OC\DB\SchemaWrapper -- one fewer private server
class for a bundled app to depend on. The psalm baseline shrinks with it.

It also removes the rule behind the MariaDB log noise fixed in the previous
commit, so that fix now has nothing left to suppress on MariaDB.

On a MariaDB test instance the page goes from 57 checks with 12 failing to
37 with 5, and the categories MyISAM, QueryCache, Security and Server
disappear entirely.

Signed-off-by: Frank Karlitschek <frank.karlitschek@nextcloud.com>
Assisted-by: Claude Code:claude-opus-5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
@kesselb
kesselb force-pushed the feat/database-checks branch from 453db0a to 1089cd4 Compare September 9, 2026 18:11
@nextcloud-command nextcloud-command added the AI assisted This PR contains AI-assisted commits label Sep 9, 2026
@kesselb
kesselb enabled auto-merge September 9, 2026 18:12
@kesselb
kesselb merged commit db69e5f into master Sep 9, 2026
44 checks passed
@kesselb
kesselb deleted the feat/database-checks branch September 9, 2026 18:15
@karlitschek

Copy link
Copy Markdown
Member Author

/backport to stable35

pull Bot pushed a commit to Mu-L/NextcloudServer that referenced this pull request Sep 9, 2026
The DB Doctor check engine now lives in serverinfo
(nextcloud/serverinfo#1123), so dbdoctor is no longer bundled with the
server tarball. Leaving it in shippedApps and defaultEnabled means the
server expects an app directory the release build no longer produces.

The release side is nextcloud-gmbh/release-script, branch
enh/unship-dbdoctor, and nextcloud-releases/server, branch
fix/unship-dbdoctor.

Assisted-by: ClaudeCode:claude-opus-5
Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
WSHAPER pushed a commit to WSHAPER/server that referenced this pull request Oct 6, 2026
The DB Doctor check engine now lives in serverinfo
(nextcloud/serverinfo#1123), so dbdoctor is no longer bundled with the
server tarball. Leaving it in shippedApps and defaultEnabled means the
server expects an app directory the release build no longer produces.

The release side is nextcloud-gmbh/release-script, branch
enh/unship-dbdoctor, and nextcloud-releases/server, branch
fix/unship-dbdoctor.

Assisted-by: ClaudeCode:claude-opus-5
Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI assisted This PR contains AI-assisted commits

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants