Skip to content

Use Lucene search engine for new RavenDB databases and flag indexes still on Corax - #5833

Merged
ramonsmits merged 4 commits into
masterfrom
ramon/lucene-for-new-databases
Sep 3, 2026
Merged

ramonsmits merged 4 commits into
masterfrom
ramon/lucene-for-new-databases

Conversation

@ramonsmits

@ramonsmits ramonsmits commented Aug 28, 2026 •

Copy link
Copy Markdown
Member

Summary

Load testing showed that Lucene indexes are smaller, use less RAM and are faster than Corax for the index definitions ServiceControl uses. This PR moves new databases to Lucene while leaving existing databases untouched, and makes it visible to operators when their indexes are still on Corax.

Switching an existing database is deliberately not done automatically: changing the search engine triggers a full rebuild of all indexes, which on very large databases can take days depending on available compute and degrades ingestion/indexing rates while it runs. Migration remains an operator decision (docs guidance to follow).

Note

The default RavenDB full-text search engine changes from Corax to Lucene to resolve performance and stability issues on large database instances. Existing indexes are not automatically updated, as an index rebuild can take significant time and can severely impact compute and storage utilization while it runs. For more information, see the 6.19 to 6.20 upgrade guide.

Changes

  • New databases are created with Indexing.Static/Auto.SearchEngineType=Lucene (Primary main + throughput databases, Audit). UpdateDatabaseSettings is unchanged, so existing databases keep whatever engine they currently have configured.
  • Startup WARN listing any index still using Corax (StartupChecks.WarnIfIndexesUseCorax, shared by both persisters).
  • Custom check Error Database Search Engine / Audit Database Search Engine surfacing the same information in ServicePulse (hourly).
  • Tests: default engine for a fresh database is Lucene; per-index override is still reset on setup; Corax detection is covered; custom check approval files updated.

New databases are created with Indexing.Static/Auto.SearchEngineType=Lucene:
Lucene indexes are smaller, use less memory and are faster for the index
definitions ServiceControl uses. Existing databases keep the engine they
currently have configured, so no full index rebuild is triggered on upgrade.
New databases use Lucene, existing databases keep their configured search
engine. Operators of existing databases now get a WARN at startup listing the
indexes still on Corax so they can plan the transition to Lucene.
Surfaces the same information as the startup warning in ServicePulse via a
custom check on both the Primary (main + throughput database) and Audit
instances, so operators see it without inspecting logs.
@ramonsmits
ramonsmits marked this pull request as ready for review September 2, 2026 13:43
@ramonsmits
ramonsmits merged commit ef3ef7a into master Sep 3, 2026
68 of 70 checks passed
@ramonsmits
ramonsmits deleted the ramon/lucene-for-new-databases branch September 3, 2026 07:32
@ramonsmits ramonsmits added this to the 6.20.0 milestone Sep 3, 2026
ramonsmits added a commit that referenced this pull request Sep 3, 2026
PR #5840 introduced InternalCustomCheckClassification and PR #5833
added the Error/Audit Database Search Engine custom checks. Both
merged independently, so the new checks were never registered and the
CustomCheckTests.VerifyCustomChecks approval tests fail on master
with "MISSING FROM InternalCustomCheckClassification".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants