docs(search): correct the 15.8 rank fusion page against the implementation - #474
Merged
Conversation
…ation Verified ja/15.8/config/rank-fusion.rst against RankFusionProcessor, SemanticChunkSearcher, FessConfigImpl, SystemHelper, fess_rankfusion.xml and fess_config.properties, then propagated the result to the other six languages. Corrections: - The troubleshooting sections recommended lowering rank.fusion.window_size to 100. init() clamps the value to paging.search.page.max.size * 2, which is 200 by default -- the same as window_size's own default. The suggested value was therefore silently raised back to 200. The page now states that paging.search.page.max.size has to be lowered first, and shows both keys together. - rank.fusion.threads is computed as availableProcessors * 3 / 2 + 1 with integer arithmetic, not "cores x 1.5 + 1"; it sizes a fixed thread pool. - content_chunker.search.enabled is a system property (conf/system.properties or -Dfess.system.<key>), not a fess_config.properties key, and it is only evaluated at startup. - rank.fusion.searchers is read with a raw System.getProperty, so the -Dfess.config. / -Dfess.system. prefixed forms do not work. The fess.in.sh snippet now uses the FESS_JAVA_OPTS form. It can also be set from the "System Property" field under System > General, which promotes arbitrary keys via System.setProperty. Added: - Verifying fusion results: the searcher and rf_score fields, that both are runtime-only and absent from the index, and that seeing them in /api/v2/search requires query.additional.api.response.fields. - Impact on hit counts: fusion adds the documents only the non-main searchers found to the reported total, and skips that correction when the count is a lower bound. - Behavior when a searcher fails: a failing searcher degrades to an empty result with a WARN and fusion continues, except for InvalidQueryException and ResultOffsetExceededException, and except on deep pages, where the fallback path has no exception handling. There is no per-searcher timeout. - All four rank.fusion.* settings require a restart, because FessConfigImpl memoizes fess_config.properties values for the lifetime of the JVM. - RRF is fixed, with no algorithm switch and no per-searcher weighting. - Troubleshooting now starts from the searcher field and the semantic-skip conditions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reviewed
15.8/config/rank-fusion.rstagainst the implementation (RankFusionProcessor,SemanticChunkSearcher,FessConfigImpl,SystemHelper,fess_rankfusion.xml,fess_config.properties) and corrected it, then propagated the result to all seven languages.The page's own reference table already documented the
window_sizelower bound correctly, but the troubleshooting section then told readers to set a value that violates it — so the most actionable instruction on the page silently did nothing. That is the main fix here.Corrections
rank.fusion.window_sizerank.fusion.window_size=100as the exampleinit()raises the value topaging.search.page.max.size * 2(200 by default) and logs a WARN. Since 200 is alsowindow_size's own default, the default configuration cannot lower it at all. The page now showspaging.search.page.max.sizeandrank.fusion.window_sizeset together, and states the page-size trade-off.rank.fusion.threadsavailableProcessors × 1.5 + 1availableProcessors() * 3 / 2 + 1, integer arithmetic (5 cores gives 8, not 8.5), sizing a fixed thread pool.content_chunker.search.enabledfess_config.propertiesgetSystemProperty, so it comes fromconf/system.propertiesor-Dfess.system.<key>. Writing it infess_config.propertieshas no effect, and it is only evaluated at startup.rank.fusion.searchersfess_config.properties"System.getProperty, so the-Dfess.config./-Dfess.system.prefixed forms do not work either. Thefess.in.shsnippet now uses theFESS_JAVA_OPTSform instead of a bare-Dline, and the page notes that the "System Property" field under System > General can set it too, sinceSystemHelperpromotes arbitrary keys from it viaSystem.setProperty.Additions
These all exist in the code but were undocumented:
searcherandrf_scorefields, that both are added at search time and never stored in the index, and that seeing them in/api/v2/searchrequiresquery.additional.api.response.fields=rf_score,searcherplus a restart. Without this there was no documented way to tell whether hybrid search was actually running, which is what the troubleshooting section needs first.InvalidQueryExceptionandResultOffsetExceededException, and except on deep pages, where the fallback path calls the main searcher without exception handling. There is no per-searcher timeout.rank.fusion.*settings require a restart, becauseFessConfigImplmemoizesfess_config.propertiesvalues for the lifetime of the JVM. Being read per request is not the same as being reloadable.rank.fusion.rank_constantis the only ranking knob.searcherfield and the semantic-skip conditions (cross-referencingsearch-semantic), and the deep-page boundary is stated concretely as "from the 101st result onward".Verification
search-semantic.rst(all languages) andproperties.rstdo not contradict any of the corrected statements.