Support hybrid search with adaptive DLS

[sharp-pixel]

Description

• Category: Bug fix

Hybrid search requests can fail when they are executed by a user with Document-Level Security (DLS). Neural Search requires a HybridQuery to remain the top-level Lucene query, while the Security plugin can wrap the user query in BooleanQuery and ConstantScoreQuery objects when enforcing DLS. Neural Search then rejects the query or fails while casting the wrapper to HybridQuery.

The original report also observed inconsistent behavior: adding a suggestion could make an otherwise equivalent request succeed because it selected a different DLS path.

Old behavior

In adaptive DLS mode, a top-level hybrid query could be wrapped by the Security plugin. This changed the query shape expected by Neural Search and could cause every shard to fail with errors such as:

• hybrid query must be a top level query and cannot be wrapped into other queries
• BooleanQuery or ConstantScoreQuery cannot be cast to HybridQuery

New behavior

For supported top-level hybrid requests in adaptive mode, the Security plugin now:

• Detects the hybrid query without adding a compile-time dependency on Neural Search.
• Applies the DLS query through QueryBuilder.filter(...), preserving the hybrid query as the top-level query and propagating DLS into every hybrid subquery.
• Uses a separate internal marker so reader-level DLS remains active for global aggregations, suggestions, and other features that do not use the top-level query.
• Propagates that marker between nodes in the local cluster and removes it before cross-cluster search requests.
• Retains the existing star-tree safeguard before skipping query wrapping.

Non-hybrid queries and unsupported hybrid configurations continue through their existing DLS paths.

Supported scope

The special hybrid handling is enabled only when all of these conditions are true:

• DLS mode is adaptive.
• The request has DLS restrictions.
• The user query is a top-level hybrid query.
• The DLS rule does not contain a term-lookup query.
• The hybrid query contains no parent/child query clauses.
• The request targets only the local cluster.
• Every node in the cluster runs OpenSearch 3.9 or newer.

Within that scope, the fix supports:

• Single or multiple hybrid subqueries.
• Existing hybrid filters, which are composed with the DLS filter.
• DLS-filtered hits.
• Global aggregations protected by reader-level DLS.
• Searches containing suggestions.
• Multi-node local clusters.

Not supported by this change

The special handling is intentionally not enabled for:

• Explicit filter_level DLS mode.
• Explicit lucene_level DLS mode.
• Adaptive mode when the DLS rule contains a term-lookup query.
• Hybrid queries nested beneath another query.
• Hybrid queries containing parent/child clauses.
• Cross-cluster search.
• Mixed-version clusters containing a node older than OpenSearch 3.9.

These cases retain their existing DLS behavior and are not claimed to be compatible with hybrid search by this PR.

This change introduces no new setting, permission, REST API, or dependency on the Neural Search plugin.

Issues Resolved

Addresses:

• [BUG] Hybrid Search not working with Document Level Security neural-search#1303
• Original transferred issue: https://github.com/opensearch-project/ml-commons/issues/3815
• Original community report: https://forum.opensearch.org/t/hybrid-search-not-working-with-document-level-security/24239

Related prior work:

• Handle a hybrid query extended with DLS rules by the security plugin neural-search#1432

Neural Search PR #1432 added support for recognizing and reconstructing the Security DLS wrapper shape available at the time. This PR complements that work on the Security side by avoiding the outer wrapper for supported hybrid requests.

This is not a backport.

These changes introduce no new permissions and require no Security Dashboards companion PR.

Testing

Security plugin unit tests

Added focused coverage for:

• Adaptive-mode selection and the DLS, version, and local-cluster gates.
• Requests without a source or query, non-search requests, and non-hybrid queries.
• Nested hybrid queries, parent/child clauses, and term-lookup DLS rules.
• Applying DLS through the hybrid filter method and composing an existing hybrid filter.
• Preserving reader-level DLS and protecting global aggregations.
• Retaining the star-tree safeguard.
• Propagating the internal marker to another local node and removing it before a remote-cluster request.

Executed:

./gradlew test \
  --tests org.opensearch.security.configuration.DlsFlsFilterLeafReaderTest \
  --tests org.opensearch.security.configuration.DlsFilterLevelActionHandlerTest \
  --tests org.opensearch.security.configuration.DlsFlsValveImplTest \
  --tests org.opensearch.security.privileges.dlsfls.DlsFlsBaseContextTest \
  --tests org.opensearch.security.transport.SecurityInterceptorTests

Result: passed.

./gradlew :precommit

Result: passed.

The Security plugin bundle was also built successfully.

Cross-plugin integration testing

A companion Neural Search integration test was run against the locally built Security plugin:

./gradlew integTest \
  --tests org.opensearch.neuralsearch.query.HybridQueryDlsIT \
  -x spotlessApply \
  -Dsecurity.enabled=true \
  -Dsecurity.plugin.path=/absolute/path/to/opensearch-security.zip \
  -PnumNodes=3

Result: passed on a three-node cluster.

The test verifies the unrestricted administrator baseline, DLS-filtered hits, DLS-protected global aggregation counts and buckets, suggestions, composition with an existing hybrid filter, a single-clause hybrid query, no duplicate hits, and no failed shards.

Running the same test with the unmodified Security plugin reproduced the original top-level BooleanQuery/HybridQuery failure.

The companion integration coverage is available in opensearch-project/neural-search#1957. That draft depends on this Security PR and should merge only after this fix is available to the Neural Search integration-test build.

Check List

• New functionality includes testing
• New functionality has been documented — behavior and support boundaries are documented above and in code comments; no user-facing setting or API is introduced
• New Roles/Permissions have a corresponding security dashboards plugin PR — N/A, no roles or permissions were added
• API changes companion pull request created — N/A, no API changes were made
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 7d11976)

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 Security concerns Trust-by-name of hybrid query: The special DLS path is selected based solely on QueryBuilder.getName().equals("hybrid"). If any plugin registers a query builder under the name hybrid that does not correctly implement filter(...) (i.e., does not propagate the DLS filter to every subquery), DLS can be silently bypassed on the top-level query path. Reader-level DLS is preserved via a separate header, mitigating suggest/aggregation paths, but hit filtering relies entirely on the third-party builder's behavior.

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Duplicate parent/child check bypass In handle(SearchRequest, StoredContext), the parent/child check now runs only when query != null before calling applyFilterLevelDls. For hybrid queries, applyFilterLevelDls re-checks parent/child and throws OpenSearchSecurityException, but this exception is thrown synchronously to the caller instead of being reported through listener.onFailure(...) like the earlier check. This changes error handling semantics: a hybrid query with parent/child clauses will propagate as an unhandled exception up the call stack rather than a listener failure, potentially leaking the exception or producing inconsistent error responses versus the non-hybrid path. SearchSourceBuilder searchSource = getOrCreateSearchSource(searchRequest); QueryBuilder query = searchSource.query(); if (query != null) { if (ParentChildrenQueryDetector.hasParentOrChildQuery(query)) { listener.onFailure(new OpenSearchSecurityException("Unable to handle filter level DLS for parent or child queries")); return false; } } applyFilterLevelDls(searchSource, filterLevelQueryBuilder, applyDlsFilterToHybridQuery); Trusting query name for security decision isHybridQuery identifies the hybrid query solely by matching getName() against the string "hybrid". Any plugin registering a query builder under the name hybrid will trigger the hybrid DLS filter path, which relies on the builder honoring filter(...) by propagating the filter into every subquery and exposing subqueries via the visitor. If a third-party or maliciously registered hybrid builder does not honor this contract, DLS could be bypassed for those subqueries because reader-level DLS is intentionally skipped by the top-level query's filter. Consider a stronger identity check (e.g., verifying the builder class comes from the Neural Search plugin) or documenting/enforcing the contract more strictly. /** * Neural Search is an optional plugin, so Security identifies its hybrid query through the public query type name * instead of depending on its query builder class. {@link QueryBuilder#getName()} is OpenSearch's unique query type * identifier. A query builder registered as {@code hybrid} must honor {@link QueryBuilder#filter(QueryBuilder)} by * applying the supplied filter to every subquery and must expose every subquery through its visitor. Reader-level DLS * remains active whenever this special path is selected, independently of the query builder's filter implementation. */ static boolean isHybridQuery(QueryBuilder query) { return query != null && HYBRID_QUERY_NAME.equals(query.getName()); }

[github-actions]

PR Code Suggestions ✨

Latest suggestions up to 7d11976

Explore these optional code suggestions:

Category	Suggestion	Impact
Security	Guard against hybrid builders ignoring filter propagation The default QueryBuilder.filter(QueryBuilder) implementation on AbstractQueryBuilder typically returns this after mutating a filter list, but there is no contract guaranteeing hybrid query builders actually apply the filter to every subquery. If a plugin's hybrid query implementation silently ignores filter() (e.g., returns this unchanged), DLS would be bypassed. Consider verifying via a marker/visitor that the filter was propagated, or falling back to reader-level DLS when propagation cannot be confirmed. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [310-319] } else if (applyDlsFilterToHybridQuery && isHybridQuery(query)) { if (ParentChildrenQueryDetector.hasParentOrChildQuery(query)) { throw new OpenSearchSecurityException("Unable to handle filter level DLS for hybrid queries with parent or child clauses"); } - // Hybrid queries must remain top-level, so apply filter level DLS query directly + // Hybrid queries must remain top-level, so apply filter level DLS query directly. + // Reader-level DLS remains active as defense-in-depth in case the hybrid builder does not honor filter(). QueryBuilder filteredHybridQuery = query.filter(filterLevelQueryBuilder); if (filteredHybridQuery == null) { throw new OpenSearchSecurityException("Hybrid query returned no query after applying the DLS filter"); } searchSource.query(filteredHybridQuery); Suggestion importance[1-10]: 6 __ Why: Valid security concern: the code relies on hybrid query builders honoring filter(), and verifying propagation or documenting the reader-level DLS fallback would strengthen defense-in-depth. However, reader-level DLS is already preserved as noted in the PR, reducing the actual risk.	Low
	Assert reader-level DLS marker before early return Returning early here skips the parsed-query rewriting that adds DLS to the search context's query. Since reader-level DLS is preserved, this should be safe, but relying solely on reader-level filtering means any code path that consults the parsed query (e.g., certain aggregation optimizations or query rewrites that read searchContext.query() before reader filtering) may operate on the unfiltered query. Verify that all query-path optimizations are gated on reader-level DLS presence, or explicitly document/assert this invariant here. src/main/java/org/opensearch/security/configuration/DlsFlsValveImpl.java [561-568] if (dlsFlsBaseContext.isDlsQueryFilterApplied()) { - // The top-level hybrid filter already protects hits, so parsed-query rewriting is not needed here. - // DlsFlsFilterLeafReader sees OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED and still applies reader-level - // DLS to aggregations, suggestions, and other paths. This check intentionally follows the star-tree - // safeguard above. + // The top-level hybrid filter already protects hits; reader-level DLS remains active via + // OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED for aggregations/suggestions/etc. + assert threadContext.getHeader(ConfigConstants.OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED) != null; log.trace("handleSearchContext(): DLS is applied to the hybrid query; preserving reader-level DLS"); return; } Suggestion importance[1-10]: 3 __ Why: The suggestion mostly asks the author to verify invariants and optionally add an assertion. The improved_code change is minor and the existing comment already documents the invariant clearly.	Low
General	Verify no dependency on prior header content The debug-mode behavior that previously stored request.toString() as the header value has been removed, but the log message in DlsFlsValveImpl still uses this header value for diagnostic output ("DLS query handling is already done for: {}"). While the change removes potentially sensitive info (good), consider verifying no other code path depends on the previous non-"true" values, and update related debug logs to not imply meaningful content in the header value. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [179-184] +threadContext.putHeader( + applyDlsFilterToHybridQuery + ? ConfigConstants.OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED + : ConfigConstants.OPENDISTRO_SECURITY_FILTER_LEVEL_DLS_DONE, + "true" +); - Suggestion importance[1-10]: 2 __ Why: The existing_code and improved_code are identical; the suggestion only asks the author to verify behavior, providing no concrete code change.	Low

---

Previous suggestions

Suggestions up to commit 5497e5c

Category	Suggestion	Impact
Possible issue	Guard hybrid branch against parent/child clauses When applyDlsFilterToHybridQuery is true and the top-level query is hybrid, the code skips the parent/child check that was performed just before this call. If a hybrid query contains parent/child clauses (which the pre-check only rejects when the top-level query is directly a parent/child query), DLS enforcement semantics may be violated. Consider running ParentChildrenQueryDetector.hasParentOrChildQuery(query) inside the hybrid branch as well, and failing closed if detected. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [305-312] } else if (applyDlsFilterToHybridQuery && isHybridQuery(query)) { + if (ParentChildrenQueryDetector.hasParentOrChildQuery(query)) { + throw new OpenSearchSecurityException("Unable to handle filter level DLS for hybrid queries with parent or child clauses"); + } // Hybrid queries must remain top-level, so apply filter level DLS query directly QueryBuilder filteredHybridQuery = query.filter(filterLevelQueryBuilder); if (filteredHybridQuery == null) { throw new OpenSearchSecurityException("Hybrid query returned no query after applying the DLS filter"); } searchSource.query(filteredHybridQuery); } else { Suggestion importance[1-10]: 7 __ Why: Valid concern: the pre-check ParentChildrenQueryDetector.hasParentOrChildQuery(query) outside applyFilterLevelDls does check the top-level hybrid query, but adding a defense-in-depth check inside the hybrid branch improves safety. Moderate impact since the outer check already handles this case in the main handle path.	Medium
Security	Harden hybrid query identification against spoofing Identifying the hybrid query solely by the string "hybrid" returned from getName() is fragile: any plugin or user-defined query registered under the same name could bypass reader-level DLS via the hybrid path. Consider also checking that the query class comes from a trusted package (e.g., neuralsearch) or hardening the contract, so a foreign query cannot claim the hybrid name and evade wrapping. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [325-327] static boolean isHybridQuery(QueryBuilder query) { - return query != null && HYBRID_QUERY_NAME.equals(query.getName()); + if (query == null || !HYBRID_QUERY_NAME.equals(query.getName())) { + return false; + } + String className = query.getClass().getName(); + return className.startsWith("org.opensearch.neuralsearch."); } Suggestion importance[1-10]: 6 __ Why: Reasonable security concern about identifying queries by name string, though hardcoding a package prefix is also fragile. The point about a foreign query masquerading as hybrid and bypassing wrapping has merit but requires design consideration.	Low
General	Verify header scoping for stored context Setting the header inside threadContext.newStoredContext(true) means the header is scoped only to this stored context and will be restored (removed) when the context is closed. Any downstream re-entry that relies on OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED to short-circuit re-processing may not see it. Confirm that the header is intentionally scoped to the child search that is dispatched via nodeClient.search(...) and not required outside; otherwise, put it on the outer context. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [176-181] +// NOTE: header put here is scoped to the stored context; ensure this matches the intended lifetime threadContext.putHeader( applyDlsFilterToHybridQuery ? ConfigConstants.OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED : ConfigConstants.OPENDISTRO_SECURITY_FILTER_LEVEL_DLS_DONE, "true" ); Suggestion importance[1-10]: 4 __ Why: The suggestion only asks for verification of header scoping without a concrete change; the improved_code merely adds a comment. Limited actionable value.	Low

Suggestions up to commit 85aa9be

Category	Suggestion	Impact
Security	Guard against null result from hybrid filter The QueryBuilder.filter() method may return null or an unexpected type for some implementations, which would silently null out the search source's query and bypass DLS enforcement entirely. Validate the returned query is non-null before assigning it, and fall back to the safe wrapping behavior otherwise to avoid a security bypass. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [305-307] } else if (applyDlsFilterToHybridQuery && isHybridQuery(query)) { // Hybrid queries must remain top-level, so apply filter level DLS query directly - searchSource.query(query.filter(filterLevelQueryBuilder)); + QueryBuilder filteredHybridQuery = query.filter(filterLevelQueryBuilder); + if (filteredHybridQuery == null) { + throw new OpenSearchSecurityException("Failed to apply DLS filter to hybrid query"); + } + searchSource.query(filteredHybridQuery); } else { Suggestion importance[1-10]: 5 __ Why: The concern about filter() returning null and silently bypassing DLS has some validity as a defensive measure, though the standard QueryBuilder contract typically returns a non-null query. Moderate impact for security robustness.	Low
General	Clarify early-return does not bypass DLS The comment states "Reader-level DLS remains active" but the method returns early here, which actually skips the code path that would apply additional query modifications for the reader. Confirm the early return does not bypass DLS enforcement — the comment appears contradictory to the behavior, and if reader-level DLS was meant to remain the sole protection, returning before other setup could still leave a gap. src/main/java/org/opensearch/security/configuration/DlsFlsValveImpl.java [560-567] if (!dlsRestriction.isUnrestricted()) { if (dlsFlsBaseContext.isDlsQueryFilterApplied()) { - // The DLS filter is already present in every hybrid subquery. Reader-level DLS remains active to protect - // aggregations, suggestions, and other search features which do not use the top-level query. This check - // intentionally follows the star-tree safeguard above. + // DLS filter has been pushed into the hybrid query at coordinator level; reader-level DLS + // (DlsFlsFilterLeafReader) still enforces restrictions for aggregations/suggestions since + // OPENDISTRO_SECURITY_DLS_QUERY_FILTER_APPLIED forces shouldApplyDlsToReader to return true. log.trace("handleSearchContext(): DLS is applied to the hybrid query; preserving reader-level DLS"); return; } Suggestion importance[1-10]: 2 __ Why: This is essentially a comment clarification suggestion asking to verify behavior; the existing comment already conveys the intent, and no functional change is proposed.	Low

Suggestions up to commit 26cb026

Category	Suggestion	Impact
Possible issue	Guard against null from hybrid filter application The QueryBuilder.filter() contract does not guarantee that the returned builder is the same instance nor that the filter is retained on all implementations. If a hybrid query implementation returns null or a different builder, searchSource.query(null) would drop the query entirely. Guard against a null return to fail loudly rather than silently disabling DLS. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [290-293] } else if (applyDlsFilterToHybridQuery && isHybridQuery(query)) { // Hybrid queries must remain top-level, so apply filter level DLS query directly - searchSource.query(query.filter(filterLevelQueryBuilder)); + QueryBuilder filtered = query.filter(filterLevelQueryBuilder); + if (filtered == null) { + throw new OpenSearchSecurityException("Failed to apply DLS filter to hybrid query"); + } + searchSource.query(filtered); } else { Suggestion importance[1-10]: 4 __ Why: The concern about QueryBuilder.filter() potentially returning null is speculative; the standard contract returns a QueryBuilder. Adding a null check is defensive but of marginal value, and the improved code changes only add minor safety.	Low
General	Prevent repeated DLS filter application on retries When applyDlsFilterToHybridQuery is true and the hybrid query's filter() mutates the original query in-place, subsequent retries/executions of the same SearchRequest (e.g. via scroll or coordinator retry) will keep stacking DLS filters on the hybrid query. Consider defensively copying/cloning the source or asserting idempotency to avoid accumulating filters. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [239-247] QueryBuilder query = searchRequest.source().query(); if (query != null) { if (ParentChildrenQueryDetector.hasParentOrChildQuery(query)) { listener.onFailure(new OpenSearchSecurityException("Unable to handle filter level DLS for parent or child queries")); return false; } } +// The DLS_QUERY_FILTER_APPLIED header prevents re-entry, but ensure a request isn't mutated twice within the same context. applyFilterLevelDls(searchRequest.source(), filterLevelQueryBuilder, applyDlsFilterToHybridQuery); Suggestion importance[1-10]: 2 __ Why: The suggestion is vague and only adds a comment without making a functional change; the improved_code is essentially identical to existing_code, providing minimal value.	Low

Suggestions up to commit 0ba09e3

Category	Suggestion	Impact
Possible issue	Guard against null from hybrid filter call The isHybridQuery check relies on identifying a query by its name string "hybrid". If the top-level query is not actually a hybrid query but happens to be some other query type that shares this name (or if the flag is set inappropriately), the DLS filter would be pushed via query.filter() without wrapping, potentially failing to enforce DLS if the query implementation ignores or improperly applies the filter. Consider defensively verifying the query type more strictly, or falling back to the wrapping approach if query.filter() returns null. src/main/java/org/opensearch/security/configuration/DlsFilterLevelActionHandler.java [290-293] } else if (applyDlsFilterToHybridQuery && isHybridQuery(query)) { // Hybrid queries must remain top-level, so apply filter level DLS query directly - searchSource.query(query.filter(filterLevelQueryBuilder)); + QueryBuilder filtered = query.filter(filterLevelQueryBuilder); + if (filtered == null) { + filterLevelQueryBuilder.must(query); + searchSource.query(filterLevelQueryBuilder); + } else { + searchSource.query(filtered); + } } else { Suggestion importance[1-10]: 3 __ Why: The suggestion is speculative; QueryBuilder.filter() is not documented to return null and the test explicitly stubs it. The defensive fallback offers marginal robustness at best.	Low
General	Preserve original filter-level DLS trigger semantics shouldUseFilterLevelDlsInAdaptiveMode now returns true only when both hasDlsRestrictions and containsTermLookupQuery are true. Previously, doFilterLevelDls was set to containsTermLookupQuery regardless of hasDlsRestrictions. This change may skip filter-level DLS for TLQ cases without DLS restrictions on the current index but with restrictions elsewhere in the map — verify this is intended, since the outer code path only reaches here when hasDlsRestrictions is already true, but the map may contain restrictions for other indices. Confirm semantics are preserved. src/main/java/org/opensearch/security/configuration/DlsFlsValveImpl.java [267-275] boolean containsTermLookupQuery = dlsRestrictionMap.containsAny(DlsRestriction::containsTermLookupQuery); -doFilterLevelDls = shouldUseFilterLevelDlsInAdaptiveMode(hasDlsRestrictions, containsTermLookupQuery); +doFilterLevelDls = containsTermLookupQuery; applyDlsFilterToHybridQuery = shouldApplyDlsFilterToHybridQueryInAdaptiveMode( request, hasDlsRestrictions, containsTermLookupQuery, isHybridQueryDlsFilterSupported(clusterService.state().nodes().getMinNodeVersion()), isLocalOnlyRequest(resolved) ); Suggestion importance[1-10]: 2 __ Why: The suggestion itself acknowledges that the outer path already requires hasDlsRestrictions to be true, making the semantic change effectively a no-op. It's a verification request with low impact.	Low

[github-actions]

Persistent review updated to latest commit 26cb026

[github-actions]

Persistent review updated to latest commit 85aa9be

[github-actions]

Persistent review updated to latest commit 5497e5c

[github-actions]

Persistent review updated to latest commit 7d11976