Harden /webSearch query validation and reject regex operators

README.md

@@ -305,7 +305,11 @@ While enabled, `POST /webSearch` requires a non-empty JSON `query` string. It al
 - `from` must be an integer greater than or equal to `0`
 - `size` must be an integer from `1` to `500`
 
-The raw-query builder preserves the original expression and uses `query_string` with an `AND` default operator. Standalone calendar dates such as `2024-01-01` and ranges such as `2024-01-01..2024-01-31` search the supported record/container date fields using inclusive boundaries. Range endpoints can also use year or month precision (`2023..2024`, `2024-01..2024-03`); a standalone year or month remains a text query. Timestamps accept UTC or explicit offsets, and timestamps without a zone are interpreted as UTC. Date-query years are limited to 1900 through 2100. Quoted dates remain literal query expressions. This date syntax is specific to `/webSearch`; `/search` retains its explicit timestamp filters.
+The raw-query builder validates every token before using `query_string` with an `AND` default operator. Ordinary text (`remote code execution`), quoted phrases, Boolean expressions and parentheses (`apache AND (nginx OR "remote code")`), and `*`/`?` wildcard searches remain supported, including leading wildcards. Notes describe all distinct term types, not only the final token. Recognized URLs containing slashes and simple relative paths are quoted as literal text before execution; URL punctuation does not become a query operator. Other supported expressions retain their syntax.
+
+Regular-expression operators, including field-qualified regex expressions, are not supported. Unsupported tokens anywhere in the query, unescaped braces, unclosed quotes, unbalanced parentheses, dangling escapes, and reversed date ranges return HTTP `400` with `WEB_SEARCH_QUERY_SYNTAX` in `searchMetadata.errors`, without contacting OpenSearch. For example, `/fre[e]type/`, `/fre[e]type/ apache`, and `apache /fre[e]type/` are rejected. Quote or escape special characters when they are intended as literal text: the JSON query strings `"\"/fre[e]type/\""` and `"\\/fre[e]type\\/"` are accepted literals, not regex queries. Quote absolute paths, for example `"query": "\"/usr/local/bin\""`. Invalid body fields, sorting, and pagination retain their existing validation-error array format. These checks do not implement the complete OpenSearch query grammar or eliminate the cost of allowed wildcard searches; other backend query errors remain sanitized HTTP `502` responses.
+
+Standalone calendar dates such as `2024-01-01` and ranges such as `2024-01-01..2024-01-31` search the supported record/container date fields using inclusive boundaries. Range endpoints can also use year or month precision (`2023..2024`, `2024-01..2024-03`); a standalone year or month remains a text query. Timestamps accept UTC or explicit offsets, and timestamps without a zone are interpreted as UTC. Date-query years are limited to 1900 through 2100. Quoted dates remain literal query expressions. This date syntax is specific to `/webSearch`; `/search` retains its explicit timestamp filters.
 
 Both endpoints use the shared OpenSearch client and reject incomplete upstream responses. A response with failed shards, a timeout, missing hit data, or an invalid total-hit count cannot produce a successful `/webSearch` response. Backend failures return sanitized HTTP 502 responses; transport timeouts return HTTP 504.
 

api-docs/openapi.json

@@ -275,7 +275,7 @@
           "CVE Record"
         ],
         "summary": "Raw query search",
-        "description": "  <h2>Expected Behavior</h2>  <p>Retrieves matching CVE records using the raw query expression supplied in the JSON request body. Service operators can disable this endpoint through runtime configuration. Standalone dates and date ranges search the supported record/container date fields; other expressions use raw query syntax with an AND default operator.</p>",
+        "description": "  <h2>Expected Behavior</h2>  <p>Retrieves matching CVE records using the validated query expression supplied in the JSON request body. Service operators can disable this endpoint through runtime configuration. Standalone dates and date ranges search the supported record/container date fields; other expressions use query syntax with an AND default operator. Every token is validated before execution. Regular expressions are not supported.</p>",
         "operationId": "webSearch",
         "responses": {
           "200": {
@@ -297,11 +297,18 @@
             }
           },
           "400": {
-            "description": "Malformed JSON, unreadable request body, or invalid raw-query request body, sort, or pagination",
+            "description": "Malformed JSON, unreadable request body, invalid request fields, sort or pagination, or unsupported query syntax. WEB_SEARCH_QUERY_SYNTAX uses the WebSearchResponse envelope with searchMetadata.errors; request validation retains its existing error format.",
             "content": {
               "application/json": {
                 "schema": {
-                  "$ref": "#/components/schemas/ApiErrorResponse"
+                  "oneOf": [
+                    {
+                      "$ref": "#/components/schemas/ApiErrorResponse"
+                    },
+                    {
+                      "$ref": "#/components/schemas/WebSearchResponse"
+                    }
+                  ]
                 }
               }
             }
@@ -380,7 +387,7 @@
                 "properties": {
                   "query": {
                     "type": "string",
-                    "description": "Raw search expression. Examples: remote code execution, apache*, 2024-01-01, or 2024-01-01..2024-01-31. Date ranges are inclusive and allow year/month endpoints; standalone years/months and quoted dates remain text queries. Date-query years must be between 1900 and 2100.",
+                    "description": "Search expression. Supports ordinary multi-term text, quoted phrases, Boolean expressions with parentheses, and * or ? wildcards, including leading wildcards. Recognized URLs containing slashes and simple relative paths are treated as literal text. Quote or escape literal special characters, including absolute paths. Regex expressions (including field-qualified regex), unsupported tokens in any position, unescaped braces, unclosed quotes, unbalanced parentheses, dangling escapes, and reversed date ranges return HTTP 400 WEB_SEARCH_QUERY_SYNTAX in searchMetadata.errors before backend execution. These checks do not implement the complete OpenSearch query grammar or bound the cost of allowed wildcard searches. Examples: remote code execution, apache*, 2024-01-01, or 2024-01-01..2024-01-31. Date ranges are inclusive and allow year/month endpoints; standalone years/months and quoted dates remain text queries. Date-query years must be between 1900 and 2100.",
                     "minLength": 1
                   },
                   "sort": {

api-docs/openapi.test.js

@@ -159,12 +159,20 @@ describe('OpenAPI contract', () => {
         const schema = operation.requestBody.content['application/json'].schema;
 
         expect(operation.deprecated).toBeUndefined();
-        expect(operation.description).toContain('raw query expression');
+        expect(operation.description).toContain('validated query expression');
         expect(operation.parameters || []).toStrictEqual([]);
         expect(operation.requestBody.required).toBe(true);
         expect(schema.required).toStrictEqual(['query']);
         expect(schema.additionalProperties).toBe(false);
         expect(schema.properties.query.type).toBe('string');
+        expect(schema.properties.query.description).toContain('WEB_SEARCH_QUERY_SYNTAX');
+        expect(schema.properties.query.description).toContain('unsupported tokens in any position');
+        expect(schema.properties.query.description).toContain('including leading wildcards');
+        expect(schema.properties.query.description).toContain('treated as literal text');
+        expect(operation.responses['400'].content['application/json'].schema.oneOf).toStrictEqual([
+            { $ref: '#/components/schemas/ApiErrorResponse' },
+            { $ref: '#/components/schemas/WebSearchResponse' }
+        ]);
         expect(schema.properties.sort.additionalProperties).toBe(false);
         expect(schema.properties.sort.properties.property.enum).toStrictEqual(['cveId', 'relevancy']);
         expect(schema.properties.size.maximum).toBe(constants.MAX_RESULTS_LIMIT);

controllers/searchController.js

@@ -230,6 +230,11 @@ async function webSearchHandler(req, res, next) {
         body.data = searchData.hits.hits;
         return res.status(200).json(body);
     } catch (ex) {
+        if (ex instanceof webSearchQueryBuilder.WebSearchQueryError) {
+            body.searchMetadata.errors = [error.webSearchQuerySyntaxError()];
+            return res.status(400).json(body);
+        }
+
         const timedOut = searchDeadline.isSearchTimeoutError(ex);
         const err = timedOut ? error.searchTimeoutError() : error.upstreamSearchError();
         body.searchMetadata.errors = [err];

controllers/searchController.test.js

@@ -1403,6 +1403,8 @@ describe('Search controller WEB_SEARCH handler', () => {
 
     test.each([
         [new Error('sensitive backend details'), 502, 'UPSTREAM_SEARCH_ERROR'],
+        [{ name: 'WebSearchQueryError', message: 'sensitive backend details' }, 502, 'UPSTREAM_SEARCH_ERROR'],
+        [{ statusCode: 400, message: 'sensitive backend parse error' }, 502, 'UPSTREAM_SEARCH_ERROR'],
         [{ name: 'TimeoutError', message: 'sensitive timeout details' }, 504, 'SEARCH_TIMEOUT']
     ])('WEB_SEARCH sanitizes transport errors: %p', async (err, status, code) => {
         mockApiSearch.mockRejectedValue(err);
@@ -1417,10 +1419,29 @@ describe('Search controller WEB_SEARCH handler', () => {
         expect(console.log).not.toHaveBeenCalled();
     });
 
-    test('WEB_SEARCH keeps unsupported expressions sanitized without contacting OpenSearch', async () => {
-        await controller.WEB_SEARCH({ body: { query: '{invalid}' } }, res, next);
-        expect(res.status).toHaveBeenCalledWith(502);
+    test.each([
+        '{invalid}',
+        '{invalid} apache',
+        'apache {invalid}',
+        '/fre[e]type/',
+        '/fre[e]type/ apache',
+        'apache /fre[e]type/',
+        'containers.cna.descriptions.value:/fre[e]type/ apache',
+        '2024-02-01..2024-01-01',
+        '"unterminated',
+        '(apache OR nginx'
+    ])('WEB_SEARCH rejects unsupported expressions before contacting OpenSearch: %s', async (query) => {
+        await controller.WEB_SEARCH({ body: { query } }, res, next);
+        expect(res.status).toHaveBeenCalledWith(400);
         expect(mockSearchOpenSearch).not.toHaveBeenCalled();
-        expect(res.json.mock.calls[0][0].searchMetadata.errors[0].error).toBe('UPSTREAM_SEARCH_ERROR');
+        expect(res.json.mock.calls[0][0]).toMatchObject({
+            searchMetadata: {
+                searchStatus: 'errors',
+                notes: [],
+                errors: [{ error: 'WEB_SEARCH_QUERY_SYNTAX' }]
+            },
+            resultsTotal: 0,
+            data: []
+        });
     });
 });

routes/index.js

@@ -485,7 +485,7 @@ router.post(
 #swagger.summary = "Raw query search"
 #swagger.description = "
     <h2>Expected Behavior</h2>
-    <p>Retrieves matching CVE records using the raw query expression supplied in the JSON request body. Service operators can disable this endpoint through runtime configuration. Standalone dates and date ranges search the supported record/container date fields; other expressions use raw query syntax with an AND default operator.</p>
+    <p>Retrieves matching CVE records using the validated query expression supplied in the JSON request body. Service operators can disable this endpoint through runtime configuration. Standalone dates and date ranges search the supported record/container date fields; other expressions use query syntax with an AND default operator. Every token is validated before execution. Regular expressions are not supported.</p>
 #swagger.requestBody = {
     required: true,
     content: {
@@ -497,7 +497,7 @@ router.post(
                 properties: {
                     query: {
                         type: "string",
-                        description: "Raw search expression. Examples: remote code execution, apache*, 2024-01-01, or 2024-01-01..2024-01-31. Date ranges are inclusive and allow year/month endpoints; standalone years/months and quoted dates remain text queries. Date-query years must be between 1900 and 2100.",
+                        description: "Search expression. Supports ordinary multi-term text, quoted phrases, Boolean expressions with parentheses, and * or ? wildcards, including leading wildcards. Recognized URLs containing slashes and simple relative paths are treated as literal text. Quote or escape literal special characters, including absolute paths. Regex expressions (including field-qualified regex), unsupported tokens in any position, unescaped braces, unclosed quotes, unbalanced parentheses, dangling escapes, and reversed date ranges return HTTP 400 WEB_SEARCH_QUERY_SYNTAX in searchMetadata.errors before backend execution. These checks do not implement the complete OpenSearch query grammar or bound the cost of allowed wildcard searches. Examples: remote code execution, apache*, 2024-01-01, or 2024-01-01..2024-01-31. Date ranges are inclusive and allow year/month endpoints; standalone years/months and quoted dates remain text queries. Date-query years must be between 1900 and 2100.",
                         minLength: 1
                     },
                     sort: {
@@ -548,11 +548,14 @@ headers: {
 }
 }
 #swagger.responses[400] = {
-description: 'Malformed JSON, unreadable request body, or invalid raw-query request body, sort, or pagination',
+description: 'Malformed JSON, unreadable request body, invalid request fields, sort or pagination, or unsupported query syntax. WEB_SEARCH_QUERY_SYNTAX uses the WebSearchResponse envelope with searchMetadata.errors; request validation retains its existing error format.',
 content: {
     "application/json": {
         schema: {
-            $ref: "#/components/schemas/ApiErrorResponse"
+            oneOf: [
+                { $ref: "#/components/schemas/ApiErrorResponse" },
+                { $ref: "#/components/schemas/WebSearchResponse" }
+            ]
         }
     }
 },

routes/search.integration.test.js

@@ -555,6 +555,51 @@ describe('search route integration', () => {
         }
     );
 
+    test.each([
+        '/fre[e]type/',
+        '/fre[e]type/ apache',
+        'apache /fre[e]type/',
+        '(apache OR /fre[e]type/)',
+        'containers.cna.descriptions.value:/fre[e]type/ apache',
+        '{invalid} apache',
+        'apache {invalid}',
+        'field:value apache',
+        '"unterminated',
+        '(apache OR nginx',
+        '2024-02-01..2024-01-01'
+    ])('POST /webSearch rejects unsupported syntax before searching: %s', async (query) => {
+        const response = await request(createApp(), { method: 'POST', path: '/webSearch', body: { query } });
+
+        expect(response.status).toBe(400);
+        expect(response.body).toMatchObject({
+            searchMetadata: { searchStatus: 'errors', errors: [{ error: 'WEB_SEARCH_QUERY_SYNTAX' }] },
+            resultsTotal: 0,
+            data: []
+        });
+        expect(mockSearchOpenSearch).not.toHaveBeenCalled();
+    });
+
+    test.each([
+        ['remote code execution', 'remote code execution'],
+        ['apache*', 'apache*'],
+        ['*etype', '*etype'],
+        ['ap?che', 'ap?che'],
+        ['apache AND (nginx OR "remote code")', 'apache AND (nginx OR "remote code")'],
+        ['"/fre[e]type/"', '"/fre[e]type/"'],
+        ['\\/fre[e]type\\/', '\\/fre[e]type\\/'],
+        ['https://example.com/a?b=1', '"https://example.com/a?b=1"']
+    ])('POST /webSearch preserves supported query behavior: %s', async (query, executedQuery) => {
+        mockSearchOpenSearch.mockResolvedValue(advancedResponse(0).data);
+        const response = await request(createApp(), { method: 'POST', path: '/webSearch', body: { query } });
+
+        expect(response.status).toBe(200);
+        expect(mockSearchOpenSearch).toHaveBeenCalledWith(
+            expect.objectContaining({
+                query: { query_string: { query: executedQuery, default_operator: 'AND' } }
+            })
+        );
+    });
+
     test('POST /webSearch normalizes fields, builds DSL, and formats OpenSearch responses', async () => {
         mockSearchOpenSearch.mockResolvedValue(advancedResponse(1, [{ _id: 'CVE-2024-0002' }]).data);
 

utils/errors.js

@@ -337,6 +337,13 @@ class errors {
         return makeError('WEB_SEARCH_QUERY_REQUIRED', 'The query field is required and must be a non-empty string.');
     }
 
+    webSearchQuerySyntaxError() {
+        return makeError(
+            'WEB_SEARCH_QUERY_SYNTAX',
+            'The query contains unsupported or malformed syntax. Regular expressions and unescaped braces are not supported. Quote or escape literal special characters.'
+        );
+    }
+
     webSearchSortPropertyError() {
         return makeError('WEB_SEARCH_SORT_PROPERTY', 'The sort.property field must be cveId or relevancy.');
     }

utils/errors.test.js

@@ -323,6 +323,15 @@ describe('error response helpers', () => {
                 message: 'The query field is required and must be a non-empty string.'
             }
         ],
+        [
+            'webSearchQuerySyntaxError',
+            [],
+            {
+                error: 'WEB_SEARCH_QUERY_SYNTAX',
+                message:
+                    'The query contains unsupported or malformed syntax. Regular expressions and unescaped braces are not supported. Quote or escape literal special characters.'
+            }
+        ],
         [
             'webSearchSortPropertyError',
             [],