diff --git a/README.md b/README.md index 6201913..a936aed 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,31 @@ An AWS load balancer can use `/api/ready-check` to stop routing search traffic t ## API Input Validation -`POST /search` accepts an optional JSON object request body and validates supported fields, boolean parameters, pagination values, CVSS severity values, required parameter combinations, CPE Name requirements, and date-time formats. A non-empty body must use the `application/json` media type with UTF-8 encoding; unsupported media types or encodings return HTTP 415, and malformed JSON returns HTTP 400. If OpenSearch is unavailable, the endpoint returns HTTP 503 with a `Retry-After` header. +`POST /search` accepts an optional JSON object request body and validates supported fields, boolean parameters, pagination values, CVSS severity values, required parameter combinations, CPE Name requirements, and date-time formats. Enum-valued fields are accepted case-insensitively and normalized to their documented canonical values. A non-empty body must use the `application/json` media type with UTF-8 encoding; unsupported media types or encodings return HTTP 415, and malformed JSON returns HTTP 400. If OpenSearch is unavailable, the endpoint returns HTTP 503 with a `Retry-After` header. + +Use the optional `scope` field to control which parts of each record participate in matching. Values are case-insensitive: `FULL` (the default) searches base record fields, CNA data, and every ADP provider; `CNA_ONLY` searches base record fields and CNA data; and `ADP_ONLY` searches base record fields and every ADP provider. Scope changes matching only, so successful responses always contain the complete CVE record. CPE applicability filters use CNA data and return HTTP 400 when combined with `ADP_ONLY`. + +Scope applies to filter matching as follows: + +| Filters | Data used for matching | +| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | +| `cveIds`, `pubStartDate`, `pubEndDate`, `lastModStartDate`, `lastModEndDate` | Base record metadata in every scope | +| `noRejected` | Base record state in every scope, plus the legacy CNA state when CNA data is selected | +| `keywordSearch`, `keywordExactMatch` | Description text in the selected CNA and/or ADP containers | +| `cweId`, all CVSS severity and metric filters | Problem types or metrics in the selected CNA and/or ADP containers | +| All SSVC filters, `knownExploited`, KEV date filters, `referenceURL`, `disputed` | Structured metrics, references, or tags in the selected CNA and/or ADP containers | +| `cpeName`, `isVulnerable`, `virtualMatchString`, and version range filters | CNA CPE applicability data; incompatible with `ADP_ONLY` | +| `resultsPerPage`, `pageNumber` | Response controls; they do not select record data | + +If the selected containers do not contain the requested data, the request is valid and returns zero matches. The API does not remove unselected containers from a returned record. + +SSVC metrics can be filtered with `ssvcExploitation` (`NONE`, `POC`, or `ACTIVE`), `ssvcAutomatable` (`YES` or `NO`), `ssvcTechnicalImpact` (`PARTIAL` or `TOTAL`), `ssvcRole`, `ssvcTimestampStartDate`, and `ssvcTimestampEndDate`. Enum and role comparisons are case-insensitive, timestamps use the API's full UTC timestamp format with inclusive boundaries, and all supplied SSVC conditions must match the same structured SSVC metric entry in a container selected by `scope`. + +Use `knownExploited` to include or exclude records with a structured KEV metric in containers selected by `scope`. The optional `kevDateAddedStartDate` and `kevDateAddedEndDate` boundaries are inclusive, require `knownExploited: true`, and use the API's full UTC timestamp format. Stored KEV `dateAdded` values are interpreted as midnight UTC. + +Use `referenceURL` for a case-insensitive literal substring search over reference URLs in containers selected by `scope`. The filter accepts one string; user-supplied wildcard syntax is treated literally. + +Use the strict boolean `disputed` filter to require (`true`) or exclude (`false`) the exact `disputed` tag in containers selected by `scope`. This does not provide a general-purpose record tag filter. String inputs are bounded before query construction: @@ -107,6 +131,8 @@ String inputs are bounded before query construction: | `cpeName` | 2,048 characters | | `virtualMatchString` | 2,048 characters | | `versionStart`, `versionEnd` | 256 characters each | +| `ssvcRole` | 256 characters | +| `referenceURL` | 2,048 characters | | CVSS metric fields | 256 characters, 32 slash-delimited tokens, and 32 characters per token | Requests that exceed a limit return HTTP 400 before contacting OpenSearch. These bounds limit input size and query complexity without changing the documented CPE matching semantics. @@ -216,13 +242,15 @@ While enabled, `POST /webSearch` requires a non-empty JSON `query` string. It al ## Swagger API docs -Swagger API docs are automatically generated on `docker compose` or `npm run dev`, and can be regenerated manually using: +Swagger API docs are regenerated by `npm run dev` and can be regenerated manually using: ```bash npm run swagger-autogen ``` -The Swagger API docs will be served at `localhost:3000/api-docs` while the CVE Search API is running via Docker or npm. +Docker builds copy the committed `api-docs/openapi.json` into the runtime image; they do not regenerate it. Regenerate and commit that file whenever the route or request schema changes. + +The Swagger API docs will be served at `http://localhost:3000/api-docs` while the CVE Search API is running via Docker or npm with the default port. ## Testing @@ -261,6 +289,10 @@ The fallback limiter uses one in-memory counter per Node.js process. Set `Search This is a process-wide safety limit, not a precise distributed quota. Each ECS task has an independent counter, so total permitted traffic scales with the number of tasks and load-balancer routing. It deliberately does not use caller IP addresses or forwarded headers until the ALB proxy chain and Express `trust proxy` setting are confirmed. AWS WAF or another infrastructure-level distributed limit remains the preferred public-facing control. +### Semantic Post-Filtering + +SSVC filters and date-bounded positive `knownExploited` searches use semantic post-filtering because OpenSearch ordinary-object arrays cannot guarantee that multiple conditions matched the same structured metric entry. These searches retrieve up to 10,000 lightweight candidates, apply same-entry checks in Node.js, and fetch the requested page as complete CVE records. The reported total and pagination are complete only within that bounded candidate set; qualifying records beyond the first 10,000 OpenSearch candidates may not be represented. This preserves the existing bounded post-filter behavior until PIT-based pagination is implemented. SSVC and KEV candidates do not use the temporary CPE cache or cold-search queue. A `knownExploited` filter without date boundaries uses a direct OpenSearch query and retains ordinary exact-total behavior. + ### CPE Post-Filter Cache Searches that use `cpeName`, `virtualMatchString`, or `virtualMatchString` with version range parameters use a semantic CPE post-filter. The API first asks OpenSearch for up to 10,000 lightweight candidate records, filters those candidates in Node.js, then fetches the requested page of full CVE records by CVE ID. diff --git a/api-docs/openapi.json b/api-docs/openapi.json index 61279e4..cd18a9d 100644 --- a/api-docs/openapi.json +++ b/api-docs/openapi.json @@ -93,6 +93,7 @@ "allParameters": { "summary": "All supported request body parameters", "value": { + "scope": "FULL", "keywordSearch": "remote code execution", "keywordExactMatch": true, "cveIds": "CVE-2024-1234,CVE-2024-5678", @@ -120,7 +121,18 @@ "versionStartType": "including", "versionEnd": "2.6", "versionEndType": "excluding", - "virtualMatchString": "cpe:2.3:o:linux:linux_kernel" + "virtualMatchString": "cpe:2.3:o:linux:linux_kernel", + "ssvcExploitation": "ACTIVE", + "ssvcAutomatable": "YES", + "ssvcTechnicalImpact": "TOTAL", + "ssvcRole": "CISA Coordinator", + "ssvcTimestampStartDate": "2024-01-01T00:00:00.000Z", + "ssvcTimestampEndDate": "2024-12-31T23:59:59.999Z", + "knownExploited": true, + "kevDateAddedStartDate": "2024-01-01T00:00:00.000Z", + "kevDateAddedEndDate": "2024-12-31T23:59:59.999Z", + "referenceURL": "vendor.example/security/advisory", + "disputed": false } }, "keywordSearch": { @@ -298,14 +310,24 @@ "schemas": { "SearchRequest": { "type": "object", - "description": "Optional filters and pagination controls for a CVE record search.", + "description": "Optional filters and pagination controls for a CVE record search. Enum-valued fields are accepted case-insensitively and normalized to the documented canonical values.", "additionalProperties": false, "properties": { + "scope": { + "type": "string", + "enum": [ + "FULL", + "CNA_ONLY", + "ADP_ONLY" + ], + "default": "FULL", + "description": "Controls which containers participate in filter matching. FULL searches base record fields, CNA data, and all ADP providers. CNA_ONLY searches base record fields and CNA data. ADP_ONLY searches base record fields and all ADP providers. Scope does not remove data from returned records. Values are case-insensitive. CPE applicability filters are not available with ADP_ONLY." + }, "keywordSearch": { "type": "string", "maxLength": 512, "example": "remote code execution", - "description": "Returns CVE records where a word or phrase is included in the record's description. Multiple keywords operate like an AND statement, returning records whose descriptions include every keyword. Limited to 512 characters and 64 whitespace-delimited terms." + "description": "Returns CVE records where a word or phrase is included in a description selected by scope. Multiple keywords operate like an AND statement, returning records whose descriptions include every keyword. Limited to 512 characters and 64 whitespace-delimited terms." }, "keywordExactMatch": { "type": "boolean", @@ -327,7 +349,7 @@ "type": "string", "maxLength": 32, "example": "CWE-79", - "description": "Returns CVE records that include the provided Common Weakness Enumeration ID." + "description": "Returns CVE records whose scope-selected containers include the provided Common Weakness Enumeration ID." }, "cvssV2Severity": { "type": "string", @@ -336,7 +358,7 @@ "MEDIUM", "HIGH" ], - "description": "Returns CVE records that contain the provided CVSS v2 qualitative severity rating." + "description": "Returns CVE records with a scope-selected CVSS v2 base score in the range represented by the provided qualitative severity. Values are case-insensitive." }, "cvssV3Severity": { "type": "string", @@ -346,7 +368,7 @@ "HIGH", "CRITICAL" ], - "description": "Returns CVE records that contain the provided CVSS v3.0 or v3.1 qualitative severity rating." + "description": "Returns CVE records with the provided qualitative severity in a scope-selected CVSS v3.0 or v3.1 metric. Values are case-insensitive." }, "cvssV3_0Severity": { "type": "string", @@ -356,7 +378,7 @@ "HIGH", "CRITICAL" ], - "description": "Returns CVE records that contain the provided CVSS v3.0 qualitative severity rating." + "description": "Returns CVE records with the provided qualitative severity in a scope-selected CVSS v3.0 metric. Values are case-insensitive." }, "cvssV3_1Severity": { "type": "string", @@ -366,7 +388,7 @@ "HIGH", "CRITICAL" ], - "description": "Returns CVE records that contain the provided CVSS v3.1 qualitative severity rating." + "description": "Returns CVE records with the provided qualitative severity in a scope-selected CVSS v3.1 metric. Values are case-insensitive." }, "cvssV4Severity": { "type": "string", @@ -376,37 +398,37 @@ "HIGH", "CRITICAL" ], - "description": "Returns CVE records that contain the provided CVSS v4.0 qualitative severity rating." + "description": "Returns CVE records with the provided qualitative severity in a scope-selected CVSS v4.0 metric. Values are case-insensitive." }, "cvssV2Metrics": { "type": "string", "maxLength": 256, "example": "AV:N/AC:L/Au:N/C:P/I:P/A:P", - "description": "Returns CVE records that contain the provided CVSS v2 vector string. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." + "description": "Returns CVE records with a scope-selected CVSS v2 vector containing every supplied slash-delimited metric token. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." }, "cvssV3Metrics": { "type": "string", "maxLength": 256, "example": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", - "description": "Returns CVE records that contain the provided CVSS v3.0 or v3.1 vector string. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." + "description": "Returns CVE records with a scope-selected CVSS v3.0 or v3.1 vector containing every supplied slash-delimited metric token. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." }, "cvssV3_0Metrics": { "type": "string", "maxLength": 256, "example": "CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", - "description": "Returns CVE records that contain the provided CVSS v3.0 vector string. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." + "description": "Returns CVE records with a scope-selected CVSS v3.0 vector containing every supplied slash-delimited metric token. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." }, "cvssV3_1Metrics": { "type": "string", "maxLength": 256, "example": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", - "description": "Returns CVE records that contain the provided CVSS v3.1 vector string. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." + "description": "Returns CVE records with a scope-selected CVSS v3.1 vector containing every supplied slash-delimited metric token. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." }, "cvssV4Metrics": { "type": "string", "maxLength": 256, "example": "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:N/SI:N/SA:N", - "description": "Returns CVE records that contain the provided CVSS v4.0 vector string. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." + "description": "Returns CVE records with a scope-selected CVSS v4.0 vector containing every supplied slash-delimited metric token. Values are limited to 256 characters, 32 slash-delimited metric tokens, and 32 characters per metric token." }, "cpeName": { "type": "string", @@ -491,6 +513,75 @@ "maxLength": 2048, "example": "cpe:2.3:o:microsoft:windows_10_1809", "description": "Returns CVE records whose CPE Match Criteria are compatible with the provided CPE 2.3 match string. CPE string literal comparisons are insensitive to lexical case. Omitted trailing components and * components act as broad matches. When used with versionStart or versionEnd, provide a partial CPE 2.3 match string without a version component." + }, + "ssvcExploitation": { + "type": "string", + "enum": [ + "NONE", + "POC", + "ACTIVE" + ], + "description": "Returns records with a scope-selected SSVC metric whose Exploitation decision has this value. Values are case-insensitive and all supplied SSVC filters must match the same SSVC metric entry." + }, + "ssvcAutomatable": { + "type": "string", + "enum": [ + "YES", + "NO" + ], + "description": "Returns records with a scope-selected SSVC metric whose Automatable decision has this value. Values are case-insensitive and all supplied SSVC filters must match the same SSVC metric entry." + }, + "ssvcTechnicalImpact": { + "type": "string", + "enum": [ + "PARTIAL", + "TOTAL" + ], + "description": "Returns records with a scope-selected SSVC metric whose Technical Impact decision has this value. Values are case-insensitive and all supplied SSVC filters must match the same SSVC metric entry." + }, + "ssvcRole": { + "type": "string", + "maxLength": 256, + "example": "CISA Coordinator", + "description": "Returns records with a scope-selected SSVC metric whose role exactly matches this case-insensitive value." + }, + "ssvcTimestampStartDate": { + "type": "string", + "format": "date-time", + "example": "2024-01-01T00:00:00.000Z", + "description": "Returns records with a scope-selected SSVC metric timestamp on or after this inclusive value. Use the format yyyy-MM-ddTHH:mm:ss.SSSZ." + }, + "ssvcTimestampEndDate": { + "type": "string", + "format": "date-time", + "example": "2024-12-31T23:59:59.999Z", + "description": "Returns records with a scope-selected SSVC metric timestamp on or before this inclusive value. Use the format yyyy-MM-ddTHH:mm:ss.SSSZ." + }, + "knownExploited": { + "type": "boolean", + "description": "When true, returns records with a structured KEV metric in a container selected by scope. When false, excludes records with a selected KEV metric. This field must be true when either KEV date boundary is supplied." + }, + "kevDateAddedStartDate": { + "type": "string", + "format": "date-time", + "example": "2024-01-01T00:00:00.000Z", + "description": "Returns records whose scope-selected KEV metric has a dateAdded value on or after this inclusive timestamp. knownExploited must be true. Stored date-only values are interpreted as midnight UTC." + }, + "kevDateAddedEndDate": { + "type": "string", + "format": "date-time", + "example": "2024-12-31T23:59:59.999Z", + "description": "Returns records whose scope-selected KEV metric has a dateAdded value on or before this inclusive timestamp. knownExploited must be true. Stored date-only values are interpreted as midnight UTC." + }, + "referenceURL": { + "type": "string", + "maxLength": 2048, + "example": "vendor.example/security/advisory", + "description": "Returns records whose scope-selected references contain this case-insensitive literal URL substring. Wildcard syntax is not supported." + }, + "disputed": { + "type": "boolean", + "description": "When true, returns records where at least one scope-selected container has the disputed tag. When false, returns records where no selected container has that tag." } } } diff --git a/api-docs/openapi.test.js b/api-docs/openapi.test.js index 0b49ee4..c2e1db1 100644 --- a/api-docs/openapi.test.js +++ b/api-docs/openapi.test.js @@ -75,6 +75,16 @@ describe('OpenAPI contract', () => { }); expect(schema.properties.noRejected.type).toBe('boolean'); expect(schema.properties.noRejected.description).toContain('When true'); + expect(schema.properties.scope.enum).toStrictEqual(['FULL', 'CNA_ONLY', 'ADP_ONLY']); + expect(schema.properties.scope.default).toBe('FULL'); + expect(schema.properties.scope.description).toContain('does not remove data'); + expect(schema.properties.ssvcExploitation.enum).toStrictEqual(['NONE', 'POC', 'ACTIVE']); + expect(schema.properties.ssvcAutomatable.enum).toStrictEqual(['YES', 'NO']); + expect(schema.properties.ssvcTechnicalImpact.enum).toStrictEqual(['PARTIAL', 'TOTAL']); + expect(schema.properties.knownExploited.type).toBe('boolean'); + expect(schema.properties.kevDateAddedStartDate.description).toContain('midnight UTC'); + expect(schema.properties.referenceURL.description).toContain('literal URL substring'); + expect(schema.properties.disputed.type).toBe('boolean'); expect(schema.properties.isVulnerable.description).toContain('same CPE Match Criteria entry'); expect(schema.properties.isVulnerable.description).toContain('False or omitted'); expect(schema.properties.cpeName.description).toContain('Maven-style ordering'); diff --git a/controllers/middleware.js b/controllers/middleware.js index 8eedf02..5be88e8 100644 --- a/controllers/middleware.js +++ b/controllers/middleware.js @@ -1,6 +1,8 @@ const errors = require('../utils/errors'); const constants = require('../utils/constants'); const cpe = require('../utils/cpe'); +const searchScope = require('../utils/searchScope'); +const ssvc = require('../utils/ssvc'); const error = new errors.errors(); const cvssV2SeverityEnum = ['LOW', 'MEDIUM', 'HIGH']; @@ -35,9 +37,24 @@ const severityValidators = [ } ]; +const ssvcEnumValidators = [ + { param: 'ssvcExploitation', allowed: ssvc.EXPLOITATION_VALUES }, + { param: 'ssvcAutomatable', allowed: ssvc.AUTOMATABLE_VALUES }, + { param: 'ssvcTechnicalImpact', allowed: ssvc.TECHNICAL_IMPACT_VALUES } +]; + const cvssMetricsParams = ['cvssV2Metrics', 'cvssV3Metrics', 'cvssV3_0Metrics', 'cvssV3_1Metrics', 'cvssV4Metrics']; -const dateParams = ['lastModStartDate', 'lastModEndDate', 'pubStartDate', 'pubEndDate']; +const dateParams = [ + 'lastModStartDate', + 'lastModEndDate', + 'pubStartDate', + 'pubEndDate', + 'ssvcTimestampStartDate', + 'ssvcTimestampEndDate', + 'kevDateAddedStartDate', + 'kevDateAddedEndDate' +]; const versionRangeParams = ['versionStart', 'versionStartType', 'versionEnd', 'versionEndType']; const versionRangeValueParams = ['versionStart', 'versionEnd']; const versionRangeTypeParams = ['versionStartType', 'versionEndType']; @@ -46,6 +63,8 @@ const nonStringSearchParams = [ 'cveIds', 'isVulnerable', 'noRejected', + 'knownExploited', + 'disputed', 'resultsPerPage', 'pageNumber', ...versionRangeParams @@ -67,6 +86,10 @@ function isSingleNonEmptyString(value) { return typeof value === 'string' && value.trim() !== ''; } +function isBoolean(value) { + return value === true || value === false; +} + function hasAnyParam(params, paramList) { return paramList.some((param) => hasParam(params, param)); } @@ -146,6 +169,17 @@ function validateParams(req, res, next) { } } + if (!searchScope.isValidScope(params.scope)) { + return res.status(400).json(error.scopeValueError()); + } + + const incompatibleParams = searchScope.getIncompatibleParams(params); + if (incompatibleParams.length) { + return res + .status(400) + .json(error.scopeCompatibilityError(searchScope.normalizeScope(params.scope), incompatibleParams)); + } + for (const [param, maxLength] of Object.entries(constants.SEARCH_STRING_LENGTH_LIMITS)) { if (typeof params[param] === 'string' && params[param].length > maxLength) { return res.status(400).json(error.searchFieldLengthError(param, maxLength)); @@ -178,6 +212,14 @@ function validateParams(req, res, next) { return res.status(400).json(error.noRejectedValueError()); } + if (hasParam(params, 'knownExploited') && !isBoolean(params.knownExploited)) { + return res.status(400).json(error.knownExploitedInputError()); + } + + if (hasParam(params, 'disputed') && !isBoolean(params.disputed)) { + return res.status(400).json(error.disputedInputError()); + } + if (hasAnyParam(params, versionRangeParams)) { if (!params.virtualMatchString) { return res.status(400).json(error.versionRangeVirtualMatchStringRequiredError()); @@ -236,6 +278,20 @@ function validateParams(req, res, next) { } } + for (const { param, allowed } of ssvcEnumValidators) { + const value = params?.[param]; + if (value && !allowed.includes(value.toUpperCase())) { + return res.status(400).json(error.enumValueError(param, allowed)); + } + } + + if ( + hasAnyParam(params, ['kevDateAddedStartDate', 'kevDateAddedEndDate']) && + !isTrueParam(params.knownExploited) + ) { + return res.status(400).json(error.kevDateRequiresKnownExploitedError()); + } + //Ensures keywordSearch is specified if keyWordExactMatch is true if (isTrueParam(params?.keywordExactMatch) && !params.keywordSearch) { return res.status(400).json(error.missingKeywordSearchError()); diff --git a/controllers/middleware.test.js b/controllers/middleware.test.js index 5079be4..dd70330 100644 --- a/controllers/middleware.test.js +++ b/controllers/middleware.test.js @@ -370,6 +370,7 @@ describe('middleware validation function', () => { ['keywordSearch', ''], ['cpeName', ' '], ['virtualMatchString', ''], + ['referenceURL', ''], ['pubStartDate', ''], ['cvssV3Metrics', ''], ['cvssV3Severity', ''] @@ -405,7 +406,8 @@ describe('middleware validation function', () => { ['pubEndDate', '2024-01-04T00:00:00.000Z'], ['resultsPerPage', '10'], ['pageNumber', '2'], - ['virtualMatchString', 'cpe:2.3:a:vendor:product:*:*:*:*:*:*:*:*'] + ['virtualMatchString', 'cpe:2.3:a:vendor:product:*:*:*:*:*:*:*:*'], + ['referenceURL', 'vendor.example/security/advisory'] ])('ValidateParams accepts supported search parameter %s.', (param, value) => { validate({ [param]: value @@ -872,6 +874,88 @@ describe('middleware validation function', () => { } ); + test.each(['FULL', 'full', 'CNA_ONLY', 'adp_only'])( + 'ValidateParams accepts case-insensitive scope value %s.', + (scope) => { + validate({ scope }); + expectValidationSuccess(); + } + ); + + test('ValidateParams rejects unsupported scope values', () => { + validate({ scope: 'CNA_AND_ADP' }); + expectValidationError('SCOPE_VALUE', 'The scope parameter must be FULL, CNA_ONLY, or ADP_ONLY.'); + }); + + test('ValidateParams rejects CNA-only CPE filters with ADP_ONLY', () => { + validate({ + scope: 'ADP_ONLY', + virtualMatchString: 'cpe:2.3:a:vendor:product' + }); + expectValidationError( + 'SCOPE_FILTER_INCOMPATIBLE', + 'The virtualMatchString parameter is not supported with scope ADP_ONLY.' + ); + }); + + test.each([ + ['ssvcExploitation', 'active'], + ['ssvcAutomatable', 'yes'], + ['ssvcTechnicalImpact', 'partial'] + ])('ValidateParams accepts case-insensitive SSVC enum %s.', (param, value) => { + validate({ [param]: value }); + expectValidationSuccess(); + }); + + test('ValidateParams rejects unsupported SSVC enum values', () => { + validate({ ssvcExploitation: 'unknown' }); + expectValidationError( + 'INVALID_ENUM_VALUE', + 'The ssvcExploitation parameter must be one of: NONE, POC, ACTIVE.' + ); + }); + + test.each(['ssvcTimestampStartDate', 'ssvcTimestampEndDate'])( + 'ValidateParams validates SSVC timestamp %s.', + (param) => { + validate({ [param]: '2024-02-30T00:00:00.000Z' }); + expectValidationError( + 'DATE_TIME_FORMAT', + `The ${param} parameter must match the following format: yyyy-MM-ddTHH:mm:ss.SSS'Z'` + ); + } + ); + + test('ValidateParams requires knownExploited true for KEV date filters', () => { + validate({ + knownExploited: false, + kevDateAddedStartDate: '2024-01-01T00:00:00.000Z' + }); + expectValidationError( + 'KEV_DATE_REQUIRES_KNOWN_EXPLOITED', + 'knownExploited must be true when kevDateAddedStartDate or kevDateAddedEndDate is provided.' + ); + }); + + test('ValidateParams accepts KEV dates with knownExploited true', () => { + validate({ + knownExploited: true, + kevDateAddedStartDate: '2024-01-01T00:00:00.000Z', + kevDateAddedEndDate: '2024-12-31T23:59:59.999Z' + }); + expectValidationSuccess(); + }); + + test('ValidateParams requires knownExploited to be a JSON boolean.', () => { + validate({ knownExploited: 'true' }); + expectValidationError('KNOWN_EXPLOITED_BOOLEAN', 'The knownExploited parameter must be a boolean.'); + }); + + test('ValidateParams requires disputed to be a JSON boolean.', () => { + validate({ disputed: 'false' }); + expectValidationError('DISPUTED_BOOLEAN', 'The disputed parameter must be a boolean.'); + }); + test('ValidateParams delegates unexpected errors to next()', () => { const err = new Error('query failure'); Object.defineProperty(req, 'query', { diff --git a/controllers/searchController.js b/controllers/searchController.js index 5deabdb..b31f590 100644 --- a/controllers/searchController.js +++ b/controllers/searchController.js @@ -1,6 +1,5 @@ const PackageInfo = require('cve-core').packageInfo; const errors = require('../utils/errors'); -const cpe = require('../utils/cpe'); const searchPerformance = require('../utils/searchPerformance'); const searchDeadline = require('../utils/searchDeadline'); const searchPostFilterCache = require('../utils/searchPostFilterCache'); @@ -8,62 +7,11 @@ const searchInFlightRegistry = require('../utils/searchInFlightRegistry'); const searchPostFilterWorkQueue = require('../utils/searchPostFilterWorkQueue'); const searchProvider = require('../utils/searchProvider'); const searchQueryBuilder = require('../utils/searchQueryBuilder'); +const searchPostFilters = require('../utils/searchPostFilters'); const error = new errors.errors(); const openSearchUnavailableErrorNames = new Set(['ConnectionError', 'NoLivingConnectionsError']); const openSearchUnavailableStatusCodes = new Set([502, 503, 504]); -function hasSearchPostFilters(postFilters) { - return Boolean(postFilters?.cpeName || postFilters?.virtualMatchString || postFilters?.virtualMatchStringRange); -} - -function getHitsContainer(resp) { - return resp?.data?.hits; -} - -function filterSearchHits(resp, postFilters) { - const hitsContainer = getHitsContainer(resp); - const cpeEvaluationContext = cpe.createEvaluationContext(); - - if (!Array.isArray(hitsContainer?.hits)) { - return { - hitsContainer, - candidateCount: 0, - filteredHits: [] - }; - } - - return { - hitsContainer, - candidateCount: hitsContainer.hits.length, - filteredHits: hitsContainer.hits.filter((hit) => { - if ( - postFilters?.cpeName && - !cpe.hitAppliesToCpeName(hit, postFilters.cpeName, cpeEvaluationContext, { - requireVulnerable: postFilters.isVulnerable === true - }) - ) { - return false; - } - - if ( - postFilters?.virtualMatchString && - !cpe.hitMatchesVirtualMatchString(hit, postFilters.virtualMatchString, cpeEvaluationContext) - ) { - return false; - } - - if ( - postFilters?.virtualMatchStringRange && - !cpe.hitOverlapsVirtualMatchStringRange(hit, postFilters.virtualMatchStringRange, cpeEvaluationContext) - ) { - return false; - } - - return true; - }) - }; -} - function paginateHits(hits, responseOptions) { const from = responseOptions?.from ?? 0; const resultsPerPage = responseOptions?.resultsPerPage ?? hits.length; @@ -72,7 +20,7 @@ function paginateHits(hits, responseOptions) { } function buildPagedSearchResponse(resp, totalValue, pagedHits) { - const hitsContainer = getHitsContainer(resp); + const hitsContainer = resp?.data?.hits; return { ...resp, @@ -90,15 +38,6 @@ function buildPagedSearchResponse(resp, totalValue, pagedHits) { }; } -function buildPostFilteredResponse( - resp, - filteredHits, - responseOptions, - pagedHits = paginateHits(filteredHits, responseOptions) -) { - return buildPagedSearchResponse(resp, filteredHits.length, pagedHits); -} - function getOpenSearchErrorStatusCode(err) { return Number(err?.statusCode ?? err?.meta?.statusCode ?? err?.meta?.body?.status); } @@ -138,19 +77,19 @@ function buildWebSearchSort(sort) { } function applySearchPostFilters(resp, postFilters, responseOptions, metrics) { - if (!hasSearchPostFilters(postFilters)) return resp; + if (!searchPostFilters.hasSearchPostFilters(postFilters)) return resp; - const { hitsContainer, candidateCount, filteredHits } = filterSearchHits(resp, postFilters); + const { hitsContainer, candidateCount, filteredHits } = searchPostFilters.filterSearchHits(resp, postFilters); if (!Array.isArray(hitsContainer?.hits)) return resp; + const pagedHits = paginateHits(filteredHits, responseOptions); if (metrics) { - const pagedHits = paginateHits(filteredHits, responseOptions); metrics.candidateCount = candidateCount; metrics.filteredCount = filteredHits.length; metrics.returnedCount = pagedHits.length; } - return buildPostFilteredResponse(resp, filteredHits, responseOptions); + return buildPagedSearchResponse(resp, filteredHits.length, pagedHits); } function getHitCveId(hit) { @@ -238,8 +177,9 @@ async function fetchFullHitsByCveIds(cveIds, metrics, signal) { } async function buildPostFilterCandidateResult(candidateBody, postFilters, metrics, signal) { + const operation = searchPostFilters.hasCpePostFilters(postFilters) ? 'cpe-candidate' : 'semantic-candidate'; const candidateStart = searchPerformance.markTime(); - const candidateData = await executeOpenSearch(candidateBody, 'cpe-candidate', signal, metrics); + const candidateData = await executeOpenSearch(candidateBody, operation, signal, metrics); metrics.openSearchElapsedMs = searchPerformance.elapsedMs(candidateStart); metrics.openSearchTookMs = candidateData?.took; @@ -247,7 +187,7 @@ async function buildPostFilterCandidateResult(candidateBody, postFilters, metric data: candidateData }; const postFilterStart = searchPerformance.markTime(); - const { candidateCount, filteredHits } = filterSearchHits(candidateResp, postFilters); + const { candidateCount, filteredHits } = searchPostFilters.filterSearchHits(candidateResp, postFilters); metrics.postFilterElapsedMs = searchPerformance.elapsedMs(postFilterStart); return { @@ -258,6 +198,12 @@ async function buildPostFilterCandidateResult(candidateBody, postFilters, metric } async function getPostFilterCandidateResult(cacheKey, candidateBody, postFilters, metrics, signal) { + if (!searchPostFilters.hasCpePostFilters(postFilters)) { + metrics.cacheHit = false; + metrics.inFlightHit = false; + return buildPostFilterCandidateResult(candidateBody, postFilters, metrics, signal); + } + const cachedResult = searchPostFilterCache.get(cacheKey); if (cachedResult) { metrics.cacheHit = true; @@ -305,9 +251,13 @@ async function executeOptimizedPostFilterSearch( metrics, signal ) { - metrics.strategy = 'cpe-candidate'; + metrics.strategy = searchPostFilters.hasCpePostFilters(postFilters) ? 'cpe-candidate' : 'semantic-candidate'; - const candidateBody = searchQueryBuilder.buildCpeCandidateSearchBody(searchOptions, queryStringObjects); + const candidateBody = searchQueryBuilder.buildPostFilterCandidateSearchBody( + searchOptions, + queryStringObjects, + postFilters + ); const cacheKey = searchPostFilterCache.buildCacheKey(candidateBody, postFilters); const candidateResult = await getPostFilterCandidateResult(cacheKey, candidateBody, postFilters, metrics, signal); searchDeadline.throwIfAborted(signal); @@ -333,7 +283,7 @@ async function executeOptimizedPostFilterSearch( } async function executeSearch(searchOptions, queryStringObjects, postFilters, responseOptions, metrics, signal) { - if (!hasSearchPostFilters(postFilters)) { + if (!searchPostFilters.hasSearchPostFilters(postFilters)) { return executeStandardSearch(searchOptions, queryStringObjects, postFilters, responseOptions, metrics, signal); } diff --git a/controllers/searchController.test.js b/controllers/searchController.test.js index 2a116f5..f0283f4 100644 --- a/controllers/searchController.test.js +++ b/controllers/searchController.test.js @@ -185,6 +185,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', keywordSearch: 'remote code', cveIds: 'CVE-2024-0001', pubStartDate: '2024-01-01T00:00:00.000Z', @@ -263,6 +264,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', keywordSearch: 'remote code', keywordExactMatch: 'true' } @@ -312,6 +314,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', noRejected: true } }, @@ -399,6 +402,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', cweId: 'CWE-79', cpeName: 'cpe:2.3:a:vendor:product:1.0:*:*:*:*:*:*:*', isVulnerable: 'true' @@ -556,6 +560,145 @@ describe('Search controller SEARCH handler', () => { }); }); + test('SEARCH semantically filters SSVC candidates and fetches full records for the requested page', async () => { + const matching = { + _id: 'CVE-2024-0001', + _source: { + cveMetadata: { cveId: 'CVE-2024-0001' }, + containers: { + adp: [ + { + metrics: [ + { + other: { + type: 'ssvc', + content: { + role: 'CISA Coordinator', + options: [{ Exploitation: 'active' }, { Automatable: 'yes' }] + } + } + } + ] + } + ] + } + } + }; + const splitAcrossEntries = { + _id: 'CVE-2024-0002', + _source: { + cveMetadata: { cveId: 'CVE-2024-0002' }, + containers: { + adp: [ + { + metrics: [ + { other: { type: 'ssvc', content: { options: [{ Exploitation: 'active' }] } } }, + { other: { type: 'ssvc', content: { options: [{ Automatable: 'yes' }] } } } + ] + } + ] + } + } + }; + const fullRecord = { + _id: 'CVE-2024-0001', + _source: { + cveMetadata: { cveId: 'CVE-2024-0001' }, + containers: { cna: { title: 'complete record' }, adp: [{ title: 'enrichment' }] } + } + }; + mockApiSearch + .mockResolvedValueOnce(rawResponse(2, [matching, splitAcrossEntries])) + .mockResolvedValueOnce(advancedResponse(1, [fullRecord])); + + await controller.SEARCH( + { + searchParams: { + scope: 'ADP_ONLY', + ssvcExploitation: 'ACTIVE', + ssvcAutomatable: 'YES', + resultsPerPage: 1, + pageNumber: 1 + } + }, + res, + next + ); + + expect(mockSearchOpenSearch).toHaveBeenCalledTimes(2); + expect(mockSearchOpenSearch.mock.calls[0][1].operation).toBe('semantic-candidate'); + expect(mockSearchOpenSearch.mock.calls[1][1].operation).toBe('page-fetch'); + expect(mockSearchOpenSearch.mock.calls[0][0]._source).toStrictEqual([ + 'cveMetadata.cveId', + 'containers.adp.metrics.other' + ]); + expect(res.json.mock.calls[0][0].data.hits.total.value).toBe(1); + expect(res.json.mock.calls[0][0].data.hits.hits).toStrictEqual([fullRecord]); + }); + + test('SEARCH semantically filters date-bounded KEV candidates within the selected scope', async () => { + const matching = { + _id: 'CVE-2024-0001', + _source: { + cveMetadata: { cveId: 'CVE-2024-0001' }, + containers: { + adp: [ + { + metrics: [{ other: { type: 'kev', content: { dateAdded: '2024-06-15' } } }] + } + ] + } + } + }; + const outsideRange = { + _id: 'CVE-2024-0002', + _source: { + cveMetadata: { cveId: 'CVE-2024-0002' }, + containers: { + adp: [ + { + metrics: [{ other: { type: 'kev', content: { dateAdded: '2023-12-31' } } }] + } + ] + } + } + }; + const fullRecord = { + _id: 'CVE-2024-0001', + _source: { + cveMetadata: { cveId: 'CVE-2024-0001' }, + containers: { cna: { title: 'complete record' }, adp: [{ title: 'enrichment' }] } + } + }; + mockApiSearch + .mockResolvedValueOnce(rawResponse(2, [matching, outsideRange])) + .mockResolvedValueOnce(advancedResponse(1, [fullRecord])); + + await controller.SEARCH( + { + searchParams: { + scope: 'ADP_ONLY', + knownExploited: true, + kevDateAddedStartDate: '2024-01-01T00:00:00.000Z', + resultsPerPage: 1, + pageNumber: 1 + } + }, + res, + next + ); + + expect(mockSearchOpenSearch).toHaveBeenCalledTimes(2); + expect(mockSearchOpenSearch.mock.calls[0][1].operation).toBe('semantic-candidate'); + expect(mockSearchOpenSearch.mock.calls[1][1].operation).toBe('page-fetch'); + expect(mockSearchOpenSearch.mock.calls[0][0]._source).toStrictEqual([ + 'cveMetadata.cveId', + 'containers.adp.metrics.other' + ]); + expect(res.json.mock.calls[0][0].data.hits.total.value).toBe(1); + expect(res.json.mock.calls[0][0].data.hits.hits).toStrictEqual([fullRecord]); + }); + test('applySearchPostFilters does not combine platform and version matches from different CPE entries', () => { const candidate = cpeHit('CVE-2026-0250', { criteria: 'cpe:2.3:a:palo_alto_networks:globalprotect_app:*:*:macos:*:*:*:*:*', @@ -1212,6 +1355,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', cvssV3_0Severity: 'high', cvssV3_1Severity: 'critical' } @@ -1240,6 +1384,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', cvssV2Metrics: 'AV:N/AC:L/Au:N' } }, @@ -1284,6 +1429,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', cvssV3_0Metrics: 'CVSS:3.0/AV:N/AC:L', cvssV3_1Metrics: 'CVSS:3.1/PR:N/UI:R' } @@ -1342,6 +1488,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', keywordSearch: 'c++ path:/tmp (test)' } }, @@ -1366,6 +1513,7 @@ describe('Search controller SEARCH handler', () => { await controller.SEARCH( { query: { + scope: 'CNA_ONLY', keywordSearch: ' ' } }, diff --git a/routes/cpeName.test.js b/routes/cpeName.test.js index 89cc369..25bd736 100644 --- a/routes/cpeName.test.js +++ b/routes/cpeName.test.js @@ -55,6 +55,7 @@ describe('POST /search cpeName route behavior', () => { expect(response.body).toStrictEqual({ route: 'search', params: { + scope: 'FULL', cpeName } }); diff --git a/routes/index.js b/routes/index.js index adca945..d4882f0 100644 --- a/routes/index.js +++ b/routes/index.js @@ -8,13 +8,23 @@ const featureFlags = require('../utils/featureFlags'); const searchDeadline = require('../utils/searchDeadline'); const searchProvider = require('../utils/searchProvider'); const searchRateLimiter = require('../utils/searchRateLimiter'); +const searchScope = require('../utils/searchScope'); const error = new errors.errors(); const limitSearchRequests = searchRateLimiter.createSearchRateLimiter(); const SEARCH_BODY_FIELDS = new Set(constants.SEARCH_QUERY_PARAMS); const SEARCH_STRING_BODY_FIELDS = constants.SEARCH_QUERY_PARAMS.filter( (field) => - !['keywordExactMatch', 'cveIds', 'isVulnerable', 'noRejected', 'resultsPerPage', 'pageNumber'].includes(field) + ![ + 'keywordExactMatch', + 'cveIds', + 'isVulnerable', + 'noRejected', + 'knownExploited', + 'disputed', + 'resultsPerPage', + 'pageNumber' + ].includes(field) ); const WEB_SEARCH_BODY_FIELDS = new Set(['query', 'sort', 'from', 'size']); const WEB_SEARCH_SORT_FIELDS = new Set(['property', 'order']); @@ -101,6 +111,10 @@ function isStrictBoolean(value) { return value === true || value === false || value === 'true' || value === 'false'; } +function isJsonBoolean(value) { + return value === true || value === false; +} + function toBooleanValue(value) { if (value === 'true') return true; if (value === 'false') return false; @@ -145,11 +159,31 @@ function requireWebSearchEnabled(req, res, next) { function normalizeSearchParams(req, res, next) { const searchParams = { ...(req.body ?? {}) }; + const upperCaseEnums = [ + 'scope', + 'cvssV2Severity', + 'cvssV3Severity', + 'cvssV3_0Severity', + 'cvssV3_1Severity', + 'cvssV4Severity', + 'ssvcExploitation', + 'ssvcAutomatable', + 'ssvcTechnicalImpact' + ]; + const lowerCaseEnums = ['versionStartType', 'versionEndType']; if (searchParams.noRejected === false) { delete searchParams.noRejected; } + searchParams.scope = searchScope.normalizeScope(searchParams.scope); + upperCaseEnums.forEach((param) => { + if (typeof searchParams[param] === 'string') searchParams[param] = searchParams[param].trim().toUpperCase(); + }); + lowerCaseEnums.forEach((param) => { + if (typeof searchParams[param] === 'string') searchParams[param] = searchParams[param].trim().toLowerCase(); + }); + req.searchParams = searchParams; next(); } @@ -175,6 +209,7 @@ router.post( allParameters: { summary: "All supported request body parameters", value: { + scope: "FULL", keywordSearch: "remote code execution", keywordExactMatch: true, cveIds: "CVE-2024-1234,CVE-2024-5678", @@ -202,7 +237,18 @@ router.post( versionStartType: "including", versionEnd: "2.6", versionEndType: "excluding", - virtualMatchString: "cpe:2.3:o:linux:linux_kernel" + virtualMatchString: "cpe:2.3:o:linux:linux_kernel", + ssvcExploitation: "ACTIVE", + ssvcAutomatable: "YES", + ssvcTechnicalImpact: "TOTAL", + ssvcRole: "CISA Coordinator", + ssvcTimestampStartDate: "2024-01-01T00:00:00.000Z", + ssvcTimestampEndDate: "2024-12-31T23:59:59.999Z", + knownExploited: true, + kevDateAddedStartDate: "2024-01-01T00:00:00.000Z", + kevDateAddedEndDate: "2024-12-31T23:59:59.999Z", + referenceURL: "vendor.example/security/advisory", + disputed: false } }, keywordSearch: { @@ -313,6 +359,8 @@ description: 'Search exceeded its OpenSearch or overall processing deadline', .custom(isStrictBoolean) .withMessage(error.noRejectedValueError()) .customSanitizer(toBooleanValue), + body('knownExploited').optional().custom(isJsonBoolean).withMessage(error.knownExploitedInputError()), + body('disputed').optional().custom(isJsonBoolean).withMessage(error.disputedInputError()), body(['resultsPerPage']) .optional() .isInt({ max: constants.MAX_RESULTS_LIMIT }) diff --git a/routes/index.test.js b/routes/index.test.js index c1cd095..415447d 100644 --- a/routes/index.test.js +++ b/routes/index.test.js @@ -106,6 +106,7 @@ describe('routes/index router', () => { expect(response.body).toStrictEqual({ route: 'search', params: { + scope: 'FULL', keywordSearch: 'test', keywordExactMatch: false } @@ -124,7 +125,9 @@ describe('routes/index router', () => { expect(controller.SEARCH).toHaveBeenCalledTimes(1); expect(response.body).toStrictEqual({ route: 'search', - params: {} + params: { + scope: 'FULL' + } }); }); @@ -258,6 +261,26 @@ describe('routes/index router', () => { expect(controller.SEARCH).not.toHaveBeenCalled(); }); + test('POST /search rejects an array for referenceURL before custom middleware runs', async () => { + const response = await request(app, { + method: 'POST', + path: '/search', + body: { + referenceURL: ['example.test/advisory'] + } + }); + + expect(response.status).toBe(400); + expect(response.body).toStrictEqual([ + { + error: 'SEARCH_FIELD_STRING', + message: 'The referenceURL field must be a non-empty string.' + } + ]); + expect(mw.validateParams).not.toHaveBeenCalled(); + expect(controller.SEARCH).not.toHaveBeenCalled(); + }); + test('POST /search lets cveIds reach custom middleware for format validation', async () => { const response = await request(app, { method: 'POST', @@ -273,6 +296,7 @@ describe('routes/index router', () => { expect(response.body).toStrictEqual({ route: 'search', params: { + scope: 'FULL', cveIds: ['CVE-2024-0001', 'CVE-2024-0002'] } }); @@ -292,8 +316,57 @@ describe('routes/index router', () => { expect(controller.SEARCH).toHaveBeenCalledTimes(1); expect(response.body).toStrictEqual({ route: 'search', - params: {} + params: { + scope: 'FULL' + } + }); + }); + + test('POST /search normalizes case-insensitive enums before custom middleware runs', async () => { + const response = await request(app, { + method: 'POST', + path: '/search', + body: { + scope: 'adp_only', + cvssV3Severity: 'critical', + ssvcExploitation: 'active', + versionStartType: 'INCLUDING' + } + }); + + expect(response.status).toBe(200); + expect(response.body.params).toStrictEqual({ + scope: 'ADP_ONLY', + cvssV3Severity: 'CRITICAL', + ssvcExploitation: 'ACTIVE', + versionStartType: 'including' + }); + }); + + test('POST /search rejects a string value for knownExploited', async () => { + const response = await request(app, { + method: 'POST', + path: '/search', + body: { + knownExploited: 'true' + } }); + + expect(response.status).toBe(400); + expect(controller.SEARCH).not.toHaveBeenCalled(); + }); + + test('POST /search rejects a string value for disputed', async () => { + const response = await request(app, { + method: 'POST', + path: '/search', + body: { + disputed: 'false' + } + }); + + expect(response.status).toBe(400); + expect(controller.SEARCH).not.toHaveBeenCalled(); }); test('POST /search rejects resultsPerPage above the configured max before custom middleware runs', async () => { diff --git a/routes/search.integration.test.js b/routes/search.integration.test.js index 5319044..86a3dbf 100644 --- a/routes/search.integration.test.js +++ b/routes/search.integration.test.js @@ -74,7 +74,7 @@ describe('search route integration', () => { simple_query_string: { query: 'remote* code*', default_operator: 'AND', - fields: ['containers.cna.descriptions.value'] + fields: ['containers.cna.descriptions.value', 'containers.adp.descriptions.value'] } } ], @@ -96,6 +96,56 @@ describe('search route integration', () => { expect(response.body.totalPages).toBe(1); }); + test('POST /search builds a scope-aware literal reference URL substring filter', async () => { + mockSearchOpenSearch.mockResolvedValue(advancedResponse(0, []).data); + + const response = await request(createApp(), { + method: 'POST', + path: '/search', + body: { + scope: 'adp_only', + referenceURL: 'EXAMPLE.test/a*?b' + } + }); + + expect(response.status).toBe(200); + expect(mockSearchOpenSearch.mock.calls[0][0].query.bool.filter).toStrictEqual([ + { + wildcard: { + 'containers.adp.references.url.keyword': { + value: '*EXAMPLE.test/a\\*\\?b*', + case_insensitive: true + } + } + } + ]); + }); + + test('POST /search builds a scope-aware disputed tag filter', async () => { + mockSearchOpenSearch.mockResolvedValue(advancedResponse(0, []).data); + + const response = await request(createApp(), { + method: 'POST', + path: '/search', + body: { + scope: 'adp_only', + disputed: true + } + }); + + expect(response.status).toBe(200); + expect(mockSearchOpenSearch.mock.calls[0][0].query.bool.filter).toStrictEqual([ + { + wildcard: { + 'containers.adp.tags.keyword': { + value: 'disputed', + case_insensitive: true + } + } + } + ]); + }); + test('POST /search rejects invalid dates before invoking OpenSearch', async () => { const response = await request(createApp(), { method: 'POST', @@ -132,6 +182,38 @@ describe('search route integration', () => { expect(mockSearchOpenSearch).not.toHaveBeenCalled(); }); + test('POST /search rejects CPE applicability filters with ADP_ONLY scope', async () => { + const response = await request(createApp(), { + method: 'POST', + path: '/search', + body: { + scope: 'adp_only', + virtualMatchString: 'cpe:2.3:a:vendor:product' + } + }); + + expect(response.status).toBe(400); + expect(response.body).toStrictEqual({ + error: 'SCOPE_FILTER_INCOMPATIBLE', + message: 'The virtualMatchString parameter is not supported with scope ADP_ONLY.' + }); + expect(mockSearchOpenSearch).not.toHaveBeenCalled(); + }); + + test('POST /search rejects KEV date filters unless knownExploited is explicitly true', async () => { + const response = await request(createApp(), { + method: 'POST', + path: '/search', + body: { + kevDateAddedStartDate: '2024-01-01T00:00:00.000Z' + } + }); + + expect(response.status).toBe(400); + expect(response.body.error).toBe('KEV_DATE_REQUIRES_KNOWN_EXPLOITED'); + expect(mockSearchOpenSearch).not.toHaveBeenCalled(); + }); + test('POST /search rejects an inverted virtualMatchString version range before invoking OpenSearch', async () => { const response = await request(createApp(), { method: 'POST', diff --git a/routes/swagger.js b/routes/swagger.js index 1cf65db..3c3e047 100644 --- a/routes/swagger.js +++ b/routes/swagger.js @@ -2,6 +2,8 @@ const path = require('path'); const swaggerAutogen = require('swagger-autogen'); const packageInfo = require('../package.json'); const constants = require('../utils/constants'); +const searchScope = require('../utils/searchScope'); +const ssvc = require('../utils/ssvc'); const projectRoot = path.join(__dirname, '..'); const outputFile = path.join(projectRoot, 'api-docs', 'openapi.json'); @@ -15,8 +17,16 @@ function searchRequestProperty(description, schema) { } const searchRequestProperties = { + scope: searchRequestProperty( + 'Controls which containers participate in filter matching. FULL searches base record fields, CNA data, and all ADP providers. CNA_ONLY searches base record fields and CNA data. ADP_ONLY searches base record fields and all ADP providers. Scope does not remove data from returned records. Values are case-insensitive. CPE applicability filters are not available with ADP_ONLY.', + { + type: 'string', + enum: searchScope.SEARCH_SCOPE_VALUES, + default: searchScope.SEARCH_SCOPES.FULL + } + ), keywordSearch: searchRequestProperty( - `Returns CVE records where a word or phrase is included in the record's description. Multiple keywords operate like an AND statement, returning records whose descriptions include every keyword. Limited to ${constants.SEARCH_STRING_LENGTH_LIMITS.keywordSearch} characters and ${constants.MAX_KEYWORD_SEARCH_TOKENS} whitespace-delimited terms.`, + `Returns CVE records where a word or phrase is included in a description selected by scope. Multiple keywords operate like an AND statement, returning records whose descriptions include every keyword. Limited to ${constants.SEARCH_STRING_LENGTH_LIMITS.keywordSearch} characters and ${constants.MAX_KEYWORD_SEARCH_TOKENS} whitespace-delimited terms.`, { type: 'string', maxLength: constants.SEARCH_STRING_LENGTH_LIMITS.keywordSearch, @@ -42,48 +52,51 @@ const searchRequestProperties = { type: 'boolean', default: false }), - cweId: searchRequestProperty('Returns CVE records that include the provided Common Weakness Enumeration ID.', { - type: 'string', - maxLength: constants.SEARCH_STRING_LENGTH_LIMITS.cweId, - example: 'CWE-79' - }), + cweId: searchRequestProperty( + 'Returns CVE records whose scope-selected containers include the provided Common Weakness Enumeration ID.', + { + type: 'string', + maxLength: constants.SEARCH_STRING_LENGTH_LIMITS.cweId, + example: 'CWE-79' + } + ), cvssV2Severity: searchRequestProperty( - 'Returns CVE records that contain the provided CVSS v2 qualitative severity rating.', + 'Returns CVE records with a scope-selected CVSS v2 base score in the range represented by the provided qualitative severity. Values are case-insensitive.', { type: 'string', enum: ['LOW', 'MEDIUM', 'HIGH'] } ), cvssV3Severity: searchRequestProperty( - 'Returns CVE records that contain the provided CVSS v3.0 or v3.1 qualitative severity rating.', + 'Returns CVE records with the provided qualitative severity in a scope-selected CVSS v3.0 or v3.1 metric. Values are case-insensitive.', { type: 'string', enum: ['LOW', 'MEDIUM', 'HIGH', 'CRITICAL'] } ), cvssV3_0Severity: searchRequestProperty( - 'Returns CVE records that contain the provided CVSS v3.0 qualitative severity rating.', + 'Returns CVE records with the provided qualitative severity in a scope-selected CVSS v3.0 metric. Values are case-insensitive.', { type: 'string', enum: ['LOW', 'MEDIUM', 'HIGH', 'CRITICAL'] } ), cvssV3_1Severity: searchRequestProperty( - 'Returns CVE records that contain the provided CVSS v3.1 qualitative severity rating.', + 'Returns CVE records with the provided qualitative severity in a scope-selected CVSS v3.1 metric. Values are case-insensitive.', { type: 'string', enum: ['LOW', 'MEDIUM', 'HIGH', 'CRITICAL'] } ), cvssV4Severity: searchRequestProperty( - 'Returns CVE records that contain the provided CVSS v4.0 qualitative severity rating.', + 'Returns CVE records with the provided qualitative severity in a scope-selected CVSS v4.0 metric. Values are case-insensitive.', { type: 'string', enum: ['LOW', 'MEDIUM', 'HIGH', 'CRITICAL'] } ), cvssV2Metrics: searchRequestProperty( - `Returns CVE records that contain the provided CVSS v2 vector string. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, + `Returns CVE records with a scope-selected CVSS v2 vector containing every supplied slash-delimited metric token. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, { type: 'string', maxLength: constants.MAX_CVSS_METRICS_LENGTH, @@ -91,7 +104,7 @@ const searchRequestProperties = { } ), cvssV3Metrics: searchRequestProperty( - `Returns CVE records that contain the provided CVSS v3.0 or v3.1 vector string. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, + `Returns CVE records with a scope-selected CVSS v3.0 or v3.1 vector containing every supplied slash-delimited metric token. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, { type: 'string', maxLength: constants.MAX_CVSS_METRICS_LENGTH, @@ -99,7 +112,7 @@ const searchRequestProperties = { } ), cvssV3_0Metrics: searchRequestProperty( - `Returns CVE records that contain the provided CVSS v3.0 vector string. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, + `Returns CVE records with a scope-selected CVSS v3.0 vector containing every supplied slash-delimited metric token. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, { type: 'string', maxLength: constants.MAX_CVSS_METRICS_LENGTH, @@ -107,7 +120,7 @@ const searchRequestProperties = { } ), cvssV3_1Metrics: searchRequestProperty( - `Returns CVE records that contain the provided CVSS v3.1 vector string. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, + `Returns CVE records with a scope-selected CVSS v3.1 vector containing every supplied slash-delimited metric token. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, { type: 'string', maxLength: constants.MAX_CVSS_METRICS_LENGTH, @@ -115,7 +128,7 @@ const searchRequestProperties = { } ), cvssV4Metrics: searchRequestProperty( - `Returns CVE records that contain the provided CVSS v4.0 vector string. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, + `Returns CVE records with a scope-selected CVSS v4.0 vector containing every supplied slash-delimited metric token. Values are limited to ${constants.MAX_CVSS_METRICS_LENGTH} characters, ${constants.MAX_CVSS_METRIC_TOKENS} slash-delimited metric tokens, and ${constants.MAX_CVSS_METRIC_TOKEN_LENGTH} characters per metric token.`, { type: 'string', maxLength: constants.MAX_CVSS_METRICS_LENGTH, @@ -225,6 +238,87 @@ const searchRequestProperties = { maxLength: constants.SEARCH_STRING_LENGTH_LIMITS.virtualMatchString, example: 'cpe:2.3:o:microsoft:windows_10_1809' } + ), + ssvcExploitation: searchRequestProperty( + 'Returns records with a scope-selected SSVC metric whose Exploitation decision has this value. Values are case-insensitive and all supplied SSVC filters must match the same SSVC metric entry.', + { + type: 'string', + enum: ssvc.EXPLOITATION_VALUES + } + ), + ssvcAutomatable: searchRequestProperty( + 'Returns records with a scope-selected SSVC metric whose Automatable decision has this value. Values are case-insensitive and all supplied SSVC filters must match the same SSVC metric entry.', + { + type: 'string', + enum: ssvc.AUTOMATABLE_VALUES + } + ), + ssvcTechnicalImpact: searchRequestProperty( + 'Returns records with a scope-selected SSVC metric whose Technical Impact decision has this value. Values are case-insensitive and all supplied SSVC filters must match the same SSVC metric entry.', + { + type: 'string', + enum: ssvc.TECHNICAL_IMPACT_VALUES + } + ), + ssvcRole: searchRequestProperty( + 'Returns records with a scope-selected SSVC metric whose role exactly matches this case-insensitive value.', + { + type: 'string', + maxLength: constants.SEARCH_STRING_LENGTH_LIMITS.ssvcRole, + example: 'CISA Coordinator' + } + ), + ssvcTimestampStartDate: searchRequestProperty( + 'Returns records with a scope-selected SSVC metric timestamp on or after this inclusive value. Use the format yyyy-MM-ddTHH:mm:ss.SSSZ.', + { + type: 'string', + format: 'date-time', + example: '2024-01-01T00:00:00.000Z' + } + ), + ssvcTimestampEndDate: searchRequestProperty( + 'Returns records with a scope-selected SSVC metric timestamp on or before this inclusive value. Use the format yyyy-MM-ddTHH:mm:ss.SSSZ.', + { + type: 'string', + format: 'date-time', + example: '2024-12-31T23:59:59.999Z' + } + ), + knownExploited: searchRequestProperty( + 'When true, returns records with a structured KEV metric in a container selected by scope. When false, excludes records with a selected KEV metric. This field must be true when either KEV date boundary is supplied.', + { + type: 'boolean' + } + ), + kevDateAddedStartDate: searchRequestProperty( + 'Returns records whose scope-selected KEV metric has a dateAdded value on or after this inclusive timestamp. knownExploited must be true. Stored date-only values are interpreted as midnight UTC.', + { + type: 'string', + format: 'date-time', + example: '2024-01-01T00:00:00.000Z' + } + ), + kevDateAddedEndDate: searchRequestProperty( + 'Returns records whose scope-selected KEV metric has a dateAdded value on or before this inclusive timestamp. knownExploited must be true. Stored date-only values are interpreted as midnight UTC.', + { + type: 'string', + format: 'date-time', + example: '2024-12-31T23:59:59.999Z' + } + ), + referenceURL: searchRequestProperty( + 'Returns records whose scope-selected references contain this case-insensitive literal URL substring. Wildcard syntax is not supported.', + { + type: 'string', + maxLength: constants.SEARCH_STRING_LENGTH_LIMITS.referenceURL, + example: 'vendor.example/security/advisory' + } + ), + disputed: searchRequestProperty( + 'When true, returns records where at least one scope-selected container has the disputed tag. When false, returns records where no selected container has that tag.', + { + type: 'boolean' + } ) }; @@ -248,7 +342,8 @@ const doc = { '@schemas': { SearchRequest: { type: 'object', - description: 'Optional filters and pagination controls for a CVE record search.', + description: + 'Optional filters and pagination controls for a CVE record search. Enum-valued fields are accepted case-insensitively and normalized to the documented canonical values.', additionalProperties: false, properties: searchRequestProperties } diff --git a/server.test.js b/server.test.js index 51ce3c8..be14e86 100644 --- a/server.test.js +++ b/server.test.js @@ -138,7 +138,10 @@ describe('server application bootstrap', () => { expect(mockSearch).toHaveBeenCalledTimes(1); expect(response.body).toStrictEqual({ route: 'search', - params: body + params: { + scope: 'FULL', + ...body + } }); }); diff --git a/utils/constants.js b/utils/constants.js index 85ae9f3..c10c1c9 100644 --- a/utils/constants.js +++ b/utils/constants.js @@ -7,6 +7,7 @@ const MAX_CVSS_METRIC_TOKEN_LENGTH = 32; const MAX_CVE_IDS = 100; const MAX_CVE_ID_LENGTH = 64; const MAX_KEYWORD_SEARCH_TOKENS = 64; +const searchScope = require('./searchScope'); const SEARCH_STRING_LENGTH_LIMITS = Object.freeze({ keywordSearch: 512, cveIds: 4096, @@ -14,7 +15,9 @@ const SEARCH_STRING_LENGTH_LIMITS = Object.freeze({ cpeName: 2048, versionStart: 256, versionEnd: 256, - virtualMatchString: 2048 + virtualMatchString: 2048, + ssvcRole: 256, + referenceURL: 2048 }); const SEARCH_POST_FILTER_CACHE_TTL_MS = 60 * 1000; const SEARCH_POST_FILTER_CACHE_MAX_ENTRIES = 100; @@ -26,7 +29,6 @@ const SEARCH_REQUEST_DEADLINE_MS = 30 * 1000; const SEARCH_READINESS_TIMEOUT_MS = 5 * 1000; const rejectedCveStates = ['REJECT', 'REJECTED', 'Rejected']; -const rejectedCveStateFields = ['cveMetadata.state', 'containers.cna.x_legacyV4Record.CVE_data_meta.STATE']; const cpeVersionRangeTypes = ['including', 'excluding']; const cpeMatchFields = { @@ -62,27 +64,10 @@ const cpeMatchFields = { * versionEnd: string, upper version bound used with virtualMatchString * versionEndType: including or excluding */ -const fields = { - keywordSearch: 'containers.cna.descriptions.value', - cveIds: 'cveMetadata.cveId', - noRejected: rejectedCveStateFields, - cweId: 'containers.cna.problemTypes.descriptions.cweId', - cvssV2Severity: 'containers.cna.metrics.cvssV2_0.baseScore', - cvssV2Metrics: 'containers.cna.metrics.cvssV2_0.vectorString', - cvssV3_0Severity: 'containers.cna.metrics.cvssV3_0.baseSeverity', - cvssV3_0Metrics: 'containers.cna.metrics.cvssV3_0.vectorString', - cvssV3_1Severity: 'containers.cna.metrics.cvssV3_1.baseSeverity', - cvssV3_1Metrics: 'containers.cna.metrics.cvssV3_1.vectorString', - cvssV3Severity: ['containers.cna.metrics.cvssV3_0.baseSeverity', 'containers.cna.metrics.cvssV3_1.baseSeverity'], - cvssV3Metrics: ['containers.cna.metrics.cvssV3_0.vectorString', 'containers.cna.metrics.cvssV3_1.vectorString'], - cvssV4Severity: 'containers.cna.metrics.cvssV4_0.baseSeverity', - cvssV4Metrics: 'containers.cna.metrics.cvssV4_0.vectorString', - cpeName: cpeMatchFields.criteria, - isVulnerable: cpeMatchFields.vulnerable, - virtualMatchString: cpeMatchFields.criteria -}; +const fields = searchScope.getFullFields(); const searchControlParams = [ + 'scope', 'keywordExactMatch', 'resultsPerPage', 'pageNumber', @@ -97,6 +82,7 @@ const searchControlParams = [ ]; const searchQueryParams = [ + 'scope', 'keywordSearch', 'keywordExactMatch', 'cveIds', @@ -124,7 +110,18 @@ const searchQueryParams = [ 'versionStartType', 'versionEnd', 'versionEndType', - 'virtualMatchString' + 'virtualMatchString', + 'ssvcExploitation', + 'ssvcAutomatable', + 'ssvcTechnicalImpact', + 'ssvcRole', + 'ssvcTimestampStartDate', + 'ssvcTimestampEndDate', + 'knownExploited', + 'kevDateAddedStartDate', + 'kevDateAddedEndDate', + 'referenceURL', + 'disputed' ]; module.exports = { diff --git a/utils/errors.js b/utils/errors.js index 4c126e0..921b2d2 100644 --- a/utils/errors.js +++ b/utils/errors.js @@ -5,6 +5,36 @@ function makeError(error, message) { } class errors { + scopeValueError() { + return makeError('SCOPE_VALUE', 'The scope parameter must be FULL, CNA_ONLY, or ADP_ONLY.'); + } + + scopeCompatibilityError(scope, params) { + return makeError( + 'SCOPE_FILTER_INCOMPATIBLE', + `The ${params.join(', ')} parameter${params.length === 1 ? ' is' : 's are'} not supported with scope ${scope}.` + ); + } + + enumValueError(param, allowed) { + return makeError('INVALID_ENUM_VALUE', `The ${param} parameter must be one of: ${allowed.join(', ')}.`); + } + + kevDateRequiresKnownExploitedError() { + return makeError( + 'KEV_DATE_REQUIRES_KNOWN_EXPLOITED', + 'knownExploited must be true when kevDateAddedStartDate or kevDateAddedEndDate is provided.' + ); + } + + knownExploitedInputError() { + return makeError('KNOWN_EXPLOITED_BOOLEAN', 'The knownExploited parameter must be a boolean.'); + } + + disputedInputError() { + return makeError('DISPUTED_BOOLEAN', 'The disputed parameter must be a boolean.'); + } + maxResultsError() { return makeError( 'MAX_RESULTS_LIMIT', diff --git a/utils/kev.js b/utils/kev.js new file mode 100644 index 0000000..fe49131 --- /dev/null +++ b/utils/kev.js @@ -0,0 +1,46 @@ +const searchContainerMetrics = require('./searchContainerMetrics'); +const searchScope = require('./searchScope'); + +const DATE_PARAMS = Object.freeze(['kevDateAddedStartDate', 'kevDateAddedEndDate']); + +function parseStoredDate(value) { + if (typeof value !== 'string') return Number.NaN; + const dateOnly = /^\d{4}-\d{2}-\d{2}$/.test(value) ? `${value}T00:00:00.000Z` : value; + return Date.parse(dateOnly); +} + +function entryMatches(other, filter) { + if (String(other?.type ?? '').toLowerCase() !== 'kev') return false; + if (!other?.content || typeof other.content !== 'object') return false; + if (!filter.dateAddedStartDate && !filter.dateAddedEndDate) return true; + + const dateAdded = parseStoredDate(other.content.dateAdded); + if (!Number.isFinite(dateAdded)) return false; + if (filter.dateAddedStartDate && dateAdded < Date.parse(filter.dateAddedStartDate)) return false; + if (filter.dateAddedEndDate && dateAdded > Date.parse(filter.dateAddedEndDate)) return false; + return true; +} + +function buildPostFilter(params) { + if (params?.knownExploited !== true || !DATE_PARAMS.some((param) => params[param] !== undefined)) return null; + + return { + scope: searchScope.normalizeScope(params.scope), + dateAddedStartDate: params.kevDateAddedStartDate, + dateAddedEndDate: params.kevDateAddedEndDate + }; +} + +function hitMatches(hit, filter) { + return searchContainerMetrics + .getOtherMetrics(hit?._source, filter.scope) + .some((other) => entryMatches(other, filter)); +} + +module.exports = { + DATE_PARAMS, + buildPostFilter, + entryMatches, + hitMatches, + parseStoredDate +}; diff --git a/utils/kev.test.js b/utils/kev.test.js new file mode 100644 index 0000000..44cb8f4 --- /dev/null +++ b/utils/kev.test.js @@ -0,0 +1,35 @@ +const kev = require('./kev'); + +function hit(dateAdded, container = 'adp') { + const metrics = [{ other: { type: 'kev', content: { dateAdded, reference: 'https://example.test' } } }]; + return { + _source: { + containers: container === 'cna' ? { cna: { metrics } } : { adp: [{ metrics }] } + } + }; +} + +describe('KEV semantic filtering', () => { + test('interprets stored date-only values as midnight UTC with inclusive bounds', () => { + expect( + kev.hitMatches(hit('2024-06-15'), { + scope: 'ADP_ONLY', + dateAddedStartDate: '2024-06-15T00:00:00.000Z', + dateAddedEndDate: '2024-06-15T00:00:00.000Z' + }) + ).toBe(true); + }); + + test('rejects dates outside the requested range', () => { + expect( + kev.hitMatches(hit('2024-06-14'), { + scope: 'ADP_ONLY', + dateAddedStartDate: '2024-06-15T00:00:00.000Z' + }) + ).toBe(false); + }); + + test('ignores KEV entries outside the selected scope', () => { + expect(kev.hitMatches(hit('2024-06-15', 'cna'), { scope: 'ADP_ONLY' })).toBe(false); + }); +}); diff --git a/utils/searchContainerMetrics.js b/utils/searchContainerMetrics.js new file mode 100644 index 0000000..61dbcbc --- /dev/null +++ b/utils/searchContainerMetrics.js @@ -0,0 +1,13 @@ +const searchScope = require('./searchScope'); + +function getOtherMetrics(source, scope) { + return searchScope + .getSelectedContainers(source, scope) + .flatMap((container) => (Array.isArray(container?.metrics) ? container.metrics : [])) + .map((metric) => metric?.other) + .filter(Boolean); +} + +module.exports = { + getOtherMetrics +}; diff --git a/utils/searchPostFilters.js b/utils/searchPostFilters.js new file mode 100644 index 0000000..4a7c059 --- /dev/null +++ b/utils/searchPostFilters.js @@ -0,0 +1,90 @@ +const cpe = require('./cpe'); +const searchScope = require('./searchScope'); +const ssvc = require('./ssvc'); +const kev = require('./kev'); + +const CPE_SOURCE_FIELDS = Object.freeze([ + 'containers.cna.cpeApplicability.nodes.cpeMatch.criteria', + 'containers.cna.cpeApplicability.nodes.cpeMatch.vulnerable', + 'containers.cna.cpeApplicability.nodes.cpeMatch.versionStartIncluding', + 'containers.cna.cpeApplicability.nodes.cpeMatch.versionStartExcluding', + 'containers.cna.cpeApplicability.nodes.cpeMatch.versionEndIncluding', + 'containers.cna.cpeApplicability.nodes.cpeMatch.versionEndExcluding' +]); + +function hasCpePostFilters(postFilters) { + return Boolean(postFilters?.cpeName || postFilters?.virtualMatchString || postFilters?.virtualMatchStringRange); +} + +function hasSearchPostFilters(postFilters) { + return hasCpePostFilters(postFilters) || Boolean(postFilters?.ssvc || postFilters?.kev); +} + +function hitMatches(hit, postFilters, cpeEvaluationContext) { + if ( + postFilters?.cpeName && + !cpe.hitAppliesToCpeName(hit, postFilters.cpeName, cpeEvaluationContext, { + requireVulnerable: postFilters.isVulnerable === true + }) + ) { + return false; + } + + if ( + postFilters?.virtualMatchString && + !cpe.hitMatchesVirtualMatchString(hit, postFilters.virtualMatchString, cpeEvaluationContext) + ) { + return false; + } + + if ( + postFilters?.virtualMatchStringRange && + !cpe.hitOverlapsVirtualMatchStringRange(hit, postFilters.virtualMatchStringRange, cpeEvaluationContext) + ) { + return false; + } + + if (postFilters?.ssvc && !ssvc.hitMatches(hit, postFilters.ssvc)) return false; + if (postFilters?.kev && !kev.hitMatches(hit, postFilters.kev)) return false; + + return true; +} + +function filterSearchHits(resp, postFilters) { + const hitsContainer = resp?.data?.hits; + const cpeEvaluationContext = cpe.createEvaluationContext(); + + if (!Array.isArray(hitsContainer?.hits)) { + return { + hitsContainer, + candidateCount: 0, + filteredHits: [] + }; + } + + return { + hitsContainer, + candidateCount: hitsContainer.hits.length, + filteredHits: hitsContainer.hits.filter((hit) => hitMatches(hit, postFilters, cpeEvaluationContext)) + }; +} + +function getCandidateSourceFields(postFilters) { + const fields = ['cveMetadata.cveId']; + if (hasCpePostFilters(postFilters)) fields.push(...CPE_SOURCE_FIELDS); + if (postFilters?.ssvc || postFilters?.kev) { + fields.push( + ...searchScope.resolveContainerPaths('metrics.other', postFilters.ssvc?.scope ?? postFilters.kev?.scope) + ); + } + + return [...new Set(fields)]; +} + +module.exports = { + filterSearchHits, + getCandidateSourceFields, + hasCpePostFilters, + hasSearchPostFilters, + hitMatches +}; diff --git a/utils/searchQueryBuilder.js b/utils/searchQueryBuilder.js index 07894bb..22bf955 100644 --- a/utils/searchQueryBuilder.js +++ b/utils/searchQueryBuilder.js @@ -1,5 +1,9 @@ const constants = require('./constants'); const cpe = require('./cpe'); +const searchScope = require('./searchScope'); +const searchPostFilters = require('./searchPostFilters'); +const ssvc = require('./ssvc'); +const kev = require('./kev'); const { MAX_RESULTS_LIMIT, RESULTS_PER_PAGE } = constants; @@ -7,30 +11,24 @@ const matchParams = Object.keys(constants.FIELDS); const pubDatePath = 'cveMetadata.datePublished'; const modDatePath = 'cveMetadata.dateUpdated'; -const cpeCandidateSourceFields = [ - 'cveMetadata.cveId', - 'containers.cna.cpeApplicability.nodes.cpeMatch.criteria', - 'containers.cna.cpeApplicability.nodes.cpeMatch.vulnerable', - 'containers.cna.cpeApplicability.nodes.cpeMatch.versionStartIncluding', - 'containers.cna.cpeApplicability.nodes.cpeMatch.versionStartExcluding', - 'containers.cna.cpeApplicability.nodes.cpeMatch.versionEndIncluding', - 'containers.cna.cpeApplicability.nodes.cpeMatch.versionEndExcluding' -]; - function buildSearchRequest(params) { + const scope = searchScope.normalizeScope(params?.scope); const processedParams = processParams(params); const resultsPerPage = getResultsPerPage(params); const from = getFrom(params, resultsPerPage); const postFilters = buildPostFilters(params); const searchOptions = { - filters: getFilterFields(processedParams.matchPhrases), + filters: [ + ...getFilterFields(processedParams.matchPhrases, scope), + ...buildSemanticCandidateFilters(params, scope) + ], rangeObjects: buildDateRangeObjects(params), resultsPerPage, from }; const request = { searchOptions, - queryStringObjects: buildQueryStrings(processedParams.queryStrings) + queryStringObjects: buildQueryStrings(processedParams.queryStrings, scope) }; if (Object.keys(postFilters).length) { @@ -71,8 +69,14 @@ function buildOpenSearchBody(searchOptions, queryStringObjects = [], { sourceInc } function buildCpeCandidateSearchBody(searchOptions, queryStringObjects = []) { + return buildPostFilterCandidateSearchBody(searchOptions, queryStringObjects, { + cpeName: true + }); +} + +function buildPostFilterCandidateSearchBody(searchOptions, queryStringObjects = [], postFilters = {}) { return buildOpenSearchBody(searchOptions, queryStringObjects, { - sourceIncludes: cpeCandidateSourceFields, + sourceIncludes: searchPostFilters.getCandidateSourceFields(postFilters), trackTotalHits: false }); } @@ -96,15 +100,27 @@ function buildPostFilters(params) { const cpeNamePostFilter = getCpeNamePostFilter(params); const virtualMatchStringPostFilter = getVirtualMatchStringPostFilter(params); const virtualMatchStringRangePostFilter = getVirtualMatchStringRangePostFilter(params); + const ssvcPostFilter = ssvc.buildPostFilter(params); + const kevPostFilter = kev.buildPostFilter(params); if (cpeNamePostFilter) postFilters.cpeName = cpeNamePostFilter; if (cpeNamePostFilter && isTrueValue(params?.isVulnerable)) postFilters.isVulnerable = true; if (virtualMatchStringRangePostFilter) postFilters.virtualMatchStringRange = virtualMatchStringRangePostFilter; else if (virtualMatchStringPostFilter) postFilters.virtualMatchString = virtualMatchStringPostFilter; + if (ssvcPostFilter) postFilters.ssvc = ssvcPostFilter; + if (kevPostFilter) postFilters.kev = kevPostFilter; return postFilters; } +function buildSemanticCandidateFilters(params, scope) { + const filters = []; + if (ssvc.buildPostFilter(params)) filters.push(buildSsvcCandidateFilter(params, scope)); + if (params.knownExploited === true) filters.push(buildKevCandidateFilter(params, scope)); + if (params.knownExploited === false) filters.push(buildNoKevFilter(scope)); + return filters; +} + function getResultsPerPage(params) { const requested = Number(params?.resultsPerPage); if (Number.isInteger(requested) && requested >= 1 && requested <= MAX_RESULTS_LIMIT) { @@ -228,14 +244,15 @@ function escapeQueryStringTerm(term, { escapeWildcards = false } = {}) { return s.replace(re, '\\$1'); } -function buildQueryStrings(params) { +function buildQueryStrings(params, scope = searchScope.SEARCH_SCOPES.CNA_ONLY) { const queryStringObjects = []; Object.keys(params).forEach((param) => { if (param === 'virtualMatchString') return; - const fieldSpec = constants.FIELDS[param]; + const fieldSpec = searchScope.resolveFields(param, scope); const fieldList = Array.isArray(fieldSpec) ? fieldSpec : [fieldSpec]; + if (!fieldList.length) return; if (param === 'keywordSearch') { queryStringObjects.push({ @@ -260,7 +277,7 @@ function buildQueryStrings(params) { return queryStringObjects; } -function getFilterFields(params) { +function getFilterFields(params, scope = searchScope.SEARCH_SCOPES.CNA_ONLY) { let fieldsToReturn = []; const validFields = Object.keys(constants.FIELDS); @@ -277,8 +294,8 @@ function getFilterFields(params) { Object.keys(params).forEach((param) => { if (!validFields.includes(param)) return; - const fieldSpec = constants.FIELDS[param]; - if (!fieldSpec) return; + const fieldSpec = searchScope.resolveFields(param, scope); + if (!fieldSpec.length) return; if (param === 'noRejected') { if (params[param] === true || params[param] === '') { @@ -288,7 +305,7 @@ function getFilterFields(params) { } if (param === 'cveIds') { - const filter = buildCveIdsFilter(params[param], fieldSpec); + const filter = buildCveIdsFilter(params[param], fieldSpec[0]); if (filter) fieldsToReturn.push(filter); return; } @@ -303,7 +320,7 @@ function getFilterFields(params) { if (isTrueValue(params[param])) { fieldsToReturn.push({ term: { - [fieldSpec]: true + [fieldSpec[0]]: true } }); } @@ -316,13 +333,21 @@ function getFilterFields(params) { return; } + if (param === 'referenceURL') { + fieldsToReturn.push(buildLiteralSubstringAcrossFields(params[param], fieldSpec)); + return; + } + + if (param === 'disputed') { + fieldsToReturn.push(buildDisputedFilter(params[param], fieldSpec)); + return; + } + if (param === 'cvssV2Severity') { const r = cvssV2SeverityToRange(params[param]); if (!r) return; - fieldsToReturn.push({ - range: { [fieldSpec]: r } - }); + fieldsToReturn.push(buildRangeAcrossFields(r, fieldSpec)); return; } @@ -379,6 +404,117 @@ function buildNoRejectedFilter(fieldSpec) { }; } +function buildSsvcCandidateFilter(params, scope) { + const clauses = [ + buildCaseInsensitiveExactAcrossFields('ssvc', searchScope.resolveContainerPaths('metrics.other.type', scope)) + ]; + const exactFields = [ + ['ssvcExploitation', 'metrics.other.content.options.Exploitation'], + ['ssvcAutomatable', 'metrics.other.content.options.Automatable'], + ['ssvcTechnicalImpact', 'metrics.other.content.options.Technical Impact'], + ['ssvcRole', 'metrics.other.content.role'] + ]; + + exactFields.forEach(([param, relativePath]) => { + if (params[param] !== undefined) { + clauses.push( + buildCaseInsensitiveExactAcrossFields( + params[param], + searchScope.resolveContainerPaths(relativePath, scope) + ) + ); + } + }); + + if (params.ssvcTimestampStartDate || params.ssvcTimestampEndDate) { + const bounds = {}; + if (params.ssvcTimestampStartDate) bounds.gte = params.ssvcTimestampStartDate; + if (params.ssvcTimestampEndDate) bounds.lte = params.ssvcTimestampEndDate; + clauses.push( + buildRangeAcrossFields(bounds, searchScope.resolveContainerPaths('metrics.other.content.timestamp', scope)) + ); + } + + return { + bool: { + filter: clauses + } + }; +} + +function buildKevCandidateFilter(params, scope) { + const clauses = [ + buildCaseInsensitiveExactAcrossFields('kev', searchScope.resolveContainerPaths('metrics.other.type', scope)) + ]; + + if (params.kevDateAddedStartDate || params.kevDateAddedEndDate) { + const bounds = {}; + if (params.kevDateAddedStartDate) bounds.gte = params.kevDateAddedStartDate; + if (params.kevDateAddedEndDate) bounds.lte = params.kevDateAddedEndDate; + clauses.push( + buildRangeAcrossFields(bounds, searchScope.resolveContainerPaths('metrics.other.content.dateAdded', scope)) + ); + } + + return { + bool: { + filter: clauses + } + }; +} + +function buildNoKevFilter(scope) { + return { + bool: { + must_not: [ + buildCaseInsensitiveExactAcrossFields( + 'kev', + searchScope.resolveContainerPaths('metrics.other.type', scope) + ) + ] + } + }; +} + +function buildCaseInsensitiveExactAcrossFields(value, fieldSpec) { + const fields = Array.isArray(fieldSpec) ? fieldSpec : [fieldSpec]; + const clauses = fields.map((field) => ({ + wildcard: { + [`${field}.keyword`]: { + value: escapeForWildcard(value), + case_insensitive: true + } + } + })); + + return clauses.length === 1 ? clauses[0] : { bool: { should: clauses, minimum_should_match: 1 } }; +} + +function buildLiteralSubstringAcrossFields(value, fieldSpec) { + const fields = Array.isArray(fieldSpec) ? fieldSpec : [fieldSpec]; + const clauses = fields.map((field) => ({ + wildcard: { + [`${field}.keyword`]: { + value: `*${escapeForWildcard(value)}*`, + case_insensitive: true + } + } + })); + + return clauses.length === 1 ? clauses[0] : { bool: { should: clauses, minimum_should_match: 1 } }; +} + +function buildDisputedFilter(value, fieldSpec) { + const exactFilter = buildCaseInsensitiveExactAcrossFields('disputed', fieldSpec); + if (value === true) return exactFilter; + + return { + bool: { + must_not: [exactFilter] + } + }; +} + function buildCpeNameFilter(cpeName) { const parsedCpe = cpe.parseCpeName(cpeName); if (!parsedCpe.valid) return null; @@ -552,6 +688,15 @@ function buildMatchPhraseAcrossFields(value, fieldSpec) { return clauses.length === 1 ? clauses[0] : { bool: { should: clauses, minimum_should_match: 1 } }; } +function buildRangeAcrossFields(value, fieldSpec) { + const fields = Array.isArray(fieldSpec) ? fieldSpec : [fieldSpec]; + const clauses = fields.map((field) => ({ + range: { [field]: value } + })); + + return clauses.length === 1 ? clauses[0] : { bool: { should: clauses, minimum_should_match: 1 } }; +} + function cvssV2SeverityToRange(sev) { const s = String(sev).toUpperCase(); if (s === 'LOW') return { gte: 0.0, lte: 3.9 }; @@ -564,6 +709,7 @@ module.exports = { buildCpeCandidateSearchBody, buildCveIdsSearchRequest, buildOpenSearchBody, + buildPostFilterCandidateSearchBody, buildQueryStrings, buildSearchRequest, getFilterFields, diff --git a/utils/searchQueryBuilder.test.js b/utils/searchQueryBuilder.test.js index b26f879..acd7477 100644 --- a/utils/searchQueryBuilder.test.js +++ b/utils/searchQueryBuilder.test.js @@ -23,6 +23,7 @@ function prefixShould(prefixes) { describe('searchQueryBuilder buildSearchRequest()', () => { test('buildSearchRequest builds filters, query strings, date ranges, and pagination options', () => { const output = queryBuilder.buildSearchRequest({ + scope: 'CNA_ONLY', keywordSearch: 'remote code', cveIds: 'CVE-2024-0001', pubStartDate: '2024-01-01T00:00:00.000Z', @@ -189,6 +190,7 @@ describe('searchQueryBuilder buildSearchRequest()', () => { test('buildCpeCandidateSearchBody limits source fields for semantic post-filtering', () => { const output = queryBuilder.buildSearchRequest({ + scope: 'CNA_ONLY', keywordSearch: 'kernel', virtualMatchString: 'cpe:2.3:o:linux:linux_kernel', versionStart: '2.2', @@ -233,6 +235,7 @@ describe('searchQueryBuilder buildSearchRequest()', () => { test('buildOpenSearchBody builds the complete standard search request', () => { const output = queryBuilder.buildSearchRequest({ + scope: 'CNA_ONLY', keywordSearch: 'remote code', cweId: 'CWE-79', resultsPerPage: 10, @@ -732,3 +735,134 @@ describe('searchQueryBuilder getFilterFields() function', () => { expect(output).toStrictEqual([]); }); }); + +describe('scope-aware search filters', () => { + test('FULL searches description fields in CNA and ADP containers', () => { + const output = queryBuilder.buildSearchRequest({ + scope: 'FULL', + keywordSearch: 'remote code' + }); + + expect(output.queryStringObjects[0].simple_query_string.fields).toStrictEqual([ + 'containers.cna.descriptions.value', + 'containers.adp.descriptions.value' + ]); + }); + + test('ADP_ONLY searches only ADP CVSS fields', () => { + const output = queryBuilder.buildSearchRequest({ + scope: 'ADP_ONLY', + cvssV4Severity: 'CRITICAL' + }); + + expect(output.searchOptions.filters).toStrictEqual([ + { + term: { + 'containers.adp.metrics.cvssV4_0.baseSeverity.keyword': 'CRITICAL' + } + } + ]); + }); + + test('SSVC filters use bounded semantic candidates with selected metric source fields', () => { + const output = queryBuilder.buildSearchRequest({ + scope: 'ADP_ONLY', + ssvcExploitation: 'ACTIVE', + ssvcRole: 'CISA Coordinator', + resultsPerPage: 10, + pageNumber: 2 + }); + const body = queryBuilder.buildPostFilterCandidateSearchBody( + output.searchOptions, + output.queryStringObjects, + output.postFilters + ); + + expect(output.searchOptions).toMatchObject({ resultsPerPage: 10000, from: 0 }); + expect(output.responseOptions).toStrictEqual({ resultsPerPage: 10, from: 10 }); + expect(output.postFilters.ssvc).toMatchObject({ + scope: 'ADP_ONLY', + exploitation: 'ACTIVE', + role: 'CISA Coordinator' + }); + expect(body._source).toStrictEqual(['cveMetadata.cveId', 'containers.adp.metrics.other']); + expect(JSON.stringify(body.query.bool.filter)).toContain( + 'containers.adp.metrics.other.content.options.Exploitation' + ); + expect(JSON.stringify(body.query.bool.filter)).not.toContain('containers.cna.metrics.other'); + }); + + test('positive KEV dates use semantic candidates and negative KEV is a direct exclusion', () => { + const positive = queryBuilder.buildSearchRequest({ + scope: 'ADP_ONLY', + knownExploited: true, + kevDateAddedStartDate: '2024-01-01T00:00:00.000Z' + }); + const negative = queryBuilder.buildSearchRequest({ + scope: 'ADP_ONLY', + knownExploited: false + }); + + expect(positive.postFilters.kev).toMatchObject({ + scope: 'ADP_ONLY', + dateAddedStartDate: '2024-01-01T00:00:00.000Z' + }); + expect(positive.searchOptions.resultsPerPage).toBe(10000); + expect(negative.postFilters).toBeUndefined(); + expect(negative.searchOptions.resultsPerPage).toBe(20); + expect(JSON.stringify(negative.searchOptions.filters)).toContain('containers.adp.metrics.other.type.keyword'); + }); + + test('knownExploited without date boundaries remains a direct search', () => { + const output = queryBuilder.buildSearchRequest({ + scope: 'FULL', + knownExploited: true + }); + + expect(output.postFilters).toBeUndefined(); + expect(output.searchOptions.resultsPerPage).toBe(20); + expect(JSON.stringify(output.searchOptions.filters)).toContain('metrics.other.type.keyword'); + }); + + test('referenceURL escapes wildcard syntax and uses a server-controlled substring query', () => { + const output = queryBuilder.buildSearchRequest({ + scope: 'FULL', + referenceURL: 'EXAMPLE.test/a*?b' + }); + const clauses = output.searchOptions.filters[0].bool.should; + + expect(clauses).toStrictEqual([ + { + wildcard: { + 'containers.cna.references.url.keyword': { + value: '*EXAMPLE.test/a\\*\\?b*', + case_insensitive: true + } + } + }, + { + wildcard: { + 'containers.adp.references.url.keyword': { + value: '*EXAMPLE.test/a\\*\\?b*', + case_insensitive: true + } + } + } + ]); + }); + + test.each([ + [true, 'should'], + [false, 'must_not'] + ])('disputed=%s builds the expected selected-container tag filter', (value, expectedClause) => { + const output = queryBuilder.buildSearchRequest({ + scope: 'FULL', + disputed: value + }); + const filter = output.searchOptions.filters[0]; + + expect(JSON.stringify(filter)).toContain('containers.cna.tags.keyword'); + expect(JSON.stringify(filter)).toContain('containers.adp.tags.keyword'); + expect(JSON.stringify(filter)).toContain(expectedClause); + }); +}); diff --git a/utils/searchScope.js b/utils/searchScope.js new file mode 100644 index 0000000..c25d1d4 --- /dev/null +++ b/utils/searchScope.js @@ -0,0 +1,170 @@ +const SEARCH_SCOPES = Object.freeze({ + FULL: 'FULL', + CNA_ONLY: 'CNA_ONLY', + ADP_ONLY: 'ADP_ONLY' +}); + +const SEARCH_SCOPE_VALUES = Object.freeze(Object.values(SEARCH_SCOPES)); + +const FIELD_PATHS = Object.freeze({ + keywordSearch: { + cna: ['containers.cna.descriptions.value'], + adp: ['containers.adp.descriptions.value'] + }, + cveIds: { + base: ['cveMetadata.cveId'] + }, + noRejected: { + base: ['cveMetadata.state'], + cna: ['containers.cna.x_legacyV4Record.CVE_data_meta.STATE'] + }, + cweId: { + cna: ['containers.cna.problemTypes.descriptions.cweId'], + adp: ['containers.adp.problemTypes.descriptions.cweId'] + }, + cvssV2Severity: { + cna: ['containers.cna.metrics.cvssV2_0.baseScore'], + adp: ['containers.adp.metrics.cvssV2_0.baseScore'] + }, + cvssV2Metrics: { + cna: ['containers.cna.metrics.cvssV2_0.vectorString'], + adp: ['containers.adp.metrics.cvssV2_0.vectorString'] + }, + cvssV3_0Severity: { + cna: ['containers.cna.metrics.cvssV3_0.baseSeverity'], + adp: ['containers.adp.metrics.cvssV3_0.baseSeverity'] + }, + cvssV3_0Metrics: { + cna: ['containers.cna.metrics.cvssV3_0.vectorString'], + adp: ['containers.adp.metrics.cvssV3_0.vectorString'] + }, + cvssV3_1Severity: { + cna: ['containers.cna.metrics.cvssV3_1.baseSeverity'], + adp: ['containers.adp.metrics.cvssV3_1.baseSeverity'] + }, + cvssV3_1Metrics: { + cna: ['containers.cna.metrics.cvssV3_1.vectorString'], + adp: ['containers.adp.metrics.cvssV3_1.vectorString'] + }, + cvssV3Severity: { + cna: ['containers.cna.metrics.cvssV3_0.baseSeverity', 'containers.cna.metrics.cvssV3_1.baseSeverity'], + adp: ['containers.adp.metrics.cvssV3_0.baseSeverity', 'containers.adp.metrics.cvssV3_1.baseSeverity'] + }, + cvssV3Metrics: { + cna: ['containers.cna.metrics.cvssV3_0.vectorString', 'containers.cna.metrics.cvssV3_1.vectorString'], + adp: ['containers.adp.metrics.cvssV3_0.vectorString', 'containers.adp.metrics.cvssV3_1.vectorString'] + }, + cvssV4Severity: { + cna: ['containers.cna.metrics.cvssV4_0.baseSeverity'], + adp: ['containers.adp.metrics.cvssV4_0.baseSeverity'] + }, + cvssV4Metrics: { + cna: ['containers.cna.metrics.cvssV4_0.vectorString'], + adp: ['containers.adp.metrics.cvssV4_0.vectorString'] + }, + cpeName: { + cna: ['containers.cna.cpeApplicability.nodes.cpeMatch.criteria'] + }, + isVulnerable: { + cna: ['containers.cna.cpeApplicability.nodes.cpeMatch.vulnerable'] + }, + virtualMatchString: { + cna: ['containers.cna.cpeApplicability.nodes.cpeMatch.criteria'] + }, + referenceURL: { + cna: ['containers.cna.references.url'], + adp: ['containers.adp.references.url'] + }, + disputed: { + cna: ['containers.cna.tags'], + adp: ['containers.adp.tags'] + } +}); + +const CPE_SCOPE_PARAMS = Object.freeze([ + 'cpeName', + 'isVulnerable', + 'virtualMatchString', + 'versionStart', + 'versionStartType', + 'versionEnd', + 'versionEndType' +]); + +function normalizeScope(value) { + if (value === undefined) return SEARCH_SCOPES.FULL; + return String(value).trim().toUpperCase(); +} + +function isValidScope(value) { + return SEARCH_SCOPE_VALUES.includes(normalizeScope(value)); +} + +function resolveFields(param, value = SEARCH_SCOPES.FULL) { + const scope = normalizeScope(value); + const paths = FIELD_PATHS[param]; + if (!paths) return []; + + const fields = [...(paths.base ?? [])]; + if (scope === SEARCH_SCOPES.FULL || scope === SEARCH_SCOPES.CNA_ONLY) fields.push(...(paths.cna ?? [])); + if (scope === SEARCH_SCOPES.FULL || scope === SEARCH_SCOPES.ADP_ONLY) fields.push(...(paths.adp ?? [])); + + return [...new Set(fields)]; +} + +function getFullFields() { + return Object.fromEntries( + Object.keys(FIELD_PATHS).map((param) => { + const fields = resolveFields(param); + return [param, fields.length === 1 ? fields[0] : fields]; + }) + ); +} + +function getIncompatibleParams(params) { + if (normalizeScope(params?.scope) !== SEARCH_SCOPES.ADP_ONLY) return []; + return CPE_SCOPE_PARAMS.filter((param) => params?.[param] !== undefined); +} + +function getSelectedContainers(source, value = SEARCH_SCOPES.FULL) { + const scope = normalizeScope(value); + const containers = []; + + if ((scope === SEARCH_SCOPES.FULL || scope === SEARCH_SCOPES.CNA_ONLY) && source?.containers?.cna) { + containers.push(source.containers.cna); + } + + if (scope === SEARCH_SCOPES.FULL || scope === SEARCH_SCOPES.ADP_ONLY) { + const adp = source?.containers?.adp; + if (Array.isArray(adp)) containers.push(...adp); + else if (adp) containers.push(adp); + } + + return containers; +} + +function resolveContainerPaths(relativePath, value = SEARCH_SCOPES.FULL) { + const scope = normalizeScope(value); + const paths = []; + if (scope === SEARCH_SCOPES.FULL || scope === SEARCH_SCOPES.CNA_ONLY) { + paths.push(`containers.cna.${relativePath}`); + } + if (scope === SEARCH_SCOPES.FULL || scope === SEARCH_SCOPES.ADP_ONLY) { + paths.push(`containers.adp.${relativePath}`); + } + return paths; +} + +module.exports = { + CPE_SCOPE_PARAMS, + FIELD_PATHS, + SEARCH_SCOPES, + SEARCH_SCOPE_VALUES, + getFullFields, + getIncompatibleParams, + getSelectedContainers, + isValidScope, + normalizeScope, + resolveContainerPaths, + resolveFields +}; diff --git a/utils/searchScope.test.js b/utils/searchScope.test.js new file mode 100644 index 0000000..a42fc44 --- /dev/null +++ b/utils/searchScope.test.js @@ -0,0 +1,43 @@ +const searchScope = require('./searchScope'); + +describe('searchScope', () => { + test.each([ + [undefined, 'FULL'], + ['full', 'FULL'], + ['cna_only', 'CNA_ONLY'], + ['AdP_OnLy', 'ADP_ONLY'] + ])('normalizes %p to %s', (input, expected) => { + expect(searchScope.normalizeScope(input)).toBe(expected); + expect(searchScope.isValidScope(input)).toBe(true); + }); + + test('resolves base fields in every scope and selected container fields only', () => { + expect(searchScope.resolveFields('cveIds', 'ADP_ONLY')).toStrictEqual(['cveMetadata.cveId']); + expect(searchScope.resolveFields('keywordSearch', 'CNA_ONLY')).toStrictEqual([ + 'containers.cna.descriptions.value' + ]); + expect(searchScope.resolveFields('keywordSearch', 'ADP_ONLY')).toStrictEqual([ + 'containers.adp.descriptions.value' + ]); + expect(searchScope.resolveFields('keywordSearch', 'FULL')).toStrictEqual([ + 'containers.cna.descriptions.value', + 'containers.adp.descriptions.value' + ]); + expect(searchScope.resolveFields('referenceURL', 'FULL')).toStrictEqual([ + 'containers.cna.references.url', + 'containers.adp.references.url' + ]); + expect(searchScope.resolveFields('disputed', 'ADP_ONLY')).toStrictEqual(['containers.adp.tags']); + }); + + test('reports CPE applicability parameters as incompatible with ADP_ONLY', () => { + expect( + searchScope.getIncompatibleParams({ + scope: 'adp_only', + cpeName: 'cpe:2.3:a:vendor:product:1.0', + isVulnerable: false, + versionStart: '1.0' + }) + ).toStrictEqual(['cpeName', 'isVulnerable', 'versionStart']); + }); +}); diff --git a/utils/ssvc.js b/utils/ssvc.js new file mode 100644 index 0000000..3b372f3 --- /dev/null +++ b/utils/ssvc.js @@ -0,0 +1,89 @@ +const searchScope = require('./searchScope'); +const searchContainerMetrics = require('./searchContainerMetrics'); + +const EXPLOITATION_VALUES = Object.freeze(['NONE', 'POC', 'ACTIVE']); +const AUTOMATABLE_VALUES = Object.freeze(['YES', 'NO']); +const TECHNICAL_IMPACT_VALUES = Object.freeze(['PARTIAL', 'TOTAL']); +const PARAMS = Object.freeze([ + 'ssvcExploitation', + 'ssvcAutomatable', + 'ssvcTechnicalImpact', + 'ssvcRole', + 'ssvcTimestampStartDate', + 'ssvcTimestampEndDate' +]); + +function getCaseInsensitiveProperty(object, propertyName) { + if (!object || typeof object !== 'object') return undefined; + const key = Object.keys(object).find((candidate) => candidate.toLowerCase() === propertyName.toLowerCase()); + return key === undefined ? undefined : object[key]; +} + +function getOption(content, optionName) { + if (!Array.isArray(content?.options)) return undefined; + + for (const option of content.options) { + const value = getCaseInsensitiveProperty(option, optionName); + if (value !== undefined) return value; + } + + return undefined; +} + +function equalIgnoreCase(actual, expected) { + return String(actual ?? '').toLowerCase() === String(expected ?? '').toLowerCase(); +} + +function timestampMatches(value, startDate, endDate) { + if (!startDate && !endDate) return true; + const timestamp = Date.parse(value); + if (!Number.isFinite(timestamp)) return false; + if (startDate && timestamp < Date.parse(startDate)) return false; + if (endDate && timestamp > Date.parse(endDate)) return false; + return true; +} + +function entryMatches(other, filter) { + if (!equalIgnoreCase(other?.type, 'ssvc')) return false; + const content = other?.content; + if (!content || typeof content !== 'object') return false; + + if (filter.exploitation && !equalIgnoreCase(getOption(content, 'Exploitation'), filter.exploitation)) return false; + if (filter.automatable && !equalIgnoreCase(getOption(content, 'Automatable'), filter.automatable)) return false; + if (filter.technicalImpact && !equalIgnoreCase(getOption(content, 'Technical Impact'), filter.technicalImpact)) { + return false; + } + if (filter.role && !equalIgnoreCase(content.role, filter.role)) return false; + + return timestampMatches(content.timestamp, filter.timestampStartDate, filter.timestampEndDate); +} + +function buildPostFilter(params) { + if (!PARAMS.some((param) => params?.[param] !== undefined)) return null; + + return { + scope: searchScope.normalizeScope(params.scope), + exploitation: params.ssvcExploitation, + automatable: params.ssvcAutomatable, + technicalImpact: params.ssvcTechnicalImpact, + role: params.ssvcRole, + timestampStartDate: params.ssvcTimestampStartDate, + timestampEndDate: params.ssvcTimestampEndDate + }; +} + +function hitMatches(hit, filter) { + return searchContainerMetrics + .getOtherMetrics(hit?._source, filter.scope) + .some((other) => entryMatches(other, filter)); +} + +module.exports = { + AUTOMATABLE_VALUES, + EXPLOITATION_VALUES, + PARAMS, + TECHNICAL_IMPACT_VALUES, + buildPostFilter, + entryMatches, + hitMatches +}; diff --git a/utils/ssvc.test.js b/utils/ssvc.test.js new file mode 100644 index 0000000..98c4343 --- /dev/null +++ b/utils/ssvc.test.js @@ -0,0 +1,64 @@ +const ssvc = require('./ssvc'); + +function metric(other) { + return { other }; +} + +function hit({ cna = [], adp = [] } = {}) { + return { + _source: { + containers: { + cna: { metrics: cna }, + adp: adp.map((metrics) => ({ metrics })) + } + } + }; +} + +const matchingSsvc = metric({ + type: 'ssvc', + content: { + role: 'CISA Coordinator', + timestamp: '2024-06-15T12:00:00.000Z', + options: [{ Exploitation: 'active' }, { Automatable: 'yes' }, { 'Technical Impact': 'total' }] + } +}); + +describe('SSVC semantic filtering', () => { + test('matches case-insensitive decisions and role with inclusive timestamp bounds', () => { + expect( + ssvc.hitMatches(hit({ adp: [[matchingSsvc]] }), { + scope: 'ADP_ONLY', + exploitation: 'ACTIVE', + automatable: 'YES', + technicalImpact: 'TOTAL', + role: 'cisa coordinator', + timestampStartDate: '2024-06-15T12:00:00.000Z', + timestampEndDate: '2024-06-15T12:00:00.000Z' + }) + ).toBe(true); + }); + + test('does not combine conditions from separate SSVC entries', () => { + const candidate = hit({ + cna: [ + metric({ type: 'ssvc', content: { options: [{ Exploitation: 'active' }] } }), + metric({ type: 'ssvc', content: { options: [{ Automatable: 'yes' }] } }) + ] + }); + + expect( + ssvc.hitMatches(candidate, { + scope: 'CNA_ONLY', + exploitation: 'ACTIVE', + automatable: 'YES' + }) + ).toBe(false); + }); + + test('ignores containers outside the selected scope', () => { + expect(ssvc.hitMatches(hit({ adp: [[matchingSsvc]] }), { scope: 'CNA_ONLY', exploitation: 'ACTIVE' })).toBe( + false + ); + }); +});