Skip to content

Define a stable public search response contract #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdaigneau5 merged 2 commits into from
Sep 10, 2026
Merged

Define a stable public search response contract #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
57bdfa0
feat: define public search response contract
afoote-mitre Sep 10, 2026
8a56e74
feat: warn for pre-1999 CVE IDs
afoote-mitre Sep 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,10 @@ An AWS load balancer can use `/api/ready-check` to stop routing search traffic t

`POST /search` accepts an optional JSON object request body and validates supported fields, boolean parameters, pagination values, CVSS severity values, required parameter combinations, CPE Name requirements, and date-time formats. Enum-valued fields are accepted case-insensitively and normalized to their documented canonical values. A non-empty body must use the `application/json` media type with UTF-8 encoding; unsupported media types or encodings return HTTP 415, and malformed JSON returns HTTP 400. If OpenSearch is unavailable, the endpoint returns HTTP 503 with a `Retry-After` header.

Successful `/search` responses use an API-owned response structure rather than exposing search-backend response objects. The top-level fields are `apiVersion`, `status`, `warnings`, `pagination`, and `records`, in that order. `records` contains complete CVE Records directly, without search-backend hit wrappers or infrastructure metadata. `pagination` contains `page`, `pageSize`, `resultsReturned`, `totalResults`, `totalPages`, and `totalIsExact`. Non-fatal conditions are reported in `warnings` before the potentially long `records` array.

CVE Records do not exist for years before 1999. A syntactically valid pre-1999 value in `cveIds` is therefore a valid search that contributes no matches rather than a request error. The response includes a `CVE_IDS_NOT_RETURNED` warning listing those IDs. When pre-1999 and supported-year IDs are requested together, matching supported-year records are still returned.

Use the optional `scope` field to control which parts of each record participate in matching. Values are case-insensitive: `FULL` (the default) searches base record fields, CNA data, and every ADP provider; `CNA_ONLY` searches base record fields and CNA data; and `ADP_ONLY` searches base record fields and every ADP provider. Scope changes matching only, so successful responses always contain the complete CVE record. CPE applicability filters use CNA data and return HTTP 400 when combined with `ADP_ONLY`.

Date filters use the API's full UTC timestamp format and inclusive boundaries. When both boundaries for publication, modification, SSVC timestamp, or KEV date-added filters are supplied, the start must be before or equal to the end.
Expand Down Expand Up @@ -287,13 +291,13 @@ Each `/search` request owns a lifecycle cancellation signal. When its overall de
Two separate 10,000-record controls apply to `/search`:

- Every requested page must fall within the first 10,000 results. Direct OpenSearch filters still report an exact total that can exceed 10,000, but pages beyond that window are rejected.
- Filters that require semantic post-filtering inspect at most 10,000 OpenSearch candidates. Their reported total and pagination describe only matches found within that candidate set.
- Filters that require semantic post-filtering inspect at most 10,000 search candidates. Their reported total and pagination describe only matches found within that candidate set.

CVSS vector filters, SSVC filters, date-bounded positive `knownExploited` searches, `cpeName`, and `virtualMatchString` searches use semantic post-filtering where OpenSearch cannot guarantee the required same-entry or version-comparison behavior. These searches retrieve up to 10,000 lightweight candidates, apply semantic checks in Node.js, paginate the matching CVE IDs, and fetch the requested page as complete CVE records. Every slash-delimited token supplied to one CVSS vector filter must match a complete token in the same eligible vector; a base token such as `AV:N` does not match a modified token such as `MAV:N`.

The reported total and pagination are complete only within that bounded candidate set; qualifying records beyond the first 10,000 OpenSearch candidates may not be represented. This preserves the existing bounded post-filter behavior until PIT-based pagination is implemented. A `knownExploited` filter without date boundaries uses a direct OpenSearch query and retains ordinary exact-total behavior.
The reported total and pagination are complete only within that bounded candidate set; qualifying records beyond the first 10,000 search candidates may not be represented. When a semantic search reaches that limit, the response sets `pagination.totalIsExact` to `false` and includes a `SEMANTIC_CANDIDATE_LIMIT_REACHED` warning. Semantic searches that remain below the limit and direct searches set `totalIsExact` to `true`. This preserves the existing bounded post-filter behavior until PIT-based pagination is implemented. A `knownExploited` filter without date boundaries uses a direct query and retains ordinary exact-total behavior.

`/search` orders OpenSearch results by CVE ID so repeated shallow-page requests use a deterministic sort. The response `pageSize` preserves the requested `resultsPerPage` value, while `resultsReturned` reports the number of records actually returned and may be smaller for a partial or out-of-range page. A stable sort does not provide a point-in-time snapshot, so index updates between separate page requests can still shift results until PIT-based pagination is implemented.
`/search` orders search results by CVE ID so repeated shallow-page requests use a deterministic sort. The response `pagination.pageSize` preserves the requested `resultsPerPage` value, while `pagination.resultsReturned` reports the number of records actually returned and may be smaller for a partial or out-of-range page. A stable sort does not provide a point-in-time snapshot, so index updates between separate page requests can still shift results until PIT-based pagination is implemented.

The API does not cache, coalesce, queue, or rate-limit searches in process. Each request executes its own required OpenSearch operations. Public rate enforcement, admission control, shared caching, and durable metrics collection must be supplied by the deployment infrastructure. The optional aggregate telemetry described above writes diagnostic events to standard output for collection by the deployment logging system.

Expand Down
127 changes: 69 additions & 58 deletions api-docs/openapi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"CVE Record"
],
"summary": "Retrieves all matching CVE records after applying the JSON request body as filters",
"description": " <h2>Expected Behavior</h2> <p>Retrieves all matching CVE records after applying the JSON request body as filters</p> <h2>Result Counts and Pagination</h2> <p>Direct searches report the exact number of matching records, including totals greater than 10,000, but requested pages must remain within the first 10,000 results.</p> <p>CVSS vector filters, SSVC filters, date-bounded positive knownExploited searches, cpeName, and virtualMatchString use semantic post-filtering. These searches evaluate at most 10,000 OpenSearch candidates, so their reported totals and pages include only matches found within that candidate set.</p>",
"description": " <h2>Expected Behavior</h2> <p>Retrieves all matching CVE records after applying the JSON request body as filters</p> <h2>Result Counts and Pagination</h2> <p>Direct searches report the exact number of matching records, including totals greater than 10,000, but requested pages must remain within the first 10,000 results.</p> <p>CVSS vector filters, SSVC filters, date-bounded positive knownExploited searches, cpeName, and virtualMatchString use semantic post-filtering. These searches evaluate at most 10,000 search candidates, so their reported totals and pages include only matches found within that candidate set. A warning is returned when that limit is reached.</p>",
"operationId": "search",
"responses": {
"200": {
Expand Down Expand Up @@ -73,7 +73,7 @@
}
},
"502": {
"description": "OpenSearch returned an invalid or unexpected response",
"description": "The search backend returned an invalid or unexpected response",
"content": {
"application/json": {
"schema": {
Expand All @@ -83,7 +83,7 @@
}
},
"503": {
"description": "OpenSearch is unavailable",
"description": "The search service is temporarily unavailable",
"content": {
"application/json": {
"schema": {
Expand All @@ -93,15 +93,15 @@
},
"headers": {
"Retry-After": {
"description": "Seconds before retrying an unavailable OpenSearch dependency",
"description": "Seconds before retrying the unavailable search service",
"schema": {
"type": "integer"
}
}
}
},
"504": {
"description": "Search exceeded its OpenSearch or overall processing deadline",
"description": "Search exceeded its processing deadline",
"content": {
"application/json": {
"schema": {
Expand Down Expand Up @@ -403,62 +403,39 @@
}
]
},
"SearchHit": {
"SearchWarning": {
"type": "object",
"additionalProperties": true,
"additionalProperties": false,
"required": [
"_source"
"code",
"message"
],
"properties": {
"_id": {
"code": {
"type": "string"
},
"_source": {
"type": "object",
"additionalProperties": true
}
}
},
"SearchHits": {
"type": "object",
"additionalProperties": true,
"required": [
"total",
"hits"
],
"properties": {
"total": {
"type": "object",
"additionalProperties": true,
"required": [
"value"
],
"properties": {
"value": {
"type": "integer",
"minimum": 0,
"description": "For direct searches, the exact number of matching records, which may be greater than 10,000. For semantic post-filtered searches, the number of matches found within the bounded 10,000-candidate set."
}
}
"message": {
"type": "string"
},
"hits": {
"cveIds": {
"type": "array",
"description": "Requested CVE IDs associated with this warning, when applicable.",
"items": {
"$ref": "#/components/schemas/SearchHit"
"type": "string"
}
}
}
},
"SearchResponse": {
"SearchPagination": {
"type": "object",
"additionalProperties": false,
"required": [
"page",
"pageSize",
"resultsReturned",
"totalResults",
"totalPages",
"status",
"data"
"totalIsExact"
],
"properties": {
"page": {
Expand All @@ -476,31 +453,65 @@
"minimum": 0,
"description": "Number of CVE records actually returned in this response."
},
"totalResults": {
"type": "integer",
"minimum": 0,
"description": "For direct searches, the exact number of matching records, which may be greater than 10,000. For semantic post-filtered searches, the number of matches found within the bounded 10,000-candidate set."
},
"totalPages": {
"type": "integer",
"minimum": 0,
"description": "Pages derived from the reported total and requested page size. Semantic post-filtered searches derive this value from matches found within the bounded 10,000-candidate set."
"description": "Pages derived from totalResults and pageSize. Semantic post-filtered searches derive this value from matches found within the bounded 10,000-candidate set."
},
"totalIsExact": {
"type": "boolean",
"description": "True when totalResults represents every match. False when a semantic search reaches the 10,000-candidate limit and additional matches may exist."
}
}
},
"CveRecord": {
"type": "object",
"additionalProperties": true,
"description": "A complete CVE Record without search-backend transport metadata."
},
"SearchResponse": {
"type": "object",
"additionalProperties": false,
"required": [
"apiVersion",
"status",
"warnings",
"pagination",
"records"
],
"properties": {
"apiVersion": {
"type": "string",
"enum": [
"1.0"
],
"description": "Version of the public search response contract."
},
"status": {
"type": "string",
"enum": [
"ok"
]
},
"data": {
"type": "object",
"additionalProperties": true,
"required": [
"hits"
],
"properties": {
"took": {
"type": "number",
"minimum": 0
},
"hits": {
"$ref": "#/components/schemas/SearchHits"
}
"warnings": {
"type": "array",
"description": "Non-fatal conditions that may affect how the results should be interpreted.",
"items": {
"$ref": "#/components/schemas/SearchWarning"
}
},
"pagination": {
"$ref": "#/components/schemas/SearchPagination"
},
"records": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CveRecord"
}
}
}
Expand Down Expand Up @@ -552,7 +563,7 @@
},
"SearchRequest": {
"type": "object",
"description": "Optional filters and pagination controls for a CVE record search. Enum-valued fields are accepted case-insensitively and normalized to the documented canonical values. When both boundaries of a date range are supplied, the start must be before or equal to the end. Direct searches report exact totals even when they exceed 10,000, although requested pages must remain inside that window. CVSS vector filters, SSVC filters, date-bounded positive knownExploited searches, cpeName, and virtualMatchString use semantic post-filtering. They evaluate at most 10,000 OpenSearch candidates, so reported totals and pages include only matches found within that candidate set.",
"description": "Optional filters and pagination controls for a CVE record search. Enum-valued fields are accepted case-insensitively and normalized to the documented canonical values. When both boundaries of a date range are supplied, the start must be before or equal to the end. Direct searches report exact totals even when they exceed 10,000, although requested pages must remain inside that window. CVSS vector filters, SSVC filters, date-bounded positive knownExploited searches, cpeName, and virtualMatchString use semantic post-filtering. They evaluate at most 10,000 search candidates, so reported totals and pages include only matches found within that candidate set.",
"additionalProperties": false,
"properties": {
"scope": {
Expand Down Expand Up @@ -591,7 +602,7 @@
"type": "string",
"maxLength": 4096,
"example": "CVE-2024-1234,CVE-2024-5678",
"description": "Returns one or more CVE records identified by a comma-separated list of CVE IDs. Limited to 100 CVE IDs, 64 characters per ID, and 4096 characters overall."
"description": "Returns one or more CVE records identified by a comma-separated list of CVE IDs. CVE Records do not exist for years before 1999; those IDs produce no matches and are identified by a non-fatal response warning without preventing other requested IDs from matching. Limited to 100 CVE IDs, 64 characters per ID, and 4096 characters overall."
},
"noRejected": {
"oneOf": [
Expand Down
Loading
Loading