docs: Add ADR 0040 - API standards for request, response, and error payloads

[mike-gorman-bitwarden]

🎟️ Tracking

Follow-up to ADR-0036 (#865).

📔 Objective

Adds ADR-0040, "API standards for request, response, and error payloads", with status Proposed. Sets what every request and response shares: the payload envelope (data, meta, and id/type/attributes on every resource), standard fields, JSON value rules (dates, enumerations, whitespace, empty strings and null), naming, content negotiation, method semantics, status codes, exhaustive request validation, and the error format, with the recorded departure from RFC 9457.

[cloudflare-workers-and-pages]

Deploying contributing-docs with    Cloudflare Pages

Latest commit:	6700cfa
Status:	✅  Deploy successful!
Preview URL:	https://a83590bd.contributing-docs.pages.dev
Branch Preview URL:	https://docs-adr-0040-api-payload-st.contributing-docs.pages.dev

View logs

[jrmccannon]

We have already started adopting problem details. Here

[mike-gorman-bitwarden]

Thanks, good to know. That's Bit.HttpExtensions, used for Admin Console validation errors today. Problem Details was one of the options considered (lines 41–43), and lines 238–246 explain why the ADR chose error objects instead: one shape for one or many failures, source for parameters and headers, and per-item failures in best-effort bulk responses. These standards apply only to new APIs in new services, so existing responses like that one don't change.

[jrmccannon]

This RFC is still proposed, not accepted.

[mike-gorman-bitwarden]

RFC 9457 is published (July 2023, replacing RFC 7807). "Proposed Standard" is its IETF maturity level, the same as most RFCs in use, such as RFC 7396 for merge patch. Either way, the ADR doesn't adopt it; it's listed as a considered option.

[Hinton]

This should really be https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.mvc.problemdetails?view=aspnetcore-10.0 which is based on https://www.rfc-editor.org/info/rfc9457.

We get a lot of value leaning into the asp.net standard and it's quite painful to roll your own validation in asp.net.

[mike-gorman-bitwarden]

The validation stays ASP.NET's: annotations, model binding, and .NET 10's minimal-API validation all still run. Only the response format changes, and that's three standard hooks (InvalidModelStateResponseFactory, an IExceptionHandler, and UseStatusCodePages). Done once in Bitwarden.Server.Sdk, that's a few hundred lines, and no service writes any of it. These standards apply only to new APIs in new services, so nothing existing has to change. Lines 238–246 give the reasons for error objects over Problem Details. Happy to talk it through if those don't hold up for you.

[Hinton]

Our clients are notoriously terrible at handling error codes and we need to be very mindful of certain error codes ever routing into client handling.

[mike-gorman-bitwarden]

These standards only cover one service calling another, so these codes go to a service, never to a client. A service that calls another decides what its own callers see, and it shouldn't pass a downstream 403 straight through to a client.

[jrmccannon]

Is the empty string the same as null? This seems to say both, then the other.

[mike-gorman-bitwarden]

Yes, with one exception. Lines 47 and 48 were two opposing options, and the first was chosen, but the list didn't say so. I've marked the chosen option in each group to make that clearer. The rules are at lines 113–114: an empty string is treated as absent, and absent is treated as null. The exception is a partial update, where absent means "leave unchanged" (ADR-0044).

[Hinton]

Is type globally unique, unique per service?

[mike-gorman-bitwarden]

It is unique per service and, in practice, almost always globally unique.

[mkincaid-bw]

I would suggest using CreationDate, RevisionDate, and DeletedDate instead of createdAt, updatedAt, and deletedAt. This would align with the existing fields in the database.

[mike-gorman-bitwarden]

These standards apply only to new APIs in new services whose database does not yet exist. Having said that, I suppose your suggested names are as good as any others but I feel like the names proposed are a bit more fluent as in 'created at [such-n-such time]' and 'created by [such-n-such user]'. Will leave this comment open to see if others have an opinion.

[rkac-bw]

From the database side, four points:

1. Overload and retry semantics. The status table has 429 (caller made too many requests) and 503 (a dependency is unavailable or timed out), but no case for a service shedding load itself, and no Retry-After. Suggest: 429 and 503 SHOULD carry Retry-After; a service that rejects work to protect its own store (for example, approaching its database worker limit) returns 503 with Retry-After; and the ADR states which statuses a caller may retry, and that it honours Retry-After. Without this, every service client invents its own retry rules, and stacked retries multiply load on the database exactly when it is struggling.
2. A timed-out write may have committed. 503 covers "timed out", but after a database timeout the write may still have committed. A caller that retries can duplicate it. That needs idempotency (see the comment on ADR-0042) or a rule that writes which time out are not retried blindly.
3. Validating identifiers owned by another service ("valid … owned by this service or another") is the application-level replacement for a cross-service foreign key. Worth saying that it should be batched where a request carries many IDs, that it is bounded by the request deadline, and that it is a check at write time only: the referenced resource can still be deleted afterwards, so the owning service still needs a delete handler or cleanup.
4. Dates should not be used for concurrency. Today's Cipher conflict check compares RevisionDate with a one-second tolerance, because precision does not survive the round trip (SQL Server datetime2(7) stores 100 ns), and it skips the check when the caller omits the date. As a result, two conflicting edits within a second are not detected. Suggest ADR-0040 says dates are informational and conflict detection uses version (ADR-0044), or else that fractional seconds are preserved exactly.

[mike-gorman-bitwarden]

Thanks.

1. Fair. The 503 row only covered dependencies. I've widened it to cover the service itself being overloaded, which matches RFC 9110. Retry-After comes with those RFCs. Which statuses a caller retries is up to the client (the SDK's resilience handler), not the API contract.
2. Covered under idempotency on ADR-0042.
3. How a service checks references, and what it does when a referenced resource is deleted later, is up to that service and the MSA design. The API standard only says the reference must be valid at the time of the request.
4. Already the case: ADR-0044 requires conflict detection to use version, and comparing updatedAt was a rejected option. These standards only apply to new services, so the Cipher RevisionDate check isn't affected.