Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Paths are from the repository root, since that is where you will be working.
| **Kotlin** | Ktor via `BaseService` | `kotlin/sdk/src/commonMain/kotlin/com/basecamp/sdk/generated/services/*.kt` |
| **Python** | httpx via `HttpClient` | `python/src/basecamp/generated/services/*.py` |

All 250 operations across the ~50-service per-SDK layer are generated. Hand-written code is limited to infrastructure:
All `250` operations across the ~50-service per-SDK layer are generated. Hand-written code is limited to infrastructure: <!-- @operation-count -->

| Purpose | Location |
|---------|----------|
Expand Down Expand Up @@ -91,7 +91,7 @@ Pull the andon cord when you see:

All new API coverage starts in `spec/basecamp.smithy`. Before writing SDK code, add operations and shapes to the spec.

`spec/basecamp.smithy` holds 250 worked operations. Copy the nearest one rather than
`spec/basecamp.smithy` holds `250` worked operations. <!-- @operation-count --> Copy the nearest one rather than
working from a skeleton here: it shows the live conventions for naming, `@http` URIs,
pagination traits and shape reuse, and it cannot drift from itself.

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ print(f"Headers: {safe}")
## Retry Behavior

Retry eligibility is decided per *operation*, not per HTTP method. `behavior-model.json` classifies
all 250 operations: the 125 GETs are retryable by method, and 83 mutations are flagged
all `250` operations: the 125 GETs are retryable by method, and 83 mutations are flagged <!-- @operation-count -->
`idempotent: true` — all 52 PUTs, all 24 DELETEs, and 7 POSTs (`CompleteTodo`, `PauseQuestion`,
`SubscribeToCardColumn`, `Subscribe`, `EnableCardColumnOnHold`, `CreateBookmark`, `PrioritizeAssignment`). The other 42 POSTs are attempted exactly once. SPEC.md §7 specifies the
three-gate algorithm and the per-SDK divergences.
Expand Down
6 changes: 3 additions & 3 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -848,7 +848,7 @@ END

### behavior-model.json Retry Patterns

All 250 operations in `behavior-model.json` use `retry_on: [429, 503]`. Three `(max, base_delay_ms)` patterns exist:
All `250` operations in `behavior-model.json` use `retry_on: [429, 503]`. <!-- @operation-count --> Three `(max, base_delay_ms)` patterns exist:
- `(2, 1000)` — most create operations
- `(3, 1000)` — most read/update/delete operations
- `(3, 2000)` — `CreateAttachment`, `CreateCampfireUpload` (file uploads)
Expand Down Expand Up @@ -1415,7 +1415,7 @@ END

### Hop-1 Retry `[conformance]`

The authenticated first hop retries on **network errors plus {429, 502, 503, 504}** — never 500. The set is declared here rather than inherited from anywhere else, and it matches neither of the two sets an SDK already has to hand: it is broader than the per-operation `retry_on` in `behavior-model.json` (`{429, 503}` for all 250 operations, which never governs `DownloadURL` because it has no entry there), and narrower than the error taxonomy's "all 5xx retryable" flag, which would sweep in the 500 this policy deliberately excludes. It is the gateway-error set Go's hand-written `singleRequest` already uses for GETs. Backoff is exponential from a 1-second base with jitter; `Retry-After` is honored on 429. The second hop is exempt: no retry, no auth.
The authenticated first hop retries on **network errors plus {429, 502, 503, 504}** — never 500. The set is declared here rather than inherited from anywhere else, and it matches neither of the two sets an SDK already has to hand: it is broader than the per-operation `retry_on` in `behavior-model.json` (`{429, 503}` for all `250` operations, which never governs `DownloadURL` because it has no entry there), and narrower than the error taxonomy's "all 5xx retryable" flag, which would sweep in the 500 this policy deliberately excludes. It is the gateway-error set Go's hand-written `singleRequest` already uses for GETs. <!-- @operation-count --> Backoff is exponential from a 1-second base with jitter; `Retry-After` is honored on 429. The second hop is exempt: no retry, no auth.

"Network error" means a transport failure, with one carve-out that SDKs inherit from their main GET loop rather than restate: an attempt that exhausted the caller's entire per-attempt time budget (a request timeout) is not retried. The timeout is per attempt, so a retry spends another full budget on the same slowness rather than riding out a blip. Kotlin implements this explicitly; SDKs whose transports surface timeouts indistinguishably from other connection failures retry them.

Expand Down Expand Up @@ -3483,7 +3483,7 @@ Every operation has a `retry` block, including non-idempotent POSTs. For non-ide

### Operation Counts

- Total operations: 250
- Total operations: `250` <!-- @operation-count -->
- Idempotent: 83 (flagged with `idempotent: true`)
- Non-idempotent: 167 (no `idempotent` field, or not present)
- All operations use `retry_on: [429, 503]`
Expand Down
85 changes: 81 additions & 4 deletions scripts/sync-doc-constants.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
# @api-version openapi.json .info.version
# @bc3-pin spec/api-provenance.json .bc3.revision / .bc3.date
# @assertion-types conformance/schema.json
# @operation-count openapi.json count of path × HTTP-method pairs
# .properties.assertions.items.properties.type.enum
#
# Only spans MARKED with an HTML comment are checked. That is deliberate:
Expand Down Expand Up @@ -114,7 +115,11 @@
$stdout.set_encoding(UTF8)
$stderr.set_encoding(UTF8)

LINE_KINDS = %w[api-version bc3-pin].freeze
LINE_KINDS = %w[api-version bc3-pin operation-count].freeze

# The OpenAPI verbs an operation can be keyed under. Anything else in a path
# item (parameters, servers, summary) is not an operation and must not count.
HTTP_METHODS = %w[get put post delete patch head options trace].freeze
BLOCK_KINDS = %w[assertion-types].freeze
KNOWN_KINDS = (LINE_KINDS + BLOCK_KINDS).freeze

Expand Down Expand Up @@ -333,6 +338,59 @@ def check_api_version(span, api_version, source)
end
end

# The count restated by @operation-count spans: every (path, HTTP method) pair
# in openapi.json, which is the same arithmetic AGENTS.md documents and the same
# number the generators report.
def operation_count(doc, source)
paths = dig!(doc, source, "paths")
paths.sum { |_path, ops| ops.count { |method, _| HTTP_METHODS.include?(method) } }
end

# A code span whose ENTIRE content is digits. Bare prose integers are not
# candidates, because the marked lines are full of them — SECURITY.md's states
# 125 GETs and 83 mutations in the same sentence as the total — and a checker
# that read those as the claim would fail on numbers it has no source for.
# Backticks are how the prose says "this one is the derived constant", the same
# device @bc3-pin uses for the SHA.
TICKED_INT_RE = /`(\d+)`/

# The single backticked integer an @operation-count span carries, or nil when the
# span is ambiguous. OCCURRENCES, not distinct values: two spans both reading
# `250` today would both be rewritten the day the count moves, and only one of
# them is the count.
#
# Both the checker and the writer go through here, and that is the point rather
# than tidiness. --write returns before the per-kind checkers run, so a writer
# that rewrote what the checker would have rejected corrupts the file first and
# is then certified by a check that can no longer see the damage.
def sole_ticked_int(text)
ints = text.scan(TICKED_INT_RE).flatten
ints.length == 1 ? ints.first : nil
end

def check_operation_count(span, count, source)
ints = span.text.scan(TICKED_INT_RE).flatten

if ints.empty?
return ["#{span.location}: @operation-count span states no backticked integer — " \
"write the count as `#{count}` so the writer can find it"]
end

# A line that needs a second backticked integer cannot carry this marker. The
# writer refuses such a span rather than rewriting every integer on it: the
# sentence in SECURITY.md states 125 GETs and 83 mutations beside the total,
# and a blanket gsub would turn both into the operation count.
if ints.length > 1
return ["#{span.location}: @operation-count span has #{ints.length} backticked integers " \
"(#{ints.join(', ')}) — exactly one is required, so the writer knows which is the " \
"count. Put the claim on a line of its own, or unticket the others."]
end

return [] if ints.first == count.to_s

["#{span.location}: @operation-count says `#{ints.first}`, #{source} has #{count} operations"]
end

def check_bc3_pin(span, revision, date)
errors = []
text = span.text
Expand Down Expand Up @@ -582,10 +640,15 @@ def check_assertion_types(span, schema_types)

# --- writer ------------------------------------------------------------------

def rewrite_line(kind, line, api_version:, revision:, date:)
def rewrite_line(kind, line, api_version:, revision:, date:, operation_count_value:)
case kind
when "api-version"
line.gsub(ISO_DATE_RE, api_version)
when "operation-count"
# Refuse an ambiguous span instead of rewriting every integer on it. Left
# untouched, it fails the next --check with a message naming the problem;
# rewritten, it would be silently corrupt and then pass.
sole_ticked_int(line) ? line.sub(TICKED_INT_RE, "`#{operation_count_value.call}`") : line
when "bc3-pin"
# Preserve the abbreviation length the prose already chose.
line
Expand All @@ -599,7 +662,18 @@ def rewrite_line(kind, line, api_version:, revision:, date:)
# --- main --------------------------------------------------------------------

def run(mode, openapi)
api_version = dig!(read_openapi(openapi), openapi, "info", "version")
openapi_doc = read_openapi(openapi)
api_version = dig!(openapi_doc, openapi, "info", "version")

# Derived lazily, and only when an @operation-count span actually needs it.
# The gate's own fixtures are minimal OpenAPI documents with no .paths at all,
# and a document without operations is a real failure only for a file that
# claims to count them — computing it eagerly turned every such fixture into
# an error about a constant it never mentions.
op_count_memo = nil
op_count = lambda do
op_count_memo ||= operation_count(openapi_doc, openapi)
end

provenance = read_json("spec/api-provenance.json")
revision = dig!(provenance, "spec/api-provenance.json", "bc3", "revision")
Expand Down Expand Up @@ -684,7 +758,8 @@ def run(mode, openapi)
body = lines[index].chomp("\n")
newline = lines[index].end_with?("\n") ? "\n" : ""
lines[index] = rewrite_line(span.kind, body,
api_version: api_version, revision: revision, date: date) + newline
api_version: api_version, revision: revision, date: date,
operation_count_value: op_count) + newline
end
updated = lines.join
next if updated == original
Expand Down Expand Up @@ -737,6 +812,7 @@ def run(mode, openapi)
when "api-version" then check_api_version(span, api_version, openapi)
when "bc3-pin" then check_bc3_pin(span, revision, date)
when "assertion-types" then check_assertion_types(span, schema_types)
when "operation-count" then check_operation_count(span, op_count.call, openapi)
else []
end
)
Expand All @@ -759,6 +835,7 @@ def run(mode, openapi)
puts " api-version #{api_version}"
puts " bc3-pin #{revision[0, 8]} (#{date})"
puts " assertion-types #{schema_types.length}"
puts " operation-count #{op_count.call}" if spans.any? { |s| s.kind == "operation-count" }
0
else
warn "ERROR: documentation constants have drifted from their sources."
Expand Down
111 changes: 106 additions & 5 deletions scripts/test-doc-constants.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,22 @@
# a reason unrelated to what it tests. The bytes are UTF-8 either way; only the
# tag is wrong. Applied here rather than at each of the five capture sites so a
# sixth cannot reintroduce it.
# THREE operations across two paths, in the --openapi source. `parameters` is a
# path-item key, not an operation, and is here so a counter that walked every
# key would report 4 and fail the positive control.
OP_COUNT = 3
SOURCE_PATHS = {
"/todos" => { "get" => {}, "post" => {}, "parameters" => [] },
"/todos/{id}" => { "get" => {} },
}.freeze

# The in-repo openapi.json is a DECOY with a different operation count, the same
# device DECOY_API_VER uses: if the gate ever read the checkout's copy instead of
# the --openapi argument, every operation-count case would report 5, not 3.
DECOY_PATHS = {
"/decoy" => { "get" => {}, "post" => {}, "put" => {}, "delete" => {}, "patch" => {} },
}.freeze

def utf8(out) = out.dup.force_encoding("UTF-8")

def expect_pass(failures, label, out, status)
Expand Down Expand Up @@ -76,7 +92,10 @@ def expect_absent(failures, label, out, fragment)

def default_files
{
"openapi.json" => JSON.pretty_generate("info" => { "version" => DECOY_API_VER }),
"openapi.json" => JSON.pretty_generate(
"info" => { "version" => DECOY_API_VER },
"paths" => DECOY_PATHS
),
"conformance/schema.json" => JSON.pretty_generate(
"properties" => { "assertions" => { "items" => { "properties" => {
"type" => { "enum" => %w[status header jsonPath] },
Expand All @@ -90,6 +109,7 @@ def default_files
"api-version" => { "SPEC.md" => 1 },
"bc3-pin" => { "COORDINATION.md" => 1 },
"assertion-types" => { "SPEC.md" => 1 },
"operation-count" => { "SPEC.md" => 1 },
}
),
"COORDINATION.md" => <<~MD,
Expand All @@ -102,6 +122,8 @@ def default_files

API_VERSION is `#{API_VER}`. <!-- @api-version -->

The surface is `#{OP_COUNT}` operations across 2 paths. <!-- @operation-count -->

<!-- @assertion-types:begin -->
| Type | Meaning |
|------|---------|
Expand All @@ -124,14 +146,17 @@ def default_files
#
# Layout: base/repo is the git checkout the gate scans; base/openapi-source.json
# is the --openapi argument, deliberately OUTSIDE the checkout.
def run_gate(mode: "--check", mutate: nil, inspect_result: nil, openapi_version: API_VER)
def run_gate(mode: "--check", mutate: nil, inspect_result: nil, openapi_version: API_VER,
openapi_paths: SOURCE_PATHS)
base = Dir.mktmpdir("doc-constants-test")
begin
dir = File.join(base, "repo")
FileUtils.mkdir_p(dir)

source = File.join(base, "openapi-source.json")
File.write(source, JSON.pretty_generate("info" => { "version" => openapi_version }),
File.write(source,
JSON.pretty_generate("info" => { "version" => openapi_version },
"paths" => openapi_paths),
encoding: "UTF-8")

files = default_files
Expand Down Expand Up @@ -168,8 +193,8 @@ def run_gate(mode: "--check", mutate: nil, inspect_result: nil, openapi_version:
end
end

def gate(mutate = nil, openapi_version: API_VER)
run_gate(mutate: mutate, openapi_version: openapi_version)
def gate(mutate = nil, openapi_version: API_VER, openapi_paths: SOURCE_PATHS)
run_gate(mutate: mutate, openapi_version: openapi_version, openapi_paths: openapi_paths)
end

def writer(mutate = nil, &inspect_result)
Expand All @@ -188,6 +213,82 @@ def read_in(dir, rel)
out, status = gate
expect_pass(failures, "crafted valid repo passes", out, status)

# --- @operation-count ----------------------------------------------------------
#
# The count is one jq away from openapi.json and was restated in prose six times
# across four files. A single new operation left five of them stale and took
# three review rounds to reconcile, which is the failure this marker retires.

out, status = gate ->(f) { f["SPEC.md"] = f["SPEC.md"].sub("`#{OP_COUNT}` operations", "`2` operations") }
expect_fail(failures, "operation-count drifted", out, status,
"@operation-count says `2`")

out, status = gate ->(f) { f["SPEC.md"] = f["SPEC.md"].sub("`#{OP_COUNT}` operations", "several operations") }
expect_fail(failures, "operation-count span states no backticked integer", out, status,
"states no backticked integer")

# Backticks are what tell the writer WHICH integer is the claim, so a span with
# two of them is ambiguous rather than merely redundant — it would silently
# rewrite both. SECURITY.md's real sentence names 125 GETs and 83 mutations
# beside the total, so this is the shape that would break it.
out, status = gate lambda { |f|
f["SPEC.md"] = f["SPEC.md"].sub("across 2 paths", "across `2` paths")
}
expect_fail(failures, "operation-count span has two backticked integers", out, status,
"exactly one is required")

# A count that is right for the wrong reason: a path item's non-operation keys
# must not be counted. Adding `parameters` to the second path keeps the real
# count at 3, so a walker that counted every key would now say 5 and fail.
out, status = gate(openapi_paths: {
"/todos" => { "get" => {}, "post" => {}, "parameters" => [] },
"/todos/{id}" => { "get" => {}, "parameters" => [], "summary" => "a summary", "servers" => [] },
})
expect_pass(failures, "path-item keys that are not operations are not counted", out, status)

# The decoy proves the count is read from --openapi, not the checkout: the
# in-repo openapi.json declares five operations, so a gate reading it would
# report 5 against a span that says 3.
out, status = gate ->(f) { f["openapi.json"] = JSON.pretty_generate("info" => { "version" => DECOY_API_VER }, "paths" => {}) }
expect_pass(failures, "operation count comes from --openapi, not the checkout", out, status)

# The writer fixes a drifted count in place, which is the whole point: nobody
# should be hand-editing six restatements again.
writer ->(f) { f["SPEC.md"] = f["SPEC.md"].sub("`#{OP_COUNT}` operations", "`999` operations") } do |out, status, dir|
expect_pass(failures, "writer rewrites a drifted operation count", out, status)
written = read_in(dir, "SPEC.md")
unless written.include?("`#{OP_COUNT}` operations")
failures << "writer: expected the operation count restored to #{OP_COUNT}, got:\n#{written[/^.*operations.*$/]}"
end
# The unticked integer in the same sentence is prose, not the claim, and the
# writer must leave it exactly where it was.
unless written.include?("across 2 paths")
failures << "writer: rewrote an unticked integer it has no source for:\n#{written[/^.*operations.*$/]}"
end
end

# The writer must REFUSE an ambiguous span, not rewrite every integer on it.
# This is the sharp edge: --write returns before the per-kind checkers run, so a
# blanket gsub corrupts the file first and the later check, comparing values that
# are now all identical, certifies the damage. Reproduced on the real SECURITY.md
# sentence before the guard existed: `125` GETs and `83` mutations both became
# `250`, and the check went green.
writer lambda { |f|
f["SPEC.md"] = f["SPEC.md"].sub("across 2 paths", "across `2` paths")
} do |out, status, dir|
expect_pass(failures, "writer exits 0 on an ambiguous operation-count span", out, status)
written = read_in(dir, "SPEC.md")
unless written.include?("across `2` paths")
failures << "writer: rewrote an integer on an ambiguous @operation-count span:\n#{written[/^.*operations.*$/]}"
end
# And having declined, the span must still be rejected rather than left to rot.
out2, status2 = Open3.capture2e({ "DOC_CONSTANTS_ROOT" => dir }, "ruby", GATE, "--check",
"--openapi", File.join(File.dirname(dir), "openapi-source.json"),
chdir: dir)
expect_fail(failures, "check rejects the span the writer declined", out2, status2,
"exactly one is required")
end

# --- @api-version --------------------------------------------------------------

out, status = gate ->(f) { f["SPEC.md"] = f["SPEC.md"].sub(API_VER, "2020-01-01") }
Expand Down
Loading
Loading