Skip to content

UTS: decide how to record a deviation whose spec-correct test can't compile because the SDK lacks the API #562

Open
Open
UTS: decide how to record a deviation whose spec-correct test can't compile because the SDK lacks the API#562

Description

@sacOO7
sacOO7
opened on Oct 8, 2026

Summary

When a UTS spec tests a public API that an SDK doesn't implement, the spec-correct derived test can't be written as is: in a statically typed SDK it doesn't compile. writing-derived-tests.md says such an absence "may itself be a deviation to record", but none of its four deviations.md categories describes this case. Failing Tests are defined as spec-correct tests that are present and env-gated, which needs the test to compile. This issue asks the spec owners to decide where a missing-API deviation goes and how its test is written, so that SDKs and the translator-skill guide (PR 558) can stop improvising.

Background: what writing-derived-tests.md says today

Quotes from uts/docs/writing-derived-tests.md and uts/README.md are from main at 747796f5. Relative links inside quotes are omitted.

Entry fields (Recording deviations, L182-L190):

  1. The spec point (e.g. RSA4b4)
  2. What the spec says — quote or paraphrase the features spec
  3. What the SDK does — concrete observable behaviour
  4. Root cause (if known) — file, function, mechanism
  5. Test impact — which test(s) are affected and how each was handled (env-gated skip, adapted assertion, fail-fast, or skipped stub)
  6. Status (for an SDK deviation) — open bug (to be fixed) vs intentional / SDK-wide deviation (and why it won't be), so it's clear whether an issue should be filed
  7. Resolution (once resolved) — the outcome after the deviation is resolved by analysis or a spec change (e.g. reclassified as a cross-SDK design divergence, or a spec point revised upstream); omit until there is one

Categories (L194-L200):

Deviations are grouped into four sections. Keep all four headings present in every deviations.md, in this order — and mark an empty category *(none)* rather than deleting its heading. [...]

  • UTS Spec Errors — the UTS spec itself is wrong (contradicts the features spec, or is internally inconsistent). The test fails fast pointing here (see "Spec-error fail-fast"); the fix belongs in the spec, not the SDK. Unlike the categories below, this is not an SDK problem.
  • Failing Tests — SDK non-compliance where the spec-correct test is present but skipped (env-gated). These are the primary output — each maps to a potential issue to file.
  • Adapted Tests — SDK non-compliance where the test asserts the SDK's actual behaviour instead of the spec's. It passes, guards against regressions, and documents the deviation. Whenever the actual behaviour is stable and assertable, prefer this and record the test here — a running adapted test is worth more than a spec-correct assertion skipped indefinitely. (Reserve Failing Tests for deviations you expect to be fixed, where preserving the spec-correct assertion as the target matters.)
  • Mock Infrastructure Limitations — tests that can't be implemented due to missing mock capabilities (e.g. msgpack support). These are skipped stubs, not SDK deviations.

This file is valuable output. It gives the SDK team a precise catalogue of spec gaps, each with a failing test that can be turned on once the fix lands.

The patterns (section 3, L114-L134):

Once you've diagnosed a failure (section 2), there are three patterns. Two are for an SDK deviation (section 2c): env-gated skip keeps the spec-correct assertion but skips it unless explicitly enabled, and adapted assertion asserts the SDK's actual behaviour with the spec expectation in a comment. [...]

Env-gated skip (preferred for a deviation you expect to be fixed) — the test contains the correct spec assertion but is skipped by default. An environment variable enables it on demand: [...]

  • When the SDK is fixed, removing the skip guard is the only change needed

Compile first (Phase 1, step 4, L56-L58):

Before moving to evaluation (or declaring the test done in a test-first scenario), make sure the test at least compiles, parses, or passes linting. Syntax errors in the translation are not interesting failures.

Test-first considerations (L233-L240):

  • Write the test to match the spec exactly. Don't preemptively accommodate likely implementation gaps — you don't know what they are yet.
  • Use the skip/pending mechanism of your test framework liberally. Tests that can't run yet should be marked as pending, not commented out.
  • Mock infrastructure may not exist yet. You may need to build it. [...]
  • The deviations file is created during evaluation, not during translation. If there's no implementation to evaluate against, there are no deviations to record yet.

Check the SDK's API surface (L246-L248), the only rule that mentions missing APIs:

Not everything in the UTS pseudocode maps 1:1 to every SDK. Before writing tests, verify that the API exists. An API that is merely named differently is idiomatic translation — use the SDK's name and move on; there is nothing to record (see Idiomatic translation vs genuine deviations). An API that is genuinely missing is different: note it — the test may be not-applicable to this SDK, or the absence may itself be a deviation to record.

Internal APIs at the unit tier (L213-L214) are handled separately: "internal APIs are not standardised across SDKs — a boolean return where the spec models a returned object, an event where it models a return value, a missing accessor — so a white-box unit test adapts to the SDK's internal surface", and "If an internal difference genuinely removes an observable, that is a real deviation, not an adaptation."

Language-inapplicable inputs (uts/README.md L129-L133):

  • Language-inapplicable inputs: a test input that cannot be constructed in a given language (e.g. a non-string map key in JavaScript, where object keys are always coerced to strings; or a null argument where the SDK's signature makes null indistinguishable from "omitted") makes that test — or that table row — not applicable to that SDK. Such omissions are sanctioned and should be noted in the derived test file rather than counted as coverage gaps.

The gap

"Check the SDK's API surface" allows a missing API to be "a deviation to record", but doesn't say in which category, or how the test is written. This issue is about missing public APIs, at any tier. A missing internal API in a white-box unit spec is already covered by the internal-shape rule above.

Going through the categories:

  • Failing Tests is defined as "the spec-correct test is present but skipped (env-gated)". In a statically typed SDK, the lines that call a missing API don't compile, so the spec-correct test can't be present, and Phase 1 step 4 requires the test to compile. "When the SDK is fixed, removing the skip guard is the only change needed" is also false: once the API exists, the test body still has to be translated. (In a dynamic language, or with dynamic access such as TypeScript's as any, the spec-correct test can often be written and env-gated as usual: a missing property reads as undefined and the spec assertion fails. A missing method fails with a missing-name error instead of a spec assertion. See ably-js below.)
  • Adapted Tests needs actual behaviour to assert. A missing API with no substitute has none: there is nothing for a test to pin as a regression guard. (Where the SDK has a different API that does the same job, e.g. compactJson() instead of compact(), an adapted test does fit.)
  • Mock Infrastructure Limitations describes the test shape ("skipped stubs") but is explicitly "not SDK deviations". A missing API that the SDK should have is an SDK gap that should produce an issue to file, which is the job of Failing Tests.
  • UTS Spec Errors doesn't apply: the spec is right; the SDK is incomplete.
  • Language-inapplicable inputs (README) are inputs that can't exist in a language, not an API the SDK hasn't built yet. They are sanctioned omissions, not counted as coverage gaps. A missing API is a coverage gap, unless the spec lets the SDK omit it.

So the only place for a missing public API is Failing Tests, with a test shape that the category's definition excludes. The test-impact field already lists "skipped stub" as a value, but the only category that defines skipped stubs (Mock Infrastructure Limitations) is the one that isn't a deviation.

A related case is an API that the spec lets an SDK omit. For example, the proposed typed-SDK points on the unmerged feature/liveobjects-cross-sdk-types-spec branch say "(RTTS3f) [...] Typed SDKs are not required to implement compact" (and RTTS7d for Instance). On main today, no spec point allows omitting compact(). An omission the spec permits is arguably not a deviation, so it would help if the decision also said how to record it (question 5).

How SDKs handle it today

The three SDKs with UTS-derived suites have each improvised, in different ways:

  • ably-js, TP3h (memberKey not exposed on PresenceMessage): spec-correct test, env-gated. test/uts/deviations.md L161-L169 records it under "Skipped Deviations (RUN_DEVIATIONS=1 to run)". The test (presence_message_types.test.ts L118-L128) is gated with if (!process.env.RUN_DEVIATIONS) this.skip(); and gets past the type checker with expect(typeof (pm as any).memberKey).to.equal('string'). Here "remove the guard once the SDK is fixed" still holds. Note that ably-js's file uses its own headings ("Skipped Deviations", "Adapted Deviations", "Spec Points Under Review", "Mock Infrastructure Limitations"), not the four in writing-derived-tests.md.
  • ably-cocoa, RSL1l1 (no publish-with-params API): Failing Tests, with an env-gated placeholder that fails. Test/UTS/deviations.md L25-L41 records it under Failing Tests. Its test-impact field says: "the spec-correct test cannot compile, so the method carries the spec pseudocode as comments, is gated behind RUN_DEVIATIONS, and fails via Issue.record pointing here when enabled." The test (PublishTests.swift L125-L146) is registered and env-gated. Its body is the spec pseudocode as comments plus Issue.record("RSL1l1: ably-cocoa has no publish-with-params API — see deviations.md (Failing Tests)"). Because the placeholder compiles, RUN_DEVIATIONS still works: enabling it gives a red test that points at the entry. (features.md RSL1l says the publish forms "should take an extra Dict<String, Stringifiable> argument".)
  • ably-cocoa, compact() (RTINS10 / RTPO13 / RTPO13c / RTPO3c1): Adapted Tests. Test/UTS/deviations.md L123-L140 ("compact() is not implemented; adapted to compactJson()") asserts the spec's values against compactJson(). Its status is "intentional deviation (documented API omission per RTTS7d/RTTS3f)".
  • ably-cocoa, RTPO13c5 (cycle reuse in compact()): Adapted Tests that pass trivially. In L222-L236 the reference-identity check has no compactJson() equivalent, so the test "retains the spec pseudocode as comments and passes trivially". Its status is "intentional deviation (documented API omission per RTTS3f)".
  • ably-java, compact() (T-3): Adapted Tests, including RTPO13c5. liveobjects/.../uts/deviations.md L91-L97 ("compact() is not implemented; compactJson() is the typed-SDK equivalent"). It adapts RTPO13c5 to the compactJson() { "objectId": ... } marker, "which makes RTPO13c5 assert the same observable as RTPO14". ably-cocoa leaves the same test trivially passing.
  • Typed Instance members that don't exist on a view: Adapted Tests in both SDKs. ably-java T-1 and T-2 (L75-L89): set/increment/subscribe "do not exist on the base Instance", so the dynamic-API call "is not expressible", and the tests assert a substitute observable (a throwing cast, the wrapped type). ably-cocoa has the same cases (L90-L121, RTINS3b/4d/9c and RTINS12d/14d/16c). Their tests keep the spec pseudocode as comments and pass.
  • Inputs that typed signatures reject: Adapted Tests, not "not applicable". ably-java T-4 (L99-L105 and L115-L121) and ably-cocoa (RTLC12e1, RTLMV4c, RTLCV4a, RTLMV4a/b, RTPO5b/RTPO6b) record these as deviations, although the README treats such inputs as language-inapplicable. (ably-java does skip the null row "per the spec's own language-applicability note".) This is the neighbouring case, and it shows that the boundary isn't clear to translators.
  • ably-java's realtime/rest file (lib/.../uts/deviations.md) has no missing-API entries. Its Failing Tests (RTL13c, RTN16f) are ordinary env-gated assertions.

So, today:

  • a missing required public API is an env-gated spec-correct test in ably-js (TP3h) and an env-gated failing placeholder in ably-cocoa (RSL1l1);
  • the same "no substitute" test shape is a trivially passing Adapted Test in ably-cocoa (RTPO13c5), while ably-java adapts that test to a substitute.

Interim approach in the translator-skill guide (PR 558)

The guide for building uts-to-<lang> translator skills (#558, uts/docs/translator-skills/writing-translator-skills.md on branch uts/translator-skills-docs at a7fc036c) has to tell skills what to do. It picks Failing Tests and labels that choice "Divergence from writing-derived-tests.md". The label is meant to stay until this issue is decided. Section 7.1, L749:

SDK lacks the API entirely, so the spec-correct test can't compile. First decide, per writing-derived-tests.md "Check the SDK's API surface", whether the test is not applicable to this SDK (7.4 case (c)) or the absence is itself a deviation. If it is a deviation: register the test, keep the uncompilable spec lines as comments with a // NOTE:, and mark it skipped (not commented out; writing-derived-tests.md "Test-first considerations"). In evaluate mode, record it under Failing Tests with the test impact "skipped stub" (one of the test-impact values in writing-derived-tests.md "Recording deviations"). Divergence from writing-derived-tests.md, whose Failing Tests are env-gated: a test that can't compile can't be env-enabled, so it stays a skipped stub until the API exists. In translate-only mode, list it in the final report (section 11).

Section 7.3, L762:

When it is written. Per writing-derived-tests.md "Test-first considerations", the file is created during evaluation. In translate-only mode, list mock-capability gaps (Mock Infrastructure Limitations) and missing APIs in the final report (section 11) instead, and record them on the first evaluate run. When the file is created, it has all four headings.

Section 7.4, L764-L772 separates the README's case from this one:

Per uts/README.md, an input that can't be constructed in your language makes that test or table row not applicable: note it in the derived test file, not in deviations.md, and don't count it as a coverage gap. Three cases:
[...]
| (c) Missing API | The SDK has no such method | Not one of the README's cases: decide not-applicable vs deviation per writing-derived-tests.md "Check the SDK's API surface" (7.1) |

For dynamic languages, the guide's compile step (L913) adds: "In dynamic languages a missing SDK API surfaces only at run time: run a type checker (mypy/pyright) over the generated tests where the SDK ships type hints, or report missing APIs as unverified in translate-only mode."

The companion agent skill (uts/skills/uts-to-lang-skill-creator/) makes the rendering a design decision that each generated skill must record (references/phase-4-design-record.md L23):

| D-11 | Pending idiom for a missing API (registered, skipped, not commented out) | 7.1, 7.4 | The framework's static skip or pending marker |

Its final-report template lists "missing APIs" next to not-applicable omissions and mock-capability gaps.

The guide's rendering (a static skip) differs from ably-cocoa's (an env-gated placeholder that fails when enabled). Both compile. The difference is whether RUN_DEVIATIONS=1 brings the gap back as a red test.

Options

(A) Widen Failing Tests to include pending stubs for missing APIs (recommended)

Failing Tests become "spec-correct test present but skipped: env-gated, or, where the SDK lacks a public API, a registered pending stub that keeps the spec lines as comments". The test-impact value is "skipped stub". PR 558 already prescribes this.

  • Pros:
    • No new heading, so the four-heading rule and the layout of existing deviations.md files stay as they are. A few existing entries may be reclassified (see Impact).
    • "Each entry maps to a potential issue to file" stays true, because a missing required API is exactly such an issue.
    • It matches what ably-cocoa did for RSL1l1.
    • "Skipped stub", already a test-impact value, gets a defined meaning for SDK deviations.
  • Cons:
    • Failing Tests then hold two kinds of test, so "remove the skip guard" doesn't fix every entry, and readers have to look at test impact.
    • You can't count the real failing assertions in the category without reading each entry.

(B) Add a fifth category, e.g. "Unimplemented APIs" (or "Missing APIs")

  • Pros:
    • The cleanest meaning: each category keeps one test shape.
    • Easy to report and count separately.
  • Cons:
    • It changes the "all four headings, in this order" rule. Every existing deviations.md that follows that rule would need a new heading (one file in ably-cocoa, two in ably-java; ably-js uses its own headings). So would every skill, checker or report that knows the four headings.
    • An entry has to change category once the API lands and the test becomes an ordinary Failing Test or a pass.

(C) Not a deviation: record it in the test file only, like a language-inapplicable input

  • Pros:
    • No deviations.md change.
    • Matches the README's rule.
  • Cons:
    • It hides real gaps. A required API that the SDK lacks is an SDK bug that should lead to an issue. Burying it in a test-file comment means nobody files it, and it doesn't count as a coverage gap.
    • It is right only when the spec lets the SDK omit the API, which is a different case (question 5).

(D) Env-gated failing placeholder (ably-cocoa's rendering), under Failing Tests

This variant of (A) changes the test shape, not the category. The stub is gated on RUN_DEVIATIONS like any Failing Test. When enabled, it fails at once with a message naming the missing API and the deviations entry. The spec lines stay as comments.

  • Pros:
    • The Failing Tests definition ("present but skipped (env-gated)") stays almost intact.
    • RUN_DEVIATIONS=1 <runner> <filter> stays meaningful for every Failing Test: it reproduces the gap as a red test, which is what an issue filed from the entry needs.
    • ably-cocoa already does this.
  • Cons:
    • The placeholder's red result doesn't test anything; it is only a reminder.
    • "Removing the skip guard is the only change needed" still doesn't hold, because the body has to be translated once the API exists.

(E) Write the spec-correct test through dynamic access, env-gated (ably-js's approach)

Where the language allows it (TypeScript as any, Python attribute access, Kotlin/Java reflection, Objective-C selector dispatch), write the spec-correct test, reaching the missing member dynamically, and env-gate it as an ordinary Failing Test.

  • Pros:
    • No change to the Failing Tests definition.
    • "Removing the skip guard is the only change needed" stays true.
    • When enabled, the test fails on the spec assertion.
  • Cons:
    • It isn't available in every language, or for every API shape. Constructing a missing parameter or overload by reflection is awkward and fragile, and it depends on the SDK's eventual signature, which isn't known yet.
    • The test bypasses the type checker, so once the API exists it should be rewritten to use the real signature anyway.
    • In statically typed SDKs it produces tests that are harder to read than the spec.

(F) Put it under Mock Infrastructure Limitations, renamed "Unimplemented tests"

Listed for completeness and not recommended. It mixes harness gaps (not SDK deviations, nothing to file against the SDK) with SDK gaps (something to file), and that distinction is the reason the category exists.

Recommendation: (A), with (D)'s rendering for the stub, and (E) allowed where the language makes it natural.

  • Widen Failing Tests, and make the stub env-gated and failing when enabled, so RUN_DEVIATIONS reproduces every Failing Test the same way.
  • Where the spec-correct test can simply be written (a dynamic language, or dynamic access that keeps the test readable), write it and env-gate it as usual (E). No stub is needed.
  • If the owners prefer a plain static skip (the PR 558 wording), (A) still works; only the rendering text below changes.

Proposed wording for (A)

Edits to uts/docs/writing-derived-tests.md:

1. Failing Tests category (replace the bullet at L196):

  • Failing Tests — SDK non-compliance where the spec-correct test is present but skipped (env-gated). This includes a missing public API: where the SDK lacks a public API that the spec requires of it, so the spec-correct test can't be written, register the test as a pending stub instead (see "Pending stub (missing API)" in section 3). These are the primary output — each maps to a potential issue to file.

2. New pattern in section 3 (after "Adapted assertion", before "Spec-error fail-fast"). It goes in section 3 so that all deviation renderings are in one place, although a missing API is usually found in Phase 1 ("Check the SDK's API surface"):

Pending stub (missing API) — when the SDK lacks a public API under test and the spec-correct test can't be written (it doesn't compile, or it could only fail with a missing-name error), there is no spec-correct assertion to gate and no actual behaviour to adapt to. Register the test under its normal name and UTS: tag, keep the spec lines that can't be written as comments, and gate it on the same environment variable as other deviations. When enabled, it fails immediately with a message naming the missing API and pointing at the deviations entry:

// DEVIATION: missing API — see deviations.md (Failing Tests)
if (!process.env.RUN_DEVIATIONS) this.skip();
// AWAIT channel.publish(message: ..., params: { "_forceNack": "true" }) FAILS WITH error
// ASSERT error.code == 40099
fail("RSL1l1: SDK has no publish-with-params API — see deviations.md")

If the spec-correct test can be written (for example a property read through dynamic access in a dynamic language), write it and use an env-gated skip instead; no stub is needed. Don't comment out the test or leave it unregistered: an unregistered test is invisible to the runner and to coverage checks. When the API is added, translate the commented spec lines into real code and remove the failing line. The test then either becomes an ordinary env-gated Failing Test (if the new API still deviates) or passes (remove the gate and close out the entry).

Use this only for a public API that the spec requires of this SDK. If the spec lets this SDK omit the API, the test is not applicable (see "Check the SDK's API surface"). If the SDK has a different API that does the same job, use an adapted assertion. A missing internal API in a white-box unit spec is an internal-shape adaptation (see "Idiomatic translation vs genuine deviations", case 3), not a pending stub.

3. Text that counts the patterns:

  • L110 (2c): "using one of the two patterns in section 3 — an env-gated skip or an adapted assertion" → "using one of the SDK-deviation patterns in section 3 — an env-gated skip, an adapted assertion, or (for a missing public API) a pending stub".
  • L116: "there are three patterns. Two are for an SDK deviation" → "there are four patterns. Three are for an SDK deviation", and add a sentence for the pending stub.
  • L175 (decision tree): "Env-gated skip or adapted assertion (section 3)" → "Env-gated skip, adapted assertion, or pending stub for a missing public API (section 3)".
  • L200: "each with a failing test that can be turned on once the fix lands" → "each with a failing test (or, for a missing API, a pending stub) that can be turned on once the fix lands".

4. Entry fields (item 5, L188). The values stay the same; define the term:

  1. Test impact — which test(s) are affected and how each was handled (env-gated skip, adapted assertion, fail-fast, or skipped stub — a pending stub for a missing public API, or a Mock Infrastructure Limitation)

5. Check the SDK's API surface (L248). Replace the last sentence:

An API that is genuinely missing is different. If the spec lets this SDK omit it, the test is not applicable: note that in the derived test file, as for language-inapplicable inputs (uts/README.md). Otherwise, for a public API, the absence is a deviation: write the spec-correct test if the language lets you, or a pending stub (section 3), and record it under Failing Tests. (A missing internal API in a white-box unit spec is an internal-shape adaptation, not a deviation of this kind.)

6. Test-first considerations (add a bullet after "Use the skip/pending mechanism"):

  • An API that doesn't exist yet gets a pending stub, not a commented-out test (section 3, "Pending stub (missing API)"). In test-first work it isn't recorded in the deviations file until evaluation; list it in your summary instead.

7. Filing issues (add to "Each issue should include"):

  • For a missing API, say so in the title (e.g. "RSL1l1: no publish-with-params API"). The reproduction command shows the placeholder failure, not a spec assertion.

Coverage and reports. A pending stub carries its UTS: tag, so a tag audit counts it as translated and present. Test-run summaries (for example a translator skill's final report) should count stubs separately from passing and env-gated tests, e.g. "N translated (P pass, F env-gated, S pending stubs for missing APIs)", so that a missing API is never shown as covered.

Impact

  • deviations.md in SDK repos. (A) needs no heading or format change. Only missing-API entries need a second look:
    • ably-cocoa RSL1l1 already matches (A)+(D), and ably-js TP3h already matches (E).
    • Trivially passing Adapted entries with no substitute (ably-cocoa RTPO13c5) would move to Failing Tests if the API is required, or become not-applicable notes once the spec allows the omission. For compact(), that would happen if the proposed RTTS points are merged.
    • (B) would add a fifth heading to the three files that follow the four-heading rule.
    • Separately, ably-js's file doesn't use the four standard headings at all.
  • Translator-skill guide and skill (PR 558). Once this is decided:
    • remove the Divergence from writing-derived-tests.md clause in section 7.1 (L749);
    • align the rendering (static skip or env-gated placeholder) with the decision;
    • update design-record item D-11 ("The framework's static skip or pending marker") and the acceptance-checklist row that cites D-11;
    • review the "Missing APIs found in translate-only mode" report line (L941) and the 7.1 end-states table (L735-L738), which lists only Pass, SDK deviation and UTS spec error;
    • if (B) is chosen, also change section 7.3's "all four headings" text.
  • The language-inapplicable boundary. The proposed "Check the SDK's API surface" text also settles the "spec allows omitting the API" case, which both typed SDKs currently record as Adapted Tests.

Questions for spec owners

  1. Which option should we take: (A), (B), (C), (A) with (D)'s rendering, (E), or something else?
  2. How should the stub be rendered: a static skip (as in PR 558) or an env-gated placeholder that fails when enabled (as in ably-cocoa)? Should RUN_DEVIATIONS=1 reproduce every Failing Test?
  3. With (A), should the category name or definition change (e.g. "Failing and Pending Tests"), or is changing the definition enough?
  4. Where the language allows dynamic access (ably-js TP3h uses (pm as any).memberKey), should translators prefer writing the spec-correct test that way over a stub? Should that apply in statically typed SDKs too (reflection, selector dispatch), or only where it stays readable?
  5. When the spec lets an SDK omit an API (e.g. the proposed RTTS3f/RTTS7d for compact() in typed SDKs), is the test "not applicable" (recorded only in the test file, per the README rule) or a deviation? Should the README's language-inapplicable rule be extended to cover it?
  6. How does a translator decide that an API is required? Features-spec wording isn't consistently normative (RSL1l says the publish forms "should take an extra [...] argument").
  7. Where an SDK has a substitute API (compactJson() for compact()), is an Adapted Test the right home, as both typed SDKs do today?
  8. Should test-run summaries show pending stubs as a separate count, as proposed above?

References

Activity

  1. sacOO7 commented on Oct 8, 2026

    @sacOO7
    sacOO7
    on Oct 8, 2026
    ContributorAuthor

    Seems, it's better to record it as a deviation with separate section in deviations.md saying Missing Public Api or Missing Api

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions