From b8bb15c05d14b2865d236701dcc38756b67f5b74 Mon Sep 17 00:00:00 2001 From: Tomas Grosup Date: Fri, 9 Oct 2026 13:59:44 +0200 Subject: [PATCH] docs: Capture release-note editorial and validation guidance Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/skills/release-notes/SKILL.md | 18 ++++++ .../references/changes-schema.md | 2 +- .../references/editorial-rules.md | 56 +++++++++++++++---- .../references/format-template.md | 27 ++++++--- .../release-notes/references/pr-layout.md | 17 +++++- .../release-notes/references/quality-bar.md | 36 +++++++++--- .github/skills/review-release-notes/SKILL.md | 27 ++++++--- .github/skills/validate-code-samples/SKILL.md | 28 +++++++--- 8 files changed, 166 insertions(+), 45 deletions(-) diff --git a/.github/skills/release-notes/SKILL.md b/.github/skills/release-notes/SKILL.md index f0380126812..6aa90040343 100644 --- a/.github/skills/release-notes/SKILL.md +++ b/.github/skills/release-notes/SKILL.md @@ -28,6 +28,24 @@ changes. Do not substitute a schema stub or defer feature selection to the component PRs. Give component writers the relevant candidate entries with their scores and reasons rather than a slim list of titles and labels. +Read previous component announcements before selecting or rewriting features. +Use their voice, structure, length, and selection as references. +Audit the pinned shipped changes and source tests. Apply the [quality bar](references/quality-bar.md) +for omission discovery and explicitly requested catch-up coverage. +Follow the [editorial rules](references/editorial-rules.md) for naming, +audience, evidence, ordering, and companion reports. + +Use the [format template](references/format-template.md) for article-level +preview setup and accurate section labels. Preserve each feature's actual +requirements and metadata-derived safety notices. + +Validate claims through `validate-code-samples`, but keep temporary projects, +checkers, and results in session or scratch storage by default. +Commit maintained fixtures only when explicitly requested or already agreed +in the task scope. Follow [PR layout](references/pr-layout.md) for publication +scope and source-branch updates. Bound research to the changed claims. +For prose-only edits, run the smallest existing documentation checks, not code builds. + Before finalizing the milestone, run `review-release-notes` against the `changes.json`, `features.json`, and completed component drafts on their branches (or the base branch after they merge). Resolve noteworthy omissions diff --git a/.github/skills/release-notes/references/changes-schema.md b/.github/skills/release-notes/references/changes-schema.md index d87e06424ab..0c73dfb31a0 100644 --- a/.github/skills/release-notes/references/changes-schema.md +++ b/.github/skills/release-notes/references/changes-schema.md @@ -231,4 +231,4 @@ The markdown release notes are a curated subset informed by these JSON files: - `features.json` helps rank which shipped changes are worth calling out - Markdown only covers features worth calling out - The agent reads `changes.json` and/or `features.json` to decide what to write about -- If a feature isn't in `changes.json`, it must not appear in the markdown +- Claims of newly shipped changes must trace to this milestone's `changes.json`. See [the fidelity rule](quality-bar.md#the-fidelity-rule) for explicitly requested catch-up coverage and full-cycle reports. diff --git a/.github/skills/release-notes/references/editorial-rules.md b/.github/skills/release-notes/references/editorial-rules.md index 89a0f28c7b7..6eec23935da 100644 --- a/.github/skills/release-notes/references/editorial-rules.md +++ b/.github/skills/release-notes/references/editorial-rules.md @@ -8,6 +8,7 @@ Tone, attribution, and content guidelines for .NET release notes. - ✅ `ProcessExitStatus provides a unified representation of how a process terminated.` - ❌ `Previously, there was no way to determine how a process terminated.` - When context about the prior state is needed, keep it brief — one clause, then pivot to the new capability +- Celebrate important capabilities through concrete developer possibilities. Put benefit-focused narrative in the body, not invented feature branding in headings. - **Don't editorialize beyond the facts** — state what concretely changed ("enabled by default", "no longer requires opt-in") rather than making claims you can't back up ("ready for production use", "signals maturity"). If a PR removes a preview attribute, say that. Don't interpret it as a promise. - ✅ `Runtime-async is now enabled by default for anyone targeting net11.0.` - ❌ `This signals that runtime-async is ready for production use.` @@ -33,7 +34,12 @@ Tone, attribution, and content guidelines for .NET release notes. ## Entry naming -- Prefer a **brief description** of what the feature does over the API name alone +- Use the **established feature name** when one exists, especially for long-running preview features. If `release-notes/features.json` lists an `official_name`, use that in headings and prose. Treat aliases as match-only metadata, not as the default wording. + - ✅ `## Unsafe Evolution remains a preview feature in .NET 11` + - ✅ `## Unsafe Evolution adds clearer diagnostics in Preview 3` + - ❌ `## Memory Safety v2 adds clearer diagnostics in Preview 3` + - ❌ `## Unsafe code adds clearer diagnostics and annotations` +- When no established name exists, prefer a **brief description** of what the feature does over the API name alone - ✅ `## Support for Zstandard compression` - ✅ `## Faster time zone conversions` - ❌ `## ZstandardStream` @@ -45,25 +51,27 @@ Tone, attribution, and content guidelines for .NET release notes. - Avoid anthropomorphic or club-like transition verbs such as `joins` when a more literal term is available. Prefer `moves to`, `is now in`, `adds support for`, or `supports`. - ✅ `## Zstandard moved to System.IO.Compression and ZIP reads validate CRC32` - ❌ `## Zstandard joins System.IO.Compression and ZIP reads validate CRC32` -- Use the **established feature name** when one exists, especially for long-running preview features. If `release-notes/features.json` lists an `official_name`, use that in headings and prose. Treat aliases as match-only metadata, not as the default wording. - - ✅ `## Unsafe Evolution remains a preview feature in .NET 11` - - ✅ `## Unsafe Evolution adds clearer diagnostics in Preview 3` - - ❌ `## Memory Safety v2 adds clearer diagnostics in Preview 3` - - ❌ `## Unsafe code adds clearer diagnostics and annotations` - Keep headings concise — 3–8 words ## Benchmarks -- Use **exact data** from PR descriptions — never round, approximate, or paraphrase -- State what was measured and the hardware/workload context +- Use source-backed measurements. Preserve exact values when quoting raw results. +- Accurate author-provided summaries can use rounded values for readability. Check them against the source and avoid invented precision. +- Name the metric: allocated bytes, allocation counts, retained heap, peak memory/RSS, CPU time, or elapsed time. +- State the workload, hardware, baseline, and comparison cohort. Distinguish source-built payloads from official SDKs. Identify changed compiler, library, runtime, or packaging inputs. - Include specific before/after measurements when compelling +- Preserve regressions and limitations. Do not sum unrelated workload deltas or claim universal speedups or future performance. - Do **not** embed full BenchmarkDotNet tables — summarize in prose +Separate public-library optimizations from compiler, compiler services (FCS), and IDE memory improvements. +FSharp.Core optimizations can benefit compiled applications. +State each audience and adoption path. Group related compiler changes across PRs into coherent categories with representative evidence. + ## What to include - **Shared audience filter** — apply the 80/20 rule from `editorial-scoring`; don't redefine a competing threshold here. Keep narrower items only when the broader audience can still see why they matter. -- **The two-sentence test** — if you can only write two sentences about a feature, it's probably an engineering fix, not a feature. Cut it. A community contribution or breaking change can lift a borderline entry, but "fixed an internal bug that happened to be visible" is not a feature. -- **Headlines should convey value** — a heading like "GC regions on macOS" doesn't tell the reader whether this is good or bad. Prefer headings that hint at the benefit: "GC regions enabled on macOS" or "Server GC memory model now available on macOS." +- **The two-sentence test** — use a thin explanation as a signal to inspect reader value, not an automatic length cutoff. Apply the shared rubric to compact capabilities and author-facing foundations. An internal fix alone does not justify a feature. +- **Headlines should convey value** — preserve established names first. Without an established name, prefer headings that state the capability or benefit. A heading like "GC regions on macOS" doesn't tell the reader whether this is good or bad. Prefer "GC regions enabled on macOS" or "Server GC memory model now available on macOS." - **Product-boundary rule** — exclude higher-level IDE, editor, or design-time tooling features from product notes unless the release is specifically about that tooling surface. For example, a Razor editor code action should not be presented as an ASP.NET Core feature just because it is adjacent to the web stack. - **TODO for borderline entries** — when a feature might deserve inclusion but you lack data to justify it (benchmark numbers, real-world impact, user demand), keep the entry but add an HTML `` comment asking for the missing information. This is better than silently including a vague claim or silently cutting something that might matter. The TODO should state what's needed and link to the PR where the data might live. - **Breaking changes are separate from hype** — a breaking change can be important even when it is not exciting. Keep the score honest; use `breaking_changes: true` to preserve a short callout instead of inflating the item into a headline feature. @@ -71,7 +79,7 @@ Tone, attribution, and content guidelines for .NET release notes. ## Feature ordering -Order features using three tiers, applied in order: +Follow explicit user priorities first. Otherwise, use these three tiers: 1. **Broad interest first** — features most developers will care about go at the top. A new `RegexOptions` value ranks above an ARM64-only instruction set. 2. **Cluster related features** — group related items together even if they differ in importance. All Regex work in one block, all System.Text.Json work in another, all JIT work together. Readers scan by area. @@ -79,6 +87,8 @@ Order features using three tiers, applied in order: Use PR and issue reaction counts as a signal for tier 1, but apply judgment — a niche feature with 100 reactions may still rank below a broadly useful one with 10. +Keep the opening list and body in the same order. Match section length to importance and predecessor articles, not a fixed feature quota. + ## Community attribution ### Inline @@ -153,6 +163,9 @@ If the fix is for a feature that was never really announced or is still obscure ## Preview-to-preview deduplication +Read the published predecessor articles, not only the current draft or PR titles. +Omit unchanged material and unnecessary carry-forward reminders, including preview-status reminders. + When prior previews already documented a feature, don't repeat the same information. But **do** document significant state changes in the current preview. A feature that was introduced as opt-in in P1 and is now enabled by default in P3 is new news — document the change in state, not the feature from scratch. Ask: "What changed about this feature since the last preview's release notes?" If the answer is meaningful to users (enabled by default, no longer experimental, major perf improvement, new sub-features), write about that. If the answer is just "more PRs landed in the same area," skip it. @@ -164,6 +177,27 @@ Examples: - ❌ Repeating the full explanation of what runtime-async is from P1 (already documented) - ❌ "More JIT optimizations landed this preview." (no specific news) +## Companion reports and supplied articles + +- Keep a useful feature summary when adding a deeper report. Add a concise sibling link that explains the report's value. +- When requested or important, add an inline read-more link in the opening TOC without changing feature order. +- Check supplied article packages for required assets, all references, and provenance before import. +- Copy only the approved article and required media. Follow the existing media layout and retain stable source and citation links. +- Do not run arbitrary supplied generators. Publication approval does not include datasets, generators, READMEs, archives, or experiment scaffolding. +- Report missing methodology or raw-result references while the user can retrieve them. Do not invent evidence URLs. +- If evidence remains missing, obtain explicit publication approval and record the limits. That approval applies only to that report. +- Describe author-provided or source-built measurements accurately. Publication approval alone does not establish independent verification or public reproducibility. + +Keep the milestone index navigation-only unless explicitly requested otherwise. +Follow [the index contract](format-template.md#readmemd-index-file), not an inferred prohibition from one observed example. + +## Link validation + +- Check relative links, heading anchors, images, remote PR/source targets, and redirects. +- If GitHub page checks are blocked, use the authenticated API to establish object existence. +- Do not count ignored targets or HTTP 429 as confirmed availability. Record unavailable checks separately from passed checks. +- After renaming headings or moving or deleting files, update affected references and relative media paths. + ## Filtered features When you cut a feature for failing the 20/80 rule or two-sentence test, record it in an HTML comment block in the output file. This creates a learning record — future runs and human reviewers can see what was considered and why it was excluded. diff --git a/.github/skills/release-notes/references/format-template.md b/.github/skills/release-notes/references/format-template.md index 76ba54ebccd..a18073eaf92 100644 --- a/.github/skills/release-notes/references/format-template.md +++ b/.github/skills/release-notes/references/format-template.md @@ -94,8 +94,8 @@ when generating and completing the index: 1. Preallocate an unlinked entry for every expected component file listed in `component-mapping.md` on the base branch, including components without noteworthy changes. Do not link a file that exists only on another branch. -2. Each component PR links exactly its own entry when adding the matching file - (and any matching validation samples). Before the base PR leaves draft, +2. Each component PR links exactly its own entry when adding the matching file. + Before the base PR leaves draft, verify that every expected component file exists and is linked exactly once. 3. Keep Libraries, Runtime, SDK, MSBuild, and NuGet before the grouped sections. 4. Use **Languages** for language release notes and @@ -131,11 +131,22 @@ Known component docs links: ## Section rules 1. **TOC at top** — every feature gets a linked entry -2. **One paragraph of context** — what the feature does and why it matters in concrete terms, with PR/issue links; avoid inferred feelings or marketing-style claims -3. **Code sample** — show the feature in use -4. **Feature ordering** — highest customer impact first +2. **Context proportional to importance** — name the audience and concrete developer possibilities, with PR/issue links. Positive benefit-focused narrative is welcome when evidence supports it. +3. **Useful scenario** — show substantial supported usage, or representative measurements for default-enabled and grouped performance changes. Validation projects stay in scratch storage unless maintained fixtures are in scope. +4. **Feature ordering** — explicit user priorities first, otherwise reader impact. Keep the TOC and body aligned. 5. **Upgrade guidance near the end** — keep version-to-version breaking changes separate from changes affecting only preview users. `breaking_changes: true` flags migration significance, not which section an item belongs in; see below. -6. **Preview feature callout** — when a feature is listed in `release-notes/features.json`, start its section with the standard blockquote callout from that file +6. **Preview requirements** — apply each feature's actual opt-ins and metadata-derived safety notices, as described below. + +### Preview setup + +Define shared component-specific opt-ins once near the article opening. +For example, F# language-version setup can apply to several sections. +Label affected sections **Preview** or refer to that setup without repeating XML or CLI flags. +Keep default-enabled improvements distinct and state any different per-feature requirements. + +When `release-notes/features.json` provides a standard callout, preserve its safety information, including broader product-preview warnings. +Shared setup does not replace those notices with a language-version assumption. +Use compact section labels only where they preserve the applicable requirements and warnings. ## Upgrade guidance @@ -146,8 +157,8 @@ Use **Changes since the previous preview** for migrations that only affect code The [incremental API diff](api-verification.md#review-incremental-api-diffs-for-preview-upgrades) and maintained samples can reveal migration candidates, but do not determine which baseline they break. Cover these when they apply. They tend to surface only when an existing project is actually upgraded -to the new build, which is one of the reasons -[`validate-code-samples`](../../validate-code-samples/SKILL.md) upgrades the component's maintained sample set under `release-notes/.0/samples//`: +to the new build. [`validate-code-samples`](../../validate-code-samples/SKILL.md) +uses scratch projects by default and upgrades maintained component samples when they are in scope: - **Renamed APIs**, especially renames that invert meaning (`EnableX` becoming `DisableX`). Show the before and after, and state the new default explicitly. diff --git a/.github/skills/release-notes/references/pr-layout.md b/.github/skills/release-notes/references/pr-layout.md index 0f1c4af5915..3b20255cf4f 100644 --- a/.github/skills/release-notes/references/pr-layout.md +++ b/.github/skills/release-notes/references/pr-layout.md @@ -16,6 +16,12 @@ The set of components and their release notes files is defined in [`component-ma - Never link an entry to a file absent from that branch. Keep unlinked entries for pending component PRs; once all component PRs merge, verify that every expected entry links to its existing file. Retain push and PR link checks throughout. - The milestone landing page `{version}.md` (for example, `11.0.0-preview.4.md`) is **not produced by this skill**. The .NET release team generates it through separate artifacts-publishing automation, so the agent leaves it alone on every branch. +Publication scope is the component article, its own index link, and explicitly approved companion reports or required media. +Validation alone does not authorize repository fixtures or other experiment artifacts. +Add or change maintained fixtures only when explicitly requested or already agreed in the task scope. +Do not silently delete existing files. +Skill-policy updates belong in a separate documentation PR, not a release-article PR. + ## Creating the PRs Order matters, and two of these steps fail silently. @@ -25,7 +31,7 @@ Order matters, and two of these steps fail silently. the component branches but **not** the base branch — verifying with that pattern reports success while the base branch is still local-only. 2. **Confirm each component branch starts from the base branch commit.** Each component PR should - show only its component file, any matching validation samples, and its single README link change; + show only its component file, its single README link change, and approved companion material or maintained fixtures; it should not restate shared metadata or other components' entries. If the base branch moves, merge it into the component branch before updating the entry and verify the diff again. 3. **Open the base PR against `main`**, then the component PRs against the base branch. @@ -37,6 +43,15 @@ Order matters, and two of these steps fail silently. edits conflict as owners merge, retain all previously merged links and add only the current component's link. +### Updating and reporting + +- Update an existing PR's head branch only. Preserve its draft state and base unless explicitly authorized otherwise. +- Use ordinary commits and non-force pushes. Do not amend commits or publish directly to the target branch or `main`. +- Leave promotion and merging to the authorized owner. Do not merge as part of a documentation edit. +- Report the actual branch and commit SHA. Distinguish local, committed, pushed, and merged state. +- Report whole-PR file counts and additions/deletions against the base, not only the latest patch. +- Bound research to the requested changes. For prose-only edits, use the smallest existing Markdown and reference checks, not code builds. + ### gh pr edit does not work on this repo `gh pr edit` fails against `dotnet/core` with a Projects (classic) GraphQL deprecation error and diff --git a/.github/skills/release-notes/references/quality-bar.md b/.github/skills/release-notes/references/quality-bar.md index bd40edcd10e..0b52ff47614 100644 --- a/.github/skills/release-notes/references/quality-bar.md +++ b/.github/skills/release-notes/references/quality-bar.md @@ -21,10 +21,10 @@ Not every PR deserves a writeup. The markdown covers features users will care ab Quality criteria: -- **High fidelity** — everything documented actually ships in this release +- **High fidelity** — claims of new availability trace to this release's manifest and build - **High value** — focused on what matters, not exhaustive - **Useful** — a developer can read a section and start using the feature -- **Honest** — no marketing fluff, no exaggeration, no invented benchmarks +- **Honest** — concrete benefits, no exaggeration, no invented benchmarks ## What to include in markdown @@ -42,32 +42,46 @@ Apply the shared **80/20 audience filter** from prioritize features most readers can immediately understand, and keep narrower items only when the broader audience can still appreciate why they matter. +State the intended audience prominently, including features that serve library authors first. +Shipped compiler or runtime foundations can merit a headline before a stock-library integration or builder ships. +Explain the real supported capability, test-only or hypothetical integrations, and unchanged stock behavior separately. +Do not promise future shipping or performance. + ## What to exclude from markdown - Internal refactoring with no user-facing change - Test-only changes - Build/infrastructure changes - Backports from servicing branches -- Features that are not independently useful yet (still need unshipped APIs) -- Anything not confirmed by `changes.json` (i.e., not in the source-manifest diff) +- Claimed capabilities that require unshipped APIs, rather than useful shipped foundations +- Claims of newly shipped changes absent from this milestone's `changes.json` - Anything that reads like unexplained engineering jargon to most readers ## The fidelity rule -**If it's not in `changes.json`, don't write about it.** +**Every claim of a new change must trace to this milestone's `changes.json`.** The `changes.json` file is generated from the VMR's `source-manifest.json` diff — it reflects exactly what code moved between release points. This is the single source of truth. Don't document features based on PR titles alone, roadmap promises, or what "should" ship. Only document what the tool confirms actually shipped. +Discover omissions against the exact pinned changes, implementation, and tests, not only the draft or titles. +Read prior component announcements to distinguish new deltas from previously announced, unchanged material. + +Explicitly requested catch-up coverage can describe earlier-shipped, unannounced capabilities. +State their actual availability and verify their original shipped provenance and current support. +Do not add them as new entries in this milestone's `changes.json` or `features.json`. +An explicitly approved full-cycle report must identify its broader baseline, not imply that all changes shipped in this milestone. + ## Every feature entry needs WHY and HOW A release note that just names a feature is useless. Every entry must answer: 1. **Why does this matter?** — what problem does it solve, what scenario does it enable? -2. **How do I use it?** — a code sample showing the feature in action +2. **How do I use it?** — a substantial supported scenario, setup, or representative measurement When relevant, add a **familiar comparison** to help the reader place the feature quickly. If a new CLI workflow resembles something people already know from another tool, say that briefly. For example, `dotnet run -e FOO=BAR` can be framed as Docker-style environment-variable injection for local runs. Use comparisons to clarify, not to imply exact parity. -If you can't write a code sample for a feature, question whether it's user-facing enough for release notes. +Use meaningful source-test scenarios, not a tiny syntax demonstration, when showing a capability. +For default-enabled or grouped performance improvements, explain adoption and measured workloads instead of forcing a code sample per change. ## Verify before you write @@ -81,9 +95,13 @@ An incorrect type name in release notes is worse than a placeholder. It teaches This applies to code samples too. A fabricated code sample that uses invented types is harmful — it looks authoritative but won't compile. +Check test conditions and expected outcomes. Negative tests, skipped cases, or unmet conditions do not prove successful runtime support. +Run documented behavior against the milestone build through [`validate-code-samples`](../../validate-code-samples/SKILL.md). +Temporary validation does not authorize committed fixtures. Follow that skill's scope rules. + ## Tone - Positive — highlight what's new, don't dwell on what was missing -- Direct — one paragraph of context, then show the code -- Precise — exact benchmark numbers only, never approximations +- Direct — concise context, then a useful scenario or measurement +- Precise — source-backed measurements and accurate summaries, following [the benchmark rules](editorial-rules.md#benchmarks) - Respectful of reader time — concise descriptions, no padding diff --git a/.github/skills/review-release-notes/SKILL.md b/.github/skills/review-release-notes/SKILL.md index 4b0e47c91e7..1b99bde3cce 100644 --- a/.github/skills/review-release-notes/SKILL.md +++ b/.github/skills/review-release-notes/SKILL.md @@ -31,7 +31,7 @@ Review these, in order: 2. `features.json` — the shipped-change entries with scored candidates 3. Draft markdown files (`libraries.md`, `runtime.md`, `sdk.md`, etc.) from each component PR branch if unmerged, or the milestone base branch if merged -4. Editorial examples in `references/examples/` +4. Editorial examples in `references/examples/` and published predecessor articles for the component 5. The scoring and quality bar references: - `../editorial-scoring/SKILL.md` - `references/feature-scoring.md` @@ -54,6 +54,9 @@ entry's `score_reason`, adjusting its score if the review changes the assessment. Do not treat a PR's presence in `changes.json` as a requirement to give it a release-note section; apply the reader-centric rubric. +Inspect pinned source and tests for important omissions and meaningful usage scenarios. +Compare against prior announcements to remove unchanged material and verify the chronology of explicitly requested catch-up coverage. + ## Core review questions Use the shared rubric from [`../editorial-scoring/SKILL.md`](../editorial-scoring/SKILL.md) @@ -66,12 +69,13 @@ rather than inventing a new one here. In particular: ### Example check -Compare the draft against the component examples: +Read the component examples and actual predecessor articles. Review these dimensions separately: + +- **Voice** — positive, concrete developer possibilities compared with the predecessors +- **Style** — established terminology, structure, and length proportional to importance +- **Selection** — feature importance, explicit priorities, order, omissions, and deduplication -- Does the length match the importance? -- Are medium-value items grouped instead of getting their own heavyweight sections? -- Are performance stories backed by evidence? -- Does the draft teach, not just enumerate? +Check grouped items and performance evidence through the existing rubric and [editorial rules](../release-notes/references/editorial-rules.md). ## Common failure modes @@ -99,12 +103,15 @@ Give both reviewers the same inputs and the same requested output: Then synthesize the overlap and disagreements. Treat consensus as a strong signal, but do **not** turn this into a blind vote — fidelity to `changes.json`, the shared `editorial-scoring` rubric, and the repo's editorial rules still wins. +Record which files each reviewer actually read, actionable findings, and resulting changes. +Naming files in a prompt is not proof of comparison. Do not claim a multi-model panel unless it ran. + ## Reviewer checklist Do not ask reviewers the vague question "do you like this?" Give them the same specific checks instead: -1. **Which headings still sound vague, passive, anthropomorphic, or promotional?** +1. **Which headings replace established names with branding, or remain vague, passive, or anthropomorphic?** 2. **Which sections fail the 80/20 reader-value test and should be cut, grouped, or demoted?** 3. **Which sentences infer feelings or outcomes (`trust`, `confidence`, `easier`, `better`) instead of stating the concrete change?** 4. **Which sections drift into API-inventory mode instead of teaching a user-facing story?** @@ -115,7 +122,11 @@ specific checks instead: 9. **Is the wording conventional, or is it inventing non-standard phrasing or terms?** 10. **Are the subject and its adjective or adverb paired in a familiar way?** 11. **Would this phrasing seem normal in release notes for another developer platform?** -12. **If `release-notes/features.json` lists this feature, does the section begin with the standard preview blockquote?** +12. **Are preview opt-ins defined once, affected sections labeled, and default-enabled changes distinct? Are actual per-feature requirements and metadata-derived warnings preserved?** +13. **Is the audience clear? Are shipped foundations distinguished from test-only integrations, unshipped builders, and unchanged stock behavior?** +14. **Do performance stories separate application and compiler audiences, metrics, baselines, and cohorts while preserving regressions?** +15. **Does publication follow the approved article, index, report/media, and maintained-fixture scope in [PR layout](../release-notes/references/pr-layout.md)?** +16. **Do relative links, anchors, images, and remote source targets resolve? Are unavailable checks reported rather than counted as passed?** Ask reviewers to answer with file + heading + issue + suggested rewrite. This produces actionable review instead of general taste feedback. diff --git a/.github/skills/validate-code-samples/SKILL.md b/.github/skills/validate-code-samples/SKILL.md index c018162d776..b4899482f9c 100644 --- a/.github/skills/validate-code-samples/SKILL.md +++ b/.github/skills/validate-code-samples/SKILL.md @@ -24,25 +24,39 @@ Verify the installed SDK reports `build.sdk_version`; stop if it does not. ## Where samples live -Keep maintained validation samples in this repository under `release-notes/.0/samples//`, for example `release-notes/11.0/samples/aspnetcore/`. These are executable fixtures primarily for verifying release notes, not a general-purpose or reader-facing samples collection. They do not replace the officially documented samples maintained by the relevant product and documentation teams. +Keep temporary validation projects, checkers, and results in session or scratch storage by default. +Validation is required, but committing its artifacts is not. +Add or change maintained repository fixtures only when explicitly requested or already agreed in the task scope. +Existing fixtures do not authorize silent deletion or unrelated expansion. -Use one working sample set per major release and component. Upgrade it from preview to preview so API renames, changed defaults, analyzer diagnostics, and runtime regressions surface naturally. Start each major release with a new sample set instead of copying or retargeting scenarios from the previous release. Component owners review changes to their sample set alongside the corresponding release notes. +When maintained fixtures are in scope, keep them under `release-notes/.0/samples//`, for example `release-notes/11.0/samples/aspnetcore/`. These are executable fixtures primarily for verifying release notes, not a general-purpose or reader-facing samples collection. They do not replace the officially documented samples maintained by the relevant product and documentation teams. -Keep samples for release-note features in the maintained component set so they can be rerun in later previews. For one-off compile checks outside feature validation, use a scratch project and remove it afterward. +For an agreed maintained set, use one working sample set per major release and component. Upgrade it from preview to preview so API renames, changed defaults, analyzer diagnostics, and runtime regressions surface naturally. Start each major release with a new sample set instead of copying or retargeting scenarios from the previous release. Component owners review changes to their sample set alongside the corresponding release notes. + +Otherwise, use scratch copies for upgrade checks without changing repository fixtures. +Remove temporary projects when no longer needed. Keep verification records in session storage. Do not commit downloaded SDKs, packages, build outputs, certificates, secrets, or generated assets. ## What to validate -- Build or update a sample for every feature in the release notes, even if the notes have no code snippet for it. Run each sample against the milestone build and verify the claimed behavior, such as a browser API call, an endpoint response, or an explicitly stated default or flag polarity. A successful build or startup alone is not enough. +- Validate every feature's documented claims, even without a code snippet. Build or update scratch samples unless maintained fixtures are in scope. Run applicable scenarios against the milestone build and verify the claimed behavior, such as a browser API call, an endpoint response, or an explicitly stated default or flag polarity. A successful build or startup alone is not enough. - Test any code snippets from the release notes as part of those samples. Confirm the snippets build, run, and behave as described. -- As you update existing maintained samples to the new preview, note changes required by the new release. Document the resulting preview-to-preview breaking changes and migration steps in the release notes. +- Check source-test conditions and expected outcomes. Negative tests, skipped cases, unmet conditions, and test-only integrations do not establish shipped runtime support. +- For grouped performance changes, check representative workloads and source measurements, not one new fixture per implementation PR. +- Do not rerun unrelated benchmarks or builds for prose-only changes. +- During scratch or maintained sample upgrades, record required changes. Document the resulting preview-to-preview breaking changes and migration steps in the release notes. ## Recording what you verified -Create `release-notes/.0/samples/README.md` when adding maintained samples. Explain their purpose and record the SDK version against which all component sample sets were last validated. Update the version after validating the sets against a new SDK. +Record the exact SDK, claim, expected and observed behavior, sources, and limits in session storage. +Keep failed or unavailable checks separate from passed checks. Do not imply verification beyond the evidence. + +Only when maintained samples are in scope, create or update `release-notes/.0/samples/README.md`. +Explain their purpose and record the SDK version against which all component sample sets were last validated. +Update the version after validating the sets against a new SDK. -In each `/README.md`, describe the component's samples, how to run them, and their expected behavior. Keep it current as the samples change. +For that maintained set, keep each `/README.md` current with its scenarios, run instructions, and expected behavior. ## When a claim fails validation