Server: Create product documentation automation#36282
Open
Server: Create product documentation automation#36282
Conversation
Create an automation for the process of opening docs PRs for dev PRs with the "Docs/Neded" label. This same automation will be applied to mobile/desktop repos. Associated PR in the docs repo: mattermost/docs#8844.
mm-cloud-bot
added
the
release-note-none
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a GitHub Actions workflow triggered by the Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor GitHub as GitHub (Event)
participant Actions as GitHub Actions
participant DevRepo as Dev PR Repo
participant DocsRepo as mattermost/docs
participant Anthropic as Anthropic Messages API
GitHub->>Actions: 'Docs/Needed' label added to PR
Actions->>DevRepo: Fetch PR metadata, milestone, body, diff
Actions->>Actions: Compute docs branch & draft path
Actions->>DocsRepo: Check for existing PR/branch
Actions->>Anthropic: Send system prompt + escaped PR data
Anthropic-->>Actions: Return generated markdown draft
Actions->>DocsRepo: Create/checkout branch, write draft file
Actions->>DocsRepo: Commit & push if content changed
Actions->>DocsRepo: Ensure/create matching milestone
Actions->>DocsRepo: Create or update docs PR, add reviewer
Actions->>DevRepo: Post comment with docs PR link
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (1)
.github/workflows/docs-needed.yml (1)
159-159: Step numbering: two consecutive steps both labelled── 3..
Check for an existing docs PRandCheckout docs repoare both numbered3.. Renumber the rest of the file to keep the section markers in sequence (the rest of the workflow already advances cleanly to 4–12).🔧 Proposed renumbering (cosmetic)
- # ── 3. Check for an existing docs PR (idempotency) ─────────────────── + # ── 3. Check for an existing docs PR (idempotency) ─────────────────── @@ - # ── 3. Checkout docs repo ───────────────────────────────────────────── + # ── 4. Checkout docs repo ─────────────────────────────────────────────…and shift the subsequent numbers (4→5, 5→6, …, 12→13).
Also applies to: 182-182
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml at line 159, Two consecutive section comments are both labeled "── 3." (the comment containing "── 3. Check for an existing docs PR" and the later "── 3. Checkout docs repo"); update the second "── 3." to "── 4." and then increment the subsequent section markers throughout the file (change 4→5, 5→6, …, 12→13) so the numbered comments remain sequential; ensure you also update the matching occurrence noted around line 182 (the other "── 3." comment) to keep all section markers consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/docs-needed.yml:
- Around line 59-64: Fix the yamllint "colons" failures by removing the
decorative alignment spaces after colons in the env mappings (e.g., change
"MILESTONE_TITLE: ${{ github.event.pull_request.milestone.title }}" and
"PR_TITLE: ${{ github.event.pull_request.title }}" and similar keys
PR_NUMBER, PR_AUTHOR, SOURCE_REPO to "MILESTONE_TITLE: ${{ ... }}" etc.) across
all flagged blocks; alternatively, if alignment is desired, relax the "colons"
rule in .yamllint to allow aligned values, but don't leave extra spaces that
currently trigger CI.
- Around line 38-43: The job open-docs-pr should declare an explicit permissions
block so the step "Comment on dev PR" using secrets.GITHUB_TOKEN doesn't fail
under org-default read-only tokens; add a permissions: section on the
open-docs-pr job granting at minimum pull-requests: write (so gh pr comment can
succeed) and ensure the GITHUB_TOKEN usage remains unchanged while other
cross-repo actions continue to use DOCS_REPO_PAT.
- Around line 437-441: The workflow message string (the --body "📝 A
documentation draft has been ${ACTION} ...") incorrectly promises automation
that will replace the Docs/Needed label; edit that literal message in the
.github/workflows/docs-needed.yml step to remove or soften the final sentence so
it no longer states "It will be replaced with `Docs/Done` once the docs PR is
merged or closed." — either delete that sentence or change it to a human-action
phrasing such as "A maintainer may replace the `Docs/Needed` label with
`Docs/Done` once the docs PR is merged or closed." Ensure you update only the
--body string content and keep the rest of the workflow unchanged.
- Around line 296-305: The heredoc content written to "$DOCS_FILE" is starting
at column 0 inside a run: | block which breaks YAML parsing; fix the block by
either indenting the heredoc body to match the run: | block and switching to a
strip-indent heredoc (use <<-HEADER so leading tabs are removed) or replace the
heredoc with a sequence of indented printf/echo lines (each staying inside the
run block) that write the same lines using the variables PR_TITLE, SOURCE_REPO,
PR_NUMBER, PR_AUTHOR and the HEADER marker; ensure the final written content
still begins with the header marker and maintains the same interpolations.
- Around line 374-380: In the Create docs PR step replace the --milestone
parameter value from "$MILESTONE_NUMBER" to "$MILESTONE_TITLE" (the gh CLI
matches milestone by title); in the Update existing docs PR step add
MILESTONE_TITLE: ${{ steps.vars.outputs.milestone_title }} to its env block and
also change its --milestone argument from "$MILESTONE_NUMBER" to
"$MILESTONE_TITLE" so both gh pr create and gh pr edit use the milestone title
output produced by steps.vars.
- Around line 29-31: The workflow currently triggers on pull_request types:
[labeled] and thus can run from forked PRs without access to
secrets.DOCS_REPO_PAT and secrets.ANTHROPIC_API_KEY, causing failures at the
"Resolve docs base branch" step; fix by either adding an explicit fork guard to
the job/step condition (e.g., require github.event.label.name == 'Docs/Needed'
&& github.event.pull_request.head.repo.full_name == github.repository) so only
internal PRs run, or change the trigger to pull_request_target to run in the
base repo context (and if you choose pull_request_target, ensure you do not
check out or execute untrusted PR code before validating/whitelisting inputs
because steps like the checkout/Claude/git usage can accept malicious content).
---
Nitpick comments:
In @.github/workflows/docs-needed.yml:
- Line 159: Two consecutive section comments are both labeled "── 3." (the
comment containing "── 3. Check for an existing docs PR" and the later "── 3.
Checkout docs repo"); update the second "── 3." to "── 4." and then increment
the subsequent section markers throughout the file (change 4→5, 5→6, …, 12→13)
so the numbered comments remain sequential; ensure you also update the matching
occurrence noted around line 182 (the other "── 3." comment) to keep all section
markers consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: ff2b0697-b4fa-4343-ad8e-531d80423c49
📒 Files selected for processing (1)
.github/workflows/docs-needed.yml
Documentation Impact Analysis — no longer neededA previous automated documentation impact comment exists, but the latest analysis determined that no documentation changes are needed. The |
amyblais
added
1: PM Review
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (2)
.github/workflows/docs-needed.yml (2)
378-384:⚠️ Potential issue | 🔴 Critical
--milestonestill receives the milestone number rather than its title.This concern from the previous review is still present:
gh pr create --milestoneandgh pr edit --milestoneboth match by milestone title (e.g.,v11.7.0), not by numeric ID, so passing$MILESTONE_NUMBERwill fail to associate a milestone (or fail outright).MILESTONE_TITLEis already an output ofsteps.vars.🛠️ Suggested fix
- name: Create docs PR id: create-pr if: steps.existing.outputs.exists == 'false' env: GH_TOKEN: ${{ secrets.DOCS_REPO_PAT }} BRANCH: ${{ steps.vars.outputs.branch }} MILESTONE_TITLE: ${{ steps.vars.outputs.milestone_title }} - MILESTONE_NUMBER: ${{ steps.docs-milestone.outputs.number }} SOURCE_REPO: ${{ steps.vars.outputs.source_repo }} … run: | … URL=$(gh pr create \ --repo mattermost/docs \ --head "$BRANCH" \ --base "${{ steps.base-branch.outputs.base }}" \ --title "docs: ${PR_TITLE}" \ --body "$BODY" \ - --milestone "$MILESTONE_NUMBER") + --milestone "$MILESTONE_TITLE") - name: Update existing docs PR if: steps.existing.outputs.exists == 'true' env: GH_TOKEN: ${{ secrets.DOCS_REPO_PAT }} DOCS_PR: ${{ steps.existing.outputs.number }} - MILESTONE_NUMBER: ${{ steps.docs-milestone.outputs.number }} + MILESTONE_TITLE: ${{ steps.vars.outputs.milestone_title }} run: | gh pr edit "$DOCS_PR" \ --repo mattermost/docs \ - --milestone "$MILESTONE_NUMBER" + --milestone "$MILESTONE_TITLE"The
Sync milestonestep (lines 331-349) is then no longer required for the--milestoneargument, though you may still want to keep it to ensure the milestone exists onmattermost/docsbeforegh pr createruns (otherwise the title-based lookup will also fail).Also applies to: 389-399
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 378 - 384, The workflow is passing the numeric milestone ID to gh commands but gh pr create / gh pr edit match milestones by title, so replace uses of $MILESTONE_NUMBER with $MILESTONE_TITLE in the gh invocations (specifically the gh pr create and gh pr edit commands in this diff) so the milestone argument is the milestone title; you can keep the "Sync milestone" step (Sync milestone) if you still want to ensure the milestone exists on mattermost/docs before running gh, but it is not required solely for providing the --milestone value.
59-64:⚠️ Potential issue | 🟠 Majoryamllint
colonsrule still failing CI on alignedenv:blocks.Per the latest pipeline, yamllint still fails at lines 59, 61, 62, 63, 64, 94, 96, 97, 170, 206 due to extra spaces after
:for decorative alignment. Either drop the alignment or relax thecolonsrule in.yamllint. As-is the workflow won't pass CI.Also applies to: 94-97, 170-170, 206-206
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 59 - 64, The CI is failing due to extra spaces after the colon in aligned env: entries (e.g., keys MILESTONE_TITLE, MILESTONE_NUMBER, PR_NUMBER, PR_TITLE, PR_AUTHOR, SOURCE_REPO and the other env blocks at the mentioned lines), so either remove the decorative alignment spaces after each colon so they follow "KEY: value" with a single space, or relax/disable the yamllint "colons" rule in .yamllint to allow aligned mappings; update the env blocks (and the other offending mapping blocks at the other line ranges) or the linter config accordingly to resolve the yamllint errors.
🧹 Nitpick comments (3)
.github/workflows/docs-needed.yml (3)
282-287: Guard against an Anthropic error response being written to disk asnull.If the Anthropic Messages API returns a non-2xx (caught by
-f) the step fails — fine. But a 200 response without.content[0].text(e.g., astop_reasonofmax_tokenswith no text block, or a transient API change) would havejq -rwrite the literal stringnullto/tmp/generated-docs.md, which would then be committed to the docs PR. Consider validating the response before writing it.🛡️ Suggested fix
- curl -sf https://api.anthropic.com/v1/messages \ + RESPONSE=$(curl -sf https://api.anthropic.com/v1/messages \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "anthropic-version: 2023-06-01" \ -H "content-type: application/json" \ - -d "$PAYLOAD" \ - | jq -r '.content[0].text' > /tmp/generated-docs.md + -d "$PAYLOAD") + TEXT=$(printf '%s' "$RESPONSE" | jq -r '.content[0].text // empty') + if [ -z "$TEXT" ]; then + echo "::error::Anthropic response did not contain content[0].text" + printf '%s\n' "$RESPONSE" | jq . >&2 || true + exit 1 + fi + printf '%s' "$TEXT" > /tmp/generated-docs.md🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 282 - 287, Capture the full Anthropic response into a variable instead of piping directly to jq, then validate the text field before writing to /tmp/generated-docs.md: run the curl command (the existing call using $ANTHROPIC_API_KEY, $PAYLOAD and anthropic-version header) into a RESPONSE variable, extract TEXT with jq like '.content[0].text // empty', and if TEXT is empty or equals "null" fail the step (non‑zero exit) or log a clear error and do not write the file; only echo TEXT to /tmp/generated-docs.md when it is present and non-empty.
165-165: Duplicate step numbering: "── 3." appears twice.Both the "Check for existing docs PR" comment (line 165) and the "Checkout docs repo" comment (line 188) are labeled
── 3.. Since step numbering is used to navigate this 448-line workflow, renumbering downstream sections to keep them sequential makes maintenance easier.Also applies to: 188-188
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml at line 165, The workflow contains duplicated step numbering: the comment "── 3. Check for an existing docs PR" and the comment "── 3. Checkout docs repo" both use "── 3."; update the latter and all subsequent step comment prefixes so numbering is sequential (e.g., change "── 3. Checkout docs repo" to "── 4." and increment following sections accordingly). Ensure you modify only the visible step-number prefixes in the comments (the strings "── 3. Check for an existing docs PR" and "── 3. Checkout docs repo" and every following "── N." header) so navigation through the 448-line workflow remains consistent.
127-131: UseGITHUB_TOKENto query public repository milestones instead ofDOCS_REPO_PAT.While fine-grained PATs do have read-only access to public repositories by default, using
GITHUB_TOKENhere is clearer and follows better separation of concerns—it's designed for accessing the current workflow's repository and public data, whereasDOCS_REPO_PATshould remain reserved for operations againstmattermost/docs. At line 94, switchGH_TOKENto useGITHUB_TOKENfor the milestone query. ReserveDOCS_REPO_PATfor the branch existence check at line 153 and the docs repo operations.🔧 Suggested change
env: - GH_TOKEN: ${{ secrets.DOCS_REPO_PAT }} + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} MILESTONE_TITLE: ${{ steps.vars.outputs.milestone_title }} MILESTONE_DUE: ${{ github.event.pull_request.milestone.due_on }} REPO_NAME: ${{ steps.vars.outputs.repo_name }}Then update line 153 to explicitly use
DOCS_REPO_PATfor the docs repo branch check:- EXISTS=$(gh api "repos/mattermost/docs/branches/${DOCS_BRANCH}" \ + GH_TOKEN=${{ secrets.DOCS_REPO_PAT }} gh api "repos/mattermost/docs/branches/${DOCS_BRANCH}" \🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 127 - 131, Change the milestone query to authenticate with the workflow's GITHUB_TOKEN instead of DOCS_REPO_PAT: set GH_TOKEN to GITHUB_TOKEN for the MM_MILESTONE gh api call (the block that assigns MM_MILESTONE) so the public repo milestone lookup uses the workflow token, and keep DOCS_REPO_PAT only for the docs-repo-specific operations (the later branch-existence check that uses DOCS_REPO_PAT).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In @.github/workflows/docs-needed.yml:
- Around line 378-384: The workflow is passing the numeric milestone ID to gh
commands but gh pr create / gh pr edit match milestones by title, so replace
uses of $MILESTONE_NUMBER with $MILESTONE_TITLE in the gh invocations
(specifically the gh pr create and gh pr edit commands in this diff) so the
milestone argument is the milestone title; you can keep the "Sync milestone"
step (Sync milestone) if you still want to ensure the milestone exists on
mattermost/docs before running gh, but it is not required solely for providing
the --milestone value.
- Around line 59-64: The CI is failing due to extra spaces after the colon in
aligned env: entries (e.g., keys MILESTONE_TITLE, MILESTONE_NUMBER, PR_NUMBER,
PR_TITLE, PR_AUTHOR, SOURCE_REPO and the other env blocks at the mentioned
lines), so either remove the decorative alignment spaces after each colon so
they follow "KEY: value" with a single space, or relax/disable the yamllint
"colons" rule in .yamllint to allow aligned mappings; update the env blocks (and
the other offending mapping blocks at the other line ranges) or the linter
config accordingly to resolve the yamllint errors.
---
Nitpick comments:
In @.github/workflows/docs-needed.yml:
- Around line 282-287: Capture the full Anthropic response into a variable
instead of piping directly to jq, then validate the text field before writing to
/tmp/generated-docs.md: run the curl command (the existing call using
$ANTHROPIC_API_KEY, $PAYLOAD and anthropic-version header) into a RESPONSE
variable, extract TEXT with jq like '.content[0].text // empty', and if TEXT is
empty or equals "null" fail the step (non‑zero exit) or log a clear error and do
not write the file; only echo TEXT to /tmp/generated-docs.md when it is present
and non-empty.
- Line 165: The workflow contains duplicated step numbering: the comment "── 3.
Check for an existing docs PR" and the comment "── 3. Checkout docs repo" both
use "── 3."; update the latter and all subsequent step comment prefixes so
numbering is sequential (e.g., change "── 3. Checkout docs repo" to "── 4." and
increment following sections accordingly). Ensure you modify only the visible
step-number prefixes in the comments (the strings "── 3. Check for an existing
docs PR" and "── 3. Checkout docs repo" and every following "── N." header) so
navigation through the 448-line workflow remains consistent.
- Around line 127-131: Change the milestone query to authenticate with the
workflow's GITHUB_TOKEN instead of DOCS_REPO_PAT: set GH_TOKEN to GITHUB_TOKEN
for the MM_MILESTONE gh api call (the block that assigns MM_MILESTONE) so the
public repo milestone lookup uses the workflow token, and keep DOCS_REPO_PAT
only for the docs-repo-specific operations (the later branch-existence check
that uses DOCS_REPO_PAT).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 72ae8402-db5b-458c-8ed7-96600f8a2ad2
📒 Files selected for processing (1)
.github/workflows/docs-needed.yml
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (5)
.github/workflows/docs-needed.yml (5)
38-43:⚠️ Potential issue | 🟠 MajorStill no
permissions:block on the job.The
Comment on dev PRstep (line 451) usessecrets.GITHUB_TOKENto post a comment; under org-default read-only token settings it will 403. Declare an explicit minimal block:🔐 Proposed fix
open-docs-pr: name: Open docs PR for ${{ github.repository }}#${{ github.event.pull_request.number }} runs-on: ubuntu-latest if: github.event.label.name == 'Docs/Needed' + permissions: + contents: read + pull-requests: writeCross-repo writes go through
DOCS_REPO_PATand are unaffected.Based on learnings,
pull-requests: writeis sufficient for bot-managed PR comments via the REST API;issues: writeis not required.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 38 - 43, The job "open-docs-pr" lacks an explicit permissions block so steps that use secrets.GITHUB_TOKEN (e.g., the "Comment on dev PR" step) will 403 under org-default read-only tokens; add a minimal permissions section to the "open-docs-pr" job that grants pull-requests: write (sufficient for REST API PR comments) so the step using GITHUB_TOKEN can post comments—leave cross-repo PAT usage (DOCS_REPO_PAT) unchanged.
59-64:⚠️ Potential issue | 🟠 Majoryamllint
colonsfailures still blocking CI.Static analysis is still failing on the decorative env-key alignment at lines 59, 61‑64, 94, 96‑97, 170, and 206. Either drop the extra spaces or relax the rule in
.yamllint. As-is the workflow won't pass CI on merge.Also applies to: 94-97, 170-170, 206-207
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 59 - 64, The CI fails due to yamllint 'colons' rule triggered by aligned environment keys (e.g., MILESTONE_TITLE, MILESTONE_NUMBER, PR_NUMBER, PR_TITLE, PR_AUTHOR, SOURCE_REPO) in the workflow; fix by either removing the extra spacing so each env line uses "KEY: ${{ ... }}" with a single space after the colon, or update .yamllint to relax/disable the 'colons' rule for this file (or add a file-specific override) so the current alignment is allowed—apply the chosen fix to all affected blocks referenced (the env sections around the shown keys and the other reported line ranges).
383-389:⚠️ Potential issue | 🔴 Critical
gh pr create --milestone/gh pr edit --milestoneexpect a title, not a number.Both steps still pass
$MILESTONE_NUMBER(numeric ID fromsteps.docs-milestone.outputs.number). The CLI matches by milestone title, so these calls won't attach the milestone — silently ingh pr edit, and noisily ingh pr create.Use
MILESTONE_TITLE(already an output ofsteps.vars) in both places, and add it to theUpdate existing docs PRstep'senv.Does the GitHub CLI `gh pr create --milestone` flag accept the milestone number, or only the milestone title?Also applies to: 394-404
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 383 - 389, The gh CLI calls use the numeric milestone ID (steps.docs-milestone.outputs.number) but gh pr create / gh pr edit require the milestone title; update the two commands that pass --milestone "$MILESTONE_NUMBER" to instead pass --milestone "$MILESTONE_TITLE" (use the output from steps.vars that exposes MILESTONE_TITLE) for both the "Create docs PR" and "Update existing docs PR" steps, and add MILESTONE_TITLE to the env of the "Update existing docs PR" step so the variable is available there.
29-42:⚠️ Potential issue | 🟠 MajorFork PRs will fail noisily — fork guard or
pull_request_targetstill missing.A maintainer labeling a fork PR will run this in the fork's restricted context with no access to
DOCS_REPO_PAT/ANTHROPIC_API_KEY, producing a red X at the "Resolve docs base branch" step. Add&& github.event.pull_request.head.repo.full_name == github.repositoryto the job-levelif, or switch topull_request_target(with the usual injection-safety caveats since you check out a PR diff and feed it to Claude).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 29 - 42, The job-level condition for the open-docs-pr job currently uses if: github.event.label.name == 'Docs/Needed' which allows maintainer-labeled fork PRs to run in the fork's restricted context; update the condition on the open-docs-pr job (job name: open-docs-pr) to also require the PR comes from the same repo by adding && github.event.pull_request.head.repo.full_name == github.repository, or alternatively change the workflow trigger from pull_request to pull_request_target and apply the usual checkout/secret-injection safety checks; ensure the modified if sits on the open-docs-pr job to prevent fork PRs from accessing secrets.
445-449:⚠️ Potential issue | 🟡 MinorComment still promises
Docs/Needed→Docs/Doneautomation that doesn't exist.The trailing line still tells authors the label "will be replaced with
Docs/Doneonce the docs PR is merged or closed." No replacement workflow ships in this PR, and per the design ofdocs-impact-review.ymltheDocs/Neededlabel is intentionally one-way (added by automation, removed only by a human). Either drop or soften the sentence, or wire up the actual replacement automation before shipping this messaging.Based on learnings: in
.github/workflows/docs-impact-review.yml, theDocs/Neededlabel is intentionally one-way — a human must explicitly remove it even when later analysis says docs are no longer needed.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 445 - 449, Update the comment text assembled into COMMENT so it no longer promises automated replacement of the "Docs/Needed" label; change or remove the trailing sentence that says the label "will be replaced with `Docs/Done` once the docs PR is merged or closed." Instead, state that the label must be removed by a human (or soften to say it "should be removed manually") to match the existing docs-impact-review.yml behavior; edit the MESSAGE built with DOCS_LINK/COMMENT accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/docs-needed.yml:
- Around line 342-352: The milestone lookup can miss closed or paginated
milestones; update the gh API listing call that sets NUM (the invocations of "gh
api repos/mattermost/docs/milestones" that use MILESTONE_TITLE to find an
existing milestone) to include pagination and request all states (use --paginate
and -F state=all) so the jq selection against MILESTONE_TITLE finds closed or
later-page matches instead of creating duplicates; keep the same jq/filter and
fallback creation logic that posts a new milestone when NUM is still empty.
---
Duplicate comments:
In @.github/workflows/docs-needed.yml:
- Around line 38-43: The job "open-docs-pr" lacks an explicit permissions block
so steps that use secrets.GITHUB_TOKEN (e.g., the "Comment on dev PR" step) will
403 under org-default read-only tokens; add a minimal permissions section to the
"open-docs-pr" job that grants pull-requests: write (sufficient for REST API PR
comments) so the step using GITHUB_TOKEN can post comments—leave cross-repo PAT
usage (DOCS_REPO_PAT) unchanged.
- Around line 59-64: The CI fails due to yamllint 'colons' rule triggered by
aligned environment keys (e.g., MILESTONE_TITLE, MILESTONE_NUMBER, PR_NUMBER,
PR_TITLE, PR_AUTHOR, SOURCE_REPO) in the workflow; fix by either removing the
extra spacing so each env line uses "KEY: ${{ ... }}" with a single space after
the colon, or update .yamllint to relax/disable the 'colons' rule for this file
(or add a file-specific override) so the current alignment is allowed—apply the
chosen fix to all affected blocks referenced (the env sections around the shown
keys and the other reported line ranges).
- Around line 383-389: The gh CLI calls use the numeric milestone ID
(steps.docs-milestone.outputs.number) but gh pr create / gh pr edit require the
milestone title; update the two commands that pass --milestone
"$MILESTONE_NUMBER" to instead pass --milestone "$MILESTONE_TITLE" (use the
output from steps.vars that exposes MILESTONE_TITLE) for both the "Create docs
PR" and "Update existing docs PR" steps, and add MILESTONE_TITLE to the env of
the "Update existing docs PR" step so the variable is available there.
- Around line 29-42: The job-level condition for the open-docs-pr job currently
uses if: github.event.label.name == 'Docs/Needed' which allows
maintainer-labeled fork PRs to run in the fork's restricted context; update the
condition on the open-docs-pr job (job name: open-docs-pr) to also require the
PR comes from the same repo by adding &&
github.event.pull_request.head.repo.full_name == github.repository, or
alternatively change the workflow trigger from pull_request to
pull_request_target and apply the usual checkout/secret-injection safety checks;
ensure the modified if sits on the open-docs-pr job to prevent fork PRs from
accessing secrets.
- Around line 445-449: Update the comment text assembled into COMMENT so it no
longer promises automated replacement of the "Docs/Needed" label; change or
remove the trailing sentence that says the label "will be replaced with
`Docs/Done` once the docs PR is merged or closed." Instead, state that the label
must be removed by a human (or soften to say it "should be removed manually") to
match the existing docs-impact-review.yml behavior; edit the MESSAGE built with
DOCS_LINK/COMMENT accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 06b7e836-46c2-4ad7-b2a5-104fe15cb37b
📒 Files selected for processing (1)
.github/workflows/docs-needed.yml
esethna
approved these changes
Apr 27, 2026
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.github/workflows/docs-needed.yml (1)
437-440:⚠️ Potential issue | 🟡 MinorComment text still promises automation that doesn't exist.
Line 440 states the
Docs/Neededlabel "will be replaced withDocs/Doneonce the docs PR is merged or closed." This promises automation that isn't implemented in this PR and contradicts the intentional design where a human must explicitly remove theDocs/Neededlabel.This was previously flagged and marked as addressed, but the problematic text persists in the current code.
📝 Suggested wording
- "The Docs/Needed label will be replaced with Docs/Done once the docs PR is merged or closed.") + "A maintainer will remove the Docs/Needed label and apply Docs/Done once the docs PR is merged.")Based on learnings: the
Docs/Neededlabel is intentionally only ever added by the workflow and never removed—a human must explicitly remove the label even if downstream signals suggest docs are no longer needed.,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 437 - 440, Update the COMMENT text constructed in the COMMENT variable so it no longer promises automation that removes the Docs/Needed label; change the sentence that currently reads "The Docs/Needed label will be replaced with Docs/Done once the docs PR is merged or closed." to clearly state that a human must explicitly remove the Docs/Needed label (e.g., "A human must explicitly remove the Docs/Needed label; it will not be removed automatically when the docs PR is merged or closed."). Ensure the revised string still uses the same COMMENT variable and printf formatting so other workflow behavior is unchanged.
🧹 Nitpick comments (1)
.github/workflows/docs-needed.yml (1)
268-286: Add error handling for the Anthropic API call.The
curl -sfflags cause silent failures. If the API returns an error,jqreceives invalid input and the workflow proceeds with an empty or malformed docs file.♻️ Suggested error handling
- curl -sf https://api.anthropic.com/v1/messages \ + RESPONSE=$(curl -s https://api.anthropic.com/v1/messages \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "anthropic-version: 2023-06-01" \ -H "content-type: application/json" \ - -d "$PAYLOAD" \ - | jq -r '.content[0].text' > /tmp/generated-docs.md + -d "$PAYLOAD") + + if echo "$RESPONSE" | jq -e '.error' > /dev/null 2>&1; then + echo "::error::Claude API error: $(echo "$RESPONSE" | jq -r '.error.message')" + exit 1 + fi + + echo "$RESPONSE" | jq -r '.content[0].text' > /tmp/generated-docs.md🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 268 - 286, The curl call that builds PAYLOAD and writes to /tmp/generated-docs.md uses silent failure flags and pipes straight to jq, so errors produce invalid output; update the curl invocation (the PAYLOAD variable and the curl -> jq pipeline) to capture HTTP status and response body separately (e.g., write response to a temp file), check the HTTP status/code and/or whether curl failed, run jq only if the call succeeded, and on failure print the API error body to the workflow log and exit non-zero so the job fails rather than producing /tmp/generated-docs.md silently; ensure the change references the existing PAYLOAD variable and the final output path /tmp/generated-docs.md.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In @.github/workflows/docs-needed.yml:
- Around line 437-440: Update the COMMENT text constructed in the COMMENT
variable so it no longer promises automation that removes the Docs/Needed label;
change the sentence that currently reads "The Docs/Needed label will be replaced
with Docs/Done once the docs PR is merged or closed." to clearly state that a
human must explicitly remove the Docs/Needed label (e.g., "A human must
explicitly remove the Docs/Needed label; it will not be removed automatically
when the docs PR is merged or closed."). Ensure the revised string still uses
the same COMMENT variable and printf formatting so other workflow behavior is
unchanged.
---
Nitpick comments:
In @.github/workflows/docs-needed.yml:
- Around line 268-286: The curl call that builds PAYLOAD and writes to
/tmp/generated-docs.md uses silent failure flags and pipes straight to jq, so
errors produce invalid output; update the curl invocation (the PAYLOAD variable
and the curl -> jq pipeline) to capture HTTP status and response body separately
(e.g., write response to a temp file), check the HTTP status/code and/or whether
curl failed, run jq only if the call succeeded, and on failure print the API
error body to the workflow log and exit non-zero so the job fails rather than
producing /tmp/generated-docs.md silently; ensure the change references the
existing PAYLOAD variable and the final output path /tmp/generated-docs.md.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 6d83032b-4fb5-4729-a96c-5d21dd9a8b1d
📒 Files selected for processing (1)
.github/workflows/docs-needed.yml
Member
Author
|
@CodeRabbit full review |
Contributor
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
.github/workflows/docs-needed.yml (3)
479-483:⚠️ Potential issue | 🟡 MinorDon’t promise automatic
Docs/Donereplacement.This workflow never swaps the label, and the current wording sets the wrong expectation for PR authors.
Suggested wording
- "The Docs/Needed label will be replaced with Docs/Done once the docs PR is merged or closed.") + "A maintainer can remove Docs/Needed and add Docs/Done once the docs PR is merged or closed, if appropriate.")Based on learnings: for
.github/workflows/docs-impact-review.yml, treatDocs/Neededas intentionally one-way — the workflow may add it, but a human must explicitly remove it.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 479 - 483, The message in the COMMENT variable incorrectly promises that the Docs/Needed label "will be replaced with Docs/Done once the docs PR is merged or closed"; update the COMMENT construction (the DOCS_LINK and COMMENT variables) to remove that promise and instead state that Docs/Needed must be removed manually by maintainers (or that the workflow does not automatically swap labels), e.g. replace the third sentence in COMMENT so it references that a human must remove or replace the Docs/Needed label rather than automatic replacement; ensure the unique symbols DOCS_LINK and COMMENT are preserved and the overall printf format remains unchanged.
389-438:⚠️ Potential issue | 🔴 CriticalPass milestone titles to
gh pr create/edit, not numeric IDs.
steps.docs-milestone.outputs.numberis a numeric milestone ID, butgh pr create --milestoneandgh pr edit --milestoneresolve by milestone title. As written, both the create and re-trigger paths can fail to attach the synced docs milestone.Suggested fix
- name: Create docs PR id: create-pr if: steps.existing.outputs.exists == 'false' env: GH_TOKEN: ${{ secrets.DOCS_REPO_PAT }} BRANCH: ${{ steps.vars.outputs.branch }} MILESTONE_TITLE: ${{ steps.vars.outputs.milestone_title }} - MILESTONE_NUMBER: ${{ steps.docs-milestone.outputs.number }} SOURCE_REPO: ${{ steps.vars.outputs.source_repo }} PR_NUMBER: ${{ steps.vars.outputs.pr_number }} PR_TITLE: ${{ steps.vars.outputs.pr_title }} PR_AUTHOR: ${{ steps.vars.outputs.pr_author }} @@ URL=$(gh pr create \ --repo mattermost/docs \ --head "$BRANCH" \ --base "${{ steps.base-branch.outputs.base }}" \ --title "docs: ${PR_TITLE}" \ --body "$BODY" \ - --milestone "$MILESTONE_NUMBER" \ + --milestone "$MILESTONE_TITLE" \ --draft) @@ - name: Update existing docs PR if: steps.existing.outputs.exists == 'true' env: GH_TOKEN: ${{ secrets.DOCS_REPO_PAT }} DOCS_PR: ${{ steps.existing.outputs.number }} - MILESTONE_NUMBER: ${{ steps.docs-milestone.outputs.number }} + MILESTONE_TITLE: ${{ steps.vars.outputs.milestone_title }} run: | gh pr edit "$DOCS_PR" \ --repo mattermost/docs \ - --milestone "$MILESTONE_NUMBER" + --milestone "$MILESTONE_TITLE"In GitHub CLI, does gh pr create/edit --milestone accept a numeric milestone ID, or only the milestone title/name?🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 389 - 438, The workflow is passing a numeric milestone ID (steps.docs-milestone.outputs.number) to gh pr create/edit --milestone, but GitHub CLI expects the milestone title/name; update both uses (the --milestone arg in the gh pr create call and the gh pr edit call) to pass the milestone title variable (e.g., the existing MILESTONE_TITLE env or another output that contains the milestone title) instead of the numeric ID so the milestone attaches correctly.
29-45:⚠️ Potential issue | 🟠 MajorSkip fork PRs here or move the workflow to
pull_request_target.A maintainer adding
Docs/Neededto a fork PR still triggers this job in the fork-scopedpull_requestcontext. The laterDOCS_REPO_PAT/ANTHROPIC_API_KEYsteps then fail because repo secrets are not available there, so external contributors get a misleading red X.Suggested guard if this automation is intended for same-repo PRs only
- if: github.event.label.name == 'Docs/Needed' + if: >- + github.event.label.name == 'Docs/Needed' && + github.event.pull_request.head.repo.full_name == github.repositoryFor GitHub Actions, when a workflow is triggered by pull_request types:[labeled] on a PR from a fork, are repository secrets available to the workflow, or is pull_request_target required for secret access?🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/docs-needed.yml around lines 29 - 45, The workflow runs on labeled pull_request events and currently triggers for forked PRs where repo secrets aren't available; update the 'open-docs-pr' job's if conditional (the if on github.event.label.name == 'Docs/Needed') to also ensure the PR is from the same repository (e.g., require github.event.pull_request.head.repo.full_name == github.repository or that github.event.pull_request.head.repo.fork == false) so the job only runs for same-repo PRs, or alternatively move this workflow to pull_request_target if you intentionally need access to repository secrets for fork PRs.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/docs-needed.yml:
- Around line 118-129: The gh api call that sets MM_MILESTONE compares
MILESTONE_DUE to the full ISO timestamp .due_on, causing mismatches when times
differ; modify the jq filter used in the MM_MILESTONE assignment to compare only
the calendar date portion of .due_on (e.g., the first 10 characters) to
MILESTONE_DUE so matching is done by YYYY-MM-DD; update the --jq expression in
the MM_MILESTONE assignment (the gh api invocation that produces MM_MILESTONE)
to use a date-only comparison such as extracting .due_on[0:10] or equivalent in
jq.
---
Duplicate comments:
In @.github/workflows/docs-needed.yml:
- Around line 479-483: The message in the COMMENT variable incorrectly promises
that the Docs/Needed label "will be replaced with Docs/Done once the docs PR is
merged or closed"; update the COMMENT construction (the DOCS_LINK and COMMENT
variables) to remove that promise and instead state that Docs/Needed must be
removed manually by maintainers (or that the workflow does not automatically
swap labels), e.g. replace the third sentence in COMMENT so it references that a
human must remove or replace the Docs/Needed label rather than automatic
replacement; ensure the unique symbols DOCS_LINK and COMMENT are preserved and
the overall printf format remains unchanged.
- Around line 389-438: The workflow is passing a numeric milestone ID
(steps.docs-milestone.outputs.number) to gh pr create/edit --milestone, but
GitHub CLI expects the milestone title/name; update both uses (the --milestone
arg in the gh pr create call and the gh pr edit call) to pass the milestone
title variable (e.g., the existing MILESTONE_TITLE env or another output that
contains the milestone title) instead of the numeric ID so the milestone
attaches correctly.
- Around line 29-45: The workflow runs on labeled pull_request events and
currently triggers for forked PRs where repo secrets aren't available; update
the 'open-docs-pr' job's if conditional (the if on github.event.label.name ==
'Docs/Needed') to also ensure the PR is from the same repository (e.g., require
github.event.pull_request.head.repo.full_name == github.repository or that
github.event.pull_request.head.repo.fork == false) so the job only runs for
same-repo PRs, or alternatively move this workflow to pull_request_target if you
intentionally need access to repository secrets for fork PRs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 56f9365e-3d87-47dd-b6b7-56dac58fa8e1
📒 Files selected for processing (1)
.github/workflows/docs-needed.yml
amyblais
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Create an automation for the process of opening docs PRs for dev PRs with the "Docs/Neded" label. This same automation will be applied to mobile/desktop repos.
Associated PR in the docs repo: mattermost/docs#8844.
Full spec doc: https://mattermost.atlassian.net/wiki/spaces/Security/pages/4450746369/Automate+Documentation+PR+Workflow+Spec+Milestone+Plan.
Change Impact: 🟠 Medium
Regression Risk: The workflow introduces cross-repository automation that creates/updates branches, PRs, and milestones in mattermost/docs and calls an external API (Anthropic); misconfiguration, token or API failures, and milestone/branch-resolution edge cases could cause unintended PRs or failures during runtime. There are untested runtime paths (API errors, branch-matching fallbacks, milestone due-date matching) that may only surface during execution.
QA Recommendation: Moderate manual QA recommended: run end-to-end tests in a staged or forked environment to verify label-trigger behavior, correct branch/milestone resolution across repos, idempotency on re-labeling, error handling for missing prompts/API failures, and secret/token scope validation.
Generated by CodeRabbitAI