From d59ddbaf9569c55a49ed03274bd4e28dd4f51d48 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <41898282+claude[bot]@users.noreply.github.com> Date: Mon, 12 Jan 2026 06:44:38 +0000 Subject: [PATCH] Create plan for issue #10 Co-Authored-By: Claude --- plans/10.md | 209 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 plans/10.md diff --git a/plans/10.md b/plans/10.md new file mode 100644 index 0000000..9f83f37 --- /dev/null +++ b/plans/10.md @@ -0,0 +1,209 @@ +# Implementation Plan for Issue #10 + +## Issue Summary + +The model may fail to post replies to review comments, resulting in wasted tokens and poor UX. This happens because: + +1. The `gh` CLI doesn't provide a straightforward way to reply to inline review comments +2. When the model doesn't follow instructions correctly, no comment is posted at all +3. Users have no visibility into what happened or why there's no response + +From the failed workflow run (https://github.com/TomzxCode/llm-augmented-workflows/actions/runs/20909952268), the permission_denials log shows that the `Skill` tool was denied, but the workflow still completed without posting any reply. + +## Feasibility Assessment + +- **Status**: ✅ Feasible +- **Complexity**: Low +- **Estimated Effort**: Low + +The solution is straightforward. The GitHub REST API provides an endpoint for replying to inline review comments: +``` +POST /repos/{owner}/{repo}/pulls/comments/{comment_id}/replies +``` + +This endpoint requires: +- `comment_id`: The database ID of the parent comment (not the node_id) +- `body`: The reply content + +This approach has been validated and works correctly. + +## Proposed Solution + +### Overview + +Instead of using `gh pr comment` which only creates general PR-level comments, we need to use the GitHub REST API to post replies to inline review comments. The solution involves: + +1. **Pass the inline comment ID through the workflow**: Update the workflow to capture and pass the inline comment's database ID +2. **Update the skill to use the correct API**: Modify the skill to detect inline comments and use the appropriate API endpoint +3. **Add fallback visibility**: Ensure users are informed even when the model can't post a proper reply + +### Implementation Steps + +1. **Update `claude-review.yml` to pass inline comment ID** + + - Files to modify: `.github/workflows/claude-review.yml` + - Approach: Add a new output variable to capture the inline comment's database ID when the event is `pull_request_review_comment` + - Testing considerations: Verify the ID is correctly extracted for both inline and general comments + + ```yaml + # In the "Check if this is a plan PR comment" step, add: + core.setOutput('comment_id', context.payload.comment.id || ''); + ``` + + Then pass it as an environment variable: + ```yaml + COMMENT_ID=${{ github.event.comment.id }} + ``` + +2. **Update `review-plan-comment.md` to handle inline replies** + + - Files to modify: `.claude/commands/review-plan-comment.md` + - Approach: Add logic to detect `COMMENT_TYPE=inline` and use the REST API for replies + - Testing considerations: Test both inline and general comment scenarios + + The skill should: + - When `COMMENT_TYPE=inline`: Use `gh api repos/{owner}/{repo}/pulls/comments/{comment_id}/replies -f body="..."` + - When `COMMENT_TYPE=general`: Use `gh pr comment $PR_NUMBER --body "..."` + +3. **Add fallback comment posting** + + - Files to modify: `.claude/commands/review-plan-comment.md` + - Approach: If posting a reply fails (e.g., API error), fall back to posting a general PR comment with context + - Testing considerations: Test error handling by temporarily using invalid comment IDs + + Example fallback: + ```bash + # If inline reply fails, post a general comment referencing the original + gh pr comment $PR_NUMBER --body "Reply to ${COMMENT_TYPE} comment on ${path}:${line}: [response]" + ``` + +4. **Add response posting verification** + + - Files to modify: `.claude/commands/review-plan-comment.md` + - Approach: After posting, verify the comment was created successfully and log any failures + - Testing considerations: Ensure the skill exits with appropriate status + +### Dependencies + +- External dependencies: None (using existing `gh` CLI) +- Internal dependencies: + - `.github/workflows/claude-review.yml` - provides the context + - `.claude/commands/review-plan-comment.md` - consumes the context + +### Risk Assessment + +- Potential risks: + - **Risk**: The comment ID from GitHub event may be the `node_id` instead of `database_id` + - **Mitigation**: The `github.event.comment.id` in GitHub Actions should already be the database ID. Test to confirm. + - **Risk**: Model may still fail to call the `gh` command + - **Mitigation**: This is a model behavior issue, but having clearer instructions in the skill helps + - **Risk**: API permissions may be insufficient + - **Mitigation**: The workflow already has `pull-requests: write` permission, which should be sufficient + +## Alternatives Considered + +1. **Use GraphQL API for replies**: Initially explored using `addPullRequestReviewThreadReply` mutation, but the REST API endpoint is simpler and works reliably. + +2. **Always post general comments regardless of comment type**: This would ensure visibility but loses the threading context for inline comments. + +3. **Modify the Claude Code action**: Could add built-in support for posting replies, but this would require upstream changes and increases dependency on external tooling. + +## Implementation Details + +### Modified `.github/workflows/claude-review.yml` (Step 1) + +The `Check if this is a plan PR comment` step needs to output the comment ID: + +```yaml +- name: Check if this is a plan PR comment + id: check-plan + uses: actions/github-script@v8 + with: + script: | + // ... existing code ... + let commentId = ''; + + if (eventName === 'pull_request_review_comment') { + // For inline comments, get the database ID + commentId = context.payload.comment.id; + // ... rest of existing code ... + } else if (eventName === 'issue_comment') { + // ... existing code ... + } + + core.setOutput('comment_id', commentId); +``` + +Then pass it to the skill: + +```yaml +- name: Respond to review comment + # ... existing config ... + with: + prompt: | + /review-plan-comment + REPO=${{ github.repository }} + PR_NUMBER=${{ github.event.pull_request.number }} + COMMENT_AUTHOR=${{ github.event.comment.user.login }} + COMMENT_BODY=${{ github.event.comment.body }} + COMMENT_TYPE=${{ github.event_name == 'pull_request_review_comment' && 'inline' || 'general' }} + COMMENT_ID=${{ steps.check-plan.outputs.comment_id }} +``` + +### Modified `.claude/commands/review-plan-comment.md` (Step 2) + +Add new input variable: + +```markdown +## Input Context +Environment variables provided by the workflow: +- `REPO`: The GitHub repository +- `PR_NUMBER`: The pull request number +- `COMMENT_AUTHOR`: The person who wrote the comment +- `COMMENT_BODY`: The content of the comment +- `COMMENT_TYPE`: Either "inline" (line-specific) or "general" (PR-level) +- `COMMENT_ID`: The database ID of the comment (for inline comments only) +``` + +Add a helper function section: + +```markdown +## Helper Functions + +### Post Response + +Use this function to post responses: + +**For inline comments:** +```bash +gh api repos/$REPO/pulls/comments/$COMMENT_ID/replies -f body="$RESPONSE" +``` + +**For general comments:** +```bash +gh pr comment $PR_NUMBER --body "$RESPONSE" +``` +``` + +Update the Type B (Question) and Type C (Discussion) response instructions: + +```markdown +**Type B: Question** +Action: +- Provide a clear, helpful answer using the appropriate comment posting method +- No plan updates needed + +**Type C: Discussion** +Action: +- Engage thoughtfully in the discussion using the appropriate comment posting method +- No plan updates unless you agree and it improves the plan +- If you do update, follow Type A process +``` + +## References + +- Related issues: #10 +- Related PRs: #9 (example of inline comment that wasn't replied to) +- Documentation: + - GitHub REST API: https://docs.github.com/en/rest/pulls/comments#create-a-reply-for-a-review-comment + - GitHub Actions event payload: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_review_comment