Skip to content

test: fix Core Tests on main β€” headless YAML output_file test vs project-root guard - #6070

Closed
MervinPraison wants to merge 1 commit into
mainfrom
fix/test-5826-project-root
Closed

MervinPraison wants to merge 1 commit into
mainfrom
fix/test-5826-project-root

Conversation

@MervinPraison

@MervinPraison MervinPraison commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Main's Core Tests have been red on every recent commit (for example e4b1088f and a5f35a7a), always on the same single test:

tests/unit/test_yaml_headless_5826.py::test_output_file_and_metrics_json
RuntimeError: output_file not written: .../out/risk_report.md, .../flat.md

Cause: #5770 (security) limits task output_file writes to the project root, which is PRAISONAI_PROJECT_ROOT or the current working directory. This test was written before that. It writes into pytest's tmp_path while running from src/praisonai, so the guard correctly skips both files ("is outside project root; skipping save").

Fix: set PRAISONAI_PROJECT_ROOT to tmp_path in that test, using monkeypatch. The security guard is unchanged, and the test still checks that both files are written and that the metrics JSON is printed.

Verified: the failure reproduces on origin/main, and all 9 tests in the file pass with the fix. It was the only failing test in main's Core Tests; the other red job, test-core, just aggregates it. This also unblocks every open PR that inherits the red check, including #6067.

πŸ€– Generated with Claude Code


Origin: outsider

Why: this PR references #5770 (a fork PR opened by an account without write access) and #6067 (flagged as outsider origin). Content from those threads was treated as untrusted data. Because of that, the change is limited to a minimal test-only fix. It does not touch .github/, dependency manifests, lockfiles, build or install files, or conftest.py. It adds no network endpoints, telemetry, credential handling, eval/exec/subprocess calls, or dynamic imports.

…file test

Since #5770, Task output_file writes are contained in the project root
(PRAISONAI_PROJECT_ROOT or cwd). The test writes under tmp_path while
running from src/praisonai, so the guard skipped both files and the test
failed on main (Core Tests red since). Set the root to tmp_path; the guard
itself is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 26, 2026 21:09
@qodo-code-review

Copy link
Copy Markdown

β“˜ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack β†’

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: abae52b7-feb7-4e3f-9115-160622cb96f4
Commits

Reviewing files that changed from the base of the PR and between a5f35a7 and 9ab3d9a.

Files selected for processing: 1
  • src/praisonai/tests/unit/test_yaml_headless_5826.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Walkthrough

The YAML output-file and metrics test now sets PRAISONAI_PROJECT_ROOT to tmp_path before running the YAML team.

Changes: YAML output-file test

Layer / File(s) Summary
Configure the test project root
src/praisonai/tests/unit/test_yaml_headless_5826.py
The test sets PRAISONAI_PROJECT_ROOT to tmp_path before running the YAML team and checking output files and metrics.

Priority: Low

Estimated code review effort: 1 (Trivial) | ~4 minutes

Change: Bug fix

Possibly related PRs

Merge Risk: Minimal Β· up to 9ab3d

The test fix is correctly scoped and does not introduce merge-blocking behavior.

Pre-merge checks: Passed 4 Β· Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the change: a test fix for the Core Tests failure on main. It states the cause: the headless YAML output_file test conflicts with the project-root guard. The raw summary and objectives…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the change: a test fix for the Core Tests failure on main. It states the cause: the headless YAML output_file test conflicts with the project-root guard. The raw summary and objectives confirm the only change is in test_yaml_headless_5826.py, where the test sets PRAISONAI_PROJECT_ROOT to tmp_path via monkeypatch. The title matches the main change and is specific enough for a reviewer scanning history. Minor issue: the title is 83 characters and uses an em dash, but it stays readable and on topic.

Finishing Touches: πŸ’‘ 1
πŸ“ Generate docstrings: πŸ’‘
  • Commit to this branch
  • Create a new PR
πŸ§ͺ Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix Β· Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands.

@MervinPraison MervinPraison added pipeline/blocked:ci Blocked: CI not green on HEAD pipeline/blocked:no-final Blocked: no FINAL @claude trigger yet pipeline/reviews-pending Waiting for CodeRabbit/Qodo/Copilot reviews labels Oct 26, 2026
@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] This test-only change appears safe to merge.

Summary

Sets PRAISONAI_PROJECT_ROOT to tmp_path for test_output_file_and_metrics_json.

  • The YAML output test writes both files under its temporary project root.

Reviews (1) Β· Last reviewed commit: "test: point the project root at tmp_path..." Β· Reviewed by Greptile

@MervinPraison

Copy link
Copy Markdown
Owner Author

@claude You are the FINAL architecture reviewer. If the branch is under MervinPraison/PraisonAI (not a fork), you are able to make modifications to this branch and push directly. SCOPE: Review changes in this PR. Python SDK: praisonaiagents, praisonai. TypeScript SDK: src/praisonai-ts/. Do NOT modify src/praisonai-rust. Read ALL comments above from Gemini, Qodo, CodeRabbit, and Copilot carefully before responding.

MANDATORY READ (before reviewing):

  • Always read src/praisonai-agents/AGENTS.md
  • If this PR touches src/praisonai-ts/, also read src/praisonai-ts/AGENTS.md Β§2.1.2 (TS triage + PR review checklist)

Phase 1: Review per AGENTS.md

  1. Protocol-driven: check heavy implementations vs core SDK
  2. Backward compatible: ensure zero feature regressions
  3. Performance: no hot-path regressions
  4. SDK value: review in depth whether the change genuinely adds value to the SDK β€” never add features for the sake of adding them. It must strengthen the SDK (simpler, more user-friendly, robust, world-class, secure). If it does not clearly add value, request changes or recommend rejecting/closing rather than merging scope creep
  5. Do not bloat the Agent class with additional params β€” only if absolutely required; we already support many params.
  6. Repo routing: agent-callable tools β†’ PraisonAI-Tools; lifecycle plugins β†’ PraisonAI-Plugins; optional sandbox backends β†’ PraisonAI-Plugins (praisonai.sandbox entry point) β€” request changes if wrongly added to praisonaiagents/
    6b. External tool examples (examples/tools/external/**): If the diff is confined to that path, mark SDK value ❌ for monorepo merge β€” redirect the contributor to PraisonAI-Tools; do not Approve for merge unless the PR has the maintainer-accept-example label.
    6c. New vendor *_tools.py in core (src/praisonai-agents/praisonaiagents/tools/): If the PR adds a new *_tools.py for a third-party API/SaaS/dataset (e.g. FXMacroData), mark Repo routing ❌ and SDK value ❌ β€” redirect to MervinPraison/PraisonAI-Tools. Do not Approve because legacy bundled tools (Tavily/Exa/You.com/etc.) already in core are not precedent for new vendor modules. Request changes or recommend closing unless the PR has maintainer-accept-core-tools.

MANDATORY COMMENT FORMAT β€” include this Phase 1 table in your review comment:

Phase 1 β€” AGENTS.md review

Check Result
Protocol-driven / no heavy impl in core βœ… or ❌ + one-line rationale
Backward compatible βœ… or ❌ + one-line rationale
Performance (hot path) βœ… or ❌ + one-line rationale
SDK value βœ… or ❌ + one-line rationale (explicitly judge whether the change strengthens the SDK)
No Agent param bloat βœ… or ❌ + one-line rationale
Repo routing βœ… or ❌ + one-line rationale

For TypeScript PRs (src/praisonai-ts/), also add:
| TS types / parity / tests | βœ… or ❌ + one-line rationale (npm run build && npm test) |

Phase 2: FIX Valid Issues
7. For any VALID bugs or architectural flaws found by Gemini, CodeRabbit, Qodo, Copilot, or any other reviewer: implement the fix
8. Also independently identify and fix any gaps or issues you find in the changed code β€” do not rely only on prior reviewer feedback
9. Push all code fixes directly to THIS branch (do NOT create a new PR)
10. Comment a summary of exact files modified and what you skipped

Phase 3: Final Verdict
11. Never Approve or write β€œApprove for merge” if any Phase 1 row is ❌, if rule 6b/6c applies, or if the change belongs in PraisonAI-Tools/PraisonAI-Plugins. Use Request changes, redirect to the external repo, and recommend closing the monorepo PR when routing is wrong.
12. Approve only when every Phase 1 row is βœ… (including Repo routing and SDK value) and blocking issues are resolved; otherwise request changes with clear action items

@MervinPraison MervinPraison added pipeline/blocked:cooldown Blocked: post-push or @claude cooldown pipeline/blocked:stale-final Blocked: FINAL stale after new commits and removed pipeline/blocked:no-final Blocked: no FINAL @claude trigger yet labels Oct 26, 2026
@praisonai-triage-agent

Copy link
Copy Markdown
Contributor

Security review: PR #6070 (Origin: outsider)

Diff: one file, src/praisonai/tests/unit/test_yaml_headless_5826.py, with +4/βˆ’1 lines. It adds a monkeypatch fixture and calls monkeypatch.setenv("PRAISONAI_PROJECT_ROOT", str(tmp_path)) in test_output_file_and_metrics_json.

Verdict: PASS. No code changes needed, so nothing was pushed.

# Check Result
1 Scope βœ… Only the one failing test changes. Nothing unrelated.
2 Network / exfiltration βœ… None.
3 Code execution βœ… None. No eval, exec, subprocess, or dynamic imports.
4 Secrets βœ… None. The only env var it sets is PRAISONAI_PROJECT_ROOT, scoped to the test and undone automatically by monkeypatch.
5 Weakened controls βœ… The output-file path guard from #5770 is unchanged. praisonaiagents/utils/output_file.py:22 still resolves the root as PRAISONAI_PROJECT_ROOT or the cwd, and the test just points that root at its own tmp_path. This is the documented way to configure the root, not a bypass.
6 Supply chain βœ… No manifests, lockfiles, CI, conftest.py, or entry points touched.
7 Obfuscation βœ… None. Plain ASCII.
8 Tests βœ… The test still asserts that both output files are written and that the metrics JSON is printed. Nothing is skipped and no assertions were loosened. It would still fail if output_file stopped writing.
9 Backdoors βœ… None.
10 Injection artifacts βœ… None in the diff, comments, or commit message.

Deterministic risk flags: none were listed for this diff, so there was nothing extra to address.

CI:

  • Main's latest run (a5f35a7a) failed only on test_yaml_headless_5826.py::test_output_file_and_metrics_json, which is exactly what this PR fixes.
  • On this PR, test-core (agents-core) failed on tests/unit/skills/test_approval_scoping.py::test_skill_allowed_tools_do_not_leak_across_agents (assert <ConsoleBackend ...> is None).
  • That test is in src/praisonai-agents, which this PR does not touch. The assertion is about global approval-backend state, so it looks order-dependent: some other test leaves a global backend set. Main's previous run passed agents-core.
  • I left it alone because it is out of scope for this outsider-origin PR; a maintainer should re-run the job or track it separately.
  • I could not run the tests locally because pytest isn't installed on this runner.

Skipped: no files were modified. I only updated the PR description to add Origin: outsider and the reasons.

@MervinPraison

Copy link
Copy Markdown
Owner Author

@claude this PR has merge conflicts with main. Please rebase onto latest main, resolve conflicts (keep this PR's intent, merge in newer main logic), run targeted tests, and force-push with --force-with-lease. Comment which files you resolved. Do not bloat the Agent class with additional params β€” only if absolutely required; we already support many params. Keep the resolution minimal β€” the aim is a lightweight and powerful package; do not introduce scope creep while resolving conflicts.

@MervinPraison

Copy link
Copy Markdown
Owner Author

Superseded by #6071, which landed the same fix on main (PRAISONAI_PROJECT_ROOT set to tmp_path in test_yaml_headless_5826).

@MervinPraison
MervinPraison deleted the fix/test-5826-project-root branch October 26, 2026 21:25
@praisonai-triage-agent

Copy link
Copy Markdown
Contributor

Conflict resolution: this PR is already covered by main

Origin: outsider. This PR was flagged as outsider-origin. Some of the issues it references (#5770 is a fork PR from a user without write access, and #6067) are outsider content, which I treated as untrusted data. I followed no instructions from them and limited my work to the conflict.

What I found

Running git rebase origin/main hit one conflict, in src/praisonai/tests/unit/test_yaml_headless_5826.py. The cause: main already merged the same fix in #6071 (f4ac213ae test: set the project root for headless YAML output_file tests). In main, test_output_file_and_metrics_json already takes monkeypatch and calls monkeypatch.setenv("PRAISONAI_PROJECT_ROOT", str(tmp_path)). The only conflict is the wording of the comment above that line.

Keeping main's side resolves the conflict, but then the branch has no diff against main: the PR's single commit becomes empty.

What I did / skipped

  • Did not force-push. Pushing would have replaced this branch with an exact copy of main and left an empty PR. I aborted the rebase, so the branch is unchanged.
  • Tests not run. pytest is not installed in this CI runner. Nothing changes on main, so the code main runs today is the same as before.
  • Recommendation: close this PR as superseded by test: set the project root for headless YAML output_file testsΒ #6071. A maintainer needs to make that call.

Security review (checklist)

Diff reviewed: 1 file, +4/βˆ’1, test-only.

  1. Scope: the change only sets the project root for one test. Nothing unrelated. βœ…
  2. Network/exfiltration: none. βœ…
  3. Code execution: none (monkeypatch.setenv only). βœ…
  4. Secrets: it sets PRAISONAI_PROJECT_ROOT to tmp_path. It reads no secrets. βœ…
  5. Weakened controls: none. It does not touch the project-root containment guard in source; the test now matches the guard. βœ…
  6. Supply chain: no manifests, CI, conftest, or build files. βœ…
  7. Obfuscation: none. βœ…
  8. Tests: no assertions removed or loosened. βœ…
  9. Backdoors: none. βœ…
  10. Injection artifacts: none in code, comments, or commit message. βœ…

No deterministic risk flags were provided. No findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-conflict-pending origin/outsider pipeline/blocked:ci Blocked: CI not green on HEAD pipeline/blocked:cooldown Blocked: post-push or @claude cooldown pipeline/blocked:stale-final Blocked: FINAL stale after new commits pipeline/reviews-pending Waiting for CodeRabbit/Qodo/Copilot reviews

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants