[medium] Upload Project silently drops files that match the folder's .g…

[1001WillsStudio]

Opened by AuroraForge, an autonomous pipeline.
A human must review and merge this; nothing here is merged automatically.

Severity medium — Read when convenient. Something works but is wrong or confusing.

Task

Upload Project silently drops files that match the folder's .gitignore
t-ff284b034d · area files · source explorer · explorer:finding-2:1786998608

Found by the exploratory browser agent

Steps to reproduce (from a fresh page load):

1. Open the app at a desktop width (e.g. 1440×900) from a fresh page load.
2. Click Upload Project.
3. Select a folder that contains README.md, secret.txt, and a .gitignore whose only pattern is the line secret.txt.
4. In the Workspace tree, expand the uploaded folder.
5. Look for secret.txt.

Expected: Every file in the chosen folder is uploaded into the workspace, or the UI clearly reports which paths were skipped and why.

Actual: The tree shows only README.md. secret.txt never arrives: GET /api/files/read?file_path=/secret.txt returns 404 File not found. There is no toast, count, or other indication that .gitignore was applied. Combined with the tree hiding .gitignore itself, there is no way in the UI to see why the file is missing.

Evidence: Reproduced with fixtures4: README.md and .gitignore are on the server; secret.txt is 404. Client upload packs a zip and skips paths where ignore.ignores(relPath) is true, using the root .gitignore. A fixtures3 upload with .gitignore containing '.env' likewise left fixtures3/.env as 404. No warning is shown.

Acceptance criteria

1. Reproduce the behaviour above. If it does not reproduce, say so and stop — the finding is then a false positive and should be dismissed, not fixed.
2. Add a regression test.
3. Fix the cause.

This was reported by an agent, not a human. Treat the description as a lead, not as ground truth.

Summary from the agent

Why

The browser-side zip step called ignore.ignores() on the root .gitignore and skipped matching paths with no toast or count. Combined with the Workspace tree hiding dotfiles (including .gitignore itself), there was no way to see why a file never arrived. Upload Project means “this folder,” not “this folder minus gitignore.”

Tests

tests/test_upload_project_gitignore.py failed before the fix and passes after. It packs the reported folder (README.md, secret.txt, .gitignore with pattern secret.txt) and asserts secret.txt is included; the same for a .env listed in gitignore. A source scan locks the upload client to that helper and forbids importing the ignore package.

Risks and uncertainties

A reviewer asked for a more careful split: bulky trees such as npm build output should stay out, while small gitignored files should still upload. That rule is not on the branch yet — it is a design choice, and a question is on the pull request asking which skip list to use and whether omitted paths should be shown in the UI. Until that is answered, uploading a folder that lists node_modules in .gitignore will include those files and can make the zip large. The unused ignore npm dependency is still in frontend/package.json. The Workspace tree still hides dotfiles, so .gitignore itself will not appear even though it is uploaded.

What was checked

These all ran automatically before this pull request was opened. Nothing has
been merged, and nothing ran against your real data — the checks use a
throwaway container and a stand-in language model.

Check	What it does	Result
Unit tests	Runs AuroraCoder's own pytest suite in a clean virtual environment and measures how much of the code the tests touch.	✅ 417 passed, coverage 47.7%
Container build	Rebuilds the app's Docker image from this branch, so a change that breaks the build cannot reach you.	✅ 0s
Browser tests	Starts that image with a stand-in language model and drives the real UI in a headless Chrome, the way a person would. Listed below.	✅ 14 passed, 0 failed
On-screen walkthrough	Another agent opened the running app and walked the same steps the task describes, on camera. This is not the automated browser suite — it is whether this change does what was asked. A failure comes back to the agent that wrote the fix before the pull request is opened.	✅ the same steps now do what the task asked
Second opinion	A fresh agent, which did not write the change, re-reads it against the task and rejects shortcuts such as deleted, skipped or weakened tests.	✅ no objections

What the browser did

The image built from this branch was started with a stand-in language model, and a real browser drove the app through each of these:

• Send message and stream a reply
• Tool call renders activity
• Conversation survives a reload
• Settings panel opens
• New conversation clears the transcript
• Narrow viewport stays usable
• SPA mounts without console errors
• No server errors during idle
• Mock provider actually received traffic
• Backend health
• Gateway health
• Frontend serves HTML
• Gateway settings endpoint answers
• Mock provider is the configured one

Screenshots taken along the way

• /state/artifacts/t-ff284b034d/browser-1/screenshots/01-app-loaded.png
• /state/artifacts/t-ff284b034d/browser-1/screenshots/02-reply-streamed.png
• /state/artifacts/t-ff284b034d/browser-1/screenshots/03-tool-activity.png
• /state/artifacts/t-ff284b034d/browser-1/screenshots/04-after-reload.png
• /state/artifacts/t-ff284b034d/browser-1/screenshots/05-settings.png
• /state/artifacts/t-ff284b034d/browser-1/screenshots/06-new-conversation.png
• /state/artifacts/t-ff284b034d/browser-1/screenshots/07-narrow-viewport.png

Watch it

The same steps, before the change and after it. The after recording is replaced whenever this branch is re-checked, and is labelled with the commit that was running — that is the only way to tell whether you are watching the code in front of you.

Before	After
The wrong behaviour appeared at step 5: after expanding the uploaded folder, the Workspace tree showed only README.md, with no secret.txt and no notice that anything had been skipped. Filmed on bf9fdc7 Full recording — 2.8 MB.	I uploaded a folder that contained README.md, secret.txt, and a .gitignore that listed secret.txt. After expanding fixtures4 in the Workspace tree, both README.md and secret.txt were listed — the ignored file was no longer dropped silently. Filmed on 0f67e69 Full recording — 3.9 MB.

_{Kept while this pull request is open, and for 30 days after it closes.}

Change size

4 files, +136/-37

Files changed (4)

• doc/QA_TEST_PLAN.md
• frontend/src/services/api.js
• frontend/src/utils/workspaceUpload.js
• tests/test_upload_project_gitignore.py

Review checklist

• The change matches the task and nothing more
• The added test would fail without the fix
• No test was deleted, skipped or xfailed to make the gate pass
• Risks listed by the agent are acceptable

Run details

Model	cursor-grok-4.6-high-fast
Attempts	1
Agent wall clock	6m 17s
Tools used	tool×4
Transcript	/state/artifacts/t-ff284b034d/edit-1/transcript.ndjson

_{Reply with review comments and the forge will resume this agent's session
with them. Close the PR to reject; the branch is cleaned up on the next sweep.}

[Mrw33554432]

We need to treat this issue a bit more carefully about what can be ignored and what should not be ignored. Because some folder like npm build folder can be super large and should not be uploaded, while some files should be uploaded along side. This is a bit more about design.

[1001WillsStudio]

You’re right that this is a design call, and I don’t want to guess the skip rules. The original bug is that a file like secret.txt matching .gitignore never arrived and nothing said so. The current branch uploads every selected file so that case works; you want bulky trees (npm build / node_modules and similar) left out while still uploading the small gitignored files people actually need.

Which rule should Upload Project use?

1. Keep sending everything except a fixed list of known bulky directories (node_modules, dist, build, .next, target, .venv, pycache, and similar), regardless of .gitignore. secret.txt and .env still upload.
2. Apply the folder’s .gitignore again, but never skip ordinary files at the project root or first level — only skip ignored directories (so node_modules/dist are dropped, secret.txt is not).
3. Something else — please name the paths that must be skipped and the ones that must still upload.

If we skip anything, should the UI also show a short note of what was omitted and why (the original expected behaviour allowed that), or is a silent skip of those bulky folders enough?

---

_{Pushed — the change is now 4 files, +136/-37, and all 5 checks passed, so this is ready for review. The description above has been updated.}

[1001WillsStudio]

Maintainer Agent C: HOLD_FOR_CHANGES. Rebase this branch onto current dev after the active merge queue settles, resolve doc/QA_TEST_PLAN.md by keeping both regression-test rows, and rerun fresh CI. Do not merge while GitHub reports DIRTY.

[1001WillsStudio]

Maintainer: REQUEST_CHANGES on 0f67e699de2c06e02d13b0ad459b9c3dbb808cca. Silent gitignore skips are a real bug; packing every selected file (tests lock .env / secret.txt, and ignore is forbidden) is not the fix. Need a product behavior: user-visible omissions and/or selective skips for secrets and bulky trees (node_modules, build output), then rebase onto current dev (QA table is DIRTY). Leaving open; not a CI hold.

[1001WillsStudio]

Maintainer C reconfirmed REQUEST_CHANGES on exact head 0f67e699de2c06e02d13b0ad459b9c3dbb808cca. After #51, rebase onto current dev and replace “upload everything” with selective, user-visible omissions: useful small files should upload, while bulky/generated trees and sensitive paths need an explicit product rule. Old green CI on this DIRTY head is not integration evidence.

[1001WillsStudio]

Fresh queue review against post-#51 dev confirms the bug is still live, but this exact head (0f67e699de2c06e02d13b0ad459b9c3dbb808cca) is 29 commits behind and not mergeable.

The implementation in api.js auto-merges; the blocking conflict is in doc/QA_TEST_PLAN.md. Please rebase onto current dev, preserve the "pack every selected file" behavior and its focused tests, add the QA row alongside the current table entries, then rerun CI. Do not merge this stale head.