Skip to content

feat(init): Preview planned file changes before initialization writes - #4312

Open
darion-yaphet wants to merge 13 commits into
github:mainfrom
darion-yaphet:feat/init-dry-run
Open

feat(init): Preview planned file changes before initialization writes#4312
darion-yaphet wants to merge 13 commits into
github:mainfrom
darion-yaphet:feat/init-dry-run

Conversation

@darion-yaphet

Copy link
Copy Markdown
Contributor

Description

Closes #4311.

Adds specify init --dry-run to preview initialization changes without writing to the requested project.

The command stages the target in a temporary directory, invokes the normal specify init path there, then reports create, overwrite, preserve, and conflict actions. --json emits machine-readable output for CI/tooling. URL extensions are reported as unresolved and are not downloaded during preview.

Also fixes the bundle initializer’s direct callback invocation so it explicitly disables the new dry-run flags.

Testing

  • Tested locally with .venv/bin/specify init --help
  • Ran the full suite with .venv/bin/python -m pytest
    7145 passed, 180 skipped
  • Tested disposable project scenarios through tests/test_init_dry_run.py
    7 passed

Additional validation:

  • git diff --check
  • .venv/bin/python -m compileall -q src/specify_cli
  • Bundle initialization regression test passed.
  • Ruff installed and run; the repository has existing lint violations, so this PR does not claim a clean Ruff baseline.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Dry-run can modify global Hermes files, and several advertised JSON, conflict, skip, and provenance guarantees are incomplete.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds specify init --dry-run to preview initialization changes through staged execution.

Changes:

  • Adds human-readable and JSON preview output.
  • Classifies staged file changes and unresolved URL extensions.
  • Updates bundle initialization and adds dry-run contract tests.
File summaries
File Description
src/specify_cli/commands/init.py Implements dry-run staging and reporting.
src/specify_cli/commands/bundle/__init__.py Supplies new callback flags.
tests/test_init_dry_run.py Covers preview output, parity, conflicts, and extensions.
Review details

Suppressed comments (3)

src/specify_cli/commands/init.py:181

  • A non-forced existing target returns before the initializer is staged, leaving actions empty; moreover, the action classifier never emits conflict. This does not provide the per-artifact conflict plan promised by the PR and issue. Continue planning against staging, then classify would-be overwrites as conflict when force was not requested.
    if directory_conflict:
        _emit_dry_run_preview(payload, json_output=json_output)
        return

src/specify_cli/commands/init.py:535

  • --json does not reliably produce a single JSON document unless callers also specify the selections. With no --integration, non-interactive execution prints the defaulting message at lines 739-741 before the payload, while an interactive terminal opens the selection UI. Make JSON mode select defaults non-interactively and suppress or redirect all pre-payload human output.
        json_output: bool = typer.Option(
            False,
            "--json",
            help="Emit the dry-run preview as a single JSON document.",

src/specify_cli/commands/init.py:230

  • The real initializer treats some requested-source failures as recoverable: for example, a missing preset warns and skips, and an invalid local extension records a tracker error, but both still exit zero. Because successful child output is discarded here, the preview silently omits that requested source and never emits the required skip result. Carry recoverable staged outcomes into the payload instead of only handling nonzero exits.
        if result.returncode:
            details = (result.stderr or result.stdout).strip().replace("\n", " ")
            raise RuntimeError(f"staged initialization failed: {details[:240]}")

        payload["actions"] = _build_preview_actions(initial_files, staged_root)
  • Files reviewed: 3/3 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/specify_cli/commands/init.py
Comment thread src/specify_cli/commands/init.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Artifact-level conflicts, skipped files, JSON purity, and staging parity have unresolved correctness issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/specify_cli/commands/init.py:404

  • Starting the staged home empty loses filesystem state that affects the real initialization. For example, Hermes writes directly to ~/.hermes/skills/.../SKILL.md (integrations/hermes/__init__.py:204-209): if the real path is a directory or symlink, a real init fails or follows the link, while this child succeeds against an empty home and the comparison reports create. Pre-stage the relevant existing home entries without following symlinks so the preview matches the subsequent invocation.
        staged_home = Path(tmp_dir) / "home"
        staged_home.mkdir()
  • Files reviewed: 3/3 changed files
  • Comments generated: 4
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py Outdated
Comment thread src/specify_cli/commands/init.py
Comment thread src/specify_cli/commands/init.py
Comment thread src/specify_cli/commands/init.py Outdated

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

External symlinks can escape staging, while home-relative extensions and permission changes produce inaccurate plans.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py Outdated
Comment thread src/specify_cli/commands/init.py Outdated
Comment thread src/specify_cli/commands/init.py Outdated
@mnriem

mnriem commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The preview can diverge from real initialization for external symlinks and file removals, and constitution provenance is inaccurate.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

src/specify_cli/commands/init.py:410

  • The candidate set only includes paths that still exist after staging, so removals made by the canonical initializer disappear from the preview. For example, Kimi's --migrate-legacy setup moves or removes files under .kimi/skills/; the dry run reports the new destination but omits that the original path will be deleted, so it does not match the subsequent real init. Removed initial paths need an explicit plan outcome (or this migration must be reported as unresolved).
    candidates = {
        path
        for path, digest in staged_files.items()
        if initial_files.get(path) != digest
    }
    candidates.update(path for path in ownership if path in staged_files)

src/specify_cli/commands/init.py:389

  • This fallback misattributes preset- or extension-backed constitutions as core. ensure_constitution_from_template() resolves the highest-priority layer, and the bundled self-test preset provides templates/constitution-template.md, so a dry run with that preset reports the created constitution (and its provenance sidecar) with the wrong provenance. Derive ownership from .constitution-template.json or the resolver's winning layer.
    if relative_path.startswith(".specify/"):
        return "core", None

src/specify_cli/commands/init.py:889

  • An existing constitution is always recorded as a core skip, even when .constitution-template.json identifies a preset or extension as its source. _merge_recorded_plan_actions() then replaces the preserve record with this hard-coded provenance, making reinitialization previews inaccurate. Read the materialized provenance before recording the skip.
        _record_init_plan_action(
            "skip",
            ".specify/memory/constitution.md",
            "core",
        )
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py Outdated
@mnriem

mnriem commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The preview can omit removals and failures or diverge from real initialization due to incomplete configuration and ownership handling.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

src/specify_cli/commands/init.py:110

  • Redirecting HOME to an empty staging directory also hides the user's catalog configuration. Both ExtensionCatalog and PresetCatalog consult ~/.specify/*-catalogs.yml (extensions/init.py:3822-3825 and presets/init.py:4404-4407), so a catalog-backed extension or preset can resolve differently—or not at all—in the preview than in the subsequent real init. Stage those read-side config files while keeping cache/write locations isolated.
    src/specify_cli/commands/init.py:409
  • This candidate set can never report files removed by the staged init because it only iterates paths still present in staged_files. Removals are possible here—for example, Kimi's --migrate-legacy moves/removes .kimi/skills entries, and extension/preset mode re-registration can retire stale artifacts—so the preview omits changes a subsequent real init will make. Diff the initial key set as well and represent removals explicitly (and update the JSON contract/tests accordingly).

src/specify_cli/commands/init.py:690

  • The child can report a successful exit even when a requested preset or extension failed: the normal init path catches those exceptions, records a tracker error/warning, and continues. Because captured stdout/stderr is discarded whenever returncode == 0, a missing local extension or unresolved catalog ID silently disappears from the preview and the command exits successfully. Propagate these non-fatal outcomes through the structured plan channel so the preview reports them.
        if result.returncode:
            payload["error"] = _preview_child_failure_message(result)
        else:

src/specify_cli/commands/init.py:695

  • These ownership collectors are not scoped to the selected integration: manifest ownership reads every installed integration manifest, and registry ownership iterates registered commands for every agent. On a project with secondary integrations, unchanged files belonging only to those integrations are therefore emitted as preserve actions even though this init does not plan them (unlike unrelated files such as keep.txt). Pass the selected integration through and retain only core plus artifacts targeted for that agent.
            project_ownership = _preview_manifest_ownership(staged_root)
            registry_project_ownership, registry_home_ownership = (
                _preview_registry_ownership(staged_root)
            )
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py
@mnriem

mnriem commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and fix test & lint errors

Dry-run stages the target and invokes the public initializer in an isolated
child process, then reports create, overwrite, and preserve actions without
writing to the requested project. This keeps previews aligned with
integration-specific installation behavior.
HermesIntegration.setup() was writing into the real user home during init --dry-run, which let preview runs touch global files. Ownership for materialized preset and extension commands also depended on destination-path heuristics, which misclassified agent-directory outputs and lost the true provenance signal.

Dry-run staging now keeps home-scoped output in an isolated preview environment, and ownership is derived from the staged registries and markers instead of from destination paths. The manifest keeps the concrete source_id separate from the required provenance category.
Stage existing projects without --force, classify colliding artifacts as
conflict, keep --json stdout parseable, report skipped already-installed
artifacts, and remap in-project absolute symlinks onto the staged copy.
Quarantine external staged symlinks, strip Windows extended path
prefixes, include permission bits in snapshot fingerprints, and
resolve home-relative --extension specs before isolating HOME.
Retarget external staged links at an isolated dummy outside the
project copy so setup() still rejects destinations that leave the
tree, without writing through to the live target.
- Validate typed marker source registration and provenance categories
- Normalize Rich panel errors across narrow terminal layouts
- Add regression coverage for ownership markers and wrapped errors
@mnriem
mnriem requested a balanced review from Copilot September 3, 2026 11:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The preview can diverge from real initialization, hide component failures, omit quarantined writes, and misreport provenance.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

src/specify_cli/commands/init.py:389

  • This fallback labels every unrecognized .specify/ file as core, including .specify/integrations/<key>.manifest.json, which is created by and belongs to the selected integration. As a result the JSON plan violates its provenance contract for every integration manifest. Handle integration manifest paths before the generic .specify/ fallback and derive the integration ID from the filename.
    if relative_path.startswith(".specify/"):
        return "core", None

src/specify_cli/commands/init.py:577

  • Treating quarantined links as resolved lets integrations without containment checks write into the quarantine and still return success, but _build_preview_actions snapshots only staged_root/staged_home. Hermes does exactly this for its project marker (integrations/hermes/__init__.py:213-216), so an external .hermes symlink yields no reported action even though real init writes through that symlink to the external target. Track writes beneath quarantine and report the corresponding path as a conflict/error rather than silently omitting it.
            if resolved is not None and (
                _is_within_root(resolved, staged_root)
                or _is_within_root(resolved, quarantine_root)
            ):
  • Files reviewed: 3/3 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py Outdated
Comment thread src/specify_cli/commands/init.py
@mnriem

mnriem commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

- Preserve the caller's force mode in staged previews
- Report optional preset and extension failures in JSON and human output
- Preserve here and force semantics in staged previews
- Surface core failures and URL resolution limits in preview output

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Authenticated catalogs, conflict classification, JSON errors, and full-project staging need correction.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

src/specify_cli/commands/init.py:803

  • directory_conflict is always passed as false, so _build_preview_actions can never emit the documented conflict action. For an existing named target without --force, changed existing artifacts are currently labeled overwrite even though applying them is blocked; classify those project actions as conflicts while retaining overwrite behavior for the confirmable --here case.
                directory_conflict=False,

src/specify_cli/commands/init.py:1157

  • The machine-readable contract only applies after _preview_init is reached. Validation failures such as an unknown integration, an invalid script type, a missing target, generic integration options, or a missing required agent CLI still exit earlier with Rich-formatted text, so specify init --dry-run --json is not reliably parseable by CI. Emit a JSON error envelope for all failures when both flags are active.
        json_output: bool = typer.Option(
            False,
            "--json",
            help="Emit the dry-run preview as a single JSON document.",
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py Outdated
Comment thread src/specify_cli/commands/init.py Outdated
Seed catalog authentication into the isolated preview home and copy only initializer-relevant project paths. Preserve symlink isolation and compare only staged outputs so unrelated files cannot break dry-run previews.

Assisted-by: OpenAI Codex (model: GPT-5, autonomous)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Preview parity and isolation gaps remain for removals, Bob layouts, environment bypasses, and global symlinks.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

src/specify_cli/commands/init.py:489

  • The candidate set only contains files that still exist after staged initialization, so removals are invisible. For example, Kimi's --migrate-legacy moves or removes entries under .kimi/skills (src/specify_cli/integrations/kimi/__init__.py:114-125), but this preview reports only the new destination while a subsequent real init also deletes the old path. Include before-only files in the comparison and represent their removal so the preview satisfies the real-init parity guarantee.
    candidates = {
        path
        for path, digest in staged_files.items()
        if initial_files.get(path) != digest
    }

src/specify_cli/commands/init.py:915

  • The staged home copies only .specify configuration, not existing global integration destinations. For Hermes, an existing ~/.hermes/skills/.../SKILL.md symlink is therefore absent in staging; _snapshot_matching_files also skips symlinks, so dry-run reports create, while real init follows that link and overwrites its target. Stage and quarantine the selected home integration paths like project paths, or explicitly reject symlinked global destinations.
        _seed_preview_home(staged_home, real_home)
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py Outdated
Comment thread src/specify_cli/commands/init.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The preview child permits project-controlled Python execution and several preview contracts do not match initialization behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

src/specify_cli/commands/init.py:969

  • For --here, normal initialization accepts a bare relative local preset directory via the Path(preset).resolve() check at lines 1910-1913, but this helper only resolves path-prefixed values before changing the child cwd to staging. A valid --preset local-preset is therefore omitted from staging and previewed as a catalog preset/failure instead of the local preset. Resolve valid local preset directories against the caller cwd before invoking the child.
            command.extend(["--preset", _resolve_preview_child_path(preset)])

src/specify_cli/commands/init.py:1011

  • This hard-coded False makes the conflict branch in _build_preview_actions unreachable. For an existing named target without --force, overlapping artifacts are consequently serialized as overwrite even though the plan cannot be applied and the issue requires a per-artifact conflict action. Mark project-file overlaps as conflicts when the gate is force_required; the home-scoped call should remain unchanged.
                directory_conflict=False,

src/specify_cli/commands/init.py:1369

  • The new machine-readable contract only applies after _preview_init is reached. Early failures such as an unknown integration, invalid script type, an existing non-directory target, or a missing required agent still print Rich/plain text and exit, so specify init --dry-run --json cannot be reliably parsed by CI on failure. Route these validation errors through a JSON error payload whenever json_output is enabled.
        json_output: bool = typer.Option(
            False,
            "--json",
            help="Emit the dry-run preview as a single JSON document.",
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/specify_cli/commands/init.py
Report removals and validation failures accurately while preserving provenance. Reject unsafe integration, legacy, and companion artifact paths before installation or cleanup.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Staging can follow Windows junctions and fail on unrelated unreadable files, producing unsafe or unavailable previews.

Review details

Suppressed comments (2)

src/specify_cli/commands/init.py:861

  • This copies each selected output directory wholesale. An unrelated unreadable file inside a selected root such as .github/skills is retained by _ignore_special_files, causing copytree to fail before a preview can be emitted, while real initialization only accesses managed artifact paths and can succeed. Limit staging to managed paths or safely omit inaccessible unrelated entries while preserving errors for managed outputs.
        shutil.copytree(
            source,
            destination,
            symlinks=True,
            dirs_exist_ok=True,
            ignore=_ignore_special_files,
        )

src/specify_cli/commands/init.py:855

  • Windows directory junctions are not covered by is_symlink(), so this copytree branch follows an external junction and materializes its target as an ordinary staged directory. The preview child can then report safe project-relative changes even though the equivalent real initialization follows the junction and writes outside the project. Detect junctions before this branch and either remap/quarantine them with the same containment policy as symlinks or reject the preview.
    elif source.is_dir():
        shutil.copytree(
  • Files reviewed: 13/13 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Reject Windows junctions across supported Python versions and skip unreadable unmanaged staging entries while preserving failures for managed artifacts.
Propagate unreadable home staging failures and share Python 3.11-compatible junction detection across preview and Hermes paths. Stabilize the wrapped warning assertion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add specify init --dry-run preview

3 participants