[Feature] Refactor leap evolve

.gitignore

@@ -221,6 +221,7 @@ __marimo__/
 *.duckdb
 *.duckdb.wal
 .leapflow/
+.learnings/
 .idea/
 
 # Swift / Xcode
@@ -234,6 +235,7 @@ Package.resolved
 # OSHost socket path (runtime)
 leapflow.sock
 
-temp
+# Entire temp directory is ignored (scratch / transient artifacts only).
+temp/
 .DS_Store
 AGENTS.md

README.md

@@ -904,6 +904,52 @@ Full authoring walkthrough: see the [Plugin Developer Guide](temp/deepseek_harne
 
 ---
 
+## World-Model-Driven Self-Evolution
+
+LeapFlow's Harness can **autonomously evolve its plugin composition** in response to dynamic environment changes. A world-model trajectory grader retrospectively evaluates execution evidence and produces a four-value **adaptation verdict** — `absorb` (update knowledge), `rebind` (select a better installed capability), `acquire` (generate a new plugin), or `escalate` (defer to a human) — so most adaptation happens without writing any code at all.
+
+When a genuine capability gap is detected, the evolution pipeline governs the entire journey from observation to production:
+
+```
+environment observation
+    → adaptation verdict (absorb / rebind / acquire / escalate)
+    → capability gap detection
+    → resolution-first check (existing catalog)
+    → proposal
+    → LLM code generation
+    → multi-stage validation (syntax → import → Protocol conformance → sandbox smoke)
+    → dual approval (content + plugin mutation)
+    → install at DRAFT trust
+    → progressive trust accrual (DRAFT → CANDIDATE → VERIFIED → PRODUCTION)
+    → governance (quarantine, rollback, proposal sweep)
+```
+
+### Enabling Self-Evolution
+
+Self-evolution is **disabled by default** as a safety constraint — the agent must be explicitly granted the ability to acquire new capabilities:
+
+```bash
+leap config set evolution.enabled true
+```
+
+When disabled, the world model still produces adaptation verdicts and distils knowledge (absorb/rebind paths remain active), but the `acquire` path that generates and installs new plugins is gated off.
+
+### Key Features
+
+- **Resolution-first** — before proposing a new plugin, the pipeline checks whether an existing capability already satisfies the requirement; duplicates are never created
+- **Dual approval gate** — generated content is reviewed for correctness, and the plugin mutation itself requires a separate HIGH-risk approval (no permanent grants)
+- **Progressive trust lifecycle** — new plugins start at DRAFT and promote through CANDIDATE → VERIFIED → PRODUCTION on consecutive successes; repeated failures trigger automatic demotion
+- **Sandbox isolation** — untrusted plugins run in a subprocess over JSON-RPC with bounded invocation timeouts
+- **Append-only causal audit trail** — 22 event types record the full causal chain from environment observation through install, validation, approval, trust transitions, and terminal outcomes
+- **Cold-path governance** — all evolution machinery (trust ledgers, proposal queues, sweep) runs on boot/reload/dispose paths with zero per-turn overhead
+- **Quarantine with recovery** — a plugin that fails hard is frozen at DRAFT with a quarantine record; it can be investigated and restored or removed
+- **DSH bundle rollback** — profile plugins maintain versioned source snapshots; `plugin_rollback` restores a previous version and hot-reloads it
+- **Proposal TTL and automated sweep** — stale proposals expire after a configurable TTL and are cleaned up by a periodic sweep
+
+For the formal specification — including system roles, architectural thesis, validation stages, and governance contracts — see [World-Model Plugin & Harness Evolution](docs/world_model_plugin_harness_evolution.md).
+
+---
+
 ## LeapBoard — Monitoring Dashboard
 
 > **Signals into insight.**

docs/plugins/plugin_lifecycle_management.md

@@ -12,7 +12,7 @@
 
 | Term | Definition |
 |------|-----------|
-| **PluginFiber** | A per-plugin lifecycle state-machine instance (`domain/plugin_fiber.py`). Tracks runtime state transitions (PENDING/LOADING/ACTIVE/FAILED/UNLOADING/DISPOSED) and owns an EffectScope for deterministic cleanup. |
+| **PluginFiber** | A per-plugin lifecycle state-machine instance (`domain/plugin_fiber.py`). Tracks runtime state transitions (PENDING/DRAFT/LOADING/ACTIVE/FAILED/UNLOADING/DISPOSED) and owns an EffectScope for deterministic cleanup. |
 | **EffectScope** | Hierarchical, LIFO-ordered cleanup collector (`domain/effect_scope.py`). Guarantees safe teardown on dispose. |
 | **Trust Level** | Progressive reliability gradient (DRAFT → CANDIDATE → VERIFIED → PRODUCTION) earned by consecutive successes, persisted in DuckDB. |
 | **Generation Counter** | Module-level monotonic integer; each new PluginFiber receives a unique generation. Engine caches key on `(id(plugin), generation)` to detect reloads. |
@@ -56,8 +56,9 @@ A plugin's state is the **composition** of three independent axes: Runtime, Trus
 
 | State | Meaning | Transitions out |
 |-------|---------|----------------|
-| `PENDING` | Created, awaiting activation or async init | `LOADING`, `ACTIVE` (fast path), `DISPOSED` |
-| `LOADING` | Async initialization in progress (dependency resolution) | `ACTIVE`, `FAILED`, `UNLOADING` |
+| `PENDING` | Created, awaiting activation or async init | `DRAFT`, `LOADING`, `ACTIVE` (trusted fast path), `DISPOSED` |
+| `DRAFT` | Isolated candidate; no live handlers are published | `ACTIVE`, `DISPOSED` |
+| `LOADING` | Async initialization in progress (dependency resolution) | `ACTIVE`, `FAILED`, `DISPOSED` |
 | `ACTIVE` | Fully operational, tools registered and available | `UNLOADING` |
 | `FAILED` | Initialization failed; retryable via `retry()`/`begin_loading()` | `LOADING`, `DISPOSED` |
 | `UNLOADING` | Graceful teardown in progress | `DISPOSED` |
@@ -129,9 +130,9 @@ A plugin's state is the **composition** of three independent axes: Runtime, Trus
 | Dimension | Built-in Plugins | Third-Party Plugins |
 |-----------|-----------------|---------------------|
 | **Discovery** | Hardcoded module list in `plugins/tool_plugins/__init__.py` → `get_all_plugins()` | Profile-dir install (`plugin_install` tool) or marketplace fetch |
-| **Boot sequence** | `discover_builtin()` → `register()` → `bind_runtime()` → `assemble()` → `adopt_existing_plugins()` | `plugin_install` → validate → sandbox smoke → register → fiber activate |
+| **Boot sequence** | `discover_builtin()` → `register()` → `bind_runtime()` → `assemble()` → `adopt_existing_plugins()` | `plugin_install` → staging → bounded sandbox smoke/behavior tests → DRAFT fiber → atomic publish |
 | **Initial trust** | Implicitly DRAFT (but never demoted/frozen in practice — no failure path for well-tested built-ins) | Explicitly DRAFT; must earn promotion through usage |
-| **Fiber creation** | `adopt_existing_plugins()` at first `get_scoped_registry()` access; starts in ACTIVE | `create_fiber()` → `scoped_register()` → `activate()` during install |
+| **Fiber creation** | `adopt_existing_plugins()` at first `get_scoped_registry()` access; starts in ACTIVE | `create_draft_fiber()` → `stage_plugin()` → `promote_draft()` after tests |
 | **Approval** | None for registration (they ARE the system); mutations still gated | ALL mutations gated (HIGH risk, no permanent grants) |
 | **Isolation** | In-process (same asyncio loop) | Optionally sandboxed (subprocess JSON-RPC via `SandboxHost`); `requires_sandbox` manifest flag defaults `True` |
 | **Reload** | `reload(plugin_id)` via scoped registry; version bump + cache invalidation | Same mechanism, but PRODUCTION trust → auto-approve; below PRODUCTION → explicit approval |
@@ -147,11 +148,11 @@ A plugin's state is the **composition** of three independent axes: Runtime, Trus
 | **plugin_list** | Agent tool | None | Always | — | No (read-only) |
 | **plugin_status** | Agent tool | None | Always | — | No (read-only) |
 | **plugin_versions** | Agent tool | None | Always | `ProfileLayout.plugin_versions_dir` | No (read-only) |
-| **plugin_propose** | Agent tool | None | Always (proposal only, no LLM/file/runtime mutation) | `ProfileLayout.plugin_proposals_path` | No (proposal store write only) |
+| **plugin_propose** | Agent tool | None | Always (proposal only, no runtime mutation) | Event-sourced lifecycle store | Yes — `proposal.created` |
 | **assess_compatibility** | Agent tool | None | Always (read-only manifest assessment; no file/runtime mutation) | — | No (read-only) |
-| **plugin_generate** | Agent tool | None | Always (code generation only, no filesystem write) | `plugin_generation_enabled` must be `True`; needs `llm_provider` bound | No (ephemeral output) |
+| **plugin_generate** | Agent tool | Content approval | Never for proposal-backed generation | `plugin_generation_enabled`; LLM provider; CAS | Yes — generated artifact and approval lifecycle events |
 | **/plugin generate** | User (slash command) | None (user invocation = consent) | Always auto-approved (user-initiated); installs at DRAFT trust level | `plugin_generation_enabled` must be `True`; needs an LLM provider | Yes — install action descriptor recorded |
-| **plugin_install** | Agent tool | `ApprovalGate` → HIGH, `allow_permanent=False` | Never (always requires human) | `plugin_install_dir`, proposal/version stores, `plugin_marketplace_root/url`, `plugin_marketplace_trusted_pubkeys` | Yes — action descriptor metadata recorded |
+| **plugin_install** | Agent tool | `ApprovalGate` → HIGH, `allow_permanent=False` | Never (always requires human) | `plugin_install_dir`, event lifecycle/version stores, `plugin_marketplace_root/url`, `plugin_marketplace_trusted_pubkeys` | Yes — action descriptor and lifecycle event recorded |
 | **plugin_rollback** | Agent tool | `ApprovalGate` → HIGH, `allow_permanent=False` | Never (always requires human) | `ProfileLayout.plugin_versions_dir` | Yes |
 | **plugin_reload** | Agent tool | `ApprovalGate` → HIGH, `allow_permanent=False` | Trust == PRODUCTION (auto-approved) | proposal/version stores when behavior tests or version labels are used | Yes |
 | **plugin_disable** | Agent tool | `ApprovalGate` → HIGH, `allow_permanent=False` | Never (always requires human) | — | Yes |
@@ -174,8 +175,8 @@ A plugin's state is the **composition** of three independent axes: Runtime, Trus
 **Trigger**: User or agent decides a new capability is needed.
 
 **Sequence**:
-1. **Generate** (optional): `plugin_generate(description="...")` → LLM produces code → `PluginValidator` multi-stage check (syntax → structure → runtime protocol conformance). Returns validated code blob. No filesystem write.
-2. **Install request**: `plugin_install(code=<blob>)` or `plugin_install(marketplace_name="...")`.
+1. **Generate**: `plugin_generate(proposal_id="...")` → LLM produces code → `PluginValidator` multi-stage check → profile CAS write → explicit proposal-content approval. No live registry mutation occurs.
+2. **Install request**: `plugin_install(proposal_id="...")` resolves the approved CAS artifact and requests a separate mutation approval. Direct code/marketplace installs still use the mutation gate.
    - **Compatibility pre-gate (marketplace path only)**: the resolved manifest is run through `assess_plugin()` (the Compatibility Assessment Engine) *before* anything else. An `INCOMPATIBLE` verdict is **rejected here with a structured error, before any file write**; an `ADAPTABLE` verdict proceeds and its adaptation notes are attached to the install result.
 3. **Approval gate**: `ActionDescriptor.platform_action("plugin_management", "install", {...})` → `gate.evaluate()` → user prompted (HIGH risk, one-time).
 4. **Duplicate check**: If `plugin_id` already exists in registry → immediate rejection with error.
@@ -491,7 +492,7 @@ These require human/product input and are not answerable from code alone:
 | Plugin discovery (built-in) | `src/leapflow/plugins/tool_plugins/__init__.py` |
 | Self-management tools (12 tools) | `src/leapflow/plugins/tool_plugins/self_management.py` |
 | Proposal domain records | `src/leapflow/domain/plugin_proposal.py` |
-| Proposal persistence | `src/leapflow/storage/plugin_proposal_store.py` |
+| Proposal persistence | `src/leapflow/storage/capability_proposal_queue.py` (`EvolutionCapabilityProposalStore`) |
 | Behavior test execution | `src/leapflow/learning/plugin_behavior_tests.py` |
 | Version snapshot store | `src/leapflow/storage/plugin_version_store.py` |
 | Trust ledger | `src/leapflow/learning/plugin_trust.py` |

docs/plugins/third_party_plugin_development.md

@@ -117,6 +117,7 @@ class ToolMetadata:
     handler: Callable[..., Any]
     x_leapflow: dict[str, Any] = field(default_factory=dict)
     mutates_state: bool = False
+    execution_policy: str = ""
 
     def to_openai_schema(self) -> dict[str, Any]:
         """Generate OpenAI function-calling schema dict."""
@@ -135,13 +136,14 @@ class ToolMetadata:
     "x_leapflow": {
       "category": "integration",
       "mutates_state": true,
-      "risk_level": "medium"
+      "risk_level": "medium",
+      "execution_policy": "mutating_once"
     }
   }
 }
 ```
 
-When `mutates_state=True`, `to_openai_schema()` folds it into `x_leapflow.mutates_state` so schema-only consumers can classify side-effecting tools without accessing the metadata object.
+When `mutates_state=True`, `to_openai_schema()` folds it into `x_leapflow.mutates_state`. A non-empty `execution_policy` is folded into `x_leapflow.execution_policy` as well, so schema-only consumers use the same declared execution semantics as the runtime.
 
 **`x_leapflow` well-known keys:**
 
@@ -157,6 +159,7 @@ omit approval/idempotency metadata.
 | `schema_cost` | `str` | `"low"` / `"medium"` / `"high"` — token cost hint for PCD |
 | `requires_approval` | `bool` | Whether the engine gates this tool behind approval |
 | `mutates_state` | `bool` | Auto-populated from the field when `True` |
+| `execution_policy` | `str` | `read_only`, `mutating_idempotent`, `mutating_once`, or `external_side_effect`; undeclared policies fail safe as external |
 
 ### 2.3 GatewayAdapterPlugin Protocol
 
@@ -388,6 +391,7 @@ ToolMetadata(
     parameters_schema={...},
     handler=handle_delete,
     mutates_state=True,
+    execution_policy="external_side_effect",
     x_leapflow={
         "category": "cloud_ops",
         "risk_level": "high",
@@ -396,7 +400,7 @@ ToolMetadata(
 )
 ```
 
-For platform actions (gateway send, external API write), use `ActionDescriptor.platform_action(platform, action, metadata)` within the handler to explicitly request gate evaluation.
+For platform actions (gateway send, external API write), use `execution_policy="external_side_effect"` and `ActionDescriptor.platform_action(platform, action, metadata)`. The action descriptor requests approval; the execution policy controls durable evidence, duplicate suppression, batch stopping, and uncertain-effect reporting. Runtime policy is derived only from these declarations, never from the tool name.
 
 ### 3.6 Code Quality
 
@@ -426,9 +430,9 @@ The following is the ordered sequence from plugin source to tool invocation:
 
 ### Step 4: PluginFiber Lifecycle
 
-5. **`ScopedToolRegistry.adopt_existing_plugins()`**: Called on first `leapflow.plugins.get_scoped_registry()` access. It creates a `PluginFiber` for every already-registered plugin and uses the fast path `PENDING → ACTIVE` for the current built-in/profile ToolPlugin runtime. The `PluginFiber` domain type also supports `LOADING` and `FAILED` retry states for future async initialization paths, but the scoped registry does not yet run a dependency-driven async activation loop.
+5. **`ScopedToolRegistry.adopt_existing_plugins()`**: Called on first `leapflow.plugins.get_scoped_registry()` access. It creates a `PluginFiber` for every already-registered plugin. Dependency-free built-ins use `PENDING → ACTIVE`; dependency-bearing plugins use `PENDING → LOADING → ACTIVE` when providers become available.
 
-Fiber domain state machine: `PENDING → LOADING → ACTIVE → UNLOADING → DISPOSED` (with `LOADING → FAILED → LOADING` retry path). Current ToolPlugin registration uses the fast path `PENDING → ACTIVE`; `LOADING`/`FAILED` are available primitives, not automatic dependency orchestration.
+Fiber domain state machine: `PENDING → DRAFT → ACTIVE → UNLOADING → DISPOSED` for isolated candidates, plus `PENDING → LOADING → ACTIVE` and `LOADING → FAILED → LOADING` for dependency-driven activation. DRAFT plugins expose no live handlers until atomic promotion.
 
 ### Step 5: Per-Turn Engine Assembly
 
@@ -485,7 +489,7 @@ Typical interceptor use cases include audit logging, execution timeout, approval
 
 ### Dependency Binding and Activation
 
-Plugins declare `dependencies`, and `ToolPluginRegistry.bind_runtime()` distributes matching runtime dependencies in topological plugin order. Current ToolPlugin activation still uses the `ScopedToolRegistry` fast path (`PENDING → ACTIVE`) after registration; plugins that require a dependency should degrade gracefully in their handler when the dependency is not bound. A future async activation loop may use the `LOADING`/`FAILED` states for dependency-driven retries, but that is not yet automatic.
+Plugins declare `dependencies`, and `ToolPluginRegistry.bind_runtime()` distributes matching runtime dependencies in topological plugin order. `ScopedToolRegistry` activates dependency-free plugins immediately and keeps dependency-bearing fibers in `LOADING` until their providers are active; handlers must still degrade gracefully when optional runtime dependencies are absent.
 
 ---
 
@@ -530,7 +534,7 @@ description:
 - **`--id <plugin_id>`** — override the auto-derived plugin id (a slug of the
   description). A colliding id is rejected cleanly.
 
-Generation is controlled by `plugin.generation_enabled` (enabled by default in current config; disable via `/config set plugin.generation_enabled false`) and requires an LLM provider. Installation still remains a separate approval-gated action.
+Generation is controlled by `plugin.generation_enabled` and requires an LLM provider. Proposal-backed generation stores validated source in profile CAS and requires explicit content approval. Installation remains a separate mutation approval.
 
 **Difference from the `plugin_generate` agent tool**: `/plugin generate` is a
 *user-initiated* control-plane command — the user's invocation is the consent, so it