[Reliability]: Publish an enterprise-runtime conformance and chaos suite

[Fangjj0621]

Is there an existing issue for the same feature?

• I have checked the existing issues.

Is your feature request related to a problem?

The published Terminal-Bench result is useful for coding-agent task success, and #466 tracks SWE-bench plus reliability-machinery ablations. Neither directly validates Astra's main product claim: a durable, governed enterprise context-to-execution runtime.

As an operator, I need evidence for system properties that final-answer benchmarks do not measure:

• crash recovery and checkpoint correctness;
• exactly-once or explicitly at-least-once side-effect semantics;
• Runner disconnect/reconnect and ownership fencing;
• approval and permission revocation races;
• duplicate and out-of-order callbacks;
• tenant isolation;
• replay completeness;
• sandbox filesystem/network escape resistance;
• trace/audit consistency after failure.

Describe the feature you'd like

Publish a reproducible enterprise-runtime conformance and chaos suite with machine-readable results. It should run against each supported deployment profile and produce evidence tied to an Astra commit, configuration, backend versions, and scenario seed.

Suggested scenario families:

1. kill -9 Server during model, tool, and result-commit boundaries;
2. disconnect/restart Edge during a non-idempotent tool call;
3. deliver duplicate, delayed, stale-generation, and out-of-order callbacks;
4. revoke permission between projection and execution;
5. attempt cross-workspace filesystem/network access;
6. reload and replay a completed multi-agent run from durable facts;
7. inject database latency, partial writes, and Memoria unavailability;
8. prove that failed/cancelled cases remain in the denominator.

Acceptance criteria

• One command runs the conformance suite in a documented environment.
• Results include pass/fail/unavailable, evidence references, and environment metadata for every planned case.
• The suite covers Server-only and Server + Edge profiles.
• Non-idempotent side-effect scenarios assert execution count, not only final text.
• Security scenarios assert actual filesystem/network denial.
• Release notes link to a result bundle for the released commit.
• The suite is clearly separated from model/task-quality benchmarks such as Terminal-Bench and SWE-bench.

Additional information

Related issues: #458, #459, #460, #462, #464, #466.