feat: add durable short and long-term memory

[zxuexingzhijie]

Summary

• Rework short- and long-term memory around a framework-first, application-assisted boundary instead of maintaining parallel framework and custom implementations.
• Use Spring AI MessageWindowChatMemory with JdbcChatMemoryRepository for the recent message window, Spring AI Alibaba MysqlSaver for graph checkpoints, Alibaba Store/MemoryStore for rebuildable exact-key projections, and Spring AI VectorStore for semantic Top-K recall.
• Keep application-owned relational tables only where product semantics require an authoritative source of truth: successful conversation turns, run/artifact audit, reviewed long-term facts, transactional outbox events, retention, and lifecycle conflict handling.
• Make terminal state transitions, deletion, projection retries, checkpoint release, and long-term-memory supersession idempotent and scope-safe across agentId, conversationId, threadId, and turnId.
• Fence schema-vector publication with schema_generation and schema_revision, serialize concurrent publication with MySQL advisory locks, and prevent stale generations from becoming visible.
• Propagate the full conversation/run identity through frontend SSE, stop, and delete requests so the backend remains the single owner of persistence and cleanup.

Responsibility split

Concern	Framework-owned capability	Application-owned supplement
Recent messages	Spring AI MessageWindowChatMemory + JDBC repository	Persist only successful user/final-assistant pairs; rebuild an empty, incomplete, or stale projection from authoritative turns
Rolling summary	Spring AI Alibaba Store + MemoryStore	Validate the cached boundary against relational state and rebuild on cache loss or cross-node staleness
Graph state	Spring AI Alibaba MysqlSaver with its node-local latest cache disabled	Record release work in the transactional outbox, retry after commit, and physically purge logically released generations after retention
Semantic recall	Spring AI VectorStore	Enforce agent/scope/status metadata filters and treat vectors as a rebuildable projection
Business truth	Framework primitives are not used as an audit database	conversation_turn, turn_run, turn_artifact, memory_item, and memory_outbox own lifecycle, review, idempotency, and recovery facts

Lifecycle and consistency changes

• A turn is added to model memory only after its business transaction commits successfully; failed, cancelled, paused, stale, or unreviewed content cannot become trusted context.
• Active graph runs are registered explicitly, and late success callbacks cannot overwrite a prior failure/cancellation terminal state.
• Conversation deletion coordinates active runs, framework chat memory, graph checkpoints, summaries, long-term items, vectors, and relational rows through one lifecycle service.
• Long-term facts use a server-derived scoped identity, row locking, a unique constraint, explicit confirmation/supersession, and HTTP 409 for lifecycle conflicts.
• Projection work uses lease tokens and retryable outbox events so another worker cannot acknowledge or overwrite a lease it no longer owns.
• Startup validation fails fast when the configured database has the old memory schema instead of allowing a partially upgraded runtime.

Database upgrade

Existing MySQL installations must apply these migrations in order before starting the updated application:

1. V20260729_01__create_durable_memory.sql
2. V20260820_01__add_datasource_schema_revision.sql

After upgrading, re-run schema initialization for each existing datasource so a stable schema_revision is published. The previous revision is invalidated before extraction; a new revision becomes visible only when all vectors from the same generation finish publishing.

Verification

• Backend clean verification: 1,868 tests, 0 failures, 0 errors, 0 skipped; application jar built successfully.
• JaCoCo aggregate coverage: 81.48% line (8,187/10,048) and 72.34% branch (2,799/3,869); all configured coverage gates passed.
• Frontend: 19/19 unit tests passed and the production build completed on Node 22.23.1.
• Real MySQL 8.4/Testcontainers integration: 3/3 tests passed, covering both production migrations from a legacy schema slice, column defaults and CHECK/unique/FK constraints, two coordinator instances serialized by GET_LOCK, and generation fencing observed from another connection.
• Spring Java Format, Spotless, Checkstyle, test-trust static checks, and git diff --check all passed on the clean commit candidate.
• Pre-landing code review and automated review audit reported no remaining actionable findings; no visual/CSS surface changed.

Documentation

• ARCHITECTURE.md / ARCHITECTURE-en.md: document the framework/application ownership boundary and the real MemoryProjectionWorker, lifecycle cleanup, long-term-memory, and WebFlux authentication flows.
• DEVELOPER_GUIDE.md / DEVELOPER_GUIDE-en.md: document memory/checkpoint configuration, outbox scheduling and retention, identifier lifecycles, long-term-memory REST/error contracts, schema migrations, generation fencing, framework-owned table creation, and the real-MySQL test command.
• ADVANCED_FEATURES.md / ADVANCED_FEATURES-en.md: replace the obsolete custom Servlet interceptor example with the implemented WebFlux API Key contract, protected endpoints, authenticated SSE and long-term-memory examples, and the native EventSource limitation.
• QUICK_START.md / QUICK_START-en.md: repair all section anchors after the new conversation-memory section and keep the Sandbox runtime version current.

Explicit validation boundaries

• No live model-provider or live Milvus end-to-end run was performed in this pass; model quality and provider execution are not claimed.
• The MySQL regression is an opt-in Failsafe *IT under the repository's existing integration profile. The default GitHub build currently runs make verify, so this real-MySQL test is not yet part of the default workflow.
• The built-in browser page uses native EventSource and cannot attach an API Key header. API-key-enabled agents therefore require a header-capable external SSE client; the documentation now states this existing product boundary, and this memory PR does not introduce an insecure query-parameter fallback.
• This PR provides the long-term-memory review API and lifecycle rules; it does not add a review UI.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.