Repository navigation
The service has no database — anchors, liquidity and settlements live in in-memory Maps and are destroyed on every restart #220
Description
Priority: High · Area: Persistence / architecture · Est. effort: 12–16 h
📌 Problem
package.json lists exactly three runtime dependencies:
compression, cors, express
There is no database driver, no ORM, no migration tool. Every repository under src/repositories/ — anchorRepository.ts, liquidityRepository.ts, settlementRepository.ts — is backed by inMemoryRepository.ts, and settlementRepository.ts:13 keeps its secondary index as private readonly anchorIndex: Map<string, Set<number>> = new Map().
Consequences:
- Every restart wipes all state. A deploy, crash, or scale event destroys the settlement history, the registered anchors and all liquidity records.
- Horizontal scaling is impossible. Two replicas hold two disjoint datasets; which one answers a request decides what the client sees.
- Unbounded growth. Nothing evicts, so the process grows until it is restarted — which is also the only thing that clears it.
Settlements are financial records. A service that forgets them on restart cannot be operated.
🎯 Design decision required
Post your proposed schema and approach in a comment on this issue before writing code, and wait for maintainer agreement. This is too large to land unsolicited. State and defend:
- Store choice. Postgres is the conventional answer for relational financial records; argue it against the alternatives rather than assuming it.
- Migration tooling and how migrations run in CI, in development, and on deploy.
- Interface preservation. The repository classes have method signatures that
src/services/*.tsdepend on. Keeping them stable means no service code changes — confirm that is achievable, and list any signature you must change. - Sequencing. All three repositories in one PR, or one at a time behind the existing interfaces? Argue which is reviewable.
- Testing without a live database. The suite currently runs with no external dependency. How do you preserve that — an in-memory implementation retained for tests behind the same interface, or a containerised database in CI?
🧩 Requirements and context
src/services/anchorService.ts,liquidityService.ts,settlementService.tsandquoteService.tsshould require no changes. If one does, justify it.- Schema must include the indexes the current code relies on —
settlementRepository.ts's anchor index is the obvious one; find the rest by reading the query methods. - Concurrency matters: two simultaneous settlements against the same liquidity must not both succeed if only one can be funded. State how your schema and transactions prevent that.
- All 42 existing test files must pass.
- Do not change API response shapes.
- Configuration must fail fast with a clear message when the database is unreachable —
src/config.tsis the place.
🛠️ Suggested execution
- Post the schema and approach; wait for agreement.
- Add the driver and migration tooling; write migrations.
- Implement one repository behind its existing interface; keep the in-memory one for tests if that is your choice.
- Prove services are untouched.
- Add a concurrency test for the double-spend case.
✅ Acceptance criteria
- The schema and approach were agreed on this issue before implementation.
- At least one repository is database-backed, with migrations committed.
- No file under
src/services/changed, or every change is justified. - Indexes backing existing query methods exist in the schema.
- A test proves two concurrent settlements cannot both consume the same liquidity.
- All 42 test files pass, with no requirement for a live database if that was your stated approach.
- Startup fails fast and clearly when the database is unreachable.
- API response shapes are unchanged.
🚫 Out of scope
- Changing the public API surface.
- Migrating all three repositories, if you argued for sequencing.
- Adding caching.
🧪 Verification
npm ci
npm run lint
npm run build
npm test📤 What your PR must include
- A link to the agreed schema comment.
- Your store, tooling, sequencing and test-strategy decisions with reasoning.
- The concurrency test.
- Confirmation services are unchanged.
Closes #<n>orPart of #<n>.
🔒 Security notes
Settlement records are the audit trail for value movement. Storing them only in process memory means there is no durable record of what the system did — a restart erases the evidence of every settlement executed before it. Beyond availability, that is an integrity and accountability failure: no reconstruction, no dispute resolution, and no way to detect after the fact that something went wrong.
📋 Guidelines
- Minimum 95% test coverage on changed lines
- Clear documentation
- Timeframe: 96 hours from assignment
- One logical change per commit; no merge commits
💬 Join our community
Working on this, or want to sanity-check your approach before you start? Come and ask — the maintainers are there and happy to help.
Telegram: https://t.me/Grainlify
Activity
Baskarayelu commentedon Aug 27, 2026
I’d love to be assigned this issue and contribute across the relevant areas, whether it involves smart contracts, UI development, backend logic, design implementation, or overall system integration. I’m comfortable working across the full stack and can adapt to the project’s technical requirements to ensure a smooth and consistent contribution.
grantfox-oss commentedon Aug 27, 2026
🦊 GrantFox — @Baskarayelu has been assigned to this issue as part of the Third Campaign campaign!
Next steps:
Closes #220)Good luck! Track your progress on GrantFox.
Baskarayelu commentedon Aug 30, 2026
Proposed persistence architecture (before implementation)
I propose PostgreSQL as the durable store, using the official
pgdriver and SQL migrations managed bynode-pg-migrate. PostgreSQL fits this domain better than SQLite (single-writer/file-oriented and awkward for horizontally scaled deploys), Redis (not the system of record and weak for relational audit queries), or a document store (the anchor/asset/settlement relationships and financial constraints are naturally relational). It also gives us row locks, transactional durability, and operational tooling for backups, replicas, and managed deployments.Schema
The first migration would create:
Indexes would mirror the existing repository access paths:
settlements(anchor, id),settlements(asset, id),settlements(status, id), andliquidity_entries(asset, anchor). The primary keys cover anchor lookup and(anchor, asset)liquidity lookup. Settlement IDs remain monotonic so current list ordering is deterministic.numeric(78,0)preserves the integer range represented by the current bigint values without converting financial amounts through JavaScript numbers. The driver layer will map these columns to strings/bigints at the repository boundary.Interfaces and sequencing
I would migrate one repository at a time behind the current repository interfaces, beginning with anchors, then liquidity, then settlements. The service classes and public response shapes should remain unchanged. I expect the existing method signatures to remain unchanged; if the concurrency boundary requires an additional internal repository method, it will be kept private to the repository composition rather than pushed into the service API. Each step can therefore be reviewed and tested independently, while the final PR can include all three migrations if the maintainer prefers one atomic rollout.
For the double-spend case, settlement creation must execute as one database transaction: lock the relevant
(asset, anchor)liquidity rows withSELECT ... FOR UPDATE, calculate the currently reserved amount under the same transaction, reject if the remaining available amount is insufficient, and insert the pending settlement before commit. A second transaction waits for the row lock and then observes the reduced availability, so both requests cannot succeed against the same balance. The concurrency test will issue two opens against one funded pool and assert exactly one succeeds.Migration and configuration
npm run migrate upagainst a disposable PostgreSQL service before integration tests, then rolls back/destroys that database after the job.DATABASE_URL, with a documented local Docker Compose PostgreSQL option.DATABASE_URLis required for the database-backed runtime.loadConfigwill validate it and the startup path will open a connection and fail fast with an actionable message if the database is unreachable.I will wait for maintainer agreement on this schema, tooling, sequencing, and test strategy before writing implementation code, as requested by this issue.