Hidden contracts: leave a token out of API responses by aditya1702 · Pull Request #764 · stellar/wallet-backend

aditya1702 · 2026-10-08T15:40:13Z

Why

The Comet LP pool (CPAL) looks like a SEP-41 token, but it mints shares without emitting an event. We compute negative balances for it and serve them today. Scam tokens need the same fix.

Part of #610.

What changes

A YAML list of contracts the API hides. serve reads it at startup.

The built-in list hides CPAL. HIDDEN_CONTRACTS_FILE replaces it with your own file.
Hidden means the API skips the contract's current state and its state changes.
Ingestion keeps writing the rows. Remove an entry and restart serve to get the full history back.

hidden_contracts:
  - contract_id: CAS3FL6TLZKDGGSISDBWGGPXT3NRR4DYTZD7YOD3HMYO6LTJUVGRVEAM
    protocol: SEP41   # whose rows to hide
    reason: ...

A bad entry stops serve at startup.

Not in this PR

No schema change, no new command, nothing deleted.
CPAL is not moved to its own processor. That comes later.

Where to look

Piece	Files
List and wiring	`internal/serve/hidden_contracts.go`, `internal/serve/serve.go`
Read filter	`internal/data/statechanges.go`, `internal/data/sep41/{balances,allowances}.go`
Docs	`docs/data-migrations/running-a-data-migration.md`

Verify

Restart serve. A CPAL holder shows no CPAL balance or history. sep41_balances is unchanged.

🤖 Generated with Claude Code

Copilot

🟡 Changes recommended

YAML streams can be partially ignored, and SAC validation can be bypassed before contract classification populates the database.

2 open findings

Reject trailing YAML documents and content · New
Validate SAC status when contract_tokens row is absent · New

What changed in this PR

Adds an API-layer hidden-contract list as a partial mitigation for #610 while preserving ingestion history.

Changes:

Loads and validates built-in or operator-supplied YAML deny lists.
Filters hidden SEP-41 balances, allowances, and protocol-scoped state changes.
Adds CLI wiring, documentation, and database-backed tests.

File	Description
`internal/services/processor_registry_ext_test.go`	Verifies protocol ordinal mappings.
`internal/serve/serve.go`	Loads and applies hidden contracts.
`internal/serve/hidden_contracts.yaml`	Defines the built-in CPAL entry.
`internal/serve/hidden_contracts.go`	Implements list parsing and model wiring.
`internal/serve/hidden_contracts_test.go`	Tests parsing and wiring.
`internal/indexer/types/types.go`	Maps protocols to ordinal ranges.
`internal/data/statechanges.go`	Filters protocol-scoped state changes.
`internal/data/statechanges_hidden_test.go`	Tests all state-change reads.
`internal/data/sep41/models.go`	Propagates hidden contracts.
`internal/data/sep41/balances.go`	Filters SEP-41 balances.
`internal/data/sep41/balances_test.go`	Tests balance filtering.
`internal/data/sep41/allowances.go`	Filters SEP-41 allowances.
`internal/data/sep41/allowances_test.go`	Tests allowance filtering.
`internal/data/models.go`	Updates SEP-41 model construction.
`internal/data/mocks.go`	Extends the contract-model mock.
`internal/data/contract_tokens.go`	Adds contract-type lookup.
`internal/data/contract_tokens_test.go`	Tests contract-type lookup.
`go.mod`	Promotes YAML to a direct dependency.
`docs/data-migrations/running-a-data-migration.md`	Documents token hiding.
`docker-compose.yaml`	Documents container configuration.
`cmd/utils/global_options.go`	Defines the configuration option.
`cmd/serve.go`	Registers the serve flag.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

🟡 Changes recommended

SAC validation mishandles empty lists and lists exceeding the 200-key RPC batch limit.

1 open finding

Handle empty and oversized hidden contract lists in batched RPC calls · New

2 resolved since last review

Validate SAC status when contract_tokens row is absent
Reject trailing YAML documents and content

🧠 Review effort: Balanced

Copilot

🟢 Approval recommended

The filtering, startup validation, protocol scoping, pagination behavior, and RPC batching are consistently implemented and comprehensively tested.

0 open findings

1 resolved since last review

Handle empty and oversized hidden contract lists in batched RPC calls

🧠 Review effort: Balanced

Copilot

🟡 Changes recommended

SAC addresses can pass validation while remaining visible through the unfiltered SAC balance model.

2 open findings

🧠 Review effort: Balanced

Copilot

🟡 Changes recommended

SAC entries are accepted but remain exposed through SAC or trustline balance APIs.

3 open findings

🧠 Review effort: Balanced

Some contracts match a protocol's interface but their balances cannot be derived the way that protocol's processor does it, and some tokens an operator does not want shown at all. The hidden-contracts list names them by contract ID. The binary embeds a default list; the --hidden-contracts-file flag (HIDDEN_CONTRACTS_FILE) replaces it with the operator's own file. Ingestion is not involved: processing continues, only the API skips the rows. Each entry names the contract and the protocol whose rows are hidden. A token belongs to one protocol at a time, so that is exactly the set of rows written for it; a token that a later, dedicated processor takes over shows that processor's rows while the old ones stay hidden. The loader validates each entry at startup: a bad address, a missing or unknown protocol, an empty reason, a repeated contract and protocol, an unknown field or a missing top-level key fails the process. It exposes the contracts in the forms the reads need: the 33-byte address stored in state_changes.token_id with the protocol's state_change_id range, and the C-address the SEP-41 tables join on. The default list hides the Comet LP pool (CPAL) from SEP-41. Its deployed code mints LP shares without emitting an event, so SEP-41 event arithmetic drives its balances negative.

…nce reads The API server loads the hidden-contracts list at startup and every read skips them. The six state_changes reads drop rows whose token_id is hidden and whose state_change_id falls in the named processor's emitter range; rows with no token are kept. The SEP-41 balance and allowance reads drop the hidden contracts by address through the contract_tokens join they already make. The predicate is a post-filter on rows already selected by account, transaction or operation: it adds no index or plan change, and with an empty list the SQL is unchanged. The read tests open their pool in pgx's Exec query mode, as the API server does, so parameter encoding is exercised the way production runs it. Rows are hidden, never deleted, and ingestion keeps writing them. Removing an entry and restarting serve shows a complete history again.

Adds a "Hiding a token" section to the data-migrations runbook: when to use it, how to add and remove an entry, that ingestion keeps processing the contract, and the file format. docker-compose shows the env var on the api service.

Copilot

🟡 Changes recommended

Stellar Asset Contracts can still remain visible through SAC balance and state-change paths.

2 open findings

2 resolved since last review

🧠 Review effort: Balanced

aditya1702 force-pushed the ignored-contracts branch from 7e07032 to 93e1ee2 Compare October 8, 2026 15:48

aditya1702 changed the title ~~Ignore list: exclude a token from protocol processing and reads~~ Hidden contracts: leave a token out of API responses Oct 8, 2026

aditya1702 force-pushed the ignored-contracts branch 5 times, most recently from 1155ec1 to c0f9867 Compare October 8, 2026 19:50

aditya1702 marked this pull request as ready for review October 8, 2026 19:50

Copilot AI balanced review requested due to automatic review settings October 8, 2026 19:50

Copilot started reviewing on behalf of aditya1702 October 8, 2026 19:51

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Comment thread internal/serve/hidden_contracts.go

Comment thread internal/serve/serve.go Outdated

aditya1702 force-pushed the ignored-contracts branch from c0f9867 to 98544ee Compare October 8, 2026 20:07

Copilot AI balanced review requested due to automatic review settings October 8, 2026 20:07

Copilot started reviewing on behalf of aditya1702 October 8, 2026 20:07

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Comment thread internal/serve/hidden_contracts.go Outdated

aditya1702 force-pushed the ignored-contracts branch from 98544ee to 92a5a73 Compare October 8, 2026 20:24

Copilot AI balanced review requested due to automatic review settings October 8, 2026 20:24

Copilot started reviewing on behalf of aditya1702 October 8, 2026 20:24

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

aditya1702 force-pushed the ignored-contracts branch from 92a5a73 to 4b5ef35 Compare October 9, 2026 20:52

Copilot AI balanced review requested due to automatic review settings October 9, 2026 20:52

Copilot started reviewing on behalf of aditya1702 October 9, 2026 20:52

View session

Copilot AI reviewed Oct 9, 2026

View reviewed changes

Comment thread internal/serve/hidden_contracts.go

Comment thread internal/serve/hidden_contracts.yaml Outdated

Copilot AI balanced review requested due to automatic review settings October 9, 2026 20:57

aditya1702 force-pushed the ignored-contracts branch from 4b5ef35 to 0c7f48a Compare October 9, 2026 20:57

Copilot started reviewing on behalf of aditya1702 October 9, 2026 20:58

View session

Copilot AI reviewed Oct 9, 2026

View reviewed changes

Comment thread docs/data-migrations/running-a-data-migration.md Outdated

aditya1702 added 3 commits October 9, 2026 17:02

docs: how to hide a token from the API

62195a9

Adds a "Hiding a token" section to the data-migrations runbook: when to use it, how to add and remove an entry, that ingestion keeps processing the contract, and the file format. docker-compose shows the env var on the api service.

Copilot AI balanced review requested due to automatic review settings October 9, 2026 21:02

aditya1702 force-pushed the ignored-contracts branch from 0c7f48a to 62195a9 Compare October 9, 2026 21:02

Copilot started reviewing on behalf of aditya1702 October 9, 2026 21:03

View session

Copilot AI reviewed Oct 9, 2026

View reviewed changes

Comment thread internal/serve/hidden_contracts.go

JiahuiWho approved these changes Oct 12, 2026

View reviewed changes

aditya1702 merged commit 7ce5c1b into main Oct 13, 2026
12 of 15 checks passed

aditya1702 deleted the ignored-contracts branch October 13, 2026 00:18

aditya1702 mentioned this pull request Oct 13, 2026

Rewrite the docs and add repo files for v1.0.0 #762

Merged

Navigation Menu

Repository navigation

Hidden contracts: leave a token out of API responses - #764

Hidden contracts: leave a token out of API responses#764
aditya1702 merged 3 commits into
mainfrom
ignored-contracts

aditya1702 commented Oct 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

aditya1702 commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changes

Not in this PR

Where to look

Verify

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

aditya1702 commented Oct 8, 2026 •

edited

Loading