feat(eth-indexer): event-driven AUDIO balance indexer#849
Merged
Conversation
Adds a new `eth-indexer` entrypoint that mirrors the structure of the
solana-indexer, but is event-driven instead of slot-polling.
Background: the discovery-provider's `cache_user_balance` task feeds the
`associated_wallets_balance` field on `GET /v1/users/...` by polling
`eth_getLogs` on the AUDIO ERC-20 contract every 30s and calling
balanceOf + totalStakedFor + getTotalDelegatorStake only for wallets that
just moved AUDIO. This is the api-side replacement.
The new indexer:
- Opens a persistent WSS connection to the configured ETH provider and
subscribes via `eth_subscribe` to the AUDIO Transfer log topic. No
polling baseline; live events stream in as they're mined.
- On every event, parses from/to from the indexed topics, joins them
against `users.wallet ∪ chain='eth' associated_wallets`, and fans out
three contract reads (balanceOf + totalStakedFor +
getTotalDelegatorStake) in parallel for each tracked address.
- Sums the three values and upserts a single row per wallet into a new
`eth_wallet_balances` table — shape (wallet, balance, blocknumber,
updated_at, created_at) so per-user totals can be rolled up via the
same JOIN pattern Solana uses.
- Persists a resume checkpoint to `eth_indexer_checkpoints` after every
processed block so reconnects backfill via `eth_getLogs` (chunks of
9000 blocks) for any gap, then resume the live subscription.
- Exposes `GET /eth/health` on :1325 reporting connected, last block
seen, checkpoint block, tracked vs cached wallet counts, and an
optional max_event_lag_secs trip wire.
Mainnet contract addresses default to the values in
`packages/sdk/src/sdk/config/production.ts`:
AUDIO 0x18aAA7115705e8be94bfFEbDE57Af9BFc265B998
Staking 0xe6D97B2099F142513be7A2a068bE040656Ae4591
DelegateManager 0x4d7968ebfD390D5E7926Cb3587C39eFf2F9FB225
Config (env vars):
ethRpcUrl - HTTPS endpoint (required)
ethWsUrl - WSS endpoint (auto-derived from
ethRpcUrl when unset)
ethAudioContractAddress - overrides
ethStakingContractAddress - overrides
ethDelegateManagerContractAddress - overrides
If `ethRpcUrl`/`ethWsUrl` are unset the indexer logs a warning and idles
until ctx.Done() so it's safe to deploy without a provider key.
Includes `cmd/eth_smoke` — a one-shot CLI that runs the same three
contract reads for a given holder against ethRpcUrl, useful for ops
debugging.
Local smoke test (this repo):
1. docker compose up -d db
2. apply migration 0203
3. seed users.wallet and an associated_wallets row
4. set eth_indexer_checkpoints to current-9000
5. `writeDbUrl=... ethRpcUrl=... go run main.go eth-indexer`
Backfill walked 9000 blocks, picked up a Transfer involving the
seeded address, called the 3 contracts, and upserted the correct sum.
Cross-checked the upserted balance with `go run ./cmd/eth_smoke
<addr>` — bytes match.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
eth-indexerentrypoint that mirrorssolana-indexer's shape:New(cfg)+Start(ctx) error+Close()+GetHealth(), plus a sibling fiberServeron:1325exposingGET /eth/health.eth_subscribeto the AUDIO Transfer log topic. On each event, joins from/to againstusers.wallet ∪ chain='eth' associated_wallets, fans outbalanceOf+totalStakedFor+getTotalDelegatorStakein parallel for matches, and upserts the sum into the neweth_wallet_balancestable.eth_indexer_checkpoints— on (re)connect weeth_getLogsthe gap (9K-block chunks) and replay.This is the api-side replacement for the discovery-provider's
cache_user_balancejob that currently feedsassociated_wallets_balanceonGET /v1/users/handle/.... Same three contract reads, same result, but live via WS instead of 30s polling.Why event-driven
For a baseline of ~50K AUDIO transfers/day across ~5M tracked wallets, this approach issues roughly one balance read per actual on-chain transfer — vs. hundreds of thousands of speculative
eth_getBalancecalls per day for a periodic poll-everyone approach. RPC cost scales with on-chain activity, not user count.Files
EthIndexerstruct with WS subscription loop, backfill, fan-out balance reads, upsert, checkpoint:1325,GET /eth/healtheth_wallet_balances+eth_indexer_checkpointscase "eth-indexer":parallelscase "solana-indexer":Config
ethRpcUrlethWsUrlethRpcUrl(https→wss)ethAudioContractAddress0x18aAA7115705e8be94bfFEbDE57Af9BFc265B998ethStakingContractAddress0xe6D97B2099F142513be7A2a068bE040656Ae4591ethDelegateManagerContractAddress0x4d7968ebfD390D5E7926Cb3587C39eFf2F9FB225If
ethRpcUrl/ethWsUrlare unset the indexer logs a warning and idles until shutdown — safe to deploy without a provider key.Test plan
go build ./...cleango vet ./...cleancmd/eth_smokeagainst mainnet:0x7d273…b060) → 0/0/0 ✓ (matches discovery-providerbalance: "0")0xe6D9…4591) → 247,024,527 AUDIO ✓80975640000000000000000weicmd/eth_smokeagainst the same holder — bytes match exactlyGET /eth/healthreturnedconnected: true, advanced checkpoint, correct tracked/cached countsethRpcUrl(and optionallyethWsUrl) in stage; let it run for ~24h and confirmeth_wallet_balancespopulates as AUDIO transfers occurOut of scope / follow-ups
eth_wallet_balancesbut does not yet roll it up into a/v1/users/...API response field. That can be a follow-up PR (or a SQL view) once the data is flowing.🤖 Generated with Claude Code