Cut process-stage CPU and allocations across the indexer and processors by aditya1702 · Pull Request #685 · stellar/wallet-backend

aditya1702 · 2026-08-13T22:17:07Z

Review commit-by-commit, in order. ~1 hour. No schema changes, no new flags, no behavior changes. Every risky rewrite has a differential test against the implementation it replaced.

Why

After #684, the process stage was the bottleneck for the p99 ≤ block-time target, and GC was 20–28% of ingest CPU. This PR only cuts CPU and allocations on that stage.

What changed

Each operation's ledger changes are decoded once and shared. Before, every processor re-decoded them through the SDK. This is the biggest win: 40–50% less wall time per ledger in the process benchmark. The shared decode also feeds ContractData collection, which moved off the serial persist goroutine and now skips classic transactions entirely (only Soroban transactions can write ContractData).
Transaction envelopes are hashed in parallel instead of single-threaded inside the SDK reader. ~3× faster ledger loading.
State changes are built with less garbage. The builder is by-value instead of heap-allocated, account comparison stopped XDR-marshaling both entries, the XDR encoder is pooled, and dead output fields are deleted.
Participants are deduped on raw ed25519 keys and the Soroban walk fills one accumulator instead of allocating a set per tree node.
Hot conversions are memoized: one shared asset→contract-ID memo serves all processors, and address→BYTEA conversion is cached per COPY batch. Riding along: an oversized address payload now errors instead of silently truncating.

State-change ordinals are assigned with a counting map — linear even for the migration path, which passes a whole window at once.

How it's verified

Rewrite	Gate
Parallel hashing	Differential vs the SDK reader over every committed fixture
ContractData at process time	Differential vs reader-based extraction on real ledgers
Participant dedupe	Old implementation kept in the test file as oracle
Field-wise account comparison	28-case differential vs the SDK function
By-value builder, ordinals	Existing exact-output suites; byte-identical IDs

Plus 583 lines of new sponsorship-effects coverage (previously none).

Numbers

Combined loadtest rig result for the campaign: 18,458 tx/s, process p99 0.992 s — this PR took process p99 under the 1 s bar. Individual PRs were not re-benchmarked.

Stack

Third of four PRs from the loadtest campaign (replaces #682). Review order: #679 → #684 → this. #686 is independent.

Follow-up, decided separately: the BALANCE_AUTHORIZATION change for a new trustline inherits trustline_limit_new from its sibling trustline change. Preserved byte-identically here; the field arguably doesn't belong there.

The pool path skips liquidity pool bodies that liquidity_pools cannot represent (non-constant-product), but shares were captured unconditionally — and now that liquidity_pool_balances references liquidity_pools, a share row without its pool aborts the whole checkpoint load at commit. A share's pool entry can arrive in any bucket, so shares are held until every pool entry has been read, then joined to the final flush minus the skipped pools'. Covered by a regression test with an unsupported pool body.

…cores runtime.NumCPU() reports the node's cores in a container with a CPU limit, so a 6-core pod on a 36-core node ran 72 indexer workers over 6 schedulable threads — pure scheduler churn on CPU-bound work. GOMAXPROCS honors the limit.

…e bar The phase histogram stepped 0.5 -> 1 -> 2 seconds, so a stage sitting at 0.9s and a stage sitting at 1.9s both reported the same p99 bucket. The pipeline's contract is that the slowest stage's p99 stays under the ledger close time, so the metric could not grade the thing it exists to grade. Boundaries now land on 0.6s and 1s, which bracket the close times this service ingests at, with 0.5 and 0.75 just below them so movement between runs stays visible rather than rounded away.

Pool-share trustlines were buffered in memory for the whole bucket scan so the final flush could drop the shares of any pool the capture had skipped. The skip it defends against cannot occur: LiquidityPoolType has one enum value, its DecodeFrom rejects every other discriminant, and GetConstantProduct returns false only for a discriminant the decoder never produces. Reaching the branch in a test required hand-building a LiquidityPoolEntryBody the wire format cannot express. The buffering bought nothing else either. The load runs in one transaction and liquidity_pool_balances references liquidity_pools with a DEFERRABLE INITIALLY DEFERRED constraint, so insert ordering within the load was already irrelevant. Shares go straight into the batch again, and an unrepresentable pool body aborts the load instead of being skipped: if a second pool type ever arrives, dropping the pool would drop every holder's shares with it, and a checkpoint that quietly loses balances is worse than one that fails. processEntry returns an error to carry it, which the scan loop already propagates.

The ledger-indexer pool already sizes off GOMAXPROCS, which the Go 1.25+ runtime derives from the container CPU limit, while the backfill pool still defaulted to runtime.NumCPU() and so to the node's core count. Both pools now use the same basis, and the comments say where the limit comes from.

The test only counted the buckets, so swapping the 0.6 and 0.75 grading bars for any other values would still pass. Assert each upper bound instead.

Re-bracketing the phase histogram on the close-time bar dropped the two tail buckets, leaving 10s as the largest finite boundary. histogram_quantile reports the highest finite bound when the quantile falls in +Inf, so an 11-second and a 60-second stall would both read as 10s and a severe slowdown would be invisible. Keep the tail.

…flag ingestService carried four fields (appTracker, getLedgersLimit, knownContractIDs, contractMetadataService) and Indexer two (ingestionMetrics, networkPassphrase) that were assigned at construction and never read — consumers get these through their own constructors or through internal/ingest.Configs. Deleting them also drops AppTracker, GetLedgersLimit, and ContractMetadataService from IngestServiceConfig (the checkpoint service keeps its own metadata-service wiring). Also removes the --latest-ledger-cursor-name flag, which only fed a deprecation warning, and the phantom load_current_state phase label from the ProtocolStateProcessingDuration help text (never emitted).

Every remaining parameter of persistLedgerData had exactly one production value: cursorName was always the latest-ledger cursor (the blind-Update branch was reachable only from tests), ledgerMeta was never nil, and the numTxs/numOps returns only fed two counters the buffer already knows via GetNumberOfTransactions/GetNumberOfOperations. The signature is now (ctx, ledgerSeq, ledgerMeta, plan, contractData, buffer) error, the cursor write is unconditionally guarded, and the contractDataMemo nil-receiver branch (test-only) is gone. IndexerBufferInterface shrinks to its real seam — the indexer's fold-in plus the five getters insertIntoDB reads; everything else is reached on the concrete *IndexerBuffer. Tests now exercise the guarded cursor path persistLedgerData actually runs in production.

…Backoff persistLedgerDataWithRetry (né ingestProcessedDataWithRetry) hand-rolled the same attempt loop, exponential backoff, cap, and context handling that utils.RetryWithBackoff already provides — the body is now a single RetryWithBackoff call with isPermanentPersistError as the classifier. Metric semantics preserved: transient retries count RetriesTotal, permanent errors count ErrorsTotal and return immediately, exhaustion counts both plus RetryExhaustionsTotal, and context cancellation counts neither. RetryWithBackoff's exhaustion return now wraps the exported sentinel utils.ErrRetriesExhausted alongside the final error, which is how the wrapper tells exhaustion apart from a permanent-error exit.

…ist stages The live loop ran fetch, process_ledger, and the DB persist strictly in sequence, so the per-ledger time was their sum — which can exceed the ledger cadence at high transaction volume even when each stage individually fits inside it. The loop is now a three-stage errgroup pipeline over depth-1 channels: while ledger N persists, N+1 processes and N+2 is fetched, making the ledger time the slowest stage instead of the sum. Persist stays strictly sequential in ledger order (the guarded cursor and per-protocol CAS chain advance N-1 → N) and is the only writing stage, so the advisory-lock liveness probe moves there. Classification planning also stays on the persist stage: its known-hash lookup is a non-transactional pool read that must observe the previous ledger's committed protocol_wasms rows. Two IndexerBuffers rotate through a freeBuffers channel, cleared only at take so a clear can never tear a persist still reading the maps the getters alias. Any stage error cancels the pipeline and the process exits, resuming from the cursor on restart — the Duration metric now sums the ledger's stage times, since wall time under a pipeline would count queue wait.

The ingestLiveLedgers comment claimed a stage error cancels the pipeline and returns. Cancellation and the producer's channel close both fire, so a downstream stage picks either and a ledger already handed off can still be persisted first — which is what we want, since every ledger that lands moves the cursor forward for the restart. Say that instead. Also record why the stage hand-off channels carry a slot: the overlap comes from receiving input before claiming run resources, not from the depth, so the slot is there to ride out a backend's bursty prefetch refill.

… error The persist stage probes the advisory-lock session before every ledger, but the probe ran on the pipeline context immediately after the receive from `processed`. On cancellation that channel usually holds a ledger, so the select can pick the ledger over ctx.Done(); the probe then fails with "context canceled" and the stage reports a lock loss that never happened, incrementing ErrorsTotal{ingest_live}. A plain SIGTERM was enough to trigger it. Check cancellation before attributing the failure to the lock, using the same wording the fetch and process stages already use.

persistLedgerData's ledgerMeta parameter became a value, which dropped the `ledgerMeta != nil` half of the guard around the protocol-processor block. LedgerCloseTime() on a zero-value LedgerCloseMeta dereferences a nil V0 arm and panics, and ledgerSeq-1 underflows to 4294967295. Three tests passed xdr.LedgerCloseMeta{} and survived only because their services register no protocol processors — the first test to register one would panic instead of failing. Use dummyLedgerMeta, the fixture already in this package, so the trap is gone at the source rather than papered over with a guard the live path can never need.

Four comments no longer described the code: - The shutdown paragraph claimed in-flight ledgers are drained and persisted so a restart lands further ahead. They are not: errgroup cancels the group context as soon as a stage fails, which fails both the lock probe and the persist transaction, so a handed-off ledger is always dropped. - The freeBuffers comment justified why handing a buffer back never blocks at today's capacities without naming the invariant that makes it true: cap(freeBuffers) >= cap(processed) + 1. Raising `processed` on its own — the obvious tuning knob — deadlocks the pipeline. - Ingestion.Duration sums three stage times that overlap in wall-clock across different ledgers, so it measures per-ledger work and is no longer a keep-up proxy. Say so, and point at LagLedgers instead. - One stale reference to ingestProcessedDataWithRetry survived the rename. Also note that channel depth bounds the pipeline by ledger count, not bytes, since peak working set grew with the extra in-flight metas and buffers.

… an attempt RetryWithBackoff only checked the context between attempts, so a cancellation landing while fn ran fell through as an ordinary failure: on the last attempt it was tagged ErrRetriesExhausted, and on earlier ones it reached onRetry, bumping the retry counter and logging "retrying" for a shutdown. persistLedgerDataWithRetry then counted a clean SIGTERM as a db_persist exhaustion, against its own documented contract. Check ctx.Err() right after fn fails, before the permanent classifier and onRetry, and return the context error untagged in both cases.

… for lock loss When one stage fails, errgroup cancels the pipeline context and the other stages' in-flight calls fail with that cancellation, so a single fault incremented ErrorsTotal{ingest_live} a nondeterministic number of times. Every stage now counts only when ctx.Err() is nil. The advisory-lock probe checked cancellation before running, so a cancel landing mid-probe still surfaced as a lost lock; it now checks after. The buffer return to the rotation is paired with ctx.Done() so it can never strand the persist stage should the rotation ever fill.

Deriving a SAC contract ID costs a SHA-256 over the asset's contract-ID preimage plus a strkey encode, and both the effects and token-transfer processors recompute it for the same few assets on every event. Each processor now memoizes by asset identity; entries are content-derived under the processor's fixed network passphrase, so they never invalidate, and sync.Map keeps the lookup safe across the indexer's pool workers.

The sponsorship effect paths had no coverage: no test exercised addSignerSponsorshipEffects, addLedgerEntrySponsorshipEffects, or the accountHasSponsoredSigner guard that spares the SponsorPerSigner build for accounts sponsoring nothing. Covers every transition (created, removed, updated, unchanged, whole-entry removal), the guard's over-admit-never-over-reject contract, deterministic signer ordering, and each sponsorable entry type's details shape.

Every NewStateChangeBuilder and Clone() heap-allocated a ~480-byte builder, and the hot paths clone constantly — a plain payment paid four heap builders for three state changes. The builder is now used by value: With* methods mutate their own copy and return it, branching is plain assignment, Clone() is gone, and builders stay on the stack. The write-only metricsService field is dropped from the builder along with the constructor parameter. parseTrustline now returns the builder it produced so the balance-authorization change emitted for a new trustline keeps inheriting the resolved token and the trustline limit exactly as it did through the shared pointer; whether that inherited limit belongs on a BALANCE_AUTHORIZATION row is flagged in the PR as a follow-up decision. Helper signatures across the effects, token-transfer, contract-deploy, SAC, SEP-41, and Blend processors take the builder by value; output is byte-identical everywhere.

The AddressMuxed removal left github.com/guregu/null unused module-wide, and the sponsorship fixtures always build entries for sponsoredAccount, so the account parameter comes off both helpers (unparam).

…tion XDR encoder The signers-only skip asked the SDK's AccountChangedExceptSigners, which deep-copies both account entries, XDR-marshals each, and compares the bytes — 2.6% of ingest CPU and 5.7% of allocated bytes at ~18k tx/s. A local field comparison preserves the exact semantics (only the Signers slice is ignored; NumSubEntries and V2 signer-sponsoring IDs still count; a missing extension equals V1 with zero liabilities), pinned by a 25-case differential test against the SDK function as oracle. The always-nil error plumbing collapses with it. ConvertOperation's per-operation XDR marshal allocated a fresh encoder and growth-doubled buffer each call; a pooled EncodingBuffer reuses the scratch across pool workers and allocates only the retained, exact-size copy of each operation's bytes.

The quadratic scan's justifying comment — streams are a handful of changes per transaction — is true for the indexer's per-transaction call but false for the sep41 and blend processors, which assign ordinals over a whole staged migration window at persist time, inside the CAS-guarded window transaction, with nothing bounding the slice but the window. A counting map keeps the assignment linear at every call site; IDs are byte-identical to the scan's. The per-call map allocation is noise on the live path: four calls per transaction against a process stage that allocates five orders of magnitude more per ledger.

…n table missed Three accountChangedExceptSigners branches had no reaching table row in TestAccountChangedExceptSigners_MatchesSDK: a differing AccountId, an InflationDest transitioning set to nil, and sponsoring-ID slices of different length (nil vs one element). Each new row differs from the base entry in exactly one field, so its verdict can only come from that field's guard; the SDK oracle agrees on all three.

Only Soroban host functions write ContractData entries, so transactionContractDataChanges now returns before walking or allocating anything when the transaction carries no SorobanTransactionData — the common case on a payment-dominated ledger. The per-transaction wrapper lookup becomes a dense slice instead of a map, and the result map is allocated on the first change actually collected instead of eagerly. The reader-extraction equivalence test still passes over the real-ledger fixtures, which pins that the gate drops nothing a Soroban transaction produces.

…m one memo AssetContractIDMemo is the package's single cache for the SHA-256-plus- strkey derivation: token_transfer's hand-inlined sync.Map becomes a memo field, and the two SAC comparison sites that recomputed the derivation per event (isSACContract, trustlineEntryMatches) now read through a memo on the processor. Three accessors normalize the three caller shapes onto one complete key, so the same asset lands on one entry no matter which path reaches it; the key's completeness and the accessor split's cost reasoning are documented on the type. Trustline matching compares contract IDs strkey-encoded — equivalent by injectivity — which retires a per-call decode. The preimage path in contract_operations stays uncached: it fires roughly once per asset lifetime (SAC deploys), not per event.

Copilot

Copilot review overview

🟡 Changes recommended

Critical migration and persistence issues, along with behavior and compatibility concerns, remain unresolved.

Review effort: Lite
Findings: 2 · 4

Open (6)

INNER JOIN hides orphaned pool-share changes · New
Add forward migration for new SAC constraint · New
Undocumented --live-persist-max-batch-size flag · New
Preserve participant links for failed classic operations · New
Route Soroban processors through shared operation changes · New
Avoid breaking Prometheus metric type changes · New

What changed in this PR

This PR optimizes indexer processing and persistence performance while updating protocol support, migrations, metrics, and integration infrastructure.

Changes:

Memoizes operation decoding and reduces processing allocations.
Adds persistence batching, cursor bounds, migration tooling, and database safeguards.
Updates protocol coverage, metrics, dependencies, documentation, workflows, and tests.

File	Summary
`README.md`	Documentation and protocol updates.
`internal/utils/sql.go`	Nullable signer-key conversion.
`internal/utils/retry.go`	Retry exhaustion classification.
`internal/utils/retry_test.go`	Retry behavior tests.
`internal/services/token_ingestion.go`	Split balance persistence APIs.
`internal/services/token_ingestion_test.go`	SAC balance gating tests.
`internal/services/sep41/validator.go`	Stored metadata lookup.
`internal/services/sep41/metadata_test.go`	Metadata caching tests.
`internal/services/protocol_validation_dispatch_test.go`	Updated retry-loop coverage.
`internal/services/protocol_processor.go`	Processor contract updates.
`internal/services/protocol_migrate_lock.go`	Migration advisory locks.
`internal/services/protocol_migrate_lock_test.go`	Lock behavior tests.
`internal/services/protocol_migrate_history.go`	History migration locking.
`internal/services/protocol_migrate_history_test.go`	History lock tests.
`internal/services/protocol_migrate_current_state.go`	Current-state locking.
`internal/services/protocol_migrate_current_state_test.go`	Current-state lock tests.
`internal/services/protocol_migrate_current_state_rebuild.go`	Current-state rebuilds.
`internal/services/mocks.go`	Updated service mocks.
`internal/services/ingest_live_test.go`	Persist error coverage.
`internal/serve/serve.go`	Bounded metrics scraping.
`internal/serve/middleware/metrics_middleware.go`	Bounded metric labels.
`internal/serve/middleware/metrics_middleware_test.go`	Label-bound tests.
`internal/serve/graphql/schema/queries.graphqls`	Query documentation.
`internal/serve/graphql/resolvers/statechange.resolvers.go`	Signer-key resolution.
`internal/serve/graphql/resolvers/statechange_resolvers_test.go`	Resolver coverage.
`internal/serve/graphql/resolvers/resolver.go`	Signer-key resolver support.
`internal/serve/graphql/generated/generated.go`	Regenerated GraphQL output.
`internal/metrics/ingestion.go`	Ingestion metrics and buckets.
`internal/metrics/ingestion_test.go`	Metric tests.
`internal/metrics/http.go`	HTTP histogram metrics.
`internal/metrics/http_test.go`	HTTP metric tests.
`internal/integrationtests/infrastructure/testconstants.go`	Protocol version updates.
`internal/integrationtests/infrastructure/main_setup.go`	Object-store naming updates.
`internal/integrationtests/infrastructure/ledger_exporter.go`	SeaweedFS exporter configuration.
`internal/integrationtests/infrastructure/containers.go`	Integration container updates.
`internal/integrationtests/data_migration_test.go`	Object-store migration setup.
`internal/ingest/timescaledb.go`	Shared hypertable handling.
`internal/ingest/ingest_test.go`	Ingest validation tests.
`internal/ingest/datastore_backend.go`	Backend documentation updates.
`internal/indexer/protocol28_test.go`	Protocol 28 decoding tests.
`internal/indexer/processors/trustlines.go`	Memoized operation changes.
`internal/indexer/processors/trustlines_test.go`	Wrapper test updates.
`internal/indexer/processors/transaction_operation_wrapper.go`	Shared change memoization and deduplication.
`internal/indexer/processors/transaction_operation_wrapper_test.go`	Participant deduplication tests.
`internal/indexer/processors/state_change_builder_test.go`	Value-based builder tests.
`internal/indexer/processors/sac_instances.go`	Memoized operation changes.
`internal/indexer/processors/sac_instances_test.go`	Wrapper test updates.
`internal/indexer/processors/sac_balances.go`	Memoized operation changes.
`internal/indexer/processors/sac_balances_test.go`	Wrapper test updates.
`internal/indexer/processors/protocol_wasms.go`	Memoized operation changes.
`internal/indexer/processors/protocol_wasms_test.go`	Wrapper test updates.
`internal/indexer/processors/protocol_contracts.go`	Executable classification.
`internal/indexer/processors/participants.go`	Participant extraction optimization.
`internal/indexer/processors/participants_test.go`	Participant behavior coverage.
`internal/indexer/processors/liquidity_pools.go`	Memoized operation changes.
`internal/indexer/processors/liquidity_pools_test.go`	Wrapper test updates.
`internal/indexer/processors/contracts/utils.go`	Storable address conversion.
`internal/indexer/processors/contracts/utils_test.go`	Address conversion tests.
`internal/indexer/processors/codes.go`	Additional result codes.
`internal/indexer/processors/codes_test.go`	Exhaustive result-code tests.
`internal/indexer/indexer_buffer_bench_test.go`	Transaction-loading benchmarks.
`internal/db/migrations/2026-08-21.0-statechanges-account-id-index.sql`	State-change indexes.
`internal/db/migrations/2026-08-13.0-liquidity_pool_balances-pool-fk.sql`	Pool-balance foreign key.
`internal/db/migrations/2025-06-13.0-contract_tokens.sql`	SAC asset constraint.
`internal/db/migrations/2025-06-10.4-statechanges.sql`	State-change index documentation.
`internal/db/db.go`	Pool and transaction abstractions.
`internal/data/trustline_balances.go`	Sorted balance upserts.
`internal/data/transactions.go`	Cursor bounds and split COPY.
`internal/data/transactions_test.go`	Transaction COPY and cursor tests.
`internal/data/sep41/wipe.go`	SEP-41 state wiping.
`internal/data/sep41/wipe_test.go`	Wipe tests.
`internal/data/sep41/balances_test.go`	Muxed balance tests.
`internal/data/sac_balances.go`	SAC validation joins.
`internal/data/query_utils.go`	Cursor-bound SQL helpers.
`internal/data/protocol_contracts.go`	Transaction-aware lookups.
`internal/data/protocol_contracts_test.go`	Lookup test updates.
`internal/data/operations.go`	Cursor bounds and split COPY.
`internal/data/operations_test.go`	Operation COPY and cursor tests.
`internal/data/native_balances.go`	Sorted balance upserts.
`internal/data/mocks.go`	Updated data mocks.
`internal/data/liquidity_pool_balances.go`	Pool validation joins.
`internal/data/liquidity_pool_balances_test.go`	Pool validation tests.
`internal/data/ingest_store.go`	Bulk-table reconciliation.
`internal/data/contract_tokens.go`	Metadata lookup API.
`internal/data/contract_tokens_test.go`	Metadata and constraint tests.
`internal/data/accounts.go`	Shared account-link COPY helper.
`go.sum`	Dependency checksums.
`go.mod`	Dependency updates.
`docs/data-migrations/running-a-data-migration.md`	Rebuild workflow documentation.
`docs/data-migrations/adding-a-protocol.md`	Processor interface documentation.
`cmd/utils/custom_set_value_test.go`	Validation expectation updates.
`cmd/protocol_setup.go`	Metadata service setup.
`cmd/ingest.go`	Live persistence batching configuration.
`.github/workflows/notify-slack-new-issue.yml`	Slack issue notifications.
`.github/workflows/build.yml`	Deployment tagging controls.
`.github/pull_request_template.md`	Contribution checklist updates.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+			SELECT u.account_id, u.pool_id, u.shares, u.last_modified_ledger
+			FROM UNNEST($1::bytea[], $2::text[], $3::bigint[], $4::bigint[])
+				AS u(account_id, pool_id, shares, last_modified_ledger)
+			JOIN liquidity_pools lp ON lp.pool_id = u.pool_id


+    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+    -- A SAC's asset is bound to its contract id, so a SAC row always carries code and
+    -- issuer. Other types may leave them NULL.
+    CONSTRAINT contract_tokens_sac_has_asset CHECK (type <> 'SAC' OR (code IS NOT NULL AND issuer IS NOT NULL))


+			Name:        "live-persist-max-batch-size",
+			Usage:       "Maximum consecutive ledgers coalesced into one persist commit when live ingestion falls behind. The default 1 persists every ledger in its own commit — right for networks whose close time comfortably exceeds the persist time; raise it on high-TPL/short-block deployments to amortize inserts across a backlog at the cost of coarser crash recovery and visibility latency under load. Peak buffer memory is roughly twice this value — check headroom before raising it under GOMEMLIMIT.",
+			OptType:     types.Int,
+			ConfigKey:   &cfg.LivePersistMaxBatchSize,
+			FlagDefault: 1,
+			Required:    false,


+	// A failed transaction changes no state beyond its fee. Payment destinations,
+	// signer keys and auth entries are unverified submitter input, so only the accounts
+	// whose state changed are attributed. The set is never empty, so the operation row
+	// still exists for the submitter.
+	if !op.Transaction.Successful() {
+		participants := set.NewThreadUnsafeSet(op.Transaction.Envelope.SourceAccount().ToAccountId().Address())
+		if op.Transaction.Envelope.IsFeeBump() {
+			participants.Add(op.Transaction.Envelope.FeeBumpAccount().ToAccountId().Address())
+		}
+		return participants, nil
+	}


+// Changes returns the ledger entry changes the operation produced, decoding them on
+// the first call and serving the memo on every later one. All of an operation's
+// processors share a single wrapper, so the decode, the allocations, and the sort
+// the SDK accessor performs happen once per operation rather than once per processor.
+//
+// The wrapper is used from a single goroutine — the per-transaction worker walks its
+// operations sequentially — so the memo needs no synchronization.


+		RequestsDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
+			Name:    "wallet_http_request_duration_seconds",
+			Help:    "Duration of HTTP requests in seconds.",
+			Buckets: []float64{0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0, 30.0},
 		}, []string{"endpoint", "method"}),


aditya1702 force-pushed the live-ingest-processor-perf branch 4 times, most recently from c4aa67e to f6407a2 Compare August 22, 2026 13:40

aditya1702 self-assigned this Aug 24, 2026

aditya1702 added this to the Wallet Backend O3 Performance Testing milestone Sep 12, 2026

aditya1702 added this pull request to stack #760 September 30, 2026 19:24

aditya1702 added 17 commits October 1, 2026 16:12

test(metrics): pin every phase-duration bucket boundary

f02e025

The test only counted the buckets, so swapping the 0.6 and 0.75 grading bars for any other values would still pass. Assert each upper bound instead.

aditya1702 added 9 commits October 1, 2026 16:35

chore: tidy unused dependency and single-caller test helper params

b98ac62

The AddressMuxed removal left github.com/guregu/null unused module-wide, and the sponsorship fixtures always build entries for sponsoredAccount, so the account parameter comes off both helpers (unparam).

Copilot AI lite review requested due to automatic review settings October 1, 2026 21:06

aditya1702 force-pushed the live-ingest-processor-perf branch from f6407a2 to 9de86a6 Compare October 1, 2026 21:06

Copilot started reviewing on behalf of aditya1702 October 1, 2026 21:07

View session

Copilot AI reviewed Oct 1, 2026

View reviewed changes

aditya1702 removed this pull request from stack #760 October 1, 2026 21:18

aditya1702 changed the base branch from live-ingest-persist-path to persist/6-hardening October 1, 2026 21:18

aditya1702 closed this Oct 1, 2026

aditya1702 reopened this Oct 1, 2026

aditya1702 added this pull request to stack #713 October 1, 2026 21:33

aditya1702 requested a review from a team as a code owner October 20, 2026 18:02

Navigation Menu

Repository navigation

Cut process-stage CPU and allocations across the indexer and processors - #685

Cut process-stage CPU and allocations across the indexer and processors#685
aditya1702 wants to merge 76 commits into
persist/6-hardeningfrom
live-ingest-processor-perf

aditya1702 commented Aug 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

aditya1702 commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

How it's verified

Numbers

Stack

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aditya1702 commented Aug 13, 2026 •

edited

Loading