feat(db): Database connection multiplexing for serverless environments (#600)#651
Conversation
|
@jadonamite Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
615f491 to
7167381
Compare
|
Possible reasons:
🤖 Drips Wave Merge Agent |
…evs17#600) Route serverless DB access through a transaction-pooling proxy (RDS Proxy or PgBouncer) so a small set of backend connections is multiplexed across many concurrent function invocations, preventing connection exhaustion. - backend/shared/db/serverlessPool.ts: transaction-pooling adapter with IAM/SCRAM-256 auth, credential refresh, withClient/withTransaction helpers, and abandoned-connection leak detection (>30s force-close). - backend/serverless/dbConfig.ts + withDatabase.ts: env-driven pool config (RDS IAM token provider) and a Lambda wrapper that releases the client in a finally block after every invocation. - backend/monitoring/connectionPoolMetrics.ts: Prometheus pool/leak metrics and structured leak alerting. - infra/terraform/{rds_proxy,pgbouncer}.tf: proxy provisioning (max ~50 pooled connections serving 500+ functions, transaction pooling). - docker-compose.yml + .env.example: local PgBouncer + Postgres for parity; all credentials read from a gitignored .env (no hardcoded secrets). Closes Smartdevs17#600
The dependencies block had a duplicate "zustand" entry and missing commas, producing invalid JSON that breaks `npm install` in CI. Keep a single zustand ^5.0.0 and the redis ^4.6.7 entry.
7167381 to
4e848fa
Compare
|
@Smartdevs17 Changes made, waiting for your review. |
|
@jadonamite merged. Thanks for contributing. Kindly drop a review 🙏 |
Summary
Implements database connection multiplexing for serverless workloads. Serverless functions (webhook handlers, auth callbacks, scheduled jobs) currently open a new DB connection per invocation, exhausting the connection limit during traffic spikes. This routes all connections through a transaction-pooling proxy (AWS RDS Proxy or self-hosted PgBouncer), so a small set of backend connections is multiplexed across hundreds of concurrent functions.
Changes
backend/shared/db/serverlessPool.ts—ServerlessConnectionPool: transaction-pooling adapter over the existingpgpool, withwithClient/withTransaction/queryhelpers, IAM and SCRAM-256 auth modes, per-connect credential refresh (rotating RDS tokens), and connection-leak detection that force-closes any client checked out longer than 30s.backend/serverless/dbConfig.ts— env-driven pool configuration, including an RDS IAM auth-token credential provider (@aws-sdk/rds-signer, lazily imported).backend/serverless/withDatabase.ts— Lambda handler wrapper that hands each invocation a pooled client and callsrelease()in afinallyblock, regardless of success/throw. Pool singleton is reused across warm invocations.backend/monitoring/connectionPoolMetrics.ts— Prometheus pool gauges (total/idle/waiting/checked-out), a*_leaked_totalcounter, and structured leak alerting.infra/terraform/rds_proxy.tf— RDS Proxy with IAM auth, transaction pooling (~50 backend connections), TLS required, 30s idle timeout,EXCLUDE_VARIABLE_SETSpinning filter.infra/terraform/pgbouncer.tf— self-hosted PgBouncer alternative (transaction mode, SCRAM-256,MAX_PREPARED_STATEMENTS, 500 client cap / 50 server pool).docker-compose.yml— local Postgres + PgBouncer for dev/test parity.Acceptance criteria
MAX_PREPARED_STATEMENTS/ RDS Proxy)release()in afinallyblock per invocationCloses #600