Add configurable Docker resource limits for local deployment #68

⚡ Distributed Traffic Control

Globally coordinated traffic control for distributed systems

A production-oriented traffic-control platform built with Java 21 and Spring Boot, using Redis-backed leases, atomic coordination, and bounded capacity allocation to enforce shared global limits across multiple application instances.

Overview · Architecture · Coordination · Quick Start · Performance · Observability · AWS · Production Readiness

🎯 Overview

Distributed traffic control is straightforward on one process and much harder once the same capacity must be shared by several independent application instances.

A naive per-node limiter can accidentally multiply a global limit:

                    Global Capacity = 100

        ┌──────────────┐
        │   Gateway 1  │ ──► 100
        ├──────────────┤
        │   Gateway 2  │ ──► 100
        ├──────────────┤
        │   Gateway 3  │ ──► 100
        └──────────────┘

                 Total = 300  ❌

This project separates the problem into two layers:

Local request enforcement — fast, in-process quota checks on each gateway.
Global capacity coordination — Redis-backed shared capacity managed through bounded leases.

The result is a distributed traffic-control engine where multiple gateways can make local decisions while still respecting a single global capacity boundary.

Why this project exists

The interesting engineering problem is not implementing a basic rate limiter. It is handling the distributed-systems concerns around it:

shared global capacity
concurrent allocation
atomic multi-key state changes
bounded reservations
lease expiration
unused-capacity reclamation
gateway failure and recovery
low-cardinality observability
measurable latency and throughput

✨ Highlights

Distributed multi-gateway traffic control
Redis-backed global capacity coordination
Lease-based capacity allocation
Atomic Redis Lua scripts for distributed mutations
Bounded lease size to prevent one node from reserving the entire pool
Lease acquisition, consumption, renewal, release, expiration, and reclamation
Protection against concurrent over-allocation
Token Bucket, Fixed Window, and Sliding Window local algorithms
Prometheus metrics and Grafana dashboards
JUnit 5 + Testcontainers integration validation
Automated GitHub Actions CI with Redis service support
Docker Compose local deployment
AWS deployment using ECR, EC2, ALB, ElastiCache Redis, and SSM
k6 single-node and distributed performance validation
Controlled gateway failure and recovery validation

🏗️ Architecture

flowchart TB
    R[Traffic Request] --> E[Traffic Decision Engine]

    E --> P[Policy Resolver]
    E --> Q[Local Quota Enforcement]

    Q -->|Quota available| A[ALLOWED]
    Q -->|Quota exhausted| C[Capacity Allocator]

    C --> L[Redis Lease Coordinator]
    L --> G[(Redis Global Capacity)]
    L --> H[(Lease Registry)]

    G --> L
    H --> L

    L --> C
    C -->|Lease acquired| A
    C -->|No global capacity| D[REJECTED]

Request decision flow

Request
   │
   ▼
Resolve Policy
   │
   ├── missing / inactive ───────────────► REJECT
   │
   ▼
Build Quota Key
   │
   ▼
Local Quota Check
   │
   ├── capacity available ───────────────► ALLOW
   │
   ▼
Capacity Allocator
   │
   ▼
Acquire Redis-backed Lease
   │
   ├── acquisition fails ────────────────► REJECT
   │
   ▼
Consume Lease Capacity
   │
   ▼
ALLOW

Local vs. global state

Concern	Identity	Store	Purpose
Local quota	`policy + subject + resource`	In-memory	Fast request-path enforcement
Global capacity	`policy + resource`	Redis	Shared distributed budget
Lease state	`leaseId`	Redis	Capacity ownership and reclamation
Lease registry	`policy + resource`	Redis sorted set	Expiration tracking

This separation keeps Redis off the normal local-capacity path while preserving a single globally coordinated capacity boundary when nodes need more quota.

🧠 Core Design

Traffic subject

A TrafficSubject identifies the entity generating traffic.

Supported subject types:

USER
API_KEY
IP_ADDRESS
TENANT

Traffic policy

A TrafficPolicy defines how traffic is controlled.

Supported policy types:

TOKEN_BUCKET
FIXED_WINDOW
SLIDING_WINDOW

Policies also carry lifecycle state and capacity/refill configuration.

Local quota

The local quota identity is:

policy + subject + resource

The gateway can therefore make many decisions without contacting Redis.

Global capacity

The distributed capacity identity is:

policy + resource

All gateway instances coordinate against the same shared pool.

Quota lease

A QuotaLease temporarily assigns a bounded portion of the global pool to a gateway.

Global capacity = 100

Gateway 1 → lease 10
Gateway 2 → lease 10
Gateway 3 → lease 10
...

The default deployment caps an individual lease at 10 units.

🔐 Distributed Coordination

Redis state is conceptually organized as:

traffic-control:quota:<policy-id>:<resource>
traffic-control:quota:<policy-id>:<resource>:leases
traffic-control:lease:<lease-id>

Why Redis Lua scripts are used

Several distributed operations update multiple pieces of Redis state together. Performing those mutations as independent commands would introduce race windows.

The coordinator therefore uses Lua scripts for operations such as:

lease acquisition
expired-lease reclamation
lease consumption
lease renewal
lease release

This makes the state transition atomic from Redis's point of view.

Lease acquisition

Conceptually:

                Request lease
                     │
                     ▼
          ┌─────────────────────┐
          │ Redis atomic script  │
          └──────────┬──────────┘
                     │
          ┌──────────┴──────────┐
          │                     │
          ▼                     ▼
  Reclaim expired       Check global
     leases               capacity
          │                     │
          └──────────┬──────────┘
                     ▼
             Reserve capacity
                     │
                     ▼
              Create lease
                     │
                     ▼
             Register expiry

Expiration and reclamation

Redis key expiration alone is not sufficient when unused lease capacity must be returned to the global pool.

The implementation keeps an explicit lease registry:

Expired lease
     │
     ▼
Find expired lease
     │
     ▼
Read remaining capacity
     │
     ▼
Return unused capacity
     │
     ▼
Delete lease state
     │
     ▼
Remove registry entry

This prevents abandoned reservations from permanently reducing the shared capacity pool.

♻️ Lease Lifecycle

                  ┌───────────┐
                  │  Acquire  │
                  └─────┬─────┘
                        │
                        ▼
                  ┌───────────┐
                  │   Active  │
                  └─────┬─────┘
                        │
             ┌──────────┼──────────┐
             │          │          │
             ▼          ▼          ▼
          Consume      Renew     Release
             │          │          │
             │          ▼          ▼
             │        Active   Return unused
             │
             ▼
        Remaining = 0
             │
             ▼
          Release

Expired lease
      │
      ▼
Read remaining capacity
      │
      ▼
Return to global pool
      │
      ▼
Delete lease + registry entry

A lease is therefore a stateful distributed resource with explicit ownership and lifecycle semantics, not just an integer counter.

🛠️ Technology Stack

Area	Technology
Language	Java 21
Framework	Spring Boot 4.1.1
Build	Maven
Runtime	Eclipse Temurin 21
Distributed coordination	Redis 7
Atomic coordination	Redis Lua
Containerization	Docker / Docker Compose
Metrics	Micrometer
Monitoring	Prometheus
Dashboards	Grafana
Integration testing	JUnit 5 + Testcontainers
Load testing	k6
HTTP server	Tomcat 11
AWS runtime	ECR + EC2 + ALB + ElastiCache Redis + SSM

🌐 API

Evaluate traffic

POST /api/v1/traffic/evaluate
Content-Type: application/json

Example request:

{
  "requestId": "req-001",
  "subject": {
    "subjectId": "user-123",
    "type": "USER"
  },
  "resource": "/api/orders",
  "requestedAt": "2026-09-07T15:20:00Z"
}

Example response:

{
  "status": "ALLOWED",
  "reason": "Request allowed",
  "remainingCapacity": 99
}

Possible decision status values include:

ALLOWED
REJECTED

Traffic admission is intentionally separate from HTTP transport semantics: a request can receive a valid HTTP response while the traffic-control decision itself is REJECTED.

🚀 Quick Start

Prerequisites

Install:

Java 21
Docker Desktop
Git

Clone

git clone https://github.com/yashdotdev13/distributed-traffic-control.git
cd distributed-traffic-control

Run the test suite

Linux/macOS:

./mvnw -B verify

Windows PowerShell:

.\mvnw.cmd -B verify

Start the distributed local environment

docker compose build
docker compose up -d

Check the services:

docker compose ps

Docker resource limits

The local Docker Compose deployment applies configurable resource limits to gateway instances and Redis to keep resource consumption predictable.

Default limits:

Service	CPU limit	Memory limit
Each gateway	1.0 CPU	512 MB
Redis	No explicit limit	256 MB

Resource limits can be customized through environment variables.

Copy the example environment file:

cp .env.example .env

On Windows PowerShell:

Copy-Item .env.example .env

Available configuration:

GATEWAY_CPUS=1.0
GATEWAY_MEMORY=512M
REDIS_MEMORY=256M

Gateway limits apply independently to each gateway instance. Adjust these values according to the resources available on your development machine.

After changing resource limits, recreate the containers:

docker compose up -d --force-recreate

Inspect the applied limits:

docker inspect traffic-control-gateway-1
docker inspect traffic-control-redis

These defaults are intended for local development and can be customized for different machine configurations.

Health checks

http://localhost:8081/actuator/health
http://localhost:8082/actuator/health
http://localhost:8083/actuator/health

Expected health state:

{
  "status": "UP"
}

Stop the environment

docker compose down

⚙️ Configuration

The application uses environment-driven configuration.

SERVER_PORT=8081
REDIS_HOST=localhost
REDIS_PORT=6379
NODE_ID=local-node
LEASE_DURATION=PT30S
LEASE_CAPACITY=10

For distributed deployment, each gateway receives a unique NODE_ID while all gateways share the same Redis-backed global capacity pool.

📊 Observability

Prometheus metrics are exposed at:

/actuator/prometheus

Key application metrics:

traffic_control_decision_duration_seconds
traffic_control_decisions_total
traffic_control_lease_allocations_total
traffic_control_quota_exhausted_total

Metrics intentionally avoid high-cardinality identifiers such as request IDs and subject IDs.

Operational signals

The metric model is designed to make the following behaviors observable:

decision throughput
allowed vs. rejected traffic
decision latency
quota exhaustion
lease allocation attempts
successful lease allocation
failed lease allocation

Prometheus scrapes the gateway instances and Grafana provides the dashboard layer.

Distributed tracing with OpenTelemetry

The application uses Spring Boot's Micrometer Observation/OpenTelemetry integration to export distributed traces over OTLP, with Jaeger providing the local trace visualization interface.

The instrumentation covers the traffic-decision workflow and Redis-backed lease operations, helping distinguish local quota evaluation from distributed coordination latency.

Key observations include:

traffic.decision — overall traffic-decision operation
traffic.policy.lookup — traffic-policy lookup
traffic.quota.evaluate — local quota evaluation
traffic.lease.allocate — lease allocation
traffic.lease.consume — consumption against an allocated lease
redis.lease.acquire — Redis lease acquisition
redis.lease.consume — Redis lease consumption
redis.lease.renew — Redis lease renewal
redis.lease.release — Redis lease release

The local Docker Compose environment includes Jaeger for inspecting traces. Span emission depends on the operation and code path being executed.

See the OpenTelemetry Tracing Guide for configuration, local setup, test requests, and troubleshooting.

🧪 Testing & Validation

The automated suite covers:

traffic decision evaluation
policy resolution
Token Bucket behavior
Fixed Window behavior
Sliding Window behavior
capacity allocation
lease acquisition
lease consumption
lease renewal
lease release
lease expiration
expired-capacity reclamation
global capacity protection
concurrent allocation safety

Current test status

Automated tests: 106
Failures:        0
Errors:          0
Skipped:         0
BUILD SUCCESS

Redis integration suite

The Redis integration tests run against a real Redis 7 instance supplied by Testcontainers.

Integration tests: 8
Failures:          0
Errors:            0
Skipped:           0

📈 Performance Validation

The system has been benchmarked with k6 in both single-node and distributed configurations.

Single-gateway baseline

Requests:          372,864
Throughput:        12,399 req/s
Average latency:   0.737 ms
p90 latency:       1.02 ms
p95 latency:       1.41 ms
Maximum latency:   73.85 ms
HTTP failures:     0.00%
Checks passed:     100.00%

Distributed 3-gateway benchmark

Requests:          128,487
Throughput:        4,273 req/s
Average latency:   2.20 ms
p90 latency:       3.66 ms
p95 latency:       5.10 ms
Maximum latency:   257.87 ms
HTTP failures:     0.00%
Checks passed:     100.00%

Comparison

Scenario	Throughput	Avg	p90	p95	HTTP Failures
Single gateway	12,399 req/s	0.737 ms	1.02 ms	1.41 ms	0.00%
3 gateways	4,273 req/s	2.20 ms	3.66 ms	5.10 ms	0.00%

These are environment-specific benchmark observations, not universal capacity guarantees.

Distributed lease evidence

During distributed validation:

Lease allocation attempts
Gateway 1 → 426,788
Gateway 2 →  36,578
Gateway 3 →  36,405

Successful leases → 10
Lease size        → 10 units
Total allocated   → 100 units

This demonstrates that requests exercised the distributed lease path and contended for the shared global capacity pool.

The current distributed k6 scenario does not guarantee perfectly even request distribution across gateways. Per-gateway request counts should therefore not be treated as a load-balancer fairness benchmark.

🧯 Failure & Recovery Validation

A controlled multi-gateway failure test was executed against the deployed topology.

Gateway failure

One gateway was stopped while Redis and the other gateways remained available.

gateway-1 → STOPPED
gateway-2 → UP
gateway-3 → UP
Redis     → UP

Traffic continued successfully through the surviving gateways.

Gateway recovery

The failed gateway was restarted:

gateway-1 → UP
gateway-2 → UP
gateway-3 → UP

The recovered gateway returned a healthy state and successfully resumed traffic evaluation.

What this validates

gateway-level failure does not require global coordination state to disappear
surviving nodes can continue serving traffic
the failed node can rejoin the deployment
Redis-backed global state remains shared across nodes

☁️ AWS Deployment

The project has also been deployed and validated on AWS using a multi-node topology:

                    Internet
                       │
                       ▼
              ┌─────────────────┐
              │ Application ALB  │
              │      :80         │
              └────────┬────────┘
                       │
          ┌────────────┼────────────┐
          │            │            │
          ▼            ▼            ▼
     ┌─────────┐  ┌─────────┐  ┌─────────┐
     │ EC2 #1  │  │ EC2 #2  │  │ EC2 #3  │
     │ Gateway │  │ Gateway │  │ Gateway │
     └────┬────┘  └────┬────┘  └────┬────┘
          │            │            │
          └────────────┼────────────┘
                       │
                       ▼
              ┌─────────────────┐
              │ ElastiCache     │
              │ Redis OSS 7.1   │
              └─────────────────┘

AWS components used:

Amazon ECR for the container image
EC2 for the three gateway nodes
Application Load Balancer for ingress
ElastiCache for Redis OSS
AWS Systems Manager for instance administration
Security groups for network isolation

The deployment was validated for:

ALB health checks
public traffic routing
shared Redis-backed capacity coordination
stopping a gateway while the other nodes continue serving requests
restarting the failed gateway and returning it to service

See the complete deployment procedure in docs/aws-deployment.md.

Important: the AWS environment is a validated deployment baseline, not a claim that every production-hardening control is already enabled.

🔒 Production Readiness

This repository is intentionally production-oriented in architecture and engineering practices, while the current AWS environment remains a controlled reference deployment.

Implemented production-oriented practices

explicit separation of local and distributed state
atomic distributed mutations
bounded capacity leasing
explicit lease ownership
expiration and reclamation
deterministic integration testing against Redis
CI validation on every push / pull request
low-cardinality operational metrics
containerized deployment
multi-node runtime validation
failure and recovery testing
environment-driven configuration

Additional hardening required for a high-risk production rollout

Before using this exact AWS baseline for a security-sensitive production workload, add and verify:

HTTPS/TLS at the public ingress
authenticated and authorized administrative APIs
secrets management instead of plain sensitive environment values
Redis TLS/authentication where required
highly available Redis configuration with automatic failover
CloudWatch alarms and durable operational alerting
centralized log retention and auditability
WAF / edge protection as appropriate
infrastructure-as-code for repeatable environments
autoscaling and capacity planning
disaster-recovery and backup/restore procedures
security review, dependency scanning, and image vulnerability scanning

This distinction is deliberate: the architecture is designed around production concerns, but a production claim should only be made after the surrounding security, reliability, and operations controls are also in place.

🧱 Project Structure

distributed-traffic-control/
│
├── src/
│   ├── main/java/
│   │   └── com/...
│   └── test/java/
│       └── com/...
│
├── load-tests/
│   ├── baseline.js
│   └── distributed.js
│
├── docs/
│   ├── phase-1-product-requirements/
│   ├── phase-2-system-design/
│   ├── aws-deployment.md
│   └── benchmark-and-validation.md
│
├── scripts/
│   └── aws/
│       └── ec2-gateway-user-data.sh
│
├── .github/
│   └── workflows/
│       └── ci.yml
│
├── Dockerfile
├── docker-compose.yml
├── prometheus.yml
├── pom.xml
└── README.md

🧭 Design Principles

Separate local and global state

Local quota enforcement handles the hot path, while Redis coordinates the shared global budget.

Make distributed mutations atomic

Operations that update multiple Redis structures are coordinated through Lua scripts.

Bound lease size

Nodes acquire bounded chunks of capacity instead of reserving the entire global pool.

Reclaim unused capacity

Released and expired leases return unused capacity to the shared pool.

Keep metrics operationally safe

Metrics remain low-cardinality so request-specific identifiers do not become monitoring labels.

Validate behavior, not just compilation

Distributed behavior is validated using integration tests, multi-node runtime tests, load tests, and controlled failure scenarios.

📚 Documentation

Detailed project documentation lives under docs/:

These documents cover requirements, architecture, distributed coordination, failure modes, deployment topology, and measured validation results.

🗺️ Roadmap

Completed

Future enhancements

✅ Validation Status

Area	Status
Core traffic-control engine	✅ Validated
Local quota enforcement	✅ Validated
Redis distributed coordination	✅ Validated
Lease lifecycle	✅ Validated
Expired-capacity reclamation	✅ Validated
Concurrent allocation safety	✅ Validated
Docker deployment	✅ Validated
3-node runtime	✅ Validated
GitHub Actions CI	✅ Validated
Prometheus	✅ Validated
Grafana	✅ Validated
Single-node benchmark	✅ Validated
Distributed benchmark	✅ Validated
Gateway failure handling	✅ Validated
Gateway recovery	✅ Validated
AWS multi-node deployment	✅ Validated
AWS gateway failure/recovery	✅ Validated

👤 Author

Yash Chauhan

📄 License

This repository does not currently declare a finalized open-source license.

Add a LICENSE file and update this section before publishing the project for external reuse.

-Original file line number
+Diff line change
@@ Expand Up / @@ -2,7 +2,15 @@ @@
     # Defaults are set in docker-compose.yml if these are unset.
     # Copy to .env and adjust if you need a different reviewed version.
+    # Image versions
     REDIS_VERSION=7.4
     JAEGER_VERSION=2.22.0
     PROMETHEUS_VERSION=v3.13.4
     GRAFANA_VERSION=13.2.4
+    # Gateway resource limits (applied independently to each gateway)
+    GATEWAY_CPUS=1.0
+    GATEWAY_MEMORY=512M
+    # Redis resource limits
+    REDIS_MEMORY=256M

-Original file line number
+Diff line change
@@ Expand Up / @@ -3,6 +3,10 @@ services: @@
       redis:
         image: redis:${REDIS_VERSION:-7.4}-alpine
         container_name: traffic-control-redis
+        deploy:
+          resources:
+            limits:
+              memory: "${REDIS_MEMORY:-256M}"
         healthcheck:
           test: ["CMD", "redis-cli", "ping"]
           interval: 5s
@@ Expand All / @@ -23,6 +27,11 @@ services: @@
       gateway-1:
         image: distributed-traffic-control:latest
         container_name: traffic-control-gateway-1
+        deploy:
+          resources:
+            limits:
+              cpus: "${GATEWAY_CPUS:-1.0}"
+              memory: "${GATEWAY_MEMORY:-512M}"
         depends_on:
           redis:
             condition: service_healthy
@@ Expand All / @@ -49,6 +58,11 @@ services: @@
       gateway-2:
         image: distributed-traffic-control:latest
         container_name: traffic-control-gateway-2
+        deploy:
+          resources:
+            limits:
+              cpus: "${GATEWAY_CPUS:-1.0}"
+              memory: "${GATEWAY_MEMORY:-512M}"
         depends_on:
           redis:
             condition: service_healthy
@@ Expand All / @@ -75,6 +89,11 @@ services: @@
       gateway-3:
         image: distributed-traffic-control:latest
         container_name: traffic-control-gateway-3
+        deploy:
+          resources:
+            limits:
+              cpus: "${GATEWAY_CPUS:-1.0}"
+              memory: "${GATEWAY_MEMORY:-512M}"
         depends_on:
           redis:
             condition: service_healthy
@@ Expand Down @@

Add configurable Docker resource limits for local deployment #68

Uh oh!

Add configurable Docker resource limits for local deployment #68

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

⚡ Distributed Traffic Control

Globally coordinated traffic control for distributed systems

🎯 Overview

Why this project exists

✨ Highlights

🏗️ Architecture

Request decision flow

Local vs. global state

🧠 Core Design

Traffic subject

Traffic policy

Local quota

Global capacity

Quota lease

🔐 Distributed Coordination

Why Redis Lua scripts are used

Lease acquisition

Expiration and reclamation

♻️ Lease Lifecycle

🛠️ Technology Stack

🌐 API

Evaluate traffic

🚀 Quick Start

Prerequisites

Clone

Run the test suite

Start the distributed local environment

Docker resource limits

Health checks

Stop the environment

⚙️ Configuration

📊 Observability

Operational signals

Distributed tracing with OpenTelemetry

🧪 Testing & Validation

Current test status

Redis integration suite

📈 Performance Validation

Single-gateway baseline

Distributed 3-gateway benchmark

Comparison

Distributed lease evidence

🧯 Failure & Recovery Validation

Gateway failure

Gateway recovery

What this validates

☁️ AWS Deployment

🔒 Production Readiness

Implemented production-oriented practices

Additional hardening required for a high-risk production rollout

🧱 Project Structure

🧭 Design Principles

Separate local and global state

Make distributed mutations atomic

Bound lease size

Reclaim unused capacity

Keep metrics operationally safe

Validate behavior, not just compilation

📚 Documentation

🗺️ Roadmap

Completed

Future enhancements

✅ Validation Status

👤 Author

📄 License

Uh oh!