Modern networks frequently experience performance degradation when multiple applications compete for limited bandwidth. Video streaming, cloud synchronization, software downloads, backups, and file transfers often share the same network path, resulting in reduced throughput and unpredictable user experience.
This project implements a complete network emulation and throughput prediction platform that reproduces these real-world conditions using Docker containers, Linux Traffic Control (tc), HTB queuing disciplines, Flask-based telemetry services, and a GRU-based machine learning model.
The system creates an isolated multi-container network topology consisting of client devices, servers, and a software router. A configurable bottleneck link is enforced using Linux HTB traffic shaping, while multiple SCP clients generate competing TCP flows. Real-time throughput measurements are continuously collected and streamed to a live monitoring dashboard. These measurements are simultaneously used by a Gated Recurrent Unit (GRU) network to predict future throughput values under changing network conditions.
The project demonstrates practical concepts from:
- Linux Networking
- Traffic Engineering
- Docker Containerization
- Network Telemetry Collection
- Real-Time Monitoring Systems
- Time-Series Machine Learning
- Network Performance Analysis
Network throughput is not constant. The performance experienced by a user depends on several dynamic factors such as:
- Available link bandwidth
- Number of competing flows
- TCP congestion behavior
- Traffic shaping policies
- Routing constraints
Traditional monitoring systems only report current network conditions. They do not provide visibility into future performance.
The objective of this project is to:
- Emulate realistic network contention scenarios.
- Observe throughput degradation under competing traffic loads.
- Collect real-time telemetry data.
- Train a machine learning model capable of forecasting future throughput.
- Visualize both actual and predicted throughput through a live dashboard.
The complete system architecture is shown below.
Figure 1: Detailed system architecture and inter-subsystem communication flow.
Figure 2: Subsystem execution pipeline.
The implementation consists of three major subsystems:
Responsible for creating the virtual network topology and generating controlled bandwidth contention.
Components:
- Stream Client
- SCP Clients
- Router
- Stream Server
- SCP Server
Responsible for collecting network measurements and exposing them through APIs and dashboards.
Components:
- Metrics Exporter
- REST APIs
- Server-Sent Events (SSE)
- Live Dashboard
Responsible for forecasting future throughput using historical telemetry.
Components:
- Prediction Engine
- Data Preprocessing
- GRU Ensemble Models
- Real-Time Inference Pipeline
The entire network topology is implemented using Docker containers connected through isolated bridge networks.
This enables:
- Reproducible experiments
- Lightweight deployment
- Isolated network stacks
- Real Linux networking behavior
The software router uses Linux Traffic Control (tc) and Hierarchical Token Bucket (HTB) scheduling to create a configurable bottleneck link.
Features include:
- Dynamic bandwidth caps
- Per-client traffic classes
- Fair bandwidth allocation
- Real-time bandwidth modification
The system continuously measures throughput by downloading large files over HTTP and recording transfer statistics.
Figure 3: Real-time telemetry web dashboard displaying live throughput measurements, active phase information, bandwidth limits, SCP flow counts, prediction statistics, and execution logs.
The dashboard receives telemetry through SSE and updates automatically.
Measurements are:
- Logged to CSV
- Published through REST APIs
- Streamed to the dashboard using SSE
- Used for machine learning inference
A GRU-based prediction engine learns throughput patterns from historical measurements and predicts future throughput values in real time. Predictions are displayed alongside live throughput measurements.
The emulated network consists of two isolated subnets connected through a dedicated software router.
172.30.0.0/24
Contains:
- Stream Client
- SCP Client 1
- SCP Client 2
- SCP Client 3
- Metrics Exporter
172.31.0.0/24
Contains:
- Stream Server
- SCP Server
Acts as the central bottleneck and forwarding device.
Interfaces:
eth0 → 172.30.0.1
eth1 → 172.31.0.1
All traffic between clients and servers must pass through this router.
The router container emulates the functionality of a physical Layer-3 router. Core responsibilities include:
Linux IP forwarding is enabled to allow packet traversal between the client and server networks.
sysctl -w net.ipv4.ip_forward=1This transforms the container from a host into a forwarding device.
The router performs source NAT using iptables MASQUERADE rules.
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADEThis ensures return traffic is correctly routed between the isolated Docker networks.
Ubuntu 24.04 applies restrictive forwarding policies by default. Explicit FORWARD rules are configured to permit traffic movement between the client and server subnets.
One of the primary objectives of the project is to reproduce realistic bandwidth contention scenarios. This is achieved using Linux Traffic Control (tc).
Without traffic shaping, all flows would consume bandwidth based solely on TCP congestion control. To create predictable and repeatable experiments, the router enforces a configurable bottleneck link.
Examples:
50 Mbps
30 Mbps
20 Mbps
10 Mbps
All client traffic must compete for this shared capacity.
The project uses the HTB queuing discipline. HTB was selected because it supports:
- Hierarchical bandwidth allocation
- Guaranteed minimum rates
- Bandwidth borrowing
- Dynamic reconfiguration
The hierarchy implemented is shown below:
Root Class
|
├── Stream Client
├── SCP Client 1
├── SCP Client 2
└── SCP Client 3
The root class represents the total available link capacity. Each client is assigned an independent child class.
Each HTB leaf class contains an SFQ scheduler.
Purpose:
- Fair sharing among concurrent flows
- Reduced flow starvation
- Improved bandwidth distribution
This ensures that multiple TCP sessions inside a class receive equitable bandwidth allocation.
Traffic is classified using source-IP based filters.
Example:
172.30.0.10 → Stream Client
172.30.0.20 → SCP Client 1
172.30.0.21 → SCP Client 2
172.30.0.22 → SCP Client 3
Each packet is assigned to the appropriate HTB class before transmission, allowing per-client bandwidth accounting and control.
The project uses two independent application protocols to generate realistic network traffic.
The Stream Server hosts a large 500 MB file using NGINX. The Stream Client continuously downloads this file using curl.
The download loop performs:
- HTTP GET request
- Byte counting
- Throughput computation
- CSV logging
- JSON export
This traffic represents:
- Video streaming
- Content delivery
- Media consumption
- Large web transfers
The SCP Server hosts another large file through OpenSSH. Three SCP clients can be independently enabled or disabled during the experiment. Each SCP client continuously downloads data from the SCP server.
This traffic represents:
- Cloud backups
- Software downloads
- File synchronization
- Enterprise data transfers
As additional SCP clients are activated, bandwidth contention increases and streaming throughput decreases.
The experiment is divided into multiple phases.
Figure 4: Multi-phase experimental sequence used for dataset generation. Each phase modifies the HTB bandwidth cap and/or the number of active SCP background transfers to create different congestion conditions.
Each phase modifies:
- Total link bandwidth
- Number of active SCP clients
The resulting throughput changes are recorded and visualized. Typical scenarios include:
| Scenario | Bandwidth Cap | SCP Clients |
|---|---|---|
| Baseline | 50 Mbps | 0 |
| Moderate Contention | 50 Mbps | 1 |
| Heavy Contention | 50 Mbps | 3 |
| Reduced Capacity | 30 Mbps | 0 |
| Severe Bottleneck | 20 Mbps | 2 |
These scenarios reproduce the behavior commonly observed in real residential and enterprise networks.
To enable real-time analysis and machine learning inference, the system continuously collects throughput measurements from the streaming client.
The telemetry subsystem was designed with the following objectives:
- Low overhead
- Real-time visibility
- Persistent logging
- API accessibility
- Machine learning integration
The complete telemetry pipeline is shown below:
Stream Client
|
v
latest.json + CSV
|
v
Metrics Exporter
|
+--> REST API
|
+--> SSE Stream
|
+--> Dashboard
|
+--> Prediction Engine
The Stream Client continuously downloads a large file from the NGINX server. For every measurement interval:
- Bytes transferred are recorded.
- Transfer duration is measured.
- Throughput is computed.
- Results are written to disk.
The throughput calculation is:
Throughput (Mbps) = (Bytes × 8) / (Time × 10^6)
Measurements are sampled periodically throughout the experiment.
Each measurement is written to a CSV log, used for:
- Dataset generation
- Offline analysis
- Model training
- Result visualization
Example:
timestamp,phase,cap_mbps,scp_flows,throughput_mbps
2025-07-01T10:00:01,1,50,0,49.1
2025-07-01T10:00:03,1,50,0,48.7
2025-07-01T10:00:05,2,50,1,24.2The most recent measurement is stored in latest.json. This file acts as a lightweight shared state between services. The Metrics Exporter continuously reads this file and publishes updates.
The Metrics Exporter is implemented using Flask and acts as the central telemetry aggregation service.
Responsibilities include:
- Reading throughput measurements
- Maintaining measurement history
- Serving REST APIs
- Streaming real-time updates
- Receiving ML predictions
- Serving the dashboard
The exporter provides a single point of interaction for all monitoring components.
GET /metrics — Returns current network telemetry.
{
"latest": {
"throughput": 24.5,
"cap": 50,
"scp_flows": 1
}
}Used by: Dashboard, Prediction Engine, external monitoring tools.
POST /prediction — Receives predicted throughput values generated by the GRU model.
{
"predicted_mbps": 23.8
}The dashboard displays both actual and predicted throughput.
Real-time updates are implemented using Server-Sent Events.
Endpoint:
GET /stream
Advantages:
- Lightweight
- Unidirectional
- Lower overhead than WebSockets
- Native browser support
The dashboard receives live telemetry without repeatedly polling the server.
The dashboard provides real-time visualization of network behavior. It is implemented using:
- Flask
- HTML
- JavaScript
- Chart.js
The dashboard receives telemetry through SSE and updates automatically.
Live Throughput Plot — Displays actual throughput and predicted throughput in real time, allowing immediate comparison between model predictions and observed network performance.
Phase Monitoring — Displays current experiment phase, active SCP flows, and configured bandwidth cap, allowing users to correlate throughput changes with network conditions.
Performance Metrics — The dashboard computes prediction quality metrics including:
Mean Absolute Error (MAE):
MAE = Σ |Actual - Predicted| / N
Measures average prediction error.
Mean Absolute Percentage Error (MAPE):
MAPE = 100/N × Σ |Actual - Predicted| / Actual
Measures prediction accuracy as a percentage.
One of the primary objectives of the project is to generate realistic throughput datasets under controlled network conditions. Unlike publicly available datasets, the generated data directly reflects:
- Configured bandwidth limits
- TCP competition
- Traffic shaping behavior
- Dynamic network state transitions
The experiment consists of multiple phases. Each phase changes one or more parameters:
Bandwidth Cap: 50 Mbps, 30 Mbps, 20 Mbps Active SCP Flows: 0, 1, 2, 3 Flows
Each combination creates a unique operating condition, for example:
| Cap (Mbps) | SCP Flows |
|---|---|
| 50 | 0 |
| 50 | 1 |
| 50 | 2 |
| 30 | 0 |
| 30 | 2 |
| 20 | 1 |
This creates a diverse dataset covering both uncongested and congested conditions.
The machine learning model uses features that directly influence achievable throughput:
| Feature | Description |
|---|---|
| Bandwidth Cap | HTB configured rate |
| SCP Flows | Number of competing downloads |
| Historical Throughput | Previous measured throughput values |
These features capture both network capacity constraints and temporal throughput behavior.
Before training, all features are normalized using Min-Max Scaling.
x_scaled = (x - xmin) / (xmax - xmin)
Benefits:
- Faster convergence
- Stable gradients
- Improved training behavior
The fitted scaler is saved and reused during inference.
Network throughput is inherently a time-series problem. Current throughput depends on:
- Previous throughput
- TCP congestion state
- Competing traffic
- Available bandwidth
To capture these temporal relationships, a Gated Recurrent Unit (GRU) network is used.
Figure 6: Host-side GRU prediction engine architecture showing telemetry collection, feature preprocessing, model inference, and dashboard integration.
Compared to traditional feedforward networks, GRUs maintain memory of previous observations.
Advantages:
- Learns sequential behavior
- Fewer parameters than LSTM
- Faster training
- Lower inference latency
- Suitable for short prediction horizons
This makes GRUs well suited for real-time throughput forecasting.
The prediction engine uses a stacked GRU architecture:
Input Window
|
v
GRU Layer (16 Units)
|
v
Dropout
|
v
GRU Layer (8 Units)
|
v
Dropout
|
v
Dense Layer
|
v
Output Throughput
The model predicts the next throughput measurement based on recent network history.
Instead of relying on a single model, the system uses an ensemble of GRU networks. Multiple models are trained using different random initialization seeds.
During inference:
Prediction 1
Prediction 2
Prediction 3
|
v
Average
|
v
Final Prediction
Benefits:
- Improved stability
- Reduced variance
- Better generalization
- More robust predictions
The prediction engine executes continuously throughout the experiment. Every few seconds it will:
- Fetch latest telemetry from
/metrics - Build feature window
- Apply Min-Max scaling
- Run GRU inference
- Average ensemble predictions
- Reverse normalization
- POST prediction to dashboard
The prediction then appears alongside actual throughput measurements.
The experiments demonstrate the effect of TCP flow competition on available throughput. Typical observations include:
| Scenario | Stream Throughput |
|---|---|
| 50 Mbps, No SCP | ~50 Mbps |
| 50 Mbps, 1 SCP | ~25 Mbps |
| 50 Mbps, 2 SCP | ~16 Mbps |
| 50 Mbps, 3 SCP | ~12 Mbps |
| 30 Mbps, No SCP | ~30 Mbps |
| 20 Mbps, No SCP | ~20 Mbps |
As competing SCP flows increase, available bandwidth is shared among active TCP connections, causing throughput degradation for the streaming application.
Figure 8: Comparison of measured throughput and GRU-predicted throughput across multiple bandwidth caps and congestion scenarios during the fifteen-phase experiment.
This project demonstrates practical experience in:
Networking
- TCP/IP
- Routing
- NAT
- Linux Networking
- Traffic Engineering
- QoS Mechanisms
Linux Systems
- Docker
- Container Networking
- Shell Scripting
- Process Management
- Traffic Control (
tc)
Backend Development
- Python
- Flask
- REST APIs
- Server-Sent Events
Data Engineering
- Telemetry Collection
- Dataset Generation
- Data Processing Pipelines
Machine Learning
- Time-Series Forecasting
- GRU Networks
- Model Deployment
- Real-Time Inference
Potential extensions include:
- Adaptive Bitrate Streaming
- Reinforcement Learning-Based Traffic Control
- Delay and Packet Loss Modeling
- Multi-Hop Network Topologies
- Kubernetes Deployment
- SDN Integration
- Real Network Interface Support
- Online Model Retraining
This project presents a complete end-to-end platform for network performance analysis and throughput prediction. By combining Linux traffic shaping, Docker-based network emulation, real-time telemetry collection, and machine learning-based forecasting, the system provides a realistic environment for studying network behavior under congestion and predicting future throughput conditions.
The project bridges concepts from networking, systems engineering, backend development, and machine learning, demonstrating how modern intelligent network management systems can be designed and implemented using open-source technologies.