Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

346 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AWS Cloud Engineering Lab

Practical project developed for the study, evolution, and demonstration of knowledge in:

AWS Terraform Docker GitHub Actions CloudWatch Status

  • AWS EC2
  • AWS SSM
  • AWS IAM
  • AWS VPC
  • AWS Security Groups
  • AWS Systems Manager
  • AWS RDS
  • AWS ECR
  • AWS ECS
  • AWS CloudWatch
  • Containers
  • Docker
  • Docker Compose
  • Terraform

Objective: To progressively build a cloud architecture similar to production environments focused on high performance.

Cost Optimization Note

Throughout this repository, the same EC2 instance is intentionally reused across multiple labs.

This approach reflects real-world engineering practices by:

  • reducing infrastructure costs;
  • minimizing unnecessary resource creation;
  • encouraging infrastructure reuse;
  • applying FinOps principles during learning.

After completing the last EC2-based lab, all AWS resources are removed following the Cleanup section.


Repository Structure

The repository is organized into two complementary learning tracks:

  1. Manual Labs — infrastructure created and validated through the AWS Management Console and supporting tools.
  2. Terraform Labs — equivalent infrastructure reproduced using Infrastructure as Code.

This structure allows each architecture to be studied manually first and then automated with Terraform while preserving the same learning objectives.

aws-cloud-engineering-lab/
│
├── README.md
│
├── docs/
│   ├── architecture/
│   ├── diagrams/
│   ├── screenshots/
│   ├── lessons-learned/
│   └── lab-notes/
│
├── labs/
│   ├── 01-ec2-and-ssm/
│   ├── 02-ec2-monitoring-with-cloudwatch-agent/
│   ├── 03-iam-roles-and-policies/
│   ├── 04-ebs-volumes-and-snapshots/
│   ├── 05-vpc-with-public-subnet/
│   ├── 06-vpc-with-private-subnet-and-nat-gateway/
│   ├── 07-docker-installation-on-ec2/
│   ├── 08-docker-compose-application/
│   ├── 09-nginx-reverse-proxy/
│   ├── 10-amazon-rds-postgresql/
│   ├── 11-amazon-s3-static-website-hosting/
│   ├── 12-aws-backup-strategy/
│   ├── 13-cloudwatch-logs/
│   ├── 14-cloudwatch-metrics/
│   ├── 15-cloudwatch-alarms/
│   ├── 16-sns-notifications/
│   ├── 17-amazon-ecr/
│   ├── 18-amazon-ecs-fargate/
│   ├── 19-ecs-service-management/
│   ├── 20-application-load-balancer/
│   ├── 21-terraform-ec2-deployment/
│   ├── 22-terraform-vpc-deployment/
│   ├── 23-terraform-ecs-deployment/
│   ├── 24-github-actions-build-pipeline/
│   ├── 25-github-actions-deployment-pipeline/
│   └── 26-end-to-end-ci-cd-pipeline/
│
├── terraform/
│   ├── README.md
│   │
│   ├── 01-ec2-and-ssm/
│   │   ├── images/
│   │   ├── .gitignore
│   │   ├── README.md
│   │   ├── compute.tf
│   │   ├── data.tf
│   │   ├── iam.tf
│   │   ├── locals.tf
│   │   ├── network.tf
│   │   ├── outputs.tf
│   │   ├── provider.tf
│   │   ├── terraform.tfvars.example
│   │   ├── variables.tf
│   │   └── versions.tf
│   │
│   └── future-labs/
│       └── Terraform implementations will be added progressively
│           as their corresponding manual laboratories are completed.
│
└── scripts/
    └── Supporting automation and validation scripts

Learning workflow

Each laboratory follows a progressive engineering workflow:

Manual provisioning
        ↓
Architecture validation
        ↓
Terraform reproduction
        ↓
Infrastructure comparison
        ↓
Automated cleanup
        ↓
Post-destroy verification
        ↓
Lessons learned

Relationship between implementations

Manual and Terraform laboratories use matching directory names:

labs/01-ec2-and-ssm/
        ↕
terraform/01-ec2-and-ssm/

The manual implementation explains how the AWS resources are created and validated step by step.

The Terraform implementation reproduces the same architecture through Infrastructure as Code and includes:

  • provider and version requirements;
  • input variables;
  • data-source queries;
  • resource provisioning;
  • outputs;
  • validation commands;
  • cost considerations;
  • automated destruction;
  • post-destroy verification.

Important

Terraform implementations are added progressively after their corresponding manual laboratories are completed and validated.

The Terraform infrastructure may be created temporarily alongside the manual infrastructure, using distinct names and tags, and then removed through an automated cleanup process.


Roadmap

AWS Cloud Engineering Lab - Roadmap

🎯 Objective

This repository documents my hands-on journey through AWS Cloud Engineering, Infrastructure, Observability, Automation, Containers, and DevOps practices.

The goal is to build practical experience with real-world cloud environments while developing skills applicable to Cloud Engineering, DevOps, Platform Engineering, Site Reliability Engineering (SRE), and Infrastructure Operations roles.


Phase 1 - AWS Foundations

Lab 01 - EC2 Instance with SSM Access

  • Configure IAM Role

Lab01

  • Create an EC2 instance

Lab01

  • Configure AWS Systems Manager

Lab01

  • Connect using Session Manager

Lab01

Lab with all the steps:

Manual Lab: labs/01-ec2-and-ssm

Terraform Lab:

Lab 02 - Modern EC2 Monitoring with CloudWatch Agent

  • Start existing EC2

Lab02

  • Install CloudWatch Agent

Lab02

  • Configure CloudWatch Agent

Lab02

  • Start Agent

Lab02

  • Verify Metrics

Lab02

  • Create Dashboard

Lab02

Lab with all the steps:

Manual Lab: labs/02-ec2-monitoring-with-cloudwatch-agent

Terraform Lab:

Lab 03 - IAM Roles and Policies

  • Create IAM Users
  • Create IAM Groups
  • Create Custom Policies
  • Apply Least Privilege Principle

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 04 - EBS Volumes and Snapshots

  • Create EBS Volume
  • Attach Volume
  • Create Snapshot
  • Restore Snapshot

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 05 - VPC with Public Subnet

  • Create VPC
  • Create Public Subnet
  • Configure Internet Gateway
  • Configure Route Tables

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 06 - VPC with Private Subnet and NAT Gateway

  • Create Private Subnet
  • Configure NAT Gateway
  • Validate Internet Access
  • Test Secure Architecture

Lab with all the steps:

Manual Lab:

Terraform Lab:


Phase 2 - Containers

Lab 07 - Docker Installation on EC2

  • Install Docker
  • Run First Container
  • Container Lifecycle Management
  • Security Best Practices

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 08 - Docker Compose Application

  • Deploy Multi-Container Application
  • Configure Networks
  • Configure Volumes
  • Persist Data

Lab 09 - NGINX Reverse Proxy

  • Install NGINX
  • Configure Reverse Proxy
  • Test HTTP Routing
  • Validate Application Access

Lab with all the steps:

Manual Lab:

Terraform Lab:


Phase 3 - Databases and Storage

Lab 10 - Amazon RDS PostgreSQL

  • Deploy PostgreSQL
  • Configure Security Groups
  • Connect from EC2
  • Execute SQL Queries

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 11 - Amazon S3 Static Website Hosting

  • Create Bucket
  • Enable Static Hosting
  • Upload Website Files
  • Configure Public Access

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 12 - AWS Backup Strategy

  • Define Backup Policy
  • Backup EBS
  • Backup RDS
  • Test Recovery

Lab with all the steps:

Manual Lab:

Terraform Lab:


Phase 4 - Observability

Lab 13 - CloudWatch Logs

  • Create Log Groups
  • Collect Application Logs
  • Query Logs
  • Analyze Events

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 14 - CloudWatch Metrics

  • Analyze Metrics
  • Create Dashboards
  • Monitor Resources
  • Configure Custom Metrics

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 15 - CloudWatch Alarms

  • Create CPU Alarm
  • Create Memory Alarm
  • Configure Notifications
  • Test Alerts

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 16 - SNS Notifications

  • Create SNS Topic
  • Configure Email Notifications
  • Integrate with CloudWatch
  • Validate Alert Delivery

Lab with all the steps:

Manual Lab:

Terraform Lab:


Phase 5 - Container Services

Lab 17 - Amazon ECR

  • Create Repository
  • Build Docker Image
  • Push Image
  • Pull Image

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 18 - Amazon ECS Fargate

  • Create ECS Cluster
  • Configure Fargate
  • Deploy Container
  • Validate Service

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 19 - ECS Service Management

  • Create ECS Service
  • Configure Desired Count
  • Test Availability
  • Review Logs

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 20 - Application Load Balancer

  • Create Load Balancer
  • Configure Target Group
  • Route Traffic
  • Validate Health Checks

Lab with all the steps:

Manual Lab:

Terraform Lab:


Phase 6 - Infrastructure as Code

Lab 21 - Terraform EC2 Deployment

  • Install Terraform
  • Create EC2 Infrastructure
  • Apply Configuration
  • Destroy Resources

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 22 - Terraform VPC Deployment

  • Create VPC Code
  • Deploy Networking
  • Validate Infrastructure
  • Refactor Modules

Lab 23 - Terraform ECS Deployment

  • Create ECS Resources
  • Deploy Containers
  • Validate Automation
  • Reuse Modules

Lab with all the steps:

Manual Lab:

Terraform Lab:


Phase 7 - DevOps and CI/CD

Lab 24 - GitHub Actions Build Pipeline

  • Create Workflow
  • Build Application
  • Validate Pipeline
  • Review Logs

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 25 - GitHub Actions Deployment Pipeline

  • Configure AWS Credentials
  • Deploy Infrastructure
  • Validate Deployment
  • Automate Releases

Lab with all the steps:

Manual Lab:

Terraform Lab:

Lab 26 - End-to-End CI/CD Pipeline

  • Build Application
  • Run Tests
  • Publish Artifact
  • Deploy Automatically
  • Validate Production Workflow

Lab with all the steps:

Manual Lab:

Terraform Lab:


Progress

Completed Manual Labs: 02 / 26

Current Lab:

  • 🚧 Manual: Lab 03 - IAM Roles and Policies

AND:

Completed Terraform Labs: 00 / 26

  • 🚧 Terraform: Lab 01 - EC2 Instance with SSM Access

About

A place where we will document, test, measure costs, improve the architecture, and automate a cloud environment on Amazon Web Services - AWS.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages