diff --git a/Common Technologies/DynamoDB/DynamoDB.md b/Common Technologies/DynamoDB/DynamoDB.md
new file mode 100644
index 0000000..b05ee11
--- /dev/null
+++ b/Common Technologies/DynamoDB/DynamoDB.md	
@@ -0,0 +1,588 @@
+# Amazon DynamoDB: How It Handles Millions of Requests Without Slowing Down
+
+
+# **The Real Problem**
+
+During large shopping events like **Amazon Prime Day**, traffic reaches an extreme scale. Millions of users simultaneously **browse products, add items to carts, update carts, and place orders**.
+
+Each of these actions generates **database reads and writes that must be processed instantly**.
+
+At this scale, traditional databases often struggle — leading to **slow responses, overloaded systems, or even outages**.
+
+Amazon, however, is built to handle this.
+
+During the **66-hour Prime Day sale in 2021**, DynamoDB processed **trillions of requests**, with peak traffic reaching around **89.2 million requests per second**, all while maintaining **low latency and stable performance**.
+
+<img src="./Resources/db_overload.gif" width="200" alt="Database Overload">
+
+This highlights a critical requirement:
+
+> **Customer actions must never be lost.**
+> 
+
+A failed cart update directly impacts both **user experience and revenue**.
+
+So the system must:
+
+- handle **millions of writes per second**
+- respond in **single-digit milliseconds**
+- remain **available even during failures**
+
+Traditional databases were not designed to satisfy all three simultaneously.
+
+This challenge led Amazon engineers to build **Dynamo**, which later evolved into **DynamoDB**.
+
+## The Engineering Approach to Handle Massive Throughput
+
+One way to scale systems is **vertical scaling** — upgrading a single machine with more **CPU, RAM, or faster storage**.
+
+This works initially, but has clear limits:
+
+- hardware becomes **expensive**
+- machines hit **physical limits**
+- creates a **single point of failure**
+
+If that machine fails, the entire system goes down.
+
+---
+
+Another approach is **horizontal scaling** — distributing data across **multiple machines** and scaling by adding more nodes.
+
+However, traditional relational databases struggle here because they require:
+
+- **coordination across nodes**
+- **joins across distributed data**
+- **strict consistency guarantees**
+
+This coordination introduces **latency and bottlenecks**, making it difficult to handle massive throughput efficiently.
+
+
+### **So the real issue here is not storing massive amounts of data. The issue is handling millions of read and write requests every second without slowing down.**
+
+This is where **DynamoDB** comes in.
+
+
+## Goal of DynamoDB Design
+
+DynamoDB was designed with **one primary goal**:
+
+> **provide consistent, low single-digit millisecond latency at any scale.**
+
+To achieve this, DynamoDB focuses on several design principles:
+
+- **Elastic scalability** – tables can grow to billions or trillions of items
+- **High availability** – the system continues operating despite failures
+- **Predictable performance** – latency remains stable under heavy load
+- **Flexible data model** – schema-less key-value storage
+
+---
+# ARCHITECTURE
+
+## DynamoDB Data Model
+
+A DynamoDB table stores data as **items**, where each item contains a set of **attributes (key-value pairs)**.
+
+Each item is uniquely identified using a **primary key**, defined when the table is created. The primary key consists of:
+
+- **Partition Key (required)**
+- **Sort Key (optional)**
+
+The partition key determines *which node* stores the item. The sort key organizes related items *within* that partition.
+
+---
+
+## Example
+
+Consider a table storing package tracking events:
+
+<img src="./Resources/table.png" width="700" alt="Table">
+
+`PackageID` is the partition key — all three events for PK102 live on the **same node**.
+
+`EventTime` is the sort key — those three events are stored in **chronological order** within that node.
+
+Together, `PackageID + EventTime` is globally unique. PK102 at 09:05 is one item. PK102 at 12:30 is a different item. They can never collide.
+
+Internally, DynamoDB organizes it like this:
+
+<img src="./Resources/tables.png" width="500" alt="Table">
+
+
+---
+
+## Primary Index
+
+The table's primary index is built using the primary key:
+
+```
+PackageID → EventTime
+```
+
+So when a query comes in:
+
+> **"Give me the full journey of package PK102."**
+> 
+
+DynamoDB:
+
+1. hashes `PK102`
+2. directly locates the partition
+3. returns all items sorted by `EventTime`
+
+No scanning required
+
+---
+
+But now someone asks:
+
+> **"Give me all packages that were picked up from Delhi."**
+> 
+
+`City` is not the partition key. DynamoDB has no idea which partitions contain Delhi data. It would have to open PK102, check city. Open PK889, check city. Open **every package in the table — one by one**. With millions of packages, this is a disaster.
+
+This is exactly what **Secondary Indexes** solve.
+
+---
+
+## Secondary Index
+
+You create a **GSI** with `City` as the new partition key:
+
+<img src="./Resources/GSI.png" width="700" alt="GSI">
+
+
+Notice: only rows that **have a City value** appear in the GSI. The InTransit and Delivered rows for PK102 have no city, so they are not indexed.
+
+The Delhi query now becomes:
+
+```
+Step 1: GSI lookup  →  City = "Delhi"  →  PK102 (09:05)
+Step 2: Base table  →  fetch PK102     →  full item returned
+```
+
+Without GSI:  O(n)  — scan every item in the table
+With GSI:     O(1)  — direct lookup by City, same as any primary key query
+
+## **Local Secondary Index (LSI)**
+
+An **LSI** keeps the same partition key but changes the sort key.
+
+Lets you query the same partition in a different order.
+
+> "Give me all events for PK102, sorted by **Status** instead of EventTime."
+> 
+
+You can't do that with the base table — EventTime is the only sort key. An LSI on `PackageID → Status` solves it instantly, and since the partition key is the same, the data is already co-located on the same node.
+
+One constraint: **LSI must be created at table creation time.** You can't add it later. GSI can be added anytime.
+
+---
+
+# PARTITIONING
+
+## How DynamoDB Distributes Data Across Nodes
+
+Now that we understand the data model, the real question is:
+
+**How does DynamoDB decide which machine stores which item?**
+
+With billions of items across thousands of servers, DynamoDB needs a method that:
+
+- distributes data **evenly** across all nodes
+- causes **minimal disruption** when a node is added or removed
+- routes any request to the **correct node quickly**
+
+## The Naive Approach — Modulo Hashing
+
+The simplest idea is modulo hashing:
+
+`node = hash(partitionKey) % numberOfNodes`
+
+Assign each node a fixed slot. Hash the key, take the remainder, done.
+
+| Key | hash(key) % 4 | Goes to |
+| --- | --- | --- |
+| PK102 | 2 | N-2 |
+| PK889 | 0 | N-0 |
+| PK415 | 3 | N-3 |
+| PK100 | 1 | N-1 |
+
+This works perfectly — until you add a node.
+
+| Key | Before (% 4) | After (% 5) |
+| --- | --- | --- |
+| PK102 | N-2 | N-1 ← moved |
+| PK889 | N-0 | N-4 ← moved |
+| PK415 | N-3 | N-0 ← moved |
+| PK100 | N-1 | N-0 ← moved |
+
+Almost **every key remaps to a different node**. At Amazon's scale — billions of items — this means moving a catastrophic amount of data every time capacity changes.
+
+---
+
+## Consistent Hashing
+
+DynamoDB solves this with **consistent hashing**.
+
+The idea is elegant: imagine the entire hash output space (say, 0 to 2³² − 1) arranged as a **ring** (circle). Each node is placed at a position on this ring by hashing its identifier.
+
+<img src="./Resources/consistent.png" width="900" alt="Consistent Hashing">
+
+When a **write or read** comes in for a key:
+
+1. Hash the partition key → get a position on the ring
+2. Walk **clockwise** until you hit the first node
+3. That node is responsible for storing and serving that key
+
+**What happens when a new node is added?**
+
+Only the keys that fall between the new node and its predecessor need to move. **All other keys stay exactly where they are.** This is the core insight that makes consistent hashing powerful.
+
+---
+
+## Virtual Nodes (Vnodes)
+
+Basic consistent hashing has one remaining problem.
+
+With only **3 servers on the ring**, each server owns one large chunk — roughly ⅓ of the ring each. That sounds fair, but in practice nodes are placed by hashing their ID, so placement is random. 
+
+In the below image a)Without Virtual Nodes, all of Key 11–60 dumps onto Server 2 (if sever 1 fails)
+
+### How Vnodes Fix This
+
+Instead of one position per server, each physical server is hashed **multiple times** — giving it multiple positions on the ring.
+
+Each position is a **virtual node**. It points back to the same physical server, but it appears at a different spot on the ring.
+
+<img src="./Resources/virtual.png" width="500" alt="Virtual Nodes">
+
+The colors now **alternate evenly** around the entire ring — no server dominates any side. Each physical server owns **3 small scattered ranges** instead of 1 large chunk, and the total adds up to the same ⅓ each.
+
+This matters because:
+
+- **Node failure** — if Server 2 goes down, its 3 small ranges scatter to 3 different neighbors instead of dumping everything onto one node
+- **New node added** — it steals small slices from many servers instead of one big slice from one server
+- **Heterogeneous machines** — a stronger server simply gets more vnodes assigned, naturally receiving more traffic
+
+**Effect:**
+
+| Property | Basic Hashing | With Vnodes |
+| --- | --- | --- |
+| Load balance | Uneven | Even across all nodes |
+| Node added/removed | Many keys move | Only neighboring ranges affected |
+| Node failure impact | One large range lost | Many small ranges redistributed |
+| Heterogeneous nodes | Can't weight by capacity | Assign more vnodes to stronger machines |
+
+With enough vnodes, load naturally distributes evenly across all physical machines.
+
+---
+
+# REPLICATION
+
+## Why Replicate?
+
+A single node storing your data is a **single point of failure**. If that server crashes, your data becomes unavailable.
+
+DynamoDB replicates every partition **three times** — across three separate **Availability Zones (AZs)** within a region. AZs are physically isolated data centers with independent power, networking, and cooling.
+
+**Replication Factor (RF)** is the number of copies DynamoDB maintains for each partition. DynamoDB hardcodes RF = 3 — one copy per Availability Zone
+
+<img src="./Resources/AZ.png" width="500" alt="Table">
+
+
+One replica is the **leader** (sometimes called primary). The other two are **followers** (replicas).
+
+- **Writes** always go to the leader first
+- **Reads** can go to any replica (with tradeoffs — more on this below)
+
+If the leader fails, **consensus** (using a protocol similar to Multi-Paxos) is used among the replicas to elect a new leader automatically. No human intervention required.
+
+---
+
+## How a Read Actually Happens
+
+## Read Consistency
+
+DynamoDB supports both strongly consistent and eventually consistent reads — where **eventual is the default**.
+
+The **leader replica** does two things:
+
+1. Serves all writes
+2. Serves strongly consistent reads — because it handles all writes, it always has the latest data
+
+That's the entire reason strong consistency works.
+
+**Strong → read goes to the leader replica**
+
+**Eventual → read goes to any replica**
+
+|  | Strong | Eventual |
+| --- | --- | --- |
+| Routes to | Leader only | Any replica |
+| Stale data? | Never | Possibly |
+| Cost | 1× RCU / 4KB | 0.5× RCU / 4KB |
+| Use case | Cart total, inventory, payments | Feed, dashboard, analytics |
+
+## How a Write Actually Happens
+
+Let's trace a write request step by step.
+
+**Scenario:** Package PK102 just got delivered. The system writes the delivery event.
+
+<img src="./Resources/quorum.png" width="800" alt="Quorum">
+
+This is called a **quorum write** (W = 2 out of 3).
+
+**Why quorum and not wait for all 3?**
+
+Waiting for all three would mean the **slowest replica** determines your write latency. If one node is slightly degraded, every write gets slower.
+
+With W=2, writes succeed as long as any **two** replicas are healthy, which provides:
+
+- **Durability**: data is on multiple machines before acknowledging success
+- **Performance**: not bottlenecked by the slowest node
+- **Availability**: one replica can be down without affecting write
+
+Two important things happening here:
+
+**WAL write is synchronous** — quorum must confirm before client gets success. Data is safe even if every node crashes immediately after.
+
+**B-Tree update is asynchronous** — the actual table update happens in the background. Client doesn't wait for it. That's why writes are fast.
+
+---
+
+# HANDLING FAILURES
+
+## What Happens When a Node Goes Down?
+
+DynamoDB is built to handle failures gracefully. Two key mechanisms make this work.
+
+**What if the Dynamo cluster cannot reach quorum? Should the write be rejected?**
+
+In traditional quorum systems, the answer is yes — writes are rejected to preserve durability.
+But DynamoDB takes a different approach using **Sloppy Quorum:**
+
+Even if the required quorum nodes are unavailable,
+the system accepts the write anyway
+and temporarily stores it on any available healthy nodes
+
+This prioritizes availability over strict durability, ensuring the system continues to accept writes even during failures.
+
+---
+
+### Hinted Handoff
+
+Imagine a write comes in for a key, but the target replica (say, Node B) is temporarily down.
+
+Instead of rejecting the write, DynamoDB:
+
+- sends it to another healthy node
+- stores it with a **“hint”**
+- later forwards it to the correct node when it recovers
+
+```
+Normal:  Write → Node A (Leader) → Node B, Node C
+
+Node B is down:
+         Write → Node A (Leader) → Node C ✅
+                                 → Node D 🔁 (hint: "this belongs to Node B")
+
+Node B recovers:
+         Node D → transfers the hinted writes back → Node B ✅
+```
+
+This is actually a direct consequence of **Sloppy Quorum in action** — writes are routed to any available nodes, not strictly the original replica set.
+
+This strategy ensures **writes are never rejected due to a single node failure.** The system remains highly available.
+
+**Limitation:** If Node D also crashes before handing off, those writes could be lost. Hinted handoff only works for **short, transient failures.**
+
+---
+
+### Anti-Entropy (Merkle Trees)
+
+For longer failures, DynamoDB uses a background sync process.
+
+Instead of comparing entire datasets (which is expensive), it uses **Merkle trees** — a structure that lets systems quickly detect differences.
+
+To compare two replicas efficiently without sending all the data, DynamoDB uses **Merkle trees** — a data structure where:
+
+- leaf nodes represent hashes of individual data items
+- parent nodes represent hashes of their children
+- the **root hash** represents the entire dataset
+
+<img src="./Resources/hash.png" width="500" alt="Merkle Tree">
+
+If two replicas have the same root hash, they are **identical** — no sync needed.
+
+If the root hashes differ, the tree can be traversed to find exactly **which keys differ** — without comparing all data. Only the divergent keys are synchronized.
+
+This makes replica synchronization efficient even at massive scale.
+
+---
+
+# THE HOT PARTITION PROBLEM
+
+## The Most Common DynamoDB Mistake
+
+Understanding consistent hashing and replication is great. But there is one problem that trips up many engineers when using DynamoDB in production.
+
+Consider an e-commerce application during an iPhone launch. Every user hitting the product page generates a read or write:
+
+| Partition Key | Sort Key | Views |
+| --- | --- | --- |
+| product#iphone-15 | 2024-01-01 | 4,200,000 |
+| product#iphone-15 | 2024-01-02 | 3,800,000 |
+| product#iphone-15 | 2024-01-03 | 5,100,000 |
+| product#samsung-s24 | 2024-01-01 | 800,000 |
+| product#pixel-8 | 2024-01-01 | 620,000 |
+
+DynamoDB hashes each partition key and distributes it across nodes:
+
+```
+hash("product#iphone-15")   → Node A
+hash("product#samsung-s24") → Node B
+hash("product#pixel-8")     → Node C
+```
+
+Perfectly spread — each product lives on a different node. This is exactly how DynamoDB is supposed to work.
+
+---
+
+But now it's iPhone launch day. Samsung and Pixel pages still get normal traffic. The iPhone page gets **millions of simultaneous hits**:
+
+```
+DynamoDB Cluster
+
+Node A ██████████████████ ← 90% of traffic (iPhone!)
+Node B ██                ← Samsung (normal)
+Node C ███               ← Pixel (normal)
+Node D █                 ← everything else
+```
+
+Node A is overloaded. Requests slow down or get throttled. **This is a hot partition.**
+
+The problem is not that data is distributed wrong — it is. The problem is that **one key receives disproportionate traffic** compared to everything else.
+
+---
+
+## Why This Happens
+
+> **Data is evenly distributed — traffic is not.**
+> 
+
+DynamoDB distributes data based on the partition key hash. If your partition key has **low cardinality** (few unique values) or **skewed access patterns** (one item is far more popular), all traffic lands on one node — regardless of how many machines you have.
+
+Adding more capacity won't help if only one partition receives all the traffic.
+
+---
+
+## Why DynamoDB Can't Just Split the Hot Partition
+
+DynamoDB can split Node A's partition into two:
+
+```
+Node A splits into A1 + A2
+
+A1                          A2
+└ product#iphone-15         ├ product#samsung-s24
+                            └ product#pixel-8
+```
+
+But `product#iphone-15` still lives in A1. Every iPhone request still goes there. Samsung and Pixel just moved to A2 — which already had light traffic anyway.
+
+You now have two partitions and the exact same hot key. Nothing changed.
+
+**Splitting redistributes keys. It cannot redistribute traffic for a single key.**
+
+That's why the fix has to come from your data model, not from DynamoDB's infrastructure.
+
+---
+
+## How to Fix It
+
+### Strategy 1: Write Sharding (Random Suffix)
+
+Instead of one key `product#iphone-15`, spread it across 10 shards:
+
+```
+product#iphone-15#0
+product#iphone-15#1
+product#iphone-15#2
+  ...
+product#iphone-15#9
+```
+
+Each shard hashes to a different node. Writes are distributed randomly across all 10. Reads must query all 10 shards and aggregate the result.
+
+**Good for:** High-write scenarios where one key is the bottleneck.
+
+---
+
+### Strategy 2: Time-Based Partition Keys
+
+Include a time component in the partition key:
+
+```
+Before: product#iphone-15
+After:  product#iphone-15#2024-01-03
+```
+
+Each day's traffic lands on a different partition. Yesterday's data naturally cools off and stops receiving writes.
+
+**Good for:** Time-series data like views, events, or logs.
+
+---
+
+### Strategy 3: Caching with DAX
+
+For read-heavy hot keys, put DAX (DynamoDB Accelerator) in front of DynamoDB:
+
+```
+Client → DAX (in-memory cache) → DynamoDB (only on cache miss)
+```
+
+Most requests for `product#iphone-15` are reads — the same product page served millions of times. DAX returns cached results in **microseconds**. DynamoDB barely sees the traffic.
+
+**Good for:** Read-heavy hot keys where the data doesn't change constantly.
+
+---
+
+## Summary
+
+| Strategy | Best For | Tradeoff |
+| --- | --- | --- |
+| Write sharding | Hot write keys | Read aggregation complexity |
+| Time-based keys | Time-series data | Natural access pattern shift |
+| DAX caching | Hot read keys | Cache invalidation, extra cost |
+
+**The best fix is always upfront design.** Choosing the right partition key is the most impactful decision you make when using DynamoDB. Get it wrong and no amount of infrastructure can save you.
+
+---
+
+
+# WHEN TO USE DYNAMODB
+
+## DynamoDB is Excellent For
+
+- **High-throughput, low-latency workloads** — gaming leaderboards, real-time bidding, session management
+- **Unpredictable or bursty traffic** — you need elastic scaling without pre-provisioning
+- **Key-value and simple query patterns** — you know your access patterns upfront
+- **Serverless architectures** — pairs naturally with AWS Lambda, no connection pooling needed
+- **Globally distributed applications** — Global Tables let you replicate across regions with local read/write
+
+---
+
+
+## The Cardinal Rule of DynamoDB
+
+> **Design your access patterns before you design your table.**
+> 
+
+In a relational database, you define a schema, and queries are figured out later. In DynamoDB, it's the **opposite**. You need to know exactly how the data will be read, then design partition keys, sort keys, and indexes around those specific patterns.
+
+Getting this right upfront means single-digit millisecond performance at any scale. Getting it wrong means redesigning your table — which is painful because DynamoDB does not support schema migrations in the traditional sense.
+
+---
+
+*Further reading: [Dynamo: Amazon's Highly Available Key-value Store (2007 SOSP Paper)](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf) — the original research paper that started it all.*
\ No newline at end of file
diff --git a/Common Technologies/DynamoDB/Resources/AZ.png b/Common Technologies/DynamoDB/Resources/AZ.png
new file mode 100644
index 0000000..0217647
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/AZ.png differ
diff --git a/Common Technologies/DynamoDB/Resources/GSI.png b/Common Technologies/DynamoDB/Resources/GSI.png
new file mode 100644
index 0000000..19036f3
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/GSI.png differ
diff --git a/Common Technologies/DynamoDB/Resources/consistent.png b/Common Technologies/DynamoDB/Resources/consistent.png
new file mode 100644
index 0000000..e68fa85
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/consistent.png differ
diff --git a/Common Technologies/DynamoDB/Resources/db_overload.gif b/Common Technologies/DynamoDB/Resources/db_overload.gif
new file mode 100644
index 0000000..ada1b23
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/db_overload.gif differ
diff --git a/Common Technologies/DynamoDB/Resources/hash.png b/Common Technologies/DynamoDB/Resources/hash.png
new file mode 100644
index 0000000..48448a1
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/hash.png differ
diff --git a/Common Technologies/DynamoDB/Resources/quorum.png b/Common Technologies/DynamoDB/Resources/quorum.png
new file mode 100644
index 0000000..b45cece
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/quorum.png differ
diff --git a/Common Technologies/DynamoDB/Resources/table.png b/Common Technologies/DynamoDB/Resources/table.png
new file mode 100644
index 0000000..27079e1
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/table.png differ
diff --git a/Common Technologies/DynamoDB/Resources/tables.png b/Common Technologies/DynamoDB/Resources/tables.png
new file mode 100644
index 0000000..c97ab9e
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/tables.png differ
diff --git a/Common Technologies/DynamoDB/Resources/virtual.png b/Common Technologies/DynamoDB/Resources/virtual.png
new file mode 100644
index 0000000..3b3889b
Binary files /dev/null and b/Common Technologies/DynamoDB/Resources/virtual.png differ
diff --git a/Common Technologies/DynamoDB/image.png b/Common Technologies/DynamoDB/image.png
new file mode 100644
index 0000000..b45cece
Binary files /dev/null and b/Common Technologies/DynamoDB/image.png differ
diff --git a/Common Technologies/Kafka/Kafka.md b/Common Technologies/Kafka/Kafka.md
new file mode 100644
index 0000000..32a9e7d
--- /dev/null
+++ b/Common Technologies/Kafka/Kafka.md	
@@ -0,0 +1,437 @@
+# Apache Kafka: Understanding Real-Time Data Streaming at Scale
+
+## The Real Problem
+
+Open the **Uber** app after booking a ride. You can see the driver moving on the map in real time. Every few seconds, the car's position changes slightly. It feels smooth and simple from the user's side.
+
+But behind that simple map, a large technical system is working continuously.
+
+<img src="./Resources/Uber_Gif.gif" width="200" alt="Uber driver tracking interface">
+
+Every few seconds, the driver's phone sends its current location to Uber's servers. This includes latitude and longitude values. Now think about how many drivers are active at the same time across different cities in the world. Each of them is sending updates again and again. This means Uber's backend is receiving millions of small updates every second.
+
+The system cannot afford to lose these updates. It also cannot slow down. The rider must see the movement instantly. Other services also depend on this data. Trip tracking, billing, analytics, and pricing systems all need the same location data. So one small update from a driver is actually useful to many different parts of the system.
+
+## The Engineering Approach to Handle This Scale
+
+A simple approach would be to write every driver location update directly into a database.
+
+Databases are excellent at **long-term storage**. They can hold billions of records and allow fast searching and querying. In other words, databases solve the **storage problem** very well.
+
+However, they are not built to handle extremely high numbers of small writes every second without limits. If millions of drivers continuously update their location directly into the database, it can become overloaded. Performance drops, latency increases, and the system slows down.
+
+### So the real issue here is not storage. The issue is **throughput**.
+
+This is where **Apache Kafka** fits in.
+
+Kafka is not an alternative to a database, and it is not competing with one. They solve different problems. Kafka is built to handle **massive streams of incoming data at very high speed**. It accepts and distributes events efficiently without becoming a bottleneck.
+
+The database, on the other hand, stores processed and structured data for long-term use and supports complex queries.
+
+They work together to make the system scalable, stable, and efficient.
+
+## How Kafka Solves the Problem
+
+In large systems, applications that generate data are called **producers**, and the systems that read and use that data are called **consumers**.
+
+<img src="./Resources/image.png" width="500" alt="Producers and Consumers">
+
+Producers send events to Kafka, and consumers read the events they need from Kafka.
+
+This creates **decoupling**. Producers do not need to know who is consuming the data. Services can scale, fail, or evolve independently.
+
+<img src="./Resources/image%20(1).png" width="500" alt="Decoupled architecture with Kafka">
+
+Kafka also acts as a **buffer**. If the database or any consumer becomes slow, Kafka temporarily stores the incoming data. Producers can continue sending updates without overwhelming downstream systems.
+
+One important question is: why can Kafka handle such high throughput?
+
+Kafka is optimized for fast, sequential disk writes instead of random writes. It treats data like an append-only log, which is much more efficient for disk operations. It also batches messages together before writing or sending them over the network. Because of this design, Kafka can handle massive streams of real-time data efficiently.
+
+Finally, consumers can read events in batches and perform **bulk inserts** into the database. This is far more efficient than inserting one record at a time and significantly reduces database load.
+
+Now that we understand what Kafka solves, let's see how it actually does it internally.
+
+# Understanding Kafka Architecture
+
+Let us go back to the simple picture we discussed earlier.
+
+Producers send data. Consumers read data. Kafka sits in the middle and manages the flow.
+
+---
+## Kafka Clusters, Brokers, and ZooKeeper vs KRaft
+
+## Kafka Clusters and Brokers
+
+But Kafka is not a single machine.
+
+It runs as a **cluster** of servers. Each server in this cluster is called a **broker**. A broker is simply one Kafka server that stores data and handles client requests. Each broker has a unique **broker ID** so it can be identified within the cluster.
+
+A Kafka cluster consists of multiple brokers working together. Each broker stores a portion of the overall data. By distributing data across brokers, Kafka can scale horizontally and handle large traffic loads.
+
+When a producer wants to send a message, it first connects to any broker in the cluster. That broker shares cluster metadata, including which broker is responsible for the required partition. In simple terms, the producer first **gets the broker information**, and then sends the message to the correct broker.
+
+Consumers work differently. They **pull messages** from Kafka. A consumer connects to a broker, fetches messages from its assigned partitions, processes them, and then **updates its offset**. Updating the offset means the consumer records how much data it has already read, so it can continue from the correct position next time.
+
+While brokers handle message storage and client communication, Kafka also needs a mechanism to coordinate the cluster, maintain metadata, and manage leader elections. Over time, Kafka has used two different approaches for this coordination: **ZooKeeper** (historical) and **KRaft** (modern).
+
+---
+
+## ZooKeeper (Historical Architecture)
+
+In early versions of Kafka, **Apache ZooKeeper** was used to coordinate the Kafka cluster.
+
+ZooKeeper acted as a central coordination service that stored cluster metadata and helped manage broker coordination. Kafka brokers communicated with ZooKeeper to keep track of the cluster state.
+
+ZooKeeper handled tasks such as:
+
+- Tracking active brokers in the cluster
+- Maintaining broker IDs
+- Leader election for partitions
+- Storing cluster metadata and configurations
+
+However, ZooKeeper did not store Kafka message data. Producers and consumers interacted only with Kafka brokers, while ZooKeeper worked in the background to coordinate the system.
+
+### ZooKeeper-based Kafka Architecture
+
+<img src="./Resources/image%20(2).png" width="500" alt="ZooKeeper Architecture">
+
+---
+
+## Problems with ZooKeeper
+
+Although ZooKeeper worked well initially, it introduced some limitations as Kafka clusters grew.
+
+### 1. Extra System to Manage
+
+Running Kafka required maintaining two distributed systems: Kafka brokers and a ZooKeeper cluster. This increased operational complexity.
+
+### 2. Metadata Stored Outside Kafka
+
+Cluster metadata such as broker registrations and partition leaders was stored in ZooKeeper instead of Kafka, creating additional coordination overhead.
+
+### 3. Scaling Challenges
+
+Large Kafka clusters with thousands of topics and partitions generated heavy metadata traffic, which ZooKeeper was not designed to handle efficiently.
+
+Because of these limitations, Kafka introduced **KRaft**, which manages cluster metadata directly inside Kafka.
+
+---
+
+## KRaft (Modern Kafka Architecture)
+
+Modern Kafka clusters use **KRaft** (Kafka Raft Metadata Mode) for cluster coordination.
+
+KRaft removes the dependency on ZooKeeper and integrates metadata management directly into Kafka using the Raft consensus protocol.
+
+Instead of storing metadata in ZooKeeper, Kafka now maintains a **metadata log** that is replicated across a set of controller nodes known as the **KRaft quorum**.
+
+These controllers manage:
+
+- Cluster metadata
+- Broker coordination
+- Leader election
+- Configuration changes
+
+Because metadata is now managed within Kafka itself, the architecture becomes simpler, more scalable, and easier to operate.
+
+### KRaft-based Kafka Architecture
+
+<img src="./Resources/kraft-architecture.png" width="500" alt="KRaft Architecture">
+
+---
+
+## ZooKeeper vs KRaft
+
+| Feature | ZooKeeper (Older Kafka) | KRaft (Modern Kafka) |
+| --- | --- | --- |
+| Coordination | External ZooKeeper cluster | Built into Kafka |
+| Metadata Storage | Stored in ZooKeeper | Stored in Kafka metadata log |
+| Architecture | Kafka + ZooKeeper clusters | Single Kafka cluster |
+| Operational Complexity | Higher | Lower |
+| Status | Removed in Kafka 4.0 | Current default |
+
+---
+
+## Topics
+
+Inside Kafka, data is organized into **topics**.
+
+You can think of a topic like a **folder** that holds all events related to a specific subject.
+
+Why do we need topics?
+
+Because not all data is the same. In the Uber example, we may have:
+
+For example, in the Uber system:
+
+- `driver-location-updates`
+- `ride-status-updates`
+- `payment-events`
+
+Each topic contains events of one type. This logical separation helps consumers subscribe only to the data they care about.
+
+<img src="./Resources/image%20(3).png" width="500" alt="Kafka topics">
+
+---
+
+## Partitions
+
+While a topic is a logical category, the actual storage happens inside **partitions**.
+
+Each topic is divided into one or more partitions. A partition is where events are physically stored.
+
+Why do we need partitions?
+
+If all events of a topic were stored in a single sequence, it would limit scalability. By dividing a topic into multiple partitions, Kafka allows data to be written and read in parallel.
+
+You can think of this like database partitioning. Instead of one large file, we split it into smaller pieces to improve performance.
+
+<img src="./Resources/image%20(4).png" width="500" alt="Topic partitioning">
+
+---
+
+## Ordering and Offsets
+
+Inside a partition, events are stored as an **append-only log file** on disk.
+
+This means:
+
+- New events are always added at the end.
+- Existing events are not modified.
+
+Events inside a partition are strictly ordered.
+
+An offset is:
+
+> The sequential ID of each message inside a partition.
+
+Offset 0 → Offset 1 → Offset 2 → Offset 3 …
+
+Offsets help consumers track where they are in the stream.
+
+<img src="./Resources/image%20(5).png" width="500" alt="Offset ordering">
+
+---
+
+## Segments
+
+A **segment** is just a physical file on disk.
+
+A partition is logically one long ordered log.
+
+But physically, Kafka breaks that log into multiple smaller files called **segments**.
+
+For example:
+
+- One segment may store events from offset 0 to 99
+- The next segment may start from offset 100
+
+This makes storage management efficient without affecting ordering.
+
+Each segment is a file like:
+
+```
+00000000000000000000.log
+00000000000000001000.log
+00000000000000002000.log
+```
+
+Kafka writes to one active segment.
+
+When it becomes full, Kafka closes it and creates a new one.
+
+<img src="./Resources/image%20(6).png" width="500" alt="Segments in Kafka">
+
+---
+
+## Consumers and Consumer Groups
+
+Consumers read messages from topics.
+
+To scale consumption, Kafka uses **consumer groups**.
+
+Within a single consumer group:
+
+- If a topic has two partitions and only one consumer, that consumer reads both partitions.
+
+<img src="./Resources/image%20(7).png" width="500" alt="One consumer reading multiple partitions">
+
+- If there are two consumers and two partitions, each consumer gets one partition.
+
+<img src="./Resources/image%20(8).png" width="500" alt="Two consumers with one partition each">
+
+- If there are more consumers than partitions, extra consumers remain idle.
+
+<img src="./Resources/image%20(9).png" width="500" alt="Extra consumers remaining idle">
+
+- If a consumer fails, Kafka automatically reassigns its partitions to another consumer in the group.
+
+<img src="./Resources/image%20(10).png" width="500" alt="Partition reassignment after consumer failure">
+
+This ensures scalability and fault tolerance.
+
+---
+
+### Why we don't use a message queue
+
+At first glance, Kafka looks similar to a traditional message queue, but there is an important difference.
+
+In most message queues, once a consumer reads a message, that message is removed. Other consumers cannot read it again.
+
+In Kafka, messages are **not deleted after being read**. They remain stored for a configured time, and consumers track their position using offsets.
+
+Because of this, multiple consumer groups can read the same data and apply different logic. Kafka also allows replaying old messages, which is not common in traditional message queues.
+
+In short, message queues focus on task distribution, while Kafka focuses on event streaming and persistence.
+
+# How Does Kafka Guarantee Durability and Fault Tolerance?
+
+We have seen how Kafka stores and distributes data.
+
+Now the real question is:
+
+What happens if something fails?
+
+Servers crash. Networks fail. Consumers restart.
+
+How does Kafka make sure data is not lost?
+
+Let's break it down step by step.
+
+---
+
+## Replication
+
+In Kafka, each partition does not live on just one broker.
+
+Instead, each partition can have **multiple copies**, called **replicas**.
+
+For every partition:
+
+- One replica is selected as the **leader**.
+- The other replicas are called **followers**.
+
+Producers send messages only to the leader replica.
+
+Consumers also read from the leader.
+
+Followers continuously copy data from the leader and stay in sync.
+
+If the broker hosting the leader crashes, Kafka automatically promotes one of the followers as the new leader.
+
+Because of replication:
+
+- Data is not lost if one broker fails.
+- The system continues running.
+- Availability is maintained.
+
+<img src="./Resources/image%20(11).png" width="500" alt="Replication in Kafka">
+
+---
+
+## In-Sync Replicas (ISR)
+
+We already discussed replication. Every partition has multiple replicas — one leader and the rest followers.
+
+But here's the important question:
+
+Are all follower replicas always fully up to date?
+
+Not necessarily.
+
+That's where **In-Sync Replicas (ISR)** comes in.
+
+ISR is simply the group of replicas that are fully caught up with the leader. This group always includes the leader itself and any followers that are actively syncing without significant lag.
+
+Kafka continuously monitors followers. If a follower falls too far behind — for example, due to network delay or system slowdown — Kafka temporarily removes it from the ISR.
+
+Let's imagine a partition with three replicas:
+
+- Broker 1 → Leader
+- Broker 2 → Follower (almost fully synced)
+- Broker 3 → Follower (lagging behind)
+
+In this case, the ISR would contain Broker 1 and Broker 2.
+
+Broker 3 would be excluded until it catches up.
+
+<img src="./Resources/image%20(12).png" width="500" alt="In-Sync Replicas (ISR)">
+
+
+Why is this important?
+
+Because **only replicas inside the ISR are allowed to become the new leader** if the current leader fails.
+
+This protects data consistency. If Kafka allowed an out-of-date replica to become leader, recent messages could be lost.
+
+There is also a connection to producer acknowledgments. When the producer uses `acks=all`, the write is considered successful only after all ISR replicas confirm the write. This ensures that committed data exists on multiple fully synchronized replicas.
+
+In simple terms:
+
+Replication gives you multiple copies.
+
+ISR ensures those copies are safe and up to date.
+
+That is what maintains reliability in Kafka's distributed environment.
+
+---
+
+## Acknowledgment Settings (acks)
+
+When a producer sends a message, it can decide how strict it wants to be.
+
+Kafka provides acknowledgment settings:
+
+- **acks = 0** → The producer does not wait for confirmation. Fast, but risky.
+- **acks = 1** → The leader confirms the write. Moderate safety.
+- **acks = all** → All ISR replicas confirm the write. Highest durability.
+
+The stronger the acknowledgment level, the safer the data — but slightly higher the latency.
+
+This allows applications to balance performance and reliability.
+
+---
+
+## Delivery Guarantees
+
+Kafka supports different delivery behaviors.
+
+**At most once**
+
+Messages may be lost, but never duplicated.
+
+**At least once**
+
+Messages are never lost, but may be processed more than once.
+
+**Exactly once**
+
+Messages are neither lost nor duplicated.
+
+Achieved using idempotent producers and transactions.
+
+This is extremely powerful for systems like payments or billing.
+
+<img src="./Resources/image%20(13).png" width="500" alt="Delivery guarantees in Kafka">
+
+---
+
+## Retention Policy
+
+Unlike traditional message queues, Kafka does not delete messages immediately after they are consumed.
+
+Messages remain stored based on a **retention policy**.
+
+Retention can be:
+
+- Time-based (for example, keep data for 7 days)
+- Size-based (delete old data after a certain storage limit)
+
+Kafka stores data in segment files. When data becomes older than the retention limit, entire segments are deleted.
+
+This design allows:
+
+- Message replay
+- Recovery from failures
+- New consumers to read historical data
diff --git a/Common Technologies/Kafka/Resources/Uber_Gif.gif b/Common Technologies/Kafka/Resources/Uber_Gif.gif
new file mode 100644
index 0000000..7c5bc5f
Binary files /dev/null and b/Common Technologies/Kafka/Resources/Uber_Gif.gif differ
diff --git a/Common Technologies/Kafka/Resources/image (1).png b/Common Technologies/Kafka/Resources/image (1).png
new file mode 100644
index 0000000..2536517
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (1).png differ
diff --git a/Common Technologies/Kafka/Resources/image (10).png b/Common Technologies/Kafka/Resources/image (10).png
new file mode 100644
index 0000000..31e49ab
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (10).png differ
diff --git a/Common Technologies/Kafka/Resources/image (11).png b/Common Technologies/Kafka/Resources/image (11).png
new file mode 100644
index 0000000..469b6f0
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (11).png differ
diff --git a/Common Technologies/Kafka/Resources/image (12).png b/Common Technologies/Kafka/Resources/image (12).png
new file mode 100644
index 0000000..ec8b17e
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (12).png differ
diff --git a/Common Technologies/Kafka/Resources/image (13).png b/Common Technologies/Kafka/Resources/image (13).png
new file mode 100644
index 0000000..086516c
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (13).png differ
diff --git a/Common Technologies/Kafka/Resources/image (2).png b/Common Technologies/Kafka/Resources/image (2).png
new file mode 100644
index 0000000..a3e10fb
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (2).png differ
diff --git a/Common Technologies/Kafka/Resources/image (3).png b/Common Technologies/Kafka/Resources/image (3).png
new file mode 100644
index 0000000..ec79c8a
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (3).png differ
diff --git a/Common Technologies/Kafka/Resources/image (4).png b/Common Technologies/Kafka/Resources/image (4).png
new file mode 100644
index 0000000..1bd4e8a
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (4).png differ
diff --git a/Common Technologies/Kafka/Resources/image (5).png b/Common Technologies/Kafka/Resources/image (5).png
new file mode 100644
index 0000000..f20b0b2
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (5).png differ
diff --git a/Common Technologies/Kafka/Resources/image (6).png b/Common Technologies/Kafka/Resources/image (6).png
new file mode 100644
index 0000000..1d5f3be
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (6).png differ
diff --git a/Common Technologies/Kafka/Resources/image (7).png b/Common Technologies/Kafka/Resources/image (7).png
new file mode 100644
index 0000000..0aa21c5
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (7).png differ
diff --git a/Common Technologies/Kafka/Resources/image (8).png b/Common Technologies/Kafka/Resources/image (8).png
new file mode 100644
index 0000000..1868e3a
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (8).png differ
diff --git a/Common Technologies/Kafka/Resources/image (9).png b/Common Technologies/Kafka/Resources/image (9).png
new file mode 100644
index 0000000..202d388
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image (9).png differ
diff --git a/Common Technologies/Kafka/Resources/image.png b/Common Technologies/Kafka/Resources/image.png
new file mode 100644
index 0000000..c2c26d2
Binary files /dev/null and b/Common Technologies/Kafka/Resources/image.png differ
diff --git a/Common Technologies/Kafka/Resources/kraft-architecture.png b/Common Technologies/Kafka/Resources/kraft-architecture.png
new file mode 100644
index 0000000..d8e5f67
Binary files /dev/null and b/Common Technologies/Kafka/Resources/kraft-architecture.png differ