End-to-End Disease Classification

A state-of-the-art Deep Learning project designed to classify kidney tumors from CT scan images. This application leverages a robust MLOps pipeline using MLflow for experiment tracking, DVC for data version control, and TensorFlow for high-performance model training. The final model is containerized with Docker and deployed on Google Cloud Run for serverless scalability.

---

🩺 Problem Statement

Kidney disease is a significant health concern worldwide, and early detection of tumors is critical for effective treatment. Radiologists often rely on CT scans to identify abnormalities, but manual interpretation can be time-consuming and subject to inter-observer variability.

The Solution: This project automates the classification process using Transfer Learning with the VGG16 architecture. By fine-tuning this pre-trained model on a dataset of Kidney CT images, we assist medical professionals with rapid, consistent assessments of whether a scan shows a "Tumor" or is "Normal".

📌 Technical Highlights

• Transfer Learning: Utilizes VGG16 (pre-trained on ImageNet) to achieve high accuracy with a relatively small dataset.
• Pipeline Orchestration: DVC manages the entire machine learning pipeline (ingestion -> preparation -> training -> evaluation), ensuring full reproducibility.
• Experiment Tracking: MLflow tracks experiments, logging metrics (accuracy, loss) and parameters to a remote server (DAGsHub).
• Serverless Deployment: Automatically builds and deploys to Google Cloud Run via GitHub Actions, featuring a highly optimized Docker container with system-level dependencies for computer vision.
• Modern Web UI: A sleek Flask-based frontend for real-time predictions.

🏗️ Project Structure

├── .github/                # CI/CD workflows for Google Cloud Deployment
├── config/                 # Configuration files (config.yaml)
├── src/                    # Source code
│   └── cnnClassifier/      # Main package
│       ├── components/     # Logic for each pipeline stage
│       ├── config/         # Configuration managers
│       ├── entity/         # Data classes for config
│       ├── pipeline/       # Pipeline orchestration scripts
│       └── utils/          # Utility functions (common.py)
├── templates/              # HTML templates for Flask UI
├── app.py                  # Flask application entry point
├── main.py                 # Main pipeline runner
├── dvc.yaml                # DVC pipeline definition
├── params.yaml             # Hyperparameters definitions
├── Dockerfile              # Container configuration
└── pyproject.toml          # Project dependencies (via uv)

🚀 Getting Started

Prerequisites

• Python 3.10+
• Docker (optional, for local container testing)
• Google Cloud Account (for deployment)

Local Installation

1. Clone the repository:

   git clone https://github.com/badri2006nathan/End-To-End-Disease-Classification-Using-MLflow.git
   cd End-To-End-Disease-Classification-Using-MLflow

2. Install dependencies using uv:

   pip install uv
   uv sync
   source .venv/bin/activate

3. Run the Web Application:

   python app.py

   Navigate to http://localhost:8080.

☁️ Google Cloud Deployment Guide

This project features a fully automated CI/CD pipeline that deploys the application to Google Cloud Run whenever changes are pushed to the main branch.

1. GCP Setup

1. Create a Google Cloud Project: Note your PROJECT_ID.
2. Enable APIs:
   • Cloud Run Admin API
   • Cloud Build API
   • Artifact Registry API
3. Create Artifact Registry:
   • Go to Artifact Registry -> Create Repository.
   • Name: disease-classification
   • Format: Docker
   • Region: us-central1 (or your preferred region).

2. Service Account Setup

Create a Service Account to verify identity from GitHub Actions:

# Create Service Account
gcloud iam service-accounts create github-deploy-sa --display-name="GitHub Actions Deployer"

# Grant Permissions (Cloud Run Admin, Storage Admin, Service Account User, Artifact Registry Writer)
gcloud projects add-iam-policy-binding <PROJECT_ID> \
    --member="serviceAccount:github-deploy-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
    --role="roles/run.admin"

gcloud projects add-iam-policy-binding <PROJECT_ID> \
    --member="serviceAccount:github-deploy-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
    --role="roles/iam.serviceAccountUser"

gcloud projects add-iam-policy-binding <PROJECT_ID> \
    --member="serviceAccount:github-deploy-sa@<PROJECT_ID>.iam.gserviceaccount.com" \
    --role="roles/artifactregistry.writer"

3. GitHub Secrets Configuration

Go to your GitHub Repository -> Settings -> Secrets and Variables -> Actions -> New Repository Secret.

Secret Name	Value
GCP_SA_KEY	The JSON Key content of the Service Account created above.

4. Handling Large Models

Since our trained model (model.h5) is ~57MB, it fits within GitHub's file limits. We force-added it to the repository to ensure it's available in the Docker container:

git add -f model/model.h5
git commit -m "Add model file"
git push origin main

For Larger Models (>100MB):

1. Upload the model to Google Cloud Storage (GCS) manually.
2. Update app.py to download the model from GCS on startup using the google-cloud-storage library.
3. Grant Storage Object Viewer role to your Cloud Run service account.

5. Deployment

Push your code to the main branch. The GitHub Action in .github/workflows/main.yaml will:

1. Authenticate with Google Cloud.
2. Build the Docker image (installing system dependencies like libgl1 for OpenCV).
3. Push the image to Google Artifact Registry.
4. Deploy the service to Cloud Run with optimized memory (2Gi) and timeout (300s) settings.

🛠️ Pipeline Stages (MLOps)

1. Data Ingestion: Downloads the Kidney CT Scan dataset from source.
2. Prepare Base Model: Loads the VGG16 model (excluding top layers) and saves it.
3. Training: Fine-tunes the model on the dataset with data augmentation.
4. Evaluation: Evaluates the trained model on a test set and logs metrics to MLflow.

📜 License

This project is licensed under the MIT License.