AggieRMP - Texas A&M University Rate My Professor Analysis

A comprehensive data collection and analysis system for Texas A&M University professor ratings, grade distributions, and course information.

🚀 Features

• Department & Course Scraping — Automated collection of department and course data from TAMU Course Catalog
• Rate My Professor Integration — Collection and analysis of professor ratings and reviews
• Database Management — PostgreSQL-based storage with SQLAlchemy ORM
• Data Analysis — AI-powered summarization and insights generation
• API Endpoints — RESTful API for accessing collected data
• Auth (SuperTokens) — Sign-in, sessions, and JWT for the web app and extension; protected routes use verify_session
• Seat alerts (Section Watcher) — Users track sections via API; a scheduled section watcher reads user_tracked_sections and section open status, then sends Web Push when a seat opens

🏗️ Architecture Flow

graph TD
    %% -- Styling --
    classDef external fill:#f9f,stroke:#333,stroke-width:2px,color:#fff;
    classDef storage fill:#555,stroke:#333,stroke-dasharray: 5 5,color:#fff;
    classDef client fill:#4a90e2,stroke:#333,stroke-width:2px,color:#fff;
    classDef default fill:#333,color:#fff,stroke:#111;
    
    %% -- Entities --
    User["TAMU Student (Browser)"]:::client

    subgraph ExternalSources ["External Data Sources"]
        direction LR
        TAMU["TAMU Catalog"]:::external
        RMP["RateMyProf"]:::external
        Anex["Anex.us (GPA)"]:::external
    end

    subgraph ClientLayer ["Client Side (Frontend)"]
        NextJS["Next.js Web App (Vercel)"]
        
        subgraph Extension ["Chrome Extension"]
            ContentScript["Content Script (Injected)"]
            Popup["Extension Popup (UI)"]
        end
    end

    subgraph AuthLayer ["Auth (SuperTokens)"]
        SuperTokens["SuperTokens — sessions, email/password, JWT"]:::auth
    end

    subgraph ServiceLayer ["Application Layer"]
        FastAPI["AggieSB+ API (FastAPI/Python)"]
    end

    subgraph WorkerLayer ["Data Pipelines (Scheduled)"]
        Scrapers["Scraper Workers (TAMU, RMP, GPA)"]
        SectionWatcher["Section Watcher — seat-open alerts + Web Push"]
    end

    subgraph StorageLayer ["Storage Layer"]
        PostgreSQL[(PostgreSQL Main DB)]:::storage
        Redis[(Redis — cache + tracked CRNs)]:::storage
    end

    %% -- Connections --

    %% User to Clients
    User -->|Interacts with| NextJS
    User -->|Uses| Popup
    ContentScript -.->|Injects UI into| User

    %% Auth: sign-in and protected API
    User -->|Sign in / account| NextJS
    NextJS <-->|Auth APIs| SuperTokens
    FastAPI <-->|verify_session + recipe| SuperTokens

    %% Clients to Backend (REST; tracking requires session)
    NextJS <-->|REST| FastAPI
    Popup <-->|Fetch Stats| FastAPI
    ContentScript <-->|Fetch Prof Metrics| FastAPI
    NextJS -->|Track section alerts| FastAPI
    ContentScript -->|Seat alerts| FastAPI

    %% Backend logic
    FastAPI <-->|SQL Queries| PostgreSQL
    FastAPI <-->|Cache / tracking sets| Redis

    %% Pipeline Flow
    Scrapers -.->|Scrape / Download| ExternalSources
    Scrapers -->|ETL / Upsert| PostgreSQL

    %% Section watcher: watches user_tracked_sections + sections.is_open → push
    SectionWatcher -->|Active watches + section status| PostgreSQL
    SectionWatcher -->|Optional Redis coordination| Redis
    SectionWatcher -.->|Web Push when seat opens| User

Loading

📁 Project Structure

AggieRMP/
├── 📁 src/aggiermp/           # Main source code
│   ├── 📁 api/                # API endpoints and routes
│   ├── 📁 collectors/         # Data collection + section_watcher (seat alerts)
│   ├── 📁 database/           # Database models and operations
│   ├── 📁 models/             # Pydantic data models
│   ├── 📁 core/               # Core utilities and configuration
│   └── main.py                # Main application entry point
├── 📁 pipelines/              # Data processing pipelines
│   ├── 📁 professors/         # Professor reviews and summarization
│   ├── 📁 gpa/                # GPA data collection and processing
│   ├── 📁 sections/           # Course section updates
│   └── 📁 courses/            # Course catalog updates
├── 📁 docs/                   # Documentation
├── pyproject.toml             # Project dependencies
└── README.md                  # This file

🛠️ Installation

1. Clone the repository

   git clone <repository-url>
   cd AggieRMP

2. Set up virtual environment

   python -m venv .venv
   
   # Windows
   .venv\Scripts\activate
   
   # macOS/Linux
   source .venv/bin/activate

3. Install dependencies

   pip install -e .

4. Set up database

   • Configure your PostgreSQL connection
   • Run database migrations

🎯 Usage

Data Pipelines

1. Upsert Professor Reviews & Summaries

   python pipelines/professors/upsert_reviews_and_summaries.py --force-update

2. Upsert GPA Data

   python pipelines/gpa/upsert_gpa_data.py

API Server

python src/aggiermp/main.py

📊 Data Sources

Source	Description
TAMU Course Catalog	Department and course information
Anex.us	Historical GPA data
Rate My Professor	Professor ratings and reviews
Manual Curation	Additional data validation & enhancement

📈 Status

Pipeline	Status
Sections Upsert
GPA Upsert
Courses Upsert
Sections Watcher

📝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

📄 License

This project is for educational and research purposes.

🙋‍♂️ Support

For questions or issues, please open a GitHub issue or contact the maintainer.