Machine Learning From Scratch Engine

A disciplined, production-grade machine learning library engineered completely from first principles using vanilla Python and a custom linear algebra matrix core. This repository serves as a rigorous exploration of mathematical optimization, algorithmic convergence, and low-level memory management.

📈 Implemented Algorithms & Progress

Tier 1: Distance & Gradients

• Linear Regression (via Gradient Descent)
  • Custom matrix-based forward pass ($Y = XW + b$).
  • Iterative weight optimization tracking Mean Squared Error (MSE).
• Logistic Regression (Binary Classifier)
  • Element-wise matrix Sigmoid activation function mapping: $\sigma(z) = \frac{1}{1 + e^{-z}}$.
  • Optimization utilizing Binary Cross-Entropy (BCE) Log-Loss.
• K-Means Clustering (In Progress)
  • Geometric grouping utilizing iterative Euclidean distance assignment.

Tier 2: Probabilistic Frameworks

• Naive Bayes Classification Suite
  • Modular parent-child object-oriented architecture leveraging log-likelihood summation to eliminate numerical underflow.
  • Gaussian NB: Built for continuous, real-valued feature sets utilizing a vectorized normal distribution probability density function.
  • Multinomial NB: Built for discrete text/token frequency counts utilizing categorical ratio mapping with hyperparameter-driven Laplace smoothing.
  • Bernoulli NB: Built for binary multivariate state tracking utilizing presence/absence probability boundaries.

---

🛠️ Core Mathematical Engine (matrix_core.py)

Rather than relying on third-party frameworks like NumPy, this library utilizes a custom-built object-oriented Matrix class. It natively implements:

• Operator Overloading: __add__, __sub__, __mul__ (Scalar), and __matmul__ (@ Matrix Dot Product).
• Dimensional Guardrails: Strict runtime shape verification protecting matrix transformations from dimension mismatches.
• Matrix Transposition: Explicit index swapping ($O(m \times n) \to O(n \times m)$) via properties (.T).

💻 Technical Execution

To run any module and verify model optimization, execute the scripts using your local Python interpreter from your workspace root:

# Verify Linear Regression Convergence
python models/linear.py

# Verify Logistic Regression Binary Classification
python models/logistic.py