MarkerSlide

A webcam-based presentation controller using classical computer vision. University Computer Vision Course Project

---

Project Purpose

MarkerSlide detects colored paper markers placed in front of a webcam and converts them into presentation keyboard commands (Next Slide, Previous Slide, Pause).

The system uses only classical computer vision algorithms taught in the course:

• RGB image processing
• RGB color thresholding
• Binary image generation
• Morphological processing (Opening, Closing)
• Connected Component Labeling
• Region property extraction (Area, Centroid, Bounding Box)
• Rule-based command generation
• Java Robot keyboard control

---

Course Restriction Statement

This project is submitted for a Computer Vision university course.

This project does NOT use:

• CNN (Convolutional Neural Networks)
• Deep Learning
• YOLO or any object detection model
• Haar Cascades
• MediaPipe
• Face Detection
• Hand Tracking
• ORB, SIFT, or SURF feature descriptors
• Machine Learning models
• HSV color space
• Any classification or prediction model

This project ONLY uses classical computer vision algorithms from the course curriculum.

---

Allowed Algorithms Used

Algorithm	Where Used
RGB Thresholding	RGBThresholdProcessor
Binary Image Generation	RGBThresholdProcessor → Core.inRange
Morphological Opening	MorphologyProcessor.applyOpening()
Morphological Closing	MorphologyProcessor.applyClosing()
Connected Component Labeling	ConnectedComponentAnalyzer → connectedComponentsWithStats
Area Calculation	CC_STAT_AREA from CCL stats
Centroid Calculation	centroids matrix from CCL
Bounding Box Calculation	CC_STAT_LEFT/TOP/WIDTH/HEIGHT from CCL stats
Region Property Extraction	RegionProperties model
Rule-Based Command Generation	CommandGenerator

---

System Flow

Webcam Frame (BGR)
        ↓
RGB Thresholding (RGBThresholdProcessor)
        ↓
Binary Image (white = marker, black = background)
        ↓
Morphological Opening (noise removal)
        ↓
Morphological Closing (hole filling)
        ↓
Connected Component Labeling (ConnectedComponentAnalyzer)
        ↓
Region Properties: Area, Centroid, Bounding Box
        ↓
Rule-Based Command Generation (CommandGenerator)
        ↓
[CV PIPELINE ENDS HERE]
        ↓
Java Robot (KeyboardCommandExecutor)
        ↓
Keyboard Event → PowerPoint / PDF Viewer

---

How to Run the Project

Prerequisites

• Java 21 (JDK 21 or later)
• Maven 3.8+
• OpenCV 4.x (4.9.0 recommended)

---

Step 1: Install OpenCV

Windows:

1. Download OpenCV from https://opencv.org/releases/
2. Run the installer (e.g., opencv-4.9.0-windows.exe)
3. Default install path: C:\opencv\
4. Find:
   • JAR: C:\opencv\build\java\opencv-490.jar
   • Native: C:\opencv\build\java\x64\opencv_java490.dll

Linux:

sudo apt-get install libopencv-dev
# or build from source at https://opencv.org/releases/

macOS:

brew install opencv

---

Step 2: Install OpenCV JAR to Local Maven Repository

Run this command once to make the JAR available to Maven:

mvn install:install-file \
  -Dfile=C:\opencv\build\java\opencv-490.jar \
  -DgroupId=org.opencv \
  -DartifactId=opencv \
  -Dversion=4.9.0 \
  -Dpackaging=jar

Alternatively, place the JAR at MarkerSlide/libs/opencv-490.jar and update pom.xml's <systemPath> accordingly.

---

Step 3: Build the Project

cd MarkerSlide
mvn clean package

This creates target/MarkerSlide.jar.

---

Step 4: Run the Application

Windows:

java -Djava.library.path=C:\opencv\build\java\x64 -jar target/MarkerSlide.jar

Linux:

java -Djava.library.path=/usr/local/lib -jar target/MarkerSlide.jar

macOS:

java -Djava.library.path=/usr/local/lib -jar target/MarkerSlide.jar

Important: -Djava.library.path must point to the directory containing:

• Windows: opencv_java490.dll
• Linux: libopencv_java490.so
• macOS: libopencv_java490.dylib

---

Step 5: Use the Application

1. Launch the application
2. Click ▶ Start Camera
3. Hold a red, blue, or green paper sheet/card in front of the webcam
4. The system detects the marker, shows the binary mask, and sends a keyboard command
5. Click ■ Stop Camera to stop

---

GUI Explanation

Panel	Location	Content
Live Webcam Feed	Left	RGB frame with bounding box + centroid drawn
Binary Mask	Center	Binary image after thresholding and morphology
System Information	Right top	Detected Color, Command, Area, Centroid, BBox, FPS
Algorithm Pipeline	Right bottom	CV pipeline steps displayed as labels
Controls	Bottom	Start Camera / Stop Camera buttons
Status Bar	Bottom	Camera, Detection, Cooldown status

---

Color Marker Rules

Marker Color	Command	Key Sent
🔴 Red paper	NEXT	Right Arrow →
🔵 Blue paper	PREVIOUS	Left Arrow ←
🟢 Green paper	PAUSE	Space
None detected	NONE	No key sent

---

Threshold Configuration

Color thresholds are defined in ThresholdConfig.java as RGB min/max ranges.

No HSV is used. All values are in RGB.

Default values:

Color	Min R	Max R	Min G	Max G	Min B	Max B
RED	150	255	0	100	0	100
BLUE	0	100	0	120	120	255
GREEN	0	120	120	255	0	120

To adjust thresholds: edit ThresholdConfig.java. No other file needs changing.

Calibration tip: Under bright or dark lighting, you may need to widen the ranges. Use plain, saturated colored paper for best results.

---

Morphology Explanation

Two morphological operations are applied in sequence:

Kernel

• 3x3 rectangular structuring element (configured in AppConfig.MORPH_KERNEL_SIZE)
• A 3x3 kernel is chosen because:
  • It is small enough to preserve the overall paper marker blob shape
  • It removes minor pixel-level noise from the binary mask
  • A larger kernel would risk merging separate regions or destroying marker shape

Operations in Pipeline

Step	Operation	Purpose
1	Opening (Erosion → Dilation)	Removes small isolated white noise pixels
2	Closing (Dilation → Erosion)	Fills small black holes inside the marker region

Opening removes stray white pixels caused by slight color variations in the background. Closing fills dark holes inside the marker caused by reflections or uneven paper color.

---

Connected Components Explanation

After morphological processing, ConnectedComponentAnalyzer applies Connected Component Labeling:

1. Uses OpenCV connectedComponentsWithStats (CCL algorithm, allowed in course)
2. Labels each connected white region with a unique integer
3. For each region extracts:
   • Area: number of white pixels (from CC_STAT_AREA)
   • Bounding Box: x, y, width, height (from CC_STAT_LEFT/TOP/WIDTH/HEIGHT)
   • Centroid: center of mass x, y (from centroids matrix)
4. Ignores regions with area < 500 pixels (configurable in AppConfig.MIN_REGION_AREA)
5. Selects the largest valid region

Command is only triggered if area > 3000 pixels (AppConfig.COMMAND_TRIGGER_AREA). This prevents false triggers from small marker glimpses.

---

Java Robot Explanation

Java Robot (KeyboardCommandExecutor) is invoked after the CV pipeline completes.

Separation of Stages

CV Pipeline         Java Robot Stage
(ImageProcessor)    (KeyboardCommandExecutor)
        |                   |
Command Generated --------> Execute Key Press

Java Robot is not part of the computer vision pipeline. It is only called when CommandGenerator returns a valid non-NONE command.

Cooldown

• Commands can only be sent once per 1000 ms (1 second)
• Prevents rapid repeated key presses while holding the marker
• Configurable in AppConfig.COMMAND_COOLDOWN_MS

---

Limitations

1. Lighting sensitivity: RGB thresholds may need calibration under different lighting
2. Color overlap: If wearing a red shirt, it may interfere with red marker detection
3. Single marker at a time: The system processes the largest valid region per frame
4. Fixed resolution: 640×480 is the target; actual resolution depends on the webcam
5. Java Robot focus: The keyboard event goes to whatever window has OS focus

---

Demo Instructions

1. Open PowerPoint or a PDF viewer in presentation/fullscreen mode
2. Start MarkerSlide
3. Click Start Camera
4. Hold a red paper card in front of the webcam → presentation advances
5. Hold a blue paper card → presentation goes back
6. Hold a green paper card → presentation pauses
7. Remove the marker to stop sending commands (cooldown resets)
8. Observe: Detected Color, Area, Centroid, Command all shown in GUI in real-time