Echo

Echo is a sentiment monitoring backend for tracking brand conversations across social media. The project combines a Go API service, a PostgreSQL/TimescaleDB data layer, an Apify-based data collection flow, and a separate Python ML service for sentiment analysis.

What the project does

• Lets users register and authenticate
• Lets users create brands and define tracking keywords
• Collects mention data for tracked brands
• Sends collected posts to an ML sentiment service
• Stores mentions, sentiment results, and aggregated snapshots for reporting
• Exposes protected brand analytics endpoints

Architecture at a glance

• Backend API: Go with Gin
• Database: PostgreSQL + TimescaleDB via pgx/sqlc
• Aggregation: Apify integration for social/X data collection
• ML layer: Python FastAPI service using Gemini-style analysis
• Infrastructure: Docker Compose for Postgres, RabbitMQ, and Redis

Repository structure

• cmd/: application entrypoint
• internal/api/: HTTP routes, handlers, and middleware
• internal/service/: business logic for auth, brands, and aggregation
• internal/aggregator/: aggregation pipeline and data normalization logic
• internal/db/: database connection, SQL migrations, and sqlc query definitions
• pkg/config/: environment/config loading
• pkg/apify/: Apify client and request/response models
• pkg/senitment-ml/: Go client for the sentiment ML service
• ML-pipeline/: standalone Python FastAPI sentiment analysis service
• docker-compose.yml: local infrastructure services
• Makefile: common local development commands

Tech stack

• Go 1.25
• Gin
• pgx + sqlc
• Viper + godotenv
• JWT + bcrypt
• PostgreSQL / TimescaleDB
• Python + FastAPI + Pydantic
• Docker Compose

Local development

1. Start infrastructure services

docker compose up -d

This brings up:

• PostgreSQL on port 5432
• RabbitMQ on port 5672/15672
• Redis on port 6380

2. Configure environment variables

Create a .env file in the project root with values like:

DB_URL=postgres://postgres:root@localhost:5432/voxpopuli
JWT_KEY=change-me
JWT_PUBLIC_KEY=change-me
OTP_SECRET=change-me
ENVIRONMENT=development
SERVER_PORT=8080
APIFY_TOKEN=your-apify-token
APIFY_X_BASE_URL=https://api.apify.com/v2/acts/...
SENTIMENT_BASE_URL=http://localhost:8000/analyze

3. Run database migrations

make migrate-up

4. Start the Go backend

make server

The API will start on the configured server port.

5. Start the ML pipeline (optional for sentiment analysis)

pip install -r ML-pipeline/requirements.txt
python ML-pipeline/main.py

Or with Uvicorn:

uvicorn ML-pipeline.main:app --host 0.0.0.0 --port 8000

Useful commands

make postgres
make createdb
make dropdb
make migrate-up
make migrate-down
make sqlc-gen

Notes

• The Go backend and Python ML service are designed to work together, but they can also run independently for local testing.
• The Apify and ML integrations depend on external credentials and URLs being configured.
• SQL queries are managed with sqlc, so regenerating the Go DB layer after schema/query changes is recommended.