FAIRagro SQL-to-ARC Middleware

This repository contains the SQL-to-ARC Converter, a core component of the FAIRagro advanced middleware architecture. It enables Research Data Infrastructure (RDI) providers to transform their relational metadata into standardized Annotated Research Context (ARC) objects and transmit them to the central FAIRagro Middleware API.

📁 Repository Structure

Folder	Description
middleware/	Source code of the converter component.
openspec/	OpenSpec domain specs and change proposals (spec-driven development).
docs/	Architectural design, database view specifications, and API documentation.
dev_environment/	Docker-based local development setup (Postgres, Mock API).
scripts/	Tooling for quality checks, Dev Container helpers, hooks, and Git LFS.
docker/	Dockerfiles and container structure tests.

🌟 Quick Start (Full Local Demo)

For the best out-of-the-box experience, you can run a complete local demonstration. This setup starts a PostgreSQL database with demo data, a local Mock Middleware API, and the SQL-to-ARC converter to process and save results locally:

# Start the full demo stack (requires Docker)
./dev_environment/start-demo.sh --build

Note: This demo does not require any secrets or mTLS keys. Generated ARCs will be saved to dev_environment/demo_output/.

🚀 Getting Started (Development)

The preferred method for working with this repository is using a Dev Container (VS Code or Cursor): Reopen in Container / Dev Containers: Reopen in Container uses .devcontainer/devcontainer.json — see docs/devcontainer.md.

While it is possible to develop without the Dev Container (see next steps below), this approach is not tested and is therefore neither documented nor officially supported.

1. Prerequisites (for manual setups only)

• Python 3.12+ (pin in versions.env / .python-version)
• uv (Dependency Management & Workspace Orchestration)
• Docker & Docker Compose
• Node.js 22+ (for npm run lint:md on host clones; installed in the Dev Container)
• Git LFS for *.sql — Dev Container restores via devcontainer-post-create.d/50-git-lfs.sh; on host clones use ./scripts/setup-git-lfs.sh; see docs/git-lfs.md

2. Environment Setup

Clone the repository and install all workspace dependencies:

uv sync --dev --all-packages
./scripts/setup-git-hooks.sh
./scripts/setup-git-lfs.sh

3. Start Local Development Environment

The dev_environment folder provides a full stack including a PostgreSQL database pre-filled with edaphobase data.

Please refer to the Development Environment README for detailed instructions on prerequisites (like secret management and mTLS keys), setup, and usage.

🔧 Component Documentation

Detailed information on how to use, configure, and deploy the specific components can be found in their respective subdirectories:

• SQL-to-ARC Converter README: Configuration (YAML/Env), CLI options, and production deployment.
• Architectural Design: Deep dive into the concurrency model, memory management, and data flow.
• Database View Spec: The SQL views required for the RDI provider database.

🧪 Quality Standards

We maintain high code quality through automated checks:

# Run all quality checks (Ruff, Mypy, Pylint, Bandit)
./scripts/quality-check.sh

# Run unit and integration tests
uv run pytest middleware/sql_to_arc/tests/

---

Maintained by: FAIRagro Middleware Team License: LICENSE