Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/hub-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Hub image

on:
pull_request:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read

jobs:
image:
name: Hub image smoke test
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: 24
- run: node hub/script/check-image/index.ts
13 changes: 13 additions & 0 deletions hub/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
.env*
!.env.example
.devcontainer
.local
.npm
node_modules
dist
coverage
storybook-static
server/generated
**/*.test.*
**/stories.tsx
.storybook
6 changes: 6 additions & 0 deletions hub/.env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Optional API port. The Vite development proxy expects the default.
PORT=3001
HOST=127.0.0.1

# The image defaults to combined; development starts the API alone.
RUNTIME_MODE=api
SHUTDOWN_TIMEOUT_MS=30000
# CLIENT_DIRECTORY=/absolute/path/to/dist/client

# API logging threshold: trace, debug, info, warn, error, fatal, or silent.
LOG_LEVEL=info
Expand Down
27 changes: 27 additions & 0 deletions hub/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# syntax=docker/dockerfile:1
FROM node:24-bookworm-slim AS build
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends openssl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY package.json package-lock.json .npmrc ./
COPY schema ./schema
COPY prisma7.config.ts ./
RUN npm ci --engine-strict
COPY . .
RUN npm run build:runtime

FROM node:24-bookworm-slim AS runtime
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends openssl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
ENV NODE_ENV=production HOST=0.0.0.0 PORT=3001 RUNTIME_MODE=combined \
CLIENT_DIRECTORY=/app/dist/client SHUTDOWN_TIMEOUT_MS=30000
COPY --from=build /app/package.json /app/package-lock.json /app/.npmrc ./
COPY --from=build /app/schema ./schema
COPY --from=build /app/prisma7.config.ts ./
COPY --from=build /app/node_modules ./node_modules
RUN npm prune --omit=dev --ignore-scripts && npm cache clean --force
COPY --from=build /app/dist ./dist
USER node
EXPOSE 3001
CMD ["node", "dist/server/index.js"]
6 changes: 4 additions & 2 deletions hub/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@

Hub is being rebuilt on `build/hub/new` using TypeScript, Express, React, Vite, and Vitest. This branch lets contributors start the new app locally. It is not ready to replace the existing Hub.

The API has one route: `GET /api/health`. The frontend displays the existing Replica logo and supports light and dark themes. Existing components, features, and pages are kept in Storybook, but they are not connected to the new API.
The API has liveness and readiness routes. The frontend displays the existing Replica logo and supports light and dark themes. Existing components, features, and pages are kept in Storybook, but they are not connected to product APIs.

Contributors can use PostgreSQL and Prisma locally or in the devcontainer. The [background-job foundation](docs/background-jobs.md) provides PostgreSQL-backed BullMQ queues and independent on-demand and scheduled workers, without product jobs yet. The database has no product models. Sign-in, saved product data, live updates, and deployment are not implemented.
Contributors can use PostgreSQL and Prisma locally or in the devcontainer. The [background-job foundation](docs/background-jobs.md) provides PostgreSQL-backed BullMQ queues and independent on-demand and scheduled workers, without product jobs yet. The database has no product models. Sign-in, saved product data, and live updates are not implemented.

The [production runtime](docs/deployment.md) packages the frontend, API, and workers in one image, with a separate release setup script.

Use [Development](docs/development.md) to start the API and frontend. Review [Architecture](docs/architecture.md), [code architecture conventions](docs/backend-architecture.md), and [coding conventions](docs/coding-conventions.md) before adding behavior.

Expand Down
1 change: 1 addition & 0 deletions hub/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Hub is being rebuilt. The [Hub overview](../README.md) describes what the baseli
- [Code architecture conventions](backend-architecture.md): repositories, integrations, utilities, and domains.
- [Coding conventions](coding-conventions.md): frontend layers and interface rules.
- [Development](development.md): local API and frontend setup.
- [Deployment](deployment.md): production image, runtime roles, setup, and health.
- [Database](database.md): Prisma schema conventions, migrations, and the shared client.
- [Authentication proof](authentication.md): internal provider proof and one-use authorization attempts.
- [Logging](logging.md): application events, automatic request records, and payload safety.
Expand Down
6 changes: 4 additions & 2 deletions hub/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@ The team has chosen TypeScript, Express for the API, React and Vite for the fron

The API responds to a health check, and the frontend displays the logo with light and dark themes. Storybook keeps the existing components and pages available for reuse.

Contributors can use PostgreSQL and Prisma locally or in the devcontainer. The database has a provider-neutral identity transaction foundation and internal GitHub authentication proof primitives. No public authentication flow is exposed. Deployment and product APIs have not been built. This app cannot yet find repositories, import Builds, host previews, save product data, or support reviews.
Contributors can use PostgreSQL and Prisma locally or in the devcontainer. The database has a provider-neutral identity transaction foundation and internal GitHub authentication proof primitives. No public authentication flow is exposed. Product APIs have not been built. This app cannot yet find repositories, import Builds, host previews, save product data, or support reviews.

The [production runtime](deployment.md) serves the built frontend and API and runs both workers in one image. Deployment runs database and job setup separately before starting that image.

## Backend

API code lives under `server/`. As features are added, route handlers will read requests, call domains, and return responses. The [code conventions](backend-architecture.md) explain repositories, integrations, utilities, domains, and which layers may import others. Put these directories under `server/` when they are needed; do not add empty implementations.

`GET /api/health` returns `{ "status": "ok" }`. Authentication routes are not registered. Unknown paths return a JSON 404 response; invalid JSON is rejected before routing. `createApp()` builds the Express app without starting a server. `server/index.ts` starts it at a local address.
`GET /api/health` returns `{ "status": "ok" }`; `GET /api/ready` reports operational readiness. Authentication routes are not registered. Unknown API paths return a JSON 404 response; invalid JSON is rejected before routing. `createApp()` builds the Express app without starting a server. `server/index.ts` selects a runtime role and defaults to the local API.

The [database guide](database.md) defines the multi-file Prisma schema and shared client in integrations. The health route remains a process check, not a database readiness check.

Expand Down
16 changes: 9 additions & 7 deletions hub/docs/backend-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

Organize backend functions into routes, repositories, integrations, utilities, and domains. Each layer has a clear job and rules for which other layers it may use.

| Layer | Responsibility | Directory example |
| --- | --- | --- |
| Routes | Parse HTTP requests, call backend functions, and send responses. Directories follow URL paths and HTTP methods. | `routes/api/health/get/index.ts` |
| Repositories | Functions that interact with the database. Each repository directory represents one RESTful resource. | `repositories/user/get-user/index.ts` |
| Integrations | Functions that interact with an API or SDK. Each integration directory represents one API or SDK service. | `integrations/github/get-user/index.ts` |
| Utilities | General functions reusable elsewhere in the codebase. | `utils/format-timestamp/index.ts` |
| Domains | Business logic functions that may compose repositories, integrations, and utilities. | `domains/user/get-user-profile/index.ts` |
| Layer | Responsibility | Directory example |
| ------------ | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| Routes | Parse HTTP requests, call backend functions, and send responses. Directories follow URL paths and HTTP methods. | `routes/api/health/get/index.ts` |
| Repositories | Functions that interact with the database. Each repository directory represents one RESTful resource. | `repositories/user/get-user/index.ts` |
| Integrations | Functions that interact with an API or SDK. Each integration directory represents one API or SDK service. | `integrations/github/get-user/index.ts` |
| Utilities | General functions reusable elsewhere in the codebase. | `utils/format-timestamp/index.ts` |
| Domains | Business logic functions that may compose repositories, integrations, and utilities. | `domains/user/get-user-profile/index.ts` |

## Import boundaries

Expand Down Expand Up @@ -37,6 +37,8 @@ Export route handlers by HTTP method, such as `get()`, and alias them to descrip

Add directories when their functions are needed. Do not add empty implementations just to fill out the structure.

Operational entry points belong under `server/scripts/`. They may compose backend functions and run deployment tools, but routes and domains must not import them. Setup runs separately from the API and workers.

## Database access

The [shared Prisma client](database.md#shared-client) lives in `server/integrations/prisma/index.ts`. Domains may pass it to repositories for database queries without letting repositories import integrations. Keep model definitions, naming, and migrations consistent with the [database conventions](database.md#schema-conventions).
Expand Down
28 changes: 15 additions & 13 deletions hub/docs/background-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Hub uses BullMQ's PostgreSQL backend for internal background work. The foundatio

Two stable queue names separate execution capacity:

| Queue | Work | Command from `hub/` |
| --- | --- | --- |
| `on-demand` | Work requested by an application action | `npm run worker:on-demand` |
| Queue | Work | Command from `hub/` |
| ----------- | --------------------------------------------------- | -------------------------- |
| `on-demand` | Work requested by an application action | `npm run worker:on-demand` |
| `scheduled` | Recurring work and work spawned by recurring sweeps | `npm run worker:scheduled` |

Each command starts one Node.js process consuming only its assigned queue. The API starts neither worker. Multiple processes may consume the same queue when more capacity is needed. Separate queues do not isolate PostgreSQL, CPU, or external service quotas.
Expand All @@ -17,7 +17,9 @@ Each command starts one Node.js process consuming only its assigned queue. The A

`server/workers/configuration.ts` is the composition point for processors and schedules. Its empty registrations are intentional until product domains exist. Worker startup logs readiness after connecting; unknown job names fail permanently rather than disappearing.

Workers stop accepting work on `SIGINT` or `SIGTERM` and wait for active jobs. After 30 seconds, the command exits unsuccessfully rather than waiting forever. Interrupted work can be retried by BullMQ's stalled-job recovery. Deployment stop timeouts must allow this shutdown window. Long-running domain work needs its own bounded execution and interruption policy; a queue lock is not a business lock.
Worker readiness confirms initial connection only; it is not continuous consumer-health monitoring. A later connection error is logged but does not clear the combined runtime's worker-initialized signal.

Workers stop accepting work on `SIGINT` or `SIGTERM` and wait for active jobs. After `SHUTDOWN_TIMEOUT_MS` (30000 milliseconds by default), the command exits unsuccessfully rather than waiting forever. Interrupted work can be retried by BullMQ's stalled-job recovery. Deployment stop timeouts must allow this shutdown window. Long-running domain work needs its own bounded execution and interruption policy; a queue lock is not a business lock.

## Database and setup

Expand All @@ -42,7 +44,7 @@ npm run start:worker:on-demand
npm run start:worker:scheduled
```

Run the last three commands as separate services, not sequential commands in one shell. These roles can use one application image in a future Compose deployment, with PostgreSQL shared and only the web service exposed. Production frontend hosting, an application image, service health checks, and release orchestration are not implemented by this foundation.
Run the last three commands as separate services, not sequential commands in one shell. The [production image](deployment.md) instead defaults to a combined process that runs the API and both workers and coordinates their shutdown.

Each queue or worker owns a small query pool and BullMQ's additional notification connections. The query pool is limited to two connections; concurrency does not increase that limit. Budget database connections across all API, producer, and worker instances. Creation functions require an explicit URL so tests and other callers cannot accidentally inherit a previously imported database client.

Expand All @@ -52,14 +54,14 @@ Keep stable names, a Zod payload schema, and retry options in a job definition.

Use the functions under `server/integrations/bullmq/`:

| Function | Contract |
| --- | --- |
| `migrateJobDatabase(databaseUrl)` | Explicitly migrate BullMQ's schema and close the migration connection. |
| `createJobQueue(options)` | Open a client for one named PostgreSQL queue. The caller must close it. |
| `enqueueJob(queue, definition, payload, options)` | Validate JSON payload data and enqueue work with job-specific retries and optional delay, priority, or job ID. Failure rejects the call. |
| `createJobProcessor(definition, process, shouldRetry)` | Bind typed execution to runtime payload validation. Invalid stored payloads fail permanently. |
| `createJobWorker(options)` | Construct a stopped consumer for registered processors. The caller starts and closes it. Duplicate processor names are rejected. |
| `upsertJobSchedule(queue, schedule)` | Validate and register an interval or timezone-explicit cron schedule on `scheduled`. |
| Function | Contract |
| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `migrateJobDatabase(databaseUrl)` | Explicitly migrate BullMQ's schema and close the migration connection. |
| `createJobQueue(options)` | Open a client for one named PostgreSQL queue. The caller must close it. |
| `enqueueJob(queue, definition, payload, options)` | Validate JSON payload data and enqueue work with job-specific retries and optional delay, priority, or job ID. Failure rejects the call. |
| `createJobProcessor(definition, process, shouldRetry)` | Bind typed execution to runtime payload validation. Invalid stored payloads fail permanently. |
| `createJobWorker(options)` | Construct a stopped consumer for registered processors. The caller starts and closes it. Duplicate processor names are rejected. |
| `upsertJobSchedule(queue, schedule)` | Validate and register an interval or timezone-explicit cron schedule on `scheduled`. |

`runJobWorker()` owns process signals and consumer lifetime outside the integration. Integration code does not import domains, repositories, or the shared logger; worker composition injects processors and logging.

Expand Down
Loading
Loading