[Sinity]

Summary

I moved the repository’s workflow validation onto a single extension-aware Postgres bootstrap story shared by CI and the local devenv shell. Instead of juggling GitHub service containers in one place, shell-initialized local clusters in another, and a dev shell that did not guarantee the same extension set, the workflows now run migrations/schema generation/SQLx checks inside nix develop against a deliberately provisioned sinex_dev database and sinity role. I also tightened extension setup for Timescale, vector, ULID, and optional pg_jsonschema, and updated the few tests that needed to tolerate ulid vs pgx_ulid differences. The branch is essentially a long stabilization loop that ends with CI and the dev shell describing the same database.

Motivation

The repository had a real “works in one environment, fails in another” problem around Postgres.

Migrations and schema/sqlx jobs need more than vanilla PostgreSQL: they rely on Timescale, vector, ULID support, and sometimes pg_jsonschema. Different workflow jobs were provisioning those extensions differently, some used GitHub Actions service images, some tried to emulate the local shell, and the dev shell itself was not guaranteeing the full extension set in one explicit package. That made failures hard to classify. Was the migration broken, or did the job just not have ulid? Did schema generation fail because the DB was wrong, or because the cluster was not ready yet?

The branch went through several approaches because the exact bootstrap point mattered. I tried leaning harder on GitHub service containers, then moved the work inside devenv, then had to fix pure-eval issues, role creation, socket readiness, extension availability, and superuser-vs-application-role behavior. I’m mentioning that because the final shape is the result of those pivots, not a first-pass design.

Changes grouped by concern

1. Run workflow database work inside nix develop

The decisive structural change is that the important database-aware jobs now run inside nix develop --no-pure-eval rather than relying on a looser outer shell plus a partially separate Postgres story.

That gives us a single environment in which to:

• run devenv test,
• start a local Postgres cluster,
• create sinity / sinex_dev,
• run migrations from crate/lib/sinex-schema,
• regenerate schemas,
• run SQLx preparation/checks.

I also made DEVENV_ROOT and DEVENV_DOTFILE explicit where nested bootstrap commands need them. That turns an implicit “hope the current directory and child shell agree” assumption into a real contract.

2. Make the Postgres cluster extension-aware everywhere

The workflows now provision the database with the extension set the repo actually needs. Over the course of the branch I tightened this from broad “CREATE EXTENSION” attempts to availability-aware logic and a local/dev shell that ships the extensions directly.

The key devenv-side change is the postgresqlWithExtensions package:

postgresqlWithExtensions =
  pkgs.postgresql_16.withPackages (ps:
    lib.filter (pkg: pkg != null) [
      (if ps ? timescaledb then ps.timescaledb else null)
      (if ps ? pgvector then ps.pgvector else null)
      (if ps ? pg_jsonschema then ps.pg_jsonschema else null)
      (if ps ? pgx_ulid then ps.pgx_ulid else null)
    ]
  );

That matters because it stops the local shell from lagging behind the workflow expectations. If CI and the dev shell are both talking about an extension-capable PostgreSQL package, the remaining failures are much easier to trust.

3. Fix bootstrap sequencing and role behavior

Several of the mid-branch fixes were about ordering and permissions rather than architecture, but they are the difference between a reproducible bootstrap and a flaky one.

The workflows now:

• wait for Postgres to be ready,
• create the role/database explicitly,
• run migrations before schema regeneration,
• use pg_ctl -w or explicit probes instead of timing assumptions,
• detect extension availability before trying to create optional extensions.

I also corrected the “who runs migrations?” story. The branch briefly tried running migrations as the application role and then reassigning ownership, but the cleaner outcome was to keep the migration/bootstrap path explicit and then make sure the application role can connect and operate normally afterward. That is less magical and easier to debug in CI logs.

4. Adjust schema-related tests to the real extension matrix

validator_schema_order_tests.rs now has an ensure_ulid_extension() helper that accepts either ulid or pgx_ulid. This is a small change, but it is exactly the kind of thing that keeps test intent focused on schema ordering rather than on whichever extension name the current Postgres build exposes.

I also carried a few harness/property-test touch-ups that were required by the new execution path so that the workflows continue to use the same sinex_test_utils shape as local development.

Testing

The branch’s strongest validation is the workflows themselves: CI, schema validation, SQLx cache, and SQLx check now all exercise the standardized bootstrap path. The schema regeneration check remains in place, so the branch does not just boot a DB—it proves the migration/generation ordering against the actual supported toolchain.

At the Rust-test level, validator_schema_order_tests.rs now explicitly handles the ulid/pgx_ulid split, which covers one of the more annoying extension-availability edge cases exposed by the branch.

The main gap is that the bootstrap logic is mostly exercised through workflow execution rather than through a standalone unit-test suite. That is acceptable here because the behavior is inherently operational and tightly coupled to the shell/database environment.

Notes

This branch is full of “fix(ci)” commits because the problem only really showed up once the workflows were forced onto a single path. I think that is a feature rather than a weakness: each fix removed one more silent assumption about extensions, readiness, or role ownership.

The final result I cared about was simple: if a developer or workflow says “run migrations, regenerate schemas, and validate SQLx,” they should all be talking to the same kind of PostgreSQL cluster with the same extension story.

Ref #103

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

Warning

Rate limit exceeded

@Sinity has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 7 minutes and 14 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 7 minutes and 14 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 64e0fc6b-42d0-466a-9962-de5089e108fa

📥 Commits

Reviewing files that changed from the base of the PR and between f9cb49a and a785260.

📒 Files selected for processing (16)

• .github/workflows/ci.yml
• .github/workflows/schema-validation.yml
• .github/workflows/sqlx-cache.yml
• .github/workflows/sqlx-check.yml
• crate/core/sinex-ingestd/tests/validator_schema_order_tests.rs
• crate/lib/sinex-core/tests/property/event_model_fuzzing_test.rs
• crate/lib/sinex-core/tests/property/schema_property_test.rs
• crate/lib/sinex-core/tests/property/ulid_property_test.rs
• crate/lib/sinex-schema/src/migrations/m20241028_000001_create_canonical_schema.rs
• crate/lib/sinex-test-utils/macros/src/lib.rs
• crate/lib/sinex-test-utils/src/lib.rs
• crate/lib/sinex-test-utils/src/property_testing.rs
• crate/lib/sinex-test-utils/src/snapshot_helper.rs
• crate/lib/sinex-test-utils/src/test_context.rs
• devenv.nix
• rustfmt.toml

Note

.coderabbit.yaml has unrecognized properties

CodeRabbit is using all valid settings from your configuration. Unrecognized properties (listed below) have been ignored and may indicate typos or deprecated fields that can be removed.

⚠️ Parsing warnings (1)

Validation error: Unrecognized key(s) in object: 'finishing_touches'

⚙️ Configuration instructions

• Please see the configuration documentation for more information.
• You can also validate your configuration using the online YAML validator.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/chore/ci-devenv-postgres-bootstrap

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Standardizes Postgres bootstrap + extension provisioning so CI workflows and the local devenv environment run migrations/schema generation/SQLx checks against the same extension-capable Postgres setup, and adjusts test utilities/macros to better capture failure context and handle ULID extension variants.

Changes:

• Reworks CI + schema/SQLx workflows to run DB-dependent steps inside nix develop and provision a consistent Postgres cluster + extensions.
• Extends devenv Postgres packaging to include required extensions (Timescale/vector/ULID/pg_jsonschema) and updates schema/tests to tolerate ulid vs pgx_ulid.
• Improves test failure artifact capture (snapshot context + proptest runner configuration).

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
rustfmt.toml	Ignores rustfmt on large property test files.
devenv.nix	Adds extension-aware Postgres package + pinned fenix fallback + devenv root wiring.
crate/lib/sinex-test-utils/src/test_context.rs	Adds failure snapshot type to persist context/logs across moves.
crate/lib/sinex-test-utils/src/snapshot_helper.rs	Extends failure persistence API to accept borrowed or snapshotted context.
crate/lib/sinex-test-utils/src/property_testing.rs	Centralizes proptest runner config + regression file path selection.
crate/lib/sinex-test-utils/src/lib.rs	Exposes proptest runner config helper + re-exports failure snapshot.
crate/lib/sinex-test-utils/macros/src/lib.rs	Updates sinex_prop config/seed handling and failure persistence calls.
crate/lib/sinex-schema/src/migrations/m20241028_000001_create_canonical_schema.rs	Adds ulid → pgx_ulid fallback when provisioning extensions.
crate/lib/sinex-core/tests/property/ulid_property_test.rs	Fixes/adjusts proptest block structure.
crate/lib/sinex-core/tests/property/schema_property_test.rs	Makes proptest returns explicit with Report turbofish.
crate/lib/sinex-core/tests/property/event_model_fuzzing_test.rs	Makes proptest returns explicit and adjusts macro block structure.
crate/core/sinex-ingestd/tests/validator_schema_order_tests.rs	Adds helper to create whichever ULID extension is available.
.github/workflows/sqlx-check.yml	Runs SQLx check inside nix develop with local Postgres bootstrap + extensions.
.github/workflows/sqlx-cache.yml	Validates SQLx cache inside nix develop with standardized Postgres bootstrap.
.github/workflows/schema-validation.yml	Validates schema generation inside nix develop with standardized Postgres bootstrap.
.github/workflows/ci.yml	Consolidates CI to a devenv-driven integration-style job with standardized Postgres bootstrap.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The CREATE EXTENSION statement is malformed: format!(r#"CREATE EXTENSION IF NOT EXISTS "{extension}""#) produces an extra trailing quote and no semicolon, so this will error at runtime. Build the statement without the trailing quote (and ideally add ;).

Suggested change

	let stmt = format!(r#"CREATE EXTENSION IF NOT EXISTS "{extension}""#);
	let stmt = format!(r#"CREATE EXTENSION IF NOT EXISTS "{extension}";"#);

[Copilot]

pgJsonschemaPkg fetches a prebuilt .deb for pg16-amd64-linux-gnu, but it’s included unconditionally in postgresqlWithExtensions. This will fail to evaluate/build on non-x86_64 Linux systems (and likely macOS). Consider guarding this derivation/package with a system == "x86_64-linux" check (or using ps.pg_jsonschema when available) so devenv remains usable cross-platform.

Suggested change

	pgJsonschemaPkg
	(if ps ? pg_jsonschema then ps.pg_jsonschema else if system == "x86_64-linux" then pgJsonschemaPkg else null)

[Copilot]

This workflow runs migrations with DATABASE_URL as the initdb superuser (line 89-93) but then verifies schema registry writes by connecting as sinity (line 95-97). Since migrations create schemas/tables owned by the migration role, sinity is likely to hit permission errors unless ownership/grants are explicitly handled. To make this check reliable, run migrations as sinity after extension provisioning (or add explicit ALTER ... OWNER TO sinity / GRANTs in the migration/bootstrap).

[Copilot]

CI is now a single Devenv Integration Test job. If the previous Nix flake checks and coverage reporting are still required quality gates, they appear to have been dropped rather than moved into devenv test/dev:test. Consider re-adding equivalent steps (or documenting that these checks are intentionally removed/replaced) so CI coverage doesn’t regress silently.

[Copilot]

The Postgres bootstrap shell (initdb/pg_ctl/role+db creation/ensure_extension) is duplicated inline here and in other workflows (sqlx-check.yml, sqlx-cache.yml, schema-validation.yml). This increases the risk of drift and subtle inconsistencies over time; consider extracting it into a shared script (e.g. scripts/ci/postgres-bootstrap.sh) or a composite action and invoking it from each workflow.