Add state migrations and reduce Postgres scheduling overhead

[rorylshanks]

Large queues currently sort JSON-derived sizes for rewrite claims, load whole jobs to choose compaction work, and repeat maintenance and progress writes across workers. This adds indexed scheduling columns and moves selection and aggregation into Postgres to reduce that work.

• Add partforge migrate with numbered, transactional migrations and a version ledger. Adopt existing hand-created tables, backfill scheduling columns, and build indexes; repeat runs are a no-op and failures roll back together. State clients check the schema version at startup.
• Claim the largest eligible unlocked artifact using ordered partial indexes and FOR UPDATE SKIP LOCKED. Preserve job, destination, and explicit partition filters; remove the idle-partition preference.
• Aggregate job summaries in SQL, index compact deadlines and stale work, and share periodic compaction maintenance across workers. Preserve immediate maintenance for worker -once.
• Combine periodic rewrite query and stage progress reports, use conditional SQL patches for progress, and surface failed rewrite heartbeats. Application writes maintain scheduling columns atomically with the JSON; no triggers. Connection pool configuration is unchanged.

Rollout: stop workers and pause uploads/state-changing admin commands, run the new binary's partforge migrate with the deployment's database settings, then start the new workers. Backfills and index builds hold table locks. Existing parts, progress, and lineage are retained. Direct SQL state changes must also maintain the derived columns.

Validation: go mod tidy, go test ./... with PostgreSQL integration tests enabled, go vet ./..., and ./e2e/run.sh passed. Tests cover migration rollback/backfill/idempotency, concurrent claims, ownership, scheduling-column consistency without triggers, and ordered query plans with 20,000 parts across 10 jobs. This is not a 2,000-worker load-test result.