Prove CORA starts, by starting it - #779
Merged
Merged
Conversation
Nothing in the suite booted the app. Every create_app() test runs app_env=test, which takes build_kernel's in-memory branch, skips drain_projections for want of a pool, and keeps the permissive kernel defaults. So the Postgres branch of the composition root, the one a deployment actually walks, first executed when someone deployed. That branch is where startup does its work: about thirty seed, drain and register calls in a fixed order, each depending on what ran before. test_seed_order_catalog_before_agents guards two of those orderings by reading main.py and asserting about the order of calls in main.py, which cannot execute them. Drives the real entry point rather than repeating its steps. A test that reproduced the sequence would assert about its own copy: delete a line from main.py and the copy still has it, so the test stays green while the deployment breaks. Two unrelated ecosystems reached the same conclusion and both are cited in the module docstring. Spring Boot builds the test context through SpringApplication, the production path, with useMainMethod=ALWAYS for applications whose main does real work. ASP.NET Core's WebApplicationFactory boots the real entry point and applies test overrides AFTER the app's own Program.cs, so swapping the database means finding and REMOVING the real registration rather than never performing it. Both run the real root and override one input. create_app's settings hook, which already existed for this purpose, is that one input. app_env stays local: not test, so build_kernel opens a pool and seeds, and not production-tier, so the auth and signing boot refusals stay clear. Everything else is the deployment's own wiring. This closes the gap #777 left open and named. Verified by mutation: deleting main.py's language_model_lookup_factory binding fails both tests here, while 52,328 unit and architecture tests, the seed-order guard, the adapter integration test and the whole catalog chain suite all stay GREEN. Nothing else in the repository catches it. The second test earns its place separately. Breaking seed_agent's ConcurrencyError idempotency fails only the restart test, since the first test never sees anything but an empty database, and a seed that lost idempotency would strand a running deployment on its next restart. Scope is deliberately one question, "does it come up". Both sources warn that broad tests become catch-alls that localise nothing, so the WHY stays with the narrower tests and the docstring says to read the chain test when this goes red. The fixture drops WITH (FORCE) because a boot that raises partway leaves the pool open, which turned a failing assertion into a failure plus a teardown error in exactly the run someone has to read. 1,379 integration + 52,360 unit and architecture passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Coverage reportClick to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item 3b. Closes the gap #777 left open and named.
The gap
Nothing in the suite booted the app. Every
create_app()test runsapp_env=test, which takesbuild_kernel's in-memory branch, skipsdrain_projectionsfor want of a pool, and keeps the permissive kernel defaults. So the Postgres branch of the composition root, the one a deployment actually walks, first executed when someone deployed.That branch is where startup does its work: about thirty seed, drain and register calls in a fixed order, each depending on what ran before.
test_seed_order_catalog_before_agentsguards two of those orderings by readingmain.pyand asserting about the order of calls inmain.py, which cannot execute them.Why it drives the real entry point
A test that reproduced the startup sequence would assert about its own copy. Delete a line from
main.pyand the copy still has it, so the test stays green while the deployment breaks. Same blindness as a guard reading the file it guards, one level up.Checked this against prior art before building it, and two unrelated ecosystems had reached the same conclusion:
SpringApplication, the production path, and offersuseMainMethod=ALWAYSprecisely for applications whosemaindoes work affecting the result. A nested@TestConfigurationis used in addition to the primary configuration, where a plain@Configurationwould replace it.WebApplicationFactoryboots the real entry point and runs test overrides AFTER the app's ownProgram.cs, so swapping the database means finding and REMOVING the real registration rather than never performing it.Both run the real root and override one input. That research also corrected my plan: I had framed the choice as "run it untouched" versus "rebuild it", and the actual pattern is neither.
create_app'ssettingshook, which already existed and is documented for this purpose, is that one input.app_envstayslocal: nottest, sobuild_kernelopens a pool and seeds, and not production-tier, so the auth and signing boot refusals stay clear. Everything else is the deployment's own wiring.Verification
Mutation, the headline. Deleting
main.py'slanguage_model_lookup_factorybinding fails both tests here, while 52,328 unit and architecture tests, the seed-order guard, the adapter integration test and the whole catalog chain suite all stay green. Nothing else in the repository catches it. That is the #777 gap, stated as an experiment and now closed.Mutation, the second test. Breaking
seed_agent'sConcurrencyErroridempotency fails only the restart test. The first test never sees anything but an empty database, so it cannot catch a seed that would strand a running deployment on its next restart.1,379 integration + 52,360 unit and architecture passing.
Scope
Deliberately one question, "does it come up". Both sources warn that broad tests become catch-alls that localise nothing (Microsoft: limit them to the most important infrastructure scenarios, prefer a unit test wherever either would do). So the WHY stays with the narrower tests, and the docstring tells a future reader to open the chain test when this goes red.
The fixture drops
WITH (FORCE), unlike its siblings, because a boot that raises partway leaves the app's pool open and a plainDROPthen fails, turning a failing assertion into a failure plus a teardown error in exactly the run someone has to read.🤖 Generated with Claude Code