Release v 1.5 - #282
Merged
Merged
Conversation
…BCs or props were not captured as part of earlier version. Default name of report includes study ID and timestamp.
…-DDS. Refreshing the schema is a deliberate.
…ord paginates it across pages.
…n reort with 'snapshot'.
…gn with data-definition-engine.
Added src/soa_builder/web/codelist_config.py to centralize the ~19 CDISC CT codelist C-codes that back dropdown selectors (Epoch Type, Arm Type, Role Type, Timing Type, etc.), each overridable through a new CODELIST_* variable in config.env instead of being hardcoded per router. The module loads config.env itself so overrides take effect regardless of import order relative to app.py. Updated all affected routers and utils.py loaders to read from the new constants instead of literal C-code strings, tightened two SQL statements to use bound parameters instead of f-string interpolation, and documented the new variables in README.md and CLAUDE.md. Added test coverage for the new config module and override behavior.
…d JSON. Introduced a legacy bug that I have now addressed. Narrative content is now added for the NCT01797120 study from a static file. No Narrative Content for any other SOAs.
pendingintent
added
bug
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR packages the v1.5 release of the SoA Workbench, adding new end-user features (DAIR generation, superscripts, intervention ↔ element linking), improving USDM export completeness, and centralizing configurable CDISC CT codelist selection for UI dropdowns.
Changes:
- Added DAIR UI + DOCX download endpoint (and dependency on
python-docx), with accompanying tests. - Introduced configurable CT codelist codes via
config.env/codelist_config.py, refactoring multiple routers/utils away from hardcoded codelist C-codes. - Expanded UI/export capabilities: activity/epoch/encounter superscripts (edit + HTML export), element intervention assignment, and study-specific narrative content injection for NCT01797120 in USDM generation.
Reviewed changes
Copilot reviewed 61 out of 77 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_routers_study_timing.py | Adds UI regression tests around the Study Timing page forms and redirect behavior. |
| tests/test_routers_study_interventions.py | Adds tests for configurable intervention role codelist behavior. |
| tests/test_routers_export_html.py | Adds tests for activity/epoch/encounter superscripts in UI and HTML export. |
| tests/test_routers_elements.py | Adds tests for element ↔ intervention assignment and USDM export wiring. |
| tests/test_routers_dair.py | Adds end-to-end tests for DAIR UI and DOCX download behavior. |
| tests/test_migrate_backfill_crf_href.py | Adds tests for a migration that rewrites dated CRF specialization hrefs to latest-version format. |
| tests/test_generate_narrative_content.py | Adds regression tests to prevent narrative content leaking into unrelated USDM exports. |
| tests/test_fetch_crf_specializations.py | Adds coverage for new CRF specializations fetching behavior (latest list vs per-BC-code). |
| tests/test_epoch_type_options.py | Adds test ensuring epoch type codelist URL honors configured codelist code. |
| tests/test_define_json_generator.py | Updates UI text assertion to match new “Data Definition Specification” wording. |
| tests/test_codelist_config.py | Adds tests for CODELIST_* env overrides and defaults. |
| src/usdm/generate_usdm.py | Wires narrative content + documentedBy into the generated USDM (and sets Study.id). |
| src/usdm/generate_narrative_content.py | New study-specific builder for NarrativeContent/NarrativeContentItem entities. |
| src/usdm/generate_elements.py | Adds element → intervention mapping and avoids N+1 queries. |
| src/usdm/create_define_json.py | Adjusts default schema validation path to schema/define.yaml and improves help text. |
| src/soa_builder/web/utils.py | Replaces hardcoded codelist C-codes with codelist_config constants across several helpers. |
| src/soa_builder/web/templates/study_timing.html | Adds section dividers and adjusts collapsible section persistence behavior. |
| src/soa_builder/web/templates/study_interventions.html | Resets “Add Study Intervention” form after HTMX request completes. |
| src/soa_builder/web/templates/soa_matrix_export.html | Renders epoch/encounter/activity superscripts in exported HTML. |
| src/soa_builder/web/templates/elements.html | Adds intervention multi-select to element create/update UI and displays assigned interventions. |
| src/soa_builder/web/templates/edit.html | Adds Study Name field; adds superscript edit affordances and rendering in the matrix UI. |
| src/soa_builder/web/templates/define_json.html | Renames Define-JSON UI copy to “Data Definition Specification (DDS) JSON”. |
| src/soa_builder/web/templates/dair.html | New DAIR page for selecting snapshots and triggering DOCX download. |
| src/soa_builder/web/templates/concept_groups.html | Splits concept groups UI into “CDISC” vs “Custom” sections and updates copy. |
| src/soa_builder/web/templates/base.html | Adds DAIR nav entry and renames Define-JSON nav text. |
| src/soa_builder/web/templates/_timings_section.html | Adjusts timing form layout class. |
| src/soa_builder/web/schemas.py | Adds intervention_ids to element create/update request models. |
| src/soa_builder/web/routers/visits.py | Refactors codelist code usage for visit environmental/contact mode fields. |
| src/soa_builder/web/routers/timings.py | Refactors timing codelist code usage to use configured constants. |
| src/soa_builder/web/routers/study_titles.py | Refactors study title type codelist code usage to use configured constants. |
| src/soa_builder/web/routers/study_interventions.py | Refactors role/type/unit codelist code usage to use configured constants. |
| src/soa_builder/web/routers/schedule_timelines.py | Refactors timing codelist code usage to use configured constants. |
| src/soa_builder/web/routers/roles.py | Refactors role type codelist code usage to use configured constants. |
| src/soa_builder/web/routers/organizations.py | Refactors organization type codelist code usage to use configured constants. |
| src/soa_builder/web/routers/objectives.py | Refactors objective/endpoint level codelist code usage to use configured constants. |
| src/soa_builder/web/routers/freezes.py | Updates amendment reason codelist usage to use the configured constant. |
| src/soa_builder/web/routers/epochs.py | Refactors epoch type codelist code usage to use configured constants. |
| src/soa_builder/web/routers/endpoints.py | Refactors endpoint level codelist code usage to use configured constants. |
| src/soa_builder/web/routers/elements.py | Adds element ↔ intervention junction table CRUD + UI wiring. |
| src/soa_builder/web/routers/dair.py | New router for DAIR UI page + DOCX streaming download. |
| src/soa_builder/web/routers/condition_assignments.py | Fixes UI delete route path prefix to /ui/.... |
| src/soa_builder/web/routers/arms.py | Refactors arm type/data-origin codelist code usage to use configured constants. |
| src/soa_builder/web/routers/amendments.py | Refactors amendment-related codelists to use configured constants. |
| src/soa_builder/web/routers/activities.py | Switches CRF specializations option loading to per-biomedical-concept filtering. |
| src/soa_builder/web/routers/_freeze_helpers.py | Enriches captured BC snapshot data and diff metadata (including data_unavailable). |
| src/soa_builder/web/migrate_database.py | Adds migrations for superscripts, element_intervention table, and CRF href backfill. |
| src/soa_builder/web/codelist_config.py | New central module defining configurable codelist constants via CODELIST_* env vars. |
| src/soa_builder/web/app.py | Registers DAIR router; adds superscript editing endpoints; updates CRF specialization fetching; updates migrations run at import. |
| src/soa_builder.egg-info/SOURCES.txt | Updates package source inventory to include new modules/tests. |
| src/soa_builder.egg-info/requires.txt | Adds python-docx requirement. |
| src/soa_builder.egg-info/PKG-INFO | Adds python-docx requirement and updates documentation content. |
| scripts/fetch_define_schema.py | Adds script to refresh vendored define.yaml schema from DataExchange-DDS. |
| schema/define.yaml.source | Adds provenance metadata for the vendored define.yaml schema. |
| README.md | Documents configurable CT codelist codes and their defaults. |
| pyproject.toml | Adds python-docx to project dependencies. |
| docs/usdm-validation-nct01797120-20260618.md | Adds validation report documentation for NCT01797120 export. |
| docs/usdm-validation-h2q-mc-lzzt-20260618.md | Adds validation report documentation for H2Q-MC-LZZT export. |
| docs/soa-workbench-improvement-plan.md | Adds an improvement/refactor plan document. |
| config.env | Adds CODELIST_* configuration entries and documentation comments. |
Comment on lines
56
to
60
| persistSection('study-timing-schedule-timelines'); | ||
| persistSection('study-timing-instances'); | ||
| persistSection('study-timing-timings'); | ||
| persistSection('study-timing-decision-instances'); | ||
| persistSection('study-condition-assignments'); | ||
| </script> |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
56
to
60
| persistSection('study-timing-schedule-timelines'); | ||
| persistSection('study-timing-instances'); | ||
| persistSection('study-timing-timings'); | ||
| persistSection('study-timing-decision-instances'); | ||
| persistSection('study-condition-assignments'); | ||
| </script> |
Comment on lines
65
to
69
| cur.execute( | ||
| "SELECT id,element_id,name,label,description,order_index,testrl,teenrl FROM element WHERE soa_id=? ORDER BY order_index", | ||
| "SELECT id,element_id,name,label,description,order_index,testrl,teenrl" | ||
| " FROM element WHERE soa_id=? ORDER BY order_index", | ||
| (soa_id,), | ||
| ) |
… element_id in memory instead of querying per row.
… other collapsible sections.
Comment on lines
+4512
to
4516
| "UPDATE soa SET name=?, study_id=?, study_label=?," | ||
| " study_description=? WHERE id=?", | ||
| ( | ||
| (name or "").strip() or None, | ||
| new_study_id, |
Comment on lines
+6400
to
+6407
| if cur.rowcount == 0: | ||
| conn.close() | ||
| raise HTTPException(404, "Activity not found") | ||
| cur.execute( | ||
| "SELECT name, label, superscript FROM activity WHERE soa_id=? AND id=?", | ||
| (soa_id, activity_id), | ||
| ) | ||
| row = cur.fetchone() |
Comment on lines
+6504
to
+6511
| if cur.rowcount == 0: | ||
| conn.close() | ||
| raise HTTPException(404, "Encounter not found") | ||
| cur.execute( | ||
| "SELECT name, label, superscript FROM visit WHERE soa_id=? AND id=?", | ||
| (soa_id, encounter_id), | ||
| ) | ||
| row = cur.fetchone() |
Comment on lines
+6620
to
+6627
| if cur.rowcount == 0: | ||
| conn.close() | ||
| raise HTTPException(404, "Epoch not found") | ||
| cur.execute( | ||
| "SELECT epoch_label, name, superscript FROM epoch WHERE soa_id=? AND id=?", | ||
| (soa_id, epoch_id), | ||
| ) | ||
| row = cur.fetchone() |
| "DELETE FROM element_intervention WHERE soa_id=? AND element_id=?", | ||
| (soa_id, element_id), | ||
| ) | ||
| for idx, iuid in enumerate(intervention_ids, start=1): |
Comment on lines
297
to
299
| study = { | ||
| "id": None, | ||
| "id": f"Study_{soa_id}", | ||
| "extensionAttributes": [], |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Latest release (version 1.5) of the SoA Workbench