feat: Domain specific language#354
Closed
Siel wants to merge 7 commits into
Closed
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an initial “DSL runtime” workflow for Pmetrics, shifting model compilation/validation, fitting, and simulation to operate from a generated DSL model source (rather than compiled .pmx artifacts), and updates both the Rust backend and the R-facing APIs/tests/docs accordingly.
Changes:
- Reworked Rust execution/configuration to build estimation problems from runtime DSL metadata (including new cache/progress settings and logging controls).
- Updated R wrappers and tests to use
PM_model$dsl()/ runtime validation, added new runtime-focused test coverage (PM_load DSL roundtrip, fixed error models, fit logging/progress/cache flags). - Regenerated/updated package documentation and namespace entries to reflect the new DSL-based APIs.
Reviewed changes
Copilot reviewed 70 out of 81 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/testthat/test-pmcore-pharmsol-migration.R | Replaces Rust-source inspection tests with DSL text assertions and adds debug_dsl() coverage. |
| tests/testthat/test-pm-model-guards.R | Updates guard tests to assert DSL output / DSL errors rather than binary-path state. |
| tests/testthat/test-pm-load-dsl-roundtrip.R | New tests for PM_load() restoring a run from saved model.dsl, and sim working from loaded DSL. |
| tests/testthat/test-pm-load-binary-fallback.R | Removes legacy tests for .pmx binary fallback on load. |
| tests/testthat/test-pm-fit-prior-parameters.R | Updates mocked Rust fit() interface to accept a model source string (DSL) instead of a binary path. |
| tests/testthat/test-fixed-error-models.R | New runtime fit tests ensuring fixed additive/proportional error models remain fixed, including file import markers. |
| tests/testthat/test-fit-progress-option.R | New subprocess-based test verifying progress toggles likelihood-matrix progress output. |
| tests/testthat/test-fit-log-controls.R | New subprocess-based tests verifying log file writing, stdout suppression, and log level filtering. |
| tests/testthat/test-fit-cache-option.R | New runtime fit test ensuring cache = FALSE is accepted and produces outputs. |
| tests/testthat/test-data-models.R | Updates expectations from binary existence to DSL generation/compile behavior; asserts model.dsl input is written. |
| tests/testthat/test-book-examples.R | Updates book example assertions to align with new arg_list/DSL runtime behavior. |
| tests/testthat/test-analytical-indexing.R | Switches analytical indexing checks from generated Rust code to DSL assertions. |
| tests/testthat/helper-models.R | Removes example temp cleanup helpers; keeps shared test helper functions. |
| tests/.dsl_runtime_validation.log | Adds a captured test run log for DSL runtime validation (new artifact file). |
| src/rust/src/settings.rs | Refactors settings parsing to build runtime fit builders from equation metadata; adds cache/progress support and tests. |
| src/rust/src/logs.rs | Adds global log controls supporting stdout/file output and log level filtering. |
| src/rust/src/executor.rs | Switches execution to use runtime equation metadata and DSL-based model sources rather than loading compiled artifacts. |
| src/rust/Cargo.toml | Updates pmcore dependency/features for DSL JIT; adjusts Rust dependencies/dev-dependencies. |
| R/PMbuild.R | Makes PM_build() a no-op informational function under the DSL runtime. |
| R/PM_valid.R | Fixes quantile handling in validation output reshaping. |
| R/PM_result.R | Simplifies PM_load() and removes binary-copy fallback; small loop style fix. |
| R/PM_post.R | Updates expected prediction file name to predictions.csv and adjusts function signatures formatting. |
| R/PM_pop.R | Updates expected prediction file name to predictions.csv and adjusts function signatures formatting. |
| R/PM_parse.R | Updates comments and parsing assumptions for renamed output files; removes binary-path field from parsed core object. |
| R/PM_op.R | Updates expected prediction file name to predictions.csv and adjusts function signatures formatting. |
| R/PM_final.R | Adds fallbacks when settings/prior fields are missing; adjusts settings handling during final object construction. |
| R/PM_cycle.R | Updates cycle parsing to iterations.csv and predictions to predictions.csv; improves parameter name fallback logic. |
| R/PM_cov.R | Updates covariate file name to covariates.csv; adjusts function signatures formatting. |
| R/extendr-wrappers.R | Regenerates wrappers to accept DSL model source (and optional solver) for fit/sim/parameter queries; adds validate_model_source(). |
| NAMESPACE | Exports validate_model_source, adds print.pm_dsl S3 method, removes obsolete exports. |
| man/validate_model_source.Rd | New generated documentation for runtime DSL validation wrapper. |
| man/temporary_path.Rd | Removes generated documentation for obsolete temporary_path(). |
| man/simulate_one.Rd | Updates generated docs to reflect DSL model source + optional solver argument. |
| man/simulate_all.Rd | Updates generated docs to reflect DSL model source + optional solver argument. |
| man/round2.Rd | Fixes/updates Rd links for base rounding documentation. |
| man/PM_valid.Rd | Regenerated R6 documentation formatting/anchors for PM_valid. |
| man/PM_pta.Rd | Regenerated R6 documentation formatting/anchors for PM_pta. |
| man/PM_pri.Rd | Regenerated R6 documentation formatting/anchors for PM_pri. |
| man/PM_post.Rd | Regenerated R6 documentation formatting/anchors for PM_post. |
| man/PM_pop.Rd | Regenerated R6 documentation formatting/anchors for PM_pop. |
| man/PM_opt.Rd | Regenerated R6 documentation formatting/anchors for PM_opt. |
| man/PM_op.Rd | Regenerated R6 documentation formatting/anchors for PM_op. |
| man/PM_fit.Rd | Regenerated R6 documentation formatting/anchors for PM_fit. |
| man/PM_final.Rd | Regenerated R6 documentation formatting/anchors for PM_final. |
| man/PM_data.Rd | Regenerated R6 documentation formatting/anchors for PM_data. |
| man/PM_cycle.Rd | Regenerated R6 documentation formatting/anchors for PM_cycle. |
| man/PM_cov.Rd | Regenerated R6 documentation formatting/anchors for PM_cov. |
| man/dummy_compile.Rd | Removes generated documentation for obsolete dummy_compile(). |
| man/compile_model.Rd | Removes generated documentation for obsolete compile_model(). |
| man/plot.PMvalid.Rd | Updates generated doc links/anchors for plotting (PMvalid). |
| man/plot.PM_valid.Rd | Updates generated “Other PMplots” link formatting. |
| man/plot.PM_sim.Rd | Updates generated “Other PMplots” link formatting and plotly schema links. |
| man/plot.PM_pta.Rd | Updates plotly link targets and “Other PMplots” link formatting. |
| man/plot.PM_post.Rd | Updates generated “Other PMplots” link formatting. |
| man/plot.PM_pop.Rd | Updates generated “Other PMplots” link formatting. |
| man/plot.PM_opt.Rd | Updates generated “Other PMplots” link formatting. |
| man/plot.PM_op.Rd | Updates plotly schema link target and “Other PMplots” link formatting. |
| man/plot.PM_model.Rd | Updates generated “Other PMplots” link formatting. |
| man/plot.PM_final.Rd | Updates plotly schema link target and “Other PMplots” link formatting. |
| man/plot.PM_data.Rd | Updates generated “Other PMplots” link formatting. |
| man/plot.PM_cycle.Rd | Updates plotly schema link target and “Other PMplots” link formatting. |
| man/plot.PM_cov.Rd | Updates plotly schema link target and “Other PMplots” link formatting. |
| man/model_parameters.Rd | Updates docs to reflect parameters extracted from DSL metadata (not compiled model path). |
| man/mic1.Rd | Updates readr link target for read_csv() documentation page. |
| man/fit.Rd | Updates docs to reflect DSL model source + optional solver argument. |
| inst/Examples/Runs/sylvain_sim/theta.csv | Adds an example theta file for simulation run output. |
| inst/Examples/Rscript/sylvain.R | Adds an example model definition script used by the sylvain simulation example. |
| inst/Examples/Rscript/sylvain_sim.R | Adds an example script to run a simulation and write theta/simulation outputs. |
| DESCRIPTION | Adds/updates RoxygenNote field. |
Files not reviewed (7)
- man/PM_cov.Rd: Language not supported
- man/PM_cycle.Rd: Language not supported
- man/PM_data.Rd: Language not supported
- man/PM_final.Rd: Language not supported
- man/PM_fit.Rd: Language not supported
- man/PM_model.Rd: Language not supported
- man/PM_op.Rd: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
11
to
13
| extendr-api = "=0.9.0" | ||
| pmcore = { version = "=0.25.2", features = ["exa"] } | ||
| libloading = "0.9" | ||
| pmcore = { version = "=0.25.2", path = "../../../PMcore-structure", features = ["dsl-jit"] } | ||
|
|
Comment on lines
259
to
263
| if (file.exists(file.path(path, "settings.json"))) { | ||
| config <- jsonlite::fromJSON(file.path(path, "settings.json")) | ||
| } else if (inherits(data, "PM_final")) { # file not there, and already PM_final | ||
| } else if (inherits(data, "PM_final") & !is.null(data$data)) { # file not there, and already PM_final | ||
| class(data$data) <- c("PM_final_data", "data.frame") | ||
| return(data$data) |
Comment on lines
1
to
5
| test_that <- function(desc, code) { | ||
| desc <- as.character(desc) | ||
| message(sprintf("[test] %s", desc)) | ||
| testthat::test_that(desc, code) | ||
| } |
Comment on lines
413
to
418
| n_cyc <- nrow(dat$mean) | ||
| n_out <- max(res$op$outeq) | ||
| dat$gamlam <- dat$gamlam |> select(starts_with("add") | starts_with("prop")) | ||
| if (ncol(gamlam) == 1 & n_out > 1) { | ||
| if (ncol(gamlam) == 1 && n_out > 1) { | ||
| gamlam <- cbind(gamlam, replicate((n_out - 1), gamlam[, 1])) | ||
| } |
Comment on lines
+1
to
+5
| ✔ | F W S OK | Context | ||
|
|
||
| ⠏ | 0 | analytical-indexing | ||
| ✔ | 1 0 | analytical-indexing | ||
|
|
Comment on lines
+17
to
+19
|
|
||
| [patch."https://github.com/LAPKB/PMcore"] | ||
| pmcore = { path = "../PMcore-structure" } |
Comment on lines
10
to
13
| [dependencies] | ||
| extendr-api = "=0.9.0" | ||
| pmcore = { version = "=0.25.2", features = ["exa"] } | ||
| libloading = "0.9" | ||
| pmcore = { version = "=0.25.2", git = "https://github.com/LAPKB/PMcore", rev = "668dd305436c75d4fdef29138261540c114c0c34", features = ["dsl-jit"] } | ||
|
|
Comment on lines
482
to
487
| n_cyc <- nrow(dat$mean) | ||
| n_out <- max(res$op$outeq) | ||
| dat$gamlam <- dat$gamlam |> select(starts_with("add") | starts_with("prop")) | ||
| if (ncol(gamlam) == 1 & n_out > 1) { | ||
| if (ncol(gamlam) == 1 && n_out > 1) { | ||
| gamlam <- cbind(gamlam, replicate((n_out - 1), gamlam[, 1])) | ||
| } |
Comment on lines
+543
to
+563
| path <- normalizePath(path, mustWork = FALSE) | ||
| dir.create(path, recursive = TRUE, showWarnings = FALSE) | ||
|
|
||
| if (!is.null(self$cycle$data)) { | ||
| write.csv(self$cycle$data, file.path(path, "iterations.csv"), row.names = FALSE) | ||
| } | ||
|
|
||
| if (!is.null(self$final$data)) { | ||
| write.csv(self$final$data$theta, file.path(path, "theta.csv"), row.names = FALSE) | ||
| } | ||
|
|
||
| if (!is.null(self$post$data)) { | ||
| write.csv(self$post$data$posterior, file.path(path, "posterior.csv"), row.names = FALSE) | ||
| } | ||
|
|
||
| if (!is.null(self$pop$data)) { | ||
| write.csv(self$pop$data$pop, file.path(path, "predictions.csv"), row.names = FALSE) | ||
| } | ||
|
|
||
| if (!is.null(self$cov$data)) { | ||
| write.csv(self$cov$data$covariates, file.path(path, "covariates.csv"), row.names = FALSE) |
Comment on lines
+555
to
+563
| write.csv(self$post$data$posterior, file.path(path, "posterior.csv"), row.names = FALSE) | ||
| } | ||
|
|
||
| if (!is.null(self$pop$data)) { | ||
| write.csv(self$pop$data$pop, file.path(path, "predictions.csv"), row.names = FALSE) | ||
| } | ||
|
|
||
| if (!is.null(self$cov$data)) { | ||
| write.csv(self$cov$data$covariates, file.path(path, "covariates.csv"), row.names = FALSE) |
| }, | ||
| error = function(e) { | ||
| try(close_live_session(live_session$session_id), silent = TRUE) | ||
| stop(e) |
Comment on lines
+237
to
+244
| if file_enabled { | ||
| let config_handle = global_log_config(); | ||
| let mut config = config_handle.lock().unwrap(); | ||
| if let Some(file) = config.file.as_mut() { | ||
| let _ = writeln!(file, "{}", plain_message); | ||
| let _ = file.flush(); | ||
| } | ||
| } |
Comment on lines
+10
to
+13
| testthat::test_that("live report session launches app and connects before fit", { | ||
| reports_path <- testthat::test_path("..", "..", "..", "Pmetricsreports") | ||
| pkgload::load_all(reports_path, quiet = TRUE, export_all = FALSE) | ||
|
|
Comment on lines
+32
to
+35
| testthat::test_that("live report close leaves time for finished report handoff", { | ||
| reports_path <- testthat::test_path("..", "..", "..", "Pmetricsreports") | ||
| pkgload::load_all(reports_path, quiet = TRUE, export_all = FALSE) | ||
|
|
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 78 out of 82 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
R/PM_parse.R:16
- The roxygen block documenting
PM_parse()is currently placed immediately beforeload_pm_parse_fit_object(), so roxygen is generating theload_pm_parse_fit_objectmanual page withPM_parse’s title/arguments (e.g., it documents a non-existentwriteargument). Split the documentation into two blocks (or move thePM_parseblock directly abovePM_parse()), and regenerateman/+NAMESPACEso they stay consistent.
#' @title Parse Pmetrics output
#' @description
#' `r lifecycle::badge("experimental")`
#'
#' A flexible parser for Pmetrics output
#' @details
#' Currently written for the Rust implementation of NPAG
#' @param path The directory containing the output from the Rust-implementation of NPAG
#' @param fit The relative path to a "fit.rds" file, which is normal output after a fit containing the data and model used.
#' @param write If `TRUE` (default), saves the output as "PMout.Rdata" in the specified path.
#' @return The output of `PM_parse` is a list containing the following elements
Comment on lines
+58
to
+60
| if (!requireNamespace("PmetricsReports", quietly = TRUE)) { | ||
| return(NULL) | ||
| } |
Comment on lines
+326
to
+327
| result <- output2List(Out = get(load(found))) | ||
| PM_result$new(result, path = dirname(found), quiet = TRUE) |
Comment on lines
+119
to
+121
| let mut state = self.state.lock().expect("live session state lock poisoned"); | ||
| state.backlog.push(payload.clone()); | ||
|
|
Comment on lines
+1
to
+5
| ✔ | F W S OK | Context | ||
|
|
||
| ⠏ | 0 | analytical-indexing | ||
| ✔ | 1 0 | analytical-indexing | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.