Result Stability Checks for Empirical R Projects
resultcheck provides lightweight helpers for checking whether empirical results remain unchanged across code revisions, platform differences, and package updates. Call snapshot() on key outputs (models, tables, derived datasets) in your analysis scripts to detect unintended result drift automatically during CI or local testing.
install.packages("resultcheck")# install.packages("devtools")
devtools::install_github("kv9898/resultcheck")The package supports a two-phase workflow:
-
Interactive development — run your analysis script and call
snapshot()on objects you care about. On first run the snapshot is saved as a human-readable.mdfile. On subsequent interactive runs, differences are shown and you are prompted to update. -
Automated testing — wrap your script in
setup_sandbox()/run_in_sandbox()/cleanup_sandbox(). Insiderun_in_sandbox(),snapshot()switches to testing mode: it errors immediately if a snapshot is missing or has changed, making the test fail.
with_example() can generate this layout for documentation/testing under tempdir():
myproject/
├── _resultcheck.yml
├── analysis.R
└── tests/
├── _resultcheck_snaps/
│ └── analysis/
│ ├── model.md
│ └── model_mismatch.md
└── testthat/
└── test-analysis.R
model <- lm(mpg ~ wt, data = mtcars)
resultcheck::snapshot(model, "model")library(testthat)
library(resultcheck)
test_that("analysis produces stable results", {
sandbox <- setup_sandbox()
on.exit(cleanup_sandbox(sandbox), add = TRUE)
expect_true(run_in_sandbox("analysis.R", sandbox))
})To try this quickly without creating files in your current project:
resultcheck::with_example({
sandbox <- setup_sandbox()
on.exit(cleanup_sandbox(sandbox), add = TRUE)
stopifnot(isTRUE(run_in_sandbox("analysis.R", sandbox)))
})Creates or verifies a snapshot of any R object.
- First interactive run: saves the object as a human-readable
.mdfile undertests/_resultcheck_snaps/<script>/at the project root by default. - Subsequent interactive runs: shows a diff and prompts to update.
- Inside
run_in_sandbox(): errors if the snapshot is missing or doesn't match. - When writing snapshots interactively: warns and shows the exact output path.
You can override the default snapshot directory in _resultcheck.yml:
snapshot:
max_print: 1000
dir: "custom/snapshots/path"
method: "print + str"
method_defaults_file: "snapshot-method-overrides.R"
method_by_class:
lm: "summary"max_print sets the base R printing limit for each snapshot method (default:
1,000 entries). Set it in _resultcheck.yml rather than relying on the session's
options(max.print = ...). It must be a whole number from 1 to 2,147,483,647.
The limit counts entries, not rows or bytes: a 200-row, seven-column data frame
contains 1,400 entries and needs a higher limit, such as max_print: 2000. Larger base-printed objects may be truncated, with an
omission notice; changes in omitted values may not be detected. Increase the
limit when full output is needed, or select a meaningful summary method.
Class-specific methods such as tibble printing can have their own limits, and
custom methods can explicitly override the print limit. The caller's R options
are restored afterwards.
Snapshot serialization also temporarily sets useFancyQuotes = FALSE, so base R
quotation marks (including model-summary significance legends) are consistent
between interactive sessions and Quarto. The caller's setting is restored even
if a snapshot method fails. Custom methods may explicitly choose other formatting.
The method argument controls how the object is serialized:
| Value | Behavior |
|---|---|
NULL (default) |
Captures both print() and str() |
print |
Only print() output is captured |
str |
Only str() output is captured |
length |
Any callable function can be used |
stats::coef |
Namespaced callables are supported |
list(print = print, summary = summary) |
Runs multiple functions in order |
When a list of methods is used, section headers are taken directly from the
method names (or list names), e.g. ## print, ## summary.
If method is omitted, defaults are resolved as:
- class override from
snapshot.method_by_class, - global default from
snapshot.method, - fallback to
list(print = print, str = str).
Config methods are parsed from strings in _resultcheck.yml, so values like
"print + str" or "stats::coef" are converted to callable methods.
You can also define class defaults in a separate R file:
# snapshot-method-overrides.R
method_by_class <- list(
lm = "summary",
glm = "summary"
)Snapshots are plain text and intended to be committed to version control.
Creates a temporary directory and copies the listed files and/or directories into it, preserving their path structure relative to the project root. Directories are copied recursively. Snapshot files do not need to be listed.
Runs an R script inside the sandbox. The working directory is set to the sandbox, but find_root() and snapshot() automatically resolve back to the original project root so snapshots are found correctly.
Returns TRUE invisibly on success, so you can use expect_true(run_in_sandbox(...)) directly in testthat.
Removes the sandbox directory. Omit the argument to clean up the most recently created sandbox.
Locates the project root by searching upward for a _resultcheck.yml (or legacy resultcheck.yml), .Rproj, or .git marker. Called automatically by snapshot() and setup_sandbox().
Place an empty _resultcheck.yml at your project root to make detection reliable:
# _resultcheck.ymlCreates a temporary example project in tempdir(), sets the working directory there while evaluating code, then cleans up automatically.
MIT © Dianyi Yang