Repository navigation
LMMs default to Satterthwaite df - #1276
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Robust covariance CIs are ignored under the new default, and three active snapshot expectations were deleted.
5 open findings
What changed in this PR
Changes linear lme4 models to default to Satterthwaite degrees of freedom when lmerTest is available.
Changes:
- Adds
lmerModS3 wrappers selecting Satterthwaite defaults. - Updates tests, snapshots, documentation, and package metadata.
- Adjusts related lavaan and random-effects tests.
| File | Description |
|---|---|
R/methods_lme4.R |
Implements the new defaults. |
R/1_model_parameters.R |
Documents Satterthwaite behavior. |
NAMESPACE |
Registers new S3 methods. |
DESCRIPTION |
Bumps package and roxygen2 versions. |
NEWS.md |
Announces the breaking change. |
tests/testthat/test-lmerTest.R |
Tests mixed-model defaults. |
tests/testthat/test-random_effects_ci.R |
Limits assertions to random effects. |
tests/testthat/test-lavaan.R |
Updates dependency and expected terminology. |
tests/testthat/_snaps/model_parameters.mixed.md |
Records Satterthwaite output. |
tests/testthat/_snaps/glmer.md |
Updates robust mixed-model output. |
man/p_value.Rd |
Updates generated documentation. |
man/model_parameters.Rd |
Updates generated documentation. |
man/model_parameters.glmmTMB.Rd |
Updates generated documentation. |
man/model_parameters.default.Rd |
Updates generated documentation. |
man/model_parameters.brmsfit.Rd |
Updates generated documentation. |
man/ci.default.Rd |
Updates generated documentation. |
Files not reviewed (6)
- man/ci.default.Rd: Generated file
- man/model_parameters.Rd: Generated file
- man/model_parameters.brmsfit.Rd: Generated file
- man/model_parameters.default.Rd: Generated file
- man/model_parameters.glmmTMB.Rd: Generated file
- man/p_value.Rd: Generated file
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
There are some failing tests, related to lmerTest,
Maybe, once lmerTest is loaded, they call the new
|
|
All tests for linear mixed models now should have |
|
|
|
Can't we just add |
|
The current implementation seems to fail for lmer-Models now, see #1276 (comment) |
jmgirard
left a comment
jmgirard
left a comment
There was a problem hiding this comment.
Thanks, this is a good change. I found one regression and a few smaller things.
Satterthwaite default fails when the model data is gone. If the data is not in scope, lmerTest cannot rebuild the deviance function. This happens, for example, with a model saved with saveRDS() and loaded in a new session. Then model_parameters(), ci() and p_value() error, where main gives Wald results:
library(lme4)
d <- sleepstudy
m <- lmer(Reaction ~ Days + (1 | Subject), data = d)
saveRDS(m, "m.rds")
# new session
m <- readRDS("m.rds")
parameters::model_parameters(m)
#> Error: Unable to extract deviance function from model fitCould the default fall back to "wald" (with an alert if verbose = TRUE) when Satterthwaite fails? An explicit ci_method = "satterthwaite" can still error.
Install prompt and repeated warning without lmerTest. insight::check_if_installed() defaults to prompt = interactive(), so in an interactive session each call on an lme4 model asks to install lmerTest. The reprex also shows the warning three times for one model_parameters() call, and it ignores verbose. Maybe pass prompt = FALSE and quietly = !verbose, and warn only once per call?
Files that look committed by accident. The PR deletes 24 snapshot files (equivalence_test/*.svg, tinyplot/*.svg, include_reference.md, lcmm.md, mipo.md, plm.md, weightit.md, and others), adds tests/testthat/_snaps/glmmTMB.new.md, and removes 8 lines from man/dominance_analysis.Rd. This looks like a local devtools::test() / document() run with some Suggests packages missing. Can these be restored?
Minor. test-random_effects_ci.R now uses effects = "random", so the fixed-effect rows with ci_random = TRUE are no longer checked. Pinning the new Satterthwaite values would keep that coverage.
|
I have updated some files and will commit soon |
|
Still not working, this is failing: > data(iris)
> set.seed(1234)
> iris$grp <- as.factor(sample(1:3, nrow(iris), replace = TRUE))
> model <- lme4::lmer(
+ Sepal.Length ~ Species * Sepal.Width + Petal.Length + (1 | grp),
+ data = iris
+ )
> params <- model_parameters(
+ model,
+ standardize = "refit",
+ effects = "fixed",
+ vcov = "CR",
+ ci_method = "wald",
+ vcov_args = list(type = "CR1", cluster = iris$grp),
+ verbose = FALSE
+ )
Error in `lmerTest::as_lmerModLmerTest()`:
! Unable to extract deviance function from model fit
Hide Traceback
▆
1. ├─parameters::model_parameters(...) at [tests/testthat/test-model_parameters_std_mixed.R:234:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
2. ├─parameters:::model_parameters.lmerMod(...) at [parameters/R/1_model_parameters.R:395:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
3. │ └─base::eval.parent(cl) at [parameters/R/methods_lme4.R:127:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
4. │ └─base::eval(expr, p)
5. │ └─base::eval(expr, p)
6. └─parameters (local) `<fn>`(...)
7. ├─base::do.call(".extract_parameters_mixed", fun_args) at [parameters/R/methods_coxme.R:140:7](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
8. └─parameters:::.extract_parameters_mixed(...)
9. ├─base::suppressMessages(do.call("ci", fun_args)) at [parameters/R/extract_parameters.R:528:7](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
10. │ └─base::withCallingHandlers(...)
11. ├─base::do.call("ci", fun_args) at [parameters/R/extract_parameters.R:528:7](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
12. ├─bayestestR::ci(...)
13. ├─parameters:::ci.lmerMod(...) at [bayestestR/R/ci.R:70:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
14. │ └─base::eval.parent(cl) at [parameters/R/methods_lme4.R:146:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
15. │ └─base::eval(expr, p)
16. │ └─base::eval(expr, p)
17. └─parameters (local) `<fn>`(...)
18. └─parameters:::.ci_generic(...) at [parameters/R/methods_lme4.R:39:5](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
19. └─parameters::ci_satterthwaite(model, ci = ci) at [parameters/R/ci_generic.R:44:5](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
20. └─parameters::dof_satterthwaite(model) at [parameters/R/ci_satterthwaite.R:4:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
21. ├─insight::get_df(model, "satterthwaite") at [parameters/R/dof_satterthwaite.R:4:3](vscode-file://vscode-app/c:/Program%20Files/Positron/resources/app/out/vs/code/electron-browser/workbench/workbench.html#)
22. └─insight:::get_df.lmerMod(model, "satterthwaite")
23. ├─insight:::.degrees_of_freedom_satterthwaite(x)
24. └─insight:::.degrees_of_freedom_satterthwaite.lmerMod(x)
25. └─lmerTest::as_lmerModLmerTest(x) |
Yes - can you think of any way other than checking if
But we do want them to prompted, no? I can try some trickery like this to have that specific prompt only happen once every say 8 hours? foo <- function() {
.last_lmerTest_prompt <- getOption(".last_lmerTest_prompt", Sys.time() - 8*3600)
.eight_hours_ago <- Sys.time() - 8*3600
if (.last_lmerTest_prompt <.eight_hours_ago) {
insight::check_if_installed("zzz", stop = FALSE)
options(.last_lmerTest_prompt = Sys.time())
}
invisible(NULL)
}
foo()
#> Warning: Package `zzz` required for this function to work.
#> Please install it by running `install.packages("zzz")`.
foo()
foo()Created on 2026-10-15 with reprex v2.1.1
@jmgirard Can you take care of that? I'm not sure how to revert that?
Fixed.
That's related to Jeffery's first point. |
|
I restored On the fallback: I do not see a better check than trying the conversion. Whether if (installed) {
converted <- tryCatch(
lmerTest::as_lmerModLmerTest(model),
error = function(e) NULL
)
if (!is.null(converted)) {
return("satterthwaite")
}
}
return(fallback)An alert for That example also has a second problem. It passes |
|
Okay I think I have it resolved now - this is the logic and it seems to work:
Then, for setting the .attempt_satterthwaite_method <- function(
method,
model,
effects,
fallback = "wald",
bootstrap = FALSE,
verbose = TRUE
) {
# Cases where we don't want to interfere
if (!is.null(method) || effects == "random" || isTRUE(bootstrap)) {
return(method)
}
if (inherits(model, "lmerModLmerTest")) {
return("satterthwaite")
}
# Then the model is DEFNITLY NOT coercible because we already tried to coerce it
if (requireNamespace("lmerTest", quietly = TRUE)) {
return(fallback)
}
# Model is potentially coercible if lmerTest were installed?
installed <- insight::check_if_installed(
package = "lmerTest",
reason = "for Satterthwaite df (default)",
stop = FALSE,
prompt = interactive() && isTRUE(verbose)
)
# This is a workaround for the fact that check_if_installed() returns FALSE
# even if the package is not installed, but the user chooses to install it. In
# that case, we need to check again if the package is actually installed.
if (!installed) {
installed <- requireNamespace("lmerTest", quietly = TRUE)
}
# if lmerTest is newly installed, and the model is coercible...
if (installed && inherits(.attempt_to_lmerModLmerTest(model, "lmerModLmerTest"))) {
return("satterthwaite")
}
return(fallback)
}I also pass I don't know why (locally) snapshots for the CI method for random effect has changed. @strengejacke any idea where this is happening? We're so close guys I can taste it! (Had I know it would be this hard I probably wouldn't have attempted this PR 😅) |
… CIs - The lmerMod wrappers share one helper. It uses Satterthwaite df when no method is given and lmerTest is installed. If that fails (for example, a saved model whose data is gone), it runs the call again with the merMod default and shows a message. Errors that the default method also raises are reported unchanged. - The wrappers no longer coerce the model to lmerModLmerTest. insight::get_df() coerces it anyway, and keeping the lmerMod class restores the footer line about random effect variances. - No install prompt; the lmerTest check respects verbose. - Satterthwaite CIs now use a robust vcov when one is given, and .extract_parameters_mixed() passes ci_method as `method` to the nested ci() and p_value() calls. - The random effect variance footer also shows for lmerModLmerTest models. - Tests for robust vcov, the fallback without model data, and missing lmerTest.
|
@mattansb I pushed a commit (160dabf) that fixes the remaining issues from my review. Please pull before your next change. What changed:
The mixed-model tests pass locally. |
|
Okay, I'm now thinking we're over engerring this. Going back to the main justification for this change - having
I think this will both save a lot of checking on our side and will be more aligned with the user's behavior (if they chose to use What do you think @jmgirard ? |
|
@mattansb Agreed, let's limit the default to Since lmerTest fits already store what Satterthwaite needs, most of the fallback in I'd keep the For the |
|
Okay, done! These are the only tests that are failing - @jmgirard maybe you can figure those out? 
|
# Conflicts: # DESCRIPTION
- The robust vcov test now fits the model with lmerTest::lmer(), because lme4::lmer() models no longer get Satterthwaite by default. - The "message once" test resets the option first, because an earlier test in the same session can already have shown the message. - The printing test for compare_parameters() expects the profile CI for the glm, because ci_method now defaults to the model's own method. - Fix typos in the lmerMod message. - Update the compare_parameters() help page for the new ci_method default.
|
@jmgirard I think we can merge this! |
jmgirard
left a comment
jmgirard
left a comment
There was a problem hiding this comment.
The other points from my earlier review are fixed, thanks. I found one more problem from the new ci_method = NULL default in compare_parameters(). With a Bayesian model and the default ci_method, the call now errors. On main it uses "eti".
m_lm <- lm(mpg ~ wt, data = mtcars)
m_stan <- rstanarm::stan_glm(mpg ~ wt, data = mtcars, refresh = 0)
compare_parameters(m_lm, m_stan)
#> Error: missing value where TRUE/FALSE neededThe same happens with only the Bayesian model. An explicit ci_method, such as "hdi" or "wald", still works. The cause is the check for Bayesian models: !NULL %in% c(...) is logical(0), so the if gets NA. I left a suggestion on that line. With it, the default gives the same table as ci_method = "eti", and the compare_parameters tests pass. A test with a Bayesian model and the default ci_method would catch this in the future.
Also, this PR and #1287 both set the version to 0.30.0.11, so the one that merges second needs a new version number.
Co-authored-by: Jeffrey Girard <me@jmgirard.com>
|
Thanks @jmgirard for all the work on this (much more than I expected...) |
Following discussion in #989 and #1273, this PR uses default Satterthwaite df for
lmerModTestobjects (from{lmerTest}) and when{lmerTest}is installed also forlmerModobjects.Created on 2026-10-14 with reprex v2.1.1