Skip to content

Incorrect eta^2 for afex objects when factor names overlap #697

Closed

Description

@crsh

Describe the bug
Hi there, the calculation of eta^2 for afex-ANOVAs is incorrect, when the name of one is part of the name of another factor. I have found this to be the case for the Anova.mlm- and the downstream afex_aov-methods, but I haven't explored this any further.

To Reproduce
Adapting the code from #389, here the relevant factor names are Block and XBlock (it does not happen, when the names are B and XB):

library(Superpower)
library(afex)
library(effectsize)
set.seed(2123)
# create data set:
aov.3way.design <- ANOVA_design(design = "2b*2w*2w",
                                n=20,
                                mu=0.025*c(-1,-1,-1,-1,1,1,1,1) + 0.05*c(-1,-1,1,1,-1,-1,1,1) + 0.05*c(-1,1,-1,1,-1,1,-1,1) + 0.05*c(-1,1,1,-1,-1,1,1,-1) + 0.05*c(-1,1,1,-1,1,-1,-1,1) + 0.05*c(-1,1,-1,1,1,-1,1,-1) + 0.05*c(-1,-1,1,1,1,-1,-1,1),
                                sd=0.15,
                                r=0.6,
                                plot=FALSE,
                                labelnames=c("XBlock","a1","a2","Block","aa1","aa2","C","c1","c2"))
aov.exact.01b <- ANOVA_exact(aov.3way.design,verbose=F)
dat1b <- aov.exact.01b$dataframe
dat1b$y <- dat1b$y+rnorm(n=dim(dat1b)[1],0,0.01)

# list partial eta_squared:
aov.01b <- aov_ez(id = "subject", dv = "y", between = "XBlock", within = c("Block", "C"), data = dat1b, anova_table = list(es = "pes"))

tibble::tibble(
  term = afex::nice(aov.01b)$Effect
  , afex = aov.01b$anova_table$pes
  , effectsize_anova_table = effectsize::eta_squared(aov.01b$anova_table, partial = TRUE)$Eta2_partial
  , effectsize_Anova = effectsize::eta_squared(aov.01b, partial = TRUE)$Eta2_partial
  # , effectsize_Anova = effectsize::eta_squared(aov.01b$Anova, partial = TRUE)$Eta2_partial
  , diff = afex - effectsize_Anova
) |>
  as.data.frame()
Contrasts set to contr.sum for the following variables: XBlock
            term       afex effectsize_anova_table effectsize_Anova       diff
1         XBlock 0.03913943             0.03913943       0.03913943  0.0000000
2          Block 0.73454670             0.73454670       0.27141502  0.4631317
3   XBlock:Block 0.25383454             0.25383454       0.04379157  0.2100430
4              C 0.52271737             0.52271737       0.22548586  0.2972315
5       XBlock:C 0.52792092             0.52792092       0.71366114 -0.1857402
6        Block:C 0.21826257             0.21826257       0.52271737 -0.3044548
7 XBlock:Block:C 0.70503499             0.70503499       0.52792092  0.1771141

Similar divergences occur, for generalized eta^2.

Expected behavior
I would expect the effect sizes to match between all approaches.

Specifiations (please complete the following information):

  • R Version 4.5.1
  • effectsize Version 1.0.2

Activity

  1. added a commit that references this issue on Mar 27, 2026
    7c0c40b
  2. mattansb commented on Mar 27, 2026

    @mattansb
    Member

    Well that was a fun puzzle!

    library(Superpower)
    #> Warning: package 'Superpower' was built under R version 4.5.3
    #> Registered S3 method overwritten by 'lme4':
    #>   method           from
    #>   na.action.merMod car
    library(afex)
    #> Loading required package: lme4
    #> Loading required package: Matrix
    #> ************
    #> Welcome to afex. For support visit: http://afex.singmann.science/
    #> - Functions for ANOVAs: aov_car(), aov_ez(), and aov_4()
    #> - Methods for calculating p-values with mixed(): 'S', 'KR', 'LRT', and 'PB'
    #> - 'afex_aov' and 'mixed' objects can be passed to emmeans() for follow-up tests
    #> - Get and set global package options with: afex_options()
    #> - Set sum-to-zero contrasts globally: set_sum_contrasts()
    #> - For example analyses see: browseVignettes("afex")
    #> ************
    #> 
    #> Attaching package: 'afex'
    #> The following object is masked from 'package:lme4':
    #> 
    #>     lmer
    library(effectsize)
    set.seed(2123)
    # create data set:
    aov.3way.design <- ANOVA_design(design = "2b*2w*2w",
                                    n=20,
                                    mu=0.025*c(-1,-1,-1,-1,1,1,1,1) + 0.05*c(-1,-1,1,1,-1,-1,1,1) + 0.05*c(-1,1,-1,1,-1,1,-1,1) + 0.05*c(-1,1,1,-1,-1,1,1,-1) + 0.05*c(-1,1,1,-1,1,-1,-1,1) + 0.05*c(-1,1,-1,1,1,-1,1,-1) + 0.05*c(-1,-1,1,1,1,-1,-1,1),
                                    sd=0.15,
                                    r=0.6,
                                    plot=FALSE,
                                    labelnames=c("XBlock","a1","a2","Block","aa1","aa2","C","c1","c2"))
    #> Warning: `aes_string()` was deprecated in ggplot2 3.0.0.
    #> ℹ Please use tidy evaluation idioms with `aes()`.
    #> ℹ See also `vignette("ggplot2-in-packages")` for more information.
    #> ℹ The deprecated feature was likely used in the Superpower package.
    #>   Please report the issue at
    #>   <https://github.com/arcaldwell49/Superpower/issues>.
    #> This warning is displayed once per session.
    #> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
    #> generated.
    #> Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
    #> ℹ Please use `linewidth` instead.
    #> ℹ The deprecated feature was likely used in the Superpower package.
    #>   Please report the issue at
    #>   <https://github.com/arcaldwell49/Superpower/issues>.
    #> This warning is displayed once per session.
    #> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
    #> generated.
    aov.exact.01b <- ANOVA_exact(aov.3way.design,verbose=F)
    dat1b <- aov.exact.01b$dataframe
    dat1b$y <- dat1b$y+rnorm(n=dim(dat1b)[1],0,0.01)
    
    # list partial eta_squared:
    aov.01b <- aov_ez(id = "subject", dv = "y", between = "XBlock", within = c("Block", "C"), data = dat1b, anova_table = list(es = "pes"))
    #> Contrasts set to contr.sum for the following variables: XBlock
    
    tibble::tibble(
      term = afex::nice(aov.01b)$Effect
      , afex = aov.01b$anova_table$pes
      , effectsize_anova_table = effectsize::eta_squared(aov.01b$anova_table, partial = TRUE)$Eta2_partial
      , effectsize_Anova = effectsize::eta_squared(aov.01b, partial = TRUE)$Eta2_partial
      # , effectsize_Anova = effectsize::eta_squared(aov.01b$Anova, partial = TRUE)$Eta2_partial
      , diff = afex - effectsize_Anova
    ) |>
      as.data.frame()
    #>             term       afex effectsize_anova_table effectsize_Anova diff
    #> 1         XBlock 0.03913943             0.03913943       0.03913943    0
    #> 2          Block 0.72727054             0.72727054       0.72727054    0
    #> 3   XBlock:Block 0.24689135             0.24689135       0.24689135    0
    #> 4              C 0.52735804             0.52735804       0.52735804    0
    #> 5       XBlock:C 0.53255624             0.53255624       0.53255624    0
    #> 6        Block:C 0.23034705             0.23034705       0.23034705    0
    #> 7 XBlock:Block:C 0.71927298             0.71927298       0.71927298    0

    Created on 2026-03-27 with reprex v2.1.1

  3. mattansb commented on Mar 27, 2026

    @mattansb
    Member

    Fixed in #699

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug 🐜Something isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions