Skip to content

test(avatar): Check the guest avatar independent of the font renderer - #65954

Merged
provokateurin merged 1 commit into
masterfrom
chore/fix-flaky-avatar-tests
Oct 29, 2026
Merged

provokateurin merged 1 commit into
masterfrom
chore/fix-flaky-avatar-tests

Conversation

@DerDreschner

@DerDreschner DerDreschner commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Summary

Our own runners are slowly being migrated from Ubuntu 24.04 to Ubuntu 26.04. As this comes with an updated FreeType version, the output of the avatars changed slightly. Due to our byte-wise comparison against a fixed PNG file, this makes the test fail each time it runs on a runner that has already been migrated. The following message is shown in the log then:

1) Test\Avatar\GuestAvatarTest::testGet
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-Binary String: 0x2e2e2ea3000000097048597300000ec400000ec401952b0e1b000000c2494441544889637cf9f93b032d01134d4d1fb560d402aa001622d57dfdf5e7f9e71f688222dc6c021c6cd4b1e0c4e3b775fbaea209669a29c7e8cb53c702086875d19117e08673853909389f640b64f8b81405b909ab430283269221e0f7bf7fbffffe8373599909bb8f340b52369c41e62e0fb59013e0a2a6058556aa52bc9c70ae180f3b412da459602021a822cc43929691535440c089276f1f7ef80ae72a0bf3280810c816a45930fdd45d646ea69932410b18472bfd510b462d18b580300000e6cc27ef3a47c03e0000000049454e44ae426082
+Binary String: 0x2e2e2ea3000000097048597300000ec400000ec401952b0e1b000000a4494441544889637cf9f93b032d01134d4d1fb560d482a161010bf14aad67ef43135112e45e1c624e350b181818dc55c4c375e5e05c3666c201409a05425cecea22bc246919fa91cc487c698a19c909860aa9264af8759116071eaa129148912cc4c546500b69160872b2a908f390a465e847326941f4eedbcf9b6f3ec3b96ccc4c8a82dcd4b460e79d973befbc847389292a4848a6e481a11fc9a3168c5a306a010303004efd1e52ea03084a0000000049454e44ae426082

/home/runner/actions-runner/_work/server/server/tests/lib/Avatar/GuestAvatarTest.php:52

Therefore, this PR makes the tests more independent from the used renderer by running the following checks:

  • Is it a PNG?
  • Does the image size match 32x32?
  • Does the image contain 27 pixels matching the font color?

The number 27 come from testing the result across different FreeType / GD versions.

Test results from Claude across different distributions
┌─────────────────────┬─────┬─────────────┬─────────────────────┬───────────────┬────────────────────┐
│     Environment     │ PHP │ GD (libgd3) │      FreeType       │ Fully covered │ Anti-aliased edges │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Ubuntu 18.04        │ 7.2 │ 2.2.5       │ 2.8.1               │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Debian buster       │ 7.3 │ 2.2.5       │ 2.9.1               │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Ubuntu 20.04        │ 7.4 │ 2.2.5       │ 2.10.1              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Debian bullseye     │ 7.4 │ 2.3.0       │ 2.10.4              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Ubuntu 22.04        │ 8.1 │ 2.3.0       │ 2.11.1              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Debian bookworm     │ 8.2 │ 2.3.3       │ 2.12.1              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ php:8.3 image       │ 8.3 │ bundled     │ 2.12.1              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Alpine 3.22         │ 8.4 │ bundled     │ 2.13.3              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Ubuntu 24.04        │ 8.3 │ 2.3.3       │ 2.13.2              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Debian trixie       │ 8.4 │ 2.3.3       │ 2.13.3              │ 27            │ 51                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Ubuntu 26.04        │ 8.5 │ 2.3.3       │ 2.14.2              │ 27            │ 27                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Fedora (latest)     │ 8.5 │ 2.3.3       │ not shown (current) │ 27            │ 27                 │
├─────────────────────┼─────┼─────────────┼─────────────────────┼───────────────┼────────────────────┤
│ Arch Linux (latest) │ 8.5 │ 2.3.3       │ not shown (current) │ 27            │ 27                 │
└─────────────────────┴─────┴─────────────┴─────────────────────┴───────────────┴────────────────────┘

As the SVG fixture has been unused since #10248 switched guest avatars to PNG, I've removed it on this occasion as well.

Backports necessary down to stable29 as they all run on Ubuntu 26.04.
Backport only necessary down to stable35, as the test didn't run until #61140

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@DerDreschner DerDreschner self-assigned this Oct 27, 2026
@DerDreschner
DerDreschner requested a review from a team as a code owner October 27, 2026 09:04
@DerDreschner DerDreschner added 3. to review Waiting for reviews tests Related to tests CI AI assisted labels Oct 27, 2026
@DerDreschner
DerDreschner requested review from come-nc, icewind1991, provokateurin and sorbaugh and removed request for a team October 27, 2026 09:04
@DerDreschner

Copy link
Copy Markdown
Member Author

/backport to stable35

Comment thread tests/lib/Avatar/GuestAvatarTest.php Resolved
@DerDreschner
DerDreschner force-pushed the chore/fix-flaky-avatar-tests branch from 5927ea0 to 04403fc Compare October 27, 2026 09:54
Comment thread tests/lib/Avatar/GuestAvatarTest.php Resolved
@provokateurin

Copy link
Copy Markdown
Member

Before merging, I would like to check if this test then also passes on my machine.

GuestAvatarTest::testGet() compared the generated PNG byte for byte with
a stored image. The anti-aliasing of the letter depends on the GD and
FreeType versions: the stored image matches Ubuntu 24.04 (libgd3
2.3.3-9ubuntu5, FreeType 2.13.2), while Ubuntu 26.04 (libgd3
2.3.3-13ubuntu2, FreeType 2.14.2) and the bundled GD of the official PHP
images render a few edge pixels differently. The nodb jobs therefore fail
whenever they run on a runner with the newer image.

Check the PNG format, the image size and the guest background color in the
light and the dark theme instead, and count the pixels in the letter
color: the anti-aliased edges differ between renderers, the strokes of the
letter don't. Also remove the SVG fixture, which has been unused since
guest avatars switched to PNG.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: David Dreschner <david.dreschner@nextcloud.com>
@DerDreschner
DerDreschner force-pushed the chore/fix-flaky-avatar-tests branch from 04403fc to 4f0609d Compare October 27, 2026 10:30

@provokateurin provokateurin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also passes on my machine 👍

@provokateurin
provokateurin merged commit bddc7d5 into master Oct 29, 2026
155 checks passed
@provokateurin
provokateurin deleted the chore/fix-flaky-avatar-tests branch October 29, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews AI assisted CI tests Related to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants