Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
916b60a
fix: clarify custom message settings
vitormattos Sep 23, 2026
3cd2594
test: enable custom message in settings screenshots
vitormattos Sep 23, 2026
6d4e565
test: cover conditional message settings
vitormattos Sep 23, 2026
4a58be8
test: verify message settings visibility script
vitormattos Sep 23, 2026
9413c87
refactor: extract message settings UI script
vitormattos Sep 23, 2026
385a7e8
refactor: make message settings UI testable
vitormattos Sep 23, 2026
e035fcb
test: cover message settings UI builder
vitormattos Sep 23, 2026
70b3b54
test: verify custom message visibility in browser
vitormattos Sep 23, 2026
b7b52fb
test: cover custom message settings browser flow
vitormattos Sep 23, 2026
716492a
test: simplify custom message browser fixture
vitormattos Sep 23, 2026
d87762a
test: match generated setting selectors
vitormattos Sep 23, 2026
665f474
fix: correct selector assertions
vitormattos Sep 23, 2026
3e1ac64
docs: describe custom message settings flow
vitormattos Sep 23, 2026
f2c498e
test: lock intentional settings order
vitormattos Sep 23, 2026
d71e668
fix: assert visible message format label
vitormattos Sep 23, 2026
28d282d
test: follow visible format label
vitormattos Sep 23, 2026
2b38652
test: match global setting selector syntax
vitormattos Sep 23, 2026
85a440a
fix: target the actual message checkbox
vitormattos Sep 23, 2026
f8acc3e
test: require real checkbox selector
vitormattos Sep 23, 2026
a8333f4
fix: refresh message controls when settings tab opens
vitormattos Sep 23, 2026
1d43b79
test: cover tab-driven visibility refresh
vitormattos Sep 23, 2026
5cad737
fix: observe dynamically rendered plugin settings
vitormattos Sep 23, 2026
34e5499
test: cover dynamic settings observer
vitormattos Sep 23, 2026
70c0560
test: wait for initial custom message hidden state
vitormattos Sep 23, 2026
b7e096a
test: cover asynchronous message visibility flow
vitormattos Sep 23, 2026
12b4e38
fix: select checkbox before narrowing setting matches
vitormattos Sep 23, 2026
3b790e2
ci: refresh screenshot for PR validation
vitormattos Sep 23, 2026
524119b
docs: refresh settings screenshot
github-actions[bot] Sep 23, 2026
f50365a
ci: remove temporary screenshot refresh step
vitormattos Sep 23, 2026
22041b1
ux: polish Telegram settings copy
vitormattos Sep 23, 2026
1eb6ac1
ux: align connection test with settings form
vitormattos Sep 23, 2026
bf526d9
ux: use current message format terminology
vitormattos Sep 23, 2026
61a9089
test: cover connection test label
vitormattos Sep 23, 2026
ccade24
test: cover message format help terminology
vitormattos Sep 23, 2026
e65fb22
test: cover polished settings labels
vitormattos Sep 23, 2026
957636d
docs: match polished settings labels
vitormattos Sep 23, 2026
9266cbf
fix: generate diffs when screenshot dimensions change
vitormattos Sep 23, 2026
5d5daa3
ci: refresh final settings screenshot
vitormattos Sep 23, 2026
b4a8d3c
test: cover dimension mismatch screenshot diffs
vitormattos Sep 23, 2026
092d6bc
fix: publish screenshot failure images for PR comments
vitormattos Sep 23, 2026
f4d132f
ci: refresh expected screenshot for PR 22
vitormattos Sep 23, 2026
60f9e3a
docs: explain screenshot failure reports
vitormattos Sep 23, 2026
99926b0
fix: follow current Telegram settings labels in screenshots
vitormattos Sep 23, 2026
199af57
test: match polished notification label
vitormattos Sep 23, 2026
aed31ee
refactor: isolate screenshot connection fields
vitormattos Sep 23, 2026
bb0ae4f
test: cover screenshot settings labels
vitormattos Sep 23, 2026
ed07306
fix: assert custom message setting containers
vitormattos Sep 23, 2026
3dbd4e0
test: import setting container helper
vitormattos Sep 23, 2026
74b15c3
test: follow setting container visibility
vitormattos Sep 23, 2026
00f0191
test: cover hidden container failure
vitormattos Sep 23, 2026
452c108
docs: refresh settings screenshot
github-actions[bot] Sep 23, 2026
a82f580
docs: refresh settings screenshot
github-actions[bot] Sep 23, 2026
2601db3
ci: serialize temporary screenshot refresh
vitormattos Sep 23, 2026
2e95a99
ci: keep screenshot validation read-only
vitormattos Sep 23, 2026
17ef05c
docs: refresh generated screenshots
github-actions[bot] Sep 23, 2026
ee61cc3
ci: remove temporary screenshot refresh workflow
vitormattos Sep 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 69 additions & 18 deletions .github/workflows/screenshot-failure-comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ on:

permissions:
actions: read
contents: write
pull-requests: write

jobs:
Expand All @@ -36,23 +37,71 @@ jobs:
echo "found=false" >> "$GITHUB_OUTPUT"
fi

- name: Checkout report branch base
if: steps.artifacts.outputs.found == 'true'
uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
path: report-repo
fetch-depth: 1

- name: Publish screenshot report images
if: steps.artifacts.outputs.found == 'true'
id: publish
env:
GH_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
RUN_ID: ${{ github.event.workflow_run.id }}
REPORT_BRANCH: screenshot-reports
shell: bash
run: |
set -euo pipefail

cd report-repo
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"

if git ls-remote --exit-code --heads origin "$REPORT_BRANCH" >/dev/null 2>&1; then
git fetch --depth=1 origin "$REPORT_BRANCH"
git switch -C "$REPORT_BRANCH" FETCH_HEAD
else
git switch --orphan "$REPORT_BRANCH"
git rm -rf . >/dev/null 2>&1 || true
fi

report_dir="pr-$PR_NUMBER/run-$RUN_ID"
mkdir -p "$report_dir"
cp ../screenshot-artifacts/*.png "$report_dir"/

cat > "$report_dir/README.md" <<EOF
Screenshot comparison generated for PR #$PR_NUMBER from workflow run $RUN_ID.
EOF

git add "$report_dir"
git commit -m "docs: screenshot report for PR #$PR_NUMBER run $RUN_ID"
git push origin "HEAD:$REPORT_BRANCH"

echo "base_url=https://raw.githubusercontent.com/$GH_REPO/$REPORT_BRANCH/$report_dir" >> "$GITHUB_OUTPUT"

- name: Comment on pull request
if: steps.artifacts.outputs.found == 'true'
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
RUN_ID: ${{ github.event.workflow_run.id }}
REPORT_BASE_URL: ${{ steps.publish.outputs.base_url }}
shell: bash
run: |
set -euo pipefail

cat > screenshot-comment.md <<'EOF'
cat > screenshot-comment.md <<EOF
## Documentation screenshot check failed

One or more generated documentation screenshots differ from the committed versions.
EOF

attachments=()
The images below are published from the workflow artifact so they can be reviewed directly in this PR.
EOF

add_report() {
local key="$1"
Expand All @@ -68,42 +117,44 @@ jobs:
cat >> screenshot-comment.md <<EOF

### $label
EOF

#### Expected

![Expected $label](./$expected)
if [[ -f "$expected" ]]; then
cat >> screenshot-comment.md <<EOF

#### Generated
#### Committed

![Generated $label](./$actual)
![Committed $label]($REPORT_BASE_URL/$key.expected.png?run=$RUN_ID)
EOF

if [[ -f "$expected" ]]; then
attachments+=("--attach" "./$expected#Expected $label")
fi

attachments+=("--attach" "./$actual#Generated $label")
cat >> screenshot-comment.md <<EOF

#### Generated

![Generated $label]($REPORT_BASE_URL/$key.actual.png?run=$RUN_ID)
EOF

if [[ -f "$diff" ]]; then
cat >> screenshot-comment.md <<EOF

#### Diff
#### Difference

![Visual diff for $label](./$diff)
![Visual difference for $label]($REPORT_BASE_URL/$key.diff.png?run=$RUN_ID)
EOF
attachments+=("--attach" "./$diff#Visual diff for $label")
fi
}

add_report settings "Plugin settings"
add_report preview "Telegram preview"

cat >> screenshot-comment.md <<'EOF'
cat >> screenshot-comment.md <<EOF

[Download the workflow artifact](https://github.com/$GH_REPO/actions/runs/$RUN_ID)

If the visual change is expected, run `npm run screenshots:update` and commit the updated PNG files.
EOF

gh pr comment "$PR_NUMBER" \
--repo "$GH_REPO" \
--body-file screenshot-comment.md \
"${attachments[@]}"
--body-file screenshot-comment.md
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,10 @@ The image below is a locally generated preview of a rendered Telegram message us
2. Add the bot to the Telegram group that should receive survey notifications.
3. Get the group chat ID. One option is to temporarily add [@RawDataBot](https://t.me/RawDataBot) to the group, read the `chat.id` value it reports, and remove the bot afterwards.
4. Open the LSTelegramNotify settings in LimeSurvey.
5. Set `Auth Token` to the token created by BotFather and `Chat id` to the destination group ID.
6. Choose the parse mode and notification options you want, then save the settings.
7. Use `Send test message` to verify that the saved bot token and chat destination work before relying on survey notifications.
5. Set `Bot token` to the token created by BotFather and `Chat ID` to the destination group ID.
6. Enable `Send a custom message` if you want a Telegram message for each completed response. When enabled, choose the `Message format` and edit the `Message template`.
7. Enable the PDF and CSV options independently if you also want those files sent with each completed response.
8. Save the settings, then use `Send test message` to verify that the saved bot token and chat destination work before relying on survey notifications.

The README screenshots below show the current plugin settings and a rendered Telegram notification example. They are generated and validated automatically by the repository screenshot workflow.

Expand Down Expand Up @@ -65,7 +66,7 @@ These placeholders work in both legacy and mustache syntax:
- `{urlExport}` / `{{urlExport}}`
- `{urlAttachments}` / `{{urlAttachments}}`

When `ParseMode` is `HTML`, the rendered template is sent through Telegram's `sendRichMessage` API using Rich HTML. This enables richer blocks such as headings, tables, `<details>` and media such as `<img src="https://example.test/photo.jpg" />`. All placeholder values are escaped before rendering so survey content cannot break the markup.
When `Message format` is `HTML`, the rendered template is sent through Telegram's `sendRichMessage` API using Rich HTML. This enables richer blocks such as headings, tables, `<details>` and media such as `<img src="https://example.test/photo.jpg" />`. All placeholder values are escaped before rendering so survey content cannot break the markup.

### Survey field placeholders

Expand Down
Binary file modified img/settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion scripts/screenshots/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,5 +46,7 @@ Useful optional overrides:
- `Documentation Screenshots` starts the repository-root disposable LimeSurvey stack with Docker Compose (`up -d --force-recreate`) and validates both `img/settings.png` and `img/telegram_example.png` in a single job.
- The workflow should break when the plugin settings UI changes enough to make `img/settings.png` drift from the committed version.

When validation fails, the workflow summary tells you which command to run next and uploads diff artifacts under `test-results/screenshots/`.
When validation fails, the workflow summary tells you which command to run next and uploads the committed, generated, and visual-diff images under `test-results/screenshots/`.

For pull requests, the `Screenshot failure comment` workflow downloads that artifact, publishes the images to the dedicated `screenshot-reports` branch, and comments on the PR with the committed screenshot, the generated screenshot, and the visual difference. The report branch is used because GitHub Actions tokens cannot upload PR comment attachments through `gh pr comment --attach`.

36 changes: 33 additions & 3 deletions scripts/screenshots/browser-flows.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,37 @@ async function waitForDocumentFonts(page) {
});
}

export async function fillTelegramConnectionSettings(page, maskedSettingsValues) {
await page.getByRole('textbox', { name: 'Bot token' }).fill(maskedSettingsValues.authToken);
await page.getByRole('textbox', { name: 'Chat ID' }).fill(maskedSettingsValues.chatId);
}

export function getPluginSettingContainer(page, settingName) {
const settingField = page.locator(
`[name="${settingName}"], [name$="[${settingName}]"]`
).first();

return settingField.locator(
'xpath=ancestor::*[contains(concat(" ", normalize-space(@class), " "), " mb-3 ") or contains(concat(" ", normalize-space(@class), " "), " form-group ")][1]'
);
}

export async function enableCustomMessageSettings(page) {
const sendMessage = page.getByRole('checkbox', { name: 'Send a custom message' });
const messageFormatContainer = getPluginSettingContainer(page, 'ParseMode');
const messageTemplateContainer = getPluginSettingContainer(page, 'DefaultText');
const messageTemplate = page.getByRole('textbox', { name: 'Message template' });

await messageFormatContainer.waitFor({ state: 'hidden' });
await messageTemplateContainer.waitFor({ state: 'hidden' });

await sendMessage.check();
await messageFormatContainer.waitFor({ state: 'visible' });
await messageTemplateContainer.waitFor({ state: 'visible' });

return messageTemplate;
}

export async function captureSettingsScreenshot(page, {
configureUrl,
pluginName,
Expand All @@ -299,10 +330,9 @@ export async function captureSettingsScreenshot(page, {
await page.goto(configureUrl, { waitUntil: 'networkidle' });
await page.getByRole('tab', { name: 'Settings' }).click();

await page.getByRole('textbox', { name: 'Auth Token' }).fill(maskedSettingsValues.authToken);
await page.getByRole('textbox', { name: 'Chat id' }).fill(maskedSettingsValues.chatId);
await fillTelegramConnectionSettings(page, maskedSettingsValues);

const defaultTextField = page.getByRole('textbox', { name: 'Default Text' });
const defaultTextField = await enableCustomMessageSettings(page);

await defaultTextField.fill(maskedSettingsValues.defaultText);
await defaultTextField.evaluate((element) => {
Expand Down
35 changes: 32 additions & 3 deletions scripts/screenshots/checker.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,26 @@ export function comparePngBuffers(expectedBuffer, actualBuffer, {
const actualImage = PNG.sync.read(actualBuffer);

if (expectedImage.width !== actualImage.width || expectedImage.height !== actualImage.height) {
const width = Math.max(expectedImage.width, actualImage.width);
const height = Math.max(expectedImage.height, actualImage.height);
const expectedCanvas = normalizePngCanvas(expectedImage, width, height);
const actualCanvas = normalizePngCanvas(actualImage, width, height);
const diffImage = new PNG({ width, height });
const diffPixels = pixelmatch(
expectedCanvas.data,
actualCanvas.data,
diffImage.data,
width,
height,
{ threshold: pixelmatchThreshold }
);

return {
matches: false,
reason: `Image dimensions differ: expected ${expectedImage.width}x${expectedImage.height}, got ${actualImage.width}x${actualImage.height}`,
diffPixels: Number.POSITIVE_INFINITY,
diffPixelRatio: 1,
diffBuffer: null,
diffPixels,
diffPixelRatio: diffPixels / (width * height),
diffBuffer: PNG.sync.write(diffImage),
};
}

Expand Down Expand Up @@ -47,6 +61,21 @@ export function comparePngBuffers(expectedBuffer, actualBuffer, {
};
}

function normalizePngCanvas(image, width, height) {
const canvas = new PNG({ width, height });

for (let index = 0; index < canvas.data.length; index += 4) {
canvas.data[index] = 255;
canvas.data[index + 1] = 255;
canvas.data[index + 2] = 255;
canvas.data[index + 3] = 255;
}

PNG.bitblt(image, canvas, 0, 0, image.width, image.height, 0, 0);

return canvas;
}

export async function assertScreenshotsUpToDate(config, { logger = console, browserType } = {}) {
const screenshots = await captureScreenshots(config, { logger, browserType });
const refreshCommand = buildScreenshotCommand('screenshots:update', config.targets);
Expand Down
2 changes: 1 addition & 1 deletion src/Plugin/DefaultTextHelpBuilder.php
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ public function build(?int $surveyId = null): string
'<ul>' . $this->buildMetadataPlaceholderHelpItems() . '</ul>',
'<p><strong>Survey field syntax</strong></p>',
'<ul>' . $this->buildFieldSyntaxHelpItems() . '</ul>',
'<p>When <code>ParseMode</code> is <code>HTML</code>, placeholder values are escaped automatically before rendering.</p>',
'<p>When <strong>Message format</strong> is <code>HTML</code>, placeholder values are escaped automatically before rendering.</p>',
];

if ($surveyId === null) {
Expand Down
Loading
Loading