Skip to content

test: establish E2E foundation with initial smoke coverage - #830

Open
scotluns wants to merge 7 commits into
developfrom
test/playwright-smoke
Open

scotluns wants to merge 7 commits into
developfrom
test/playwright-smoke

Conversation

@scotluns

@scotluns scotluns commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Why

Establish a foundation for E2E tests that gives contributors confidence that changes preserve Navigator's core workflows. Begin with smoke coverage and a minimal happy path, then add scenarios within the same test setup as coverage grows.

What Changed

  • Adds a Chromium Playwright suite under nav-app/e2e, run with npm run test:e2e. Tests tagged @smoke can be selected with --grep @smoke; the README documents the convention and shared setup.
  • Starts with one happy-path smoke test: create an Enterprise layer, select and score a technique, rename the layer, and validate the downloaded JSON. It uses existing synthetic STIX fixtures and existing UI selectors, without application changes.
  • Runs the suite against the production build in GitHub Actions, with one worker and no retries. Artifacts go under tmp/playwright/test-results; CI uploads failure results and traces for seven days.
  • Documents local and optional Docker execution; the container runs tests as a non-root user. Protractor remains untouched for removal in a separate change.

How To Verify

  • Follow nav-app/e2e/README.md to build and run locally. The current tagged smoke test passes and writes its export to the configured artifact directory.
  • The current revision passed GitHub CI and SonarCloud checks. A deliberately incorrect score assertion confirmed CI fails and uploads the exported JSON and a readable trace; that assertion was restored.
  • The current Docker image builds and the smoke test passes on Linux arm64 as a non-root user, using the documented run command without platform or shared-memory overrides.

Known Gaps

Initial coverage uses synthetic data and checks interaction and export behavior. Full ATT&CK data, layer comparison, import, visual rendering, and other browsers remain future coverage.

Exercise layer creation, scoring, naming, and JSON export against the production build using existing synthetic STIX fixtures.

Run the single Chromium test after the existing quality gate builds Navigator, and retain failure artifacts for seven days.
Comment thread .github/workflows/ci.yml Fixed
Comment thread .github/workflows/ci.yml Fixed
Temporarily expect score 2 after entering score 1 to prove that the draft PR fails and uploads its diagnostic trace. Restore the assertion after validation.
Restore the exported-score assertion to 1. The separate CI run for the preceding commit retains the deliberately incorrect expectation for failure-path validation.
Use the lockfile-installed CLI for Chromium installation so CI cannot fall back to fetching a package through npx.
Move Playwright tests into e2e alongside the existing Protractor files,
with separate filename matching and a smoke tag for focused runs.

Use test:e2e across local and CI execution, keep artifacts under tmp,
and consolidate contributor and Docker instructions in the E2E README.
Leave Protractor removal for a separate change.
@scotluns scotluns changed the title test: add Chromium layer smoke test to CI test: establish E2E foundation with initial smoke coverage Sep 14, 2026
Comment thread nav-app/e2e/Dockerfile Fixed
Use a shared browser installation path and grant the built-in node user
ownership of the test artifact directory. This resolves the SonarCloud
finding for the optional Docker setup.

Validated the production build and smoke test on Linux arm64 with the
documented default Docker run command.
Keep the README focused on the current test setup and rely on the existing tmp ignore rule for Playwright artifacts.
@sonarqubecloud

Copy link
Copy Markdown

@scotluns
scotluns marked this pull request as ready for review September 14, 2026 18:12
@scotluns
scotluns requested a review from jondricek September 14, 2026 18:29
@scotluns scotluns self-assigned this Sep 14, 2026

@jondricek jondricek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I approve this pull request, but i'd like you to open a new GitHub issue to update our approach to using mock data for tests in ATT&CK Navigator. We should probably continue to test with various versions of ATT&CK, so we shouldn't completely discount old stuff, but we should also test with current data too, whether mock data or real ATT&CK releases

Comment thread nav-app/e2e/layer.spec.ts
Comment on lines +10 to +20
// Reuse existing synthetic STIX fixtures; only data responses are replaced.
await page.route('**/assets/config.json', route => route.fulfill({
json: { ...config, collection_index_url: '', versions: configData },
}));
await page.route(configData.entries[0].domains[0].data[0], route => route.fulfill({
json: {
type: 'bundle',
id: 'bundle-smoke',
objects: [matrixSDO, TA0000, { ...T0001, name: 'Smoke technique' }, { ...T0003, name: 'Untouched technique' }],
},
}));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, but to me it is highlighting that our mock data that it uses here hasn't been updated in 2+ years and probably longer than that, since https://github.com/mitre-attack/attack-navigator/blob/master/nav-app/src/tests/utils/mock-data.ts has ATT&CK v13 in it which is from April 2023.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants