Skip to content

test: add emulator-free JVM unit tests + JaCoCo coverage gate (97.8%)#13

Open
AkashBrowserStack wants to merge 5 commits into
mainfrom
test/coverage-100-gate
Open

test: add emulator-free JVM unit tests + JaCoCo coverage gate (97.8%)#13
AkashBrowserStack wants to merge 5 commits into
mainfrom
test/coverage-100-gate

Conversation

@AkashBrowserStack

Copy link
Copy Markdown

The :espresso library module had zero JVM unit tests (only emulator-bound androidTest) and no coverage gate in CI. This adds a real emulator-free suite + a JaCoCo gate.

What

  • 71 JVM unit tests under espresso/src/test/java/... covering all 9 production classes: Environment, Cache, ScreenshotOptions, Tile, CliWrapper (stub HTTP server), AppPercy, MetadataHelper, Metadata, GenericProvider. Uses Robolectric (for Build/Resources/Bitmap/Base64 — no emulator), Mockito, and a java.net StubHttpServer.
  • espresso/build.gradle: applies jacoco, adds test deps, includeAndroidResources/returnDefaultValues, and a jacocoTestCoverageVerification (LINE floor 0.977) wired into the gate.
  • .github/workflows/test.yml (PR + push): setup-java@v4 (JDK 17) + setup-android@v3 (installs the SDK platform — no emulator), runs :espresso:testDebugUnitTest :espresso:jacocoTestCoverageVerification.

Behavior-preserving testability seam

GenericProvider.captureBitmap() extracts the Screenshot.capture().getBitmap() call into a protected overridable method — production behavior is identical (it still calls Screenshot.capture()), but a test subclass can supply a fake Bitmap. No public-API change.

Coverage (local, Android SDK installed)

71 tests, 0 failures, line coverage 97.79% (266/272). 8 of 9 classes at 100%; MetadataHelper 38/44.

Honest scope note

The 6 uncovered lines are the deviceNameFromCSV remote-storage.googleapis.com fallback + its catch(IOException)/return null tail — unreachable on the JVM because parseBufferReader never returns null (it returns a CSV match or Build.MANUFACTURER + " " + Build.MODEL). Defensive/dead code that would need a live network call to exercise. No JaCoCo excludes used; the floor is the honest achieved value, documented inline.

(Also noted, not fixed: devices.csv is UTF-16 LE but read with the default charset — tests assert against the actual parsed output rather than altering prod.)

🤖 Generated with Claude Code

Adds real, emulator-free unit test coverage for the :espresso library and a
CI coverage gate. Achieves 97.79% line coverage (266/272 lines); the only 6
uncovered lines are dead/defensive remote-CSV-fallback code in MetadataHelper
that cannot run on the JVM (or a device) without a live network call.

- 71 JVM unit tests across all 9 production classes:
  - pure-JVM: Environment, Cache, ScreenshotOptions, Tile, CliWrapper
    (CliWrapper driven against a small ServerSocket-based StubHttpServer that
    covers every response / version-gate / exception branch)
  - Robolectric: Metadata, MetadataHelper, AppPercy, GenericProvider
  - Mockito (inline + mockStatic/mockConstruction) for AppPercy's internal
    CliWrapper/GenericProvider and the androidx Screenshot.capture() bridge
- GenericProvider: extract Screenshot bitmap acquisition into an overridable
  protected captureBitmap() seam. Behavior-preserving: production still calls
  Screenshot.capture(); tests can supply a fake Bitmap. Public API unchanged.
- espresso/build.gradle: apply jacoco; add test deps (junit, org.json,
  robolectric 4.10.3, mockito 5.2); includeAndroidResources + returnDefaultValues;
  includeNoLocationClasses=true so Robolectric-loaded classes are counted;
  jacocoTestReport + jacocoTestCoverageVerification (LINE floor 0.977).
- .github/workflows/test.yml: PR + push gate on JDK 17 + setup-android (no
  emulator) running :espresso:testDebugUnitTest + jacocoTestCoverageVerification.

NOTE (latent bug, NOT fixed here): espresso/src/main/resources/devices.csv is
UTF-16 LE but MetadataHelper.parseBufferReader reads it with the default charset.
It works only because sanitizedString() strips the interleaved NUL bytes; tests
assert against the ACTUAL parsed output rather than masking this.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@AkashBrowserStack AkashBrowserStack requested a review from a team as a code owner June 15, 2026 10:05
Comment on lines +12 to +47
name: ":espresso unit tests + JaCoCo coverage gate"
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'

- name: Set up Android SDK
uses: android-actions/setup-android@v3

- name: Make gradlew executable
run: chmod +x ./gradlew

- name: Run :espresso unit tests and coverage verification
run: ./gradlew :espresso:testDebugUnitTest :espresso:jacocoTestCoverageVerification --stacktrace

- name: Upload JaCoCo HTML report
if: always()
uses: actions/upload-artifact@v4
with:
name: jacoco-report
path: espresso/build/reports/jacoco/jacocoTestReport
if-no-files-found: warn

- name: Upload unit test results
if: always()
uses: actions/upload-artifact@v4
with:
name: unit-test-results
path: espresso/build/reports/tests/testDebugUnitTest
if-no-files-found: warn
GitHub's built-in Automatic Dependency Submission (the 'submit-gradle' check)
fails repo-wide because it runs './gradlew help' without the Android SDK. This
workflow installs the SDK via setup-android before submitting the dependency
graph. A repo admin should disable the built-in feature (Settings -> Code
security -> Automatic dependency submission) so the failing check stops.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@AkashBrowserStack

Copy link
Copy Markdown
Author

Claude Code PR Review

PR: #13Head: 86f710dReviewers: stack:code-reviewer

Summary

Adds an emulator-free JVM test suite (Robolectric + Mockito + an in-process stub HTTP server) and a JaCoCo line-coverage gate (floor 0.977) to the :espresso module. The only production change is a behavior-preserving testability seam in GenericProvider: Screenshot.capture().getBitmap() is extracted into a new protected Bitmap captureBitmap() and captureTiles now calls it. Also adds two GitHub Actions workflows — a PR/push unit-test + coverage gate, and an SDK-aware Gradle dependency-submission job that installs the Android SDK before generating the dependency graph.

Review Table

Priority Category Check Status Notes
High Security No hardcoded secrets or credentials Pass No secrets in source, tests, or workflows. dependency-submission.yml uses only the implicit GITHUB_TOKEN.
High Security Authentication/authorization checks present N/A SDK library + test infra; no auth surface.
High Security Input validation and sanitization Pass No untrusted input path. SDK consumes its own bundled resources and a localhost Percy CLI; StubHttpServer is a test-only fixture.
High Security No IDOR — resource ownership validated N/A No multi-tenant resource access.
High Security No SQL injection (parameterized queries) N/A No database/SQL in this codebase.
High Correctness Logic is correct, handles edge cases Pass captureBitmap() seam is a verbatim extraction; single call site; NPE semantics of Screenshot.capture() unchanged. Verified production diff and call sites independently.
High Correctness Error handling is explicit, no swallowed exceptions Pass Prod unchanged. StubHttpServer accept-loop catch is intentional (quiet exit on stop()-triggered socket close) and commented; acceptable for a test stub.
High Correctness No race conditions or concurrency issues Pass StubHttpServer uses a daemon accept thread guarded by a volatile running flag; cleanly stopped in @After. No shared mutable prod state introduced.
Medium Testing New code has corresponding tests Pass The seam is covered both via override (FakeBitmapProvider) and via the REAL path (mockStatic(Screenshot.class) in testCaptureBitmapUsesScreenshotCapture). Broad suite across lib/metadata/providers.
Medium Testing Error paths and edge cases tested Pass Healthcheck bad-code / missing-header / connection-refused; postScreenshot 500 + server-down; rethrow-vs-swallow on ignoreErrors; IOException-wrapped reader; malformed-CSV fallback.
Medium Testing Existing tests still pass (no regressions) Pass Prod change is behavior-preserving; new JVM tests are additive and module-scoped to :espresso (legacy androidTest untouched).
Medium Performance No N+1 queries or unbounded data fetching N/A No data-access layer; no loops over external resources in changed prod code.
Medium Performance Long-running tasks use background jobs N/A Not applicable to an SDK seam + tests.
Medium Quality Follows existing codebase patterns Pass Package layout, naming (testutil, StubHttpServer, FakeBitmapProvider), and Gradle conventions are consistent. Actions pinned to major tags matching repo style.
Medium Quality Changes are focused (single concern) Pass One concern: enable emulator-free coverage. Prod change is the minimal seam needed to test it.
Low Quality Meaningful names, no dead code Pass No commented-out code or placeholder TODOs. The 6 uncovered prod lines are genuinely dead/defensive (documented in the gate).
Low Quality Comments explain why, not what Pass Exemplary: the JaCoCo floor rationale and the MetadataHelperTest UTF-16 charset note explain why, and transparently flag latent prod bugs without papering over them.
Low Quality No unnecessary dependencies added Pass (nit) Test-only deps are appropriate. mockito-inline is redundant on Mockito 5.x (inline mock-maker is the default) — harmless.

Findings

No Critical or High findings. All findings are Low / informational and do not block merge.

  • File: espresso/build.gradle (test dependencies block)

  • Severity: Low

  • Reviewer: stack:code-reviewer

  • Issue: mockito-inline:5.2.0 is redundant — Mockito 5.x uses the inline mock-maker by default, so the artifact is effectively a no-op alias. A reviewer may waste time wondering why two Mockito artifacts are declared.

  • Suggestion: Drop mockito-inline and rely on mockito-core's default inline maker, or add a one-line comment noting it is intentional/explicit.

  • File: espresso/build.gradle (jacocoClassDirs fileTree → intermediates/javac/debug/classes)

  • Severity: Low

  • Reviewer: stack:code-reviewer

  • Issue: classDirectories points at a single AGP output path. If a future AGP upgrade relocates compiled classes, the fileTree silently resolves to empty and JacocoCoverageVerification passes vacuously (no classes = no rule violations), masking a broken gate.

  • Suggestion: Add a fail-loud guard that errors if classDirectories resolves to an empty set, so path drift fails the build instead of silently green-lighting it.

  • File: espresso/build.gradle (jacocoTestCoverageVerification violation rule, counter = 'LINE')

  • Severity: Low (awareness)

  • Reviewer: stack:code-reviewer

  • Issue: The gate enforces LINE coverage only; there is no BRANCH floor, so future untested branches on already-covered lines would not trip the gate.

  • Suggestion: Optional — add a branch-coverage rule later if branch protection is desired. Reasonable to defer; not in scope for this PR.

  • File: espresso/src/test/java/io/percy/espresso/AppPercyTest.java:120-154 (vs prod AppPercy.java:77,79)

  • Severity: Low (informational)

  • Reviewer: stack:code-reviewer

  • Issue: The log tests pass interned string literals, so they happen to pass despite the prod logLevel == "debug" / == "info" reference-equality comparison (a latent prod bug the PR intentionally does not fix). A future reader refactoring the literal into a variable could break the test mysteriously.

  • Suggestion: Add a one-line test comment (like the UTF-16 note in MetadataHelperTest) flagging that string-identity is load-bearing here.

  • File: .github/workflows/dependency-submission.yml:19-20 and .github/workflows/test.yml

  • Severity: Low (awareness)

  • Reviewer: stack:code-reviewer

  • Issue: contents: write on the dependency-submission job is the documented minimal permission for gradle/actions/dependency-submission and the job triggers only on push/workflow_dispatch (not fork PRs), so there is no fork escalation surface — noted for awareness. Actions are pinned to major tags (@v4/@v3) rather than full SHAs. test.yml has no Gradle dependency caching.

  • Suggestion: Optional hardening — SHA-pin actions if/when org supply-chain policy tightens; add Gradle caching to test.yml to cut CI time. Neither is required.

Independently verified (no issue)

  • captureBitmap() is a verbatim extraction of Screenshot.capture().getBitmap(); the sole call site (captureTiles) now calls it; protected is the minimal visibility for the test override. Behavior preserved.
  • AppPercy wraps capture errors as RuntimeException("Error taking screenshot " + name, e), matching the test assertion.
  • MetadataHelper.parseBufferReader can never return null (returns a CSV match, returns Build.MANUFACTURER + " " + Build.MODEL, or throws), so the device == null network-fallback branch in deviceNameFromCSV is genuinely dead — the 0.977 floor (266/272 = 97.79%, no JaCoCo excludes) is honest and un-gamed.

Verdict: PASS

AkashBrowserStack and others added 3 commits June 22, 2026 14:18
…ndency submission

The built-in Automatic Dependency Submission check (dynamic / submit-gradle)
runs a newer Gradle (8+) than the repo wrapper (7.5). Plugin 1.1.0 relies on
org.gradle.util.ConfigureUtil, removed in Gradle 8, so it fails evaluating
scripts/publish-root.gradle with NoClassDefFoundError: org/gradle/util/ConfigureUtil.
1.3.0 is the first release that dropped that internal-API usage while still
supporting Gradle 6.2+, so the normal 7.5 build is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Raises the JaCoCo LINE gate 0.977 -> 1.00. The previously-uncovered
remote-CSV fallback in MetadataHelper.deviceNameFromCSV (the
storage.googleapis.com/.../supported_devices.csv fetch plus its
catch(IOException)/return-null tail) is now exercised by real tests.

No JaCoCo excludes, no @generated, no pragmas. The remote fetch was made
reachable by minimal, behavior-preserving testability seams in
MetadataHelper (localLookup / openLocalCsvReader / openRemoteCsvReader /
remoteCsvUrl), letting MetadataHelperTest force a local miss and drive the
fallback against a loopback StubHttpServer (success path) or an injected
IOException (catch/return-null path). Production control flow is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants