Skip to content

ci: fix smoke test expectation after rule #00279 landed - #978

Merged
haeter525 merged 1 commit into
ev-flow:masterfrom
pulorsok:fix/smoke-test-a4db-count
Oct 1, 2026
Merged

haeter525 merged 1 commit into
ev-flow:masterfrom
pulorsok:fix/smoke-test-a4db-count

Conversation

@pulorsok

Copy link
Copy Markdown
Member

smoke_test.yml has failed on every run since 2026-09-24, on master as well as on open PRs. The docs PRs it failed on did not cause it.

Cause. The smoke test pins how many 100%-confidence behaviours each fixture APK yields. Rule #00279 (TelephonyManager.getDeviceId + getSubscriberId, added for the Exodus analysis) reaches full confidence on 13667fe3b0ad496a0cd157f34b7e0c991d72a4db.apk, so that sample now yields one behaviour more than the pinned value.

Verified locally against quark-engine/apk-samples with the current rule set:

fixture with #279 without #279 pinned
13667fe3…a4db.apk 22 21 20 → 21
Ahmyth.apk 40 40 42 (unchanged)
14d9f1a9…e273e.apk 44 44 43 (unchanged)

(The absolute numbers differ from CI's because the local run filters rules differently; the delta of exactly one behaviour on the a4db sample, and none on the other two, is what matters.)

Only the a4db expectation moves. I also checked the rule proposed in quark-rules #92 (Runtime.exec + Process.getErrorStream): it changes none of the three counts, so no further update is needed when that merges.

🤖 Generated with Claude Code

Rule ev-flow#279 (TelephonyManager.getDeviceId + getSubscriberId) reaches 100%
confidence on 13667fe3b0ad496a0cd157f34b7e0c991d72a4db.apk, so the sample
now yields 21 behaviours instead of 20. The smoke test has failed on every
run since that rule landed in quark-rules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.28%. Comparing base (7136a64) to head (c0dbe47).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #978   +/-   ##
=======================================
  Coverage   81.28%   81.28%           
=======================================
  Files          84       84           
  Lines        7923     7923           
=======================================
  Hits         6440     6440           
  Misses       1483     1483           
Flag Coverage Δ
unittests 81.28% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@haeter525
haeter525 merged commit 845a35f into ev-flow:master Oct 1, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants