Skip to content

Commit ff450cc

Browse files
committed
fix(sms): keep reply bait to real openers and to texts from the inbox
"Wrong number" counted anywhere in a message, so a clinic's first text ending "Wrong number? Reply STOP to opt out" went from 40 to 70 and was blocked in the default mode. It now counts only in the opening 40 characters of a text with no opt-out wording. RCS and chat notifications never reach the SMS inbox, so every one of them read as a stranger's first text. The notification path now says the message isn't from the inbox and never counts as first contact. Test changes: the IMC evaluation scored reports whose sender the dataset didn't show as texts from a phone number, which let reply bait fire on them and overstated recall. Only phone-number senders count now, and the recall floors are reset to the corrected measurement (all 0.122, reply-based 0.257, en 0.165, es 0.179, de 0.136), which is lower than the floors the flattered run had set.
1 parent 898a44a commit ff450cc

12 files changed

Lines changed: 102 additions & 25 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,12 @@ All notable changes to CallShield will be documented in this file.
2626
Dutch, Portuguese, Italian, French and a few more, and so are "Is this
2727
Sarah?" and "sorry, wrong number". On its own it blocks only in aggressive
2828
mode, since a real kid with a new phone writes the same words. Someone
29-
else's parent ("this is Maya's mom, here's my new number") doesn't count.
29+
else's parent ("this is Maya's mom, here's my new number") doesn't count,
30+
and neither does a business's "Wrong number? Reply STOP" footer or an RCS
31+
or chat message, which never reaches the inbox that says who's a stranger.
3032
- The SMS regression test runs 5,000 real smishing reports from the IMC 2025
3133
dataset (CC BY 4.0) and reports recall for each language. Content rules
32-
catch 13% of them, up from 10% before reply bait, and 35% of the wrong
34+
catch 12% of them, up from 10% before reply bait, and 26% of the wrong
3335
number and "hi mum" scams, up from 4.5%.
3436
- The report Worker runs the current code. It had been serving a build from
3537
March, so reports carried no reporter identity and the trending lists and

‎README.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
<p align="center">
1313
<a href="https://github.com/SysAdminDoc/CallShield/releases/latest"><img src="https://img.shields.io/github/v/release/SysAdminDoc/CallShield?style=flat-square&color=a6e3a1" alt="Release"></a>
1414
<img src="https://img.shields.io/badge/Spam%20Numbers-51%2C363-f38ba8?style=flat-square" alt="51,363 Numbers">
15-
<img src="https://img.shields.io/badge/JVM%20unit%20tests-1740-94e2d5?style=flat-square" alt="1740 JVM unit tests">
15+
<img src="https://img.shields.io/badge/JVM%20unit%20tests-1742-94e2d5?style=flat-square" alt="1742 JVM unit tests">
1616
<img src="https://img.shields.io/badge/Android-10%2B-89b4fa?style=flat-square" alt="Android 10+">
1717
<img src="https://img.shields.io/badge/License-MIT-cba6f7?style=flat-square" alt="MIT License">
1818
<img src="https://img.shields.io/badge/API%20Keys-None-fab387?style=flat-square" alt="No required API keys">
@@ -216,7 +216,7 @@ The maintainer trains it by hand from the CallShield database (50K positive and
216216

217217
SMS content regressions use a separate CC0 corpus that CallShield wrote itself. It covers eight languages, the three sender forms, each kind of link, legitimate messages and hard negatives, and the test reports precision, recall and false-positive rate by language and message type without shipping personal data. Every spam message it catches is checked again in disguise (zero-width characters between letters, Cyrillic look-alike letters, fullwidth text, a hidden character in its link) and each copy has to be caught too: `./gradlew :app:testDebugUnitTest --tests com.sysadmindoc.callshield.data.SmsEvaluationCorpusTest`.
218218

219-
The same test runs 5,000 real smishing reports sampled from the IMC 2025 dataset (Agarwal, Papasavva, Suarez-Tangil and Vasek, "Fishing for Smishing", CC BY 4.0) in 63 languages. It prints recall for each language, with precision and false alarms taken from the clean messages above, and each large language has a floor it can't drop below. The number is humbling. Content rules alone catch 13% of those reports, and 35% of the wrong number and "hi mum" scams. Part of the gap is the dataset: it replaced every link with a placeholder, so the link rules see a link but never its host. `python scripts/sample_imc25_corpus.py` redraws the sample from a pinned copy of the dataset, and `IMC25-NOTICE.txt` next to it in `app/src/test/resources/sms-corpus/` credits it.
219+
The same test runs 5,000 real smishing reports sampled from the IMC 2025 dataset (Agarwal, Papasavva, Suarez-Tangil and Vasek, "Fishing for Smishing", CC BY 4.0) in 63 languages. It prints recall for each language, with precision and false alarms taken from the clean messages above, and each large language has a floor it can't drop below. The number is humbling. Content rules alone catch 12% of those reports, and 26% of the wrong number and "hi mum" scams. Part of the gap is the dataset: it replaced every link with a placeholder, so the link rules see a link but never its host. `python scripts/sample_imc25_corpus.py` redraws the sample from a pinned copy of the dataset, and `IMC25-NOTICE.txt` next to it in `app/src/test/resources/sms-corpus/` credits it.
220220

221221
## Features
222222

@@ -515,12 +515,12 @@ RELEASE_KEY_PASSWORD=...
515515
## Testing
516516

517517
```bash
518-
./gradlew testDebugUnitTest # 1740 tests
518+
./gradlew testDebugUnitTest # 1742 tests
519519
./gradlew connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.sysadmindoc.callshield.platform.TargetSdkBehaviorSmokeTest
520520
./gradlew verifyPipelineTests # Cloudflare Worker (node) + data-pipeline and translation checks (python)
521521
```
522522

523-
The suite is **1740 total JVM unit tests**, run with Robolectric wherever a screen, a service or the database is involved.
523+
The suite is **1742 total JVM unit tests**, run with Robolectric wherever a screen, a service or the database is involved.
524524

525525
Two GitHub workflows run without building the app. **Validation** runs the Worker and
526526
pipeline suites on every push except report-only ones (`run-pipeline-tests.ps1 -CorrectnessOnly`),
@@ -557,7 +557,7 @@ language in [issue #7](https://github.com/SysAdminDoc/CallShield/issues/7).
557557
| Community API | Cloudflare Workers |
558558
| URL Safety | Local spam-domain data, with optional PhishTank and OpenPhish |
559559
| Verification | Local Gradle, lint, and release-artifact checks |
560-
| Tests | 1740 JVM unit tests (JUnit) |
560+
| Tests | 1742 JVM unit tests (JUnit) |
561561
| Strings | 1731 string resources and 37 plural groups (translation-ready) |
562562
| Accessibility | 100+ content descriptions, 48dp touch targets |
563563
| Min SDK | 29 (Android 10) |

‎app/src/main/java/com/sysadmindoc/callshield/data/SmsReplyBait.kt‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,9 @@ internal object SmsReplyBait {
2020
/** How far back from a parent word to look for "Maya's" or "my". */
2121
private const val POSSESSIVE_WINDOW = 24
2222

23+
/** "Sorry, wrong number" opens a stranger's text; further in, it's usually a business's opt-out footer. */
24+
private const val WRONG_NUMBER_OPENING = 40
25+
2326
private const val START = "(?<![\\p{L}\\p{N}])"
2427
private const val END = "(?![\\p{L}\\p{N}])"
2528
private const val SPACE = "[\\s\\p{Zs}]"
@@ -86,6 +89,10 @@ internal object SmsReplyBait {
8689
"|did(?:n$APOSTROPHE?t| not) you saved? my number|long time no (?:see|talk|chat|speak))$END",
8790
)
8891

92+
// A business's first text ends "Wrong number? Reply STOP to opt out".
93+
private val optOut =
94+
Regex("(?iu)$START(?:(?:reply|text|send)$SPACE+[\"'“]?stop|opt$SPACE?-?$SPACE?out|unsubscribe)$END")
95+
8996
// "Hi, is this Sarah?" The name has to be capitalized: "is this the
9097
// number for the flyer?" is someone with a reason to write.
9198
private val isThisName =
@@ -110,7 +117,12 @@ internal object SmsReplyBait {
110117
val signsOff = parentSignsOff.containsMatchIn(text)
111118
if ((parents.isNotEmpty() || signsOff) && newPhone.containsMatchIn(text)) return true
112119
if ((signsOff || parents.any { it !in ambiguousParents }) && chatApp.containsMatchIn(text)) return true
113-
return wrongNumber.containsMatchIn(text) || (visibleLength <= MAX_OPENER_LENGTH && isThisName.containsMatchIn(text))
120+
return isWrongNumberOpener(text) || (visibleLength <= MAX_OPENER_LENGTH && isThisName.containsMatchIn(text))
121+
}
122+
123+
private fun isWrongNumberOpener(text: String): Boolean {
124+
val match = wrongNumber.find(text) ?: return false
125+
return match.range.first <= WRONG_NUMBER_OPENING && !optOut.containsMatchIn(text)
114126
}
115127

116128
/** The parent words [text] speaks to, skipping ones that name someone's parent. */

‎app/src/main/java/com/sysadmindoc/callshield/data/SpamRepository.kt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -738,12 +738,14 @@ class SpamRepository(
738738
body: String,
739739
realtimeCall: Boolean = true,
740740
prefsSnapshot: Preferences? = null,
741+
fromSmsInbox: Boolean = true,
741742
): SpamCheckResult =
742743
spamRepositoryImpl.isSpamSms(
743744
number = number,
744745
body = body,
745746
realtimeCall = realtimeCall,
746747
prefsSnapshot = prefsSnapshot,
748+
fromSmsInbox = fromSmsInbox,
747749
)
748750

749751
// ── Pipeline trace (diagnostic) ─────────────────────────────────────

‎app/src/main/java/com/sysadmindoc/callshield/data/repository/SpamRepositoryAdapter.kt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,18 @@ class SpamRepositoryAdapter(
4949
prefsSnapshot = prefsSnapshot,
5050
)
5151

52+
override suspend fun checkSpamMessageOutsideInbox(
53+
number: String,
54+
body: String,
55+
prefsSnapshot: Preferences?,
56+
): SpamCheckResult =
57+
repository.isSpamSms(
58+
number = number,
59+
body = body,
60+
prefsSnapshot = prefsSnapshot,
61+
fromSmsInbox = false,
62+
)
63+
5264
override suspend fun syncDatabase(force: Boolean): SyncResult = repository.syncFromGitHub(force = force)
5365

5466
override suspend fun blockNumber(

‎app/src/main/java/com/sysadmindoc/callshield/data/repository/SpamRepositoryImpl.kt‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,7 @@ class SpamRepositoryImpl(
275275
body: String,
276276
realtimeCall: Boolean = true,
277277
prefsSnapshot: Preferences? = null,
278+
fromSmsInbox: Boolean = true,
278279
): SpamCheckResult {
279280
val prefs = prefsSnapshot ?: settingsRepository.readPrefsSnapshot()
280281
val senderProvenance = senderProvenanceResolver.resolve(number, senderRegionIso)
@@ -283,9 +284,12 @@ class SpamRepositoryImpl(
283284
canonicalPhone.isNotBlank() &&
284285
checkerDependencies.smsContextChecker.isTrustedSender(context, canonicalPhone)
285286
// Reply bait only means something in a stranger's first text. A rescan
286-
// finds the old message itself in the inbox, so it applies only live.
287+
// finds the old message itself in the inbox, so it applies only live,
288+
// and an RCS or chat message never reaches the inbox, so every one of
289+
// them would read as a first text.
287290
val smsFirstContact =
288291
realtimeCall &&
292+
fromSmsInbox &&
289293
!smsContextTrusted &&
290294
canonicalPhone.isNotBlank() &&
291295
checkerDependencies.smsContextChecker.isFirstMessageFrom(context, canonicalPhone)

‎app/src/main/java/com/sysadmindoc/callshield/domain/repository/SpamCheckRepository.kt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,15 @@ interface SpamCheckRepository {
1919
realtimeCall: Boolean,
2020
prefsSnapshot: Preferences?,
2121
): SpamCheckResult
22+
23+
/**
24+
* A live message the SMS inbox never holds (RCS, a chat app's notification).
25+
* Whether it's someone's first text can't be read from the inbox, so it's
26+
* never treated as one.
27+
*/
28+
suspend fun checkSpamMessageOutsideInbox(
29+
number: String,
30+
body: String,
31+
prefsSnapshot: Preferences?,
32+
): SpamCheckResult = checkSpamSms(number, body, realtimeCall = true, prefsSnapshot = prefsSnapshot)
2233
}

‎app/src/main/java/com/sysadmindoc/callshield/domain/usecase/CheckSpamSmsUseCase.kt‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,16 @@ class CheckSpamSmsUseCase(
1212
body: String,
1313
realtimeCall: Boolean = true,
1414
prefsSnapshot: Preferences? = null,
15+
fromSmsInbox: Boolean = true,
1516
): SpamCheckResult =
16-
repository.checkSpamSms(
17-
number = number,
18-
body = body,
19-
realtimeCall = realtimeCall,
20-
prefsSnapshot = prefsSnapshot,
21-
)
17+
if (fromSmsInbox) {
18+
repository.checkSpamSms(
19+
number = number,
20+
body = body,
21+
realtimeCall = realtimeCall,
22+
prefsSnapshot = prefsSnapshot,
23+
)
24+
} else {
25+
repository.checkSpamMessageOutsideInbox(number, body, prefsSnapshot)
26+
}
2227
}

‎app/src/main/java/com/sysadmindoc/callshield/service/RcsNotificationListener.kt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -251,7 +251,7 @@ class RcsNotificationListener : NotificationListenerService() {
251251
val result =
252252
senderNumber
253253
.takeIf { spamBlockingEnabled && filterAsciiDigits(it).length >= 7 }
254-
?.let { number -> checkSpamSms(number, effectiveBody.orEmpty(), prefsSnapshot = prefs) }
254+
?.let { number -> checkSpamSms(number, effectiveBody.orEmpty(), prefsSnapshot = prefs, fromSmsInbox = false) }
255255
val contentVerdict =
256256
effectiveBody?.takeIf { spamBlockingEnabled && result == null }?.let { bodyText ->
257257
contentVerdict(

‎app/src/test/java/com/sysadmindoc/callshield/data/ReplyBaitFirstContactTest.kt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ import android.database.MatrixCursor
88
import android.net.Uri
99
import android.provider.Telephony
1010
import androidx.test.core.app.ApplicationProvider
11+
import com.sysadmindoc.callshield.data.repository.SpamRepositoryAdapter
12+
import com.sysadmindoc.callshield.domain.usecase.CheckSpamSmsUseCase
1113
import kotlinx.coroutines.runBlocking
1214
import org.junit.After
1315
import org.junit.Assert.assertEquals
@@ -92,6 +94,16 @@ class ReplyBaitFirstContactTest {
9294
assertFalse(fixture.repository.isSpamSms(number, body, realtimeCall = false).isSpam)
9395
}
9496

97+
@Test
98+
fun `an RCS or chat message never counts as a first text`() =
99+
runBlocking {
100+
// They never reach the SMS inbox, so every one would read as a stranger's first.
101+
val result = CheckSpamSmsUseCase(SpamRepositoryAdapter(fixture.repository))(number, body, fromSmsInbox = false)
102+
103+
assertFalse(result.signals.toString(), "reply_bait" in result.signals)
104+
assertFalse(result.isSpam)
105+
}
106+
95107
@Test
96108
fun `a short code is not a stranger's phone`() {
97109
val context = ApplicationProvider.getApplicationContext<Application>()

0 commit comments

Comments
 (0)