Skip to content

Commit 5147fa1

Browse files
committed
feat(sms): measure on real smishing reports and catch reply bait
The SMS corpus was CallShield's own synthetic set, where the analyzer catches 90%+. A stratified sample of 5,000 real reports from the IMC 2025 dataset (Agarwal, Papasavva, Suarez-Tangil and Vasek, CC BY 4.0) now runs beside it: scripts/sample_imc25_corpus.py draws it from a pinned commit and SHA-256, keeps every wrong-number and hey mum/dad report, and samples the rest by language (63 languages). The test swaps the dataset's <URL>/<PHONE_NUMBER>/<NAMED_ENTITY> placeholders for neutral stand-ins and prints recall per language, with precision and false alarms from the clean messages in that language. Baseline: 10.3% overall, 4.5% of reply-based scams. Reply bait (reply_bait, +30): a stranger's first text with no link (a WhatsApp chat link aside) that addresses a parent about a lost or new phone, says "this is your mum", uses a wrong-number phrase, or is a short "is this Sarah?". A possessive or article before the parent word ("Maya's mom", "la mamá de") doesn't count, and "mam"/"pap" need a phone cue, not WhatsApp alone. It applies only when isSpamSms sees a live text from a full phone number the user never texted and has no inbox message from older than two minutes. An unreadable inbox means no. Alone it clears aggressive mode's bar but not the default one, since a real kid with a new phone writes the same words. With it: 13.0% overall, 35.3% of reply-based reports, and still no false alarm among the 66 clean messages. Recall floors pin those numbers. Tests: SmsReplyBaitTest (phrases in seven languages, possessives, ma'am, links, length, disguise) and ReplyBaitFirstContactTest (Robolectric, a fake inbox and sent folder through isSpamSms). SmsEvaluationCorpusTest scores phone-number senders as first contacts, gains ten hard negatives that look like reply bait, the IMC report and floors, and an attribution check. No existing assertion changed.
1 parent 8d81639 commit 5147fa1

17 files changed

Lines changed: 6112 additions & 28 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,17 @@ All notable changes to CallShield will be documented in this file.
44

55
## Unreleased
66

7+
- SMS checks catch reply bait: a first text from a phone number you've never
8+
texted or heard from that fishes for an answer without a link. "Hi mum,
9+
this is my new number, my phone broke" is caught in English, Spanish, German,
10+
Dutch, Portuguese, Italian, French and a few more, and so are "Is this
11+
Sarah?" and "sorry, wrong number". On its own it blocks only in aggressive
12+
mode, since a real kid with a new phone writes the same words. Someone
13+
else's parent ("this is Maya's mom, here's my new number") doesn't count.
14+
- The SMS regression test runs 5,000 real smishing reports from the IMC 2025
15+
dataset (CC BY 4.0) and reports recall for each language. Content rules
16+
catch 13% of them, up from 10% before reply bait, and 35% of the wrong
17+
number and "hi mum" scams, up from 4.5%.
718
- The report Worker runs the current code. It had been serving a build from
819
March, so reports carried no reporter identity and the trending lists and
920
spam domains couldn't count independent reporters. It now records a daily

‎README.md‎

Lines changed: 28 additions & 24 deletions
Large diffs are not rendered by default.

‎app/src/main/java/com/sysadmindoc/callshield/data/SmsContentAnalyzer.kt‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -336,8 +336,15 @@ class SmsContentAnalyzer
336336
* multi-MB string is a real ReDoS risk on the 5-second screening
337337
* deadline — at this length the message is almost certainly spam
338338
* anyway, so we sample the first 16 KB and stop.
339+
*
340+
* [firstContact] says it's the first text from a phone number the user
341+
* has never texted or heard from, the only place a friendly opener with
342+
* no link is suspicious (see [SmsReplyBait]).
339343
*/
340-
fun analyze(body: String): SmsAnalysisResult {
344+
fun analyze(
345+
body: String,
346+
firstContact: Boolean = false,
347+
): SmsAnalysisResult {
341348
var score = 0
342349
val reasons = mutableListOf<String>()
343350

@@ -429,6 +436,11 @@ class SmsContentAnalyzer
429436
reasons.add("short_msg_with_url")
430437
}
431438

439+
if (firstContact && SmsReplyBait.matches(ruleText, urls, visibleText.length)) {
440+
score += REPLY_BAIT_SCORE
441+
reasons.add("reply_bait")
442+
}
443+
432444
return SmsAnalysisResult(score.coerceAtMost(100), reasons)
433445
}
434446

@@ -437,6 +449,11 @@ class SmsContentAnalyzer
437449

438450
internal const val MAX_ANALYSIS_LENGTH = 16_384
439451
internal const val MAX_REPORT_DOMAINS = 10
452+
453+
// On its own, reply bait clears aggressive mode's bar (25) but not
454+
// the default one (50): a real child with a new phone writes the
455+
// same words, so it takes a second signal to block by default.
456+
internal const val REPLY_BAIT_SCORE = 30
440457
private const val MAX_VERIFICATION_BODY_LENGTH = 1_024
441458
private const val MIN_REPORT_DOMAIN_LENGTH = 5
442459
private const val MAX_REPORT_DOMAIN_LENGTH = 253
@@ -539,7 +556,10 @@ class SmsContentAnalyzer
539556
return normalized.takeIf { isValid }
540557
}
541558

542-
fun analyze(body: String): SmsAnalysisResult = shared.analyze(body)
559+
fun analyze(
560+
body: String,
561+
firstContact: Boolean = false,
562+
): SmsAnalysisResult = shared.analyze(body, firstContact)
543563

544564
fun extractReportableIndicators(body: String): SmsReportIndicators = shared.extractReportableIndicators(body)
545565

‎app/src/main/java/com/sysadmindoc/callshield/data/SmsContextChecker.kt‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,41 @@ class SmsContextChecker
6060
}
6161
}
6262

63+
/**
64+
* True when [number] is a full phone number (not a short code) with no
65+
* inbox message older than a two-minute grace, so the one arriving is
66+
* its first. The grace keeps the arriving message itself from counting
67+
* when the SMS app has already stored it. False when the inbox can't
68+
* be read, so an unreadable inbox never makes a sender a stranger.
69+
*/
70+
fun isFirstMessageFrom(
71+
context: Context,
72+
number: String,
73+
nowMillis: Long = System.currentTimeMillis(),
74+
): Boolean {
75+
val normalized = normalize(number)
76+
if (normalized.length < MIN_FIRST_CONTACT_DIGITS) return false
77+
78+
return try {
79+
context.contentResolver
80+
.query(
81+
Telephony.Sms.Inbox.CONTENT_URI,
82+
arrayOf(Telephony.Sms.Inbox.ADDRESS),
83+
"${Telephony.Sms.Inbox.ADDRESS} LIKE ? AND ${Telephony.Sms.Inbox.DATE} < ?",
84+
arrayOf("%${normalized.takeLast(7)}", (nowMillis - FIRST_CONTACT_GRACE_MS).toString()),
85+
null,
86+
)?.use { cursor ->
87+
while (cursor.moveToNext()) {
88+
val address = cursor.getString(0) ?: continue
89+
if (normalize(address) == normalized) return false
90+
}
91+
true
92+
} ?: false
93+
} catch (_: Exception) {
94+
false
95+
}
96+
}
97+
6398
/**
6499
* Strong trust check backed exclusively by prior outbound history.
65100
* Called from SpamRepository before keyword/content analysis.
@@ -248,6 +283,8 @@ class SmsContextChecker
248283
private const val MAX_SMS_BURST_ROWS = 500
249284
private const val SMS_BURST_PHONE_DIGITS = 10
250285
private const val SMS_BURST_PREFIX_DIGITS = 6
286+
private const val MIN_FIRST_CONTACT_DIGITS = 8
287+
private const val FIRST_CONTACT_GRACE_MS = 2 * MILLIS_PER_MINUTE
251288

252289
fun hasSentMessageTo(
253290
context: Context,
Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
package com.sysadmindoc.callshield.data
2+
3+
/**
4+
* Reply bait: a text that opens a conversation and carries no link, the way
5+
* "Hi mum, this is my new number" and "Is this Sarah?" scams wait for an
6+
* answer before they ask for money or move the chat to WhatsApp. It only
7+
* means something in a stranger's first text, which the caller decides; this
8+
* reads the words.
9+
*
10+
* Measured on the IMC 2025 smishing sample in SmsEvaluationCorpusTest. Most
11+
* wrong-number openers ("How are you doing", "Is the house still for sale?")
12+
* read exactly like a real first text, so only the unambiguous ones count.
13+
* Word edges are \p{L} lookarounds and spaces include \p{Zs}, as in
14+
* [SmsContentAnalyzer], because the JVM and Android's ICU disagree on \b.
15+
*/
16+
internal object SmsReplyBait {
17+
/** A real first text that asks "is this Sarah?" says why it's writing, so the opener counts only when it's short. */
18+
private const val MAX_OPENER_LENGTH = 80
19+
20+
/** How far back from a parent word to look for "Maya's" or "my". */
21+
private const val POSSESSIVE_WINDOW = 24
22+
23+
private const val START = "(?<![\\p{L}\\p{N}])"
24+
private const val END = "(?![\\p{L}\\p{N}])"
25+
private const val SPACE = "[\\s\\p{Zs}]"
26+
private const val APOSTROPHE = "['’]"
27+
28+
private val chatLinkHosts = setOf("wa.me", "api.whatsapp.com", "chat.whatsapp.com")
29+
30+
// A parent the text speaks to.
31+
private val parent =
32+
Regex(
33+
"(?iu)$START(?:mum|mom|mam|mama|mamma|mammy|mummy|mommy|dad|daddy|papa|pap|pappa|m[aã]e|mam[aã]e|papai" +
34+
"|mam[aá]|pap[aá]|mami|papi|mutti|vati|maman|ayah|bunda)$END|엄마|아빠|妈妈|爸爸|お母さん|お父さん",
35+
)
36+
37+
// "mam" is also ma'am and "pap" is also porridge, so they count with a lost
38+
// or new phone but not with a request to use WhatsApp alone.
39+
private val ambiguousParents = setOf("mam", "pap")
40+
41+
// Whose parent: "Maya's mom", "my dad", "la mamá de Lucía", "le papa de Léo".
42+
private val possessive =
43+
Regex(
44+
"(?iu)(?:$APOSTROPHE(?:s)?|s$APOSTROPHE|$START(?:my|your|his|her|their|our|the|a|la|el|lo|los|las|su|sus|mi|mis|tu" +
45+
"|der|die|das|dem|den|meine?|deine?|seine?|ihre?|unsere?|het|mijn|jouw|je|zijn|haar|onze" +
46+
"|le|les|mon|ma|ton|ta|son|sa|notre|il|gli|mio|mia|tuo|tua|suo|sua" +
47+
"|o|os|as|meu|minha|teu|seu|nossa|de|van|von|di|du|des|do|da|della|del))$SPACE*$",
48+
)
49+
50+
// "This is your mum", "It's your son".
51+
private val parentSignsOff =
52+
Regex(
53+
"(?iu)^[^\\p{L}\\p{N}]*(?:(?:hi|hello|hey)[^\\p{L}\\p{N}]+)?(?:this is|it$APOSTROPHE?s|it is)$SPACE+" +
54+
"(?:your$SPACE+)?(?:mum|mom|mother|dad|father|son|daughter)$END",
55+
)
56+
57+
// A lost, broken or new phone, in the languages the reports use.
58+
private val newPhone =
59+
Regex(
60+
"(?iu)$START(?:" +
61+
"new (?:mobile |phone |cell |cellphone |whatsapp )?(?:number|phone|no\\.?$END|nr\\.?$END)" +
62+
"|temporary (?:number|phone)|(?:save|store) (?:this|my|the) (?:new )?number" +
63+
"|(?:lost|broke|smashed|dropped|cracked|damaged) my (?:old )?(?:phone|mobile|cell|sim)" +
64+
"|my (?:old )?(?:phone|mobile|cell)[^.!?\\n]{0,20}(?:broken|broke|smashed|lost|dead|damaged|cracked|stolen" +
65+
"|won$APOSTROPHE?t (?:turn on|work)|doesn$APOSTROPHE?t (?:turn on|work))" +
66+
"|friend$APOSTROPHE?s phone|using a spare|water damage" +
67+
"|(?:nuevo|otro) n[uú]mero|n[uú]mero (?:nuevo|temporal)" +
68+
"|(?:tel[eé]fono|m[oó]vil|celular)[^.!?\\n]{0,15}(?:roto|rompi|ca[ií]d|perd[ií])" +
69+
"|neue[sn]? (?:handy)?nummer|nummer gewechselt|nummer (?:ein)?speichern" +
70+
"|(?:handy|telefon)[^.!?\\n]{0,15}(?:kaputt|verloren|defekt|gestohlen|zertr)" +
71+
"|nieuwe? (?:mobiele |telefoon|gsm |06[- ]?)?(?:nummer|nr$END)|leentoestel" +
72+
"|(?:telefoon|mobiel|gsm)[^.!?\\n]{0,15}(?:kapot|kwijt|gevallen|gestolen)" +
73+
"|(?:novo|outro) n[uú]mero|n[uú]mero novo" +
74+
"|(?:telem[oó]vel|celular|telefone|ecr[aã])[^.!?\\n]{0,15}(?:avariad|partid|quebrad|estragad|perdi)" +
75+
"|nuov[oi] numero|(?:telefono|cellulare)[^.!?\\n]{0,15}(?:rott|cadut|pers[oa])" +
76+
"|nouveau num[eé]ro|(?:t[eé]l[eé]phone|portable)[^.!?\\n]{0,15}(?:cass[eé]|perdu|tomb[eé])" +
77+
"|nomor baru|hp[^.!?\\n]{0,10}rusak" +
78+
")|폰이? ?고장|새 번호|手机坏|换号|新号码|携帯[がを]?壊|新しい番号",
79+
)
80+
81+
private val chatApp = Regex("(?iu)${START}whats ?app")
82+
83+
private val wrongNumber =
84+
Regex(
85+
"(?iu)$START(?:wrong (?:number|person)|do you (?:still )?remember me" +
86+
"|did(?:n$APOSTROPHE?t| not) you saved? my number|long time no (?:see|talk|chat|speak))$END",
87+
)
88+
89+
// "Hi, is this Sarah?" The name has to be capitalized: "is this the
90+
// number for the flyer?" is someone with a reason to write.
91+
private val isThisName =
92+
Regex(
93+
"^[^\\p{L}\\p{N}]*(?:(?iu:hi|hello|hey|good (?:morning|afternoon|evening))[^\\p{L}\\p{N}]+)?" +
94+
"(?iu:is this|are you|is that)$SPACE+(?iu:you$SPACE+)?(?:(?iu:mr|mrs|ms|miss|dr)\\.?$SPACE+)?" +
95+
"\\p{Lu}\\p{Ll}+(?:${APOSTROPHE}s)?(?:$SPACE+(?iu:number|phone|cell))?$SPACE*\\?",
96+
)
97+
98+
/**
99+
* True when [text] fishes for a reply. [urls] are the lowercase links it
100+
* carries: any link but a WhatsApp chat link makes it a link scam instead.
101+
* [visibleLength] is its length without invisible characters.
102+
*/
103+
fun matches(
104+
text: String,
105+
urls: List<String>,
106+
visibleLength: Int,
107+
): Boolean {
108+
if (urls.any { !isChatLink(it) }) return false
109+
val parents = parentsAddressed(text)
110+
val signsOff = parentSignsOff.containsMatchIn(text)
111+
if ((parents.isNotEmpty() || signsOff) && newPhone.containsMatchIn(text)) return true
112+
if ((signsOff || parents.any { it !in ambiguousParents }) && chatApp.containsMatchIn(text)) return true
113+
return wrongNumber.containsMatchIn(text) || (visibleLength <= MAX_OPENER_LENGTH && isThisName.containsMatchIn(text))
114+
}
115+
116+
/** The parent words [text] speaks to, skipping ones that name someone's parent. */
117+
private fun parentsAddressed(text: String): List<String> =
118+
parent
119+
.findAll(text)
120+
.filterNot { match ->
121+
val start = match.range.first
122+
possessive.containsMatchIn(text.substring((start - POSSESSIVE_WINDOW).coerceAtLeast(0), start))
123+
}.map { it.value.lowercase() }
124+
.toList()
125+
126+
private fun isChatLink(url: String): Boolean {
127+
val host =
128+
url
129+
.substringAfter("://")
130+
.substringBefore('/')
131+
.substringBefore('?')
132+
.removePrefix("www.")
133+
return host in chatLinkHosts
134+
}
135+
}

‎app/src/main/java/com/sysadmindoc/callshield/data/checker/Checkers.kt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1283,7 +1283,7 @@ internal class SmsContentChecker(
12831283

12841284
override suspend fun check(ctx: CheckContext): BlockResult? {
12851285
val body = ctx.smsBody ?: return null
1286-
val result = smsContentAnalyzer.analyze(body)
1286+
val result = smsContentAnalyzer.analyze(body, firstContact = ctx.smsFirstContact)
12871287
val aggressive = ctx.prefs[SpamRepository.KEY_AGGRESSIVE_MODE] ?: false
12881288
val threshold = if (aggressive) 25 else 50
12891289
return if (result.score >= threshold) {

‎app/src/main/java/com/sysadmindoc/callshield/data/checker/IChecker.kt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,12 @@ data class CheckContext(
132132
* relationship so the later, globally ordered SMS trust rule can decide.
133133
*/
134134
val smsContextTrusted: Boolean = false,
135+
/**
136+
* True for a live SMS that is the first text from a full phone number the
137+
* user has never texted and has no earlier message from. Only the reply
138+
* bait rule in [com.sysadmindoc.callshield.data.SmsContentAnalyzer] reads it.
139+
*/
140+
val smsFirstContact: Boolean = false,
135141
/** Region-scoped sender-ID/numbering evidence for SMS only. */
136142
val senderProvenance: SenderProvenance? = null,
137143
/**

‎app/src/main/java/com/sysadmindoc/callshield/data/repository/SpamRepositoryImpl.kt‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -272,6 +272,13 @@ class SpamRepositoryImpl(
272272
val smsContextTrusted =
273273
canonicalPhone.isNotBlank() &&
274274
checkerDependencies.smsContextChecker.isTrustedSender(context, canonicalPhone)
275+
// Reply bait only means something in a stranger's first text. A rescan
276+
// finds the old message itself in the inbox, so it applies only live.
277+
val smsFirstContact =
278+
realtimeCall &&
279+
!smsContextTrusted &&
280+
canonicalPhone.isNotBlank() &&
281+
checkerDependencies.smsContextChecker.isFirstMessageFrom(context, canonicalPhone)
275282
var trustedAllowSource: String? = null
276283
var sharedDiagnostics: ScreeningDiagnostics? = null
277284
if (canonicalPhone.isNotBlank()) {
@@ -311,6 +318,7 @@ class SpamRepositoryImpl(
311318
prefs = prefs,
312319
trustedAllowSource = trustedAllowSource,
313320
smsContextTrusted = smsContextTrusted,
321+
smsFirstContact = smsFirstContact,
314322
senderProvenance = senderProvenance,
315323
)
316324
val pipelineRun = CheckerPipeline.runWithDiagnostics(smsExtensions, ctx)

‎app/src/main/java/com/sysadmindoc/callshield/ui/MatchReasonLabels.kt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ fun signalLabelRes(signal: String): Int? =
7272
"special_chars" -> R.string.signal_special_chars
7373
"short_msg_with_url" -> R.string.signal_short_message_with_link
7474
"disguised_text" -> R.string.signal_disguised_text
75+
"reply_bait" -> R.string.signal_reply_bait
7576
"spam_domain" -> R.string.signal_spam_domain
7677
"lookalike_host" -> R.string.signal_lookalike_host
7778
else -> overlayReasonLabelRes(signal)

‎app/src/main/res/values-zh-rCN/strings.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -377,6 +377,7 @@
377377
<string name="signal_special_chars">符号过多</string>
378378
<string name="signal_short_message_with_link">带链接的简短消息</string>
379379
<string name="signal_disguised_text">链接或品牌名中藏有伪装字符</string>
380+
<string name="signal_reply_bait">陌生号码的第一条短信在诱你回复</string>
380381
<string name="signal_spam_domain">链接指向已知垃圾网站</string>
381382
<string name="signal_lookalike_host">链接仿冒知名网站</string>
382383
<string name="signal_list_separator">、</string>

0 commit comments

Comments
 (0)