Skip to content

Commit 03deb86

Browse files
committed
Keep user rules through database corruption, refetch an emptied spam list
Android's SQLite deletes a database the moment a query finds a damaged page (SupportSQLiteOpenHelper.Callback.onCorruption), before the error reaches any caller, and Room carries on with an empty file. User blocks, the allow list and rules went with it. The screener's own recovery then closed the Room singleton that SpamRepository and Hilt still held, so every later DAO call threw a cancellation and calls rang through unscreened until the process died. A wrapping open-helper factory now reads the user tables out of the damaged file with raw SQL, one table at a time, saves them in the portable backup format under noBackupFilesDir, lets SQLite delete, then merges them back into the rebuilt database in-process and resyncs. A start that finds a leftover rescue imports it before anything screens. recoverFromCorruption is gone. Shard hashes and the data commit live in settings, which cloud backup restores without the database. Sync no longer reports "up to date" with zero database rows, and a shard whose hash matches but whose rows are all missing is fetched again. Tests: CorruptionRescueTest, RescuingOpenHelperFactoryTest, EmptiedDatabaseResyncTest, and CorruptionRescueInstrumentedTest, which overwrites a real page on a phone. FlaggedTextNumbersTest was wrong: it relied on whatever home region an earlier test left in the canonicalizer's 60-second cache, so in a full run a bare ten-digit number wasn't read as North American and "a later flagged text moves the date forward" failed. It now pins a US SIM and resets the cache, as OutgoingCallHoldRobolectricTest does. IsolatedRepositoryFixture gains sqlite and damageTable helpers.
1 parent 46c5daf commit 03deb86

15 files changed

Lines changed: 894 additions & 50 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,14 @@ All notable changes to CallShield will be documented in this file.
44

55
## Unreleased
66

7+
- Your own blocks, allow list and rules now survive a damaged database.
8+
Android deletes the file as soon as it finds a bad page, which used to
9+
take all of them with it. CallShield now copies them out first and puts
10+
them back into the new database. The screener also stopped letting every
11+
call through after a rebuild until the app restarted.
12+
- A phone that comes back from a cloud backup, or whose database had to be
13+
rebuilt, downloads the whole spam list again. Before, the restored
14+
settings claimed the list was current and only changed parts came down.
715
- The outgoing call check now also holds a call to a number that appeared
816
in a text CallShield flagged as spam during the last 30 days. Fake bank
917
and delivery texts ask you to call a number back, and the hold notice

‎README.md‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
<p align="center">
1313
<a href="https://github.com/SysAdminDoc/CallShield/releases/latest"><img src="https://img.shields.io/github/v/release/SysAdminDoc/CallShield?style=flat-square&color=a6e3a1" alt="Release"></a>
1414
<img src="https://img.shields.io/badge/Spam%20Numbers-51%2C363-f38ba8?style=flat-square" alt="51,363 Numbers">
15-
<img src="https://img.shields.io/badge/JVM%20unit%20tests-1777-94e2d5?style=flat-square" alt="1777 JVM unit tests">
15+
<img src="https://img.shields.io/badge/JVM%20unit%20tests-1789-94e2d5?style=flat-square" alt="1789 JVM unit tests">
1616
<img src="https://img.shields.io/badge/Android-10%2B-89b4fa?style=flat-square" alt="Android 10+">
1717
<img src="https://img.shields.io/badge/License-MIT-cba6f7?style=flat-square" alt="MIT License">
1818
<img src="https://img.shields.io/badge/API%20Keys-None-fab387?style=flat-square" alt="No required API keys">
@@ -367,6 +367,9 @@ scrape Nomorobo's restricted carrier feed.
367367
- **Scoped backup**. Cloud backup includes non-secret settings only. The
368368
sensitive database is limited to direct device transfer and explicit
369369
user-created portable backups
370+
- **Damaged database recovery**. Android deletes a database the moment it finds a
371+
damaged page. Before it does, CallShield copies out your own blocks, allow list and
372+
rules, puts them into the fresh database and downloads the spam list again
370373
- **APK privacy gate**. Builds package only the five runtime protection feeds.
371374
Verification rejects raw community submissions and maintainer files
372375
- **Direct-boot boundary**. A minimal device-encrypted mirror keeps explicit
@@ -516,12 +519,12 @@ RELEASE_KEY_PASSWORD=...
516519
## Testing
517520

518521
```bash
519-
./gradlew testDebugUnitTest # 1777 tests
522+
./gradlew testDebugUnitTest # 1789 tests
520523
./gradlew connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.sysadmindoc.callshield.platform.TargetSdkBehaviorSmokeTest
521524
./gradlew verifyPipelineTests # Cloudflare Worker (node) + data-pipeline and translation checks (python)
522525
```
523526

524-
The suite is **1777 total JVM unit tests**, run with Robolectric wherever a screen, a service or the database is involved.
527+
The suite is **1789 total JVM unit tests**, run with Robolectric wherever a screen, a service or the database is involved.
525528

526529
Two GitHub workflows run without building the app. **Validation** runs the Worker and
527530
pipeline suites on every push except report-only ones (`run-pipeline-tests.ps1 -CorrectnessOnly`),
@@ -558,7 +561,7 @@ language in [issue #7](https://github.com/SysAdminDoc/CallShield/issues/7).
558561
| Community API | Cloudflare Workers |
559562
| URL Safety | Local spam-domain data, with optional PhishTank and OpenPhish |
560563
| Verification | Local Gradle, lint, and release-artifact checks |
561-
| Tests | 1777 JVM unit tests (JUnit) |
564+
| Tests | 1789 JVM unit tests (JUnit) |
562565
| Strings | 1738 string resources and 37 plural groups (translation-ready) |
563566
| Accessibility | 100+ content descriptions, 48dp touch targets |
564567
| Min SDK | 29 (Android 10) |
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
package com.sysadmindoc.callshield.data
2+
3+
import android.content.Context
4+
import androidx.test.core.app.ApplicationProvider
5+
import androidx.test.ext.junit.runners.AndroidJUnit4
6+
import com.sysadmindoc.callshield.data.local.AppDatabase
7+
import com.sysadmindoc.callshield.data.model.BlockedCall
8+
import com.sysadmindoc.callshield.data.model.SmsKeywordRule
9+
import com.sysadmindoc.callshield.data.model.WildcardRule
10+
import kotlinx.coroutines.flow.first
11+
import kotlinx.coroutines.runBlocking
12+
import org.junit.After
13+
import org.junit.Assert.assertEquals
14+
import org.junit.Assert.assertTrue
15+
import org.junit.Before
16+
import org.junit.Test
17+
import org.junit.runner.RunWith
18+
import java.io.File
19+
import java.io.RandomAccessFile
20+
21+
/**
22+
* The corruption path on real SQLite: one table's pages are overwritten on
23+
* disk, Room reports the damage as corruption, and the user's own rows come
24+
* through the rebuild.
25+
*/
26+
@RunWith(AndroidJUnit4::class)
27+
class CorruptionRescueInstrumentedTest {
28+
private val context: Context = ApplicationProvider.getApplicationContext()
29+
private val stores = TestSettingsStores(context)
30+
private val rescueFile = File(context.noBackupFilesDir, "corruption-rescue.json")
31+
32+
@Before
33+
fun setUp() {
34+
context.deleteDatabase(TEST_DB)
35+
rescueFile.delete()
36+
}
37+
38+
@After
39+
fun tearDown() {
40+
context.deleteDatabase(TEST_DB)
41+
rescueFile.delete()
42+
stores.close()
43+
}
44+
45+
@Test
46+
fun aMalformedCallLogKeepsTheUsersRowsThroughTheRebuild() =
47+
runBlocking {
48+
val original = AppDatabase.builder(context, TEST_DB, afterRebuild = {}).build()
49+
val seeded = stores.repository(context, original)
50+
assertTrue(seeded.blockNumber("+12125550101", "robocall", "kept calling"))
51+
assertTrue(seeded.addToWhitelist("+14155550100", "Dr. Patel's office", rangeDigits = 2))
52+
original.spamDao().insertWildcardRule(WildcardRule(pattern = "+1832555*"))
53+
original.spamDao().insertKeywordRule(SmsKeywordRule(keyword = "gift card"))
54+
original.spamDao().insertBlockedCalls(
55+
List(LOG_ROWS) { BlockedCall(number = "+1212555${1000 + it}", isCall = false, smsBody = "x".repeat(LOG_BODY_CHARS)) },
56+
)
57+
val sqlite = original.openHelper.writableDatabase
58+
val pageSize =
59+
sqlite.query("PRAGMA page_size").use {
60+
it.moveToFirst()
61+
it.getLong(0)
62+
}
63+
val rootPage =
64+
sqlite.query("SELECT rootpage FROM sqlite_master WHERE name = 'call_log'").use {
65+
it.moveToFirst()
66+
it.getLong(0)
67+
}
68+
sqlite.query("PRAGMA wal_checkpoint(TRUNCATE)").use { it.moveToFirst() }
69+
original.close()
70+
RandomAccessFile(context.getDatabasePath(TEST_DB), "rw").use { file ->
71+
file.seek((rootPage - 1) * pageSize)
72+
file.write(ByteArray(pageSize.toInt()) { 0x5A })
73+
}
74+
75+
var rebuilds = 0
76+
val corrupt = AppDatabase.builder(context, TEST_DB, afterRebuild = { rebuilds++ }).build()
77+
try {
78+
val failure =
79+
runCatching {
80+
corrupt.openHelper.readableDatabase.query("SELECT smsBody FROM call_log").use { cursor ->
81+
while (cursor.moveToNext()) cursor.getString(0)
82+
}
83+
}.exceptionOrNull()
84+
assertTrue("reading the damaged table should fail as corruption: $failure", AppDatabase.isCorruptionException(failure))
85+
assertEquals("SQLite replaced the file once", 1, rebuilds)
86+
assertTrue(CorruptionRescue.isPending(context))
87+
88+
// The same Room instance carries on with the empty file SQLite made.
89+
assertTrue(CorruptionRescue.importPending(context, corrupt.spamDao(), stores.repository(context, corrupt)))
90+
val dao = corrupt.spamDao()
91+
assertEquals(
92+
0,
93+
corrupt.openHelper.readableDatabase.query("SELECT COUNT(*) FROM call_log").use {
94+
it.moveToFirst()
95+
it.getInt(0)
96+
},
97+
)
98+
assertEquals(listOf("+12125550101"), dao.getUserBlockedNumbers().first().map { it.number })
99+
assertEquals(listOf("+14155550100" to 2), dao.getAllWhitelist().first().map { it.number to it.rangeDigits })
100+
assertEquals(listOf("+1832555*"), dao.getAllWildcardRules().first().map { it.pattern })
101+
assertEquals(listOf("gift card"), dao.getAllKeywordRules().first().map { it.keyword })
102+
} finally {
103+
corrupt.close()
104+
}
105+
}
106+
107+
private companion object {
108+
const val TEST_DB = "corruption-rescue-test.db"
109+
const val LOG_ROWS = 200
110+
const val LOG_BODY_CHARS = 400
111+
}
112+
}

‎app/src/main/java/com/sysadmindoc/callshield/CallShieldApp.kt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import android.util.Log
1515
import androidx.hilt.work.HiltWorkerFactory
1616
import androidx.work.Configuration
1717
import com.sysadmindoc.callshield.data.BackupRestore
18+
import com.sysadmindoc.callshield.data.CorruptionRescue
1819
import com.sysadmindoc.callshield.data.RestoreSentinel
1920
import com.sysadmindoc.callshield.data.SpamHeuristics
2021
import com.sysadmindoc.callshield.data.SpamRepository
@@ -100,6 +101,17 @@ class CallShieldApp :
100101
} catch (e: Exception) {
101102
Log.e("CallShieldApp", "Failed to reconcile an interrupted restore", e)
102103
}
104+
try {
105+
// Rows saved from a damaged database that the last process
106+
// didn't get to put back go in before a worker or the screener
107+
// reads the database.
108+
if (CorruptionRescue.isPending(this)) {
109+
runBlocking { CorruptionRescue.importPending(this@CallShieldApp) }
110+
SyncWorker.syncNow(this)
111+
}
112+
} catch (e: Exception) {
113+
Log.e("CallShieldApp", "Failed to put back the rows saved from a corrupt database", e)
114+
}
103115
val restoreCheckMillis = SystemClock.elapsedRealtime() - restoreCheckStartedAt
104116
// After the restore check, so the caches hold the restored rules.
105117
// Boot and app updates start the process here too, so the first

‎app/src/main/java/com/sysadmindoc/callshield/data/BackupRestore.kt‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -980,6 +980,20 @@ object BackupRestore {
980980

981981
internal fun backupToJson(backup: Backup): String = moshi.adapter(Backup::class.java).toJson(backup)
982982

983+
internal fun backupFromJson(json: String): Backup? = moshi.adapter(Backup::class.java).fromJson(json)
984+
985+
/**
986+
* Adds rows this device wrote itself back without the import checks, the
987+
* way an Undo does (see [toUndoPayload]).
988+
*/
989+
internal suspend fun restoreOwnRows(
990+
context: Context,
991+
backup: Backup,
992+
dao: SpamDao,
993+
repo: SpamRepository,
994+
sections: Set<BackupSection>,
995+
): RestoreResult = restorePayload(context, backup.toUndoPayload(sections), RestoreMode.MERGE, dao, repo, sections)
996+
983997
private fun Backup.rawItemCount(): Long =
984998
blockedNumbers.size.toLong() +
985999
whitelistNumbers.size +

0 commit comments

Comments
 (0)