fix(security): add retry and error handling for PCP upload to prevent biometric data loss

[failsafesecurity]

Security Fix: Fire-and-Forget PCP Upload Race Condition Allows Biometric Data Loss

Severity: Medium
Finding ID: cmsdegzmm003tob01xjqt0r8f

Problem

The PCP (Personal Custody Package) upload in src/agents/data_uploader.rs is fire-and-forget. After a successful signup, the PCP upload runs asynchronously via FuturesUnordered with these issues:

1. No retry limit: Network errors (non-4xx) retry infinitely in a tight loop with no backoff, hammering a failing server
2. Silent data loss on 4xx: Client errors (400, 401, 403, etc.) silently break and drop the biometric data — no log, no metric, no recovery path
3. No failure surfacing: Failures are not tracked or reported to the operator. The upload task simply returns (tier, id) regardless of success or failure
4. No backoff: Even for transient network errors, retries happen immediately with zero delay

Code Location

// src/agents/data_uploader.rs — upload_pcp()
loop {
    let response = backend::upload_personal_custody_package::request(...).await;
    match response {
        Ok(()) => { break; }
        Err(err) => {
            // 4xx: silently break and lose data
            if status.is_client_error() { break; }
            // Other errors: retry immediately, no backoff, no limit
        }
    }
}

Fix

1. Exponential backoff: Added PCP_RETRY_INITIAL_BACKOFF (1s) with exponential growth, capped at PCP_RETRY_MAX_BACKOFF (30s). Each retry logs the backoff duration.

2. Max retry limit: Added PCP_MAX_RETRIES (5 attempts). After exhausting retries, the failure is surfaced via:

• Error log: PCP upload tier {tier} failed after {N} retries. Biometric data may be lost. Operator intervention required.
• Metric: main.count.signup.result.failure.upload_custody_images with type:max_retries_exceeded

3. Better error logging for 4xx: Client errors now log Biometric data may be lost. Manual intervention required. instead of silently dropping.

4. Retry attempt tracking: Each retry logs the attempt number (attempt {N}/{MAX_RETRIES}) for observability.

Constants

const PCP_MAX_RETRIES: u32 = 5;
const PCP_RETRY_INITIAL_BACKOFF: Duration = Duration::from_secs(1);
const PCP_RETRY_MAX_BACKOFF: Duration = Duration::from_secs(30);

Testing

• Network errors now retry with exponential backoff (1s, 2s, 4s, 8s, 16s) instead of immediately
• After 5 retries, the upload fails gracefully with clear error logging
• 4xx errors still break immediately (non-retryable) but now log operator guidance
• Successful uploads are unaffected

---

Submitted by FailSafe Security Research