Skip to content

fix(activesync): separate PING watermark from SYNC modseq to stop iOS…#41

Draft
TDannhauer wants to merge 5 commits into
FRAMEWORK_6_0from
fix/ActiveSync_loops
Draft

fix(activesync): separate PING watermark from SYNC modseq to stop iOS…#41
TDannhauer wants to merge 5 commits into
FRAMEWORK_6_0from
fix/ActiveSync_loops

Conversation

@TDannhauer
Copy link
Copy Markdown
Contributor

@TDannhauer TDannhauer commented Jun 5, 2026

Summary

Fixes ~3-second PING loops on iOS devices with many IMAP folders. Two separate
failure modes were sharing the same state logic:

  1. MODSEQ drift: PING compared against the SYNC modseq but intentionally
    did not advance it, so the same change was re-reported every few seconds.
  2. sync_pending during PING: An interrupted SYNC with <MoreAvailable />
    left a UID batch in sync_pending. PING treated that batch as a change
    signal, never called ping(), and could not persist a PING checkpoint.

The fix is a three-state model with deliberately separate code paths:

State Purpose Storage
SYNC watermark CHANGEDSINCE $_status / modseq()
PING watermark "client notified?" $_pingStatus / ps in sync_data
SYNC pending MOREAVAILABLE batch sync_pending (SYNC only)

All EAS Protocol versions are affected handling emails via imap

Changes

Core logic

  • Folder/Imap.php: $_pingStatus, pingModseq(), acknowledgePingStatus();
    optional ps key in sync_data serialization
  • Imap/Adapter.php: ping() compares against the PING watermark, not SYNC modseq()
  • State/Base.php: documented SYNC vs. PING paths in getChanges(); ignore
    sync_pending when ping=true; persist PING checkpoint via
    save(['preservePending' => true])
  • State/Sql.php / State/Mongo.php: preservePending keeps sync_pending
    intact via _syncPendingBlob

Documentation

  • Architecture comments at the central decision points (getChanges(), $_pingStatus,
    sync_pending column, preservePending)
  • Cross-references between Base, Folder, and Adapter for overall context

Tests

  • ImapFolderTest — PING checkpoint, serialization, stale SYNC modseq scenario
  • ImapAdapterTest — PING uses pingModseq(), not SYNC modseq()
  • StateSqlPreservePendingTestpreservePending does not wipe sync_pending

Verification

Tested against a production iOS device with 100+ mail folders:

Metric Before After
Found changes! bursts ~12,200 / ~3s interval None
False PING folder responses Every ~3s Stopped
ps in sync_data Missing for affected collections Present
Heartbeat behavior Immediate re-response Normal Sleeping for N seconds

Note: Collections with an open sync_pending batch from an earlier interrupted
SYNC are expected to retain that state. PING no longer loops on it; the client must
complete a SYNC for those collections to drain the batch.

Test plan

  • phpunitImapFolderTest, ImapAdapterTest, StateSqlPreservePendingTest
  • Device log: no Found changes! burst, no false PING folder responses
  • DB: ps present in sync_data for affected collections; sync_pending preserved on PING save
  • Manual: observe iOS mail client for 10–15 minutes under normal use
  • Optional: force SYNC on a collection with open sync_pending → batch clears after SYNC completes

TDannhauer and others added 5 commits June 8, 2026 08:31
… mail loops

x(activesync): stop iOS PING loops with separate SYNC and PING watermarks
Email collections used one modseq for both SYNC (CHANGEDSINCE) and PING
(IMAP STATUS), and PING treated sync_pending as a change signal. That
caused infinite ~3s PING loops when SYNC modseq lagged behind the server,
and when a MOREAVAILABLE batch remained in sync_pending.
Introduce a three-state model: SYNC watermark ($_status), PING watermark
($_pingStatus / ps in sync_data), and sync_pending (SYNC only). PING now
polls IMAP STATUS via pingModseq(), advances the checkpoint on detection,
and saves with preservePending so in-flight MOREAVAILABLE batches survive.
Add unit tests for checkpoint serialization, PING-vs-SYNC modseq separation,
and preservePending saves.
fix(activesync): reject corrupt collection sync_data on load and save

Treat deserialized empty arrays and non-Folder_Base blobs as missing state
instead of using them as folder objects. Refuse save() when _folder is not a
valid Folder_Base instance to prevent writing a:0:{} into horde_activesync_state.
…export

fix(activesync): repair initial sync, corrupt state, and nested MIME export

Follow-up to the PING/SYNC watermark split. Several bugs still blocked
reliable mail delivery on iOS ActiveSync:

- Reject empty or non-folder sync_data on load and refuse to persist
  invalid collection state on save, preventing corrupted rows from
  breaking PING/SYNC and new-mail push.
- Defer marking initial sync complete until exported UIDs are
  acknowledged; fix unserialize treating haveInitialSync=false as
  complete when the hi key is present.
- Stop the SYNC export loop from treating failed exports as progress
  and from dropping sync_pending entries on retriable backend errors.
- Retry IMAP body fetches without BODY[].SIZE when the server returns
  no data for deeply nested MIME parts (Dovecot), so signed messages
  with inline PGP key material export correctly.
- Treat application/pgp-keys like S/MIME signatures for attachment
  detection and log message build failures instead of silently
  omitting messages from export.

Add unit tests for initial-sync acknowledgement, corrupt state
handling, preservePending saves, nested MIME body fetch fallback,
and PGP-key MIME structures.
…ow locks

Parallel SYNC and PING workers could load and save the same sync_key
concurrently, overwriting sync_data and sync_pending and causing Invalid
sync_data corruption during heavy initial sync.
Hold a row lock from state load through save() or updateSyncStamp():
SQL uses SELECT ... FOR UPDATE in a transaction; Mongo uses a sync_lock
field with findAndModify(), including stale-lock recovery. Release locks
on collection switch, save, stamp-only updates, and __destruct().
Add updateSyncStamp() to State_Mongo for parity with SQL. Add unit tests
under StateTest/Sql and StateTest/Mongo (RowLockTest, InitialSyncTest).
@ralflang ralflang force-pushed the fix/ActiveSync_loops branch from 8901cd1 to c7e372f Compare June 8, 2026 07:25
Copy link
Copy Markdown
Member

@ralflang ralflang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase before adding code to this. I will put it back into draft status now.

@ralflang ralflang marked this pull request as draft June 8, 2026 07:27
@ralflang ralflang mentioned this pull request Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants