Smart Memories - surface photo collections proactively

[rohan-pandeyy]

Problem

PictoPy's library is entirely user-initiated. Nothing surfaces unless you go looking for it: no "this day last year", no trip recap, no prompt to revisit anything. Every other photo app does this, and PictoPy already stores everything needed to — capture dates, GPS, face clusters, SigLIP2 embeddings and semantic labels — it just never uses them to offer anything.

There is an existing memory_clustering.py, but it is not a foundation to build on. It re-clusters the entire library on every API call, persists nothing, and has no notion of ranking or of "have I already seen this?". A large portion of it is unreachable, and it carries a hardcoded 30-city reverse-geocode table.

Proposal

Persisted, scored photo collections that surface on their own, shown in a full-screen story viewer.

Curation — three triggers

• Anniversary — photos from this calendar date in previous years
• Import event — a burst of photos that hangs together in time and place
• Semantic event — photos SigLIP2 recognises as one occasion (wedding, diwali, a day at the beach)

Memories should be idempotent: regenerating must not produce a different set from the same photos, and must never resurface something already watched or dismissed.

Ranking

• Score every candidate photo from signals already in the database — favourites, named people, face presence, semantic/event confidence, GPS novelty, album membership
• Weights adjustable from settings
• Missing data must not be a penalty: a photo with no GPS should score the same as one taken at home, not worse
• Suppress near-duplicate shots, and spread the survivors across the event's span

Naming

• Title a memory from what the AI recognised, when it recognised something confidently
• Fall back to a readable stand-in ("Remember this day?") rather than a bare date, chosen stably so a rebuild doesn't rename a memory the user has already seen

Videos

• Include short clips shot during the same span, as punctuation between stills
• Guardrails so a story doesn't become a playlist: a cap per memory scaled to its size, a maximum clip length, and a total-video budget
• Clips play muted with no controls; unmuting one should silence the background music and vice versa

Story viewer

• Full-screen playback with segmented progress bars, autoplay, keyboard and swipe navigation
• Grid page of all memories, plus a filmstrip for jumping between them
• Settings page: enable/disable, slide duration, scoring weights, music toggle

API

• Persisted endpoints under /memories with proper response models — list, single story, "the one to surface now", generate, mark viewed/dismissed, delete
• A status endpoint so a background task can tell whether a run is in progress

Data quality prerequisites

Memories are only as good as the capture dates behind them, and there are known problems here that have to be fixed first — a library copied from another machine currently reads as though every photo was taken on import day:

• Read EXIF capture dates correctly (they live in the sub-IFD, not IFD0)
• Stop filesystem mtimes being stored as if they were capture times
• Read Google Takeout sidecar metadata, since Takeout strips EXIF from part of its own export
• Read capture dates from MP4/MOV containers — videos currently have no date source at all

Pipeline integration

• Curate after indexing, after folder sync, and after AI tagging completes
• Never curate mid-index — scoring a half-written library produces nonsense
• Curation must never fail an import

Out of scope for the first pass

• Desktop notifications (Tauri background task, launch-time trigger) — worth its own PR, but the schema should be designed with it in mind from the start
• Rediscovery — bringing a memory back after it hasn't been seen for months. The first pass will produce a queue that drains; this is what turns it into something ongoing

Acceptance criteria

• Memories persist across restarts and are not recomputed per request
• Regenerating twice over an unchanged library produces identical memories
• A watched or dismissed memory never resurfaces
• Two consecutive memories don't recycle the same photos
• Curation failure never breaks an import or a sync
• Backend and frontend tests cover the scoring rules, each trigger, and the viewer