docs: document Smart Memories and sync the backend directory tables

docs/backend/backend_python/database.md

@@ -7,7 +7,7 @@ PictoPy uses several SQLite databases to manage various aspects of the applicati
 ## Database Schema
 
 <!-- markdownlint-disable MD033 -->
-<iframe width="560" height="315" src='https://dbdiagram.io/e/6a593dd1c3a90dd98d55554d/6a5b4a02067336e1dea2347a'> </iframe>
+<iframe width="765" height="600" src='https://dbdiagram.io/e/6a593dd1c3a90dd98d55554d/6a5b4a02067336e1dea2347a'> </iframe>
 <!-- markdownlint-enable MD033 -->
 
 Alternatively, [click here to view the interactive DB schema diagram in a new tab](https://dbdiagram.io/d/PictoPy-6a593dd1c3a90dd98d55554d).

docs/backend/backend_python/image-processing.md

@@ -76,6 +76,105 @@ For the full technical breakdown — architecture diagrams, database schema,
 model calibration details, and known limitations — see the dedicated
 [Semantic Search](semantic-search.md) page.
 
+## Capture Dates and Location
+
+Indexing records two different dates per file, and they are not
+interchangeable.
+
+| Where it lives                                    | What it holds                                                                             | Who reads it                           |
+| ------------------------------------------------- | ----------------------------------------------------------------------------------------- | -------------------------------------- |
+| `date_created`, inside the `metadata` JSON column | The best date available, always populated — it falls back to the file's modification time | Gallery display and sorting            |
+| `captured_at`, a column on `images` and `videos`  | When the shutter actually fired; `NULL` when nothing trustworthy was found                | The Memories feature, and nothing else |
+
+The same `metadata` JSON carries a `date_source` field saying where the date
+came from:
+
+| `date_source` | Meaning                                          | Trusted |
+| ------------- | ------------------------------------------------ | ------- |
+| `exif`        | Read from the image's own EXIF tags              | Yes     |
+| `sidecar`     | Read from a Google Takeout sidecar JSON file     | Yes     |
+| `container`   | Read from an MP4/MOV container box               | Yes     |
+| `filesystem`  | The file's mtime, standing in for a capture date | No      |
+| `unknown`     | The file could not be opened or stat'd at all    | No      |
+
+The trusted set is `TRUSTED_DATE_SOURCES` in
+`app/utils/extract_location_metadata.py`. `MetadataExtractor.extract_datetime()`
+returns `None` when `date_source` is anything outside it, and
+`video_util_prepare_video_records()` applies the same test, so an untrusted
+date is written to `captured_at` as `NULL` while `date_created` keeps it for
+the UI.
+
+### EXIF
+
+`DateTimeOriginal` is not in IFD0 — it sits behind the EXIF sub-IFD pointer
+`0x8769`, and Pillow's `getexif()` returns IFD0 only. `_extract_capture_datetime()`
+searches both: for each of `DateTimeOriginal`, `DateTimeDigitized` and
+`DateTime`, in that order, it reads the sub-IFD first and then IFD0. If nothing
+parses, `date_created` falls back to the file mtime and `date_source` becomes
+`filesystem`.
+
+GPS comes off the same EXIF object via `_extract_gps_coordinates()`, which
+reads the GPSInfo IFD and converts degrees/minutes/seconds to signed decimal
+degrees.
+
+### Google Takeout sidecars
+
+A Google Photos export strips EXIF from part of its own library and parks the
+real values in a sibling JSON file. When EXIF yields no date or no coordinates,
+`takeout_sidecar_read()` (`app/utils/takeout_sidecar.py`) looks for that file.
+
+| Sidecar spelling                       | How it is found                        |
+| -------------------------------------- | -------------------------------------- |
+| `<file>.supplemental-metadata.json`    | Probed by exact name                   |
+| `<file>.json`                          | Probed by exact name                   |
+| `<file>.supplemental-metadata(1).json` | Found by prefix in a directory listing |
+
+The exact spellings cost one `stat` each and are tried first; the directory is
+only listed if neither produced a usable sidecar, and that listing is cached —
+one entry, keyed on the directory path and its mtime — so a folder of N photos
+does not cost N scans.
+
+Once a sidecar is open:
+
+- `photoTakenTime` is preferred over `creationTime`, which is the upload time.
+- Coordinates come from `geoData`, falling back to `geoDataExif`.
+- Takeout writes `0/0` rather than null for "no location", so an exact
+  (0.0, 0.0) pair is discarded.
+- Album-level metadata files match no image and are skipped by key: a payload
+  containing `entries` or `albumData` is rejected.
+
+The datetime is returned in EXIF format (`%Y:%m:%d %H:%M:%S`) so callers parse
+it exactly as they parse a real EXIF value.
+
+### Videos
+
+Video capture dates are parsed straight out of the ISO base media container
+boxes by `app/utils/video_capture_date.py`. There is no ffmpeg or ffprobe
+dependency — PictoPy ships through PyInstaller and cannot assume either exists
+on the machine. OpenCV, which handles the poster frame, exposes no creation
+date at all.
+
+`_resolve_capture_date()` in `app/utils/videos.py` takes the first source that
+answers:
+
+| Order | Source                                                          | `date_source` | Notes                                                                          |
+| ----- | --------------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------ |
+| 1     | `com.apple.quicktime.creationdate` in `moov/meta` (keys + ilst) | `container`   | Carries its own UTC offset, so it yields the wall-clock time of the shot       |
+| 2     | Google Takeout sidecar                                          | `sidecar`     | Same reader as photos                                                          |
+| 3     | `moov/mvhd` creation time                                       | `container`   | UTC seconds since 1904-01-01, converted to local time; a re-encode rewrites it |
+| 4     | File mtime                                                      | `filesystem`  | Untrusted, so it never reaches `captured_at`                                   |
+
+Values outside 1990-01-01 through two days from now are treated as noise — a
+container can carry 0, or a clock that was never set — and a malformed keys
+table is bounded at `MAX_METADATA_KEYS = 512` entries.
+
+The filesystem mtime is recorded in its own field, `metadata["file_modified"]`.
+`video_util_source_is_unchanged()` compares a video's size and that stored mtime
+to decide whether to re-index it; a row without `file_modified` is treated as
+changed, so its container is read once and its date corrected.
+
+For what consumes `captured_at`, see the [Memories](memories.md) page.
+
 ## How It All Fits Together
 
 When you add a new photo, we first look for objects and faces. If we find faces, we generate embeddings for them. These embeddings then get added to our face clusters.
@@ -126,13 +225,13 @@ Here are some key parameters for the main models used in PictoPy's image process
 
 ### Semantic Search (SigLIP2)
 
-| Parameter | Value | Description |
-| -------------------------- | ---------------------------------- | ---------------------------------------------------------------------------- |
-| Default checkpoint | `base` | Set via `SIGLIP2_ACTIVE_CHECKPOINT`; `large` and `so400m` also exist but ship placeholder registry entries only (see [Semantic Search](semantic-search.md#model-distribution-and-checkpoints)). |
-| Input resolution (`base`) | 224 × 224 | Larger checkpoints use 384 × 384. |
-| Embedding dimension | 768 | Same dimensionality for both the image and text towers. |
-| `SIGLIP2_EMBED_BATCH_SIZE` | 8 | Images per batch during the background embedding pass. |
-| `SIGLIP2_MATCH_THRESHOLD` | 0.01 | Minimum sigmoid score to count as a match. SigLIP2's absolute scores run low even for real matches — this is expected, not a bug. |
-| Output | Sorted, scored image list | Scores are rounded to 4 decimal places server-side and never shown in the UI. |
+| Parameter                  | Value                     | Description                                                                                                                                                                                     |
+| -------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Default checkpoint         | `base`                    | Set via `SIGLIP2_ACTIVE_CHECKPOINT`; `large` and `so400m` also exist but ship placeholder registry entries only (see [Semantic Search](semantic-search.md#model-distribution-and-checkpoints)). |
+| Input resolution (`base`)  | 224 × 224                 | Larger checkpoints use 384 × 384.                                                                                                                                                               |
+| Embedding dimension        | 768                       | Same dimensionality for both the image and text towers.                                                                                                                                         |
+| `SIGLIP2_EMBED_BATCH_SIZE` | 8                         | Images per batch during the background embedding pass.                                                                                                                                          |
+| `SIGLIP2_MATCH_THRESHOLD`  | 0.01                      | Minimum sigmoid score to count as a match. SigLIP2's absolute scores run low even for real matches — this is expected, not a bug.                                                               |
+| Output                     | Sorted, scored image list | Scores are rounded to 4 decimal places server-side and never shown in the UI.                                                                                                                   |
 
 Note: Some of these values are default parameters and can be adjusted when initializing the models or during runtime, depending on the specific use case or performance requirements.