Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,26 @@
# Changelog

## 6.12.0

A review of the whole history and sync design against how AdGuard actually stores userscript data. AdGuard for Mac, Windows and Android give each tab the stored values as they were when the tab loaded, and a tab never sees what other tabs save after that. The script had been designed as if every tab read the same, current storage. With many tabs left open, that broke in several ways, each now covered by a test that reproduces it.

- Fix (data loss): a tab left open for a while wrote its old copy of the history over the newer one. Visits other tabs recorded after it loaded were lost once their journals were removed ten minutes later. Unmarked items and per-card overrides came back, and with sync off nothing could recover them. Now, on managers that don't report other tabs' writes, a tab open for more than eight minutes saves only to its own journal, and newer tabs fold that journal into the shared history.
- Fix (data loss): a long-open tab could remove another page's journal after that page had written to it again, because its check read the tab's own stale copy. Each journal key is now written for five minutes at most and then left unchanged. It is removed only once it has been unchanged for longer than any tab is allowed to keep writing the shared history, and only after a newer tab has found it already in the saved history.
- Fix (data loss): a visit could be stranded on the device for good. A long-open tab that synced recorded "everything up to now is uploaded", including visits other tabs had recorded that it could not see. The single upload mark is replaced by a list of unconfirmed changes, kept in the history and in each journal and combined across all copies. A change leaves the list only when the worker has confirmed it; an older copy can at worst offer a confirmed change again, which the worker ignores.
- Fix: a long-open tab that synced wrote back its whole copy of the sync settings, which could turn sync back on, restore an old token or endpoint, or undo a pasted sync link. Sync no longer writes the sync settings at all.
- Fix: with other JavStore tabs open, "Restored N visited URLs from the worker" could appear on page after page, and each time the device downloaded its whole history again. Old tabs saved their outdated copy of the history, with the history id they had loaded, so the stored history and the sync settings kept naming different histories, and every new page took that for a lost history. The download position now lives inside the history, in the same save as the visits it covers. It is kept per worker (the worker is named by a hash, never by its address), and journals carry it along with their rows. A page that finds an older copy catches up from where that copy reaches, quietly and only for the difference. In the same situation, an older copy could also lose visits for good, because the position claimed rows the copy did not hold; that can no longer happen either. Histories saved by 6.11.0 or earlier are downloaded again once, without the note. The note is reserved for a history that has lost most of what the worker holds.
- Fix: at the 5,000-visit limit, each new visit could upload the whole history again.
- Fix: a sync that only uploaded changes rewrote the whole history document a second time. It now updates the page's journal only.
- Fix: a device browsing listing pages without pause never downloaded other devices' visits. Page loads inside a session now check the worker again once the sync interval has passed.
- Fix: importing a backup could not restore a visit unmarked after the backup was taken: the newer unmark won the next merge. Such entries are now restored just after the unmark.
- Fix: a sync request that never finished held up syncing in every other tab. Requests now time out on the page's side as well, and no tab holds the sync lock for more than five minutes. Bringing a tab to the front runs one sync instead of two.
- Fix: on AdGuard, the storage-based lock used when Web Locks are missing could not keep tabs apart, and a lease left behind could hold a tab up for 95 seconds. It is no longer used on managers that don't report other tabs' writes; tabs sync side by side, at the cost of at most a duplicate request.
- Fix: opening a new tab could show a visit that had just been unmarked, or cleared, in another tab, and save it back.
- Change: the worker now gives its store a random id and returns it with every answer. A device that finds a different store at the same address, such as one recreated empty, starts over and sends it everything it holds; until now such a worker stayed empty. The worker needs redeploying; the Cloudflare build does that when this reaches `main`. Until it does, an older worker still works, and a reset is noticed from its sequence numbers going backwards.
- Change: a first sync with a worker, or after its store was reset, downloads first and then uploads only what the worker lacks or holds an older version of, rather than everything the device holds. After the update, each JavStore tab reloaded at once on AdGuard starts this way from its own copy, so this keeps that to one download per tab and almost no uploads. Each worker's confirmations are kept apart, so moving between two workers sends each one everything it is missing.
- After updating, reload or close JavStore tabs that were open before the update. Until then they keep running the old code.
- The worker version moves to 6.12.0 with the userscript.

## 6.11.0

- Share initial cloud sync across a browsing session: page navigation and new tabs reuse locally stored history instead of starting another download. Sessions expire after 30 minutes without startup/sync activity; a new session resumes from the durable cursor.
Expand Down
14 changes: 8 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,21 @@ Userscript-manager storage is local to one device and AdGuard has been known to

Once it is deployed, open the `JVS` panel → **Cloud sync**, enter the worker URL and the access token you generated, tick the box, and save. Repeat on every device with the same URL and token.

- The first page in a browsing session checks the worker; later pages and new tabs immediately use the shared local history. A session is shared across tabs and expires after 30 minutes without startup/sync activity. Starting another session checks for new changes using the saved cursor; it does not download the entire history again.
- The first page in a browsing session checks the worker; later pages and new tabs use the shared local history, and check again once the sync interval has passed so that visits from other devices keep arriving while you browse. A session is shared across tabs and expires after 30 minutes without startup/sync activity. Starting another session checks for new changes from the saved position; it does not download the entire history again.
- Opening 30–50 item tabs from the context menu still records every visit when its item page loads. Uploads wait a few seconds and collect visits from all tabs; tabs whose changes were already uploaded skip their redundant requests. Advancing the listing to page 2 does not restart restoration.
- **Sync now** always checks the worker. Focus refreshes and the configured periodic refresh use shared progress, so tabs do not independently download the same changes. Hidden tabs skip periodic refreshes, but their new visits can still upload. The panel's status line reports the last sync or the reason the last one failed.
- Devices exchange only what changed: each remembers the sequence number it last saw and the moment it last pushed, so incremental checks exchange only new rows, and an ordinary page load during an active session needs no cloud request. The first sync on a device transfers everything, and so does the next sync on a device whose userscript manager lost its history but kept the sync settings: the history carries an id that the saved position is tied to, so a history without it is downloaded again rather than left behind.
- Tabs on one device share history, a sync cursor, and session progress. Before deciding what to send, a sync merges what the device has on disk, so a visit recorded in one tab is not passed over by another tab's sync; and an entry that turns up stamped before the last push—a click replayed from the previous page, a restored backup—pulls the mark back to itself rather than being left behind it.
- Devices exchange only what changed. The stored history records, in the same save as the visits, how far into the worker's history it is complete, and keeps a list of the changes made on the device that the worker has not confirmed yet. Incremental checks exchange only new rows, and an ordinary page load during an active session needs no cloud request. The first sync with a worker downloads everything and uploads only what the worker lacks, and so does the next sync on a device whose userscript manager lost its history but kept the sync settings: a history with no position for the worker is downloaded again rather than left behind.
- Each worker store has a random id. A device that finds a different store at the same address (for example one recreated empty) starts over against it and sends it everything it holds.
- Sync never writes the sync settings themselves (endpoint, token, on/off, interval); only the panel and sync links do, so a tab that was opened before a change cannot undo it.
- The worker applies the same rules as the local merge—newest timestamp wins per URL—so a device that has been offline for a week cannot overwrite what the others recorded. A device that pushes a stale entry is handed the winning one back.
- No device can delete visited history from the worker. **Clear visited history**, retention pruning, unmarking a card and a backup import that leaves entries out all apply to the device they happen on; the worker and the other devices keep their copies. The worker drops only its oldest visits beyond 50,000, to bound its storage. Per-card overrides are choices rather than history, so removing one still reaches every device.
- Because a Durable Object is single-threaded, each sync's read-merge-write is serialized and strongly consistent: two devices syncing in the same second queue behind one another rather than racing.
- Settings sync too, including the match and excluded terms, and they merge setting by setting: each one goes to the device that changed it last. A setting a device has never changed carries no timestamp and cannot win, so a new install picks up your keywords from the worker instead of pushing its defaults over them, even if you changed something else on it first. **Restore defaults** and a backup import set every setting on purpose, so they win everywhere.
- The Cloud sync panel shows the worker version from the worker's last answer. When the worker is behind the userscript, a **⚠ Redeploy worker** button appears on screen next to the JavStore controls; tap it for the versions. The worker redeploys through Cloudflare whenever `main` changes, so a lasting warning means its Cloudflare build did not go through. When the worker is ahead instead, an **⚠ Update script** button appears on screen and links to the latest release, because the userscript manager has not installed it yet.
- AdGuard for Android clears a userscript's storage whenever it installs an update. After setting sync up, press **Copy sync link** and keep the link somewhere private, such as a password manager. After a wipe, open it on JavStore, or paste it into the endpoint box and save, to set everything up again. The link carries your token, so treat it like a password.
- The endpoint and token live only on the device they were entered on. They are never written into the synced document, never appear in an exported backup, and are not left in the page: the token box stays empty once a token is stored and only reports that one exists.
- A worker that is unreachable does not affect anything locally: history is still saved to the userscript manager, and the next sync picks it up. Failed or incomplete restoration is not marked complete. If local history is actually lost, recovery bypasses the session marker and downloads it again; **Restored … visited URLs** reports that recovery, not ordinary page navigation.
- Current browsers coordinate cloud work with Web Locks. Browsers without that API use a userscript-storage lease with expiry and ownership checks, so a tab that disappears cannot hold sync indefinitely. Session/progress metadata stays in userscript-manager storage; credentials and history are not placed in page `localStorage`.
- A worker that is unreachable does not affect anything locally: history is still saved to the userscript manager, and the next sync picks it up. Failed or incomplete restoration is not marked complete. If local history is actually lost, recovery bypasses the session marker and downloads it again; **Restored … visited URLs** reports that recovery, not ordinary page navigation. A history that is merely behind catches up on the difference without the note.
- Current browsers coordinate cloud work with Web Locks, and no tab holds them for more than five minutes. Managers that notify tabs of each other's writes fall back to a userscript-storage lease where Web Locks are missing; elsewhere tabs sync side by side, which costs at most a duplicate request. Session metadata stays in userscript-manager storage; credentials and history are not placed in page `localStorage`.
- **Sync now** reports both directions—what went up and what came down—so a device that is failing to push is not mistaken for one with nothing to push.
- Restoring a backup keeps the backup's own timestamps, so the restored entries reach the worker and the other devices instead of being discarded by them. Entries the backup leaves out are dropped on the importing device only. Entries a history clear on this device already covers cannot be restored, and the panel says how many were skipped.

Expand All @@ -66,6 +67,7 @@ Once it is deployed, open the `JVS` panel → **Cloud sync**, enter the worker U
- Optional userscript APIs: `GM_addValueChangeListener` and `GM_registerMenuCommand`. AdGuard does not implement either; the script polls for changes instead of being notified, and its own on-page control replaces the manager menu.
- `GM_getValue`/`GM_setValue` are supported in both the synchronous Tampermonkey style and the asynchronous GM4 style AdGuard uses. Values are stored as JSON strings, because GM4 only promises to keep strings, numbers and booleans, and AdGuard for Android drops stored objects when the page reloads.
- `GM_listValues` and `GM_deleteValue` let each page keep its own journal of what it recorded, so tabs saving at the same moment cannot overwrite each other's visits. AdGuard and Tampermonkey provide both; an engine without them falls back to a single stored document.
- AdGuard (for Mac, Windows and Android) gives each tab the stored values as they were when the tab loaded, and never shows it what other tabs save afterwards. On managers without `GM_addValueChangeListener`, a tab open for more than eight minutes therefore saves only to its own journal and leaves the shared history to newer tabs, which fold its journal in. Journal keys are each written for five minutes at most and removed only once a newer tab has found them in the saved history, so a long-open tab can neither write an older history over a newer one nor remove another tab's journal. Long-open tabs do not show what other tabs recorded after they loaded until they sync or are reloaded.

The script depends on JavStore's current page structure. If the site changes, please [open an issue](https://github.com/jitdor/userscript-javstore/issues).

Expand All @@ -75,6 +77,6 @@ The distributable is [`javstore-full-layout-cleanup.user.js`](./javstore-full-la

Releases publish themselves: a push to `main` whose `@version` header names a version with no release yet runs the tests and publishes that release, which is what `@updateURL`/`@downloadURL` point at. A push that does not bump the header is a no-op, so releasing a change means bumping `@version` and `SCRIPT_VERSION` and adding the matching `## <version>` section to `CHANGELOG.md` in the same pull request. The workflow can still be run manually to release from a branch.

`npm install && npm test` runs the link, startup, storage-persistence, and cloud-sync suites (a current Node.js 22 or newer with `node:sqlite`; CI uses Node.js 22). The suites load the userscript into jsdom windows that share one asynchronous value store with no change notifications—an AdGuard-shaped engine—and assert that history survives reloads, concurrent tabs, interrupted saves, and explicit removals. The store can be given random per-call latency, which is how the suite reproduces several tabs' saves overlapping on a slow engine. The sync suite additionally runs the real worker from [`worker/src/worker.mjs`](./worker/src/worker.mjs): its Durable Object class executes against a `node:sqlite` database standing in for `ctx.storage.sql`, with each jsdom window acting as a separate device, so both halves of a sync are covered against real SQL.
`npm install && npm test` runs the link, startup, storage-persistence, and cloud-sync suites (a current Node.js 22 or newer with `node:sqlite`; CI uses Node.js 22). The suites load the userscript into jsdom windows that share one asynchronous value store with no change notifications, or (with `snapshotPerTab`) give each window its own copy of the store taken when it opened, as AdGuard does, and assert that history survives reloads, concurrent tabs, interrupted saves, and explicit removals. The store can be given random per-call latency, which is how the suite reproduces several tabs' saves overlapping on a slow engine. The sync suite additionally runs the real worker from [`worker/src/worker.mjs`](./worker/src/worker.mjs): its Durable Object class executes against a `node:sqlite` database standing in for `ctx.storage.sql`, with each jsdom window acting as a separate device, so both halves of a sync are covered against real SQL.

Session regression tests assert request counts and downloaded rows across navigation, concurrent first tabs, a 50-tab context-menu burst, session expiry, stale tab cursors, failed progress writes, and actual history loss.
Loading
Loading