diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ed79dfb..9095761 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -6,7 +6,7 @@ better-link is a single Cloudflare Worker with three kinds of request. A convers ## Codemap -- `src/index.ts` is the only entry point. `createWorker` routes `/` to `convert`, paths under `/media/` to `mediaRedirect`, and every other path to `shareRedirect`. +- `src/index.ts` is the only entry point. `createWorker` routes `/` to `convert`, paths under `/media/` to `mediaRedirect`, `/log` to `sharedLinkLogPage` when the log binding exists, and every other path to `shareRedirect`. - `src/clean.ts` removes tracking parameters (`cleanUrl`). - `src/expand.ts` expands `/share/` short links (`expandShareLink`) and decides which URLs can be converted (`isShareable`). - `src/crawler.ts` tells crawlers from people by User-Agent (`isCrawler`). @@ -16,6 +16,7 @@ better-link is a single Cloudflare Worker with three kinds of request. A convers - `src/og.ts` renders a `Post` into og and twitter card HTML (`renderOgPage`). - `cloudflare.config.ts` holds the Worker settings that `cf` reads. It declares the `SHARED_LINKS` KV binding only when the build variable `LOG_SHARED_LINKS` is `1`. - `src/shared-link-log.ts` (`logSharedLink`) writes one record per successful `GET /?url=` to `env.SHARED_LINKS` through `ctx.waitUntil`. Without the binding it does nothing. +- `src/shared-link-log-page.ts` (`sharedLinkLogPage`) renders `GET /log`: one `list` of a UTC day's keys, shown only when Cloudflare Access set `ctx.access`. Without the binding `/log` falls through to `shareRedirect`. ## Invariants diff --git a/README.md b/README.md index ba890a4..127919d 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,7 @@ The `url` parameter must be percent-encoded in both modes. The examples assume t | `GET //` | Crawlers: og preview page or 302 to a fix service. Regular browsers: 302 to the original URL | | `GET /www.instagram.com/p//` | The n-th image of a multi-image post (1-based) | | `GET /media/...` | 302 to a freshly signed CDN image or video URL, used by og tags | +| `GET /log[?day=YYYY-MM-DD]` | With the shared-link log enabled: one UTC day of logged links, newest first. `403` unless the request passed Cloudflare Access | ## Deploy your own @@ -76,6 +77,8 @@ Cloudflare copies this repository into your GitHub or GitLab account, builds it Optional: to record each share link that `GET /?url=` produces, set the build variable `LOG_SHARED_LINKS` to `1` in Workers Builds. The config then declares a KV binding named `SHARED_LINKS`, and each record expires after 90 days. Without the variable no binding is declared and the Worker behaves as before. Unset the variable and redeploy to remove the binding; the namespace and its records stay. +To read the log, open `/log` on your Worker. The page answers `403` until Cloudflare Access protects it: in Zero Trust > Access > Applications, add a self-hosted application whose domain is your Worker's hostname with the path `/log`, and allow only yourself. Protect only that path, not the whole Worker; protecting the Worker also blocks share links and `/media/`, so chat apps lose their previews. + Once deployed, the Worker is reachable at `https://better-link..workers.dev` (or under the name you picked). Use that as `https://link.example` in the examples above, or attach a custom domain in the Cloudflare dashboard. To deploy manually instead: diff --git a/README.zh-TW.md b/README.zh-TW.md index ea13a59..903798b 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -65,6 +65,7 @@ curl 'https://link.example/?url=https%3A%2F%2Fwww.instagram.com%2Fp%2FABC%2F%3Fi | `GET /<原網域>/<路徑>` | 爬蟲：og 預覽頁或 302 到修正服務。一般瀏覽器：302 回原網址 | | `GET /www.instagram.com/p//` | 多圖貼文的第 n 張（1 起算） | | `GET /media/...` | 302 到當下重新簽章的 CDN 圖片或影片網址，供 og 標籤使用 | +| `GET /log[?day=YYYY-MM-DD]` | 啟用分享連結紀錄時：某一個 UTC 日的紀錄，新的在前。請求沒有通過 Cloudflare Access 時回 `403` | ## 部署自己的版本 @@ -76,6 +77,8 @@ Cloudflare 會把這個 repo 複製到你的 GitHub 或 GitLab 帳號、建置 可選功能：要記錄 `GET /?url=` 產生的每一筆分享連結，在 Workers Builds 的建置變數設定 `LOG_SHARED_LINKS` 為 `1`。設定檔會宣告一個名為 `SHARED_LINKS` 的 KV binding，每筆紀錄 90 天後過期。沒有設定變數時不宣告任何 binding，Worker 的行為和以前相同。要取消時移除變數並重新部署即可移除 binding；命名空間和其中的紀錄會保留。 +要查看紀錄，打開 Worker 的 `/log`。在 Cloudflare Access 保護這個路徑之前，頁面一律回 `403`：到 Zero Trust > Access > Applications 新增一個 self-hosted application，網域填 Worker 的主機名稱、路徑填 `/log`，並只允許你自己登入。只保護這個路徑，不要保護整個 Worker；保護整個 Worker 會連分享連結和 `/media/` 一起擋住，聊天軟體就拿不到預覽。 + 部署完成後網址是 `https://better-link.<你的子網域>.workers.dev`（或你取的名稱），把上面範例中的 `https://link.example` 換成它即可，也可以在 Cloudflare 後台綁自訂網域。 手動部署： diff --git a/docs/spec/shared-link-log-page-behind-access.md b/docs/spec/shared-link-log-page-behind-access.md new file mode 100644 index 0000000..c974f08 --- /dev/null +++ b/docs/spec/shared-link-log-page-behind-access.md @@ -0,0 +1,22 @@ +--- +id: shared-link-log-page-behind-access +status: accepted +scope: + - "src/index.ts" + - "src/shared-link-log-page.ts" +verify: check:bun test ./test/shared-link-log-page-behind-access.test.ts +related: [shared-links-logged-to-kv-at-conversion, free-plan-bindings-only, cpu-budget-10ms] +source: null +adr: null +--- +`GET /log` 只在請求經過 Cloudflare Access 驗證（`ctx.access` 有值）時才讀取 `SHARED_LINKS`，否則回 403 且不呼叫任何 KV 方法；沒有綁定時 `/log` 照原本的分享連結路由處理（404）。 + +登入由部署的人在 Zero Trust 建立的 self-hosted application 負責，應用程式網域只填 `<網域>/log` 這個路徑。保護整個 Worker 會連分享連結和 `/media/` 一起擋住，爬蟲拿不到預覽。Worker 不自己驗 JWT，只看執行環境填入的 `ctx.access`；這一層檢查是為了 Access 沒設好時（例如只保護自訂網域，卻從 `workers.dev/log` 連進來）仍然不會吐出紀錄。 + +一次瀏覽只呼叫一次 `list`，`prefix` 是一個 UTC 日期（`YYYY-MM-DD`，由 `?day=` 指定，預設今天），因為 key 以 ISO 時間戳開頭。免費方案每天最多寫 1,000 筆、一次 `list` 最多回 1,000 筆，所以一天的紀錄一次拿完（https://developers.cloudflare.com/kv/platform/limits/ 、https://developers.cloudflare.com/kv/api/list-keys/ ，2026-10-16 查閱）；list 請求在免費方案每天 1,000 次（https://developers.cloudflare.com/kv/platform/pricing/ ），所以不做跨日翻頁或搜尋。資料只讀 key 的 metadata，不逐筆 `get`；metadata 沒有原網址的紀錄只顯示提示。原網址來自使用者輸入，輸出到 HTML 前一律跳脫。回應帶 `Cache-Control: no-store`。`list` 失敗回 503，不影響其他路由。 + +`ctx.access.aud` 不比對：帳號裡任何涵蓋這個 Worker 的 Access 應用程式都是部署的人自己授權的。以本地時區分日也不在範圍內。 + +違反時不會被察覺：紀錄含有所有使用這個部署的人的連結，頁面若在沒登入時也能讀，外人打開 `/log` 不會留下任何讓部署的人注意到的痕跡；網址沒跳脫時，惡意的轉換請求會把腳本存進紀錄，等部署的人登入後打開頁面才執行。 + +由 `test/shared-link-log-page-behind-access.test.ts` 檢查（即 `verify` 指令）：沒有 `ctx.access` 時回 403 且不呼叫 `list`；沒有綁定時 `/log` 回 404；`list` 的 `prefix` 等於指定或今天的 UTC 日期，且只呼叫一次；新的紀錄排在前面；網址經過 HTML 跳脫；回應帶 `no-store`；`list` 失敗時回 503；不合法的 `day` 回 400 且不呼叫 `list`。 diff --git a/docs/spec/shared-links-logged-to-kv-at-conversion.md b/docs/spec/shared-links-logged-to-kv-at-conversion.md index 86ed68a..619a694 100644 --- a/docs/spec/shared-links-logged-to-kv-at-conversion.md +++ b/docs/spec/shared-links-logged-to-kv-at-conversion.md @@ -6,7 +6,7 @@ scope: - "src/index.ts" - "src/shared-link-log.ts" verify: check:bun test ./test/shared-links-logged-to-kv-at-conversion.test.ts -related: [free-plan-bindings-only, fail-open-to-original, tracking-params-cleaned-at-conversion] +related: [free-plan-bindings-only, fail-open-to-original, tracking-params-cleaned-at-conversion, shared-link-log-page-behind-access] source: null adr: null --- diff --git a/src/index.ts b/src/index.ts index d66c080..f80dfc0 100644 --- a/src/index.ts +++ b/src/index.ts @@ -6,6 +6,7 @@ import { fixServiceUrl } from "./fix-services"; import { instagramPost, isShortcode, shortcodeOfMediaId, type Post, type PostCache } from "./instagram"; import { renderOgPage } from "./og"; import { logSharedLink } from "./shared-link-log"; +import { hasSharedLinkLog, sharedLinkLogPage } from "./shared-link-log-page"; const CONVERT_HINT = "pass a percent-encoded http(s) URL as ?url="; const POST_PATH = /^\/(p|reel)\/([^/]+)(?:\/([1-9]\d?))?\/?$/; @@ -206,6 +207,7 @@ export function createWorker(deps: { cache: () => PostCache }): { const url = new URL(request.url); if (url.pathname === "/") return convert(url, env, ctx); if (url.pathname.startsWith("/media/")) return mediaRedirect(url, deps); + if (url.pathname === "/log" && hasSharedLinkLog(env)) return sharedLinkLogPage(url, env, ctx); return shareRedirect(url, request.headers.get("User-Agent"), deps); }, }; diff --git a/src/shared-link-log-page.ts b/src/shared-link-log-page.ts new file mode 100644 index 0000000..f884ee2 --- /dev/null +++ b/src/shared-link-log-page.ts @@ -0,0 +1,108 @@ +const DAY = /^\d{4}-\d{2}-\d{2}$/; + +type LoggedKey = { name: string; metadata?: unknown }; + +type SharedLinkList = { + list(options: { prefix: string }): Promise<{ keys: LoggedKey[]; list_complete: boolean }>; +}; + +function escapeHtml(value: string): string { + return value + .replaceAll("&", "&") + .replaceAll("<", "<") + .replaceAll(">", ">") + .replaceAll('"', """) + .replaceAll("'", "'"); +} + +function text(status: number, body: string): Response { + return new Response(body, { + status, + headers: { "Content-Type": "text/plain; charset=utf-8", "Cache-Control": "no-store" }, + }); +} + +function parseDay(value: string | null): Date | null { + if (value === null) return new Date(`${new Date().toISOString().slice(0, 10)}T00:00:00.000Z`); + if (!DAY.test(value)) return null; + const day = new Date(`${value}T00:00:00.000Z`); + return Number.isNaN(day.getTime()) || day.toISOString().slice(0, 10) !== value ? null : day; +} + +function shiftDay(day: Date, days: number): string { + return new Date(day.getTime() + days * 86_400_000).toISOString().slice(0, 10); +} + +function row(key: LoggedKey, origin: string): string { + const metadata = (key.metadata ?? {}) as { url?: unknown; raw?: unknown }; + const time = escapeHtml(key.name.slice(11, 19)); + if (typeof metadata.url !== "string") { + return `${time}(URL exceeded the metadata limit)`; + } + const url = escapeHtml(metadata.url); + const share = escapeHtml(metadata.raw === true ? metadata.url : `${origin}/${metadata.url.replace(/^https:\/\//, "")}`); + const badge = metadata.raw === true ? " raw" : ""; + return `${time}${url}${share}${badge}`; +} + +function render(day: Date, keys: LoggedKey[], complete: boolean, origin: string): string { + const label = day.toISOString().slice(0, 10); + const rows = keys.map((key) => row(key, origin)).reverse().join("\n"); + const truncated = complete ? "" : "

Only the first 1,000 records of this day are shown.

"; + return ` + + + + + +Shared links + + + + +

${keys.length} records. Records expire after 90 days.

+${truncated} + + + +${rows} + +
Time (UTC)Original URLShare link
+ + +`; +} + +export function hasSharedLinkLog(env: Env): boolean { + return (env as { SHARED_LINKS?: SharedLinkList }).SHARED_LINKS !== undefined; +} + +export async function sharedLinkLogPage(requestUrl: URL, env: Env, ctx: ExecutionContext): Promise { + if (ctx.access === undefined) return text(403, "forbidden"); + const day = parseDay(requestUrl.searchParams.get("day")); + if (day === null) return text(400, "pass ?day=YYYY-MM-DD"); + const store = (env as { SHARED_LINKS?: SharedLinkList }).SHARED_LINKS; + if (store === undefined) return text(404, "not found"); + let listed: { keys: LoggedKey[]; list_complete: boolean }; + try { + listed = await store.list({ prefix: day.toISOString().slice(0, 10) }); + } catch { + return text(503, "shared link log unavailable"); + } + return new Response(render(day, listed.keys, listed.list_complete, requestUrl.origin), { + headers: { + "Content-Type": "text/html; charset=utf-8", + "Cache-Control": "no-store", + "X-Robots-Tag": "noindex", + "Referrer-Policy": "no-referrer", + }, + }); +} diff --git a/test/shared-link-log-page-behind-access.test.ts b/test/shared-link-log-page-behind-access.test.ts new file mode 100644 index 0000000..cd18113 --- /dev/null +++ b/test/shared-link-log-page-behind-access.test.ts @@ -0,0 +1,102 @@ +import { afterEach, expect, setSystemTime, test } from "bun:test"; +import { createWorker } from "../src/index"; +import { fakeCache } from "./support/fake-cache"; +import { type KvKey, type KvMode, fakeKv } from "./support/fake-kv"; + +const ORIGIN = "https://link.example"; + +afterEach(() => setSystemTime()); + +function ctxWith(access: boolean): ExecutionContext { + const ctx = { + waitUntil() {}, + passThroughOnException() {}, + ...(access ? { access: { aud: "aud-tag", getIdentity: () => Promise.resolve(undefined) } } : {}), + }; + return ctx as unknown as ExecutionContext; +} + +function key(at: string, metadata: Record): KvKey { + return { name: `${at}-00000000-0000-4000-8000-000000000000`, metadata: { at, ...metadata } }; +} + +async function page( + path: string, + options: { access?: boolean; binding?: boolean; mode?: KvMode; keys?: KvKey[] } = {}, +): Promise<{ response: Response; body: string; lists: { prefix: string }[] }> { + const kv = fakeKv(options.mode ?? "ok", options.keys ?? []); + const env = (options.binding ?? true) ? { SHARED_LINKS: kv.binding } : {}; + const worker = createWorker({ cache: () => fakeCache().cache }); + const response = await worker.fetch(new Request(`${ORIGIN}${path}`), env as unknown as Env, ctxWith(options.access ?? true)); + return { response, body: await response.text(), lists: kv.lists }; +} + +test("without Access the page is forbidden and the store is not listed", async () => { + const { response, lists } = await page("/log", { access: false }); + expect(response.status).toBe(403); + expect(lists).toHaveLength(0); +}); + +test("without the binding /log is routed like any other unknown path", async () => { + const { response, lists } = await page("/log", { binding: false }); + expect(response.status).toBe(404); + expect(lists).toHaveLength(0); +}); + +test("the default day is today in UTC and the store is listed once", async () => { + setSystemTime(new Date("2026-10-16T23:59:59.000Z")); + const { response, lists } = await page("/log"); + expect(response.status).toBe(200); + expect(lists).toEqual([{ prefix: "2026-10-16" }]); +}); + +test("?day= picks the listed day", async () => { + const { lists } = await page("/log?day=2026-01-02"); + expect(lists).toEqual([{ prefix: "2026-01-02" }]); +}); + +test("an invalid day is rejected before listing", async () => { + for (const day of ["2026-1-2", "2026-13-01", "2026-02-30", "x"]) { + const { response, lists } = await page(`/log?day=${day}`); + expect(response.status).toBe(400); + expect(lists).toHaveLength(0); + } +}); + +test("records are shown newest first with their share links", async () => { + const keys = [ + key("2026-10-16T01:00:00.000Z", { url: "https://www.instagram.com/p/OLDER/" }), + key("2026-10-16T02:00:00.000Z", { url: "https://www.youtube.com/watch?v=newer", raw: true }), + ]; + const { body } = await page("/log?day=2026-10-16", { keys }); + expect(body.indexOf("youtube")).toBeLessThan(body.indexOf("OLDER")); + expect(body).toContain(`${ORIGIN}/www.instagram.com/p/OLDER/`); + expect(body).not.toContain(`${ORIGIN}/www.youtube.com`); + expect(body).toContain("02:00:00"); +}); + +test("a record without a url in its metadata still renders", async () => { + const { response, body } = await page("/log?day=2026-10-16", { keys: [key("2026-10-16T01:00:00.000Z", {})] }); + expect(response.status).toBe(200); + expect(body).toContain("01:00:00"); +}); + +test("urls are HTML-escaped", async () => { + const keys = [key("2026-10-16T01:00:00.000Z", { url: `https://www.youtube.com/watch?v=">` })]; + const { body } = await page("/log?day=2026-10-16", { keys }); + expect(body).not.toContain("