Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/features/media.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,19 @@ Folder routes (`/admin/api/cms/media/folders/...`) are matched **before** asset
| `server/repositories/mediaMigration.ts` | Migrating assets between storage adapters |
| `server/repositories/mediaStorageAdapters.ts` | Adapter registry persistence |

### What still uses a file

Permanently deleting an asset asks `POST /admin/api/cms/media/usage` first, so the confirmation can say which files in the selection something still depends on, and where. The answer merges two sources that are deliberately built differently:

| Source | How it is known | Why |
|---|---|---|
| **Settings** — a user's avatar | Recorded in `media_usage_refs` when the setting is written (`setMediaUsageRef` in `server/repositories/media.ts`) | One writer and an explicit set/unset, and nothing to walk. Setting a new value MOVES the reference instead of adding a second one. |
| **Page content** — image/media props, background images, Visual Component bodies, site-wide style backgrounds | Computed on request from each branch's draft site document (`collectContentUsageRefs` in `server/media/contentUsage.ts`) | Content is written continuously by the collab relay and removing an image emits no event, so a stored index would drift and the warning would start naming pages that are fine. |

The content walk reuses `collectPageMediaPaths` from the publisher's prefetch, so it recognises exactly the props the publisher resolves. It reads drafts, not published artefacts, and it reads every branch: media is shared across branches while pages are not, so a purge removes a file from all of them. A use on main is reported plainly; a branch is named only when the use exists on that branch and not on main.

The cost — O(branches × site) — lands only on a permanent delete, never on a page load or a trash. `buildUsageWarning` (`src/admin/pages/media/utils/usageWarning.ts`) turns the refs into the confirmation's copy: it counts per asset ("1 of 11 is still in use") but names per place, and it never blocks the delete.

### Upload pipeline

Uploads initiated outside the Media page use the same pipeline. In particular, the Agent Panel's explicit **Save to Media** image action resolves the private chat image, wraps it in a MIME-correct `File`, and calls `uploadCmsMediaAsset`; it does not create an AI-specific storage route. On success, `mediaAssetEvents.ts` upserts the new row into an already-mounted Site → Media explorer while the normal media cache is primed for canvas consumers.
Expand Down Expand Up @@ -364,6 +377,7 @@ See [docs/features/plugin-system.md](plugin-system.md). The plugin SDK's `api.cm
| Adding a docked panel to the Media page | Use a floating window — Media is canvas-style by design |
| Calling `api.cms.media.*` from a plugin without the matching media permission | Declare `media.import`, `media.storage.adapter`, `media.url.transform`, or `media.variant.delegate` as appropriate |
| Treating `deleted_at IS NOT NULL` rows as gone | They're in Trash; restore is supported until purge |
| Recording page-content usage in `media_usage_refs` | Compute it (`collectContentUsageRefs`); the table is for single-writer settings |
| Skipping `parent_id, slug` uniqueness when creating folders | The unique constraint enforces it — handle the error path |

---
Expand Down
29 changes: 29 additions & 0 deletions server/db/migrations-pg.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1359,4 +1359,33 @@ export const pgMigrations: Migration[] = [
id: '030_iso_timestamps',
sql: 'select 1',
},
{
// Existing avatars predate anything writing `media_usage_refs`, so the
// first build that warns before a delete would still have said nothing
// about the avatar already set — the one case the feature exists for.
//
// Idempotent by construction: `not exists` on the same key
// `setMediaUsageRef` writes, so re-running inserts nothing and the row a
// later avatar change moves is the row this created. Re-running is also
// what makes the id itself safe to change: an installation that recorded
// it under another number runs it again and inserts nothing.
//
// `032` sits between two ids other open PRs claim: `031` (#495) and `033`
// (#335). Ids are only ever sorted, so the gaps cost nothing, and
// whichever lands first, nobody has to renumber.
id: '032_backfill_avatar_usage_refs',
sql: `
insert into media_usage_refs (asset_id, ref_kind, ref_id, ref_path)
select u.avatar_media_id, 'user.avatar', u.id, ''
from users u
where u.avatar_media_id is not null
and not exists (
select 1 from media_usage_refs r
where r.asset_id = u.avatar_media_id
and r.ref_kind = 'user.avatar'
and r.ref_id = u.id
and r.ref_path = ''
);
`,
},
]
29 changes: 29 additions & 0 deletions server/db/migrations-sqlite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1515,4 +1515,33 @@ export const sqliteMigrations: Migration[] = [
id: '030_iso_timestamps',
sql: isoTimestampRewrite030(),
},
{
// Existing avatars predate anything writing `media_usage_refs`, so the
// first build that warns before a delete would still have said nothing
// about the avatar already set — the one case the feature exists for.
//
// Idempotent by construction: `not exists` on the same key
// `setMediaUsageRef` writes, so re-running inserts nothing and the row a
// later avatar change moves is the row this created. Re-running is also
// what makes the id itself safe to change: an installation that recorded
// it under another number runs it again and inserts nothing.
//
// `032` sits between two ids other open PRs claim: `031` (#495) and `033`
// (#335). Ids are only ever sorted, so the gaps cost nothing, and
// whichever lands first, nobody has to renumber.
id: '032_backfill_avatar_usage_refs',
sql: `
insert into media_usage_refs (asset_id, ref_kind, ref_id, ref_path)
select u.avatar_media_id, 'user.avatar', u.id, ''
from users u
where u.avatar_media_id is not null
and not exists (
select 1 from media_usage_refs r
where r.asset_id = u.avatar_media_id
and r.ref_kind = 'user.avatar'
and r.ref_id = u.id
and r.ref_path = ''
);
`,
},
]
14 changes: 14 additions & 0 deletions server/handlers/cms/me.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import {
acceptUploadedMedia,
readUploadForm,
} from './mediaUpload'
import { setMediaUsageRef } from '../../repositories/media'
import { Type } from '@core/utils/typeboxHelpers'
import { isValidEmail } from '@core/utils/email'
import { MIN_PASSWORD_LENGTH, PASSWORD_TOO_SHORT_MESSAGE } from '@core/utils/passwordPolicy'
Expand Down Expand Up @@ -328,6 +329,15 @@ export async function handleMeRoutes(
if (asset instanceof Response) return asset

const updated = await setUserAvatarMediaId(db, user.id, asset.id)
// Register the dependency so the media library stops treating a profile
// picture as an anonymous upload. Without it the asset is indistinguishable
// from a decorative one, and purging it nulls `avatar_media_id` through
// the column's `on delete set null` — silently, from the operator's side.
await setMediaUsageRef(db, {
assetId: asset.id,
refKind: 'user.avatar',
refId: user.id,
})
if (!updated) {
// The user row vanished between auth and the update (e.g. concurrent
// soft-delete). The uploaded asset stays in the media library — it's
Expand All @@ -351,6 +361,10 @@ export async function handleMeRoutes(
const updated = await setUserAvatarMediaId(db, user.id, null)
if (!updated) return jsonResponse({ error: 'User not found' }, { status: 404 })

// The asset stays in the library on purpose (see the file header), but it
// is no longer depended on — so the warning has to stop firing for it.
await setMediaUsageRef(db, { assetId: null, refKind: 'user.avatar', refId: user.id })

await createAuditEvent(db, {
actorUserId: user.id,
action: 'user.update',
Expand Down
40 changes: 40 additions & 0 deletions server/handlers/cms/media.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@
* permitted on already-trashed
* assets) and removes the file
* (`media.delete`)
* POST /admin/api/cms/media/usage — which of these assets are still
* depended on, and by what:
* recorded settings refs plus
* page content, computed live
* POST /admin/api/cms/media/:id/restore — restore a soft-deleted asset
* (`media.write`)
* POST /admin/api/cms/media/:id/replace — overwrite the bytes for an asset
Expand All @@ -38,11 +42,13 @@
*/
import type { DbClient } from '../../db/client'
import { requireCapability } from '../../auth/authz'
import { collectContentUsageRefs } from '../../media/contentUsage'
import {
assignAssetToFolders,
deleteMediaAsset,
getMediaAsset,
listMediaAssets,
listMediaUsageRefs,
restoreMediaAsset,
softDeleteMediaAsset,
updateMediaAssetMetadata,
Expand Down Expand Up @@ -293,7 +299,41 @@ async function handleDeleteMedia(

const ID_PATTERN = '(?<id>[^/]+)'

/**
* Which of the given assets something still depends on.
*
* A POST because the id list is a selection and can be long — a query string
* of a hundred ids is the wrong shape for a read this cheap.
*
* The UI calls this before a destructive action so it can name what is about
* to break instead of warning in the abstract. Requires `media.read`: the
* response reveals nothing beyond what the library already lists.
*/
async function handleMediaUsage(req: Request, db: DbClient): Promise<Response> {
const user = await requireCapability(req, db, 'media.read')
if (user instanceof Response) return user

const body = await readValidatedBody(req, MediaUsageQuerySchema)
if (!body) return badRequest('Invalid request body')

// Two sources, one answer. Settings (an avatar, later a favicon) are
// recorded in `media_usage_refs` because they have one writer and an
// explicit set/unset. Page content is COMPUTED, because it has neither —
// see `server/media/contentUsage.ts` for why a table would go wrong there.
// Both run in parallel; the caller cannot tell which side a ref came from.
const [stored, content] = await Promise.all([
listMediaUsageRefs(db, body.assetIds),
collectContentUsageRefs(db, body.assetIds),
])
return jsonResponse({ usage: [...stored, ...content] })
}

const MediaUsageQuerySchema = Type.Object({
assetIds: Type.Array(Type.String(), { maxItems: 500 }),
}, { additionalProperties: false })

const MEDIA_ROUTES: readonly Route<[]>[] = [
{ method: 'POST', pattern: `${MEDIA_PREFIX}/usage`, handler: handleMediaUsage },
{ method: 'GET', pattern: MEDIA_PREFIX, handler: handleListMedia },
{ method: 'POST', pattern: MEDIA_PREFIX, handler: handleUploadMedia },
{
Expand Down
166 changes: 166 additions & 0 deletions server/media/contentUsage.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
/**
* Which pages use these files — worked out from the site itself, not from a
* stored index.
*
* The counterpart to `media_usage_refs`, and the split between them is the
* point: a stored reference suits a SETTING, which has one writer and an
* explicit set/unset — an avatar, a favicon, a logo. Page content has
* neither. It is written continuously by the collab relay, and removing an
* image produces no event at all, so a table would fill with references to
* nodes that no longer exist and the warning would start being wrong.
*
* A wrong warning is worse than none: an operator who is misled once stops
* reading it. So this computes the answer at the moment it is asked, from
* `getDraftSiteDocument` — which cannot drift, because there is nothing to
* keep in sync.
*
* Deliberately reads DRAFTS, not the published artefacts: an image placed on
* an unpublished page is still in use, and a warning that only knew about live
* pages would let a delete quietly break the next publish.
*
* And it reads EVERY branch. Media is shared across branches while pages are
* not, so purging a file removes it from all of them at once — an image that
* only a branch uses breaks that branch's preview now, and the live site the
* moment the branch merges. Main is reported plainly; a branch is named only
* where it adds something main does not already say, so a site with five
* branches does not list the same page five times.
*
* The cost lands where it belongs. The walk is O(branches × site), and it runs
* only when someone asks to permanently delete something — never on a page
* load, never on a trash. For the site sizes this product is built for, that
* is milliseconds on an action that is about to be irreversible. If a site ever
* grows past that, the fix is to cache this — with the walk still the source
* of truth, so the cache can be checked against it.
*/

// Registry population. The walk asks the registry which props are
// image/media-typed, so without the base modules registered it matches
// nothing and reports NO usage — a warning that is silently always empty,
// which is the one failure mode worse than not having it. Same import
// `pageDiff.ts` and the collab relay make, and for the same reason.
import '@modules/base'
import { MAIN_BRANCH_ID } from '@core/branches'
import { registry } from '@core/module-engine'
import type { SiteDocument } from '@core/page-tree'
import { collectSiteStyleBackgroundImagePaths } from '@core/publisher'
import { MAIN_SCOPE } from '../branches/scope'
import { placeholder, type DbClient } from '../db/client'
import { listBranches } from '../repositories/branches'
import type { MediaUsageRef } from '../repositories/media'
import { getDraftSiteDocument } from '../repositories/publish'
import { collectPageMediaPaths } from '../publish/mediaPrefetch'

/**
* `ref_kind` values this module produces. They share the namespace with the
* stored kinds (`user.avatar`), so a caller merges the two lists without
* caring which side each one came from.
*/
export const PAGE_CONTENT_REF_KIND = 'page.content'
export const SITE_STYLES_REF_KIND = 'site.styles'

/**
* Map the requested asset ids to the `public_path` each one is stored under.
*
* Content props hold the path, not the id — and `replaceMediaAssetBinary`
* keeps the path stable across a file swap precisely so page references
* survive it. The path is therefore the join key, and this is the one query
* that translates.
*/
async function pathsForAssetIds(
db: DbClient,
assetIds: string[],
): Promise<Map<string, string>> {
const placeholders = assetIds.map((_, i) => placeholder(db.dialect, i + 1)).join(', ')
const { rows } = await db.unsafe<{ id: string; public_path: string }>(
`select id, public_path from media_assets where id in (${placeholders})`,
assetIds,
)
const byPath = new Map<string, string>()
for (const row of rows) byPath.set(row.public_path, row.id)
return byPath
}

/** One place a file is used, before it is decided which branch to name. */
interface Sighting {
assetId: string
refKind: string
refId: string
label: string
}

/**
* Identity of a sighting, independent of branch. Page ids are LOGICAL on every
* branch, so the same page on main and on a fork produces the same key.
*/
function sightingKey(sighting: Sighting): string {
return JSON.stringify([sighting.assetId, sighting.refKind, sighting.refId])
}

/**
* Everywhere one branch's draft uses one of the requested files.
*
* One sighting per (asset, page) — a file used by four nodes on one page is
* one page to fix, and repeating its title four times would turn the warning
* into the wall of text it exists to avoid.
*/
function sightingsInSite(
site: SiteDocument,
assetIdByPath: ReadonlyMap<string, string>,
): Sighting[] {
const found = new Map<string, Sighting>()
const add = (sighting: Sighting) => found.set(sightingKey(sighting), sighting)

for (const page of site.pages) {
// `collectPageMediaPaths` descends into the definition tree of every
// Visual Component the page references, so an image inside a VC body is
// attributed to the page that renders it — which is the page that would
// break, and so the one worth naming.
for (const path of collectPageMediaPaths(page, site, registry)) {
const assetId = assetIdByPath.get(path)
if (!assetId) continue
add({ assetId, refKind: PAGE_CONTENT_REF_KIND, refId: page.id, label: page.title || page.slug })
}
}

// Site-level style backgrounds belong to no single page — every page that
// matches the rule renders them, so naming one page would be misleading.
for (const path of collectSiteStyleBackgroundImagePaths(site)) {
const assetId = assetIdByPath.get(path)
if (!assetId) continue
add({ assetId, refKind: SITE_STYLES_REF_KIND, refId: 'site', label: 'site styles' })
}

return [...found.values()]
}

/**
* Which of `assetIds` the site's own content references, and where — across
* every branch.
*/
export async function collectContentUsageRefs(
db: DbClient,
assetIds: string[],
): Promise<MediaUsageRef[]> {
if (assetIds.length === 0) return []

const assetIdByPath = await pathsForAssetIds(db, assetIds)
if (assetIdByPath.size === 0) return []

// Main first, read explicitly: everything a branch reports is measured
// against it, so a branch that shares a use with main adds nothing.
const mainSite = await getDraftSiteDocument(db, MAIN_SCOPE)
const refs: MediaUsageRef[] = mainSite ? sightingsInSite(mainSite, assetIdByPath) : []
const onMain = new Set(refs.map(sightingKey))

for (const branch of await listBranches(db)) {
if (branch.id === MAIN_BRANCH_ID) continue
const site = await getDraftSiteDocument(db, { branchId: branch.id })
if (!site) continue
for (const sighting of sightingsInSite(site, assetIdByPath)) {
if (onMain.has(sightingKey(sighting))) continue
refs.push({ ...sighting, branchName: branch.name })
}
}

return refs
}
Loading
Loading