feat: add themed image responses, browser caching, and shared upload handling

docs/catalog-image-uploads.md

@@ -82,8 +82,15 @@ insight upload routes retain their existing request contracts.
 
 ## Shared stock images
 
-When a project or engine has no saved image, the download route serves a shared
-stock image directly from the configured stock collection. It does not copy that
+The client uses the instance-managed badge for registered system projects and
+uploaded images or themed stock artwork for other project catalogs (including
+agents, skills, apps, notebooks, and automations). Engine catalog
+cards and detail headers use their database-type or model-provider icons from
+the existing frontend asset library. Engine image endpoints remain available
+to other consumers.
+
+When an ordinary project or engine has no saved image, the download route serves
+a shared stock image directly from the requested stock collection. It does not copy that
 fallback into the resource's version folder or upload it to cluster image storage.
 The CouchDB project/database fallback likewise returns stock bytes without saving
 a per-resource attachment. Uploaded images continue to take precedence.
@@ -95,13 +102,103 @@ up to 30 seconds to appear on a node that is currently using a stock fallback.
 
 Selection uses the existing deterministic key: the resource name for local
 version paths, the resource ID for cluster image paths, and the partition/name
-for CouchDB. The selection stays stable while that key, the configured theme,
-and the stock collection stay unchanged. No per-resource reference file or
-database record is required. `DEFAULT_IMAGE_THEME` selects the light or dark
-stock collection, with `images/stock-engines` as the legacy fallback directory.
-
-Existing images, including stock copies created by previous versions, are still
-served as saved images. This change does not delete or migrate them. Removing old
+for CouchDB. Paired light and dark collections use matching, sorted filenames,
+so changing the theme keeps the same artwork. No per-resource reference file or
+database record is required.
+
+Both download routes accept `?theme=light` or `?theme=dark`. The catalog UI sends
+its resolved theme, including the operating system preference when set to
+System, and updates the URL whenever that theme changes. Upload cache revisions
+remain in the URL as a separate `v` parameter. Ordinary projects' uploaded images
+are returned unchanged for either theme.
+
+Requests with no theme or an unsupported value use `DEFAULT_IMAGE_THEME`.
+If the requested collection is missing or empty, the server tries the configured
+theme and then the legacy `images/stock-engines` directory. Local and cluster
+response ETags include the stock file path, so light and dark variants have
+different cache validators; CouchDB ETags reflect the returned image bytes.
+
+For ordinary projects, existing images, including stock copies created by
+previous versions, are still served as saved images. This change does not delete
+or migrate them. Removing old
 copies requires a separate migration that identifies stock files without removing
-custom uploads. Build and deploy both SEMOSS core and Monolith for this behavior;
-the SDK and frontend download URLs are unchanged.
+custom uploads. Build and deploy SEMOSS core, Monolith, and the client together
+to enable theme switching. The download route paths and existing callers remain
+compatible; the theme query parameter is optional.
+
+### Instance-managed projects
+
+The project download endpoint identifies system projects using
+`SystemDefaultEngines.isSystemProject(projectId)`, which includes the registered
+platform apps, skills, MCPs, and agents. This uses exact IDs from the same lists
+that seed projects at startup. Editable `SYSTEM` tags, project names, `platform__`
+prefixes, missing creators, and global visibility do not classify user projects
+as instance-managed. Additional managed project types can join these registry
+lists without changing the image endpoint.
+
+After the normal session and project permission checks, registered projects
+receive `images/system-projects/instance-managed-light.svg` or
+`instance-managed-dark.svg` from the SEMOSS base directory. These badges take
+precedence over saved images for system projects only, consistently across local,
+cluster, and CouchDB deployments. No saved images are deleted or modified, and
+ordinary projects and engine provider/type icons keep their existing behavior.
+
+An omitted or unsupported theme uses `DEFAULT_IMAGE_THEME`. If the selected badge
+is missing, the resolver tries the configured theme and the other available badge.
+If neither badge exists, the endpoint continues through its normal image lookup.
+The badge files stay outside the random stock collections so they are never
+assigned to user-created projects. They use the same private browser cache policy
+and theme-specific ETags as other project images.
+
+Publish SEMOSS core and Monolith together with the two SVG assets to enable this
+selection. Existing catalog UI image requests need no frontend changes.
+
+Image responses set `Content-Type` from the returned file or bytes. PNG stock
+artwork is served as `image/png`; SVG uploads use `image/svg+xml`. Advertising
+SVG in `@Produces` alone is insufficient: a browser's `Accept` header can select
+SVG even when the response contains PNG bytes.
+Image ETags include a representation version so previously cached responses
+with incorrect content types are replaced when the browser revalidates.
+
+When publishing an incremental local build, include newly added classes as well
+as changed route classes. In particular, both routes require
+`WEB-INF/classes/prerna/web/services/util/CatalogImageResponse.class`. A missing
+helper causes `NoClassDefFoundError` and failed image requests, so the UI keeps
+its initials fallback. Publish the complete build and reload the application;
+clearing the browser cache cannot repair a missing server class.
+
+## Browser caching
+
+Successful engine and project image downloads use:
+
+```http
+Cache-Control: private, max-age=300, must-revalidate
+ETag: "<image-version>"
+Vary: Cookie, Authorization
+```
+
+The browser can reuse a fresh image for five minutes without contacting the
+server. Once stale, it sends `If-None-Match`; an unchanged image returns
+`304 Not Modified` with the same ETag, Cache-Control, and Vary headers and no
+image body. This applies to local files, cluster files, and CouchDB responses.
+Authentication and resource permissions are checked before any server response,
+including a 304. Private caching avoids shared proxy storage, and Vary keeps
+browser cache entries separate when session cookies or authorization change.
+
+The UI's `theme` query parameter separates light and dark entries. A successful
+upload changes `v`, bypassing the previous cached URL immediately in that browser
+session. Other viewers can retain the previous image for the remaining five-minute
+freshness window, plus any cluster synchronization delay. These URLs are mutable,
+so they are deliberately not marked `immutable` or cached for a year.
+
+To verify a deployed environment, open browser DevTools > Network and leave
+**Disable cache** unchecked. Load a catalog image once, then navigate away and
+back: the same URL should show a memory/disk cache hit. After five minutes, its
+conditional request should return 304 if the image has not changed. Changing
+theme should request the other URL once; switching back can reuse its cached
+variant. Uploading an image should request a new `v` URL. Reload and hard-refresh
+can force revalidation, so use normal navigation for the freshness check.
+
+Inspect the final response headers through your deployed proxy. A proxy that
+adds `no-store` or `no-cache`, drops the ETag, or changes query-string handling
+can override or defeat this behavior.

src/prerna/semoss/web/services/local/EngineRouteResource.java

@@ -62,7 +62,6 @@
 import jakarta.ws.rs.core.MediaType;
 import jakarta.ws.rs.core.Request;
 import jakarta.ws.rs.core.Response;
-import jakarta.ws.rs.core.Response.ResponseBuilder;
 import prerna.auth.User;
 import prerna.auth.utils.SecurityAdminUtils;
 import prerna.auth.utils.SecurityEngineUtils;
@@ -83,6 +82,7 @@
 import prerna.util.EngineUtility;
 import prerna.util.NotificationConstants;
 import prerna.util.Utility;
+import prerna.web.services.util.CatalogImageResponse;
 import prerna.web.services.util.WebUtility;
 
 @Path("/e-{engineId}")
@@ -372,10 +372,10 @@ public Response updateSmssFile(@Context HttpServletRequest request, @PathParam("
 		ClusterUtil.pushEngineSmss(engineId, engine.getCatalogType());
 		if (Utility.isNotificationDatabaseEnabled()) {
 			// Adding notification
-			String engineType = String.valueOf(SecurityEngineUtils.getEngineType(engineId)).toLowerCase();
-			NotificationDbUtils.createNotification(user, null, null, engineId, NotificationConstants.Type.SMSS_UPDATE,
-					engineType, NotificationConstants.Priority.MEDIUM, null, null,
-					NotificationConstants.DisplaySurface.BELL);
+			String engineType = String.valueOf(SecurityEngineUtils.getEngineType(engineId)).toLowerCase();
+			NotificationDbUtils.createNotification(user, null, null, engineId, NotificationConstants.Type.SMSS_UPDATE,
+					engineType, NotificationConstants.Priority.MEDIUM, null, null,
+					NotificationConstants.DisplaySurface.BELL);
 
 			// Adding email notification
 			EmailUtility.sendSmssUpdateEmailNotification(user, engineId, EmailUtility.RESOURCE_TYPE.ENGINE);
@@ -408,7 +408,9 @@ public Response uploadImage(@Context ServletContext context, @Context HttpServle
 	 * Download the image associated with this engine. The lookup falls through
 	 * three sources in order: CouchDB (if enabled), cloud storage (if running in
 	 * cluster mode), and finally the engine's local version folder. If no image
-	 * exists locally the shared stock image is served without copying it.
+	 * exists locally the shared stock image is served without copying it. The
+	 * optional {@code theme=light|dark} query parameter selects the stock artwork
+	 * theme.
 	 * <p>
 	 * Honors HTTP {@code If-None-Match} via an entity tag built from the file's
 	 * path, last-modified timestamp, and size, so an unchanged image returns 304.
@@ -422,7 +424,8 @@ public Response uploadImage(@Context ServletContext context, @Context HttpServle
 	 */
 	@GET
 	@Path("/image/download")
-	@Produces({ MediaType.APPLICATION_OCTET_STREAM, MediaType.APPLICATION_SVG_XML })
+	@Produces({ "image/png", "image/jpeg", "image/gif", "image/webp", "image/svg+xml",
+			MediaType.APPLICATION_OCTET_STREAM })
 	public Response imageDownload(@Context final Request coreRequest, @Context HttpServletRequest request,
 			@PathParam("engineId") String engineId) {
 		// not required for containment. getEngineTypeAndSubtype and
@@ -482,13 +485,15 @@ public Response imageDownload(@Context final Request coreRequest, @Context HttpS
 			return WebUtility.getResponse(returnMap, 400);
 		}
 
+		String imageTheme = request.getParameter("theme");
 		File exportFile = null;
 		// is the image in couch db
 		if (CouchUtil.COUCH_ENABLED) {
 			try {
 				Map<String, String> selectors = new HashMap<>();
 				selectors.put(couchSelector, engineId);
-				return CouchUtil.download(couchSelector, selectors);
+				return CatalogImageResponse.withBrowserCache(coreRequest,
+						CouchUtil.download(couchSelector, selectors, imageTheme));
 			} catch (CouchException e) {
 				classLogger.error(
 						"Failed to download engine image from CouchDB for engine '{}' using selector '{}'; falling through to other sources",
@@ -498,7 +503,7 @@ public Response imageDownload(@Context final Request coreRequest, @Context HttpS
 		// is the image in cloud storage
 		else if (ClusterUtil.IS_CLUSTER) {
 			try {
-				exportFile = ClusterUtil.getEngineAndProjectImage(engineId, engineType);
+				exportFile = ClusterUtil.getEngineAndProjectImage(engineId, engineType, imageTheme);
 			} catch (Exception e) {
 				classLogger.error("Failed to fetch engine image from cluster storage for engine '{}' (type {})",
 						engineId, engineType, e);
@@ -512,32 +517,16 @@ else if (ClusterUtil.IS_CLUSTER) {
 			if (exportFile == null) {
 				// Resolve the shared stock file without creating an engine asset.
 				String fileLocation = engineVersionPath + "/" + "image.png";
-				exportFile = DefaultImageGeneratorUtil.getStockImageForPath(fileLocation);
+				exportFile = DefaultImageGeneratorUtil.getStockImageForPath(fileLocation, imageTheme);
 			}
 		}
 
 		if (exportFile != null && exportFile.exists()) {
 			String exportName = engineId + "_Image." + FilenameUtils.getExtension(exportFile.getAbsolutePath());
-			// want to cache this on browser if user has access
-//			CacheControl cc = new CacheControl();
-//			cc.setMaxAge(86400);
-//			cc.setPrivate(true);
-//			cc.setMustRevalidate(true);
 			EntityTag etag = new EntityTag(Integer.toHexString(exportFile.getAbsolutePath().hashCode()) + "-"
 					+ exportFile.lastModified() + "-" + exportFile.length());
-			ResponseBuilder builder = coreRequest.evaluatePreconditions(etag);
-
-			// cached resource did not change
-			if (builder != null) {
-				return builder.build();
-			}
-
-			return Response.status(200).entity(exportFile)
-					.header("Content-Disposition", "attachment; filename=" + exportName)
-//					.cacheControl(cc)
-					.tag(etag)
-//					.lastModified(new Date(exportFile.lastModified()))
-					.build();
+			return CatalogImageResponse.withBrowserCache(coreRequest, Response.ok(exportFile)
+					.header("Content-Disposition", "attachment; filename=" + exportName).tag(etag).build());
 		} else {
 			Map<String, String> errorMap = new HashMap<>();
 			errorMap.put(Constants.ERROR_MESSAGE, "Error sending image file");