Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,14 @@ object IcebergCatalogInstance {
*/
def getInstance(warehouse: Option[String] = None): Catalog = {
val name = warehouse.getOrElse(defaultWarehouse)
synchronized {
catalogs.getOrElseUpdate(cacheKey(name), createCatalog(name))
val key = cacheKey(name)
// Read the cache outside the lock: building a catalog can block on the
// REST endpoint, and holding the monitor for that long stalls every other
// warehouse's first access.
if (!catalogs.contains(key)) {
catalogs.put(key, createCatalog(name))
}
catalogs(key)
Comment on lines +77 to +80

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🔴 Critical | 🏗️ Heavy lift

Restore atomic catalog initialization.

contains, createCatalog, put, and get are not one atomic operation. Two concurrent IcebergDocument instances can miss the same key, create separate catalogs, and overwrite each other. Callers can receive different instances, and a discarded PostgreSQL catalog can retain its connection pool.

Use a thread-safe cache with atomic per-key initialization, or protect the complete lookup/create/insert sequence with the same lock used by replaceInstance. Add a concurrency test that proves one catalog is created for concurrent misses.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@common/workflow-core/src/main/scala/org/apache/texera/amber/core/storage/IcebergCatalogInstance.scala`
around lines 77 - 80, Make catalog initialization in IcebergCatalogInstance’s
lookup path atomic per key: synchronize the complete contains/create/put/get
sequence using the same lock as replaceInstance, or replace the cache with a
thread-safe compute-if-absent mechanism. Ensure concurrent misses return the
same catalog and create only one instance, and add a concurrency test covering
simultaneous initialization.

}

private def createCatalog(warehouse: String): Catalog =
Expand Down