Skip to content

feat(swift)!: read pasted and dropped files lazily, off the main thread - #138

Merged
tmm merged 1 commit into
mainfrom
swift-lazy-data-transfer-files
Nov 14, 2026
Merged

tmm merged 1 commit into
mainfrom
swift-lazy-data-transfer-files

Conversation

@tmm

@tmm tmm commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Closes #112. Rebased onto main after #137 (macOS refused files dropped from Finder), which ships in 0.1.10. Merge only after Swift 0.1.10 is released: this is a minor changeset, and merging it earlier would turn the open version PR into 0.2.0.

What changes

Pasted and dropped files are no longer read into memory on the main thread. As a web File is a lazy Blob, a DataTransferFile now has a size and reads its contents only when a handler awaits bytes(). This is the approved proposal: a clean break, not an API next to data, since the cost was in building the DataTransfer itself.

public struct DataTransferFile: Sendable {
  public var name: String
  public var type: String
  public var size: Int?                                       // File.size, nil when unknown
  public init(name: String, type: String, data: [UInt8] = [])  // in memory, as before
  public init(name: String, type: String, size: Int?,
              load: @escaping @Sendable () async throws -> [UInt8])
  public func bytes() async throws -> [UInt8]                 // File.bytes()
}
  • macOS: a file URL is stat'ed for its size and type, and read with a @concurrent read when bytes() is called. Folders are skipped, as before. An image on the pasteboard with no file behind it (copied from an app) is read from the pasteboard on the main actor when asked. It throws once the pasteboard has changed, as a web File throws NotReadableError for a changed file.
  • iOS: paste reads only the text types it needs. The old code read UIPasteboard.items, which loads every representation of every item on the main thread. Files come from itemProviders. iOS drops and pastes now carry files of any type, not only images (a PDF from Files was lost before), and canHandle accepts public.data.
  • Playground: DragDropPaste awaits bytes() in order, as mediaFileReader does, and builds the data URLs off the main thread.
  • DataTransferFile.data and its Equatable conformance are removed. Docs: DESIGN.md, JSParity.md, the DRAG_DROP_PASTE doc comment, and a platform-clipboard deviation, native/pasteboard-files.

Where the proposal was wrong

  • iOS can't be fully lazy. The proposal said the drop loaders would "capture the URL or item provider and read only when asked". UIKit doesn't allow that: a drop's item providers can be read "within the scope of [performDrop] only and not at any other time" (Apple's docs), and a later read fails silently. A paste read outside the paste would also bring up the paste permission prompt. So on iOS each file's loadFileRepresentation starts during the paste or drop, the file is moved into a temporary directory off the main thread, and bytes() waits for it. The temporary file is removed once no DataTransferFile holds it (checked: 1 copy while held, 0 after release). Files load from disk rather than into memory, so a dropped 2 GB video won't get the app killed. size is nil on iOS, so size: Int? is needed, as the proposal had it.
  • iOS drop main-thread time doesn't go down. UIKit's loadFileRepresentation costs about 2.5 ms per file on the calling thread, against 0.05 ms for loadDataRepresentation. The data version would hold every dropped file in memory, which is what this change removes. The handler latency and memory gains are below.
  • Before fix(swift): take files dropped from Finder on macOS #137, files dropped from Finder never reached the editor on macOS, so on main the main-thread read happened for paste (⌘V of a file copied in Finder), not for real drops.

Performance

Measured on a Mac satellite (12-core, macOS 27, Xcode 27.3) and the iOS 27.2 Simulator, against main.

LexicalBenchmarks (release, main and branch run alternately, 3 runs each): the same within noise; none of these paths changed. For example, insert a character p50 0.544–0.546 ms on main and 0.551–0.555 ms on the branch; load to rendered storage p50 103–111 ms and 110–111 ms; Enter p50 0.88–0.90 ms and 0.90–0.95 ms. Every target was met.

Paste and drop of 4 × 256 MiB files (the view's performDragOperation / paste(_:) with a handler that takes the files; scratch harness, not committed):

main: main-thread time branch: main-thread time branch: reading all bytes (off the main thread)
macOS drop, warm cache 85–87 ms 0.35–0.42 ms 84–87 ms, longest main-thread stall 2.3 ms
macOS drop, cold cache (purge) 238–246 ms 8.4–8.9 ms (first run, one-time setup) 196–218 ms, stall 2.3 ms
macOS paste, warm 86–89 ms 0.33–0.37 ms 84–86 ms, stall 2.2 ms
iOS paste, 4 × 64 MiB images 44–54 ms (and only 1 file arrives) 3.3–4.3 ms (all 4 arrive) 75–78 ms, stall 2.3 ms
iOS drop, file-backed providers 11–18 ms 12–14 ms 413–459 ms (includes the provider's copy)

iOS drop: DROP_COMMAND now runs after 12–14 ms instead of 159–204 ms, because it no longer waits for 1 GiB to load into memory, and nothing is held in memory until a handler reads.

Before and after (macOS playground: 8 × 1 GiB clips and a PNG dragged from Finder, cold cache, then a click and typing)

Top: before (main plus #137); the app freezes for about 4 s after the drop. Bottom: after; the image and the typed text appear within a second.

Video. "Before" is main plus #137, because main refuses the drop altogether. In the before take, the drag cursor stays stuck over the editor for about 3.8 s, then the image and the typed text appear together. In the after take they appear within a second of the drop. iOS has no visible change in the playground: drops already loaded off the main thread, and paste time went from about 50 ms to 4 ms. The table above has the iOS numbers.

Tests

  • AppKit readsADroppedFileOnlyWhenItsBytesAreRead: the drop's file has its name, type and size. The file is changed after the drop and bytes() returns the new contents, then it's deleted and bytes() throws. Dragover still has no files.
  • AppKit readsAPastedImageOnlyWhileThePasteboardHoldsIt: a pasteboard PNG reads when asked, and throws once the pasteboard has changed.
  • UIKit pastesAnImageAsAFile and dropsAFileThatIsNotAnImage: a PDF provider becomes Report.pdf (application/pdf) with its bytes, and a text provider adds no file. Neither passes on main.
  • Scratch check: an image put on the pasteboard as a UIImage still arrives as image.png and decodes.
  • swift format lint --strict, swift build --build-tests, swift test on macOS (all 28 test runs pass); LexicalUIKitTests on the iOS 27.2 Simulator; pnpm --filter lexical-conformance run test and check-ledger; the Linux build and the rich-text and utils file tests; scripts/api-breaks.sh, which reports DataTransferFile.data and != as no longer public and allows them under the pending minor changeset.

Migration (0.2.0 release notes)

Read a file's contents with try await file.bytes() where you read file.data, as upstream's mediaFileReader awaits each File. A DRAG_DROP_PASTE handler that needs the bytes returns true and reads them in a Task. DataTransferFile(name:type:data:) still makes an in-memory file, and DataTransferFile(name:type:size:load:) makes a lazy one. DataTransferFile is no longer Equatable, so compare files by name and type.

Base automatically changed from swift-accept-file-drops to main November 14, 2026 11:51
@tmm
tmm force-pushed the swift-lazy-data-transfer-files branch 2 times, most recently from cde8e97 to 3594ac1 Compare November 14, 2026 16:01
A DataTransferFile held its bytes in a public data: [UInt8], so building
a paste's or a drop's DataTransfer read every file into memory on the main
thread before PASTE_COMMAND or DROP_COMMAND ran, even when no handler
wanted files (#112). A web File is a lazy Blob read through promises
(arrayBuffer(), bytes()), and the playground's mediaFileReader awaits them.

DataTransferFile now has a size and an async bytes(), and the loaders
capture where each file lives instead of its contents:

- macOS: a file URL is stat'ed for its size and read off the main thread
  when bytes() is called. An image on the pasteboard with no file behind
  it is read from the pasteboard on the main actor when asked, and throws
  once the pasteboard has changed.
- iOS: UIKit lets an item provider load only during the paste or drop
  (performDrop is "the only opportunity"), so each file's
  loadFileRepresentation starts then, the file is moved into a temporary
  directory off the main thread, and bytes() waits for it; size is nil.
  The temporary file is removed once no DataTransferFile holds it. Paste
  reads only the text types it needs instead of UIPasteboard.items, which
  loaded every representation of every item on the main thread.
- iOS drops and pastes now carry files of any type, not only images (a
  PDF from Files was lost), and the drop interaction accepts them.

This is a clean break, not an API next to data: the cost was in building
the DataTransfer, so only a lazy file removes it. DataTransferFile.data and
its Equatable conformance are gone; DataTransferFile(name:type:data:)
still makes an in-memory file, and init(name:type:size:load:) a lazy one.
The playground's DragDropPaste reads with await file.bytes() and builds
the data URLs off the main thread. Minor changeset, DESIGN.md, JSParity
and a platform-clipboard deviation (native/pasteboard-files) updated.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d884-2859-717d-8ddd-80ee5e237c13
@tmm
tmm force-pushed the swift-lazy-data-transfer-files branch from 3594ac1 to 1f3ad98 Compare November 14, 2026 16:31
@tmm
tmm merged commit 59c4205 into main Nov 14, 2026
19 checks passed
@tmm
tmm deleted the swift-lazy-data-transfer-files branch November 14, 2026 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

swift: pasted and dropped files lose their URL and are read into memory on the main thread

2 participants