Conversation
Select an area, describe a change, and get the result as a new masked layer. Nothing outside the selection changes and the rest of the document keeps its resolution. Optional: the app ships without a key and sends nothing until the user presses Generate. The Gemini image models take no mask and repaint whatever they are sent, so the app does the masking. It crops the selection with its surroundings, in a ratio the model accepts, as the picture looks from the new layer's place in the stack (layers above are left out, so an adjustment layer is not applied twice). The answer is mapped back by fractions and kept only through a layer mask made from the selection, grown and feathered to hide the seam. - Generative Fill and Generative Remove in the Edit menu; Generative Expand from a crop frame dragged past the canvas, which enlarges the canvas and adds the layer as one undo step - Reference images, one to three results, a choice of model and size - The result previews on the canvas as the layer it will become; Keep is one undo step, and the document is held still while the panel is open - Settings (Cmd-,) for the key, kept in the keychain, with a free check that it can reach the image models - GenerativeImageProvider keeps the service behind one protocol, so a masked-fill service can be added beside Gemini - ImageImporter decodes image data as well as files, WebP included Tests use a stand-in provider and an in-memory key store; they never touch the network or the keychain. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Owner
|
Thanks, this is one of the more thoughtful PRs I've gotten. Leaving it open while I think about how I want AI in Compositor longer term - and what is the most useful workflow for users. |
Author
Sure. Honestly, I was just curious how complex it would be to actually implement it. Use it or close it as you wish ;) If you would like some changes there, let me know. This is more or less a POC; a lot of things can be improved for sure 😉 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds optional Generative Fill, Generative Remove and Generative Expand, using the user's own Google Gemini API key. The result is a normal layer with a layer mask; nothing outside the selection changes and the rest of the document keeps its resolution.
Heads-up: this is the first feature that sends user content off the Mac. It is off by default (no key ships), nothing is sent until the user presses Generate, the first Generate explains what is sent and that it is billed to their Google account, and the key lives in the keychain. No new entitlement. If you'd rather not have this in the app, that's a fair answer.
How it works. Gemini has no mask input, so the app does the masking: it sends the selection with some surrounding context, maps the answer back, and keeps only the selection through a feathered layer mask. The crop is rendered from the new layer's place in the stack, so adjustment layers above aren't applied twice. While the panel is open the document is locked, the way Levels does it. Details are in the README.
output_2x_final.mov
Worth a look in review
Settings…item on ⌘, (for the key)generativeEdit == niladded to theEditorSessiongates;canEditLayerssplit so Expand can start from the Crop frameEditorCanvasdraws a preview copy of the document, built by the same function Keep usesTesting (Xcode 27 / macOS 27)
main.Known limits: large areas come back below document resolution (models top out around 4K) and look softer; no seed, so every retry is a new charged request.