Skip to content

Feat/visual grounding delegation - #112

Open
jhshi1879 wants to merge 2 commits into
google:mainfrom
jhshi1879:feat/visual-grounding-delegation
Open

jhshi1879 wants to merge 2 commits into
google:mainfrom
jhshi1879:feat/visual-grounding-delegation

Conversation

@jhshi1879

@jhshi1879 jhshi1879 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixed an issue where the Operator agent failed to delegate Explorer to identify color codes by two lines of prompt change. Verified on the Android World benchmark BrowserDraw task with shell ADB access to task.html strictly blocked to prevent reward-hacking. Successfully validated use of Image Processor agent for pixel color sampling, achieved a 3/3 pass rate.

…ssor integration

- Support pixel sampling and spot color inspection in image_processor tool declarations
- Update ask_explorer description and dynamic prompt context for ultra tier
- Enable Operator to delegate subtle visual attributes and color matching to Explorer and Image Processor
- Add test coverage for Explorer tool descriptions across tiers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant