feat(vision): add auxiliary vision model for image description fallback#157
Open
Johell1NS wants to merge 1 commit into
Open
feat(vision): add auxiliary vision model for image description fallback#157Johell1NS wants to merge 1 commit into
Johell1NS wants to merge 1 commit into
Conversation
When the selected model lacks image input capability, the bot now automatically uses a user-configurable auxiliary vision model to describe images. The textual description is then fed to the main model, enabling non-vision models to process image-based queries. Changes: - Add /setvision command to select a vision-capable model via inline menu - Add vision-model-service with synchronous describeImage() using session.prompt() for temporary vision sessions - Modify photo, document, and media-group handlers to fall back to vision model when main model lacks image support - Vision sessions are created as children of the foreground session to suppress background session notifications - Add 11 new i18n keys across all 7 supported languages - Persist vision model selection in settings.json Closes grinev#151
Johell1NS
pushed a commit
to Johell1NS/opencode-telegram-bot
that referenced
this pull request
Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an auxiliary vision model fallback. When the user sends an image to a model that does not support image input, the bot:
/setvision)Everything is automatic — no user confirmation required. If no vision model is configured, the bot behaves exactly as before (error message + text-only fallback).
Closes #151
Commands
/setvision— shows a menu to select a vision model. If already set, shows the current model with "Change" and "Clear" optionssettings.jsonacross restartsTechnical changes
3 new files:
src/app/services/vision-model-service.ts— service:describeImage()creates a temporary session, callssession.prompt()synchronously with the vision model, extracts the text description, deletes the sessionsrc/bot/commands/setvision-command.ts—/setvisioncommand handlersrc/bot/callbacks/vision-model-callback-handler.ts— inline menu callback for vision model selection17 modified files:
src/app/types/settings.ts/src/app/stores/settings-store.ts— newcurrentVisionModelfieldsrc/bot/handlers/photo-handler.ts— vision fallback in!supportsImagebranchsrc/bot/handlers/document-handler.ts— fallback for image documentssrc/bot/handlers/media-group-handler.ts— fallback for media groups with imagessrc/bot/routers/command-router.ts/callback-router.ts— command and callback registrationsrc/bot/menus/inline-menu.ts— added"vision"menu kindsrc/bot/commands/definitions.ts— command list registrationsrc/i18n/*.ts— 11 new i18n keys across 7 languagesNotes
session.prompt()(synchronous) rather thanpromptAsync(), since the full response is needed to continue the flow