Synchronize tokenizer and multimodal preprocessing across shared sessions

[Baiju Meswani (baijumeswani)]

Summary

Fixes an intermittent Windows invalid string_view position failure when concurrent vision sessions share the same loaded model.

Root cause

ORT Extensions tokenizer encoding is not reentrant. Foundry Local synchronized its model tokenizer previously, but ORT GenAI's shared multimodal processor owns a separate tokenizer that remained unguarded. Concurrent preprocessing could therefore corrupt mutable tokenizer state.

See discussion #1008 (comment) for details.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview	Aug 18, 2026 6:07am

[Copilot]

Pull request overview

Centralizes shared tokenizer and multimodal preprocessing synchronization to prevent concurrent tokenizer-state corruption.

Changes:

• Replaces Tokenizer with a synchronized Preprocessor abstraction.
• Routes chat, audio, and embeddings preprocessing through shared synchronization.
• Adds a concurrent vision-session regression test.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
sdk_v2/cpp/CMakeLists.txt	Registers the new preprocessor source.
sdk_v2/cpp/src/inferencing/generative/tokenizer.h	Removes the superseded tokenizer wrapper.
sdk_v2/cpp/src/inferencing/generative/tokenizer.cc	Removes the old tokenizer implementation.
sdk_v2/cpp/src/inferencing/generative/preprocessor.h	Defines synchronized preprocessing ownership and APIs.
sdk_v2/cpp/src/inferencing/generative/preprocessor.cc	Implements synchronized tokenizer and multimodal operations.
sdk_v2/cpp/src/inferencing/generative/genai_model_instance.h	Replaces separate preprocessing resources with Preprocessor.
sdk_v2/cpp/src/inferencing/generative/genai_model_instance.cc	Creates and exposes the consolidated preprocessor.
sdk_v2/cpp/src/inferencing/generative/embeddings/embeddings_session.cc	Routes embedding tokenization through the preprocessor.
sdk_v2/cpp/src/inferencing/generative/chat/onnx_chat_generator.cc	Synchronizes chat and media preprocessing.
sdk_v2/cpp/src/inferencing/generative/chat/chat_template.cc	Routes template application and encoding through the preprocessor.
sdk_v2/cpp/src/inferencing/generative/audio/onnx_audio_generator.cc	Uses synchronized audio preprocessing and stream creation.
sdk_v2/cpp/src/inferencing/generative/audio/audio_session.cc	Uses the shared preprocessor for streaming decode setup.
sdk_v2/cpp/test/sdk_api/vision_session_test.cc	Tests concurrent sessions sharing one vision model.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.