fixes flaky test in c# test suite

[Prathik Rao (prathikr)]

Vision_ImageBytesInput_ProducesResponse was hitting the following error intermittently...

FoundryLocalException: invalid string_view position 

Added a class-local SemaphoreSlim gate around vision requests to preserve parallelism while also allowing model sharing across tests

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview	Aug 18, 2026 12:18am

[Copilot]

Pull request overview

Serializes vision inference tests that share a model, preventing intermittent native request failures while preserving parallelism elsewhere.

Changes:

• Adds a class-level SemaphoreSlim.
• Gates both streaming and non-streaming vision requests with guaranteed release.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Scott McKay (skottmckay)]

Do we know why we hit the exception? I would have expected multiple tests using different sessions against the same model to work as the FL session has its own state, and ORT sessions are typically stateless. Is there something at the GenAI level that doesn't support this sort of usage?

[Baiju Meswani (baijumeswani)]

my guess without seeing the code is that the different sessions sharing a multi-modal pre-processor?

[Scott McKay (skottmckay)]

Is this the correct analysis about what's happening in GenAI?

---

Root cause identified: PR #1008 is exposing the same ORT Extensions tokenizer race previously addressed for text inference in foundry-local#934, but through a second, unprotected tokenizer owned by ORT GenAI’s multimodal processor.

The failing path is:

1.  VisionTests  shares one loaded  IModel  while TUnit runs both tests concurrently.
2. Each test creates a separate  ChatSession , but both sessions resolve to the same  GenAIModelInstance .
3.  GenAIModelInstance  owns one shared  OgaMultiModalProcessor  in  sdk_v2\cpp\src\inferencing\generative\genai_model_instance.h .
4. Both requests concurrently call that processor from  onnx_chat_generator.cc .
5. ORT GenAI’s  MultiModalProcessor  owns its own shared  Tokenizer  and calls  Tokenizer::Encode()  without synchronization:
   •  D:\src\github\ort.genai\src\models\model.cpp 
   • Present in v0.15.1, v0.15.2, and current  main .
6. ORT GenAI 0.15.1 embeds ORT Extensions commit  fe4e13f... , which contains onnxruntime-extensions#1068. That optimization caches one mutable  CachedSplitters::reg_splitter  per tokenizer. Every call performs:
   cached_splitters_->reg_splitter.Set(seg_id.first);
   // ...
   tok = cached_splitters_->reg_splitter.GetNextToken();
   ``` Set() changes the backing `std::u32string_view`. Concurrent calls overwrite that shared view, producing invalid offsets and MSVC’s `invalid string_view position` exception.

This matches the exact Windows-only exception and mechanism documented in #934. FL’s  fl::Tokenizer  mutex fixed the primary model tokenizer, but it cannot protect the separate tokenizer created internally by  OgaMultiModalProcessor .

Recommended fix: fix ORT GenAI, not the C# tests. Add synchronization inside  Generators::Tokenizer  around  Encode() —and other operations touching the same ORTX tokenizer state—so every consumer, including  MultiModalProcessor , is protected. Add a concurrent shared-tokenizer/multimodal-processor regression test. Longer-term, ORT Extensions could make cached splitter invocation state per-call or thread-local, but its current code explicitly assumes concurrent calls on one kernel instance do not occur.

[Baiju Meswani (baijumeswani)]

Yes, this is what I would guess the problem is. I fixed the tokenizer usage in FL to work around the not-thread sage tokenizer. But it looks like the processor also embeds the tokenizer which causes this problem.

Solution is to guard the tokenizer in genai. Or to make the ort-extensions tokenizer thread safe so individual .encode calls don't mutate the state in a way that it leads to a crash.

[Baiju Meswani (baijumeswani)]

for now, we could put workaround the problem by guarding the processor and the tokenizer with the same mutex so only one thread can access either at a given time (in foundry-local)