test(vision): activate Ollama vision e2e once granite-vision-4.1 lands on Ollama

[planetf1]

Context

PR #1185 fixes the failing Ollama vision tests by splitting them into two tiers:

1. Structural payload tests — mocked unit tests that verify mellea correctly embeds ImageBlock instances into the Ollama conversation payload. These run in CI and locally with no vision model required.
2. Dormant live e2e tests — a thin round-trip smoke test that is currently skipped because no current-generation Granite vision model runs on Ollama yet.

What needs to happen

granite-vision-4.1 (ibm-granite/granite-vision-4.1-4b) is the current Granite vision model. It is not yet in the Ollama library (ollama.com/library/granite-vision-4.1 → 404). When it lands, the dormant tests in test/backends/test_vision_ollama.py should be activated.

Activation steps

1. Verify ollama pull granite-vision-4.1 works.
2. In test/backends/test_vision_ollama.py:
   • Remove the pytest.skip(...) call from the vision_session fixture.
   • Remove this tracking comment.
3. Add granite-vision-4.1 to the CI model-pull step in .github/workflows/quality.yml so the e2e tests run in CI.
4. Optionally add IBM_GRANITE_VISION_4_1_4B to mellea/backends/model_ids.py and update the _VISION_MODEL constant to use it.

Related

• Fixes: test(vision): test_vision_ollama CI path sends images to non-vision model, causing 400 Bad Request #1185
• Blog post tracking Ollama availability: mellea-website branch blog/granite-vision-structured-extraction (see the "Model availability" note in that draft)

[AngeloDanducci]

I had a PR to open remove the pinning #1392 but after further investigation this was not the cause of the memory usage being too high that we had seen.

[psschwei]

Since the vision model is on HF, this may be doable now
https://huggingface.co/ibm-granite/granite-vision-4.1-4b

[planetf1]

Updated to use the vision model from Hugging Face.

granite-vision-4.1 still isn't in the Ollama library, but IBM publishes an official GGUF build, so Ollama can pull it straight from HF:

ollama pull hf.co/ibm-granite/granite-vision-4.1-4b-GGUF:Q4_K_M

The dormant e2e tests are now active and passing in CI on 3.11/3.12/3.13, with no meaningful change to job runtime. Docs updated too. PR #1527.