Summary
_aggregate rejects Choice answers whose probabilities sum to 0.99, which the API routinely returns for Choices with many options. The whole system_one call then raises ValueError("Backend probabilities must sum to approximately 1").
Cause
The API reports each probability rounded to two decimals, so the rounding error of the sum grows with the number of options (up to 0.005 × K). The check in pijev/__init__.py uses a fixed tolerance:
if not isclose(mass, 1.0, rel_tol=0, abs_tol=1e-3):
raise ValueError("Backend probabilities must sum to approximately 1")Observed (jev-1.13.0, live)
Permuted copies of real Choices, one request each, repeated a few times:
| Options (K) |
Returned sum |
| 12 |
0.99 |
| 14 |
0.99 |
| 26 |
0.99 |
About half of our requests containing a 12–26 option Choice failed this way at the default budget of 8, since any one of the 8 copies can trip it. Small Choices (3–4 options) were never affected.
Suggested fix
Make the tolerance rounding-aware and keep renormalising (which _aggregate already does):
if mass <= 0 or not isclose(mass, 1.0, rel_tol=0, abs_tol=0.005 * len(labels) + 1e-9):
We applied the equivalent change in a Java port and have not seen the error since.
Summary
_aggregaterejects Choice answers whose probabilities sum to 0.99, which the API routinely returns for Choices with many options. The wholesystem_onecall then raisesValueError("Backend probabilities must sum to approximately 1").Cause
The API reports each probability rounded to two decimals, so the rounding error of the sum grows with the number of options (up to 0.005 × K). The check in
pijev/__init__.pyuses a fixed tolerance:Observed (jev-1.13.0, live)
Permuted copies of real Choices, one request each, repeated a few times:
About half of our requests containing a 12–26 option Choice failed this way at the default budget of 8, since any one of the 8 copies can trip it. Small Choices (3–4 options) were never affected.
Suggested fix
Make the tolerance rounding-aware and keep renormalising (which
_aggregatealready does):We applied the equivalent change in a Java port and have not seen the error since.