Clarify TopP cutoff semantics: threshold-crossing move excluded

[Dash1971]

Could you confirm whether Maia-3's TopP option is intended to follow the original nucleus sampling definition, or whether excluding the threshold-crossing move is a deliberate variation?

Documented terminology and original definition

The Maia-3 README describes TopP as:

nucleus sampling threshold. 1.0 disables top-p filtering.

In Holtzman et al., The Curious Case of Neural Text Degeneration, section 3.1, equation (2), the nucleus is defined as the smallest set whose combined probability is greater than or equal to p. The accompanying explanation specifies the highest-probability candidates, and equation (3) renormalizes the retained distribution.

Under that definition, the candidate that first brings the cumulative probability to or above the threshold is included. I have not found an explicit statement from the Maia team about this boundary behavior, so the interpretation above is based on the named algorithm rather than confirmed maintainer intent.

Observed behavior

At commit 1e13597c42d4858b7cfd7cfdae01e297263364b2, sample_from_logits uses:

cumulative = torch.cumsum(sorted_probs, dim=-1)
keep = cumulative <= top_p
keep[0] = True

For a legal-move distribution of [0.6, 0.3, 0.1], Temperature=1, and TopP=0.8:

Behavior	Retained moves	Combined probability before renormalization	Sampling probabilities
Current Maia-3 code	First move only	0.6	[1.0]
Original nucleus definition	First and second moves	0.9	[2/3, 1/3]

Here, 0.6 is below 0.8; adding 0.3 gives 0.9, which reaches/exceeds the threshold. The current mask excludes that second move because 0.9 exceeds 0.8.

For some settings with Temperature > 0 and TopP < 1, this retains fewer candidates and can make sampling deterministic. TopP=1 bypasses this filtering, and Temperature=0 already uses argmax.

Proposed change and reproduction

PR #13 proposes aligning the cutoff with the original definition. It includes a checkpoint-free reproduction, a small mask change, a proposed README clarification, and 11 deterministic regression tests using real PyTorch operations (only the final random draw is mocked). All 11 pass with the change; five test methods fail on the unchanged implementation.

Those tests establish the behavioral difference, not the team's intention. If the current cutoff is deliberate, documenting its difference from the original definition would help downstream integrations. Otherwise, PR #13 provides a tested correction for review.