Why is the training probability scaled by 0.5 in GaussianEncoder.get_prob_train？

[Zyan20]

Hi, thanks for the great work and open-sourcing the code!

In src/models/common_model.py, GaussianEncoder.get_prob_train (around line 177) computes the bin probability and then halves it before converting to bits:

upper = _cdf2((0.5 - values) / scales)
lower = _cdf2((-0.5 - values) / scales)
prob = upper - lower
prob = torch.clamp_min(0.5 * prob, 1e-9)

Since bits = -log2(0.5 * prob) = -log2(prob) + 1, this charges each symbol roughly +1 bit relative to the true bin probability mass. Meanwhile, the quantization CDF table built in GaussianEncoder.update() uses the unscaled pmf, and the actual arithmetic coder encodes at that physical rate.

Observations from my experiments (image model, Kodak, cvpr2026_image.pth.tar, skip_thres both 0 and 0.15):

QP	forward BPP (estimated)	compress BPP (actual bitstream)
0	0.198	0.031
20	0.282	0.107
40	0.511	0.338

If I remove the 0.5 * factor and recompute bits_y, the estimated BPP becomes very close to the actual bitstream BPP (e.g. QP 0: 0.207 → 0.035 vs 0.031 actual). The gap is almost entirely in bits_y (z uses bit_estimator_z, which has no such scaling), and it is not caused by skip_thres.

My questions:

1. What is the intended role of this 0.5 scaling? Is it (a) a deliberately conservative entropy estimate during training to leave headroom for CDF discretization / the skip mechanism, (b) only about making the reported training bpp conservative, or (c) something that actively shapes the training objective?

2. Since a constant multiplicative factor does not change the gradient direction of most symbols, my understanding is that it does not change the optimum of the RD objective — is that correct, or does it interact with the clamp boundary / some other mechanism in a way that matters for training dynamics?

Any clarification would be very helpful. Thank you!

[aviadlazar]

I think this is meant to shape the training objective because it directly affects the rate portion of the loss.

The added bit only affects probabilities that are significantly higher than 1e-9, which should encourage the entropy model to learn a distribution that tends to produce a lower probability, and avoid the added cost. This would lead to a lower entropy in general, and better compression.

Hopefully we could get a confirmation from the authors, but you could run a test that checks how the entropy during training changes with and without scaling.