BUG: Use label integers instead of float to count pixels - #6759
Conversation
There was a problem hiding this comment.
Thank you for contributing a pull request! 🙏
Welcome to the ITK community! 🤗👋☀️
We are glad you are here and appreciate your contribution. Please keep in mind our community participation guidelines. 📜
More support and guidance on the contribution process can be found in our contributing guide. 📖
This is an automatic message. Allow for time for the ITK community to be able to read the pull request and comment
on it.
|
|
TODOs: merge into one commit, add commit message prefix as per ghostflow. Preferably before merging, latest upon merge. |
|
Thank you for identifying the issue and contributing. I started to look into this and then saw you PR here. Here are some additional AI recommendation I found: The PR correctly fixes InitializePriorProbabilities() but leaves the same float-counting bug unfixed in InitializeConfusionMatrixArrayFromVoting(): This is arguably the more impactful site because these counts initialize the confusion matrices used throughout all EM iterations. A consistent fix using a local std::vector<size_t> (or Array2D) for counting, followed by normalization into TWeights, should be applied there as well, mirroring the pattern used in InitializePriorProbabilities(). Additional observations: EM M-step accumulation unaddressed — m_UpdatedConfusionMatrixArray[k][j][ci] += W[ci] still accumulates TWeights (float) sums over all pixels. For large 3D images this is a secondary but real precision issue; using double internally for m_UpdatedConfusionMatrixArray (which is private, so no API impact) would resolve it. ITK style nit — std::vector<size_t> is fine, but SizeValueType (ITK's canonical pixel-count type, defined as uint64_t) would be more idiomatic. |
|
Cool, thanks for the feedback. Of course initially I wanted to be as surgical as possible and only change the things that directly affected the problem I found instead of giving the class a general overhaul. But all your points are valid, am I "allowed" to go ahead and make the changes myself? |
|
That can also be a follow-up PR. |
|
I made the changes and pushed them already. |
Float precision is lost when incrementing a float counter beyond 2^24 (~16.7M). Replace float-based pixel counting with integer accumulators in InitializePriorProbabilities and InitializeConfusionMatrixArrayFromVoting. Use double for m_UpdatedConfusionMatrixArray to avoid accumulation loss in the EM M-step.
8ada944 to
29d724e
Compare
dzenanz
left a comment
There was a problem hiding this comment.
I made through review earlier, and a glancing review after the scope increase. Everything looked OK. Is this functionality being exercised by any tests?
|
It looks to me like there are extensive tests in |
|
The Greptile P1 ("Automatic-prior array leaves its final allocated element uninitialized") is a false positive — this->m_PriorProbabilities.SetSize(1 + static_cast<SizeValueType>(totalLabelCount));
this->m_PriorProbabilities.Fill(0.0); // <-- retained
for (SizeValueType l = 0; l < totalLabelCount; ++l)
{
this->m_PriorProbabilities[l] = static_cast<WeightsType>(labelCounts[l]) / static_cast<WeightsType>(totalCount);
}The trailing entry is deterministically zero, not indeterminate. Greptile's own T-Rex run observed I also audited the head commit for any remaining integer-count-in-float accumulators. There are none — all three items from @blowekamp's review are addressed, and every surviving floating-point accumulator is genuinely fractional. Audit detail
Remaining floating-point accumulators, all correct as-is:
One note for the commit message: Separately and out of scope here: |
The original count uses ++ on a
WeightsType, which defaults to float, to count pixels.When there are more than ~ 16.7 million pixels to be counted, which happens quickly in 3D images (times number of inputs), the ++ has no more effect as 16,777,217 has the same float representation as 16,777,216.
The fix is simple: Do the counting in integers (
size_t) and then the final operationin float (
WeightsType).PR Checklist
Refer to the ITK Software Guide for
further development details if necessary.