New feature: black and white color mode

[toolicious]

Implements the black and white mode from #150. Measurements are in the issue comment.

Adds ColorMode.BLACK_AND_WHITE, selectable per page and as a default in the settings. The binarization runs on the output of the existing grayscale pipeline and adds a Sauvola threshold. PDF export embeds these pages as 1 bit per pixel with CCITT group 4, rebuilt from the original capture. Automatic detection never selects the new mode, page geometry and the other two modes are untouched, and there is no new dependency.

Three things you may want to decide differently:

• Resolution for this mode is expressed in dpi (150 / 300 / 450) rather than as a pixel budget, because dots per inch is what decides whether 1 bit per pixel looks sharp. It reuses the physical page size the app already estimates for the PDF page box, and falls back to A4 when that estimate is not available. The highest setting interpolates beyond the capture resolution to get finer edges.
• The name, "Black & white" next to "Grayscale".
• I put the three new strings into all locale files, machine translated, the way the other string commits in the repo do it. Happy to drop those and leave you the base file only.

While measuring I also noticed two things unrelated to the feature: during export the page image was fetched twice per page, and the OCR bitmap was decoded even when no OCR language is enabled. Both are fixed in this branch, since they distorted my numbers. I can split them into a separate PR if you prefer.

Tested on a device and with ./gradlew clean license check assembleRelease.

[pynicolas]

Thanks a lot for your PR!
I will have a look at it but probably not in the next few days.

[toolicious]

I just received an email with an error:

1 error
Android CI
This workflow run required approval but was not approved before it expired.

Looks like this has been pending for a month and expired now. Is this still on your radar?
For what it's worth: I've been using this feature many times now for my own purposes during that time and it works very well.

[pynicolas]

It is on my radar, but I didn't manage to work on it so far. I'm sorry for that.
I hope I manage to do that soon.

[pynicolas]

Thanks again for this pull request!
I can tell that you put quite some effort to come up with a good binarization and to stay true to the spirit of the app.
Nice job!

Here are my thoughts:

• The binarization itself looks good, it goes much further than what I had tried myself before. However, I must say I haven't looked into it deeply yet. I noticed some small issues (like disappearing dots with an 8pt font). We may discuss this later.
• CCITT G4 gives good results in terms of PDF size: around 40-80kB (vs 300kB for grayscale JPEG) for ExportQuality.BALANCED.
• Performance is my biggest concern, especially at export time. So far, I tried to make the export almost instantaneous (<100ms) for ExportQuality.BALANCED. For BLACK_AND_WHITE, it takes more than 1 second per page on my device, because the full black-and-white processing is recomputed at export time. I guess I know why you did that: FairScan uses JPEG for its internal storage, and that would have introduced artifacts. Beyond that, the binarization itself takes about 800ms at capture time, which is significant.
• DPI vs pixel budget: I don't see that as related to this PR, so I suggest we put it aside.
• black_and_white_jpeg_hint: I understand the rationale but I prefer not to have it, I can explain.

About the "two things unrelated to the feature":

during export the page image was fetched twice per page

I don't see that. Did I miss something?

the OCR bitmap was decoded even when no OCR language is enabled

Yes, but it's minor (25ms per page), so we can handle it separately.

I started a new branch based on your code to:

• Refactor internal storage to use PNG (8 bits per pixel) for BLACK_AND_WHITE
• Revert other changes I would prefer not to take

It's not ready to be merged:

• my refactoring definitely requires some tests and adjustments.
• It seems to give slightly different results compared to what you did (everything looks slightly "bolder") and I don't know why.

Would you like to have a look at it and tell me whether this could be a good starting point for further discussion?

No rush. And sorry for the time it took me.
Thanks again!

[toolicious]

Thanks for the review, and yes, I'd continue on your branch. Binarizing once at capture and storing it lossless is simpler than my export path and the export is fast. The dpi budget can go, your 4x gives about the same resolution at the balanced setting.

I'd like to hear your reasoning on the JPEG hint. Without it, someone who exports to JPEG picks black and white for a small, crisp file and gets the opposite, with nothing telling them why. In your branch that JPEG is the 8 MP image at q75, 775 kB for a text page against about 250 kB in grayscale. Maybe you have a better solution. Side note: I find it difficult to switch between the output format as it is currently a global setting rather than an export setting for the current file. I saw that there are already open issues for that. Maybe this could somehow be solved together. If the export page allows to directly change the output format, then it could be automatically adjusted and forced to PDF when B&W is selected. Just a thought.

The bolder look comes from the 2 MP step. Your branch shrinks the capture to 2 MP and interpolates it back up 4x, the PR binarized the real pixels. The threshold sits much closer to paper than to ink, so softer edges mean more ink. On a rendered 300 dpi page the strokes come out 6 to 20 % wider. With the dataset photos you won't see it, at 1024x768 nothing gets shrunk. The fix is to resize the warp to 4 * maxPixels for black and white and let binarizeDocument only fill up to that instead of always going 4x. The flattening itself can stay at 2 MP, only the division and the threshold need the real pixels. Tried it: stroke width within 2 % of the PR, about as fast as your branch.

The disappearing dots at 8 pt come from the despeckle step. It drops ink below 12 px at 300 dpi, and an 8 pt period or i-dot is 7 to 10 px. If specks are only removed when there is no other ink close by, all the dots survive in my test. It takes one more connectedComponents call and on old, stained paper more specks get through.

"Fetched twice": Sorry, that was my mistake. main fetches once, the double fetch was in an earlier version of my own branch. I mixed that up.

A few things I noticed in the branch:

• The PNG is written with three channels. One channel halves it (185 to 97 kB for a page here), IMWRITE_PNG_BILEVEL gets it to 59 kB.
• ExportQuality.LOW shrinks the binary image to 1 MP and re-thresholds it, which gives jagged text. CCITT at 8 MP is already small, so black and white could just ignore LOW.
• The document screen and the capture preview decode the 8 MP PNG into a 32 MB bitmap. inSampleSize = 2 for PNGs gives the same 2 MP as the other modes.
• 4x at HIGH is 16 MP, and Sauvola holds several float copies of that. A cap around 9 MP would leave the balanced setting as it is and spare small devices.
• OCR gets the 8 MP bitmap for these pages, four times what Tesseract gets otherwise. Maybe better at ~290 dpi, but slower, worth measuring.

If you want, I can push the resolution fix and the despeckle change onto your branch.

[pynicolas]

Thanks for your answer. I can see that you have in mind a lot of implications for this PR.

On the JPEG hint. Both black&white and JPEG are non-default choices, so I expect almost only advanced users to use such a combination and to know what they are doing. I also assume that some people may use the black&white filter for the contrast it gives, not for the reduced file size: in that case, if they actually need JPEG, the hint may be annoying. I don't have a strong opinion, though.

I think you can go ahead with your PR: take what you want in my commits and add what you think makes sense.
I suggest that we only focus on BALANCED for now. Once we get good results on it, it shouldn't be difficult to handle HIGH and LOW in a way that makes sense.

Thank you very much for your work on this topic, it's impressive!

[pynicolas]

Hi,

First thing: I discovered a performance issue in the grayscale processing. I fixed it on the main branch (#236), which also makes the black and white processing faster.

I continued working on my branch. Here are the main changes:

• The full black and white processing now works at 8 MP (if available) for BALANCED.
• I had a deeper look at the binarization itself and I simplified it. I changed the Sauvola window from maxDim/60 to maxDim/10, which seems to give better results on medium-sized dark areas (e.g. a light heading on a dark background). I also dropped the fill and despeckle steps. I'm not sure this is the right approach yet, and I'm sure there are cases I haven't noticed, so we can definitely discuss this.
• The JPEG export now resizes to get the same resolution as COLOR or GRAYSCALE (2 MP for BALANCED).

Performance seems acceptable: on my test device, enhanceCapturedImage takes about 750ms for BALANCED.
OCR takes about 3 seconds in B&W, compared to 2 seconds in Grayscale. I think that's acceptable for now.

Here's an example that we can discuss:

• source image
• PDF (BALANCED)

Would you be OK if I merged that to the main branch and if we worked on improvements as separate PRs?
Thanks again!

[toolicious]

I had continued on my side in the meantime, but most of it is in your branch now, so I'll drop my branch.

I ran your branch through my test pages: Text pages come out like the PR, same stroke width, 8 pt dots intact. The 12 pt box in your photo is even cleaner than with the PR (image 1).

I found two problems:

• Large dark areas (bigger than about 6 cm) come out white inside. Image 2 is a plain page with one dark rectangle and nothing in it, half of it turns white. Image 3 is a dark area with a reflection, there the light text is gone. Sauvola takes the threshold from the pixels around each point. Deep inside a dark area those are all fill, so the threshold ends up just below the fill level, close to black. With a little noise, half the fill pixels are above it and turn white. The fill step in the PR was there for exactly this.
• Preview and OCR use the 8 MP bitmap for black and white pages, 32 MB per page on screen, and probably your extra second in OCR.

Smaller things, not urgent: the PNG has three channels (twice the size needed), HIGH processes 16 MP, and without despeckle old or textured paper gets more grit. The export screen line I'd still propose as a small PR (your decision, but please consider it after reviewing it again).

I'd fix the dark areas before this reaches a release: the fill logic on top of your window, no despeckle needed. Could you give me until Sunday before merging to main? I'd send that and the 2 MP bitmaps as small PRs against your issue150 branch, one topic each. If you'd rather merge now, I target main instead, but then please no release before the dark areas are fixed.

[pynicolas]

With a window of maxDim/10, the fill algorithm is very slow, about 7 seconds on my device.
I'm going to look into the possibility of combining sauvola with a global binarisation to keep dark areas.

I don't really see why the 8 MP image would be a serious problem for preview and OCR:

• For preview, there's only one bitmap at a time, so the 32MB in memory seems reasonable to me. At least it makes it possible to see the actual result precisely when zooming.
• For OCR: OCR takes time anyway and it's variable. I'm open to simple solutions, though.

[toolicious]

The 7 seconds are the elliptical kernel. With a rectangle the fill step takes about 300 ms at 8 MP on a current phone. PR against your branch: #238. Dark rectangle and reflection fixed, dots of light print stay, your 12 pt box stays clean, text pages unchanged.

A global cutoff does the same on these pages, I checked on your flattened image, and it's simpler and costs no time. What it doesn't do is close small glare specks inside a fill when they're brighter than the cutoff, like the ones at the right end of your first box. If that isn't worth 300 ms to you, take the cutoff.

Preview at 8 MP: fair enough. For OCR, decoding at 2 MP for the OCR run only would be a few lines, if you want it.

[pynicolas]

@toolicious I just merged #243. Huge thanks for your help on this topic, your suggestions and your patience.
I certainly would not have come up with such advanced image processing by myself.

I will probably release it in the coming days, after doing some more testing.

The most important thing I need from you now is to know how you would like to be credited in the GitHub release notes and in the announcement on Mastodon.
Feel free to contact me by email: contact@fairscan.org

Thank you!

[toolicious]

Just my username is fine, with a link to my GitHub profile if you like. I keep this account completely separate from my other (commercial) projects. I'd like to give something back to the open source community, with a few new FOSS projects of my own and with contributions like this one. I'm really glad projects like FairScan exist, and if I can help in an area I know a bit about and deal with anyway, for myself and for people around me, that's good for everyone. Thanks for taking this feature on and getting it into the app!