Optical character recognition has quietly become very good, which makes its failures more confusing than they used to be. A page comes back perfect and the next one comes back as fragments. The difference is almost never the engine.
What OCR is doing
Three stages, and they fail in different ways:
- Preprocessing. Convert to greyscale, decide what is ink and what is paper, straighten the page, find the text regions.
- Layout analysis. Work out blocks, columns, lines and reading order.
- Recognition. Turn each glyph shape into a character, using a language model to resolve ambiguity from context.
Step three is the part people picture and the part that is now rarely the bottleneck. Step one decides whether step three has anything to work with, and step two decides whether the correct characters come out in the correct order.
Resolution is the hard floor
OCR wants around 300 DPI relative to the text: roughly 20–30 pixels of height for a normal body character.
A phone photo of an A4 page framed edge to edge at 12 megapixels is comfortably above that. The same page photographed from across a desk, at an angle, with half the frame being the table, may have body text only 10 pixels tall. At that point strokes merge, and no amount of sharpening invents the gap between the two verticals of an n.
The fix is to re-take the photo: fill the frame with the page, hold the camera parallel to it, and let the autofocus settle. That single change usually outperforms every processing option combined.
What to fix before running it
Skew. More than about two degrees of rotation and line detection starts merging or splitting lines. Straighten first.
Contrast and shadow. A shadow falling across half the page makes a global black-and-white threshold pick the wrong cut-off on one side. Raising contrast and flattening brightness before recognition is often the difference between half a page and a full page.
Colour backgrounds. Text on a coloured or patterned background loses contrast in the greyscale conversion. Increasing contrast hard, even to the point where the image looks ugly, usually improves recognition.
Perspective. A photo taken at an angle makes characters trapezoidal, which is a shape no glyph model expects. Shooting square is much easier than correcting afterwards.
Where it still fails
Handwriting. Standard OCR is trained on print. Neat handwriting gets partial results; ordinary handwriting does not.
Tables. The characters are usually right and the structure is usually wrong. Cells get merged, columns get read as one line, and the output needs rebuilding by hand.
Multi-column layouts. If layout analysis misses the column break, you get sentences from column one and column two interleaved: every character correct, the text meaningless.
Very stylised type. Logos, condensed display faces and heavy italics have glyph shapes that fall outside what the model saw in training.
Low-contrast scans. Faded thermal receipts and second-generation photocopies lose the strokes themselves, and the engine's language model starts guessing whole words.
Getting the best out of a single page
- Re-photograph if the text is small in the frame. This is the highest-value step.
- Crop to the page.
- Straighten.
- Raise contrast; remove shadow.
- Recognise.
- Proofread digits and proper nouns.
For a scanned PDF rather than a photo, the same logic applies one layer up: see what OCR can and cannot read off a scan for how the text layer gets attached to the page.
For difficult images, adjust the contrast before running OCR, and always proofread numbers, names and anything else where a single misread character changes the meaning.