Using multiple PSMs to improve accuracy

[ehcloninger]

I have PDFs of newspapers from 1904-1947 that I'm trying to get online. I am working to clean up and deskew the source images (~13,000) so that the OCR can do its best. It doesn't help that some issues have a vertical line between the columns and other times it's just blank space. Also, the number of columns varies between years between 6 and 8 columns.

I have scripts that bundle each issue of the newspaper as a single PDF. I am currently using OCRmyPDF with --tesseract-pagesegmode 1 as part of the command line. The OCR works, but it doesn't do a great job of finding the columns. Sometimes it can find a paragraph of text properly in a column, but most of the time it reads all the way across the page. I've been reading some CV blogs and StackOverflow posts that suggest to run multiple passes of tesseract on the images to improve the column detection.

Is this possible in OCRmyPDF? Can I issue more than one --tesseract-pagesegmode in the same invocation or can I run it twice in a row, once with --tesseract-pagesegmode 0 and the next with --tesseract-pagesegmode 1 and have it do the correct thing?

Thanks

[jbarlow83]

I don't think PSMs would be a productive direction with Tesseract. Broadly speaking, Tesseract is not really seeing any R&D to improve its capabilities, and state of the art OCR is using GPT type solutions now.

What would likely work pretty well is do some template matching to detect two column documents, then mask each columns, OCR, and combine the results. For the 6-8 column thing you could use a machine learning vision model and train it to find the bounding boxes of columns or return the count of columns; and there's traditional non-ML methods using binary morphology that aren't half bad (and are probably similar to what the machine learning models learn to do anyway).

Alternately, we maybe some of the improved OCR engines floating about will give better results.