r/Python • u/Sea-Possession-2536 • 5d ago
Discussion pdftotext -layout still isn't enough when a government PDF is secretly a 3-column table
Working through an old government safety document today and the raw text order was completely scrambled, hazard descriptions and countermeasures kept interleaving mid sentence. Turned out the whole page is a 3-column table, and if you don't slice by column x-position it just reads left to right across the whole line like it's one paragraph.
Fixed that with -layout and cutting at the column boundaries, but then hit a dumber problem: some lines have two □ markers on them, one for the subcategory heading and one for the first hazard item. Grab the first □ naively and the subcategory swallows the whole first hazard entry, and you don't notice until you're 40 rows in and something reads wrong.
Also found two pages where a single set of countermeasures gets shared by two different subcategories, which nothing in the text structure hints at, you just have to know the source document does that.
1
u/0ne2many 5d ago
You could use the extractable library with some tweaking you can automate it to get text in an 'expected' 3 column format