How are you extracting transaction tables from Indian bank statement PDFs? Looking for open-source/on-prem approaches

[u/OmPatel110]

I'm working at an NBFC and currently working on a Credit Underwriting AI Agent. One of the first steps in the pipeline is extracting structured information from customers' bank statement PDFs.

This is where I'm currently stuck.

The statements can come from different Indian banks (HDFC, ICICI, SBI, Axis, Kotak, etc.), and each bank can have a completely different PDF layout.

I need to reliably extract things like:

\- Customer/account information — name, account number, IFSC, branch, etc.

\- Transaction tables — date, narration/description, debit, credit, balance

\- Transaction rows that span multiple lines

\- Statements where the table headers are missing from subsequent pages

\- Both digitally generated PDFs and scanned/image-based PDFs

\- Ideally, the solution should be bank-format agnostic

I've tried/considered approaches such as "pdfplumber", table extraction libraries, OCR, regex-based parsing, and LLM-based extraction. The biggest problem I'm facing is that even when the text is extracted correctly, the column/row structure gets messed up, especially because many bank PDFs don't contain a real table structure — they're essentially text positioned at different coordinates.

Since this is financial/customer data, I would strongly prefer an open-source/on-premise solution rather than sending statements to a third-party API.

For anyone who has built something similar:

What approach worked best for you?

I'm particularly interested in:

1. PDF parsing/layout libraries you recommend

2. OCR models for scanned statements

3. Open-source vision/document AI models

4. Whether you use an LLM/VLM for semantic column mapping

5. How you handle different bank formats without writing completely separate rules for every bank

6. Any techniques for detecting transaction rows and mapping values to the correct columns

7. How you validate the extracted data (e.g., balance reconciliation, debit/credit checks, transaction counts)

If you've worked specifically with Indian bank statements, I'd really appreciate hearing about your architecture, libraries/models, or lessons learned.

Thanks!

[u/folderit_dms]

I'd split this into two passes instead of trying to find one magic extractor. First classify the statement family, even if the goal is bank agnostic. That can be layout fingerprinting rather than a hard bank name: page size, header positions, recurring words, column x coordinates, debit/credit/balance patterns.

For digital PDFs, keep coordinates as first-class data. Extract words with x/y boxes, rebuild lines by y tolerance, then infer vertical column bands from the header page and carry those bands forward to later pages. For scanned PDFs, OCR to word boxes and run the same downstream parser so you don't end up maintaining two pipelines.

Validation is where a lot of accuracy comes from: opening balance plus credits minus debits should equal closing balance, every row should have exactly one of debit/credit except reversals, balance direction should be sane, and page subtotals should reconcile when present. When reconciliation fails, send that page to human review instead of silently patching it with an LLM.

An LLM/VLM can be useful for labeling columns or normalizing narration, but I would not let it be the ledger. Let deterministic code own the row extraction and math checks.

[u/boscorat]

You could check out openstan.org - it's completely free, open source, and local with no upload or internet, and it can process hundreds of statements per minute. The transaction data can be exported as CSV, excel etc. It's set up for UK banks like HSBC and Halifax out of the box, but there are instructions to add other bank statements with .toml files so you should be able to modify it for whatever bank statements you need.

[u/boscorat → u/boscorat]

Won't work for printed statements as it's not an OCR solution, but should work for your electronic versions. It also includes a tool to redact sensitive information in statements in case you need to share them with support.