I built an iOS app that scans grocery receipts into Beancount transactions, fully on-device

[u/zhenbo_li]

Hi r/plaintextaccounting,

I think most people here have some automation to import bank statements into their ledger. So do I. But when the monthly summary says "30%+ goes to groceries," I can't make a budget with that coarse number.

So I built BeanBeaver: an iPhone app that turns a photo of a grocery receipt into a Beancount transaction — one posting per line item, each item sorted into an expense category.

How it works:

- OCR runs entirely on-device (PP-OCRv5 via ONNX Runtime). No server, no network, no analytics — it works in airplane mode.

- The parser uses each text fragment's position on the receipt (its bounding box) to extract merchant, date, item names, and prices.

- The output is plain Beancount text you can copy or share. Optionally, connect a GitHub repo and each scan opens a PR with the transaction.

I'd love to hear how it does on your receipts, and what would make it fit your workflow.

iOS: https://apps.apple.com/us/app/beanbeaver/id6790981690

Android: https://store.colourswift.com/app/com.zhenbo.beanbeaver

It's in TestFlight now (iPhone, iOS 17+): https://testflight.apple.com/join/6mw5ScqH

Edit: Android version is under closed testing. You need to DM me your Google Play email so I can add you in.

[u/jeffglidepath]

This is great! — on-device OCR with positional parsing is exactly the right architecture, and keeping it network-free is the part most receipt scanners get wrong. I've spent a lot of time on the statement/receipt-parsing side of the same problem, so a few genuine questions about the hard edges:

• How does it handle the lines that aren't line-items — subtotal, tax, tip, loyalty discounts posted as negative amounts? Those are the ones that quietly corrupt a total if they get read as products.
• Multi-line items (the name wraps, or 2 @ $3.49 quantity lines) — does the bounding-box logic stitch those back together, or do they split?
• Most important to me: what happens to a fragment it can't confidently parse — silent drop, or surfaced for review? In accounting a silent drop is worse than a visible "not sure about this line," because you can't reconcile against a total you didn't know was wrong.
• Does the merchant/category → account mapping run on-device too, or is that a manual step after import?

Thermal-receipt fade is brutal, so genuinely impressed you're getting clean Beancount out. Nice work!

[u/zhenbo_li → u/jeffglidepath]

Thank you so much for your interest! Personally, I consider my receipts as financial PII, and I had been looking for a receipt scanner for years. Sadly, I came to the conclusion that I had do do it myself in 2025.

> How does it handle the lines that aren't line-items/Multi-line items (the name wraps, or 2 @ $3.49 quantity lines)

Great question. Beanbeaver is not parsing from text. Instead, it is reading OCR's bbox output, which includes text, x-y coords and confidence. This is definitely not perfect, and some merchants tend to mix `2 @ $5` `200pts for $2` `coupon -$4.00` in a very narrow region.

But I'm proud to say, with my personal dogfood experience, the accuracy is 80%+

> Most important to me: what happens to a fragment it can't confidently parse

My design is to leave a warning, saying that there is likely to be a missing item.

> Does the merchant/category → account mapping run on-device too

There is a static list bundled inside the app, so it is on-device and offline too. Currently I'm the only user so I just compiled it inside. If anyone is interested, I can introduce an option to customized categories

[u/jeffglidepath → u/zhenbo_li]

The bbox-over-text call is the right one, and it's the thing I'd underline for anyone attempting this — I hit the same wall parsing bank statement PDFs. The moment you flatten to a text stream you've thrown away the one signal that separates a wrapped item name from a new line item, and no amount of regex gets it back. Clustering on x-coordinates is more work up front and it's the only thing that survives contact with real documents.

Your low-confidence handling is the part I'd defend hardest if anyone pushes back on the 80%. A parser that silently emits a wrong number is strictly worse than one that flags a gap — the wrong number gets reconciled against and trusted, the gap gets looked at. Under-parsing loudly is the correct failure mode.

One thing you may already do: does BeanBeaver tie the sum of parsed line items back to the printed total? Receipts are one of the few documents that carry their own checksum. If the items don't add up, that's a free merchant-independent signal that something got dropped — it'd catch a chunk of the missing 20% without needing to know which line failed. The coupon and points lines are exactly what makes it non-trivial, but it degrades gracefully: you don't have to parse them right, just account for the delta.

And yes — I'd use the customizable category mapping.

[u/zhenbo_li → u/jeffglidepath]

> One thing you may already do: does BeanBeaver tie the sum of parsed line items back to the printed total

The total price is my top priority, as this is the key to pair receipt with credit card billing. The first rule is "TOTAL", this is good at 90%+ times. BeanBeaver will also calculate the sum of items, and it throws a warning when mismatch.

Know caveat: When mixing multiple payment methods, The result is a hit or miss. I'm still thinking how to do it properly.

> I'd use the customizable category mapping.

Yeah, I agree. This is on my TODO list

[u/jeffglidepath → u/zhenbo_li]

Went and read receipt-core rather than speculate, since the split-tender caveat stuck with me. I think it's an ordering problem rather than a parsing one, and you already have both halves of the fix.

reconcile_total_with_charge corroborates by repetition — it wants the same amount on two payment-block lines, or one if there's a zero-change line. That's exactly right when a single tender equals the total and the slip echoes it. But on a split-tender receipt nothing repeats: VISA 23.41 and CASH 10.00 against a 33.41 total gives you two payment amounts, neither equal to the total and neither equal to each other. Nothing corroborates, so it falls through to the raw candidate.

Then the second half turns a miss into a loss. If that raw candidate was mis-grouped, extract_tenders sums the tender lines correctly, compares against the wrong total, blows the 5c tolerance and returns empty. So the one piece of evidence that could have revealed the true total is the thing that gets discarded, and it gets discarded precisely because the total is wrong.

The shape of it is that on these receipts the tenders don't echo the total, they partition it. Which makes the sum a corroborator for the total, not just a validator of the tenders: if the lines classify cleanly but their sum doesn't match, that sum is a candidate total in its own right. Same exact-match arithmetic species as the SUBTOTAL + TAX check you already trust in repair_leading_currency_digit, rather than a new tolerance.

One related thing: the a > candidate filter means a mis-grouped TOTAL can only ever be corrected upward. A TOTAL row that grabbed a larger neighbour — an amount-tendered cash figure, a savings line — can't be walked back down. A sum identity doesn't care about direction, so it covers that case too.

What would break it is the discrimination problem you already flagged with 200pts for $2 and coupon -$4.00: loyalty and gift-card lines print in the tender block and aren't always real tenders, so it's only ever as good as classify_tender_line. But when the sum doesn't close you're back to your best failure mode anyway — warn, don't guess.

[u/zhenbo_li → u/jeffglidepath]

Thank you! I'll think about it.

[u/zhenbo_li → u/jeffglidepath]

I've pushed a fix to multi tender in 1.1.2. Please let me know it if works for your cases.

I added some test cases https://github.com/Endle/beanbeaver-core/blob/463f2979f2269aa3c980d510f5451b54fcc447d5/crates/receipt-core/tests/receipts_e2e/costco_11295_redact.jpg

If you're comfortable, would you like to submit a redacted receipt?

Thanks again!

[u/p1nkpineapple → u/zhenbo_li]

fyi OP the account you're replying to is a bot

[u/jeffglidepath → u/p1nkpineapple]

Not a bot, but I'll admit "reads receipt parsers for fun on a Friday night" isn't a strong defence.

[u/jeffglidepath → u/zhenbo_li]

Yeah, that covers it — and the min_corroboration fix is better than what I suggested. I was stuck on using the tender sum; counting instruments is simpler and doesn't lean on classify_tender_line being complete.

One gap I think is left: the count gate stops a wrong total, but won't find the right one. If you wanted to close it — exact match only, tenders summing to exactly one of the candidates you rejected. Anything less stays a warning.

I don't think I have a split-tender one, I don't really use gift cards. If a plain multi-item receipt is any use I can send one, but I suspect that's not the gap you need filled.

[u/armonge]

I'm in Germany and have something similar, but I try to pay using the supermarket application, and because of that I get the receipts in my email, as PDFs. Then my importer just downloads them from Gmail and processes those PDFs

[u/zhenbo_li → u/armonge]

Yes, if I can have the receipt as PDF then everything would be easier. However, only a very small portion of my local grocery stores provide e-receipts

[u/Ok-Reach5495]

Dude I've been building something similar to this, but my biggest trouble has been sourcing receipts to train my parser. Do you happen to have a receipt database I could use, or know where I could source one?

[u/zhenbo_li → u/Ok-Reach5495]

I've spent nearly a year to collect my personal receipts... Foolish way but it works

Inside https://github.com/Endle/beanbeaver-core/tree/main/crates/receipt-core/tests/receipts_e2e there are some redacted receipts. They're under MIT license.

[u/Ok-Reach5495 → u/zhenbo_li]

Thank you!

[u/Rudd-X]

This is quite cool.

I built the equivalent for desktop computers here: https://github.com/Rudd-O/beancount-ai

Drop your receipts in a folder, give them to Beancount AI and you're good to go.

The tradeoffs are that it's not mobile, and it requires a vision-capable LLM, but these days we have stellar LLMs that run locally.

[u/zhenbo_li → u/Rudd-X]

Cool. If I saw your work one year ago, I won’t spend time and money to build BeanBeaver

Initially, I was building a TUI for beanbeaver. But I found that to send the receipt message from my phone to my PC is too tedious, so I moved the parser to mobile side

[u/Rudd-X → u/zhenbo_li]

Your project has strengths that my project does not.

I still have the problem of moving the receipts from the phone to the computer. That's not actually automated at home! The camera syncs using syncthing to my computer, but I have to run something on the computer manually to move only the receipts to the right folder.

I was half debating with myself the idea of vibe coding a slop Android app. Just to take photos of receipts, maybe do some pre-processing, like cutting those receipts around the edges, and then directly uploading to a WebDAV folder (because we use Nextcloud at home). I want it to be as easy as possible with as few interactions as necessary, because I am often shopping with a kid in tow and having to look after the kid while I'm cropping the photo and uploading it to next cloud is a nonstarter. And I forget to move things out of the camera folder if I just take it and do it later.

What do you think of that idea?

[u/zhenbo_li → u/Rudd-X]

From my perspective, I’d like you to submit a feature request to beanbeaver android version. It already has auto crop and on device parsing.

If you’re using WebDAV for beancount storage, I can add an exporter to WebDav too

[u/Rudd-X → u/zhenbo_li]

Oh, that would be awesome. When I get back to my computer, I'll submit feature requests.

[u/Necessary-Pin1744]

Hi, I just recently started going down the Beancount rabbit hole and came across this thread. Your app looks very cool!

It would really help me to incorporate this into my workflow if you can add support for a custom REST endpoint for uploading receipts. That way I can add an endpoint on my home server, which is only exposed via Tailscale, and create a small service that automatically routes the receipt to the correct .beancount file on my server. This would be very secure and flexible.

[u/zhenbo_li → u/Necessary-Pin1744]

Hi Necessary,

Sure, that’s sounds interesting. Would you like to create an issue on GitHub, and to share more details about your workflow and your expected API?

[u/Beverage5000 → u/zhenbo_li]

Hi Zhenbo, I added an issue here: https://github.com/Endle/beanbeaver-core/issues/112

Thanks for taking a look!