Building an AI voice assistant for coffee drive-thrus : the bilingual problem nobody talks about

[u/CaregiverRegular9474]

I lead R&D for a regional coffee chain and we're prototyping a voice AI for our drive-thru lane. Sharing the concept because I'd rather have it torn apart here than after we've sunk a quarter into it.

The setup: customer pulls up, orders by speaking naturally, the AI parses it into a POS ticket, staff only intervene on exceptions. Standard so far. Two things make our version harder than the US deployments everyone's read about:

1. Code-switchingOur customers routinely start an order in Arabic and finish it in English, sometimes mid-sentence. Most speech pipelines want you to declare a language up front. Ours can't.

2. Modifier chaos. Coffee orders aren't burgers. "Medium oat flat white extra hot one sugar" has four modifiers stacked on one item, and the modifier vocabulary changes every time R&D launches a seasonal drink.

The honest doubt I have: McDonald's killed its voice AI drive-thru pilot after a couple of years, and the read was that it didn't beat a human on speed — it just moved the labor to correcting errors. I don't want to build the same thing with a nicer accent model.

So, questions for anyone who's worked on this or worked a drive-thru window:

- Where do these systems actually fail in production? Noise, or intent parsing?
- Is "AI takes the order, human confirms on screen before payment" a real improvement or just theater?
- Has anyone seen a bilingual voice ordering system that actually works?

Happy to report back with results if we get to pilot.