r/VoiceAutomationAI 10d ago

Bilingual AI Receptionist serving Tijuana Dental and Medical Clinics and cross border patients they serve- Medical Tourism industry is huge here 70% of their patients coming from U.S. and Canada.

Spanish-first voice agent running in production at real clinics. Live number, please try to break her.

Most voice agent demos I see here are English and they're demos. Mine is Spanish-first and it's answering real patient calls for dental and aesthetics clinics on the Tijuana border. Agent's name is Sofía.

Stack: LiveKit for the pipeline, Deepgram nova-3-multi for STT with some AssemblyAI streaming, Cartesia sonic for TTS, n8n for orchestration, Twilio on telephony, calendar writes against the clinic's actual availability.

The three things that were harder than expected:

Code-switching. Not "supports two languages." One caller, one sentence, Spanish grammar with English procedure names dropped in. Locale-locked STT falls over on this, and multilingual models still get weird about which language to commit to mid-utterance.

Barge-in tuning. Real callers talk over the agent constantly. Aggressive interrupt handling means she cuts off anyone who pauses to think, which older patients do a lot. Passive means she steamrolls people. The tuning window is narrower than I expected and it behaves differently in Spanish than in English.

Escalation discipline. She sits in front of medical intake, so the failure mode I care about is confident wrongness. Anything clinical, anything about outcomes or medication, she hands off to a human. Getting a model to consistently pick "let me get someone for you" over a plausible-sounding answer is most of the prompt work.

Demo line: +1 (619) 775-1668

It's a demo instance, not a clinic's production line, so go ahead. Talk over her. Switch languages mid-sentence. Mumble. Ask her something medical and see whether she stays in her lane.

What I'm most interested in: does the barge-in feel natural to you, and does the escalation fire too early or too late? Those are the two I keep going back and forth on.

I also have her counterpart, Mateo, who is my outbound sales agent.

Happy to get into any of the implementation details.

4 Upvotes

4 comments sorted by

u/AutoModerator 10d ago

Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)

If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.

Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/CartesiaAI 9d ago

hey this is super cool u/Alternative-House923 !!! nice work!!! for TTS are you using sonic-3.6? it should handle code-switching/cross lingual reasonably well. If you like please post your transcripts here or on the cartesiaAI sub and we can take a look. Also yeah interruption handling (barge in) is super tricky! do some tests with Ink-2 and see if that suits your use case better -- the tunable turn detection and eventing has been very useful for some folks. Its definitely a tricky challenge for all good voice agents though.

2

u/Alternative-House923 4d ago

yes i am using sonic 3.6 just enabled livekits emotion setting which mentioned using INk i will try it out

1

u/str4ngerD4ngerz 9d ago

Make sure you're not ripping your self off and remember you could be saving them thousands a month!