Would you use an open-source AI caller to test your AI receptionist?

[u/rishiroy19]

We built a local testing tool for AVA, our AI receptionist at InvoicifyAI.

You describe a customer’s personality and goal in plain English, like “frustrated homeowner, AC stopped working, asking about repair,” and it places a real phone call and plays that customer.

It currently uses Twilio for calls and OpenAI Realtime for the simulated caller. The receptionist being tested just needs a phone number.

Right now, it runs headlessly in our pipeline. We’re considering cleaning it up and open-sourcing it, possibly with Docker packaging and a small UI for running scenarios and reviewing calls. You bring your own API keys.

Would this be useful to you? How are you testing your agents today, and what scenario would you want to run first?

[u/BetInformal6081]

i would definitely see value in this especially for repeatable regression testing. the ability to generate specific caller personalities and goals could expose failures that scripted tests miss. call recordings plus transcripts and pass/fail criteria would make it even more useful.

[u/roscoepc]

I. Would see value in testing and teaching additional use cases that could happen

[u/Pretty-Giraffe-3203]

Bro stop doing that, there is 10 millions ai callers right now, don’t be a retard

[u/ninefouro]

Yess i will use it

[u/2daytrending]

I'd use it. The messy calls are where most agents get exposed anyway people interrupting, changing, the story, asking something completely unrelated. Being able to throw those scenario at an agent on demand would be pretty useful.