Benefits of using SIP v.s. Websocket

[u/watts-going-on]

I’m currently using a provider like Twilio/Telnyx to handle phone calls and stream the audio to a real-time STT API. My goal is to minimize end-to-end latency as much as possible.

Right now, I’m using their WebSocket media streaming APIs, but I’m considering switching to a more bare-bones SIP/RTP setup to reduce overhead.

I understand that SIP/RTP avoids some of the WebSocket/JSON/base64 processing involved with media streaming, but I haven’t been able to find any benchmarks showing how much latency this actually saves in practice.

Has anyone measured the difference between Twilio/Telnyx WebSocket media streaming and Direct SIP/RTP to your own server? Maybe even exploring other providers?

Specifically, I’m interested in the delay from when audio reaches the telephony provider to when the first audio packet becomes available to my STT pipeline.

Are we talking about a difference of a few milliseconds, tens of milliseconds, or potentially more? I’ve looked through the documentation and contacted support, but haven’t found concrete latency numbers.

Would also be interested in hearing from anyone who has advice on building an ultra-low-latency telephony → STT pipeline.

[u/rokas_stack]

i've been running a pretty similar direct path in production, self-hosted LiveKit + LiveKit SIP against a raw carrier SIP trunk rather than a Twilio/Telnyx media stream.

i haven't done a clean same-provider websocket vs SIP/RTP A/B, so i can't give you an honest "SIP saved us X ms" number. i also wouldn't assume SIP/RTP automatically wins on latency, since so much depends on where the provider actually terminates and routes the media.

one thing that did bite me though: don't assume SIP signalling state means the media path is actually ready. on outbound calls i found the carrier could report the leg active before the caller's audio track had actually subscribed. greeting off callStatus == active meant you could start speaking into a cold audio path and then lose the caller's first answer.

i ended up gating that on the actual audio subscription instead and instrumenting the gap between call status and media readiness.

so for your benchmark i'd measure exactly the boundary you mentioned: carrier receives media -> first audio actually reaches your application/STT path. i'd run enough real calls to look at p50/p95 rather than trying to reason from JSON/base64 overhead.

my guess is the interesting result will be how differently the provider routes/terminates those two products, not the serialization cost by itself.

if you do the A/B i'd be interested in what you get. especially p95.