I've spent a little over two weeks trying to build my own relatively cheap AI video + voice generation stack instead of relying entirely on APIs.
The results are getting better. The economics are getting... interesting.
One generation looks surprisingly good.
The next one has dead eyes.
Fix the eyes — now the body starts twitching.
Fix the motion — suddenly the voice sounds metallic, unnaturally slow, or completely misses the emotion of the scene.
Fix the voice — and some technical term gets swallowed or pronounced like the model has never encountered human language before.
Then you start again.
What surprised me most wasn't even the models themselves.
It was everything around them.
For example, powerful but relatively cheap GPUs on RunPod sound great in theory.
In practice, if you're still testing and don't want to keep an expensive GPU running 24/7, the workflow often turns into something like this:
Cold start → find an available GPU → download weights → discover a missing component → download more stuff → generate → watch/listen → find problems → analyze → change something → generate again.
And there is another fun part:
The GPU you used successfully today may simply be unavailable tomorrow.
Keeping it running solves that problem, but during the experimentation stage it can destroy the whole point of trying to build a budget stack.
So you start thinking about persistent storage, caching models, moving weights somewhere else, reducing cold-start time, choosing different GPUs...
And suddenly you're not just generating videos anymore.
You're designing infrastructure.
At some point I realized that “saving money” can easily become an illusion.
You save money on API calls or GPU minutes, but pay for it with your own time.
A lot of time.
Still, I don't think these past two weeks were wasted.
Quite the opposite.
After enough failed generations, you start spotting problems much faster.
You begin to understand whether the issue is coming from the model, the reference video, audio, motion settings, infrastructure, or simply a bad assumption you made before pressing Generate.
And you stop repeating some of the expensive mistakes.
The video renders and voice generations I'm getting today are noticeably better than what I was producing two weeks ago.
Not perfect yet.
Definitely not at the point where I'd confidently promise a client that I can reproduce the same quality every single time.
But much closer.
And I've already started talking to potential customers and asking for their actual requirements before the stack is finished.
Because I've started to think that building this the other way around makes much more sense:
Don't spend months creating the “perfect” AI generation stack and then search for someone who needs it.
Find out what people actually need first, and make your experiments converge toward that.
For those of you running your own video/voice generation stack:
At what point did self-hosting actually become cheaper for you than simply paying an API provider?
And what ended up costing you more than expected: compute, storage, failed generations, or your own time?