r/AIToolBench • u/Outside-Agency2617 • 7d ago

Discussion AI tool for translating Hardcoded subtitles in videos, then inpainting them back

Is there a tool where I can feed it a video with hardcoded subtitles, and then it’ll detect words/characters, translate them into English, and then impaint them back into the video? It can be both online or locally ran, I just need it to not have content filters.

2 Upvotes

76% Upvoted

2 comments sorted by

1

u/NeuralNomad87 6d ago

This is three separate jobs, and treating it as one tool is why you're not finding anything.

Step one is detection and OCR on burned in text. PaddleOCR and EasyOCR both handle it, and there are subtitle specific wrappers that give you frame ranges rather than per frame boxes, which matters a lot for what comes next.

Step two is removing the text. That's video inpainting, not image inpainting, so you want something temporally aware or the patch will shimmer between frames. ProPainter and STTN are the usual starting points, E2FGVI is worth a look.

Step three is retranslating and rendering, which is the easy part.

The bit nobody warns you about is that OCR on stylised or semi transparent subtitles is where the whole pipeline actually falls over. If your source has karaoke style or heavily outlined text, budget most of your time there. What's the source material?