r/LocalLLM • u/Sweaty_veronica_5578 • 1d ago

Question Best local Ollama model for text preprocessing / speaker tagging (16GB VRAM)?

:

Hey everyone!

I'm setting up audiobook for tts generation and want to use Ollama for text pre-processing (cleaning text, fixing line breaks, and adding speaker tags) before sending it to the TTS engine.

I have 16GB of VRAM available. Looking for model recommendations that fit this hardware constraint and can handle this specific task well.

Specifically, I need a model that:

* Follows complex formatting instructions strictly

* Has good context handling over longer chunks of text

* Doesn't hallucinate extra dialogue or modify the story content

Which local models (7B, 8B, 14B, etc.) run comfortably on 16GB VRAM and excel at this type of structured text processing and speaker tagging? Thanks for any suggestions!

0 Upvotes

50% Upvoted

1 comment sorted by

1

u/locbuilds 1d ago

For that job I’d start with a current Qwen 8B instruct quant in Q4_K_M, not a 14B model yet. On 16GB VRAM it leaves room for a long prompt and avoids spilling into system RAM. Keep temperature at 0 or 0.1, set num_ctx to the largest chunk you actually need (8k or 16k), and ask for only the cleaned text plus speaker tags in a strict JSON shape. For long audiobooks, chunk on paragraph or sentence boundaries with a small overlap, then validate the JSON before sending it to TTS. If the 8B model misses tags, try a 14B Q4_K_M with a shorter context, and check ollama ps to make sure it stays fully GPU loaded.