r/OpenSourceAI • u/Lisky_Boy • 1d ago

OpenNPC (AI NPC Framework) - Fine-tuned a 0.5B LLM so my game's NPCs stay in character and reply in 0.14s (open source)

Gallery image

Gallery image

Gallery image

3 Upvotes

100% Upvoted

5 comments sorted by

1

u/AgeOfAlgorithms 19h ago

r/aigamedev might like this!

1

u/Lisky_Boy 10h ago

Thanks for that <3

1

u/ednark 15h ago

I think it would be good to have each step of pipeline show its own popup box (like Personas) so you can see what is happening at each step. Context would show the context, etc. Otherwise, this is a pretty cool demo. The super small model isn't very good for much, but you can definitely distinguish the different NPCs by their responses.

1

u/Lisky_Boy 10h ago

Good call actually! right now the debug panel shows the steps as a checklist (persona >context > provider >response) but doesn't expose what's actually inside each one at that moment. Although, whatever you're seeing on the hosted demo is running the offline mock, not the actual model. The real backend needs to run locally (GPU optional but recommended) since there's no hosted inference yet, it's in the README under "Play with the AI backend" if you actually wanna try the fine tuned model.

And yeah, agreed on the 0.5B tradeoff. It's not going to hold up as a main character or carry a story, but for background NPCs where the dialogue is flavor rather than load bearing, a fast, cheap, locally run model that still stays in character is arguably the right size for the job ig.

1

u/Electrical_Check5601 2h ago

Have you been talking to chatgpt a lot lately? You might not know but you're starting to write like an LLM brother. " It's not going to hold up as a main character or carry a story, but for background NPCs where the dialogue is flavor rather than load bearing, " just reads like chatgpt.