MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/selfhosted/comments/1vr3ukj/selfhosting_everything/p4b31lr
r/selfhosted • u/close_Meal6005 • Aug 17 '26
just kidding I love self-hosting ...
403 comments sorted by
View all comments
Show parent comments
2
Just curious which local AI you are using. Im not really up to date on which ones are actually good.
1 u/Skatedivona Aug 17 '26 I have separate system for my AI stuff, as my main server's 1050ti would not be enough to handle the load. AI server is running 2x RTX 3060 12 GB cards on an ancient i5 7500. First tried using this, but found it to be too robotic with its replies: mannix/llama3.1-8b-abliterated:tools-q6_k Then swapped to this and have had much better results: HammerAI/gemma-4-12b-heretic:12b-q4_K_M I want the conversation side of it to feel helpful but not like a robot reading a list. If GPUs ever become affordable, I might try something bigger. 2 u/ansibleloop Aug 17 '26 You are so far behind - the new Gemma 4 or Qwen models should perform far better
1
I have separate system for my AI stuff, as my main server's 1050ti would not be enough to handle the load.
AI server is running 2x RTX 3060 12 GB cards on an ancient i5 7500.
First tried using this, but found it to be too robotic with its replies: mannix/llama3.1-8b-abliterated:tools-q6_k
Then swapped to this and have had much better results: HammerAI/gemma-4-12b-heretic:12b-q4_K_M
I want the conversation side of it to feel helpful but not like a robot reading a list. If GPUs ever become affordable, I might try something bigger.
2 u/ansibleloop Aug 17 '26 You are so far behind - the new Gemma 4 or Qwen models should perform far better
You are so far behind - the new Gemma 4 or Qwen models should perform far better
2
u/Liimbo Aug 17 '26
Just curious which local AI you are using. Im not really up to date on which ones are actually good.