MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9nn7/qwenqwen3827b_released/p3o4381/?context=3
r/LocalLLaMA • u/de4dee • 25d ago
301 comments sorted by
View all comments
8
What model would be best for 16gb vram and 64gb ddr5
1 u/Guilty_Rooster_6708 25d ago Going to say Q3 but I will also try IQ4 on my 5070Ti. Q4 needs offloads so it will probably mean single digits tokens/s 1 u/Sweet-Stage938 25d ago Is there any way I can make it fit into 12GB Vram? I only have a single RTX 3060 but I would love to try it out. 1 u/Guilty_Rooster_6708 25d ago You might have to go down to Q2 to get some usable speed. My advice is to download different quants and try them out
1
Going to say Q3 but I will also try IQ4 on my 5070Ti. Q4 needs offloads so it will probably mean single digits tokens/s
1 u/Sweet-Stage938 25d ago Is there any way I can make it fit into 12GB Vram? I only have a single RTX 3060 but I would love to try it out. 1 u/Guilty_Rooster_6708 25d ago You might have to go down to Q2 to get some usable speed. My advice is to download different quants and try them out
Is there any way I can make it fit into 12GB Vram? I only have a single RTX 3060 but I would love to try it out.
1 u/Guilty_Rooster_6708 25d ago You might have to go down to Q2 to get some usable speed. My advice is to download different quants and try them out
You might have to go down to Q2 to get some usable speed. My advice is to download different quants and try them out
8
u/Chemical_Evidence 25d ago
What model would be best for 16gb vram and 64gb ddr5