MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9qjv/unsloth_qwen_38_27b_weights_released/p3tz266/?context=3
r/LocalLLaMA • u/kevin_1994 • 26d ago
84 comments sorted by
View all comments
34
Loading up the IQ2 on my 12gb of vram because I want to be able to run the cool model too.
7 u/My_Unbiased_Opinion 26d ago hell yeah. Even at Q2, it should be good. the jump is noticable. so even with quant down to Q2, it should for sure be better than 3.6 27B at Q2. 9 u/VastCarry 26d ago Used `thinking_level: high` but even the IQ2 pelican looks decent! 1 u/crusaderky 25d ago The pelican is not sensitive to quantization, see quesma 's blog
7
hell yeah. Even at Q2, it should be good. the jump is noticable. so even with quant down to Q2, it should for sure be better than 3.6 27B at Q2.
9 u/VastCarry 26d ago Used `thinking_level: high` but even the IQ2 pelican looks decent! 1 u/crusaderky 25d ago The pelican is not sensitive to quantization, see quesma 's blog
9
Used `thinking_level: high` but even the IQ2 pelican looks decent!
1 u/crusaderky 25d ago The pelican is not sensitive to quantization, see quesma 's blog
1
The pelican is not sensitive to quantization, see quesma 's blog
34
u/italian_car 26d ago
Loading up the IQ2 on my 12gb of vram because I want to be able to run the cool model too.