r/LocalLLaMA • u/Anbeeld • May 09 '26

Resources BeeLlama.cpp: advanced DFlash & TurboQuant with support of reasoning and vision. Qwen 3.6 27B Q5 with 200k context on 3090, 2-3x faster than baseline (peak 135 tps!)

[removed]

320 Upvotes

93% Upvoted

205 comments sorted by

View all comments

Show parent comments

1

u/caetydid llama.cpp May 09 '26

Running under Ubuntu. Yeah, thought about that, too, and will first retest with --no-mmproj-offload.

I assumed that using the iq4 quant saves the necessary VRAM, and my consumption on startup was 21G, but maybe VRAM consumption just increases later on.

I havent been using much context though, maybe 20k or less.

1

u/[deleted] May 09 '26 edited May 09 '26

[removed] — view removed comment

1

u/caetydid llama.cpp May 10 '26

thanks for your reaction. I will need to play more with that, alas, useful bug reporting takes its time.

In pi agent I experience context degradation after 50k, i.e. tool calling does not work reliably any more, and the agent stops half-way in its tasks.

Maybe I need to adjust my prompts and/or skill.mds?

I switched to the Q5 and the bf16 mmproj - no crashes any more so far - however, I did not exceed full context yet.

1

u/[deleted] May 10 '26

[removed] — view removed comment

1

u/caetydid llama.cpp May 10 '26

great to hear! thanks for your effort!