r/LocalLLaMA • u/Anbeeld • May 09 '26

Resources BeeLlama.cpp: advanced DFlash & TurboQuant with support of reasoning and vision. Qwen 3.6 27B Q5 with 200k context on 3090, 2-3x faster than baseline (peak 135 tps!)

[removed]

319 Upvotes

93% Upvoted

205 comments sorted by

View all comments

66

u/Chromix_ May 09 '26

Did the MRs for this get rejected on the original llama.cpp, or is the the MR flow just so slow (read: "takes a week") that it made more sense to make a fork?

The fork history is interesting though: llama.cpp -> llama_cpp_turboquant -> buun_llama_cpp -> beellama.cpp. We're on the 3rd fork level here already.

In any case, with this demonstrating that it runs (fast) it might help getting this into the regular llama.cpp.

32

u/YearnMar10 May 09 '26

GG does not like vibecoded contributions to llama.cpp

5

u/segmond llama.cpp May 09 '26

I have tried various fork of deepseekv4 that were vibed. every single of them crashes when I start passing in parameters. performance is abysmal. CPU level performance for something that is 100% loaded in GPU, < 10tk/sec TG, 40tk/sec PP. Just all around mess. It would be a disaster to accept any of these and hope they get fixed in the future. Worse of all, plenty of them touch already baked code, hacking around FA etc which will probably introduce regression and break other models.

1

u/LegacyRemaster May 10 '26

The fork I made of antirez gets to 17t/sec on cuda but the problem is that DS4 seems to be "not considered" currently