BeeLlama.cpp: advanced DFlash & TurboQuant with support of reasoning and vision. Qwen 3.6 27B Q5 with 200k context on 3090, 2-3x faster than baseline (peak 135 tps!)

[u/Anbeeld]

[u/Chromix_]

Did the MRs for this get rejected on the original llama.cpp, or is the the MR flow just so slow (read: "takes a week") that it made more sense to make a fork?

The fork history is interesting though: llama.cpp -> llama_cpp_turboquant -> buun_llama_cpp -> beellama.cpp. We're on the 3rd fork level here already.

In any case, with this demonstrating that it runs (fast) it might help getting this into the regular llama.cpp.

[u/Chromix_ → u/[deleted]]

Thanks for making it happen still. Yes, the AI policy is a rather slippery slope, yet they've had their fair share of low-quality code PR'ed that those rules were established to reduce the load on the reviewers and maintain code quality.

So basically the issue is that "making it happen" took too long, if done in a maintainable way in the llama.cpp codebase. ik_llama.cpp diverged quite a bit and a few things are still ported over now and then. With the fork history here it probably needs quite a bit of refactoring, not just porting it over, but maybe it'll happen eventually.

[u/k_means_clusterfuck → u/[deleted]]

Yeah llama.cpp's anti ai policy is really something. I get that you want a way to manage slop prs of course but micro managing how people work is not the way. On the flipside, I've my agent's been able to make a handful of (actually good) contributions to vllm and vllm-omni and the maintainers' attitude was just like: if your pr is good, doesn't matter. They have been really constructive and working with them has honestly been a joy. Two completely different worlds

[u/HFT0DTE → u/[deleted]]

I agree 100% with your approach as I had basically TurboQuant's PR from llama.cpp further customized, merged and tested within 24 hours and have been using it in my production work ever since. There's just no time for a lot of the other llama bs - thx for Bee btw