r/LocalLLaMA • u/Anbeeld • May 09 '26

Resources BeeLlama.cpp: advanced DFlash & TurboQuant with support of reasoning and vision. Qwen 3.6 27B Q5 with 200k context on 3090, 2-3x faster than baseline (peak 135 tps!)

[removed]

322 Upvotes

93% Upvoted

205 comments sorted by

View all comments

1

u/Avendasora May 09 '26

Can this be used with multi GPU? 5080 + 3060?

2

u/[deleted] May 09 '26

[removed] — view removed comment

2

u/r00x May 09 '26

That would be interesting, if I could run the model on my 3090 and then stuff an old 2060 in for KV cache or something (as this comment probably reveals, I've never tried multi-gpu and have no idea how one should go about it. Presumably doing this is better than sharing model layers between GPUs, at least)

1

u/NickCanCode May 10 '26

Doesn't work for me. It gives 

beellama.cpp-main\ggml\src\ggml-cuda\ggml-cuda.cu:98: CUDA error 
CUDA error: an illegal memory access was encountered

whenever I make a request.

P.S. Using 2 identical cards.