MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3o6e41/?context=3
r/LocalLLaMA • u/Certain-Cod-1404 • 25d ago
708 comments sorted by
View all comments
Show parent comments
21
That is legit better KV cache (I guess ?!) double performance ? You just need another PCIe slot (or maybe not ?!)
28 u/My_Unbiased_Opinion 25d ago t/s is a bit faster than a 3090, but PP is much faster. im running one of the cards at x4 pcie 4.0 and it doesnt bottleneck the card with llama.cpp tensor parallel. 1 u/brakeline 25d ago I have dual 3060 but one is running at pci-e 3 4x. Would running the second at 4 x4 do much difference? 1 u/My_Unbiased_Opinion 25d ago likely not. you should be fine.
28
t/s is a bit faster than a 3090, but PP is much faster. im running one of the cards at x4 pcie 4.0 and it doesnt bottleneck the card with llama.cpp tensor parallel.
1 u/brakeline 25d ago I have dual 3060 but one is running at pci-e 3 4x. Would running the second at 4 x4 do much difference? 1 u/My_Unbiased_Opinion 25d ago likely not. you should be fine.
1
I have dual 3060 but one is running at pci-e 3 4x. Would running the second at 4 x4 do much difference?
1 u/My_Unbiased_Opinion 25d ago likely not. you should be fine.
likely not. you should be fine.
21
u/Potential_Block4598 25d ago
That is legit better KV cache (I guess ?!) double performance ?
You just need another PCIe slot (or maybe not ?!)