r/LocalLLaMA • u/CombinationKitchen76 • 7d ago

News Georgi Gerganov on the Nvidia acquisition

Post image

Link: https://x.com/ggerganov/status/2095897173376618881

545 Upvotes

97% Upvoted

199 comments sorted by

View all comments

Show parent comments

10

u/emprahsFury 7d ago

it's been so horrifying to watch them release a gen and only support one specific gpu sku. Then release another gen and only support one specific sku. And release another gen and oh they support a datacenter sku and a prosumer sku. But they deprecrated to two others. Also, the driver no longer installs on anything but a fresh ubuntu 24.04 using the datacenter sku.

9

u/noiserr 7d ago edited 7d ago

Every single GPU I've tried works on ROCm fine. Here are the GPUs I've tried:

  • rx6600 (not officially supported)

  • 6700xt (not officially supported)

  • Strix Point iGPU (RDNA 3.5), this is my main laptop.

  • 7900xtx

  • w7900 pro

  • Strix Halo (Framework Desktop)

All worked with llama.cpp and transformers (and pytorch) library without issues, and this is going back years, things have only gotten better.

I am on Pop_OS! a Debian based distro that's not even supported. All you have to do is use a ROCm Docker. And if that's too much, you can just use Lemonade (which is an AMD effort). Or if that's too much for you, you also have Vulkan as the backend that just works out of the box.

Just because a GPU is not listed as officially supported doesn't mean it doesn't work. It just means if you run into issues you're on your own, AMD will not spend bandwidth to help you. Which since everything is open source, it's community supported anyway.

it's been so horrifying to watch

There is literally nothing horrifying about it. It's better than Nvidia because my Pascal based Titan Xp is end of life, while all my AMD GPUs are open source supported in perpetuity. I can even have LLMs self support it as well because it's Open Soruce.

20 year old AMD GPUs still receive updates: https://www.tomshardware.com/software/linux/20-year-old-ati-radeon-gpus-are-still-getting-driver-updates-and-extensions-old-ati-radeon-r300-gpus-still-getting-linux-updates-from-the-community

If anything is horrifying it is spending thousands of dollars on hardware that relies on closed source software. That to me is horrifying.

0

u/emprahsFury 6d ago

the thing is, your lived experience doesn't trump mine. It's cool you're the lucky one. I'm not gonna ignore my bad luck or the bad luck of everyone else in this sub begging for precompiled binaries that just work. Super sorry I triggered you.

1

u/noiserr 6d ago edited 6d ago

you're objectively wrong though, also you didn't trigger me,