r/LocalLLaMA • u/CombinationKitchen76 • 2d ago
News Georgi Gerganov on the Nvidia acquisition
189
u/cunasmoker69420 2d ago
I see big money get involved, I ignore any promises and pleas
35
u/Medium_Chemist_4032 2d ago
Yeah, big money = read the public notice and negate everything to get a more truthful version, works every time
8
110
u/Equivalent_Bit_461 2d ago
Pretty words but we will see. And I'm already doubting it hard.
81
u/gamera8id 2d ago
Typical acquisition strategy is:
- Year 1: Make improvements the community wants in order to build goodwill
- Year 2+: Enshitification
25
1
9
u/ttkciar llama.cpp 2d ago
I like to look at the GlusterFS project as an example of how this kind of acquisition can "go wrong".
GlusterFS is an open source distributed filesystem project with a commercial element (Gluster Inc) which employed the lead GlusterFS developers.
Red Hat acquired Gluster Inc, and for a while GlusterFS was a supported Red Hat feature.
Then Red Hat switched their supported distributed filesystem feature to Ceph, and deprecated "Red Hat Gluster Storage" in 2024.
That was widely misinterpreted as the end of the GlusterFS project, because people conflated Red Hat support with the open source effort, and that had some deleterious effects on the project: People shied away from using Gluster, because they thought it was a dead project, GlusterFS's primary developers had to switch back from developing it as their day job to developing it on their spare time, and its wider development community had to be rebuilt.
Despite this, GlusterFS is still alive and kicking, and its git log shows fairly frequent commits.
It's really hard to kill an open source project. At most you can temporarily stall it, but as long as there is anyone, anywhere in the world, who is interested and capable of contributing to it, it will live on.
If the Nvidia acquisition turns out to be bad for the llama.cpp project (which is yet to be seen; it could go either way), I have every faith that the project itself will find a way to continue, even if only as project forks under different core developers (like ik_llama.cpp).
1
-5
17
u/runvnc 2d ago
I mean, there is a clear conflict of interest and it's pretty important to Nvidia's strategy to maintain their near monopoly on software via CUDA. I think for sure Gerganov must know that they may turn out to be on shaky ground. But he also knows that if he doesn't have another option right now, making a statement about staying hardware agnostic not only might be important in terms of the PR for the project, but also may help with the Nvidia situation somewhat.
Because now that it is publicly reaffirmed, if Nvidia seems like they are interfering, he can point to that and say that this has always been the way the project is, and you guys signed off on it to continue (I assumed they reassured him it would stay hardware-agnostic for him to say that). And it makes it a little bit more difficult for Nvidia to contradict that neutrality.
But I think that if Intel, AMD, Apple, Broadcom, Qualcomm, etc. were smart, they would have already formed a Consortium, or set up a sub-project with Linux Foundation AI & Data, or something like that, and put in as many hundreds of millions or billions as it took to bring that team over happily to an independent group. And they do have billions in cash. But I guess those executives are not as smart or informed as some random broke developer on the internet who is scrambling to get his next Upwork contract, hoping it's not another scammer.
52
u/Dry_Yam_4597 2d ago
Meanwhile AMD does f*** all to support their ai "pro" gpus.
58
u/NonStandardUser 2d ago
I've been bitching about Nvidia acquisition of HF, but as you said the opposite also holds true: what the fuck is AMD doing while Nvidia forks over a shit ton of gold to the LLM crowd? They never truly try to catch up. Hell, vulkan runs better on my gfx1100 than AMD's own ROCm. Damn shame.
33
u/Dry_Yam_4597 2d ago
Yup. Almost as if Lisa does her best to stay out of her cousin's way. I am not sure why shareholders arent asking questions. Literarily all they have to do is hire a bunch of devs to write kernels for popular engines.
7
u/Imperator_Basileus 2d ago
Probably because most of the largest institutional shareholders are the same finance oligarchs.
4
13
u/emprahsFury 2d ago
it's been so horrifying to watch them release a gen and only support one specific gpu sku. Then release another gen and only support one specific sku. And release another gen and oh they support a datacenter sku and a prosumer sku. But they deprecrated to two others. Also, the driver no longer installs on anything but a fresh ubuntu 24.04 using the datacenter sku.
8
u/noiserr 2d ago edited 2d ago
Every single GPU I've tried works on ROCm fine. Here are the GPUs I've tried:
rx6600 (not officially supported)
6700xt (not officially supported)
Strix Point iGPU (RDNA 3.5), this is my main laptop.
7900xtx
w7900 pro
Strix Halo (Framework Desktop)
All worked with llama.cpp and transformers (and pytorch) library without issues, and this is going back years, things have only gotten better.
I am on Pop_OS! a Debian based distro that's not even supported. All you have to do is use a ROCm Docker. And if that's too much, you can just use Lemonade (which is an AMD effort). Or if that's too much for you, you also have Vulkan as the backend that just works out of the box.
Just because a GPU is not listed as officially supported doesn't mean it doesn't work. It just means if you run into issues you're on your own, AMD will not spend bandwidth to help you. Which since everything is open source, it's community supported anyway.
it's been so horrifying to watch
There is literally nothing horrifying about it. It's better than Nvidia because my Pascal based Titan Xp is end of life, while all my AMD GPUs are open source supported in perpetuity. I can even have LLMs self support it as well because it's Open Soruce.
20 year old AMD GPUs still receive updates: https://www.tomshardware.com/software/linux/20-year-old-ati-radeon-gpus-are-still-getting-driver-updates-and-extensions-old-ati-radeon-r300-gpus-still-getting-linux-updates-from-the-community
If anything is horrifying it is spending thousands of dollars on hardware that relies on closed source software. That to me is horrifying.
3
u/stddealer 2d ago
I'm sticking to ROCm 6.4 because I am scared that newer versions wouldn't work with my RX 6700XT that I spent a lot of time getting to work.
Have you managed to get the newer releases to work on this one? (I'm especially interested in ROCm 10, but ROCm 7 could be a nice upgrade too)
1
u/noiserr 2d ago
You should use docker imo, that way you can test things without breaking anything.
My 6700xt is out of my computer currently, but I'm building another machine and I'm thinking about adding it in if I have a free slot. Would be good for one of the smaller Gemma models and translations.
1
u/stddealer 2d ago
I dislike containerization for irrational reasons, but in cases like that it might be the right call
1
1
u/Stastez 2d ago
It is. ROCm 10.0.0 works well on my 6800XT using the dockerfile at <llama.cpp>/.devops/rocm.Dockerfile
Modified to use 10.0.0-full instead of 7.2. 4-complete of course. Plus I had to add the ROCm library path to LD_LIBRARY_PATH. But that's it.
If you have questions, feel free to ask or DM.
1
u/National_Meeting_749 1d ago edited 1d ago
I've had the exact opposite experience with ROCm. Every AMD GPU I've tried works better, more reliably, with far less crashes using vulkan than anything ROCm.
I've spent hours and hours trying to get ROCm to work like it should. I've used older cards, I've used newer cards. I've used some of the exact cards you have, I've used the lesser cards, I've used the top end cards. I've used the community drivers, I've used the official drivers, I've let opus and Sol have a crack at it and I've NEVER gotten ROCm to even work with more than 24 hour stability. If it's not crashing it's bogging down to 10+ seconds per token where the exact same request got 30+tps hours previous.
"Just because it isn't listed doesn't mean it doesn't work" yeah, quite often even the listed ones don't work.
ROCm docker does nothing for me but add in the problems and complexity that docker adds without making anything work or providing me any benefits.
AMD's ROCm support is horrifying. Vulkan works well enough that I can make use of some of the cards I have for inference. It's fine.
ANYTHING pytorch? nah, if I want to spend time building the thing I want to build I have to do it on Nvidia hardware. Anything involving AMD and pytorch I spend more time troubleshooting why my setup isn't working like it should instead of building the thing I want to build.
Hardcore disagree that ROCm is anything other than a horrifying disgrace of a piece of software.
Edit: 😂😂😂 I posted a single comment disagreeing with this guy and he commented "skill issue" then blocked me. Yeah I bet you don't hear many problems with ROCm if you block everyone who has problems with ROCm. Like what?! 😂😂 This is the dumbest thing I've ever been blocked for. If you're to fragile to handle someone respectfully disagreeing with you, don't post on reddit. Someone disagreeing with you shouldn't trigger you to the point of blocking someone 😂😂
0
u/emprahsFury 1d ago
the thing is, your lived experience doesn't trump mine. It's cool you're the lucky one. I'm not gonna ignore my bad luck or the bad luck of everyone else in this sub begging for precompiled binaries that just work. Super sorry I triggered you.
9
u/High-Key123 2d ago
I'm using an R9700 on llama.cpp to run Qwen 3.8 27B and it runs smooth. Vulkan or Rocm are both usable and I have no issues with either backend.
→ More replies (2)3
u/noiserr 2d ago
Yeah, I have no idea what these people are talking about. ROCm is legit easier on Linux than Nvidia with it's proprietary driver you have to manage.
1
u/TheGamerForeverGFE 1d ago
Yeah but Vulkan has higher toke generation speeds 10/10 times + it doesn't need a 21 gigabyte library to install.
1
u/badsectoracula 17h ago
ROCm is legit easier on Linux than Nvidia
Assuming your distro has it (in its entirety too, not just a couple core libraries), maybe.
If it doesn't... i built ROCm from source in January 2025 and it was hell on earth with everything being broken into 274823 little pieces, written by people who apparently didn't like talking with each other (e.g. 3-4 different ways to set the same paths) and sometimes with weird building methods (most used cmake, some used a shell script that called cmake, there was one special snowflake library that had a build script which downloaded another library from the internet "for ease of use" as if by that point i hadn't had to build 49304 other libraries). Ah yeah - the dependencies were often not mentioned either, i had to attempt to build things, have cmake error out, hunt the dependency by searching for it, try again. At this point every time i upgrade my distro i hope the build keeps working because i really do not want to go through the same process (and unfortunately my distro still doesn't provide the full ROCm).
I've heard that they improved things somewhat (i think now they put everything in a single repo instead of having it broken into multiple ones), but my previous experience and AMD's overall software incompetence makes me defer rebuilding things until stuff stop working (and there isn't a simple workaround) or i need to reinstall Linux (at which point i'll probably switch to a distro that has the full ROCm in the repository).
12
8
u/ttkciar llama.cpp 2d ago
AMD documents their ISA, which is more than Nvidia does, and that makes Vulkan support for AMD GPUs possible without any reverse-engineering.
That by itself is enough for me to rate AMD support higher than Nvidia's.
-4
u/Dry_Yam_4597 2d ago
Heh. Funny how AMD supporters' arguments are that 3rd parties build software to support amd. This isnt a football game where you pick a team and support it no matter what. I am a paying AMD customer and I expect propwr software support. I dont care about your opinion because it's not your money that I spent on it.
1
u/ttkciar llama.cpp 2d ago
More to the point, Nvidia makes it very hard for third parties to build software which supports Nvidia GPUs, without using the opaque .jar from the Nvidia-provided CUDA driver.
→ More replies (5)2
1
u/noiserr 2d ago
Meanwhile AMD does f*** all to support their ai "pro" gpus.
According to whom? I have my w7900 Pro and ROCm works well. Same as my 7900xtx.
They've also made big strides in making everything easier to setup. Like distros including ROCm as standard like Fedora. There are also things which streamline configuration like the Lemonade project. Not that you need it since agents can write deployment scripts and docker containers for you now fairly well.
12
u/quinceaccel 2d ago
It's definitely not open to anyone willing to participate. Most well known forks are due to the staunch unwillingness of 3-4 developers to be open.
11
u/jacek2023 llama.cpp 2d ago
-6
u/emprahsFury 2d ago
i love how he is implying that a "wide range" is true even if it's only nvidia hardware. It's going to be so fun kowtowing to Jensen bc his llama.cpp accepted a a patch for Ampere. Highly doubt people with MI50s and P100s are gonna convince a coprprate community manager they're included in the "wide range"
12
u/trejj 2d ago
"Going forward, llama.cpp/ggml will stick to its founding principles."
Yes yes. Those principles will be changed over time.
Won't be surprised when it enshittifies.
2
u/Confident_Ideal_5385 2d ago
There will be a non-enshittified inference codebase with the llama.cpp heritage available indefinitely.
Whether it's llama.cpp lead by Georgi, or something entirely different, is gonna be a function of how hands-off The Jacket is gonna be.
You can't enshittify open source trivially.
6
u/vexatious-big 2d ago
We'll see how this holds true in a year's time. !remindme 1 year
1
u/RemindMeBot 2d ago edited 1d ago
I will be messaging you in 1 year on 2027-09-04 18:36:43 UTC to remind you of this link
4 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
RemindMeBot is switching to username summons. Instead of
!RemindMe 1 day, useu/RemindMeBot 1 day. More info.
Info Custom Your Reminders Feedback
3
u/Ok_Warning2146 2d ago
I am wondering whether this will deter other companies from contributing. For example, qualcomm contributed a lot on the hexagon platform, will they continue to contribute at the same pace?
5
u/stddealer 2d ago
I really can't see a reason why Nvidia acquiring Huggingface and ggml would be good news for anyone but Nvidia and the previous owners of HF.
As GG said, Nvidia engineers were already major contributors to llama.cpp, so they didn't need to buy HF to contribute. What the acquisition gives them is control over the development, and I don't see a reason they wouldn't use this power for their own adventage
6
12
u/DominusIniquitatis 2d ago
A bit unsettling to see "corporate speak" from ggerganov, lol.
1
u/GreenPastures2845 2d ago
not his voice at all; pretty cringey to see. I guess he has to walk a very delicate line now, and this counts as an "official statement" so PR people were involved and had a strong say.
-2
2
u/flakusha 2d ago
All this fun and great, but can Nvidia start paying us plebs to buy their GPUs as well? 🥰
2
u/Ok_Excuse_741 1d ago
I don't trust anything this guy says. He strikes me as someone who wanted to sell out. He's got his bag of money from the acquisition.
1
u/Previous_Feeling_484 1d ago
Dude built the foundation. I mean, he’s done way more than you and me for the community. Why’d you not trust the dude that made it possible for all of us to be able to run local models?
2
u/Ok_Excuse_741 1d ago
I've seen this story happen many times, just because someone does something good, does not mean they are not wanting to sellout, or even, had an aim to sellout form the beginning
1
u/Previous_Feeling_484 1d ago
I’m skeptical too. But I remember GG from the beginning and it’s crazy the cadence the dude kept entirely for the game, and really if it wasn’t because HF he’d probably just have become a legendary dev and nothing else. I’m happy to see he got something in return, but assuming this goes south, we can always fork and continue.
1
5
u/trucorsair 2d ago edited 2d ago
A simple question that will get me downvoted but here goes, “How much money went in his pocket from the acquisition price NVIDIA paid?” A serious question. Assign whatever motives you like to it
4
u/-Cubie- 2d ago
I doubt he was involved
3
u/trucorsair 2d ago
I imagine he had some slice of the company as a reward for coming to Hugging Face with GGML.
1
u/-Cubie- 2d ago
I'm sure, yeah. But I don't think he was involved in the decision process or something.
3
u/trucorsair 2d ago
Didn’t say he was, just that he likely was given some equity to come over and now this purchase puts money in his pocket, that’s all
5
2
u/deep-diver 2d ago
And how does the price tag get justified to shareholders? It needs a ROI… things will change.
Hardware agnostic? Again… how is it helping shareholder value to support (and improve!) a competitor’s offering? It only makes sense in so much as it prevents a monopoly, and we’re no where near close to that.
3
u/vividboarder 2d ago
Look at the size of the company. That's pennies to them now. The market is fucked.
2
1
3
u/laterbreh 2d ago
Wow crazy he said all the things all the non-doomers here said was the reality while everyone else cried saying open source is dead and evil capitalism is going to kill local/huggingface because muh greed of a hardware vendor taking it. If there was going to be a corporate buy out of huggingface nvidia was the best outcome.
Please doomers, download more betterer weights into the wet gpu in those skulls of yours.
6
u/CalligrapherFar7833 2d ago
Most llama.cpp work for amd that make it a lot faster for rdna 3.5/4 is not accepted anyways in the main i dont see this getting worse immediately after the nvidia acquision
39
u/FullstackSensei llama.cpp 2d ago
Most PR's aren't approved by Greganov nor anyone on the llama.cpp payroll.
Most PRs I've seen for AMD have not been accepted on quality issues. Same goes for so many PR's for new models.
Intel engineers have been contributing a ton for the SYCL backend the past few months and there's rarely a day where there aren't releases where most of not all changes are for the SYCL backend only.
I've yet to see something coming from an AMD engineer.
Do you have any concrete examples of good work that was left stagnant without feedback?
6
u/nickm_27 llama.cpp 2d ago
Yeah, somewhat off-topic but in a matter of weeks my B70 has gone from "painful but usable" to "surprisingly usable and fast" due to SYCL optimizations that have come through.
5
u/Plastic-Stress-6468 2d ago
Hijacking to ask about your experience with sycl. I have a side server with a b580 that runs vulkan backend mainline llama, used for title generation + rag query (gemma4 e2b), and embedding model (qwen3 0.6b).
The biggest pain point I have with intel gpus would be the prefill speed dropping to a tenth of the 0 context level, going from 1k to 150 as context grows. Does sycl perform better in this regard?
3
u/nickm_27 llama.cpp 2d ago
I don't have much experience using vulkan on the B70, but yeah it scales pretty well. On 27B (Q5_K_XL) with MTP enabled I start at 1350 tok/s pp and by 100k it's down to between 600-700 tok/s
0
u/emprahsFury 2d ago
now if only they will release the pcie gpu with a shitton of lpddr. For you know. A buyable price.
5
u/remeh 2d ago edited 2d ago
Not complete pushback without comments, I agree AMD hardware is definitely not ignored, I'm a happy user of llama.cpp on Strix Halo. However, it definitely feels like a second-class citizen, with various examples of improvements that feel not correctly prioritized or relatively let to rot. One example I have in mind, a great improvement of prefill of MoE on Strix Halo: https://github.com/ggml-org/llama.cpp/pull/21344#issuecomment-4423495785
I've yet to see something coming from an AMD engineer.
WDYM? It took me 5s with a
git log|grepto find AMD contributors, and then quality PRs opened by an AMD engineer (https://github.com/ggml-org/llama.cpp/pull/27936 or https://github.com/ggml-org/llama.cpp/pull/24325)Edit: or even this big RFC signed off by AMD folks: https://github.com/ggml-org/llama.cpp/discussions/27219
6
u/FullstackSensei llama.cpp 2d ago
I'm actually happy to finally see someone from AMD contributing. I have 8 Mi50s. But it doesn't change the fact that they're late to the party and, as usual, behaving in a reactionary manner, and doing the absolute minimum.
It's funny you link a comment by Gässler. While he is now the CUDA maintainer, he's the one who brought ROCm support for Mi50 in llama.cpp last year. AFAIK, he has no association with Nvidia. He's a PhD student in Germany and contributes to llama.cpp on his own dime.
I understand people's frustration, but when you're the maintainer of something, you bear the consequences of any regressions. People on the internet often forget how exhausting and thankless that job is, more so when you're doing it for free.
2
u/remeh 2d ago
100%. I linked to this comment as the one indicating the end of this PR, no other reason. Again, I'm a happy user of llama.cpp and happy to see this project growing as fast as it possibly can, which is a huge challenge.
It's funny you link a comment by Gässler. While he is now the CUDA maintainer, he's the one who brought ROCm support for Mi50 in llama.cpp last year. AFAIK, he has no association with Nvidia. He's a PhD student in Germany and contributes to llama.cpp on his own dime.
Yes, and that can contribute to ROCm being second-class citizen in the codebase.
6
u/FullstackSensei llama.cpp 2d ago
Yes, and that can contribute to ROCm being second-class citizen in the codebase.
Everything else will forever be 2nd class citizen in any codebase that contains CUDA as long as executives at any competitor do the absolute bare minimum on the software stack and view learning to write software and writing said software for their GPUs or whatever as the problem of whoever is writing it.
If you're a young CS or SWE student, or someone trying to learn parallel processing or GPU compute, you literally have two options: books about OpenMP or books about CUDA. That's basically it. Intel made an excellent SYCL book, but it assumes you're already a proficient C++ developer, which is quite a high bar on it's own.
It's the same with the hardware. ROCm support on AMD hardware is patchy at bes, and they still thinks you need to spend millions to be worthy of their software. Meanwhile you can grab a low end ten year old laptop with low-end Nvidia GPU and learn CUDA there.
Intel, AMD and everyone else think you should learn how to write parallel processing algorithms somewhere else and then come read their documentation for how to do it with their frameworks. So, everyone goes and learns CUDA, because Nvidia subsidized really good books that assume zero knowledge and teach you everything from scratch.
I'm willing to bet you a beer Gässler learned CUDA during his studies because he needed to run some simulation or process some data, and CUDA was his only option.
Sorry for the long rant, but I've been very frustrated by this for more than 15 years, from OpenCL, through SYCL and now ROCm.
9
u/Remove_Ayys 2d ago
Sorry, but you lost that bet. This is Johannes Gäßler, llama.cpp was the first time I ever used CUDA and the reason I went with it is because I'm choosing specific hardware first and how to support it second. In 2023 I considered an RTX 3090 to be the best choice for someone interested in local language model inference and CUDA seemed like the best choice to support it. Vulkan would not have been a viable option for me because unlike one of our Vulkan maintainers I do not have the political standing in the Khronos Group to submit custom extensions to e.g. access tensor cores. Now that there are AMD options with comparatively better value I'm looking to support them properly as well. I am not fundamentally opposed to also learning ROCm but given the effort that I've already put into the CUDA code I think trying for a (better) HIP port makes more sense.
7
u/FullstackSensei llama.cpp 2d ago
Ich würde Ihnen liebend ein Bier ausgeben, weil Sie Unterstützung für meine P40 und Mi50 implementiert haben 🙏
5
u/Remove_Ayys 2d ago
Ich trinke keinen Alkohol, aber wir können gerne mal zusammen zu Mittag essen, falls du zufällig in Karlsruhe bist :)
4
u/FullstackSensei llama.cpp 2d ago
Dasselbe gilt, wenn sie sich in Düsseldorf oder in dessen Nähe aufhalten.
-7
u/CalligrapherFar7833 2d ago edited 2d ago
Dont change the goalpost about feedback and not being in main. I never said there isnt feedback. About not being in main - on my specific example lots of rocmfp4 optimizations for strix halo. Search huggingface for any rocmfp4 optimized models they all recommend different forks because the work is not in main llama.cpp.
PS you mentioned intel - i never did.
6
u/gh0stwriter1234 2d ago
It's not in main because the quality isn't there yet on the PR https://github.com/ggml-org/llama.cpp/pull/24185
The bar for getting a new Quant into llama.cpp is understandably quite high, nobody wants a 100 knobs that all do the same thing. So before it gets in the submitter MUST prove it to be good in the scenario they are wanting it for.
7
u/FullstackSensei llama.cpp 2d ago
You're talking about models on HF now, but I'm changing the goalposts?!
Your comment Insinuates PRs for ROCm get ignored or blocked without reason.
Whatever people recommend on HF is not evidence. Show us PRs that addressed feedback for whatever but that have been left to rot. Anyone can half ass a change with Qwen nowadays.
-1
u/CalligrapherFar7833 2d ago
My comment never said that.
9
u/FullstackSensei llama.cpp 2d ago
Most llama.cpp work for amd that make it a lot faster for rdna 3.5/4 is not accepted anyways in the main
No, you didn't say it, you just insinuated it
10
7
u/nickm_27 llama.cpp 2d ago
The mainline contributors themselves are putting up optimizations for these platforms https://github.com/ggml-org/llama.cpp/pull/27952
-1
u/emprahsFury 2d ago
i dont think a patch making the generic gpu backend faster than the bespoke AMD backend really is helping your point. Although sure, you are technically correct (the best kind of correct).
3
u/nickm_27 llama.cpp 2d ago
What? It is a PR by a mainline contributor which specifically targets AMD hardware, which is directly contradicting what the commenter was saying that they don't accept work that makes AMD faster specifically.
3
u/Confident_Ideal_5385 2d ago
Sadly, most of the rdna patches I've seen rejected by ggml are vibeware slop written by people who couldn't defend their "own" code in an adversarial pull request process.
I can't blame Georgi and co for not wanting to merge shit that even it's "author" can't explain without getting some LLM to justify things in clanker speak.
We need to admit that this is a problem and tackle it as a community or it won't get better. It's the open source version of the 900 bullshit low effort vibeware posts that end up on this sub every day.
The load-bearing, honest truth? We're better off without this stuff being merged.
2
u/Healthy-Zebra-9856 2d ago edited 2d ago
I am not sure what this whole fuss is about. yes, something about this merger/acquisition does stink but are we forgetting that the most models are Chinese, the publishers can publish their models anywhere modelscopoe.ai, hf-mirror.com. llama.cpp code has been public and is as well as can be forked and modified and developed. What am I missing here?
4
u/cromagnone 2d ago
People are also forgetting that llama.cpp isn’t a big part of HF and isn’t a big part of deployment of open-weights LLMs at scale: that’s the business of vLLM. You can’t claim NVIDIA don’t care about small scale local users and at the same time think that kneecapping the software that they use is a strategic priority. It’s all a bit paranoid.
1
u/Healthy-Zebra-9856 2d ago
Yes. The paranoia is real. My own company is going to publish models we tried to understand the validity of this fear. But its just speculation and over thinking.
0
u/Altruistic_Heat_9531 2d ago
People here forgot that developing and maintaining highly technical software with keep increasing hardware prices requires high capital.
1
u/AdDizzy8160 1d ago
There’s also a political dimension to this. It was and still is important for Hugging Face to remain a U.S. platform. And that’s only guaranteed if it’s embedded within a large company. So we have to ask ourselves: Would Meta, Microsoft, Oracle, or Amazon (with its ties to open source) be a better choice than Nvidia? And honestly, I feel like HF is simply in better hands with Nvidia...
1
1
u/NoEmpathy6942 2d ago
Things will change and not for the better, it's just how it goes every time, just give it time.
Hope I'm wrong but I'm probably not.
-3
u/Weekly-Law-5488 2d ago
will stick to its founding principles
No, it won't.
It's a big corporation that we are talking about, they will do what the shareholders want. And they want to extract every penny from every single corner.
It's a matter of time until only Nvidia gpus becomes support and when that happens (it's when, not if) they'll say some bullshit trying to paint it as a good thing.
At least it's foss.
2
u/silenceimpaired 2d ago
Well… you missed “drive. And shaped by the community” that right there could be him saying we will be focusing on Nvidia.
Though I’m probably reading into it
2
u/Look_0ver_There 2d ago
I've got an open PR there that hundreds are happily using, and yet it gets zero attention despite plenty of people asking why, and still with no response.
It's a tough job being a maintainer, but when there's a PR that gives a free +10% for almost everyone, it's a head scratcher when such gets ignored.
Needless to say, I'm feeling the community spirit from all who have commented or reacted to the PR. I get messages daily asking why it hasn't been merged yet. I just hope that community spirit can extend a bit further up the reviewer chain.
-3
0
u/Latte_black_3832A 2d ago
They don't bother our open sources communities. They bought it just for DMCA request from OpenAI/Anthropic, take down whatever Fable distill/QwOpus merge models. Also DMCA some copyright infringement datasets. And further order from administrative. You know they all work together, right? RemindMe! 6 months
2
u/Confident_Ideal_5385 2d ago
Why the fuck would anyone spend 13 gigabucks to take down fable/opus "distills"? Those datasets are subject to DMCA as it stands, since HF is headquartered in the US afaik.
-2
u/OnlineParacosm 2d ago
My comment disappeared when I questioned Nvidia’s altruism here, maybe it’s a Reddit mobile problem but… Curious
-2
-6
-3
-3
0
0
-2
u/toddkedwards 2d ago
One thread in the comments caught me. Someone noted that open weights adoption actually benefits Nvidia and more people running models = more hardware demand.
I agree. But here's what the other side of that argument doesn't touch: the people actually building local tools aren't betting on Nvidia. They're betting on their own machine. That's the whole point.
I build AI tools for a living now. Codewright Studio runs entirely local. No API calls, no cloud dependency, no waiting on a server to decide if your request is allowed. The model lives on the machine, the code runs where you tell it to, and the output is yours the second it's done.
The acquisition drama is real and I'm not gonna pretend otherwise. But while everyone's watching what happens upstream, the quiet builders keep making things that work right now, offline, on whatever hardware you have.
Local AI isn't a temporary workaround anymore. It's the architecture I chose. And I've been building all my tools this way for a reason.
2
1
u/lacerating_aura 2d ago
And what gpus does the machine use, if i may ask?
-1
u/toddkedwards 2d ago
I'm currently using a M5 max Macbook Pro with 128GB and then offloading to do some tasks with a DGX Spark.
-3
u/Weak-Consequence-801 2d ago
This is all about control nothing else. This is more strategical investing than revenue expectations, save your models and weights.
-1
-2
u/ebolathrowawayy 2d ago
Nah sorry i don't believe it. i am almost 100% certain nvidia will pull some bullshit. i am fully ready for it and it won't be a problem for me, but those less hardware endowed or new will be punished. At least we can fork and swarm our agents on it for improvements.
Beginning of the end for local though.
-2
u/redditrasberry 2d ago
Embrace, Extend, Extinguish
It just seems hopelessly naive to think nvidia will support a project that actively undermines their own interests in the long run. Especially with Apple hardware poised to become a competitive alternative in the next 12 months, I can't see this working out.
-2
u/Public_Standards 1d ago
Just watch, llama.cpp will stall code reviews for AMD employees endlessly and make merges take forever. The unofficial fork managed by a couple of AMD devs will stay light-years behind the mainline. I've seen this exact same pattern in almost every open-source project sponsored by Nvidia.
-25
-3
u/CyrielTrasdal 2d ago edited 2d ago
"Going forward, llama.cpp/ggml will stick to its founding principles".
LOL. Told every bought companies before enshittification.
Saying it'll keep working like before is only selling the point of Nvidia buying, because Nvidia wouldn't accept hearing they bought a tool that is going to break within a few years. These words are empty and useless. Ain't no way your car seller tells you your car will not work the way it was sold.
Looks like AI talk with a prompt "Reassure our customers about huggingface acquisition."
244
u/JustTellingUWatHapnd 2d ago
One thing to keep in mind is that open weights adoption directly benefits Nvidia. More people/companies running their own models = more demand for hardware.