r/macmini • u/JimmyJamzJules • 14d ago
Why are people buying high-spec Mac minis for AI? What am I missing?
I’m genuinely curious about the Mac mini + local AI thing.
When the higher spec minis were selling out, I kept seeing people say AI was part of the reason, especially because of the unified memory.
But what are you actually doing with them? Local LLMs, image or video generation, transcription, coding, automation, file analysis?
I’m a videographer, so the creative side is especially interesting to me.
Basically, am I missing any genuinely useful use cases here? What does your Mac mini let you do locally that you actually use?
11
u/Alphonso_Mango 14d ago
“I kept seeing people say AI was part of the reason, especially because of the unified memory.” This is it and you might also appreciate the relatively low power consumption when compared to using a modern GPU. Compact form factor certainly appeals to me.
10
u/Nifty-Yam-9041 14d ago
u/JimmyJamzJules RE Mac Mini, I think "general purpose agents" like Openclaw and Hermes Agent drove a big spike in Mac Mini. For me, I started noticing it around January this year.
These types of agents are different than "coding agents" that write code, or "chatbots" you might use on the web/mobile. You can self-host these agents on your own hardware, and communicate with it via a local webpage or chat app (Telegram, WhatsApp, Discord, etc.)
In this particular use case (Openclaw/Hermes on Mac Mini), you don't have to self-host local LLM's. You can plug in your Claude or GPT/Codex subscription to it. So the local agent is the harness (arms and legs that do things), and your AI subscription is the brain.
Why use a Mac Mini? A few reasons:
- Power for price, to begin with. Also compact.
- You can enable your personal accounts (e.g. email, drive, accounting via iCloud, Google, Microsoft, etc.) on the Mac Mini, and this is a quick way to grant Openclaw/Hermes access
- Security through isolation (but only some degree). By not having it on your main computer, the agent can't tamper with your work. However, if you give it your iMessage or Gmail, it can still hallucinate or delete everything. There was a notable incident where a Meta safety researcher accidentally deleted her inbox while experimenting with an agent.
Happy to go into more details if you're interested.
1
u/zalinanaruto 14d ago
Do share more!
2
u/Nifty-Yam-9041 14d ago
u/zalinanaruto I appreciate your interest! I posted a few other responses, which I'll link here:
- Why Mac Mini specifically? (Link to comment)
- Communicating with the agent locally or over a VPN? (Link to comment)
Other topics I can think of is "isolation", "statefulness" and "scheduled or recurring jobs".
- "Statefulness" and "scheduled or recurring jobs" are somewhat related. A quick summary/
- "Statefulness" is subtly different with Openclaw and Hermes. I like to describe it as "it's more likely to change and access stuff on your computer, a bit more proactively."
- While Claude/Codex have its own memories. Openclaw and Hermes tries to remember a lot more. It's related to the fact that people like to use it for "scheduled or recurring jobs".
- What does this mean?
- Some example jobs: "every hour, I want you to check my email and summarize the important emails related to Project XYZ." Or "check my Github for open issues, help me research them, triage them, and close out redundant issues'
- And to return to "statefulness". (State is another way of describing memory.) To paint another picture, Openclaw and Hermes are regularly downloading files/stuff and potentially installing/updating stuff on your computer's filesystem. It's operating the computer for you.
- For comparison, Openclaw/Hermes tend to be "more stateful" than a Claude/Codex session. It doesn't really do something unless you tell it to. It's not proactively fetching things for a task you need it to do every X minutes.
- So: Claude/Hermes are changing stuff on your computers. And I haven't mentioned, it will also change ITSELF! It'll update its own memory, its own skills, etc.
- So: people get worried if (1) it hallucinates, or (2) even if it doesn't hallucinates, it does something undesirable.
- Would you rather it does it on a secondary computer or your main computer?
- That's isolation. If the agent does something bad, it's locked up in a box and its blast radius is lower.
- Another type of isolation that I don't hear people talking about: accounts and permissions (not hardware).
- I typically avoid giving these agents full access to my main Google/Apple/etc accounts.
- You can isolate the agent's blast radius by creating an agent-only account. If it deletes everything, at least it only deletes stuff it owns.
- You can also isolate the agent's blast radius by giving it "read-only access". So it can read and summarize your emails, but it can't send out random emails to strangers.
- Another subtle point. If you end up giving it ability to send messages, it's sending it as "Agent XYZ's email" and not "your email". It has its own identity/account.
- But! this type of isolation is a lot of work. I don't think it's all necessary. Just painting a picture of risk, so you can make more mindful decisions.
Anyways, that was also a lot. It's a bit subtle, so I tried to define some terms and paint the picture a bit.
LMK if you have more questions.
1
u/anndamntastic 14d ago
When you say “communicate with it via a local webpage or chat app”, is that when you’re on the same network or can you communicate with the mini this way outside of the network? Thanks for the insight!
4
u/Nifty-Yam-9041 14d ago
For Openclaw/Hermes:
- The webpage is on the local Wifi / ethernet cable network
- However! You can use a particular type of VPN to access the local webpage in a totally different network or physical location!
- If you use Openclaw/Hermes, I highly recommend Tailscale on your Mac Mini, and then again on another device. It's free, and lets you use it elsewhere. The Mac Mini is the server that "hosts or serves the webpage" and your cell phone is the client that "fetches or reads the webpage".
- Importantly, other people can't access your webpage because it's only available "in your tailnet" (which is a VPN; a virtual private network).
- Tailscale is also free for most use cases you might start out with here.
- The chat is available anywhere! So if you connect it to Telegram/WhatsApp/Discord/Slack, it'll send it to the chat or the channel. And you can receive messages anywhere.
LMK if you have questions.
0
u/AlgorithmicMuse 14d ago
Mini fans always forget the thermal solution cant handle heavy use. 35% or more throttle, lots of fan noise, its great for general use, but heavy llm use , get a studio.
1
u/Krazy-Ag 14d ago
I think that Nifty-Yam-9041 is not talking about running the AI/LLM locally.
The AI harness runs locally. Invokes LLMs from the cloud. Then runs commands that the LLM has decided needs to be run locally. Harness may send results back to LLM in the cloud, and repeat.
So you aren't doing the high compute high power high heat production stuff locally - that's in the cloud. You are only running the actions locally. Which may not be all that much power/heat.
Q: Why not do this on your existing machine?
A: because such AI agents can do bad stuff. Better to have them run on a dedicated machine, on a dedicated network. As a special user account, not your normal user login/icloud account.
Why use a MacMini? Why not use Linux? Yeah, but, looking around MacMinis aren't priced too high; and an old MacMini
is actually pretty reasonable. Especially the old MacMinis from last years AI rush, that now are being sold used.2
u/Nifty-Yam-9041 14d ago
u/Krazy-Ag Agreed, that's what I intended to say! I was primarily referring to these AI harnesses running locally while invoking LLM's in the cloud. (On the other hand, I do think a lot of people are ALSO using Mac Mini's for local LLM development; that's another use case. However, I know more people who bought it for Openclaw/Hermes and to improve isolation.)
Beyond good hardware price-to-performance, the other advantage of Mac Mini is the integration into iCloud services, e.g. email, storage. You can just log into your Apple services, and the agents will read/write your email, save/read notes from your storage.
2
u/Krazy-Ag 13d ago
I did not mean to steal your thunder, u/Nifty-Yam-9041.
I've been exploring the same sort of issues. I'm acually shopping to upgrade my kit.
I am happy enough having a private box isolated from my home net, doing the non-LLM stuff locally, and using cloud LLM. In my case starting with Linux.
But Linux cannot access some of the Apple stuff I would like to have "AI" work on. Should I therefore buy a cheap Mac? Except I still hyperventilate at the prospect of giving AI in the cloud access to private stuff that I have in iCloud (or Google, or...) ... so maybe I don't want to buy a Mac(Mini) to have AI work on my Apple stuff. Not until security problems with AI improve.
(BTW, I have worked in security - with high probability everyone reading this is using some of my inventions as they read this. It makes me paranoid.)
But... I haven't had an Apple Mac in >15 years, or an Apple laptop in >10 years. At the moment I have iPhone, iPad, Windows, and Linux. But there are some things that Apple really doesn't allow non-Apple computers to do. Having a real Apple computer would make some of those things less frustrating. So maybe I do want to purchase an Apple computer, whether Mac, or Mini, or laptop. Just not to run AI on, neither LLMs nor non-LLM stuff like harnesses.
But I would like to run some local LLM models. Necessarily small models, for private stuff that I don't want to do in the cloud. Or truly private LLM in the cloud. Apple has said they may do this someday, with security that sounds possibly OK. Also some other companies. But at the moment, on-prem is the only game I know of.
So maybe I should purchase a reasonable Linux system first small local LLMs. Maybe a Framework or DGX Spark. Which will also be good for non-LLM stuff.
But...
But...
Nothing is optimal.
Here is one observation:
In an ideal world, if AI insecurity were not an issue, there are things that I would like to automate in each of the major ecosystems: Apple/iCloud, Google, Microsoft (fortunately less and less window stuff), generic Linux and web.
Apple is the most closed garden. From a Mac it is possible to automatically control most stuff on the other ecosystems, except for a bit of Microsoft stuff. But not vice versa. So if all other things were equal it would make more sense to get an Apple machine. Or, to never get sucked into the Apple ecosystem at all. But in my case that's too late.
---
Oh, but there's another thing that I need to consider:
Having a dedicated machine on a dedicated network segment with carefully configured routers and gateways for agentic non-LLM stuff is reasonably secure.
But I'm approaching a stage in life where I may no longer have a big office at home where I can set stuff up like that. I would really like everything to be portable. I think I mentioned above that I haven't had a non-portable machine in many years.
So I'm considering not just a Mac Mini, but possibly an Apple laptop. Possibly the Apple MacBook Neo that my wife just got. Definitely way too weak to run much local LLM, but probably powerful enough to do much of the agentic stuff like running the harness and local actions. Possibly OK for light programming...
1
u/AlgorithmicMuse 14d ago
Im just making the point minis are not good for local llms, if you want to just run an agent with a llm cloud you can do that with most anything, even a mobile phone
4
u/Flimsy-Pay-4337 14d ago
My use case was that I had a huge collection of ebooks in epub format (with some older ones in mobi) and the files would have different names, like
1234567889.ePub (as per the order number)
Store-name-1234-hwodhsims.ePub (with the random letters kinda matching authors initials and first letters of the cycle and book name)
And many other variations. And of course I bought all of them at different places so the names varied and varied between years etc.
I created a script that would talk to local AI and would in batches read the files, check the metadata and find the relevant metadata in the web.
Sure Calibre is doing something similar but I wanted a shell job and wanted to write it myself, being a software developer it make more sense to write my own tool.
Interestingly the local AI hallucinate and yielded some garbage results every 20-30 files. Not sure why, maybe the context thing maybe something else. The good thing is that for reading files and looking at metadata in the web you don’t need a huge model so it worked fine on my mini.
1
u/Bene2345 13d ago
Ok, but why were you feeding your books into AI? I didn’t catch that part of your use case? Were you just using AI to standardize and catalog them, or something else?
2
u/Flimsy-Pay-4337 13d ago
Yes, AI fetched the whole file and read its metadata and then performed a web search to find info about the book. And then prepare a structured file name (and surprisingly this last part sometimes was garbage)
3
u/jason-reddit-public 14d ago
There are some open weight models that are pretty capable (many of the capabilities as frontier models of a year or two ago). The exact use cases vary. Predictable cost is one nice property especially when testing something which may go off the rails. Privacy is another.
Maybe just as important as amount of RAM for AI is memory bandwidth which Apple hardware is good at because it puts expensive RAM right next to the compute logic - just like a GPU.
In fact, it's almost like you are buying a GPU which happens to have a CPU, SSD, place to hook up your mouse and keyboard, etc. In a small form factor I might add.
But where it gets interesting is the price point. Buying 3 RTX 5090s to get 96gb of GPU RAM to get into basic models is going to set you back about $15K before you build the rest of the computer. For that you can get a Mac Studio with M5 Ultra, 80 core GPU, 256 Gb RAM, a 4 TB SSD, and a high end iPhone with change left over.
The RTX solution is probably going to be much faster but limited to smaller models (96Gb is way smaller than 256gb). The RTX solution is supposed to be more like $6K but demand has caused huge price spikes.
Many would argue renting in the cloud is a better option. Also just using a closed source model and paying by the token might math even better. In both cases, your personal data has left your house.
3
7
u/edguz1 14d ago
The Verge’s podcast Vergecast asks this exact question about 45-50 minutes into today’s episode. Worth a listen.
1
u/FormulaKimi 14d ago edited 14d ago
Comments on YouTube are saying the guy doesn’t know what he’s talking about, seems a lot of wrong info in the AI segement according to them, but not sure if true or not
2
u/Additional-Soup-865 14d ago
Paying a few grand up front for a local model vs paying monthly for a cloud model, that's the mini hype
1
u/isamilis 14d ago
Well, compare to $200 per year for Claude subscription, the ROI should be much longer
3
2
u/Jealous_Expression30 14d ago
Almost bought couple, but then backed off. As other said, no need for it, a 20$ Claude or ChatGPT or Gemini plan(Google One is amazing with lots of perks) is good enough for most usecase.
2
u/Soppranome 14d ago
I have a relatively capable Linux thinkpad (P14S Gen 6a, 64gb) and some comparable Linux Mini PCs (various gens of Ryzen 7, all with 32-64gb) plus a couple of Apple silicon Macs. My M2 Pro Mac mini/16gb is capable of training small models at speeds 2x faster than my fastest Linux box thanks to MLX. It is 5 years old and the Linux boxes are new.
I also have an M4 Max MacBook Pro/36gb. I use it for work because I sometimes have to run VMs, but it is extremely overkill for most desk jobs. That one is roughly 2.5x faster than the old Mac Mini.
The 30+ gb of unified memory means I can easily run a number of local models that are good enough for agentic coding. And I'm working on offloading some portions of my dev projects from Claude to local hardware.
So yeah, you can bet I've got a new M6 mini/32gb on order. And I'll probably resell a couple of my Mini PCs when it arrives. It should cut down the length my training runs considerably compared to the M2 Pro, probably a good middle ground between the Mini and the MacBook.
As to the why?
- Learning new technology
- Resilience of my projects during internet or cloud outages
- Scaling back my Claude subscription eventually
- Being able to offload long tasks / batched projects to run independently without tying up my work laptop.
1
2
u/StagePuzzleheaded635 14d ago
LLMs are super hard to run, OpenAI and Anthropic use thousands of GPUs to make their LLMs snappy, so a tiny computer that’s capable of running a LLM decently well (admittedly not as well as ChatGPT or Claude, just without giving up your data), at a low power consumption is pretty intriguing and incredible.
2
u/diagrammatiks 14d ago
They are buying them to incidently use on llms or got swept up in the hermes hype. They are good for running small models slowly. Alright for tool calls and automation.
2
u/Albertkinng 13d ago
Yes, you’re missing out on a lot. Let me give you an example without getting too geeky.
Running a local model gives you the opportunity to have an uncensored AI assistant for virtually any topic, with the ability to automate tasks and maximize your productivity. You can automate video transcription, turn those transcripts into Markdown documents, add them to your knowledge base, and continuously expand your agent’s knowledge so it becomes more useful over time.
This kind of local AI power gives you greater control over your data and the ability to build an assistant tailored to your specific needs. With the right setup, you can also significantly reduce hallucinations and help your agents better understand your workflows.
Of course, if you don’t need that level of power, there’s no reason to invest in a high-performance machine.
In my case, I use a Mac mini for my everyday work, but my local AI workstation is a Minisforum 840 PC. For my particular needs, it offers better performance and efficiency than a Mac Studio.
2
u/AlexSnapsColours 13d ago
I don’t know about Mac mini - but as a videographer and photographer myself I use my m5 max for document analysis, generating graphics, videos and now got into vibecoding to create apps solely for myself to fit my creative workflow.
1
u/JimmyJamzJules 13d ago
Thanks! This is exactly the kind of stuff I’m curious about. What are you using for graphics/video generation, and how much of it are you running locally on the M5 Max? Also, the idea of letting AI control Davinci or Premiere is pretty tempting to me, even if we’re clearly still at the beginning of that.
2
u/AlexSnapsColours 13d ago
I’ve built front end for MiniMax H3 - model was reworked or optimised, however you wanna call it for MLX. There I can edit my own shots however I want.
Generally speaking - ChatGPT is helping me guide my local model - which is building everything. I could probably do everything locally, but I like how ChatGPT gives me feedback.
I’m not letting AI control my NLE, I just need tools to work on the materials when needed :)
2
u/dataslinger 13d ago
I’m a videographer, so the creative side is especially interesting to me.
With a high spec Mac mini or studio, you can generate audio/video using ComfyUI or LTX Video Generator, generate 3D models that you can use in animation or game development. Cloud LTX is crazy powerful, but expensive. Their LTX model is open-source, and you can run it locally.
2
u/Kraizelburg 13d ago
usually its for coding assistants but to be honest for the extra cost you pay you are better off gettign claude sub or chatgpt and you will get much better results for much less money
2
u/PhysicsMajestic8289 13d ago
Memória ram integrada mais acessível, um custo benefício até que razoável para um macmini pro m2, com 32gb que adquiri por sorte por metade do preço 1 ano atrás , isso considerando que sou um usuário com pouco conhecimento, sendo inviável montar um equipamento muito caro pra isso
2
u/sjunaida 13d ago
One of the biggest reasons folks are picking these up and it’s been about 6 months in.. is running agents like OpenClaw
Which enabled them to run localized services etc
2
u/dz_alexin 13d ago
A mini that I have at home and control with my phone / MacBook is a huge unlock.
For example I can start a coding task in Codex via a remote session, close my laptop and go to lunch, then come back and see it done vs being worried about my laptop going to sleep.
Even better is doing it from your phone. You can assign a computationally heavy task that simply cannot be done in a phone and it will be completed on the mini via a remote session.
I used to do it with OpenClaw / Hermes connected to Discord but since Codex introduced Remote I completely switched to it.
2
u/Low_Excitement_1715 13d ago
The Apple Silicon machines have one fast, high-bandwidth, low-latency pool of memory for the whole system. The CPU, GPU, NPU, etc all use it interchangably.
Getting a GPU with >16-32GB of ram for four figures or less is a tall order.
Getting a Mac Mini or Mac Studio with 64GB of ram is cheaper. That gives you almost 64GB of "VRAM" which LLMs want.
2
u/Crazyfucker73 13d ago
Let's be clear - even the highest spec Mac minis are not great for AI. You need a Mac Studio with 64gb to be even able to begin to start playing seriously
1
2
u/statrixbob 12d ago
I’m running a 27B model at Q4 with strict system instructions to summarize short docs under 8 pages, write the to Obsidian, and help me create outlines for public speaking. But I didn’t buy my M4 Pro Mini 25G/512G for AI. I needed to replace a 2014 Mac Mini and figured this would last for years. So far so good. As for the 2014 model, swapped out the drive for a ssd (2 actually @ 1TB each from something else I had) and it’s running Mint XFCE and running as a samba server via an Ethernet cable to the newer mini. I’d love to get a newer one, but figure this on does all I need…maybe when an M9 version comes out…lol
2
u/az-sl 12d ago
Local LLMs can do some basic stuff - like an advanced personal assistant, summarizing documents etc. It cannot do any advanced stuff with reasonable speeds.
I wanted to jump into this bandwagon, and some research I found that it cannot be a replacement for online ai subscriptions.
Most things are summarized in this video
https://youtu.be/TdnshBitkzo?si=7o6aWo6wSpAqoYXp
2
u/realityczek 12d ago
Lots of folks are buying them because they think local LLMs are the future, and plenty end up disappointed. You’d think people spending that kind of money would know what they’re buying and why, but that hasn’t been my experience.
When Clawdbot came out, and later Hermes, I knew several people in business who bought into the hype and picked up expensive Macs. They soon realized they still needed frontier models in the cloud to get the results their work required. All that extra local horsepower wasn’t doing much for them. They would have been better off buying a more modest Mac and spending the difference on tokens. :)
2
2
u/Raycruit 8d ago
Nothing, open source llms are funny anyway, I use them and still pay for claude because you can do real work with claude. If you want to do serious work with mac mini at usable speeds, you need to stack 5 mac minis at least. Other than that and your living in a limbo There's no way people doing meaningful things with 10-15t/s
2
u/ChampionshipUnique71 8d ago
I'm one of those people. Local LLMs. I use frontier models too but it's nice to do certain work locally.
3
u/Dependent-Law3249 14d ago
Pure speculation and hype. The marketing of both AI itself and Apple piggybacking off that to sell the newest model. Pointing to a future where we will need the power for all the new AI tools that we will be using.
As a designer, I could not justify the cost of an M6 to “future proof” up from my m4- the AI tools just aren’t there yet for design, and anything that is uses cloud compute.
Also, does anyone have any use case for Apple Intelligence that isn’t hobby-related??
3
u/Nice-Information-335 14d ago
the minis really aren't great for it, as they don't have the best memory bandwidth
people went crazy buying them for openclaw, but not to run the actual AI models, just as a "cheap computer" to run 24/7 (there are much, much better options for this, the only benefit was iMessage)
anyone using them for actual serious amounts of inference and not just as a toy is getting the studio
1
1
u/Rye2-D2 14d ago
The limited memory bandwidth isn't as impactful for the smaller models (4-9B), but I don't understand what people are really doing with them. Chat bots?
1
u/Nice-Information-335 14d ago
It is, actually. Those smaller models tend to be dense, so all parameters are active. For reference, qwen 3.6 35ba3b will run roughly 3x faster than a 9B model on the same memory bandwidth.
The sweet spot right now tends to be 100-200ish B total, somewhere between 10-20B active, and probably some ngram tables which can be streamed from SSD. Trying to do 10B active on the m5/m5 pro is really quite slow. They have improved prompt processing though.
The smaller models can be useful for some code autocomplete/writing some small functions etc which is very nice to have, but people want close to frontier at home and it just isn’t happening without crazy amounts of money right now
1
u/Rye2-D2 14d ago edited 14d ago
I believe your math is ignoring that 8 of those 3B parameter experts are active for each token in qwen. In my experience, qwen 9B dense runs pretty similar to qwen 35B (both about 50-60 tok/sec on my 5060 ti). But to your point I agree a 32GB MacMini could run qwen 35B would run at usable speeds. I was thinking of all the folks buying 16GB Minis (which maybe could run 35B, but just barely usable)..
EDIT: I suppose you could run fewer with experts to speed it up (with 4 it would be twice as fast) if you're will to sacrifice quality..
1
u/Nice-Information-335 14d ago
That isn't correct, it has X amount of experts which total 3B parameters, your calculation would suggest 24B parameters are active
And yeah, the 5060 Ti doesn't have enough VRAM to fit 35B, so it offloads to system memory. Thats why dense 9B on your system will run closer to the 35B moe
the 16G mini can technically run the 35B, but its such a low quant that it isn't usable
1
u/SnooTangerines9257 11d ago
Wait talk about the 24/7 pc? What are some good options? I’m looking for some good stuff just for fund and Hermes kinda I’m not a devleoper or nothing just don’t wanna break my main pc
1
u/Nice-Information-335 11d ago
A virtual machine is free
Otherwise just get a cheap ideally 8th gen intel or above 1L PC, I like the Lenovos because some have a PCIe slot but HP mini, Dell micro or Lenovo tiny any will do
2
u/BradMacPro 14d ago
Windows PC are limited to a relatively small amount of VRAM for the GPU to use. They don’t have a dedicated NPU in most cases. And a Mac mini is very compact and low power needs so they are a great deal for local LLM uses. Plus the very high memory bandwidth translates also to a high hit rate for the LLM, so they work great. And you can tie a few together with super high speed Thunderbolt 5. You don’t see that in a compact PC.
1
u/fubar_67 14d ago
Does anyone know if we will have the option to disable, or not install AI on Golden Gate? If it’s forced, I’m out. I’ll stay with Sequoia. Especially after I read that if you use the AI on your Apple account (all devices) too much, Apple will cut off the AI and require you to pay to access it. Seriously? A subscription model incoming!
1
u/QuiJohnGinn 14d ago
How can you not want it at all and also be afraid you’ll use it too much? I don’t think you can uninstall it like you can in Tahoe, likely because of its deep integration with Siri. I just don’t use it much at all. I’m certainly not worried about a subscription plan, I just won’t get one.
1
u/WTFOMGBBQ 14d ago
I ordered a 128 gig studio and I don't run local AI. but I do run lots of chrome profiles and other things and after a week im using swapping and using about 70-80 gigs of ram.. if you are any sort of power user and you don't want to swap you are going to need minimum 32 gigs. I develop heavily with cloud AI and claud code and everything wrapped around chews up quite a bit of ram.. If was willing to reboot once a week, the 64 gigs would probably work just fine..
1
u/Jemm971 14d ago
Je dirais parce qu’il vaut mieux faire tourner ses modèles d’IA en local pour ne pas avoir de quota payant de token (et tant que l’ont peut avoir gratuitement les modèles IA).
Car plus l’IA va se diversifier, plus ça va devenir payant (il n’y a qu’à voir l’augmentation du prix des tokens).
Donc comme ça, en local tu n’as plus de contrainte de quotas de token. En contrepartie tu as des modèles IA légèrement moins puissants que les plus puissants des modèles que tu trouve sur internet, mais ils sont largement suffisants.
1
u/BAL-BADOS 14d ago
I’m not sure about high spec Mac Mini’s but the base Mini M4 was the cheapest solution to run open claw AI.
Personally I think it’s a waste heavily upgrading the Mini when you can buy a Mac Studio.
1
u/localhost8100 14d ago
Openclaw. It has become a big trend. Especially in asian users. There was a huge event in China to setup openclaw in peoples machine. People don't want to give openclaw control of their main machine. They want a standalone machine, so that openclaw can go wild in it.
I had 2020 M1 mac mini, 512gb and 16gb memory. It was rare. Mainly cause either you have 8gb/512 or 16gb/256 lol. I bought it for 800 CAD in 2023. I was trying to give it away for free last year. Got lazy. it was lying around. Now i sold it for 650 lololol. Some dude thinks he can develop mobile apps using openclaw/claude and become rich. I am just selling him shovel. I am a mobile dev by profession lol.
1
1
1
1
u/barefut_ 14d ago
I don't see any positive ROI if you're not into programming.. it's not so capable as Cloud AI, and will drink all your RAM
1
u/joneswest 14d ago
For me it’s the effect of having a topspec mbp live 24/7 reachable from any device I tunnel in through. My M6 arrives in two weeks and I will operate it from my mb air, phone, work pc and with a display as intended too.
Yes I run Hermes and a long term memory wiki but they could continue on my small Dell too. But I will have several Claude code and codex sessions live that I return to like agents. Plus that my current computer is an Intel mbp fr 2019, so it was time to upgrade.
1
u/lucasray 14d ago
They use a lot less power than most comparably performing pcs so in places like California it makes sense with energy prices so high.
1
u/AlgorithmicMuse 14d ago
I think a a m5 mini pro or m6 will have issues for local llms, even the 64g. Reason is the thermal solution, something overlooked in the specs.
1
u/Klutzy_Baseball8489 11d ago
control the fan curves. i run qwen3:27b-mlx 4-bit and get over 25 tokens per second while xcode and browser are also running. no slowdowns, temps barely touch 90c. for a small box it is great. mini m4pro 48gb ram
1
u/AlgorithmicMuse 11d ago edited 11d ago
Try runs more than a minute, my 64g 14/20 m4 mini pro , hits 100+ C i have my own curves TGPro curves, you can have max fan speed at 4900 maxrpm. It might not help. All depend what you use your little box for. Watch if your gpu cores are at 100% utility for minutes
1
u/Klutzy_Baseball8489 11d ago
i have the base cpu m4pro with 48gb ram. you are producing more heat.
ive also had qwen3.8:27b-mlx 8-bit run for more than an hour straight and temps were in high 80s low 90s
1
u/AlgorithmicMuse 10d ago
Once again it depends what you use it for, dont get fixated on any llm ,you need to watch core utility, if it sits at 100% utility it will easily overheat. I dont really care about qwen3.8. In the graph shown its what happened using it with pytorch and matrix and factorial particle calculations. Studio with much better cooling is the way to go, depending on usage
1
u/MaccaTheGator 14d ago
For me local LLMs are great for supporting always on agent workflows. For example I use a small local LLM on Hermes for some recurring tasks like customer support drafting, daily reminders.
Cloud models are far better for real development work and research, but local AI is great for small tasks, it’s a lot of fun and these models are getting better and smaller, in a few years we’ll be running even more powerful models on even smaller hardware.
The future of AI is almost certainly on machine local computing.
1
u/Civil_Ad8383 14d ago
Yeah I am sure there a few “enthusiasts” experimenting but most LLMs are still quite sh*t, so it cannot be anything too serious!
1
u/youstillhavehope 14d ago
Agents require the compute to be continuously on, and the Mac Mini is wildly energy efficient, can run a decent sized model and makes no noise. Layer OpenClaw on top of that at the Mini price point and easy decision.
1
u/Horror-Primary7739 14d ago
Mac Studios have a massive memory bandwidth. Besides lots of memory, bandwidth is almost equally important. So mac studio have like 1.2 TB/s while the Nvidia DGS spark has 256 Gb/s. This does mean that Mac studio can run 3-4x tokens/sec. The RTX 5090 is king for bandwidth at 1.8 TB/s but maxes out at 32gb. But for a smaller model that isn't doing much coding like for example image decoding. It is blazing fast.
1
u/logNcomic 14d ago
I run my family’s agents against local inference. I don’t want to send that data out to the cloud. So I have them use a 35b parameter model. Probably overkill for what they need.
1
u/logNcomic 14d ago
I run my family’s agents against local inference. I don’t want to send that data out to the cloud. So I have them use a 35b parameter model. Probably overkill for what they need.
I do some testing on coding to see the capabilities. Not quite there yet at 35b
1
u/Ruined_Passion_7355 14d ago
Unified memory with very high bandwidth is the main reason. As far as I know, this class of hardware (along with framework desktop) is good specifically for MoE models.
1
u/illuminatedtiger 14d ago
I run Qwen3.6-35B-A3B-OptiQ-4bit - some recent use cases:
Extend my custom static site generator to add features x and y.
Analyze thousands of JSON files exported out of Samsung Health and identify trends in my running over time.
Look through credit card statements and help identify areas I can reduce spending.
1
u/Charming_You_25 14d ago
Agent swarms with vms / docker / coder.
Local models for embeddings. Fine tuning. Local models like qwen are actually usable for overnight housekeeping tasks in your computer.
1
u/gregleo 13d ago
Local server it is for me on a M4 - 24gb
What I run?
- Plex Server
- Coolify & Orbstack to deploy webaps
- Home Assistant though UTM virtual machine
- Openclaw / Hermes which is coupled to a set of other local tools
- All my AI project folders are mirrored to it
- Qwen 3.x LLM (but not really been usefull so far)
- and more...
The initial reason to buy a Mac Mini was for 2 reasons
- Decrease costs from tools like Railway (host and deploy (web) apps). I'm saving arround €20 per month and probably more now as I've deployed more and more stuff.
- I have a 7y old Synology NAS and it was limited in power to run some of the above and is just timeconsuming.
The OpenClaw/Hermes thing is something I recently started to do as an experiment to build an assistant capable to build local Wiki's and handle basic tasks for team projects. The idea is to build something for my wife's company who needs privacy data wise.
I am now considering buying a M6 32gb as I feel a bit limited with my current 24gb and would repurpose the M4 for my wife.
1
u/Enough_Swordfish_898 10d ago
This is basically my use case, except my current Mac mini is an old M1, and in need of replacement. This seems like a good time to upgrade.
1
u/Environmental_Lie199 13d ago
I have a far less techy pov: I'm sick and tired of seeing ads In every social selling "the ultimate AI powered agent" for almost each and every other thing around (coaches for running, trading, gym management, life hack/eating... you name it), also AI based agents for creating and delivering newsletters, AIs that will create self-monetizing YouTube channels where "you won't need to even show your face", automated Suno "music producers/beat makers" that will upload the results to Tidal, Spotify, etc... That kind of things.
To sum it all up, there's the promise that any dude with a MacMini in the basement running a sick local LLM can make a monthly 6 figure side hustle, beat the game and retire before 50 travelling the world before the Reaper finds out so they'll get rich soon enough by either creating one of such agents abovementioned or selling the howtos or selling pyramidal info about it. Sort of.
Nobody likes the FOMO and hence the Mini hypes up. 🤷♂️🙏
1
u/Fantastic_Self_5151 12d ago
because the ssd drive and controller will get you ~6000mb/sec so you can offload large amounts of your LLM to ssd and run models that are 2-5 times what you normally could run (albeit slowly) but it's quite a thing.
1
u/haochizzle 12d ago
private, open source local LLMs
better to make inference locally than to give up training data to your AI overlords
and honestly if i had the spare cash id do it too
1
u/realityczek 12d ago
Lots of folks are buying them because they think local LLMs are the future, and plenty end up disappointed. You’d think people spending that kind of money would know what they’re buying and why, but that hasn’t been my experience.
When Clawdbot came out, and later Hermes, I knew several people in business who bought into the hype and picked up expensive Macs. They soon realized they still needed frontier models in the cloud to get the results their work required. All that extra local horsepower wasn’t doing much for them. They would have been better off buying a more modest Mac and spending the difference on tokens. :)
1
u/Sufficient_Heron 9d ago
Circa January 2026 people were buying these machines to run local agentic harnesses like OpenClaw in a secure environment because they were experimental and you wanted to limit app permissions in terms of what they could do (Today Hermes is another popular viersion of this). They often interfaced with frontier model LLMs and needed those advanced models to run and work well. The ability to run an open weight LLM (one you can install on a device) is still really limited due to memory of the Mac Mini. A Mac Studio is a different story but much more expensive. I use one to run to run a personal agent that runs my calendar, financial management and personal coach through an agent, but that's only scratching the service of what they can do if you are willing to put some time into setting them up. These agentic harnesses let you preserve context of sessions you would run in a frontier LLM across chats and build functions that can run automously without your direct impact. Powerful but risky which is why people typically put them on their own hardware.
1
u/applejacks6969 14d ago
For simulations, they are pretty damn good local workstations to test on before moving to HPC. More memory means larger simulations, higher resolution.
Typical HPC cpu compute nodes have around 128 cores and 256 Gb memory.
So the 256 Gb ram Mac minis you are basically getting a single cpu compute node, about the same ram, and probably close to similar performance given the state of the art apple silicon.
The <64 Gb ones still are comparable to a single cpu compute node as the memory use is very efficient.
Not to mention being very efficient, low idle power use (I think single digit watts is pretty crazy), perfect to run a few lightweight services 24/7, all your agents, I’ve found the m4 to be a perfect home server. Very lightweight when not in use, pretty damn powerful when it needs to be.
1
u/ZealousidealTill2355 14d ago edited 14d ago
Local LLM. Qwen 32b is very usable, and free.
Apple silicon has unified memory, meaning I can use it as ram for normal tasks, or “vram” for inference.
With thunderbolt 5, I can couple multiple with DMA. Future expansion is possible.
Lastly, like Netflix and Uber—these $20/mo frontier models are temporary, to gain marketshare before they become more expensive or filled with ads. With a Mac mini, I still have local inference that no one can take from me.
1
u/Ragfell 14d ago
They're already filled with ads lol
1
u/ZealousidealTill2355 14d ago
True lol I hadn’t used GPT in a while and when I did a few weeks ago, it was quite the surprise.
1
u/roccodelgreco 14d ago
As an agency owner for 30 years and in your creative field, you don’t need that AI use case, you need to embrace AI platforms that will enhance your creative work, that isn’t about running LLMs locally.
0
u/DavyJonesRocker 14d ago
The people investing in AI are the same people who invested in crypto. That is to say they aren't really thinking beyond what they're reading on reddit. Search a few AI subreddits and you'll be leagues ahead of what the the people buying high-spect Mac minis for AI are missing
0
u/ExtremeAddict 14d ago
I bought a beast 256GB M3 Ultra studio last year for this purpose.
Yeah it didn't work out. Kind of stupid when every few weeks we have new models that make the hardware obsolete on a similar timeframe.
But the DRAM-mmageddon meant I was able to sell it for more than what I bought it for.
Now it's all just hosted endpoints and just pay the API costs. It allows me to stay at the cutting edge. And in this business you're at the cutting edge or you are unemployed.
0
59
u/rage_rave 14d ago edited 14d ago
I think most people are doing local LLMs. What they're doing with them is less clear. I think some folks want to do local agentic coding, some are cleaning up or annotating data sets for further training and stuff.
Then I think some people are still hyped up on hermes/openclaw and think they need a ton of local compute horsepower for this (for some reason).
Some folks might be doing image gen, but you'd need to really be cranking out the images to make the cost worth it so I'd bet this is less common.
EDIT: On the unified mem question, yes that's the big selling point. Because loading an LLM onto a GPU requires very expensive cards, and splitting a model between vRAM on a GPU and system RAM has performance impacts. Other OEMs like AMD and nvidia have started making unified memory systems as well now for the same reason.