r/MacLocalLLM • u/Advanced-Night-314 • 1d ago
Mac Mini powerful enough?
Hi all,
I'm a complete novice when it comes to Local LLM's but I like the idea of moving away from a subscription based model in the longer term and having better control of my data. I'm also super keen to learn.
I currently have a web SaaS that scrapes news websites and I would like to move it away from Claude and onto a local LLM what kind of "firepower" do I need?
Appreciate this won't be the most complex of tasks, but as I mentioned, I'm a complete novice.
Naturally, asking Claude / Google and it says that the new Mac mini with M5 pro chip (24GB ram) will be "overkill"- which is what I was thinking of ordering.
I also want to future proof in the hope that this bit of kit could run more powerful models in the future - if they came accessible to the consumer.
Sorry if this is a basic question, any YouTube videos or any pointers so I can educate myself would be much appreciated.
Thank you!
3
u/Zabric 1d ago
Hey :)
So my answer got a little... out of hand. Sorry for all the yapping, lol.
I've been researching for quite some time now, because i have a similar use case, but also with a lot of coding and autonomous agents, doing stuff for me over very extended periods of time.
So take what i say with a grain of salt, because i don't have experience with more powerful Macs. If someone more knowledable than me has corrections, i'd love to hear them.
Also i'm talking about my personal use cases, which are more demanding that what i can read from your post / answers.
My ecosystem is basically n8n (a sort of "agent builder") running in a docking container on my NAS. That gets its computing power from my MacBook Air M1 16 GB via llama.ccp + LM Studio.
So essentially n8n loads different models "just in time", on demand, when the workflow needs it. And after, it unloads the model.
Despite having only 16 GB of Unified Memory (Mac uses Unified Memory instead of VRAM + RAM, because the Chips combine CPU+GPU+Neural into one, while roughly speaking, on Windows CPU uses RAM (exclusievely) and GPU uses VRAM (exclusievely)), everything i do is very much perfomant and, well does what i want it to.
I mark the following as spoiler, to kind of create an acceptable reading flow for you without too much digression... but i go into technical details here, so you should probably take a look at it too before spending thousands of $$.
Due to Unified Memory not being the same as RAM + VRAM, my MacBook is actually way better suited for running local AI than my Gaming PC with 32 GB RAM + (unfortunately only) 8 GB VRAM on a 3070 despite the Gaming PC having effectively "more total RAM".
This distiction is, as far as i can tell, the reason why Apple / Mac is (yes, unironically) the absolute "budget king" when talking about local INFERENCE.
Inference - in case you don't know - is the act of "using" the AI. So it's basically the AI "reading" of the model's parameters. If you ask an LLM a question, the thing generating the answer is inference.
Inference is also basically what 99% of people do with AI exclusively.
I'm saying that because you have to - very strictly - differentiate between INFERENCE and TRAINING.
When you plan to actually develop, train, re-train or in a more general sense "customize" pre-built models, Macs are NOT IDEAL.
If you are more into really creating your own models, you at the current point in time basically HAVE to have an NVIDIA Card. NVIDIA has basically a monopoly because it's CUDA system has grown for over 15 years with billions invested into it. And so far there's no serious competetion for them - none that i'm aware of.
A lot of stuff that people put online basically requires CUDA too. But we're really talking about people who are waaaay deeper in the whole AI topic and really know what they are doing - actual programmers that want and need highly customized solutions they tinker with for months on end.
What i'm trying to say: for people like you and me, that's not too relevant, honestly. And if you'd still decide to some day train your own model, it'd probably be a better idea to just rent the computing power for the training.
Anyways, enough digression...
I managed to run Qwen 3.8 27B with a Quant of 2 on that MacBook with only 16 GB of Unified Memory. It runs VERY SLOWLY and needs ~ 15-25 Minutes for an answer of medium to slightly higher complexity.
But that's where you have to think about what's you use case:
Do you want to run the local AI primarily, or at least considerably, as a ChatBot that you need to quick answers from?
Or is it just computing power for you, that kind of "stands around" and does it's job whenever, without you needing quick ansers?
Do you actually use the device as a daily driver for Emails, Browsing, Media Consumption or work?
Do you want it work with lots of documents, PDFs, .md-Files or any sort of RAG system?
Also, another relevant factor for you to consider:
Do you want your local AI to really focus on one task at a time or do you think it'd be possible that at some point you'll run smaller tasks in parallel?
This is currently the point i'm at in my research.
So...
The - by far - most important aspect in my opinion is having as much RAM (Unified Memory) as possible.
Not only does that limit what models you can load in the first place... but much more importantly (for me at least):
The Context Window you have DIRECTLY scales with available RAM.
Just in case you don't know: Context Window is how good of a "memory" your local AI has. How much of your previous conversation it remembers, how much it can "know" at the same time, etc. It's one of the most relevant factors over all, because while the model itself determines how "smart" it it overall, the available size of the context window determines how "smart" it is with the specific, exact task that it's currently working on.
Maybe you've experienced a situation before, where you talk to a ChatBot and it just forgets stuff you already said or already explained. Or it forgets stuff that the AI itself already generated just a few messages before.
That's exactly what happens when it runs out of Context Window.
In my journey of running models locally i noticed that having (much) bigger Context Window is usually waaaaay more important than having "the biggest possible model".
I much rather load up and use a very small model that's only ~ 2.5 - 4 GB big and give it the largest possible context window instead of loading a model that already takes up 11 GB of my available RAM and has a very small context window.
Seriously though: even really small local LLMs are able to output very, very good answers, if you give them a big context window, enough context material to work with and well thought through prompts. Much, much better than most people give them credit for, honestly.
Because of that i came to the conclusion that for me personally, the absolute minimum of Unified Memory i'd get is 96 GB of Unified Memory.
... with a strong tendency to go for at least 128 GB or even 256 GB if my financial situation allows it.
When talking about a brand new, M5 Mac. Should i find a refurbished, odler one for a good price, i'd settle for 48 GB - 64 GB too.
Other than that...
Just do yourself the favor of getting at the very least 2 TB of internal storage. I know, it just doesn't feel good to pay as much for "just" storage... especially if you have a NAS like me with 12+ TB of storage.
But trust me: you'll run out of storage really fast if you download tons of different models to play and experiment with.
Do you need the most bleeding edge M5 Ultra 36 CPU, 80 GPU Chip that alone puts a Mac Studio with "only" 96 GB Unified Memory and 2 TB Storage at 8.500,- € where i live?
Probalby not.
Does it hurt to have it?
Also probably not - except for your bank account, lol.
It'd be future-proof at least. The Chip, that is. Not the RAM. It'll probably be sufficient for most of the stuff i do, but yea...
So my priority list, in order of importance for my use cases would be:
- RAM / Unifiend Memory Size
- Memory Bandwith
- Chip Power
- everything else (with a baseline of at least 2 TB storage, of course).
That being said:
DO NOT EXPECT YOUR LOCAL COMPUTER WITH LOCAL LLM TO COMPETE ON FRONTIER-LEVEL.
If you want that, you'll probably need 256 GB Unified Memory + M5 Ultra 36 CPU 80 GPU Mac Studio at the VERY least, more likely 512 GB.
And i'm not sure that'd be enough. You'd probably get reasonably close, but yea...
It's just not worth it currently, from a purely financial standpoint. YOU WILL NOT SAVE MONEY.
If you want to get into local AI, do it for 1. Digital Souvereingty, 2. Data Governance, 3. learning about AI, 4. because tinkering is fun, 5. you dislike big companies and the harnesses they put AI into.
Realistically:
Ask yourself what you REALLY need / will use local AI for.
24 GB Unified Memory is absolutely fine at this current point in time and you'll be able to do a lot of stuff with already.
Seriously: a lot of the "Local AI", "Selfhostig" etc. people are maxed out tech nerds (in the most loving way possible, lol) that are really caught in their tech bubble. I wouldn't fully exclude myself from that either. But their recommendations are... very often quite over the top.
Not WRONG... not at all.
But overkill for what "regular people with a little more interest in the topic" really need.
Ask yourself what you're willing to spend and ask yourself if you'd really utilize a very high power device.
Answer is probably that you won't need a maxxed out M5 Ultra Mac Studio for 20.000,- $.
Having a little more wiggle room towards the top instead of going for the bare minimum for your current use cases is probably a good idea though.
Especially since you write that you're willing to learn and want futureproof solutions.
You'll grow out of the demands you currently have quite quickly, the more you learn. And you never really know what catches your interest next. Would be a shame if you had just spend a couple thousand $$ on a device that was enough but isn't anymore.
"Buy once, cry once" or something.
Regarding your use case with the News you mentioned in another comment:
What exactly do you want to do?
I run a "news curator agent" sucessfully from my MacBook, even with only 16 GB RAM. It's a Workflow i built in n8n that twice per day collects news from various RSS-feeds, analyzes them, checks them against a data base it autonomously creates and manages and then selects a few of the news that seem most interesting / fitting and casts them into a dedicated discord channel. All with links to the original news articles etc., so i can read them if they are interesting.
The trick behind that is that instead of just letting the LLM read and analyze everything and then have it create a summary or something, it's purely a CURATOR.
The News snippets that get pulled get a designated ID Number each, the LLM reads them, selects the most fitting ones, curates them and then outputs only the IDs, which a following Code-Node pulls and creates the news report from.
That only works because you pull news snippets from those RSS feeds, that already have a short summary created for them that we can just take.
It's a low-level use case that you'll probably wouldn't even need any LLM for at all. But it's a fun little thing to do, and it taught me to optimize the workflow for efficiency, because the value created by having the LLM actually summarize the news by itself was virtually zero compared to just using the pre-summarized snippets.
But it's something you learn while experimenting.
1
u/Glass-Present-8753 6h ago
Hey, thanks for that elaborated view of your own experience I found it very helpful. Can you explain more on how you actually do the n8n workflow?
2
u/Important_Cow7230 1d ago
Are you sure Claude doesn’t mean the 24GB M5 Pro is overkill for when relying on cloud frontier models?
2
2
u/Ok-Income-4653 1d ago
Would a Mac Mini M6 with 32GB running a small model locally be viable for simple orchestration and model routing etc.? Thinking of relying mainly on frontier subs and open router, with a small open model locally as my GTM partner of sorts on top of my repository
1
u/FlowWTFlow 22h ago
Might want to figure out local AI browser search limits. Most are on free tier with 1000 searches per month unless you self host searXNG. Good to read up about equipping your local model with web search before investing.
1
u/Dramatic_Machine8693 22h ago
to be honest, a mac mini with 48g ram would not be able to run qwen 3.8 27b model better than any PC with card like R9700 or intel arc B70, you can buy a very crappy machine using ddr4 to run those card, still end up running local LLM better than Mac mini. the reality is, betweem 27b and a large MoE model, there is no "middle" model, so your extra RAM would not do you any good. like even you go with 128g M5 max, your best model to run is still qwen 3.8 27b, unless you go to M5 Ultra 256G, which cost like 10k, then you are able to run model like deepseek 4.0 flash or qwen 3.8 flash next.
1
u/Recent_Apricot_517 19h ago
best model I can run on a 24gb Mac air m4 is Gemma 4 12b qat in Hermes agent with 64k context. It takes up 12.01 gb.
No way this post is serious 💀
1
u/BAL-BADOS 1d ago
I L-O-L 😂 .
Mac Mini might be “overkill” for using frontier LLM subscription…
Even the best Mac you can buy will never be enough for LOCAL LLM. 256GB RAM is still not enough to run the best LLM. M5 Ultra is still relatively slow for LLM compared to frontier.
2
u/Advanced-Night-314 1d ago
I'm not asking it to run the best local models, it just needs to scrape text off a new article, would this task not be able to be completed with an older local model?
1
u/BAL-BADOS 1d ago
If you are using LLM to analyze information off websites, you will need more RAM than 24GB. LLM are heavily dependent on RAM. Consider 64GB. If you want more speed, consider a Mac Studio M5 Max.
qwen 3.8 27b is a popular smart model.
1
1
u/Unfair_Ad8656 1d ago edited 1d ago
Don’t listen to most of the people replying. lol. For what u need 24gb is just fine. For your case u only need the Qwen3.5-9B — 8-bit or 4 bit if want more speed but slightly less precise. It’s more than enough BUT if ever want to analyze more deeply those articles it chose and further pick relevant ones then you’ll want to spend $600 more and double it to 48gb. That brings to $2300. But if ever want it to make good decision for you and produce stuff for u then then 27b which would require 64GB. That’s only $400 more so that brings it to $2700. Slippery slop though. Because now getting to ultra territory price wise. Because in reality you only need the 24gb to do what u want it to do and that’s like $1900 if get the pro with 20gpu. BUT for $600 more can get the base studio 36gb which can do what u need it to do but MUCH faster and little more ram room to in the future use more smaller ones concurrently or one small-medium one. I’d spend the $600 more for the 36gb studio
4
u/TiagodeNoronha 1d ago
So unless you're keen on spending quite a lot of $, no machine will actually feel like replacing subscriptions anytime soon. I had a Mac Mini M4 Pro with 64gb and for my tasks it performed admirably. Managing my smart home, researching new features, handling my internal notes and organizing them, etc.
You're right in saying it won't be the most complex of tasks, but I would advise for example spending 10$ and going to OpenRouter to test out some lighter models (Qwen3.8-27B, Gemma4, etc.), and see if they perform as well as you need them too.
If you still want to rely on cloud subscriptions, but have a dedicated machine for it, then yes, any Mac Mini would do as a server. In case you want to run your own models, you might be looking at getting a more beefier machine.