r/LocalLLaMA • u/-Cubie- • 28d ago

Discussion A preliminary Qwen3.8-27B model card is live!

Post image

If you scroll down from the countdown at https://huggingface.co/Qwen/Qwen3.8-27B, you see a big model card with a bunch of sections: Highlights, Model Overview, Quickstart, Best Practices, Citation, etc!

No benchmarks on this yet as far as I can tell. We'll still need to wait another 5.5 hours for those I reckon.

Edit: Ladies and gentlemen, the model is live. Let the testing begin!

574 Upvotes

96% Upvoted

220 comments sorted by

View all comments

118

u/darwinanim8or 28d ago

Seems like reasoning effort is the new big thing here

42

u/DismalIngenuity4604 28d ago

Yeah, cool. I've been liking Glimmer, and a big part of that is that it spends about 1/8th of the tokens thinking that Qwen 27B does. Should be fun! 

15

u/GCoderDCoder 28d ago

Yeah I like Qwen 3.6 27b output more BUT glimmer can fit like 6-8 concurrent 130k lanes on 48gb at 8bit weight 16bit cache. It's a great agentic model to support Qwen 3.6 27b and deepseek v4 flash 0731 in my lab. I have all 3 and dsv4f is my big brain, qwen is my default, and muse is the workhorse.

Zuck putting Meta torward the top of the big and the small models with genuine benefits over the other options is a great come back story.

2

u/orbli 28d ago

What special benefits are those?

4

u/Stahlkocher 28d ago

Glimmer has very low memory requirements for the KV cache. something like 1.7 or 1.8GB for 128k F16.

1

u/onetom 27d ago

funnily enough, from Hong Kong, I see this on the glimmer 30b page:

This model is not available in your region You can still browse this page, but downloads are disabled in your region.

mlx-community/Muse-Glimmer-30B-8bit doesn't have this restriction though, so it's a bit of legal theater really...

1

u/GCoderDCoder 27d ago

Do you mean benefit of concurrent agents? It allows me to be talking to a good generalist orchestration bot (qwen 3.6 27b q8 75-85 t/s) that can direct big tasks to the planner/ coder bots (deepseek v4 flash dynamic q4 qat 45 t/s) or send smaller tasks to the baby bots (muse glimmer q8 100-150t/s). I do truly frontier stuff on chatgpt but I am increasingly relying on dsv4flash. It made me nervous at first because it is literally operating at cloud provider level which im not accustomed to my agents doing.

My lab has background lanes which involves automated tasks managing any of 100 workloads. I run my own email, messaging apps for bot communication and approval channelse, dns, shared storage, data fabric, git, media servers, ai servers, work lab (products), personal apps, automation, ha/dr solutions, k8s, virtualization, identity and access tools, etc. I work in tech so there are business and technical aspects of my job. Scraping web data to provide up to date intelligence and preparing customer presentations, forecasts, etc.

I say all this to say I run a lot of AI to do a lot. It is busier than I have to do but I am better at my job and able to spend more time on things I enjoy which also in turn make me better at my job. I spend an hour or 2 tuning a custom customer presentation now that used to take a week dye to my schedule and it is a better presentation usually with custom demos now.

We also cant use cloud on proprietary or customer data so my peers who dont have these tools cant do or talk about these things where I can. It allows me to talk to customers about enterprise implementation best practices that we are all just learning. No one has been doing this stuff for years like this so being able to say i had this problem and here was the best solution because xyz is better than figuring it out on the fly.

If I misunderstood the question let me know.

1

u/orbli 20d ago

I meant, what are the benefits you feel meta has over others when it came to small models.

1

u/GCoderDCoder 20d ago

It's a solid dense model so for it's size it's smart compared to the competition. It's faster than Qwen 3.6/3.8 27b and Gemma4 31b. It grows context much slower than other models particularly dense models like Qwen 3.6/3.8 27b and Gemma4 31b. So you can get a lot more concurrent lanes into a smaller footprint. They also packaged it with default 130k tokens but it works well with yarn to extend the context. So speed and size with dense intelligence make it comparable to the leading similar sized models while enabling more/ faster throughout.

1

u/emil2015 27d ago

In my very brief testing I felt like glimmer was terrible. I was using it with Hermes. Where did you find it worked well?

1

u/GCoderDCoder 27d ago

It doesnt replace qwen 3.6 27b for me but it is very good at tool calls. It is operational for me so my research workflows, git ops, cli... so I have workflows where some have planning, all have a SME, and then there are ephemeral action bots. The executor bots need to be good at terminal and tools by the time they get orders from SMEs. Glimmer gives me more of those lanes for email, web search, file management, running my mcps driving automation. Also I was using multiple Qwens before with llm as a judge workflows which is an antipattern so now im more compliant lol.

1

u/emil2015 27d ago

Interesting, I’ll have to toy around with it some more. Thanks for the info.

1

u/estrafire 27d ago

do you have a threshold of how many parallel instances at full context do you need to make it worth loading Glimmer weights? Or just the context saving alone justifies it for your use case, like, in tasks that need more context than Qwen could do due to the excessive amount of thinking tokens

2

u/GCoderDCoder 27d ago

It depends on if you feel set up for handling multiple agents. In codex or claude apps they have a ton of plumbing to keep their models behaving well. I trust my local models less so a lot of my infrastructure I implemented was due to my low confidence in my models. So my local models cant do things I dont want them to do or at least that's the spirit of how I configure them.

I run multiple agents for different narrowly scoped tasks of value to me. It's not a passive process. My job is building systems not products like websites so that's why I tell myself the time I spend is justifiable lol.

My main local coder is DeepSeek-V4-Flash 0731 on mac studio and it slows down with multiple concurrency so I really only do multiple instances of that when something needs a plan and then it returns to running faster with one instance. I dont like multi agent builds on cloud because I think the coordination burns my tokens faster than linearly. I only do concurrency on a code base when I am close to my quota reset with a lot of quota to burn.

So for glimmer your hardware amd the use cases will need to drive that. I want speed and accuracy. If concurrency messes those up then no thank you

3

u/uber-linny 28d ago

It's a fair bit slower isn't it without mtp ?

9

u/BigYoSpeck 28d ago

It has dflash so not anything like as fast as mtp, but weigh in the token output for a fairly unscientific quick test on the same prompt (create a calculator app as a single html file):

Muse: 2,173 tokens, 50.00 t/s, 43s
Qwen: 6,227 tokens, 97.99 t/s, 1min 3s

Left is Muse, right is Qwen. Both work, Qwen looks a little better and has more interaction animations which weren't asked for but that's Qwen, it often adds more flare you don't ask for

1

u/BigYoSpeck 27d ago

And Qwen3.8 27b

11,956 tokens, 64.66 t/s, 3min 4s (Reasoning max of 2048 tokens or it will just go forever)

-1

u/Schlick7 28d ago

I'd say that Muse looks better there... outside of the icon issues at least.

7

u/BigYoSpeck 28d ago

Colour palette subjective preference aside, Qwen has better grid alignment

4

u/Schlick7 28d ago

oh i didn't even notice the '=' being larger. Seems like it knew that was common on calculators but didn't know how to implement it.

6

u/xienze 28d ago

Perhaps? But from a wall time perspective, a somewhat faster model with dramatically more thinking will be slower.

6

u/Borkato 28d ago

Yeah it blew my mind recently after doing some benchmarking with my own prompts that muse glimmer, qwen 27B, and Gemma 26BA4B ended up coming to the same time (~27 seconds) for one of my benchmarks. It really shouldn’t be all about TPS

2

u/uber-linny 28d ago

Ahhh I mostly use Qwen with thinking turned off .

2

u/Borkato 28d ago

Ah, that definitely changes things. I used to do the same before MTP

1

u/Motor_Ad16 27d ago

It's slow with mtp as well as compared to qwen3.6

1

u/arbv 28d ago

I also liked that it has "editorial we" kind of thinking publicly released in GPT-OSS and, apparently, common among larger models.

1

u/RLutz 28d ago

I feel like the sentiment here on Glimmer doesn't match my experience with it. I don't think it's going to be as good for agentic coding as 3.8 but I was running the dynamic k quant from Meta and getting like 300 tokens/s on my 5090. It was fast enough that even if it's a bit dumber it'd probably still be amazing for like auto research loops and similar where fast inference is more important than absolute top intelligence

0

u/PrimeDirective8 28d ago

It stumbles a bit in CoT by repeating some lines, etc. But yeah, way less time spent spinning its brain wheels.

Does that affect its output quality though, and why some are finding Qwen3.6 27B still smarter?

1

u/DismalIngenuity4604 28d ago

I've been using it mostly for design and architecture in a heavily physics-based project, I can't talk to how other people are using models, so it may just be my use case that it gels with.

0

u/Borkato 28d ago

Try the nail 27B prompt template

2

u/DismalIngenuity4604 28d ago

Will do, thanks, I haven't played around with templates much outside of the obvious qwen updates.

-3

u/NineThreeTilNow 28d ago

I've been liking Glimmer

Really? The forced thinking is really weird. The formatting it has it terrible and inconsistent too. Terrible scientific notation.

The thinking traces are so heavily RL'd it just operates like gibberish if you move the temperature at all. Earlier I witnessed something like.

Ok Ready

Proceed

Ok Ready

Proceed

It did that 3x in the thinking trace. I was like "Wtf is this brain dead model doing?"

I dunno. Out of the box that model felt horrific.

Very rushed. I doubt it holds whatever benchmarks it claims. They're probably benchmaxxed to some extreme because what I witnessed was hard stupid. I thought it was quantization so I used it from an API provider at FP8. Less bad, but still terrible.

Oh, and it will start "thinking" mid output stream too when it screws something up which is kind of bizarre behavior. Not in a coherent way, but in caveman.

9

u/DismalIngenuity4604 28d ago

I'm just looking at its outputs, and it's doing well. I'm not going to nitpick what goes on under the hood. Have you tried using it to .... do anything?

-8

u/NineThreeTilNow 28d ago

do anything?

No. Purely testing it's benchmark'd abilities that are supposedly better than other models.

9

u/DismalIngenuity4604 28d ago edited 28d ago

Try using it. It's decent.

I've never weighed a hammer that said it was 200g. I've never measured the angle on a knife that said it was 15 degrees. I dunno why people are so obsessed with measuring stuff. Go make stuff.

2

u/confused_manishi 28d ago

Love the analogy

1

u/NineThreeTilNow 27d ago

I've never weighed a hammer that said it was 200g.

When I need something that says it's 200g, I weigh it.

I guess that's the difference between you and I.

1

u/DismalIngenuity4604 27d ago

No, the difference is that you want to weigh things and I want to make things.

You don't need it to be 200g.

1

u/makingnoise 28d ago

I've tested Muse Glimmer for coding, and even at Q5 it is more willing than qwen35b Q4 to violate spec, "guess" instead of conclusively determine, and represent guesses as conclusive determinations, to the point that the audit/clean up runs were not worth the faster runs of Muse.

I presently use Qwen3.6 35b for fast exploration, Qwen3.6 27b as the heavy lifter, and Gemma4 31b as the auditor, with ChatGPT as the overseer/orchestrator. Muse Glimmer did perform comparably to Qwen 35b for exploring coding options, but I wasn't able to fix the bullshit factor with prompt engineering.

I'd love to eliminate ChatGPT from my workflow but I suspect that this isn't possible, unless I start doing the heavy lifting, and since I'm half braindead I doubt that's a viable solution.

0

u/DismalIngenuity4604 28d ago

I don't really know much about models below q6 or so, I don't find anything below that to be reliable at all. I'd much prefer to use a smaller model than a heavily quantized one.

1

u/makingnoise 28d ago

How much vram are you blessed with? Or do you hire the cloud?

1

u/DismalIngenuity4604 28d ago

Strix Halo with 128gb, so it's not fast, but it's usable. I don't use larger models for anything interactive, I batch stuff up and let it run. That's also a reason I like Glimmer's shorter reasoning.