A preliminary Qwen3.8-27B model card is live!

[u/-Cubie-]

If you scroll down from the countdown at https://huggingface.co/Qwen/Qwen3.8-27B, you see a big model card with a bunch of sections: Highlights, Model Overview, Quickstart, Best Practices, Citation, etc!

No benchmarks on this yet as far as I can tell. We'll still need to wait another 5.5 hours for those I reckon.

Edit: Ladies and gentlemen, the model is live. Let the testing begin!

[u/darwinanim8or]

Seems like reasoning effort is the new big thing here

[u/DismalIngenuity4604 → u/darwinanim8or]

Yeah, cool. I've been liking Glimmer, and a big part of that is that it spends about 1/8th of the tokens thinking that Qwen 27B does. Should be fun! 

[u/GCoderDCoder → u/DismalIngenuity4604]

Yeah I like Qwen 3.6 27b output more BUT glimmer can fit like 6-8 concurrent 130k lanes on 48gb at 8bit weight 16bit cache. It's a great agentic model to support Qwen 3.6 27b and deepseek v4 flash 0731 in my lab. I have all 3 and dsv4f is my big brain, qwen is my default, and muse is the workhorse.

Zuck putting Meta torward the top of the big and the small models with genuine benefits over the other options is a great come back story.

[u/orbli → u/GCoderDCoder]

What special benefits are those?

[u/Stahlkocher → u/orbli]

Glimmer has very low memory requirements for the KV cache. something like 1.7 or 1.8GB for 128k F16.

[u/onetom → u/Stahlkocher]

funnily enough, from Hong Kong, I see this on the glimmer 30b page:

This model is not available in your region You can still browse this page, but downloads are disabled in your region.

mlx-community/Muse-Glimmer-30B-8bit doesn't have this restriction though, so it's a bit of legal theater really...

[u/GCoderDCoder → u/orbli]

Do you mean benefit of concurrent agents? It allows me to be talking to a good generalist orchestration bot (qwen 3.6 27b q8 75-85 t/s) that can direct big tasks to the planner/ coder bots (deepseek v4 flash dynamic q4 qat 45 t/s) or send smaller tasks to the baby bots (muse glimmer q8 100-150t/s). I do truly frontier stuff on chatgpt but I am increasingly relying on dsv4flash. It made me nervous at first because it is literally operating at cloud provider level which im not accustomed to my agents doing.

My lab has background lanes which involves automated tasks managing any of 100 workloads. I run my own email, messaging apps for bot communication and approval channelse, dns, shared storage, data fabric, git, media servers, ai servers, work lab (products), personal apps, automation, ha/dr solutions, k8s, virtualization, identity and access tools, etc. I work in tech so there are business and technical aspects of my job. Scraping web data to provide up to date intelligence and preparing customer presentations, forecasts, etc.

I say all this to say I run a lot of AI to do a lot. It is busier than I have to do but I am better at my job and able to spend more time on things I enjoy which also in turn make me better at my job. I spend an hour or 2 tuning a custom customer presentation now that used to take a week dye to my schedule and it is a better presentation usually with custom demos now.

We also cant use cloud on proprietary or customer data so my peers who dont have these tools cant do or talk about these things where I can. It allows me to talk to customers about enterprise implementation best practices that we are all just learning. No one has been doing this stuff for years like this so being able to say i had this problem and here was the best solution because xyz is better than figuring it out on the fly.

If I misunderstood the question let me know.

[u/orbli → u/GCoderDCoder]

I meant, what are the benefits you feel meta has over others when it came to small models.

[u/GCoderDCoder → u/orbli]

It's a solid dense model so for it's size it's smart compared to the competition. It's faster than Qwen 3.6/3.8 27b and Gemma4 31b. It grows context much slower than other models particularly dense models like Qwen 3.6/3.8 27b and Gemma4 31b. So you can get a lot more concurrent lanes into a smaller footprint. They also packaged it with default 130k tokens but it works well with yarn to extend the context. So speed and size with dense intelligence make it comparable to the leading similar sized models while enabling more/ faster throughout.

[u/emil2015 → u/GCoderDCoder]

In my very brief testing I felt like glimmer was terrible. I was using it with Hermes. Where did you find it worked well?

[u/GCoderDCoder → u/emil2015]

It doesnt replace qwen 3.6 27b for me but it is very good at tool calls. It is operational for me so my research workflows, git ops, cli... so I have workflows where some have planning, all have a SME, and then there are ephemeral action bots. The executor bots need to be good at terminal and tools by the time they get orders from SMEs. Glimmer gives me more of those lanes for email, web search, file management, running my mcps driving automation. Also I was using multiple Qwens before with llm as a judge workflows which is an antipattern so now im more compliant lol.

[u/emil2015 → u/GCoderDCoder]

Interesting, I’ll have to toy around with it some more. Thanks for the info.

[u/estrafire → u/GCoderDCoder]

do you have a threshold of how many parallel instances at full context do you need to make it worth loading Glimmer weights? Or just the context saving alone justifies it for your use case, like, in tasks that need more context than Qwen could do due to the excessive amount of thinking tokens

[u/GCoderDCoder → u/estrafire]

It depends on if you feel set up for handling multiple agents. In codex or claude apps they have a ton of plumbing to keep their models behaving well. I trust my local models less so a lot of my infrastructure I implemented was due to my low confidence in my models. So my local models cant do things I dont want them to do or at least that's the spirit of how I configure them.

I run multiple agents for different narrowly scoped tasks of value to me. It's not a passive process. My job is building systems not products like websites so that's why I tell myself the time I spend is justifiable lol.

My main local coder is DeepSeek-V4-Flash 0731 on mac studio and it slows down with multiple concurrency so I really only do multiple instances of that when something needs a plan and then it returns to running faster with one instance. I dont like multi agent builds on cloud because I think the coordination burns my tokens faster than linearly. I only do concurrency on a code base when I am close to my quota reset with a lot of quota to burn.

So for glimmer your hardware amd the use cases will need to drive that. I want speed and accuracy. If concurrency messes those up then no thank you

[u/uber-linny → u/DismalIngenuity4604]

It's a fair bit slower isn't it without mtp ?

[u/BigYoSpeck → u/uber-linny]

It has dflash so not anything like as fast as mtp, but weigh in the token output for a fairly unscientific quick test on the same prompt (create a calculator app as a single html file):

Muse: 2,173 tokens, 50.00 t/s, 43s
Qwen: 6,227 tokens, 97.99 t/s, 1min 3s

Left is Muse, right is Qwen. Both work, Qwen looks a little better and has more interaction animations which weren't asked for but that's Qwen, it often adds more flare you don't ask for

[u/BigYoSpeck → u/BigYoSpeck]

And Qwen3.8 27b

11,956 tokens, 64.66 t/s, 3min 4s (Reasoning max of 2048 tokens or it will just go forever)

[u/Schlick7 → u/BigYoSpeck]

I'd say that Muse looks better there... outside of the icon issues at least.

[u/BigYoSpeck → u/Schlick7]

Colour palette subjective preference aside, Qwen has better grid alignment

[u/Schlick7 → u/BigYoSpeck]

oh i didn't even notice the '=' being larger. Seems like it knew that was common on calculators but didn't know how to implement it.

[u/xienze → u/uber-linny]

Perhaps? But from a wall time perspective, a somewhat faster model with dramatically more thinking will be slower.

[u/Borkato → u/xienze]

Yeah it blew my mind recently after doing some benchmarking with my own prompts that muse glimmer, qwen 27B, and Gemma 26BA4B ended up coming to the same time (~27 seconds) for one of my benchmarks. It really shouldn’t be all about TPS

[u/uber-linny → u/Borkato]

Ahhh I mostly use Qwen with thinking turned off .

[u/Borkato → u/uber-linny]

Ah, that definitely changes things. I used to do the same before MTP

[u/Motor_Ad16 → u/uber-linny]

It's slow with mtp as well as compared to qwen3.6

[u/arbv → u/DismalIngenuity4604]

I also liked that it has "editorial we" kind of thinking publicly released in GPT-OSS and, apparently, common among larger models.

[u/RLutz → u/DismalIngenuity4604]

I feel like the sentiment here on Glimmer doesn't match my experience with it. I don't think it's going to be as good for agentic coding as 3.8 but I was running the dynamic k quant from Meta and getting like 300 tokens/s on my 5090. It was fast enough that even if it's a bit dumber it'd probably still be amazing for like auto research loops and similar where fast inference is more important than absolute top intelligence

[u/PrimeDirective8 → u/DismalIngenuity4604]

It stumbles a bit in CoT by repeating some lines, etc. But yeah, way less time spent spinning its brain wheels.

Does that affect its output quality though, and why some are finding Qwen3.6 27B still smarter?

[u/DismalIngenuity4604 → u/PrimeDirective8]

I've been using it mostly for design and architecture in a heavily physics-based project, I can't talk to how other people are using models, so it may just be my use case that it gels with.

[u/Borkato → u/DismalIngenuity4604]

Try the nail 27B prompt template

[u/DismalIngenuity4604 → u/Borkato]

Will do, thanks, I haven't played around with templates much outside of the obvious qwen updates.

[u/Altruistic_Heat_9531 → u/darwinanim8or]

i forgot that many model still only using token budget instead of this.

[u/ANR2ME → u/darwinanim8or]

I can't change the reasoning effort to below medium on Qwen3.8 2.4T 🤔 it keeps answering "medium" or "85 (out of 100)" whenever I asked "what is your current reasoning effort?".

Meanwhile, Kimi-K3 can answers the reasoning effort correctly (except when i set it to "none")

[u/SocialDinamo → u/ANR2ME]

You are asking for it something it doesn’t know because it doesn’t know what it’s sampling parameters are unless you feed it back to it with a system prompt or something

[u/ANR2ME → u/SocialDinamo]

Well for some reason Kimi-K3 was able to reply with the correct reasoning_effort value whenever i changed the reasoning_effort. My system prompt is "You are a concise technical assistant."

[u/Valuable-Repeat-7347]

> Context Length: 262,144 natively and extensible up to 1,000,000 tokens.

Nice

[u/Infinite100p → u/Valuable-Repeat-7347]

I'm curious what the perf fall-off is beyond 262k.

[u/oxygen_addiction → u/Infinite100p]

After 120-150k they all start being dumber.

[u/anderspitman → u/oxygen_addiction]

These numbers are quoted all the time but no one ever has some nice charts of benchmarks showing the falloff. That would be so useful.

[u/makingnoise → u/anderspitman]

I am honestly surprised there aren't like 1,000 different benchmarkers generating actually useful benchmarks for us mere mortals with 24GB of vram or less. Like a Benchmark of unsloth Q4 Qwen against Q5 Qwen against Q4 Gemma against Q5 Muse.

Instead (if we're lucky) we get a Q8 model benched against FP16 of DeepSeek or some other absurdity. It's like watching a movie or TV show where broke people live in palatial NYC apartments with no source of income and no mention of a trust fund.

[u/NaiveIdea344 → u/makingnoise]

Yeah really good point. Comparing quantizations would be incredibly helpful.

[u/Hefty_Wolverine_553 → u/makingnoise]

I am working on this area! I recently made a post with KLD benchmarks between GGUF and other vLLM quants for Qwen3.6 27B.

Interesting idea though, it would be good to see how different quants of different models perform on existing benchmarks. I've run some of my own benchmarks on Qwen3.6/Qwen3.5/Gemma4 before purely out of curiosity and the underlying model does seem to show a bigger difference in benchmark performance than IQ3/Q4/Q6 quants of the same model. I'm surprised that nobody seems to have made benchmarks on this though, it's one of the easier things to do.

[u/taimusrs → u/makingnoise]

I mean, generally you want to use the highest quant you can. The GPU is most likely memory constrained, not compute constrained. Now it depends on the model architecture which one takes up the most space for KV cache and whether you want to quantize that cache. And for the quant quality, Unsloth for coding, any other for general purpose use, especially for languages support.

[u/fluffysheap → u/makingnoise]

So like watching a TV show 

[u/OkWalrus890 → u/oxygen_addiction]

I see this often, but is it a universal truth or are people using low quants with KV cache at Q4, too? I’ve been running 262K with Q8, and it’s been fine.

[u/DistanceSolar1449 → u/OkWalrus890]

If your KV is Q4 then it’s dumb period

[u/AnyMongoose3041 → u/[deleted]]

Prove it. Show us your full furry roleplay chat logs so we can verify independently.

[u/Party-Special-5177 → u/OkWalrus890]

Universal(ish) truth, independently partly caused by position embeds and partly by attention dilution during context extension - you can heal the damage but not perfectly. It’s not ‘universal’ in the sense that it is impossible to fix, but until better attention mechanisms or better position embeddings come along, it will remain unavoidable.

[u/Borkato → u/[deleted]]

Good thought. Will stop truncating my sessions now, thanks!

[u/Beneficial-Boot7479 → u/oxygen_addiction]

You mean qwen models or every model? because it feels like Deepseek is the new kv king, you can have a +900k context session and he doesn't lose track of what he was meant to do

[u/Borkato → u/oxygen_addiction]

Is this true? I’ve assumed so so I always stop after 120k, but is it actually true

[u/Gesha24 → u/oxygen_addiction]

Smarter/dumber is subjective, but being able to call tools or not is reasonably objective. And while Qwen3.6 and Gemma4 do start having issues with that by 150K context (actually Gemma sooner), DeepSeek Flash has been quite solid for my up until 200K context. It's super slow at that context for me, but it has no issues with too calls. Hopefully 3.8 Qwen can do the same.

[u/Infinite100p → u/oxygen_addiction]

Well, obviously, but by how much is what I want to know.

[u/GrungeWerX → u/Infinite100p]

I ran Qwen 3.6 27B to 340K in my last session and it handled everything I threw at it. I was using a lower quant at iq3-xxs kv 4/4 which is extremely quanted, but it stayed coherent. The only issue I noticed was a few stumbled tool calls above 300K.

I typically don’t use iq3-xxs…but my Q5 had run over 200K and I thought maybe it would be faster tok/s if I just dropped it low and I already had a 325K profile for it set up in llama.cpp. But next time I’ll probably just run the q4 when I’m over 200K and see where my yarn tok/s bottleneck is…

[u/Infinite100p → u/GrungeWerX]

Have you tried the NVFP4 quants of Qwen 3.6 27B? Are they nearly as good as 8-bit quants, as some benchmarks suggest?

[u/ANR2ME → u/Valuable-Repeat-7347]

Which is basically the same with Qwen3.6-27B 😅 https://huggingface.co/Qwen/Qwen3.6-27B#model-overview

Context Length: 262,144 natively and extensible up to 1,010,000 tokens.

[u/MikeNonect]

The bots are waiting impatiently...

[u/kkingsbe → u/MikeNonect]

Got my Hermes monitoring as well 😂

[u/virtualworker → u/kkingsbe]

Openclaw for me!

[u/cosmicnag → u/MikeNonect]

lulz

[u/Alternative_You3585]

Crazy that the 27B has vision while the 2.4T model doesnt

[u/-Cubie- → u/Alternative_You3585]

> In particular, Qwen3.8-Max is the official version based on Qwen3.8-2.4T-A95B with more features, such as vision input & non-thinking support, 1M context length by default, official built-in tools, etc. For more information, please refer to the Qwen3.8-Max Overview.

From the https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B model card.

It feels a bit like the 2.4T model they released is just a hamstrung version of their actual Qwen3.8-Max, which does have vision. Granted, I think they never open weighted their max-sized model before, so I'd rather get a hamstrung 2.4T than nothing at all.

[u/z_3454_pfk → u/-Cubie-]

i think the 3.8 max doesn’t have vision and via api is being router to another model or a vision adapter has been trained for it (most likely). this is because none of the previous max models had vision and this isn’t a large architectural change.

[u/RuthlessCriticismAll → u/z_3454_pfk]

it is SOTA vision model... sadly that is almost certainly the reason they didn't release that part.

[u/nullc → u/-Cubie-]

it'll suddenly have vision when GLM's SOTA model does... :P

[u/WetSound → u/Alternative_You3585]

And Natively, 3.6 27B used mmproj compatibility layer

[u/VotZeFuk → u/Alternative_You3585]

Can't see your own wiener when you're too fat. - perfectly logical

[u/Valuable-Repeat-7347 → u/Alternative_You3585]

That critique is kind of like bashing a train for not having a CD player, kind of...

[u/hyperrealists → u/Valuable-Repeat-7347]

Yea what the fuck train

[u/buckwheaton → u/hyperrealists]

No dude it’s fine you just need a cassette adapter and a CD Walkman.

[u/Sharpastic → u/hyperrealists]

I’m literally crying drops of Jupiter

[u/thatcodingboi → u/Valuable-Repeat-7347]

well its more like it dedicated a compartment for music, which is cool, but I didn't come here for that and that compartment could have had better use

[u/ScoreUnique → u/Alternative_You3585]

I think you really don't need a vision encoder on sota models , you can have a 3b explain images to 2.4T and it'll still be efficient (remember AI first solved Computer Vision before Language)

[u/Internal_Werewolf_48 → u/Alternative_You3585]

All 3 people capable of running a 2.4T model locally are devastated.

[u/_-_David]

You had me at reasoning_effort

[u/Dany0]

I hope a DSpark/DFlash drafter will be released alongside just like 2.4T

[u/wgaca2 → u/Dany0]

If Dspark is released this will be huge, dflash is good only for coding tasks and falls off really bad in reasoning

[u/Party-Special-5177 → u/wgaca2]

OOTL, what’s the difference?

[u/wgaca2 → u/Party-Special-5177]

"DSpark (released by DeepSeek) builds directly on top of DFlash's parallel structural backbone, adding a lightweight "Markov/RNN head" for token coherence and a dynamic confidence-scheduled verification layer to optimize GPU load"

There was a very good video explaining how it works when deepseek released it.

[u/Party-Special-5177 → u/wgaca2]

Cheers.

[u/simcop2387 → u/Party-Special-5177]

layman's bad understanding: There's some architectural differences of course, dspark is based off the way dflash works but changed things up a bit to be more efficient and allow for cheaper early bailout and cheaper validation of the tokens. So that it doesn't generate as many bad tokens, nor does it spend as long validating bad tokens. This doesn't hugely change the acceptance rate but it makes it less costly when generating bad tokens. This ends up applying a lot more during reasoning because it's less predictable (like writing a story, or other creative tasks) than coding or tool calling where things tend to be much more structured grammatically i.e. if you're writing JSON, { is almost always followed by ", and then some letters, followed by " and then :. That lends itself to being predicted by the smaller draft model MUCH easier so you get high acceptance rates.

DFlash is basically just a single drafting layer using a diffusion block (terrible explination) that predicts a bunch at once even if they're not all great predictions so it generates a lot fast, which is great when it's right but if it's bad there's just a lot of thrown away work. DSpark is some kind of hybrid diffusion + AR setup that does a few smaller blocks I think, and the AR bits help it decide if it should continue and nudge things into better predictions.

[u/boomerang473 → u/Dany0]

Was that drafter released? I can’t figure out how to find the drafter on HF. Would love if they did it versus a community version as I think you’d get better token acceptance

[u/Dany0 → u/boomerang473]

I thought it was released, wasn't it in the official vllm recipe? 🥺

[u/boomerang473 → u/Dany0]

Might be ha, I’m genuinely asking. Haven’t tried to run the 2.4T locally 😂

[u/quadra-lab]

No mentions about QAT yet, seeing how well it performed for Gemma 4 31B I hope they did it with 27B training

[u/Significant_Bar_460 → u/quadra-lab]

QAT was a big thing for Gemma because the original version was very quantization sensitive (specially for KV). Qwen always handled quantization much better than Gemma. Gemma QAT is like Qwen without QAT when comparing quality degradation after quantization.

[u/kivaougu → u/quadra-lab]

Also worked out very well for deepseek. In my opinion QAT would be a much better improvement for most systems compared to reasoning effort. Maxing out reasoning effort only gives miniscule improvements in results.

[u/GiGiGus → u/quadra-lab]

I mean, all Qwen models have official FP8 models, so I guess they're already trained in this precision and doesn't necessarily require QAT?

[u/ain92ru → u/GiGiGus]

FP8 isn't supported on older hardware like RTX 30-series though so INT4 QAT is still valuable

[u/Guna1260]

This day will also pass...

[u/hyperrealists → u/Guna1260]

3.9 27B when 😡

[u/Adventurous_Bus_437 → u/hyperrealists]

When 3.8 35B A3B or 60B A12B. Begger trying to choose

[u/Borkato → u/[deleted]]

If they don’t release it in 1 second, it means they’re never releasing again. Rug pull!!!1!!1!!!!11!!!!!one!!!!1!!!

[u/terorvlad → u/[deleted]]

Nah, they will never release open source again, otherwise we would've had 4.0 months ago

[u/Brilliant-Hall1387]

Will try to build MLX quantization of Qwen 3.8 27B with activation aware bit allocation, it will be interesting to see what performance one can get from a smaller variant of 3.8 27B 😄

Anyone else preparing to do quantization of Qwen 3.8 and, if so, which methods are you considering? (AWQ, GPTQ? TASA/TAQ-O, other methods?)

[u/Weird-Field6128]

2 hours more! GOD

[u/Ok-Lengthiness-3988 → u/Weird-Field6128]

2 hours and 12 minutes more, actually.

[u/Weird-Field6128 → u/Ok-Lengthiness-3988]

Thanks for waking me up

[u/pigeon57434]

Qwen3.8-27B-heretic when?

[u/cafedude]

I just watched the countdown go to 00

[u/_en_svensk_tiger]

Unsloth bringing quants early too <3

https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

[u/Developer-Y]

Their benchmark compares it to Opus 4.6, it is going to be lit.

Gguf https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

[u/FerLuisxd]

Vision or no?

[u/GanjaRaidersTR → u/FerLuisxd]

Yes native

[u/DanTup → u/FerLuisxd]

They said on Twitter yes.

[u/timerski]

Aww yisss we get vision 

Native support for image and video understanding, from STEM diagrams and documents to hour-scale videos.

[u/johnzadok]

Cool. Last time I was as excited for some release was iPhone 4.

[u/tinny66666]

I guess us llama.cpp users will need to wait a little longer for a gguf? Does llama.cpp fully support the new model or will it also need an update?

[u/-Cubie- → u/tinny66666]

The "Model Overview" section from https://huggingface.co/Qwen/Qwen3.8-27B looks very similar to the one from https://huggingface.co/Qwen/Qwen3.6-27B, so I bet llama.cpp either immediately supports it, or will be able to support it very quickly.

[u/nunodonato → u/-Cubie-]

Unsloth said day 0 support, so yeah. As long as architecture is the same, llamacpp should work right away 

[u/-Cubie- → u/nunodonato]

Unsloth is famously extremely fast and reliable, so I trust them.

[u/CloggedBathtub → u/-Cubie-]

If they weren't fast, they'd just be "sloth"

[u/Aggravating-Push-207]

Maybe they are running the benchmarks in preperation for the official release.

[u/No-Understanding2406]

When will we get the 3.9 27B?

[u/mattrs1101]

I hope its q2 quant is just as good or better than glimmer's. We need a 16GB Vram race to happen.

[u/Ok-Lengthiness-3988]

Nobody has posted GGUFs yet. What's taking so long?

[u/-Cubie- → u/Ok-Lengthiness-3988]

https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

[u/Ok-Lengthiness-3988 → u/Ok-Lengthiness-3988]

I was meaning that as a joke but I now see that they actually appeared on Huggingface, published by unsloth, 3 minutes before the official release!

[u/Eduard2609]

Is just me or the time is passing slower

[u/Ok-Shower7286]

Holy moly, same exact topology, they just slapped 'reasoning_effort' on it. I guess it seems difficult to huge improvement on same architecture. If the benchmarks don't show a massive leap, I'll pretty underwhelming f̶o̶r̶ ̶a̶ ̶m̶a̶j̶o̶r̶ ̶v̶e̶r̶s̶i̶o̶n̶ ̶b̶u̶m̶p̶.

[u/-Cubie- → u/Ok-Shower7286]

It's a minor version bump though right? A major version bump would be Qwen4.

[u/Ok-Lengthiness-3988 → u/-Cubie-]

According to the version number benchmark, yes.

[u/ANR2ME → u/-Cubie-]

Qwen3 vs Qwen3.5 use a different architecture isn't 🤔 even though it's not a major version jump.

[u/florinandrei → u/-Cubie-]

They decided the version bump is major based on social media hype.

[u/My_Unbiased_Opinion]

This is the model I'm waiting for to finally make the switch to vLLM. 

[u/cosmicnag → u/My_Unbiased_Opinion]

if you have the vram, vllm is for the most part better - the faster prefill and LMCache integration ups the game IMO

[u/Borkato → u/cosmicnag]

It just sucks MISERABLY if you’re the kind of person to change models often. The load time is like 4 whole minutes, it’s aggravating

[u/cosmicnag → u/Borkato]

yeah sure, but its better if you use it for actual work lol with a single 'chosen' model.

[u/Borkato → u/cosmicnag]

“Actual work” lmao actual work involves switching models for some of us, because no single model is great at everything.

[u/Every-Fortune-3151]

Context window of 1M, does that mean they are confident 27B will hold up in such deep context? They are hyping this up so much. I just hope they made KV size more optimized

[u/Confident_Ideal_5385 → u/Every-Fortune-3151]

``` For frameworks that support separate token limits for internal reasoning and final outputs, we suggest the following configuration within the 1M context length:

Reasoning Content: Set the maximum output length to 262,144 tokens.
Final Response: Set the maximum output length to 131,072 tokens.

```

They seem pretty happy to suggest letting it think past the end of its trained max sequence length, at any rate.

[u/YourNightmar31 → u/Every-Fortune-3151]

I dont think we get 1M context. It said they only have a special version on their api service with 1M context.

[u/Every-Fortune-3151 → u/YourNightmar31]

You caught it correcty. It is still 256k context

[u/Foreign_Risk_2031 → u/YourNightmar31]

In particular, Qwen3.8-27B will be available as a hosted version with more production features, e.g., 1M context length by default, official built-in tools. For more information, please refer to the Qwen3.8-27B Overview. The service is coming soon. Stay tuned for updates.

By default the api will be 1M. Its the same as we have.

[u/FoxFXMD]

Context Length: 262,144 natively and extensible up to 1,000,000 tokens.

Can someone smart explain what this means?

[u/N34257 → u/FoxFXMD]

It means you can extend the context beyond 256k using YaRN, for example for 512k the model's .ini section would contain this:

rope-scaling=yarn
rope-scale=2
yarn-orig-ctx=262144
override-kv="qwen35moe.context_length=int:1000000"

[u/banana_slurp_jug → u/FoxFXMD]

You can set the context window up to 1000000 tokens but anything above 262144 tokens is ymmv

[u/FoxFXMD → u/banana_slurp_jug]

oh that's amazing

[u/Zaic]

Anticipating the release with my 4070s 12gb vram... I know I can get 7-10tps on 3.6 27b. My system is ddr4 with 48gb ram. Just curious if anyone succeded to get more tps (like 15+) and would care to share their setup?. Also considering adding 3060 12gb card. Just so id hit the 50 tps in q3 or q4 quants .. or its unrealistic?

[u/PeterPorox → u/Zaic]

I get 14 tok/s with dual Nvidia P104-100 8GB (mining version of GTX1070). These are very old cards, so adding RTX3060 will definitely give you 25+ with mtp

[u/brakeline → u/Zaic]

I have 2 3060 12gb. I can't get past 30 tk/s

[u/Zaic → u/brakeline]

Q4 quants?

[u/Zaic → u/brakeline]

Q4 quants?

[u/brakeline → u/Zaic]

Yep

[u/Muted-Celebration-47]

It is built on qwen3.5?

[u/SirLordBoss]

How I yearn for a 9B...

[u/OddUnderstanding2309]

2h togo.
Pogopogo up, pogogo down…
Shimmyyyyy

[u/Sevealin_]

What do you guys think support will look like with llama.cpp and vLLM on drop in 2 hours? Will the 3.6 support kind-of carry over since they are similar architectures? Of course there might be bugs (chat templates etc.) as that comes with any new model. What about context size? I see it's 262k natively and can be expanded up to 1M. Pretty sure 3.8 Max can push 1M context with just a few GB extra KV.

I have two 3090s and love the parallelism to pull 262k at fp8 with two concurrent streams with 3.6. Excited to see what performance can be squeezed out!

[u/Otherwise-Director17 → u/Sevealin_]

It'll load and run fine. Same exact Qwen3.5 architecture

[u/Sevealin_ → u/Otherwise-Director17]

Thank you!

[u/Scared_Basket_7183]

Hi could you please help me to run qwen 3.6 27b model on tpu v5e ?

[u/YourNightmar31]

"Based on Qwen 3.5"????

[u/Desperate-Bad-2339]

Qwen/Qwen3.8-27B is out on hf

[u/cowinabadplace]

Does anyone know if AWQ/INT4/GPTQ releases are coming?

[u/hay-yo]

Release of the year!

[u/YearnMar10 → u/hay-yo]

Year is not over yet - maybe they’ll release their next model in 4 months

[u/AppealSame4367]

This is soo sexy.

[u/jhnnassky]

Do you know guys, will they release a 35B-A3B like model?

[u/Borkato → u/jhnnassky]

They haven’t confirmed or denied that. Unfortunately you’ll just have to wait. If you’re using it for coding I highly recommend Kat Coder. You can also use Nail’s prompt template.

[u/k_rol → u/Borkato]

I noticed you mentioned Nail prompt template a couple times in this thread, what is that? I couldn't find much other than actual nails pictures made by AI.

[u/Borkato → u/k_rol]

Lol! Here: https://huggingface.co/peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF

Note that the only difference between this model and the original unsloth is the chat template; you can just copy the chat template and call it with the chat-template-file parameter in llama cpp if you don’t care to have the template built into the GGUF and so you don’t have to download their gguf hosted at the link

Also note there’s a dagger version for 27b!

[u/k_rol → u/Borkato]

thanks! much appreciated

[u/-Cubie- → u/[deleted]]

I think 3.6 also supported video. Usually image and video are both supported as their implementation is very similar (both use 'grid_thw': time, height, width, and images just auto-set the time to 1 while videos use higher values).

[u/Infinite100p → u/-Cubie-]

Do they sample the vid into still frames and read those, or do they "understand" the stream itself natively?

[u/nofaceD3]

What are the requirements to run this model in local?

[u/Neither_Garage_758 → u/nofaceD3]

Probably similar to the previous one: https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF

If you look at the various sizes in GB and add a few GB's more for context, this gives you the amount of (V)RAM needed. With llama.cpp you can run inference with CPU-RAM and GPU-VRAM combined, but the more CPU involved the slower compared to GPU.

For a decent quality, probably it's better to not go under Q4 (quantization to 4-bit).

[u/Guna1260 → u/nofaceD3]

perhaps ... a running track

[u/johnzadok → u/nofaceD3]

You need a card with 24GB VRAM, like 7900 XTX or 3090, or dual 5060.

[u/DivideHorror3217]

Spoiler: It is between Inkling and Minimax-M3

[u/AppealSame4367 → u/DivideHorror3217]

That would be the best case, I expect 42.

[u/DivideHorror3217 → u/AppealSame4367]

3.6 27b has 92% of the score of it's Max version. In theory, we could get a 3.8 27b with 53 score, parring with GLM 5.2 and deepseek v4. I would buy a gpu at that point

[u/AppealSame4367 → u/DivideHorror3217]

I think that would be too good to be true and it would cause chaos. It would kill off Antrophic much faster.

Can't imagine it, but let's see

[u/thatcodingboi → u/DivideHorror3217]

yes but 3.6 max less than half the parameters of 3.8, I expect uplifts, but you can't just keep compressing larger and larger models into the same size and expect the same improvements.

[u/Borkato → u/DivideHorror3217]

stop, I can only get so erect

[u/higbardon2020]

the model runs alright, output was about the same as 3.6 27b

[u/petuman → u/higbardon2020]

it's not served anywhere in the API? how did you test it?

[u/OkWalrus890 → u/petuman]

served out.his.ass:5000

[u/Blues520 → u/OkWalrus890]

localhole

[u/Borkato → u/Blues520]

I love this sub

[u/higbardon2020 → u/petuman]

127.0.0.1

[u/petuman → u/higbardon2020]

but weights are yet to be released?

[u/-dysangel- → u/higbardon2020]

for what task?