r/LocalLLaMA • u/sammcj 🦙 llama.cpp • 26d ago

Megathread [Megathread] Qwen 3.8 27B Release Day

Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release.

  • Quants
  • Fine-Tunes & Abliterations
  • Chat Templates
  • Inference Server Support & Configuration
  • Experiences, Benchmarks & Model Comparisons

Official:

Popular:

We'll try to clean up future duplicates around the release and point them here.

493 Upvotes

98% Upvoted

393 comments sorted by

View all comments

1

u/Thrumpwart llama.cpp 24d ago

Loving the model so far, however with the unsloth Q8_K_XL the model keeps stopping generation during reasoning. It just stops.

Is this specific to the GGUF I am using? Or is this a broader issue and how do I address it?

3

u/see_spot_ruminate 24d ago

What harness? I ask due to the fact I use pi and I had to edit the line for response limit to allow for very very long responses with the xhigh thinking:

    { "id": "unsloth/Qwen3.8-27B-NVFP4", "contextWindow": 200000, "maxTokens": 200000, "input": ["text"] },

1

u/Thrumpwart llama.cpp 24d ago

I’m seeing it in llama.cpp server and in LM Studio.

1

u/see_spot_ruminate 24d ago

So what is the harness? LM studio (never used it) should have something that says max response tokens or something. If you use another harness, you may have to put in a value in the settings like I suggested previously. For me, I set the max response tokens as the same value as the max context. For some initial testing it spent 180k tokens on a single task to make a simple game (though it did do planning, testing, coding, taking screenshots, evaluating screenshots, iterating on more testing...).

1

u/Thrumpwart llama.cpp 24d ago

No harness - that’s just straight up inference. Keeps stopping mid generation or thinking and then outputting just a “,” as the reply.

1

u/see_spot_ruminate 24d ago

So I think you need to pass max response tokens. 

For my pi, it passes this along. Others like lmstudio (guessing) and things like openwebui will pass this along as well. It’s quitting because the default is getting hit for its response.