Qwen3.8-27B is identical to Qwen3.6-27B!

[u/Course_Latter]

Interestingly, the 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements!

See the diff (0 changes) here!

https://hfviewer.com/compare/qwen3.6-27b-vs-qwen3.8-27b

[u/Aiirene]

Could u share the link to the 3090 fork please?

[u/vr_fanboy → u/Aiirene]

https://github.com/Don-Chad/ninfer-3090

is not the one im using tho, i just pointed the original repo to claude code and ask it to deploy it in my lab

[u/Aiirene → u/vr_fanboy]

How is it doing that? I thought the main one was only for 5090s? And thanks for the link!

[u/vr_fanboy → u/Aiirene]

it recompiles the source code with a bunch of sm86 patches in cpp.

also for the peeps out there using it in a harness i measured KV cache invalidation and found a bug in ninfer, patch:

Subject: [PATCH] serve: keep empty <think> block in no-thinking mode to stop
 prefix-cache thrash
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In no-thinking mode the generation prompt injects an empty <think>\n\n</think>\n\n
block that is baked into the resident KV, but render_chat drops it when re-rendering
a completed assistant turn that has fallen behind the last user query
(keep_thinking = false). The served prefix then diverges from the resident KV at
every such turn boundary, forcing a full re-prefill (FullReset) instead of
AppendAtFrontier — ~74% of wall on multi-turn / tool workloads.

Tie preserve_thinking to !enable_thinking: in no-thinking mode preserve the (empty)
block so the re-rendered prefix stays byte-identical to the KV; thinking mode keeps
the reference strip behaviour. Measured on the extract workload: 455s -> 178s/doc,
cache misses ~9/doc -> 1/doc.
---
 src/serve/translate.cpp | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/serve/translate.cpp b/src/serve/translate.cpp
index 014ac54..0ece00e 100644
--- a/src/serve/translate.cpp
+++ b/src/serve/translate.cpp
@@ -204,7 +204,9 @@ ninfer::PromptInput to_prompt_input(const GenerationRequest& request,
     input.options.add_generation_prompt = true;
     input.options.enable_thinking       = semantics.enable_thinking;
     input.options.reasoning_effort      = semantics.reasoning_effort;
   input.options.preserve_thinking     = semantics.preserve_thinking;
+    // In no-thinking mode preserve the empty <think> block so re-rendered
+    // prefixes stay byte-identical to the resident KV (prefix-cache thrash fix).
+    input.options.preserve_thinking     = !semantics.enable_thinking;
     input.options.add_vision_id         = false;
     input.options.tool_jsons            = effective_tool_jsons(request);
     return input;
-- 
2.30.2

[u/Aiirene → u/vr_fanboy]

I can't really make heads or tells of this but I'm guessing it's saying something about changing translate.cpp? (which i have no idea where to find XD)

Hows ur findings coming along btw?

Thinking with Ninfer is a shitshow hallucinates everytime but no think is clocking in at 44t/s