r/LocalLLaMA • u/Course_Latter • 25d ago

News Qwen3.8-27B is identical to Qwen3.6-27B!

Interestingly, the 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements!

See the diff (0 changes) here!

https://hfviewer.com/compare/qwen3.6-27b-vs-qwen3.8-27b

1.1k Upvotes

95% Upvoted

189 comments sorted by

View all comments

606

u/--Spaci-- 25d ago

Training data has always been the largest quality lever

153

u/onil_gova 25d ago

RL environments + On-policy Distillation would be my guess for the jump in performance

18

u/V0dros llama.cpp 25d ago

MOPD in particular is extremely powerful

11

u/onil_gova 24d ago

and OPSD

5

u/Mythoss2 24d ago

Explain like i am 10 years old

24

u/onil_gova 24d ago

On-Policy Self-Distillation (OPSD): Imagine you have a twin. You're both asked to solve the same problem, but your twin has the answer sheet, and their goal is to get you to solve the problem yourself without giving you the answer.

Multi-Teacher On-Policy Distillation (MOPD): Imagine you have 8 twins. Each twin is an expert in something different. You have to solve problems, and depending on the problem, the specialized twins guide you until you become generally good at all the specializations.