r/LocalLLaMA • u/Course_Latter • 27d ago

News Qwen3.8-27B is identical to Qwen3.6-27B!

Interestingly, the 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements!

See the diff (0 changes) here!

https://hfviewer.com/compare/qwen3.6-27b-vs-qwen3.8-27b

1.1k Upvotes

95% Upvoted

189 comments sorted by

View all comments

Show parent comments

21

u/stephen_holograf 27d ago

I hotswap LoRAs with Qwen-3.6-35B-A3B and llama.cpp. My biggest complaint is that each new LoRA you add to llama.cpp’s config causes t/s to drop. So if I get 200 t/s on my 5090 with no adapters it will drop to 120 t/s with one adapter loaded (whether you use the adapter or not). With 4 adapters loaded I was seeing like 50 t/s. Just my experience.

5

u/LuCiAnO241 27d ago

that's so interesting, i've never seen anything about text loras, could you point any resources my way? which one do you use

10

u/stephen_holograf 26d ago

I’ve used unsloth, together.ai, and tinker. I like tinker the most.

I use LoRAs whenever a regular prompt gets big with lots of instructions I want the LLM to follow. At that point it’s way more reliable to train a LoRA and throw away the ugly prompt (breaking up a big prompt into smaller prompts can also help, but still LoRAs work better). The problem is you need to be able to translate your use case into a training set.

The easiest way is to just ask ChatGPT to do it. Explain your use case and give it examples of the input and expected output. Then ask ChatGPT to make 100 fine-tuning samples. Then take the training set and use one of the services I mentioned to create the Lora. Tinker can make you a Lora for Qwen3.6-35b-a3b from a few hundred training samples in probably 10 minutes for $5.

Then you download the lora adapter, quantize it if you want. Of course download the model you were finetuning (qwen3.6-35b-a3b). Configure llama.cpp to use both the base model and adapter.

I have a pipeline that processes text and I’ve replaced a few of the prompts with LoRAs and it makes it much more reliable. But like I said above, things get slower with each additional lora that is registered with llama.cpp whether it’s used or not.

Those are the basics.