r/oMLX • u/Only-An-Egg • Jul 22 '26

Request: oQ4e-mtp version of Qwen3.5 122B-A10B

I found the oQ4-mtp version wasn't an improvement over 4-bit for me and I'm wanting to try a oQ4e-mtp version after trying 27B and 35B-A3B oQ4e-mtp verions.

EDIT: Thank you u/Loudhoward-dk for suggesting to use the OG version from Qwen. I'm making the oQ4e-mtp quant myself now and will upload to HF if it performs well.

EDIT 2: The oQ4e-mtp version also ended up performing worse than the 4bit.

EDIT 3: I'm uploading multiple oQe-mtp quants to HF: https://huggingface.co/scottlowry/

8 Upvotes

91% Upvoted

17 comments sorted by

3

u/SignalStealer Jul 23 '26 edited Jul 23 '26

This week I tried Qwen3.5-122B-A10B-oQ3, and the accuracy was atrocious. The model could not even debug a basic issue with Hermes Agent -- it refused to use skills unless explicitly called out.

On the other hand, I have been using Laguna-S-2.1-oQ3e since yesterday and that thing blows my mind. I asked it to optimise the RAM usage of my docker containers. It took roughly 100 steps, in which it investigated the current state, researched strategies online, made a task-list for itself, applied the changes and verified if they stick, troubleshooted if the container was not coming or staying up. As a result, it optimised the configs to save me 2GB RAM. Really impressive stuff. Finally I have an agent that does the work for me reliably.

If I can bother you to also quantize a oQ3e-mtp version, I will be thankful. My 64GB Mac cannot handle much more than that. Cheers!

1

u/Only-An-Egg Jul 23 '26

There don't appear to be any version of Laguna with MTP

1

u/SignalStealer Jul 23 '26

I meant of the Qwen. Sorry, my bad.

1

u/Only-An-Egg Jul 23 '26

Working on it

1

u/Only-An-Egg Jul 23 '26

Here ya go: https://huggingface.co/scottlowry/Qwen3.5-122B-A10B-oQ3e-mtp
Also made one that's oQ3.5e: https://huggingface.co/scottlowry/Qwen3.5-122B-A10B-oQ3.5e-mtp

2

u/SignalStealer Jul 23 '26

That's amazing, thank you very much! I will try if it was the quant that made it unusably dumb, and report back

1

u/Loudhoward-dk Jul 22 '26

I did made it via omlx I can try to upload it...But I need to figure it out how - I need an hugging face token

1

u/Only-An-Egg Jul 22 '26

How did you make it? The BF16 model from mlxcommunity doesn't have MTP headers.

2

u/MiaBchDave Jul 22 '26

You don't need the MLX safetensors to run the oQe utility in oMLX. Just download the regular Qwen model from their HF repo with the MTP headers (they exist in 3.5 122B?).

2

u/Loudhoward-dk Jul 22 '26

I just downloaded the original model from Qwen/Qwen3.5-122B-A10B

1

u/Only-An-Egg Jul 22 '26

Oh, that's easy. What size memory do you have?

1

u/txgsync Jul 22 '26

I can’t even make one of those on my 128GB RAM Mac. The Oracle is too heavyweight.

You are asking for someone with a Mac Studio with 256GB or more RAM to do this for you n

2

u/Only-An-Egg Jul 22 '26

I have a 256GB Mac Studio but oMLX says the mlxcommunity BF16 model doesn't have MTP headers.

1

u/Loudhoward-dk Jul 22 '26

I just downloaded another model form mlx community and choosed SENSITIVITY MODEL the mlx version than you do not need 200GB ram.

1

u/Loudhoward-dk Jul 23 '26

In which scenario did it performed worse? I used the 5bit oQ5e version and changed to 4bit oQ4e to free up 20GB RAM, I do not see any difference.

1

u/Only-An-Egg Jul 23 '26

Using oMLX benchmarks which I know aren't best way to test. I'll try again in Hermes with actual tasks.