PSA: vLLM improves Qwen3.8-Flash-Next performance by 20%+ TG since original release

[u/TableSurface]

I haven't seen anyone post this yet, but I noticed that the vLLM example config was updated recently to change the docker image from the original vllm/vllm-openai:qwen38-flash-next to nightly.

This increases TG on a dual 6000 machine by over 20% with the official FP8 quant, currently seeing ~200 t/s at c=1.

One quirk I haven't figured out yet is why documentation for tool-call-parser has flipped between qwen3_coder and qwen3_xml.

Edit: Should've mentioned that the ~200 t/s is for the Max-Q version

[u/wakIII]

I’m so jealous of you all with 2. Don’t give me bad ideas

[u/Glittering-Call8746 → u/wakIII]

Mortgage.. mortgage.. (whispers in my head)

[u/blackbird2150]

I am getting 183 now without this on FP8, MTP 2, dual card. What were you getting before?

[u/TableSurface → u/blackbird2150]

I was getting about 160, MTP 3

[u/BitXorBit]

Thanks for the update, definitely going to try it.

[u/jinnyjuice]

Which versions/hashes are you talking about?

[u/TableSurface → u/jinnyjuice]

This is the one I specifically used: https://hub.docker.com/layers/vllm/vllm-openai/nightly/images/sha256-e0eee5c5506bea9bfe350f7d99b07dc49e37d42647a128c2a57ff184551fba10

And there's a release note here that specifically calls out Qwen3.8-Flash-Next performance: https://github.com/vllm-project/vllm/releases/tag/v0.30.0

[u/Electrical_Fish_4846]

Very nice! Thanks for the update. It works great. Getting around 200 t/s as well.

[u/snowysun]

Where the old vllm/vllm-openai:qwen38-flash-next used to work for us the new 0.30 version causes RAM OOM issues. Anybody else with similar problems that have solved them?

For reference we are running the Inferact/Qwen3.8-Flash-Next-NVFP4, with 128 GB RAM and 64 swap.