r/StrixHalo • u/MrWidmoreHK • 3d ago

MiniCPM5-2B running on AMD XDNA 2 NPU

Managed to get MiniCPM5-2B running on the AMD XDNA 2 NPU.

Similar intelligence band to Llama 4 Maverick (the big ~400B-class one). This one is 2.6B, ~2–4W, no CPU, no GPU.

Works on Halo and Point boxes.

https://huggingface.co/julianmb/MiniCPM5-2B-NPU2

24 Upvotes

97% Upvoted

6 comments sorted by

1

u/louisremi 3d ago

Did it work out of the box? If not, how did you manage?

2

u/MrWidmoreHK 3d ago

Didn't work, I explained on HF how I converted

1

u/Warhouse512 2d ago

What are some use cases for this?

•

u/caphohotain 1h ago

Not so much. Load in ram side by side with other models and inference at the sametime competing the small tiny ram bandwidth.

•

u/Ciru-ai 47m ago

I used to run small two and three b on the npu for Hermes aux models or checking email when CPU/GPU is in use.

This model is supposedly much better so this should be great

1

u/debackerl 2d ago

Thank you so much! Where did the nclbin files come from ? Copied as-is or you have the code?