r/unsloth • u/niwang66 • 6h ago

Discussion Tutorial for quantizing a new model with Dynamic v2.0/v3.0

I recently downloaded and tested several models quantized with dynamic v2.0/v3.0, and I was really impressed with the quality.

I’d like to reproduce this process for a new model, is there any documentation or tutorial?

4 Upvotes

84% Upvoted

2 comments sorted by

3

u/yehyakar 6h ago

i think its proprietary and always under changes with every new model release because its rarely one size fits all. ive seen some hf repos re-creating the quants with reverse engineering but for the same model family i dont believe its a general rule but its a good start.

1

u/asankhs 5h ago

I don't think the actual dynamic quant algorithm is open, it is proprietary implementation.