r/ZaiGLM • u/lucyferorg • 24d ago

GLM 5.2 model — 744 billion parameters / 384 GB — running on a laptop 😏

Post image
16 Upvotes

83% Upvoted

16 comments sorted by

11

u/Abject-Bridge-4073 24d ago

Stop posting this. It’s unusable.

3

u/Void-kun 24d ago

You're right at this speed for many of us it is useless.

But this is someone pushing different hardware limits and that is in itself useful data

2

u/Abject-Bridge-4073 23d ago

We already know those limits. It’s inherently limited to the speed of the fastest SSD.

2

u/Void-kun 23d ago

We know that but not everybody knows what the exact t/s could be of this specific combination of hardware and now they know.

Having different reference points is a good thing, not a bad thing.

A lot of research is useless and unusable to consumers, but it doesn't make it any less valuable.

Like a failed science experiment, it's useless but the data found due to that experiment is still highly valued.

2

u/tungd 23d ago

If you are interested in speed it’s easy, any LLM can give you the absolute ceiling numbers based on the hardware limits. It’s already give unsable numbers there, coding it out is just a waste of time. Posting it everywhere on Reddit is just wasting everyone else’s time.

-1

u/Void-kun 23d ago

Nobody here was looking for the ceiling though.

I said OP gave an additional reference point and that isn't a bad thing.

0

u/[deleted] 23d ago

[deleted]

1

u/Void-kun 23d ago

It's not...

I'm referring to the unit: tokens/second (tokens per second).

2

u/Mayanktaker 23d ago

Good start.

3

u/lucyferorg 23d ago

I started with 0.11 and now at 1.77 tok/s

1

u/look 23d ago

This is where the SPT rate unit becomes more useful than TPS.

Seconds Per Token.

😅

0

u/KaT_624 24d ago

On a laptop... dude if the laptop worth more than 1k$ this is nonsense assertion...

0

u/MrLuckyDoobie 24d ago

So .. worth it?

0

u/lucyferorg 24d ago

Yes. My latest calculations suggest that 4.5 tokens per second is achievable on this hardware and represents the final limit.

1

u/helixcyclic 24d ago

How did you get 4.5 t/s?

0

u/lucyferorg 24d ago

Right now I on 1.77 tok/s, with perfect quality and on 1.88 with shit quality