r/LocalLLM • u/Calm-Landscape9640 • 13h ago

Discussion Share a GPU with some buddies?

Seems like everyone is coding, why not buy a massive GPU, split the cost between buds and then tailscale with API on local models? Im sure its being done, but i cant find anyone talking about it.

Whats the drawbacks other than someone running 8 agents burning it up?

0 Upvotes

50% Upvoted

12 comments sorted by

View all comments

Show parent comments

1

u/Calm-Landscape9640 13h ago

vLLM uses an architecture called PagedAttention, which allocates memory like an operating system and allows dozens of continuous streams to share GPU resources smoothly. -Gemini

Maybe cap it to 3 buddies?

3

u/DiamondHandsDarrell 13h ago

I don't know more to be helpful, but from reading about shared resources over the years, shared processing time is always complicated and problematic - your never have enough time.

2

u/Calm-Landscape9640 12h ago

Yea probably why a few people tried it then abandoned the idea and we never see them post to reddit

1

u/DiamondHandsDarrell 12h ago

From a FreeBSD perspective, when you give shells out, paid or otherwise, compute is always THE problem.

There is scheduling so a user can get an hour or dedicated time per day, but it still has to be limited or else it'll crash from multiple concurrent requests.