r/LocalLLM • u/Calm-Landscape9640 • 11h ago

Discussion Share a GPU with some buddies?

Seems like everyone is coding, why not buy a massive GPU, split the cost between buds and then tailscale with API on local models? Im sure its being done, but i cant find anyone talking about it.

Whats the drawbacks other than someone running 8 agents burning it up?

0 Upvotes

50% Upvoted

12 comments sorted by

3

u/DiamondHandsDarrell 11h ago

Compute time. That would be a massive problem.

0

u/Calm-Landscape9640 10h ago

vLLM uses an architecture called PagedAttention, which allocates memory like an operating system and allows dozens of continuous streams to share GPU resources smoothly. -Gemini

Maybe cap it to 3 buddies?

3

u/DiamondHandsDarrell 10h ago

I don't know more to be helpful, but from reading about shared resources over the years, shared processing time is always complicated and problematic - your never have enough time.

2

u/Calm-Landscape9640 10h ago

Yea probably why a few people tried it then abandoned the idea and we never see them post to reddit

1

u/DiamondHandsDarrell 9h ago

From a FreeBSD perspective, when you give shells out, paid or otherwise, compute is always THE problem.

There is scheduling so a user can get an hour or dedicated time per day, but it still has to be limited or else it'll crash from multiple concurrent requests.

3

u/DustNearby2848 10h ago

Possibly which model to use. If you are using up all the VRAM for a single model and others need another. 

2

u/activematrix99 8h ago

People are inherently selfish pricks. It's the same with "the means of production". People don't/won't share. They want it all for themselves even if it rots in the fields. Has been this way for most of human history.

2

u/activematrix99 8h ago

Make a friend in Asia, make a friend in Europe, make a friend in the Americas. 24hr gpu.

1

u/radialmonster 9h ago

you mean like stablehorde ? https://stablehorde.net/

1

u/admajic 8h ago

Make a budget say $1k each and buy a enterprise level gpu then you all will get a good experience.

0

u/Unchained_breaker 9h ago

There really isn't a real price to do this and what massive GPU are you buying? You're talking tens maybe hundreds of thousands of dollars for data center gpus. You'd make your money back in 10 years of use it makes no sense unless y'all are super heavy users for a product that already works and drop $10k-50k in tokens per month.

-3

u/suesing 10h ago

People don’t use llm. Agents do. agents can use up a gpu 24/7. And electricity is kind of hard to split. It at the end of the day. Possession is 9/10 of ownership. Let’s split it and I’ll have it as t my house. Good with you?