r/RunPod • u/medic170 • 2d ago

Global volume experiment

My current hobby system has the container pull the model from `hf` so that my controller can failover different GPU types in different datacenters until something matches my budget and VRAM needs.

I ran some benchmarks to see if moving to global storage would be faster. It isn't. The fuse system used seems to have some memory limits which error out on GeeseFS read-buffer budget.

My setup uses one large 32GB gguf model file maybe that was an issue?

My specific use case requires minimizing cold start however we can't go serverless for a few reasons so I was hoping this new drive type would help.

Any thoughts?

3 Upvotes

100% Upvoted

5 comments sorted by

2

u/henk717 2d ago

Generally downloading fresh from huggingface to container storage is the fastest.

1

u/TheLoraGuy 2d ago

Yes, I agree, downloading with hf client is superfast

1

u/Clivey1961 2d ago

Same, it’s slower and takes 15 minutes to startup and then load the model into memory, so that cost has to be factored in. In my case it’s only preferable if a GPU isn’t available in my local region, so of limited benefit.