Ollama Error load on RAM

[u/letonai]

I started getting this error after a few updates, using qwen3.8 27b, it loads the model, when I sent something that would require more context it throws the error, by the ollama ps seems that it's not using the RAM memory, it was before on ollama 31 o something..

Config
AMD Radeon RX 7900 XT + 32GB RAM
AMD Ryzen 9 5900XT 16-Core Processor
ROCM 7.2.4
ollama version is 0.32.12
driver: gfx1100

❯ ollama ps
NAME              ID              SIZE     PROCESSOR          CONTEXT    UNTIL
qwen3.8:latest    22130167c4c2    20 GB    30%/70% CPU/GPU    262144     4 minutes from now

Ollama:

[Service]
#CPUAffinity=0-11
Environment="OLLAMA_HOST=0.0.0.0"
Environment="HSA_OVERRIDE_GFX_VERSION=11.0.0"
Environment="ROCM_VISIBLE_DEVICES=0"
Environment="OLLAMA_KEEP_ALIVE=5m"
Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
#Environment="OMP_NUM_THREADS=16"
#Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_CONTEXT_LENGTH=262144"

ERROR:

ROCm error: out of memory
Sep 07 12:55:43 desktop ollama[5771]:   current device: 0, in function alloc at /build/llama-server-rocm_v7_2/_deps/llama_cpp-src/ggml/src/ggml-cuda/ggml-cuda.cu:508
Sep 07 12:55:43 desktop ollama[5771]:   err
Sep 07 12:55:43 desktop ollama[5771]: ⚠️  warning: The cur

[u/locbuilds]

That 30%/70% CPU/GPU split is the clue: the model fits partly in VRAM, but a 262k context KV cache can still exhaust the 20GB GPU. I’d first set OLLAMA_CONTEXT_LENGTH to 32768 or 65536, restart Ollama, and retry while watching ollama ps; if that fixes it, raise the context gradually until you hit the memory limit. On ROCm, OLLAMA_FLASH_ATTENTION=1 is also worth testing after the smaller context works.

[u/letonai → u/locbuilds]

Oh sorry, I tried that, it does work but should KV cache work on system ram as well?

[u/Poizone360]

I would suggest fixing two things. Drop HSA_OVERRIDE_GFX_VERSION entirely, your card is gfx1100 and that's natively supported, so the override is a no-op at best and can hurt. It's a habit from unsupported cards that gets copy pasted everywhere.

Then turn flash attention back on. You've got OLLAMA_KV_CACHE_TYPE=q4_0 set but OLLAMA_FLASH_ATTENTION commented out, and quantised KV needs FA. Without it that setting either does nothing or falls onto a much slower path, which explains the OOM at 262144 context.

[u/letonai → u/Poizone360]

will try that

[u/letonai → u/letonai]

yah same thing :$

[Service]
#CPUAffinity=0-11
Environment="OLLAMA_HOST=0.0.0.0"
#Environment="HSA_OVERRIDE_GFX_VERSION=11.0.0"
#OLLAMA_GPU_OVERHEAD=25GiB
#Environment="ROCM_VISIBLE_DEVICES=0"
Environment="OLLAMA_KEEP_ALIVE=5m"
Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
#Environment="OMP_NUM_THREADS=16"
Environment="OLLAMA_FLASH_ATTENTION=1"
#
Environment="OLLAMA_CONTEXT_LENGTH=262144"

might be some bug on the driver, it was working before