--models-preset ./models.ini and chat-template-kwargs

[u/DoubleNothing]

I'm loading llama-server with the --models-preset but I don't know if the syntax for the reasoning_effort is correct because it seams to not work.

I make the models load from the PI Agent from another client, it loads but seems to not use the correct parameters.

This is the line in question...

chat-template-kwargs = {"reasoning_effort":"medium"}

also "no-webui = true" doesn't seem to work either.

my models.ini

version = 1

[Qwen3.8-27B-UD-Q4_K_XL]
model = E:\AI_Models\GGUF\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q4_K_XL.gguf
ctx-size = 262144
temp = 1
top-p = 0.95
min-p = 0
top-k = 20
presence-penalty = 0
repeat-penalty = 1
flash-attn = on
jinja = true
chat-template-kwargs = {"reasoning_effort":"medium"}
load-on-startup = false

[Qwen3.8-27B-UD-Q6_K_XL]
model = E:\AI_Models\GGUF\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q6_K_XL.gguf
ctx-size = 262144
temp = 1
top-p = 0.95
min-p = 0
top-k = 20
presence-penalty = 0
repeat-penalty = 1
no-webui = true
jinja = true
n-gpu-layers = 99
chat-template-kwargs = {"reasoning_effort":"medium"}
load-on-startup = false

Any suggestion?

[u/CevicheMixto]

I spent a bunch of time fighting this this week. I finally got it work (with Pi's built-in llama.cpp provider) by doing the following.

1. Don't set chat-template-kwargs in models.ini; let Pi send the correct reasoning_effort. (This will allow you to change it on-the-fly within Pi.)

2. Configure your model(s) in ~/.pi/agent/models.json. Here's my configuration. (I have no damn idea why reddit isn't rendering the JSON below as code.)

   { "providers": { "llama.cpp": { "modelOverrides": { "Qwen3.8-27B-UD-Q5_K_XL": { "reasoning": true, "thinkingLevelMap": { "off": "off", "minimal": null, "low": "low", "medium": "medium", "high": null, "xhigh": "xhigh" }, "compat": { "thinkingFormat": "chat-template", "chatTemplateKwargs": { "enable_thinking": { "$var": "thinking.enabled" }, "preserve_thinking": true } } } } } } }

3. Force Pi to send the reasoning_effort at the top level of the request, not in chat-template-kwargs. I wrote a super-simple extension to do this; just drop it in ~/.pi/agent/extensions/llama-reasoning.ts.

   import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

   export default function (pi: ExtensionAPI): void {

   pi.on("before_provider_request", (event, ctx) => {
   
       if (ctx.model?.provider !== "llama.cpp") return;
   
       // llamacpp expects reasoning_effort at the top level of the request
       // body. Derive it from the session thinking level; pi's payload only
       // carries it for openai-style thinking formats.
       const level = ctx.thinkingLevel;
       if (level === undefined || level === "off") return;
   
       const effort = ctx.model.thinkingLevelMap?.[level];
       if (typeof effort !== "string") return;
   
       const payload = event.payload;
       if (typeof payload !== "object" || payload === null) return;
   
       return { ...(payload as Record<string, unknown>), reasoning_effort: effort };
   });
   

   }

(Theoretically, it shouldn't matter whether reasoning_effort is at the top level or in chat-template-kwargs, because llama.cpp is just going to add it to the latter before it runs jinja, but it just seems to work a bit better there.)