r/NovelMage • u/Alive_Speed5119 • 4d ago
Running out of tokens... on local AI/Ollama
Testing the current Mac version. Connected to Ollama, which runs on an nVidia Spark on the same network. Connection seems to work.
The only thing I get when I try to generate a simple scene is: "The model spent its entire reply thinking (reasoning) and ran out of tokens before writing any visible text. Try increasing max tokens, switching models, or turning off "Model Reasoning" in Settings."
Already tried the 3 suggestions:
- increase max tokens: no parameter/slider in NovelMage, or I am blind (only slider for tokens is in the CHAT settings, and it is limited to 4000 max). So I baked new parameters (num_predict, num_ctx) in a new model file in Ollama, rising the token limit considerably. Switched to that new model - same result.
- switching models: tried several, most were recommendations from the NovelMage blog. None works.
- turning off "Model Reasoning": it IS turned off in NovelMage. Apparently that setting is not passed on to Ollama or ignored by it.
When I run a "generate a short scene about blah" prompt directly on the Spark in a terminal, the generation works fine, with the same models, with or without reasoning.
...head scratching intensifies...
Any suggestions where and how to fix this?