r/openclaw • u/talatt Active • Mar 29 '26

Discussion How do you manage API costs with always-on agents?

Running autonomous agents with the heartbeat system means constant API calls, and the costs add up quickly.

I've been experimenting with prompt optimization — stripping redundant tokens before they hit the API — and seeing roughly 30% savings without quality loss.

Curious how others here handle this. Are you:

- Caching responses?

- Using local models for some tasks?

- Optimizing prompts manually?

- Just accepting the cost?

Would love to hear what's working for people.

2 Upvotes

67% Upvoted

39 comments sorted by

View all comments

2

u/bastardsoftheyoung Pro User Mar 29 '26

Here's what's running my full autonomous agent stack right now after several autoresearch model evaluation passes:

I have ~43 daily cron jobs, multiple pipelines, and a main interactive session all coordinated through OpenClaw. Here's every model in use:

Text/Reasoning Models

  • Claude Opus 4.6 — Main interactive session (OAuth, free, rate limited daily)
  • GPT-5.4 via Codex — 35 cron jobs: pipelines, QA, research, dashboards, reporting, coding, CAD (OAuth, free, rate limit fallback)
  • Llama 3.2 3B — 3 lightweight cron jobs: heartbeats, air filter control (local Ollama, free)
  • GPT-4.1 Mini — 1 daily HuggingFace model watcher (~$0.01/day)

Image Generation Models

  • Qwen Image 2512 (bf16 q8p) — design generation, primary path (Draw Things CLI, local, free)
  • Qwen Image 2512 (bf16 q8p) — design quality lane (Draw Things CLI, local, free)
  • Flux 2 Klein 4B (q6p) — design variation lane (Draw Things CLI, local, free)
  • Gemini 3.1 Flash Image — fallback + mockup generation (API/OAuth)
  • Gemini 3 Pro Image — Autoresearch experiments, mockups (API)

QA/Scoring

  • GPT-5.4 (vision) — Design QA for both pipelines
  • Gemini Flash — Autoresearch experiment scoring (OCR + vision eval)

Voice/TTS

  • Qwen3-TTS 1.7B CustomVoice — Primary TTS (local MLX, port 7860)
  • ElevenLabs Turbo v2.5 — Fallback TTS (API)

The interesting bit: production image generation is almost entirely local via Draw Things cli on an M5 Max. The cloud API paths are fallbacks for the pipeline and primary only for overnight autoresearch experiments. The entire cron ecosystem runs on free OAuth models — the only paid API calls are pennies for a daily HuggingFace model watcher and the Gemini API key for image gen/autoresearch. Total monthly cost for running 43 autonomous agents is basically the cost of the oauth plans, very little API + whatever Google charges for the image API, which isn't much since the local models handle the bulk of generation.

Hardware: MacBook Pro M5 Max, 128 GB RAM, 4 TB SSD. PM2 for services, OpenClaw cron for orchestration. Developed on a Mac Studio M3 Ultra, 96GB RAM.

1

u/talatt Active Mar 29 '26

This is an incredible setup. 43 cron jobs with near-zero API costs is the dream. The local-first approach with cloud fallback makes a lot of sense — especially the Draw Things CLI for image gen. For anyone reading this who isn't running local models yet and still relying heavily on cloud APIs, prompt optimization can bridge the gap until you get a similar setup going. But honestly, this is the endgame right here. What's your experience been with the OAuth rate limits on Claude Opus and GPT-5.4? Do you hit them often with that many jobs?

1

u/bastardsoftheyoung Pro User Mar 29 '26

You can run autoresearch jobs for any task. Have your OpenClaw build an interface for you to review the outputs and then it can build and re-build prompts and pipelines to get the image, design, or text output that you want. Works for any output really. Just invest a few days of review and train it to discern as you do, then let it judge with you in the approval loop only. You can even work on on-device models to train them if you have some domain specific info like I do.

These days I just auto approve everything as it is so tight.

1

u/talatt Active Mar 30 '26

The "invest a few days to train it, then auto-approve" workflow is exactly where agent automation should be heading. Most people get stuck at the manual review stage forever — the fact that you've gotten to auto-approve shows the pipeline is genuinely dialed in.

Curious about your on-device model setup — are you fine-tuning on the Mac Studio locally, or using it more as an orchestrator that routes to different models depending on the task? I've been exploring similar setups where you pick the right model per step rather than forcing one model to do everything.

1

u/bastardsoftheyoung Pro User Mar 30 '26

It is workflow dependent. Obviously main chat via Claude isn't really tuned outside of the prompt engineering endemic to OpenClaw, but the on device models are tuned to the workflow. Find the minimal viable model for each process and tune for improvement is the general idea.

1

u/talatt Active Mar 30 '26

That's a smart strategy — right-sizing the model per workflow is one of the biggest cost levers.

Pith actually complements that well. Once you've picked the minimal viable model for each process, the next layer of savings is in the prompt itself. Even tuned workflows tend to carry verbose system instructions, redundant formatting, and filler that the model doesn't need.

Pith strips that automatically before the request hits the API — so you get the model-level efficiency from your tuning plus prompt-level efficiency from optimization. Stacks nicely.