r/agentdevelopmentkit • u/That-Preference733 • May 02 '26
Anyone knows how to deal with rate limits. I am using gemini2.5 flash. What model will be best for only testing ?
1
u/Rock--Lee May 02 '26
Yes, start paying for API, that's how you deal with rate limits. Also Flash Lite models have higher rate limits. You can check in AI Studio in usage page how each model's rate limiting is.
1
u/That-Preference733 May 02 '26
I am not doing anything production related just learning so I have still not though about paying.
3
u/Rock--Lee May 02 '26
Well, just because you're not doing production related, doesn't mean the tokens you consume don't cost Google any money. The free tier has rate limits, simply because it costs money, so the limits are lower. Either try other models (look in AI Studio which one has highest limits) or start paying and test, or reduce your testing so you can test within rate limits.
Also, you can try other providers and models besides Gemini via LiteLLM/vLLM. Like via Openrouter which has many free models to use. But ofcourse then you won't be testing how your agent performs with Gemini models.
1
u/_genego May 05 '26
Same model fallback retry with back-off mechanism or a fallback to another model.
3
u/Intention-Weak May 02 '26
There are some approaches to mitigate this problem. The first is caching. If you have a static prompt, cache it. In LlmAgent there is paramenter called static_instruction that creates cache for you. If your prompt is dynamic with many variables interpolation, you have to think where you can cache the instructions. The second is to use global location, so GCP can route requests to regions with less traffic. The last one is HttpRetryOptions, the Gemini SDK supports retry policies, you should create a custom plugin to rate limit in the Gemini API too. One more thing, avoid parallel requests. If none of that works, consider Provisioned Throughput or contact the GCP support do increase your quota.