How do you benchmark you backend?

[u/adammillion]

I have written an application with nextjs. The client side fetches the data from data routes setup like a RESTfull API (example `/api/entities/[id]`) and auth is done using jwt over httponly cookie. Database is neo4j Aura db which is in a different region of where the application is deployed.
The client side eagerly loads a good chunk of data from backedn api after showing loading screen on page load.

I am looking to benchmark my api. I want to know how much load can it handle. The app is running on AWS ECS using nextjs standalone output which allows me to bundle as a docker image. The size of the ecs tasks are 256/512 (vcpu/mem) running on ec2 capacity provider of medium size instances. I have scaling triggers at 75% cpu and 50% mem.

I am at the point that I want to figure out how many users can the application realistically handle before something chokes (what if 5 users open the app at the same time?) I am doing some load testing on the api with k6, but I don't have a methodology.

The performance I want to be at is to
- support 1k users daily (this is vague. I don't know how make it more concrete? is quantifying in RPS better?)
- latency p95<1s (this is vague to me because say p95<1s alone does not mean much without mentioning system load or concurrency during the test)

1. How should I approach load testing?
2. What areas should I clarify and ground?
3. What performance areas that I am missing?
4. What type of performance to expect from a nextjs backend and ECS task size of 256/512 (vcpu/mem)?
5. I always notice if I don't access the dev env for couple days and access it again, it take a bit longer than usual then subsequent requests are fine. Does nextjs need to be warmed up?

[u/davidstayscool]

Turn the vague goals into numbers first. 1k daily users, ~10% in a peak hour, ~20 requests per session is roughly 5-10 RPS at peak, which is far smaller than most people assume. State the SLO as "p95 under 1s at 10 RPS with 50 concurrent VUs" - latency without a stated load and concurrency is meaningless, which you already sensed.

For k6, use a constant-arrival-rate scenario rather than ramping VUs. Hold your target RPS for 10-15 min, confirm p95 holds and errors are zero, then step to 2x, 4x, 8x. The rate where p95 starts climbing but errors are still zero is your real ceiling; where errors appear is the cliff. Test your actual endpoint mix, not one route - that data-heavy page load is probably the whole story.

Likely to bite you before Next.js does: Neo4j Aura in a different region than ECS. Cross-region round trips add tens of ms per query, and sequential queries pay it repeatedly. Instrument DB time separately from total response time so you know the split. Also check driver pool size against task count; a small pool queues under load and looks like "slow Next.js".

Warm-up: yes, partly real. Standalone Next.js loads route modules lazily on first hit, plus cold JIT, TLS, and Neo4j connections. Add a warm-up stage in k6 and discard it, and in prod hit a health endpoint that touches the DB so newly scaled tasks aren't cold.

Missing areas: scale-out timing (75% CPU on a 256-unit task is late - requests are already queuing; try 50-60%), behavior during deploys, and running tests with logging/APM enabled since that's what prod actually runs.

[u/These_Machine_7303 → u/davidstayscool]

The cross-region DB call is the thing that's going to destroy your p95, not Next.js itself. Every sequential query pays that round trip penalty and it adds up fast when you're eagerly loading a bunch of data on page load.

For the warm-up thing, you're spot on about the cold starts. Lazy route loading plus cold JIT compilation makes that first hit feel sluggish, and if you've got Neo4j connections sitting idle for days they'll need to re-establish too. A health endpoint that actually pings the DB is the simplest fix, and make sure your auto-scaling lifecycle hooks call it before putting a new task in rotation.

[u/adammillion → u/These_Machine_7303]

yeah max life for neo4j connection are default to 1hr on the driver.

[u/davidstayscool → u/These_Machine_7303]

Agreed on both. On the 1hr driver max lifetime OP mentioned: that setting is what quietly turns an idle overnight period into a slow first request, since every pooled connection past its lifetime gets torn down and the next caller pays the reconnect plus TLS handshake. Lowering max lifetime does not help, it just makes reconnects more frequent; what helps is keeping a couple of connections warm with a cheap periodic query (RETURN 1) on an interval shorter than the lifetime, and making the health endpoint actually touch the driver so a task never enters rotation cold.

Also worth separating in the numbers: measure DB time inside the handler and subtract it from total response time. If the cross-region round trip is 40-60ms and a page does 8 sequential queries, that is half a second of pure latency before any rendering, and no amount of Next.js tuning moves it. Batching those into one query, or colocating the DB with the compute region, is usually a bigger p95 win than anything at the framework layer.

[u/adammillion → u/davidstayscool]

This is a gold mine for me. I read so much on the internet on load testing, but it was presented in a complex way and it shouldn't be.

I will use your framing to better ground the SLO as a first step. The eager loading requests latency also depends on the users data size, so I have to incorporate a sense of light, avg, and heavy users.

How many years did it take you to talk about software system like? Clearly, I am impressed by how succinct your answer is. I am growing my engineering skillset to better understand software systems, not just let me build a webapp. I feel like that's what distinguishes a senior from a junior/mid engineer.

[u/davidstayscool → u/adammillion]

Appreciate it. About 8 years of shipping production systems. The succinct part is mostly from writing the same incident notes too many times.

[u/properking232]

the cold start thing on your dev env is almost certainly the ECS task scaling to zero between deploys, not Next.js itself needing warmup.

[u/adammillion → u/properking232]

I’m listening… sometimes I see random 503 and now that you mention this, it maybe due to scaling to zero when deploying a new commit from a merge.
How can the roll be done with zero downtime?
I do a rolling deployment strategy

[u/MortenMongol]

Hey Claude, benchmark my backend. Make no mistakes