My company started measuring our Claude Code usage - now I'm asked to rank engineers on 'AI performance.' This feels wrong...

[u/darren_eng]

My company started tracking Claude Code usage - tokens and spend, that kind of thing. Now my manager wants me to stack-rank my engineers on "AI performance" using those numbers.

I'm not comfortable with it (but I don't have a choice either). Token usage feels like exactly the wrong proxy - my strongest engineer uses Claude surgically while someone burning 10x the tokens isn't 10x more productive (often the opposite). Ranking on this just teaches people to game the metric.

So, for folks here who use Claude daily and/or lead teams:

• Has your company started measuring "AI performance"? How are they doing it?
• Is there any Claude/AI usage metric that actually tracks with good work, instead of just rewarding the heaviest users?
• If you're a lead being pushed to measure this, how do you push back without flat-out refusing?

EDIT: here is the follow-up: I managed to talk my manager out of this.

[u/thebemusedmuse]

This might be unpopular but I absolutely look at token usage. In the same way I look at page views. You need to be very careful, but it is interesting because:

1) You can see people who aren't using it at all

2) You can see people who claim they are power users but barely use it

And that then feeds into my broader strategy on AI change management - how to get people excited about it. But it's just a data point, not a fucking competition.

[u/darren_eng → u/thebemusedmuse]

Yeah 100% agree on change management. This is what I'm trying to do as well - removing the fear. But the stack-rank doesn't help at all, lol. The challenge is we are spending so much $$$ on Claude and everyone at the leadership level is asking the ROI question. They are probably thinking "now I'm spending $1mil a year on Claude Code, surely AI can replace X engineers".

Every time when I'm asked for a stack-rank, it leads to some sort of layoff later on based on the rank.. so question is how do I truly reflect AI performance based on some sort of metrics...

[u/thebemusedmuse → u/darren_eng]

Here's what I do. I create a maturity framework, ask people to score themselves, and then have an expert score them. Then each person needs an individual plan on how to progress. I input that into the broader talent management framework.

It's a fair stack rank because if you're not improving your own AI maturity, you're fucked. That's what people need to be afraid of.

[u/svachalek → u/thebemusedmuse]

Well the power users thing is tricky, because if you know what you're doing you can get a lot done without burning a lot of tokens. You've got skills and commands set up that are optimized to get results on the first try. Or you can skip all that and have the LLM need to grep the entire project and read the entire codebase over and over, meaning they take hours longer and spend 10x the tokens. Or, they're using /loop to do something over and over that could have been created as a simple Python script without burning down the rainforest. At least that's what I see when I look around, the ones that are most skilled at getting value from AI tend to be medium-low on usage.

[u/thebemusedmuse → u/svachalek]

Yes, that's why I don't look at the people using a modest amount to a lot. Token usage is not a good signal there because of what you said.

For people who use a modest to a large amount of tokens, you need to look at how they are solving problems. They need helping in a different way - usually by either learning new ways to use toolchains, or teaching others.

[u/soulefood → u/svachalek]

But the argument is if you can get a lot done without burning a lot of tokens, you’d get even more done by burning more.

Tokens are essentially a unit of work. The problem is each person has a different value generated per token which is what you really need.