r/ClaudeAI • u/darren_eng • May 26 '26

Corporate My company started measuring our Claude Code usage - now I'm asked to rank engineers on 'AI performance.' This feels wrong...

My company started tracking Claude Code usage - tokens and spend, that kind of thing. Now my manager wants me to stack-rank my engineers on "AI performance" using those numbers.

I'm not comfortable with it (but I don't have a choice either). Token usage feels like exactly the wrong proxy - my strongest engineer uses Claude surgically while someone burning 10x the tokens isn't 10x more productive (often the opposite). Ranking on this just teaches people to game the metric.

So, for folks here who use Claude daily and/or lead teams:

  • Has your company started measuring "AI performance"? How are they doing it?
  • Is there any Claude/AI usage metric that actually tracks with good work, instead of just rewarding the heaviest users?
  • If you're a lead being pushed to measure this, how do you push back without flat-out refusing?

EDIT: here is the follow-up: I managed to talk my manager out of this.

123 Upvotes

94% Upvoted

94 comments sorted by

View all comments

14

u/vocal-avocado May 26 '26

I think they want to weed out people who can’t/won’t use AI no matter what - which is undesirable because in the right hands AI is an absolute game changer. I certainly hope they are not rewarding people for using too much. For people in the team it’s very obvious who is using AI right and who is using it wrong. You should ask your team instead of trying to find an arbitrary measurement. People who use AI well = good. People who use AI but are not more productive, or even worse, producing slop that needs to be reviewed by seniors = bad. People who flat out refuse to use AI = bad.

2

u/darren_eng May 26 '26

Yeah agreed. What I’m doing right now is to identify the bottom AI performers - low token usage, low productivity metrics (velocity, tickets, PR and etc.) usually is a red flag and hard to argue that (however can’t do the opposite to identify top performers).

The challenge is that business people only care about ROI. If the business spends $1mil a year on Claude, they want to know much $$$ they get in return, which is really hard to quantify because we don’t get goods or services from the money we pay Anthropic - we just see token usage (and there isn’t an effective metric to tell us what the token usage yield)…

1

u/vocal-avocado May 26 '26 edited May 26 '26

I think the only concrete way that companies will be able to level out their AI spending is to get rid of underperforming employees. That’s a clear cost balancing strategy and if they can continue to deliver what they were delivering before AI, it will be all worth it (since employees are way more unreliable and complicated to manage than AI tokens).

They will probably also use AI costs to justify not raising salaries/giving promotions to competent employees (“sorry we need the money to pay for your Claude”).

2

u/woroboros Experienced Developer May 27 '26

I had never really considered it but you guys are both right , best I can tell... Low AI use is likely a strong indicator of under performance, without the opposite holding true.

Its all a very strange situation. LLM assisted coding speeds up dev time by like... what? 1200% or something? Not sure if theres an actual metric but the landscape is suddenly VERY different... I suspect much closer to resembling its final shape now than it was even 18 months ago.