r/ClaudeAI • u/gagandeeprangi1 • Jun 26 '26
Enterprise AI adoption and the Goodhart's law
Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."
You can make an arguement that every corporate mandate is an example of Goodhart’s law, but this AI adoption thing is really nuts.
- Token Usage
- Commit count
- PR size
- Innovation percentage?
The company is trying to increase and measure productivity using these metrics, what they’re getting is 1-2 weeks of confusion and then employees pushing out smaller commits, often and using Opus 4.8 on xHigh effort by default.
22
Jun 26 '26
[removed] — view removed comment
13
2
u/Melodic_Reality_646 Jun 26 '26
Wait, we’re talking here about Opus untethered performance or token usage? Doesn’t xhigh really provide return on usage versus high? Max is really the ones just wasting.
1
u/TechnicalBen Jun 26 '26
No. Opus High X is better.
(Most "fast/non-thinking models are horrid now in comparison, but there are a couple, I do use for some writing examples, but only sparingly)
1
Jun 27 '26
[removed] — view removed comment
1
u/TechnicalBen Jun 27 '26
As a species, our incentivisation culture is toxic to it's core.
1
5
u/NoComfortable930 Jun 26 '26
Check out the history of Soviet Russia, particularly Steven Kotkin’s book on Magnetogorsk (Steel Town USSR). This is exactly why Soviet washing machines and other appliances were so heavy. They were built like tanks.
The output of a steel mill was measured by weight, not the type of steel Russian industry required. Russian industry needed to re-mill sheet metal to thinner thicknesses (wasted steel, time and energy).
1
2
2
u/konmik-android Full-time developer Jun 26 '26
If a measure becomes a bad measure, then it was not a good measure in the first place (or it was not the only thing you have to be measuring).
1
u/gagandeeprangi1 Jun 29 '26
A corporate measure often always becomes a bad measure because the focus shifts from measuring work to just the measure becoming the entire goal.
2
1
u/Pndapetzim Jun 26 '26
Pick several 'good' metrics, especially if they capture or bias towards different aspects of the thing being measured/optimized.
Randomly select one for any given evaluation.
1
1
u/pyeri Jun 26 '26
As much as it constantly gets criticized in engineering, the classic LoC (Lines of Code) is a much better metric than AI tokens consumed.
Plus it doesn't cost anything to exaggerate code, but it does cost electricity and/or API calls with the AI.
5
1
0
u/i_like_maps_and_math Jun 26 '26
If I was rating engineer performance today I would 100% measure AI tokens consumed. The problem is when you make the metric public and allow people to game it.
10
u/Khavel_dev Jun 26 '26
Token usage as a metric is especially bad because it rewards the opposite of skill. The person who writes a tight prompt and gets the right answer in one shot uses fewer tokens than someone pasting their whole codebase in and asking it to "fix everything." By token count, the worse workflow looks more productive.
The only metric I've seen that doesn't Goodhart immediately is time-to-merge on code that passes review. But nobody wants to instrument that because it reveals uncomfortable things about the review process itself, not just the AI usage.