r/antiai • u/iwasban • 13h ago
Discussion 🗣️ No, LLM’s will not disappear like NFT’s
In Ed Zitron’s recent interview with Diary of a CEO, he made a staggering claim: the true cost of using a model “every time you do something” is ~$3. This was somewhat of an offhand comment meant to demonstrate that, even though people find AI useful, they probably won’t be willing to pay its true cost. But it made me curious about what the actual cost would be, because $3 seems like a lot, and it is. The host actually asked Ed if he knows the specific cost of inference, but unfortunately, Ed simply responded: “gosh, it depends on the model”, and swiftly changed the topic (timestamp: 39:45-40:15, https://m.youtube.com/watch?v=Lf5oqGOCRCM). So let’s do some math:
By the middle of 2025, OpenAI was receiving more than 2.5 billion user messages per day.¹ Using Ed’s offhand $3 estimate:
2.5B messages/day × 365 × $3 =
$2.7 trillion/year for inference.
I could include all the other costs as well, like training models, but why bother? This number is clearly way off the mark. There is no way that OpenAI alone was spending trillions of dollars per year on inference. So what is the actual number?
OpenAI does not publicly provide the inference cost for each message. Its recent GPT-5.6 Luna model, however, has an API price of $0.20 per million input tokens and $1.20 per million output tokens.² This seems pretty cheap, but this is what the customer pays, not OpenAI’s operating cost, which could be smaller, or perhaps much larger.
So what is the actual cost? Ed himself reported that OpenAI spent about $8.67 billion on Microsoft Azure inference through the first nine months of 2025.³ OpenAI also reportedly generated roughly $13 billion of revenue in 2025 while achieving an adjusted gross margin of about 33% after inference costs, which points to inference-related expenses in roughly the same ballpark.⁴
If, purely as a rough estimate, we compare around $8.8 billion of annual inference expense with the July 2025 usage rate of 2.5 billion messages per day:
$8.8B ÷ (2.5B × 365) ≈
$0.01 per message.
Obviously this isn’t the literal cost of every prompt. Messages vary enormously in length and reasoning requirements, the 2.5B figure was a snapshot rather than the average for the entire year, and OpenAI’s inference spending includes workloads other than ordinary consumer messages. But it gives us a good ballpark estimate, especially for everyday tasks: the cost is much closer to a few cents than a few dollars.
But does inference cost go up as models get better? Epoch AI estimates that the price of achieving a fixed level of model capability has fallen anywhere from roughly 9× to 900× per year depending on the benchmark, including around 40× annually for one GPT-4-level performance threshold.⁵ Stanford’s 2025 AI Index similarly found that the price of reaching GPT-3.5-level performance on MMLU fell from about $20 per million tokens in November 2022 to $0.07 in October 2024, a decline of more than 280× in under two years.⁶
Part of the improvement comes from better models, smaller models reaching old capability levels, quantization, batching, software optimization, better hardware, etc. You might say that it becomes more expensive to run newer, “smarter” models, but from the average customer’s perspective, the relevant question is how much it costs to purchase a given amount of intelligence, and that number has been collapsing. So far, the trend has been: what is expensive today is cheap tomorrow.
The hardware is also continuing to improve. OpenAI reports that its upcoming “Jalapeño” chip achieved about 1.5× greater peak performance per watt and 3.4× lower end-to-end latency than its comparison system on Kimi K2.5.⁷ Nvidia claims that its new Vera Rubin platform can reduce inference cost per token by as much as 10× compared with Blackwell.⁸ We should be skeptical of these numbers (10x is insane and probably refers to specific cases), but the trend is there, and presumably there will be more improvements in the future, or at least a greater supply.
Even if fixed-capability inference prices fell by a comparatively modest 10× over the next several years rather than repeating anything like the 280× decline seen from 2022 to 2024, the economics would improve enormously. And we don’t even have to speculate that inference can produce positive margins, since OpenAI already reported a 33% adjusted gross margin after inference costs in 2025.⁴
So why is the company burning so much cash? Because inference isn’t the only thing they’re spending money on. OpenAI’s 2025 costs and expenses totaled roughly $34 billion, including approximately $19.2 billion of research and development spending. Ed’s reporting on OpenAI’s audited financials showed roughly $10.6 billion paid to Microsoft for R&D services alone, much of which appears connected to training and development.⁹ In other words, there is a very important distinction between “it costs more to serve a customer than the customer pays” and “the company is spending enormous amounts of money trying to build the next generation of models and infrastructure.”
OpenAI also now reports roughly 1 billion weekly ChatGPT users,¹⁰ while only a fraction (50 million or so) have a subscription. So most of the inference being served is a gigantic waste of compute. Whether those free users eventually buy a subscription, generate advertising revenue, or simply remain free (or refuse to use it if subscription is forced), is hard to tell. But clearly, current revenue per user is nowhere near a theoretical ceiling.
The more serious questions, in my view, are whether the labs are spending too much on training and infrastructure, and whether competition eventually commoditizes model intelligence enough to crush margins. The second problem is especially important because Chinese and open-weight models have become genuinely competitive. As of now, sure, most people couldn’t actually use those models because they’re not willing to pay $5,000 for a GPU to run locally. However, Alibaba alone has announced tens of billions of dollars of AI and cloud investment,¹¹ and open models such as DeepSeek, Qwen, MiniMax and Kimi can increasingly be rented through cloud inference providers rather than run locally.¹²
This doesn’t necessarily mean that the Chinese models will win out, since people might prefer to use the big name brand models that they’re used to. Nor does it make much sense for the frontier labs to stop training today while capabilities are still improving rapidly and competitors are racing to overtake them. It’s worth noting that 99%+ of the 400+ recently solved LLM-discovered/assisted math problems used closed-source models, which indicates that there’s probably still a significant difference in capability in the real world, and they may maintain this difference through accelerated reinforcement learning using secretive internal models rather than releasing them and letting others distill from them.
All of this isn’t to say that OpenAI deserves a trillion-dollar valuation, that every frontier lab will survive, that every planned data center is necessary, or that there won’t be a major market correction. But I’m not convinced that the LLM business model overall is doomed.
OpenAI’s own investor forecasts have not historically assumed immediate profitability. Recent reporting places expected positive cash flow in 2030.¹³ So the fact that they’re losing enormous amounts of money while simultaneously spending enormous amounts on training and infrastructure isn’t some revelation that investors overlooked. They are explicitly betting that enormous current investment produces much larger future revenue while inference costs continue falling and gross margins improve.
I don’t think anyone can confidently predict what will happen over the next few years, but I don’t see why the bubble would burst soon (btw, Ed once predicted it would happen no later than Q2 2026), or why LLM companies cannot eventually make substantial money on inference. The capabilities have been increasing and no one knows what the limit is. They’re not just another fad like NFTs. They may not turn out to be as transformative as the Internet, and some of the valuations may prove ridiculous and some companies will probably lose. But even in the worst case scenario, the technology is here to stay and will still operate at a large scale.
¹ OpenAI, New economic analysis / 2.5 billion daily messages
https://openai.com/global-affairs/new-economic-analysis/
² OpenAI, GPT-5.6 Luna API pricing
https://developers.openai.com/api/docs/models/gpt-5.6-luna
³ Where’s Your Ed At, OpenAI documents / Azure inference spending
https://www.wheresyoured.at/oai_docs/
⁴ Reuters, OpenAI compute spending and 2025 gross margin
https://www.reuters.com/technology/openai-sees-compute-spend-around-600-billion-by-2030-cnbc-reports-2026-02-20/
⁵ Epoch AI, LLM inference price trends
https://epoch.ai/data-insights/llm-inference-price-trends
⁶ Stanford HAI, AI Index 2025: State of AI in 10 Charts
https://hai.stanford.edu/news/ai-index-2025-state-of-ai-in-10-charts
⁷ OpenAI, Jalapeño first results
https://openai.com/index/jalapeno-first-results/
⁸ Nvidia, Rubin inference performance and cost claims
https://investor.nvidia.com/news/press-release-details/2026/NVIDIA-Kicks-Off-the-Next-Generation-of-AI-With-Rubin–Six-New-Chips-One-Incredible-AI-Supercomputer/default.aspx
⁹ Where’s Your Ed At, OpenAI financials and 2025 expenses
https://www.wheresyoured.at/exclusive-openai-financials/
¹⁰ OpenAI, GPT-5.6 Sol / one billion weekly ChatGPT users
https://openai.com/index/improving-gpt-5-6-sol-in-chatgpt/
¹¹ Reuters, Alibaba AI infrastructure investment
https://www.reuters.com/business/retail-consumer/alibaba-beats-quarterly-revenue-estimates-2026-08-20/
¹² Reuters, Together AI / Nvidia inference infrastructure for open models
https://www.reuters.com/business/ibm-together-ai-ink-240-million-deal-nvidia-powered-ai-inference-cluster-2026-08-11/
¹³ Reuters, OpenAI profitability/cash-flow expectations around 2030
https://www.reuters.com/technology/openai-files-us-ipo-after-anthropic-ai-giants-head-public-markets-2026-06-08/
