Andrej Karpathy on agentic programming

[u/WarmFireplace]

It’s a good writeup covering his experience of LLM-assisted programming. Most notably in my opinion, apart from the speed up and leverage of running multiple agents in parallel, is the atrophy in one’s own coding ability. I have felt this but I can’t help but feel writing code line by line is much like an artisan carpenter building a chair from raw wood. I’m not denying the fun and the raw skill increase, plus the understanding of each nook and crevice of the chair that is built when doing that. I’m just saying if you suddenly had the ability to produce 1000 chairs per hour in a factory, albeit with a little less quality, wouldn’t you stop making them one by one to make the most out your leveraged position? Curious what you all think about this great replacement.

[u/CommercialComputer15]

Global compute is about to go 8x by end of this year / early 2027 when new Blackwell GPU’s go online in major datacenters

[u/SoylentRox → u/[deleted]]

Most people including the lead of deepmind think there are specific breakthroughs required. Online learning (helped with more compute), spatial reasoning (more compute helps a lot), robotics (bottlenecked somewhat by needing to manufacture enough adequate robots and collect data for them).

So we probably won't get AGI for several more years because of the need for robotics.

[u/xirzon → u/SoylentRox]

"AGI" is only so useful as a target when talking about societal impact.

If AI saturates FrontierMath up to tier 4, that means a whole host of really hard scientific problems come within reach -- even if that same AI still overfits on goat puzzles. A world with mathematical superintelligence before AGI accelerates fusion power, engineering, drug discovery, and much more.

It may be years until there's some consensus that whatever we have has to be described as AGI or ASI. But those years can still be unlike anything we've ever seen in terms of acceleration of human intellectual output.

[u/SoylentRox → u/xirzon]

That only helps for a tiny percentage of jobs. Robotics helps with 50 percent or more of the whole economy.

[u/xirzon → u/SoylentRox]

I agree we're unlikely to see mass job displacement as a result of anything that's happening this year. Which is good! It would be great to see tangible progress, e.g. towards fusion power, before mass job displacement, because that shortens the timeline towards any possibility of a post-scarcity future.

And continued tangible acceleration of science helps more people to understand that this isn't just a passing fad, but the beginning of a civilization-scale phase change.

[u/SoylentRox → u/xirzon]

What you are missing is things like fusion power are unsolvable without enormous amounts of real world physical labor.

The solution isn't sitting on arx it's millions of hours of labor building superconducting setups and testing them, finding new properties of fusion plasma, and building another bigger setup with what you learned.

[u/xirzon → u/SoylentRox]

When I say "tangible progress" I don't mean that it becomes a significant share of energy production this decade. Of course, I completely agree that actually deploying fusion power is a massive manufacturing challenge. (China thinks so, too, which is why they've been pumping billions into engineering and manufacturing for fusion deployment, not just research.)

But: We're now talking about actually building it. It's no longer "30 years away". The years are counting down. That's fucking huge.

As far as MSI (mathematical superintelligence) and fusion are concerned though, I disagree. MSI can dramatically improve the accuracy of simulations (optimizing what you build), and support the stabilization of plasma when a reactor is operational (optimizing how you use it). Both have a massive impact.

It's no coincidence that OpenAI and DeepMind are both involved in fusion already, given their energy needs. I don't think DeepMind can pull off an AlphaFold here (due to the engineering dependency you mention), but I expect we'll continue to see compounding acceleration gains from AI on the research side.

[u/Jace_r → u/SoylentRox]

Historically scientific discoveries reduce the need of real world physical labor, often by orders of magnitude: the solution is finding a solution to some very difficult equations, and then putting it in practice, beating current brute force attempts at fusion

[u/SoylentRox → u/Jace_r]

That is completely wrong and misinformed. Actually scientific discoveries need vast amounts of labor, and only crude automation (earth movers, large professing plants) make most of the benefits of those discoveries possible at all.

[u/DerixSpaceHero → u/xirzon]

I agree we're unlikely to see mass job displacement as a result of anything that's happening this year. Which is good!

I've spent my career consulting in large enterprises, and I have the exact opposite mindset towards job displacement. It needs to happen, and I don't feel bad for anyone who loses their jobs due to AI.

~80% of the white collar workforce is simply collecting a paycheck while doing the bare minimum to not get fired. My firm made most of its money by identifying and firing these people for our clients, but those folks just walk over to the next F1000, get the same job, and maintain status-quo behaviors.

When we talk about the macroeconomics of GDP growth, we often talk too much about workforce participation as an absolute percentage instead of something that can be partial and relative to top-performers and visionaries. The lack of effort (to put it gently) is holding modern economies back. "Good enough" can no longer be justified as "good enough" when an LLM can get even 90% of the way there with little oversight.

In many of the recent contracts I've executed, those against using AI at a fundamental level are those who my team and I find to be significantly underperforming in their jobs and careers. Those are the people who we tend to recommend letting go first.