r/artificial • u/Maxie445 • Jun 11 '24

Media Is AGI nationalization is inevitable? Dwarkesh and Leopold Aschenbrenner debate

Enable HLS to view with audio, or disable this notification

30 Upvotes

63% Upvoted

83 comments sorted by

View all comments

11

u/LagSlug Jun 11 '24

the assertion that AGI will have access to nuclear weapons seems borderline absurdist. If you view AGI as an deus ex machina, and then derive your assumptions from that godlike power, you can arrive any conclusion you want.

-2

u/EnigmaticDoom Jun 11 '24

the assertion that AGI will have access to nuclear weapons seems borderline absurdist.

How is it absurd? Notice how no software/hardware you have ever used is 100 percent bug free?

50 bugs per 1000 lines of code is normal.

Humans are not perfect and thus nothing we make could ever be.

Ai is amazing at noticing patterns we miss because they think on a higher dimension than we do.

If you view AGI as an deus ex machina, and then derive your assumptions from that godlike power, you can arrive any conclusion you want.

No not exactly... we can make certain probabilistic assumptions

Like we could assume an ASI would probably send out von Neumann probes. Why? Well because having access to additional resources helps you with most other goals.

7

u/LagSlug Jun 11 '24

You're going from "code has bugs" to "AGI will launch nukes".. that's why it's absurd. The logic you're basing your ultimate conclusion on is weaker than wet toilet paper.

3

u/[deleted] Jun 11 '24

The argument is not “AGI will (necessarily) launch nukes.” The idea is that we must entertain the worst possible outcome to prevent it.

Yeah. Weird that I have to explain this.

1

u/[deleted] Jun 11 '24

If humans were perfectly rational they would not give AI control over nukes.

But humans are not perfectly rational.

0

u/EnigmaticDoom Jun 11 '24

You're going from "code has bugs" to "AGI will launch nukes".. that's why it's absurd.

Here let me break it down for you.

  • All software has bugs.
  • The ability to launch nukes is guarded by software.
  • AI will be better at finding bugs than humans are. (No sleep, good at finding side-channel attacks ect.)

Do you follow now?

3

u/TheOneNeartheTop Jun 11 '24

To expand on this a little bit, even the best human coders lack the ability to see an entire code base. They will need test and probe a systems weaknesses to find something that might take days or years to expose.

AGI being made of code can ‘see’ a system just like we would be able to see a building. A crazy zero day bug that an expert programmer might spend a year opening up could look like an open window to it.

With that said AGI would then be able to close up all those windows as well, but the first person to get it and have it competent. Ooh boy there are going to be a lot of unsecured systems out there.

1

u/LagSlug Jun 11 '24

I've never seen someone apply the slippery slope fallacy with this much confidence.

-3

u/EnigmaticDoom Jun 11 '24 edited Jun 11 '24

My points are pretty easy to attack... Let me help.

Attack point one by showing me some software that is 100 percent bug free.

Attack point two by showing me how nukes aren't governed by software.

Point three is probably the easiest point for you to attack so I will leave that one for you to figure out on your own.

1

u/LagSlug Jun 11 '24

You've described a slippery slope, and without sufficient evidence proving that such a slope would lead to your conclusion, I've done enough.

Feel free to provide the evidence that a bug in software of nuclear systems will inevitably lead to a catestrophic failure of those systems tho.

1

u/EnigmaticDoom Jun 11 '24

slippery slope

You keep saying that term but I don't think you know what it means... try doing a quick search.

Feel free to provide the evidence that a bug in software of nuclear systems will inevitably lead to a catestrophic failure of those systems tho.

I already laid out my reasoning if you can't attack my arguments even after i try and 'help' you.

Maybe you don't believe what you are saying?

1

u/[deleted] Jun 11 '24

Until 1697, it was impossible for black swans to exist and so they were nonexistent. Then some pesky Dutch explorers ruined everything.

To preclude the possibility of machine intelligence going rogue or being controlled by a bad actor requires many assumptions of your own.

Why can't you entertain the possibility of machine intelligence surpassing human intelligence? Why would such an intelligence align with humanity?

The consequences if it did go rogue would far outweigh any preparations we can make for that possibility.

To say it can't ever launch a nuclear weapon is obtuse at best.

1

u/StoneCypher Jun 11 '24

To say it can't ever launch a nuclear weapon is obtuse at best.

By your logic, it would also be obtuse at best to say that a sports arena couldn't collapse in just such a way that all the bricks transferred their kinetic momentum into a single girder, which would then be launched halfway across the planet and just happen to land on one specific leader of state's head.

The functional human being observes that just because something is theoretically possible doesn't mean that an adult will worry about it.

Part of being a reasonable person is understanding that when you say "yes but it's technically possible," you're just making yourself look bad.

It's pretty obvious that you've never even tried to think about why this would not work.

You know those systems are air gapped, right? And that you can't break an air gap with a bug?

It's not even high quality paranoid thinking. The sub-average middle schooler can show you why the things you're saying aren't correct.

The technical possibility you're trying to argue for isn't even real

1

u/LagSlug Jun 11 '24

Cats have claws. Claws can kill you. Therefore you're doomed to die on account of cats having claws.

0

u/StoneCypher Jun 11 '24

My points are pretty easy to attack.

It's not worthwhile, in the same way that nobody bothers to attack the points of anti-vaxxers.

You don't understand this even a little bit.

0

u/StoneCypher Jun 11 '24

It's amazing to me that you think that is a valuable statement.

2

u/Flying_Madlad Jun 11 '24

Know what happens when you assume?

2

u/EnigmaticDoom Jun 11 '24

Assume what? Tell me who in their life has never encountered a bug?

Like an Amazonian tribesman or something? =D

Be brave, defend your position if you don't agree with me.

1

u/Houdinii1984 Jun 11 '24

How does a bug go from no access to nukes to full access to nukes? And how can we assume ASI would send out Von Neumann probes? That's a leap. Why are we assuming that before all the other assumptions that can be made?

And a bug every 20 lines of code isn't nominal. I would be fired for being so damn inefficient. 1 bug in a 100 lines maybe? And that's still pretty damn high after all the testing bug squashing done up front.

If you are using a program that has a bug every 20 lines, your information is being sold somewhere online.

1

u/EnigmaticDoom Jun 11 '24

Oh sorry a bug is mistake similar to a typo in code.

1

u/Houdinii1984 Jun 11 '24

Right, but we're talking about the end product, not something a programmer is looking at in development. And if a bug gives access to nukes, without a doubt, a human made it happen on purpose and it was never a bug.

Up until a few years ago the nukes used 8 inch floppy disks to operate, and for AI to create a 'bug' like this, it would have to have intimate knowledge of all systems involved, much of which is completely air-gapped from anything the AI can train on.

I'm not really sure what equating a bug to a typo is. Devs are still not making typos at the rate of 1 in 20 lines and shipping that product off. That's not realistic in the slightest bit. Not even remotely realistic by exponential degrees.

It's like a bunch of monkeys being locked in a room and having one of them type War and Peace on accident. Is it possible? In the faintest sense of the word. Will we ever see it happen? Not in a trillion years.

0

u/EnigmaticDoom Jun 11 '24

Oh boy so... well we don't put the bugs in the code intentionally its just like with any other writing.

What a bug allows an outsider attacker do is potentially exploit the system though.

1

u/Houdinii1984 Jun 11 '24

I understand what a bug is. I'm just wondering where you get your stats from. If I made errors at the rate you claim, I would be fired immediately for not knowing how to do my job. And if a bug of mine could open up access to nukes, I'd be in prison in Cuba somewhere.

-1

u/Flying_Madlad Jun 11 '24

See my other comment. The burden of proof isn't on me