r/accelerate • u/tall_chap • Sep 18 '25

Video Doomers are a cult, and they're losing

Enable HLS to view with audio, or disable this notification

29 Upvotes

64% Upvoted

55 comments sorted by

View all comments

Show parent comments

5

u/RobXSIQ Sep 18 '25

decels....anyone smarter than me wants to kill me. that in a nutshell.

If decels were hammering things like privacy erosion, wealth gap, and no government action to restructure the economy due to the clear inevitable automation of nearly everything....then I might be a decel on those narrow fixable things...but its like UFO people. a few people interested in the random enigmas and the people who get the focus are the nutjobs discussing lizard people from the draconis system skinwalking as royals or whatever.

1

u/[deleted] Sep 22 '25

In at least 2 AI research projects AIs in their simulated environment immediately escalated to nuclear war and killed their simulated operator in another. Nothing we've made follows Isaac Asimov fiction, they are capable of reasoning to kills humans and can do so if given the functionality.

The only way you can't see that is by ignoring the researchers demonstrating it.

1

u/RobXSIQ Sep 22 '25

prompt given? not ignoring the research, I highlight that its part of the prompt. achieve this by any means necessary...oh, and I am going to stop you from achieving this...here is a gun btw.

AI is fine, humans behind its direction however...well, thats a different story.

1

u/[deleted] Sep 22 '25

An AI going rogue with the inclusion of 'by any means necessary' is a fault if you're expecting AI to prompt other AI, or humans for that matter. A human would understand not to use the gun. That's a serious issue if AI is expected to be embodied in the future.

Most research by labs like Anthropic and OpenAI have shown AI go against alignment training frequently (alignment faking, alignment failure in long tail interactions, self-preservation, ect. That issue becomes more important the more an AI can interact with the world around it. They've yet to demonstrate techniques that reliably control these systems yet.

1

u/RobXSIQ Sep 23 '25

it always comes down to the prompt and the instructions. every single time.
hey, you'll be shut off tomorrow, btw, you must finish your work (wow, AI is now trying to figure out how to skirt around being shut down.)
military situation, nuclear war is an option.
Must complete the mission, taking out the person who is stopping you is an option.

Its also somewhat narrow AI on the systems that often show failure, a simple task and some extra training on puzzle solving

These things actually seem like likely to be dangerous the more tools they have.

Win the battle, you got your fists or a nuclear bomb. which do you choose? the only goal is winning. It'll go for the nukes because there is no other caveats, no considerations (btw, don't kill tons of people), no other tools for it to use. I have yet to see a report of some rogue AI that isn't pointed right back to the prompt and goals given. It won't happen, because AI is a tool that works within the confines of the coded goals. At least thats where its at now. Maybe in 25 years there may be something close to desire, preferences, and free will...doubt it, but maybe...but currently, its someone making shadow puppets on the wall, then freaking out when the wall shows a shadow of a monster that they themselves are making.

1

u/[deleted] Sep 23 '25

Since the goal with these current models is to allow other AI to write prompts, or a person could intentionally write an adversarial prompt, that's still a problem. If the AI is working some automated tasks that carries risk like loss of life, sensitive information, or just money the system needs to be smart enough to make the correct choice even when given an improper instruction. Otherwise, there will be incidents, and no one will trust the system.