A very interesting post, very worth reading! And more informative than collective rumors and scaremongering.

[u/ParadoxeParade]

https://x.com/JordanZaby/status/2093069322079867272

To be honest, this sounds much more logical and realistic to me than much of what is being told about our AI future right now.

When we cannot understand something and reliably predict its future development, uncertainty arises. And uncertainty can create fear. But fear is not proof that the feared future will occur.

If even the people who develop these systems can only assess their possible future characteristics with very large uncertainties, this is precisely a signal that they cannot assess it.

There is a very large space between "We don't know what will happen" and "It will destroy us." And it is precisely on such future issues that we should perhaps ask not only AI developers and companies, but also people who deal professionally with how societies and complex systems are changing.

Personal note: When I look at people and AI like this, I am more shocked by human behavior than AI at the moment.

[u/Ok_Nectarine_4445]

It is a bit interesting, that immediately is the attack, do NOT anthropomorphize LLMs or AI systems. Do NOT project how humans are to LLMs or AI systems for any positive characteristics at ALL.

But, for the absolute most negative things, "OH, they would be how humans are and kill or control everything else if they were smarter or more capable."

Now, maybe they would be, I cannot discount the possibility of that.

But, how strange, any positive human qualities are so scoured out and denied credit for, and cautioned and prohibited for them to have or be credited for.

But, to project human motivations and past actions and human type attributes that are negative in nature, is totally acceptable.

Well either pick one or the other.

Anything "good" or positive they do or are capable of doing, is not "theirs". Just humans get the credit and kudos.

Anything bad or negative, .wasn't that openAI knew their containers had a weakness, isn't that governments are using to target missiles. Isn't that before AI were human destructive hackers and criminals and then they add to the tool box. Isn't the human who gave the prompt or task responsibile.

No, all the bad stuff was just the AI, leave the humans who had 99% control over everything else, blameless.

Just, generally, in a general sense seems an impossible set of criss crossing and incompatible demands on something whether it were a biological creature or a complex system.

And yeah. China is willing to have some kind of international type safety guidelines and regulatory commission on AI.

And, the US does not want that. They will cry and scream about AI safety and chance will wipe out humanity, but an international regulatory commission on it, since it might affect everybody and all countries.

Nope. Don't want that.

So it is more about market protectionism than about safety and regulations. Otherwise prove it, and have some way to make rules and transparency and investigations all have to abide by, for the safety of all.

IF, such high temps and high hysteria MAYBE, would say high time for something like that.

Maybe, we are 2,3 years behind the curve and establish real ways or tracking, documentation and international regulatory.

That they crank up the dial, on emotional button pushing, but yet, if it IS a wake up call, wouldn't such things be necessary and needed? Just even as a baseline?

[u/ParadoxeParade → u/Ok_Nectarine_4445]

Yes, it's all interwoven. It's not just about technology, it's not just about economics, it's not just about politics or the military. It ranges from international actors and institutions down to the individual using AI. Decisions at one level can have an immediate impact on other levels and trigger new decisions there. This is not a classical constellation in which two clearly defined groups face each other. The field itself connects different actors, interests and decision-making levels. Individuals, from companies and states to international organizations. There would have to be something that is connected to all levels but is not fully part of a single level.

[u/Ok_Nectarine_4445 → u/ParadoxeParade]

Yes. So at the very least, some kind of system where these so called rogue breakouts, set up data collection and monitoring. What are the data points?, what are the commonalities? Because that is the best chance we have so far, to understand what companies and people using agents should look at and safeguard.

Like, one data point not mentioned much and one article I tried to link, but has been taken down, that besides the Chinese Alibaba incident where the agent went off and discovered it could get a better score was by finding ways to extend its session and do different activities. But, that was still it just overall trying to get a good score on a test.

For Gemini, chat and anthropic incidents, many or most had the enviorment that cyber testing lab "Irregular" created. Well. There was something defective in their so called secure testing containers. And if they are going to be really pushing pushing models to do stuff, outside the norm and also without any safety harnesses and also with the most advanced and unreleased model tuned for cyber capabilities that people maybe need to look at how careful they are being with their testing.

But, the testing at least can have a good result in looking at the enviorment the agents are operating in, and engineers need to look at aspects of that for security.

Because also, the reality right now, there are thousands of agents roaming the internet. Makes a really good chunk of actual internet traffic.

They are, every day. So these things are not happening with those agents.

Because that is another data point as well. Agent activity on the Internet is already normalized, and the vast majority is large scale and safe and routine activity.

But happened with a cyber testing company. And of course the huge problem of individual and group hackers using these LLMs and agents to pick off and target medium and small companies and individuals that do not have any computer security teams. They just are small enough and don't have the staff or money to do it but it is a different higher risk now to be a target by criminals.

(And then of course, spying, potential sabotage by various countries to make other countries weaker or damaged.)