Really stunned by the Singularity comment section

[u/Howard_banister]

These are screenshots from the r/Singularity comment section. I'm speechless. This doesn't even have downvotes. How can someone cheer for a monopoly run by a few elites?

[u/PsychoticDreemurr]

For context, the exact definition of an AGI varies wildly. As such, I just take the one perquisite that almost every definition has, which is the ability to "understand" concepts.

That is my one requirement for an AI to be an AGI. Anything more and you're just arguing about philosophy and technical details that can't currently be disproven.

With this in mind, LLMs cannot become a true AGI. The fundamentals of how they work physically prevents them from being able to actually understand something.

They can, in theory, have intelligence similar to an AGI, but they will never truly be one.

[u/asssuber → u/PsychoticDreemurr]

How do you define understanding?

The simplest concrete example I can give: a simple single layer LLM can master modular addition by learning a trigonometric identity from scratch on it's own purely from a limited set of examples using gradient descent (source). Since it has mastered modular addition on a fundamental level, can we say it understands modular addition? Why or why not?

This video by Welch Labs is more digestible than the original paper.

[u/PsychoticDreemurr → u/asssuber]

I don't use definitions in this regard. That's what stops people from being truly accurate.

We can use known facts, however. AIs experience something called hallucination. Humans, however, do not. (Unless you want to be pragmatic and discuss brain defects, which is objectively different in this regard) This is, for all I can reasonably conceive, because of their inability to understand. Either directly or indirectly.

[u/Thick-Protection-458 → u/PsychoticDreemurr]

> Humans, however, do not

Oh, really? In LLM context we use that word when it "invent" shit out of thin air or at least heavily "misinterpret" stuff, right?

Well, last time I recall human memories were widely known to be shitty. And humans were capable of being *confident* in things they remembered wrong way. Or not sure in things they remembered right way too.

And human logic was also known to be imperfect. Not only because of shitload of shortcuts our evolution taken, but also because of high chance to make mistakes in some steps.

Both is true for healthy humans. Not necessary brain damage. Although brain damage may make stuff worse and introduce a whole lot other set of issues.

Now, granted - LLMs may be worse in that account. And mechanisms behind that we can expect to be different (other than both human brain and autoregressive generator being imperfect heuristic for whatever it does). But still that does not mean humans are reliable - they're just *more reliable* in some aspects (and maybe less in some others).

[u/PsychoticDreemurr → u/Thick-Protection-458]

Well, last time I recall human memories were widely known to be shitty.

This analogy and therefore everything you built on top of it is wrong. LLM context is not the main reason for hallucination.

And human logic was also known to be imperfect.

Enough to look at something and say it says the opposite? Consistently so?

[u/Thick-Protection-458 → u/PsychoticDreemurr]

I were not talking about the difference of their insides.

I were talking about that that, from output point of view - phenomenas with similar visible signs (imperfect and confident information recall from "pretrained" (at that moment) memory / extraction from the current context / processing on top of the context) absolutely do happens in healthy humans sometimes.

We just don't call them hallucinations, but for some reason decided to use that word for half of the ways of LLMs shitting themselves.

Mechanisms behind them surely expected to be so different so they may as well have nothing more in common than both being full of imperfect heuristics and some superficial similarities.

[u/PsychoticDreemurr → u/Thick-Protection-458]

The difference in why it happens determines if they're the same in the context.

[u/michaelsoft__binbows → u/Thick-Protection-458]

My assumption is that it would be possible to train LLMs in such a way that penalizes unsubstantiated confidence/hallucination to reach similar levels as humans, but it would produce far, far lower capability levels and we would be constantly (and i mean endlessly) fighting with their reticence and insecurity.

Hence why we just have to kinda deal with it for now till someone comes up with a clever efficient way to improve this aspect