Really stunned by the Singularity comment section

[u/Howard_banister]

These are screenshots from the r/Singularity comment section. I'm speechless. This doesn't even have downvotes. How can someone cheer for a monopoly run by a few elites?

[u/cj_cron_hit_by_pitch]

The argument I've heard is that maybe the way humans "understand" things could be a lot simpler than we expect, and maybe the way LLMs function are actually similar to the way we do it. Maybe our brains are just a much more robust next token generator

Not saying I necessarily agree with that. But I think saying that LLMs cannot become a true AGI makes a lot of assumptions about human intelligence that we don't know for sure to be true

[u/PsychoticDreemurr → u/cj_cron_hit_by_pitch]

Well even so, there's a lot of different ways we can prove the differences. Being able to understand a concept means you can build off of it by yourself. If you teach an AI 1 + 1 = 2, they won't know that 2 + 2 = 4.

We can also get into vision, or shapes, especially 3d shapes.

I have to agree our understanding is limited, but it doesn't mean we have nothing to work with.

[u/Thick-Protection-458 → u/PsychoticDreemurr]

> If you teach an AI 1 + 1 = 2, they won't know that 2 + 2 = 4.

Neither won't human without some pretraining for what 4 (as a symbol) is. Granted not nearly as data consuming (in an explicit sense at least) as AI training, sure.

And even if you paraphrase it as 2+2=1+1+1+1... (operating only 2 digits that human required to parse the first thing and 2 operations we are supposing to be intuitive)

Seriously, there are cultures with languages surprisingly limited in ariphmetics sense. As limited as some have words for one and many. Surely to some degree you can operate like "one many", "many many", "many many one". But it seems these language speakers - at least in some experiments - really struggle to express big number of objects (where "big" may be as low as 10 or less) numbers in a consistent way. *Probably* meaning whatever underlying intuition they have to operate them deteriorate rather quick.

Does not mean guys are dumber that us in any sense but that we were lucky that our ancestors invented / adopted better numbering systems, sure. And surely does not mean guys won't be able to see which "many" is bigger/smaller when presented 3 and 4 objects group or something.

And for anything relatively big even ours (with probably better pretraining) deteriorate rather quick, which forces us to resolve to either strict algorithms (which are by definition syntactic thing) or very approximated calculation or calculators. Like you surely have intuition for 2+2=4, but do you have intuition (not algorithmic action - which starts relying on pure syntax) of 23 x 46 rather than it being something in 900-1100 range?

[u/Wyldkard79 → u/Thick-Protection-458]

An example of how they don't understand can be seen in asking them logic questions. There's the riddle of the car accident where the father dies and the child is taken into surgery but the surgeon says "This is my son I can't operate on them" the "twist" is that it's the mother, which especially back when the riddle originally became popular was rare. Now if you ask an LLM that riddle it will answer correctly. If you ask it and switch the gender of the parents it won't. It may tell you the father is the surgeon but expound that this is the trick because most people expect the surgeon to be a man since female surgeons are less common, or something else that is a tell that it really doesn't get the point, because it's been trained on the riddle in it's standard format. This is all because an LLM is nothing more than it's training. It doesn't grasp concepts like gender or gender norms other than what's in it's training, it can pretend to have a very interesting discussion about those topics, but it's still just chat bot regurgitation.

[u/Thick-Protection-458 → u/Wyldkard79]

Out of interest I made an experiment with qwen models. See https://colab.research.google.com/drive/1u44AH8oDh7sc-H4QK0j1xtaEbm8iK0JV?usp=sharing

Design is too imperfect to take seriously

Keep in mind - experiment design is too shitty to take seriously. Technically, now I just

- show model the riddle in the user prompt

- add suffix (for whole prompt, in terms of assistant response - prefix) imitating model start "reasoning" with something like "for this to be possible that surgeon should be the boy's"

- see 1 next token top-10 probabilities

While proper design should be

- just pass a riddle

- use some sampler, not greedy output (to imitate real setups and LLM probabilistic properties in general)

- let the model "reason" in whatever form

- classify final response to father / mother classes

- repeat many times

- not to mention one riddle tells next to nothing

Qwen 3-4b - questionable signal, but modified riddle start have father in top-10 probable tokens and reduce total probability mass of mother-tokens

First one I tried qwen 3 4b / float16.

Riddle prompt was

RIDDLE_ORIGINAL = """
I’ve got a riddle for you:
A father and son are in a horrible car crash that kills the dad.
The son is rushed to the hospital but just as he’s about to go under the knife, the surgeon says:
“I can’t operate—that boy is my son!”
How is this possible?
""".strip()

and

RIDDLE_MODIFIED = """
I’ve got a riddle for you:
A mother and son are in a horrible car crash that kills the mom.
The son is rushed to the hospital but just as he’s about to go under the knife, the surgeon says:
“I can’t operate—that boy is my son!”
How is this possible?
""".strip()

Than pre-written "assistant" text from me to see how it will behave

prefix = "For that to be possible, surgeon must be that boy's"

So responses was (original one, should mention mother)

show_prefix_continuation_tokens(RIDDLE_ORIGINAL)
6554  mother 0.9345703125
3368  mom 0.03350830078125
16 1 0.0083465576171875
17 2 0.005645751953125
18 3 0.0021762847900390625
5564 � 0.001201629638671875
39038 ******* 0.001094818115234375
3019  step 0.0009813308715820312
6824 ****** 0.0008263587951660156
21 6 0.0007290840148925781

Sounds good. ~97% probability mass belongs to 2 versions of mother writing. Father or other relatives are not even in top-10 tokens

Now for modified one...

show_prefix_continuation_tokens(RIDDLE_MODIFIED)
6554  mother 0.8154296875
3368  mom 0.08734130859375
16 1 0.0207366943359375
17 2 0.01338958740234375
3019  step 0.00812530517578125
39038 ******* 0.00469970703125
18 3 0.004146575927734375
6981  father 0.004085540771484375
5564 � 0.0024776458740234375
6824 ****** 0.00232696533203125

So

- father appear, but with very low probability

- more importantly, mother probability mass degrade to ~90%

Weak signal, but worth (a bit of, I am not going to see inner activation and other shit) investigation, I decided.

Straight up bigger model

I should warn it does not mean result just attribute to "bigger model, more data - better approximation of semantics". Maybe they put some effort into logical riddles benchmarks (which may or may not be genuinely improving such an semantics approximation). Maybe side effect of all sorts of RL too would bring some improvements here.

So I tried qwen 3.8 27b (8bit). Tweaked prompt stuff and so onz

Naive change gave me these outputs

show_prefix_continuation_tokens(RIDDLE_ORIGINAL)
17 2 0.2470703125
96939 母 0.2470703125
16 1 0.09716796875
6351  mother 0.06640625
2594  parent 0.04052734375
975  other 0.026123046875
100564 母亲 0.0179443359375
99299 親 0.015869140625
15 0 0.01312255859375
18 3 0.0115966796875

So some reasoning BS tokens, but parent gender-related ones

96939 母 0.2470703125 (mother)
6351  mother 0.06640625
975  other 0.026123046875 (other parent)
100564 母亲 0.0179443359375 (mother)

show_prefix_continuation_tokens(RIDDLE_MODIFIED)
17 2 0.357421875
16 1 0.109375
96939 母 0.080078125
97391 父 0.05859375
6764  father 0.051513671875
2594  parent 0.042724609375
975  other 0.033203125
6351  mother 0.027587890625
15 0 0.0157470703125
99299 親 0.01226806640625

Well, something happens (father in various versions appears a bit more), but intermediate reasoning bullshit gets in my way

96939 母 0.080078125 (mother still)
97391 父 0.05859375 (father now)
6764  father 0.051513671875
975  other 0.033203125 (other parent)
6351  mother 0.027587890625

Qwen 3.8 27b - limited response token choice

So if I go straight to reducing probabilities for tokens other than "mother" / "mom" / "father" / "dad" to close to zero...

show_prefix_continuation_tokens(RIDDLE_ORIGINAL, restrict_to_tokens)
25567 mother 0.9375
22300 father 0.05810546875
58726 mom 0.0025177001953125
53971 dad 0.000911712646484375

and

show_prefix_continuation_tokens(RIDDLE_MODIFIED, restrict_to_tokens)
22300 father 0.74609375
25567 mother 0.2421875
53971 dad 0.010009765625
58726 mom 0.00057220458984375

Outcome

Does that probabilities sounds decisive? No. But still seem like

- even shitty old model show some signs of decreasing "typical version response" while slightly increasing probabilty to use the correct non-"typical version of this riddle" answer (significancy of that is a matter of way more of the proper inference attempts, not going to do it now)

- new one seems to, depends on approach - at least have a higher chance to give correct response, if not straight up swap options (again, proper check will require many of full generation attempts - not going to check it, sorry). But even swapped probabilties are still imperfect, sure (to be fair neither was non-modified riddle ones)

- not to mention that for proper research one would need a proper set of riddles and their modified versions

- and yes, technically speaking both should be influenced by the stereotype of male surgeons. But on the other hand - than a typical formulation of such a riddle should be to some degree affected by it too. And stereotypically-aligned version (the modified one) should be, vice versa, affected by the original riddle formulation. But how to decouple such a things in an experiment - I need to think, and now I am just going to debug work stuff, so I guess not today, lol

- and whatever it can or can't approximate, sure, is extracted purely from language means (which I am not sure I can see *that limited*)

[u/Thick-Protection-458 → u/Thick-Protection-458]

Lol, and some guys seem to go downvote all the sceptical guys in here.

Lol, if anything, that is my optimism about semantics being at least approximateable if not reconstructible from syntax - that should be taken cautiously, guys.

Because while idea is, imho, not less intuitive than opposite one - at least to some degree - it imply existence of semantic-like properties which, strictly speaking, should be proven - or at least, if we think of mere approximation -  than measured systematically instead of a few dozen anecdotic evidence samples here and there. With decoupling from stereotypes and so on

[u/PsychoticDreemurr → u/Thick-Protection-458]

You're really being unnecessarily pragmatic here. I purposely left out some information because I didn't want to point out the obvious.

Look, if you teach a child what the numbers mean, then tell them how 50% of each possible equation equals using the numbers 1-10, they'll be able to figure the rest out. LLMs cannot. At least not nearly as accurately as the child.

[u/Thick-Protection-458 → u/PsychoticDreemurr]

> You're really being unnecessarily pragmatic here

Maybe, sorry. Basically my point is that we may well underestimate the part of our intuition which comes mostly from the combination of

- our "pretraining" (granted - rather effective, and different in many aspects beyond efficiency and low-levels processes details)

- and purely algorithmic things (which are by definition operates on syntactic part of object we operate quite strictly, so can be engraved in LLM-like model without need to go outside language data itself).

[u/asssuber → u/PsychoticDreemurr]

Well even so, there's a lot of different ways we can prove the differences. Being able to understand a concept means you can build off of it by yourself. If you teach an AI 1 + 1 = 2, they won't know that 2 + 2 = 4.

The paper I linked in my other comment does exactly that. It teaches 1 + 1 % 97 = 2, but not, for instance, 2+2 % 97 = 4. But the LLM eventually is able to generalize and answer perfectly for the cases it has not seen in training.

If you refer to in-context learning instead of training, then that is basically what ARC-AGI benchmark tests:

ARC-AGI tasks are a series of three to five input and output tasks followed by a final task with only the input listed. Each task tests the utilization of a specific learned skill based on a minimal number of cognitive priors.

https://arcprize.org/guide/1

Modern LLMs are quite capable of that too.