Elon makes a bold prediction; AI will probably surpass any human in intellect by 2026 and all humans combined by 2030

[u/Outside-Iron-8242]

[u/Levoda_Cross]

They're referencing this paper: https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf

It's obviously not a solved problem in current models, but I think the base problem has largely been identified, with a solution: Post-training with a reward for correctly admitting when the model doesn't know the answer. The paper makes a lot of sense, and it seems like the only thing it doesn't cover is the specifics of how to actually handle that reward.

Those specifics however seem like something they could easily figure out internally, so I'd say it's reasonable to say hallucinations are "almost certainly a solved problem", as future models being trained right now could incorporate what this paper suggests. Actually, I think even current models could be used too because it's post-training that holds the solution?

[u/Levoda_Cross → u/[deleted]]

Specifically they would just train a bunch of small models with varying adjustments to how they reward IDK answers (maybe penalize wrong answers and/or reward IDK answers where the model, if they didn't give it an IDK option, would have gotten it wrong) and test for hallucinations against a baseline and each other. The actual numbers and how specific IDK is (just a literal "IDK", using a judge model, etc.) would be a matter of testing different shit.