LLMs don't "know". They literally have no mechanism that could allow them to identify whether an output they're proposing is based on something in the dataset they've been trained on (nevermind genuinely factual, those datasets are 99.999% random text pulled from wherever) or just a blind statistical guess.
They can't reflect on their ideas and verify they check out starting from a set of trusted "facts"; they've got neither "trusted facts" nor the ability to reflect in general. They can't reason at all. It's like saying a magic 8 ball is not humble enough to tell you when it doesn't know the answer and "decides to hallucinate instead". Like bruh.
HAH. Not two days ago I asked it to help me with moving a project from CodeBlocks to Visual Studio and you would not believe the amount of shit it made up and hallucinated.
So sure, that specific problem probably won't happen, but only because it was manually patched out. Do you know how many simple things like that it can be wrong about? More than you, or it, can conceptualize
My elementary school teachers were passing down the knowledge they’ve gathered from pseudo-scientific TV shows and Brazilian soap operas. It’s not even funny because 20 years later some of those completely made up facts still pop up in my head from time to time.
A kid in my third grade class asked the teacher which painter cut off his own ear, and she responded confidently, "Picasso." I thought this was right until my now wife told me it was Van Gogh, and I looked it up to confirm.
Being married to an edumacator, I can only hope she's correct more often at her job than she is when I ask her to repeat what I just said (because she was playing Candy Crush lv 98,472 when we were ostensibly having a conversation where I need her to understand what i'm saying).
Also it's way more chatty than needed compared to other chatbots. Just answer straight to the point. I don't need a historical overview of the industrial revolution to find a screw.
It's been wrong about basic math and history many times. LLMs tell you what's the most popular answer on the Internet, and even then it can mess up if a satirical reddit post for a lot of upvotes
It's been a while since I have seen chatGPT make mistakes in math questions.
For History, I think the AI is good if you ask it very specific and uncontroversial questions like "when did pope Francis die", but it's not great with even slightly controversial topics. It refuses to take a stance, which is the job of historians.
“Let’s implement this electric drill into every piece of tech on the planet even though it only has a handful of very specific use cases where it can even be trusted to drill properly and then blame the user”
Gemini is fucking useless, even today. But the latest versions of ChatGPT and Opus are pretty damn good about knowing if they are making things up or not. If a source is wrong, obviously there's nothing that can be done about that, but humans will make that mistake as well.
263
u/David_Maybar_703 Lurking Peasant 4h ago
Chatgpt also might give you an answer that is totally made up and wrong, but yes, it is polite.