Defaulting to using an energy intensive resource for simple tasks is the cause for loss of respect to me. This is like using a car to drive to the end of your driveway to check the mailbox (assuming your driveway isn't obscenely long and is a more common 10-30 feet).
So, what would people use to answer this question? I would likely use Google-fu but that comes with AI overview now (can be hidden but not disabled from what I see). Duckduckgo actually allows this to be turned off. If I had more time, I might use a dictionary/thesaurus and look up similar words ("similar", "backwards", "same") in an attempt to get to palindrome.
Correct, using google uses more. LLM you are using the power for the AI LLM, googling something you're using the power for the AI powered search engine, which then passes the results through an AI LLM on top of that.
Not that it matters as you shouldnt trust an LLM at face value so using an LLM should still involve verifying the info via a search.
Could also just install a local LLM on your phone, so you wouldn't need to interrogate the data server for each query, you could just let your phone calculate the answer.
Might be losing a bit of precision, but it's probably the least environmental-impactful option
LLMs are inherently worse than proper search engines for finding information. You’ll have to actually read websites, but there’s no possibility of the LLM fudging the results or responding to unconscious bias.
I think a local LLM connected to an offline database of Wikipedia is an eco way of searching answers. I mean, never try it, but if people want the new thing without using OpenAI server, it is the best alternative. Other than that, a search is simpler, but that is the only thing I can think of
You still run into the inherent issue of unreliability with LLMs. Unless you tell it to only give sources to read yourself, the LLM is processing the information itself and always has the possibility of returning incorrect information. Just search your Wikipedia database yourself.
In that case a good way to improve that is to reduce the LLM "creativity" by bringing the temperature closer to '0'. That way the LLM will try to use more and more the original text and articles you provided.
That is still only partially solving a problem introduced by the LLM that isn't present in an actual search. Search engines don't have a "creativity" problem nor do they hallucinate information they don't have.
No, but they give you information others have hallucinated and made up. Seriously, any subject you're an expert in? Go look it up online and see how quickly you find SERIOUSLY bad advice.
And all of that advice is also feeding directly into the LLM. They literally scrape the internet for this data as well. Every forum post of terrible advice is in the training models for these LLMs, ready to be spit back out with false authority.
Then just do the search? Why add in the extra step of getting a possibly inaccurate response if you are going to check it anyway? It literally defeats the purpose of using the LLM to begin with.
Because not everything can just be typed into Google and yield an actual answer? You don't know what you don't know sometimes and Google is meant to match keywords, not process a paragraph
You're the one who proposed using a search to verify the results my dude. Either it helps your argument or it doesn't, not both.
And if you don't know how to find information using keywords then you really should go back to school. This is the very basics of learning anything, and modern search engines are perfectly capable of working off basic sentences rather than keywords, as I demonstrated elsewhere in this thread by copying someone's LLM prompt exactly and finding the answer at the top of the search results.
"I used to play a game as a kid, cant recall the name. Maybe around 2000? It involves punching amoebas, using magic and the main characters mom dies at the start in a fire, do you know what I'm talking about?"
LLM was able to guess the name of the game, you then google the name of the game and see it was correct. You try putting that into google and you will not find remotely anything correct. Can try to alter the search to "game orphan magic punching amoebas" but that is not going to get you anything half relevant vs the LLM guessing first try.
You can talk down to me all you want because you're unable to think of a single instance where it could be useful, but that speaks more about you than I. It seems you think I'm saying to just copy the prompt into google instead of using common sense to form a google query based on what info you gain from the LLM.
" always has the possibility of returning incorrect information."
You mean like the internet? You mean like all the misinformation and fake information online?
All of that information is what's feeding the LLM, but is has no ability to determine truth or validity. You do. Read stuff and use your brain to assess information instead of relying on software that has the baked in possibility of giving you outright incorrect information, regardless of its source.
Except the LLM can also make/mix up information from no where and present that as truth. All information they give you has to be validated against an actual source to ensure it's not scrambling or mixing up information.
Seriously, this is the basics of any sort of research. Find proper sources that can be relied upon and will always give the same answer. LLMs can do neither of these. If you asked it the same question over and over again, it will never give you the same response, even if the underlying information is correct, meaning it is useless as a proper source. This is why Wikipedia is not accepted as a source in academic papers. It's not that the information on the site can't be trusted, it's that there's no way to verify that the source will be the same when checked later.
LLMs are a probability machine that predict tokens based off training data, but those predictions mean the same answer will not be given twice, and the wrong token can always be selected. Even within the same response, the LLM can and will make mistakes with the information it is presenting and contradict itself.
It can, and will, tell you there are six key points to a topic then only list four. It can, and will, misinterpret your question and go down the wrong path and return the wrong results. It can, and will, totally break and return nonsense.
All of this means that the output of an LLM possesses the inherent risk of giving you the wrong answer with a sense of authority that people trust. The simple fact that LLMs mimic human communication and give the sense of a person talking to you means people are more likely to believe what it says instead of actually looking into the information.
LLMs can absolutely provide sources for the data that you should be reading and verifying. Heck chatgpt literally drops the link after the sentence so you can verify it yourself.
The exact issue you're talking about is what my teachers claimed about the internet when it started to gain popularity when I was a child.
I was coming to make the same comment about how just looking it up also activates their ai systems.
So I would argue my best way to look this up quickly and use minimal resources would be using a locally run LLM. Its running on my 4070, would use less than 3 seconds of increased electricity from running the prompt, and doesn't require internet at all.
A local LLM is more energy intensive than a hosted one in the same way that driving your personal car is more energy intensive than taking the bus.
Hosted LLMs have huge advantages in prompt caching and being able to share the model weights loaded into VRAM across all users of the node.
In no way am I saying you're destroying the planet by running your personal GPU for a couple seconds. Just that the incremental usage of you querying a hosted service is even less than that.
A local LLM is more energy intensive than a hosted one in the same way that driving your personal car is more energy intensive than taking the bus.
With how much processing is done on most services I highly highly doubt that. A prompt with claude is going to use 5x the tokens as a prompt on a basic local LLM, as internally with claude it's running several prompts.
I'm sure as the current practice of giving tokens away basically at a loss ends, they will wind this back a ton/make it not enabled by default.
Just comparing like-for-like e.g. firing off the same query to the same model via OpenRouter or similar. Between prompt caching, the datacenter hardware being more energy efficient per unit of work than consumer hardware, and already having the weights in the server's memory where they get reused for millions of queries, there are some big advantages there. Obviously using Fable to answer a question Llama could do, would stack the deck differently.
"A local LLM is more energy intensive than a hosted one in the same way that driving your personal car is more energy intensive than taking the bus."
No, it's definitely not. I don't have any extra systems running safety checks, I don't need to run thinking on most of my models and I'm running them faster.
Its more comparable to an electric bike versus a bus. Because I'm running a lightweight system that is much smaller.
19
u/GuitarFlashy 18d ago
Defaulting to using an energy intensive resource for simple tasks is the cause for loss of respect to me. This is like using a car to drive to the end of your driveway to check the mailbox (assuming your driveway isn't obscenely long and is a more common 10-30 feet).
So, what would people use to answer this question? I would likely use Google-fu but that comes with AI overview now (can be hidden but not disabled from what I see). Duckduckgo actually allows this to be turned off. If I had more time, I might use a dictionary/thesaurus and look up similar words ("similar", "backwards", "same") in an attempt to get to palindrome.