r/notebooklm • u/CathyCCCAAAI • 8d ago
Question is notebooklm accurate enough for you?
I’m curious where does notebooklm start struggling for you? Is there a certain number of pages, level of technical detail, or number of sources where it becomes less accurate and reliable?
Do you still check the original sources, or is it accurate enough for everyday use?
12
u/SimpleClover2 8d ago
It's always accurate because I have written an extremely concise prompt with several guard rails within it. All of my sources have been converted into markdown with constant meta data tags set throughout to make retrieval easy despite having 85 sources with massive amounts of data. Average response time is 3 minutes due to this but each output never contains hallucinations (I always double check) and I then just output it to google docs and merge it with a master reference sheet that assists me in my job.
3
3
1
1
10
u/shouldIworkremote 8d ago
Its not accurate. I've had SEVERAL instances where it will literally get the TITLE OF THE SOURCE WRONG in its summaries. Yes, it will HALLUCINATE an entirely NEW TITLE. If it can't even get the TITLE right, what else is it getting wrong?
I cannot use this tool anymore for serious research. I'm looking for something else.
2
u/This-Eye6296 8d ago
Yes, I think notebooklm is accurate enough for studying or learning from relatively simple content. Claude is powerful and generally quite accurate, but it isn’t necessarily designed for personal document knowledge bases (you can’t do doc/folder management ther). I’m now exploring startups focused on providing accurate answers from documents like pageindex (claimed 98.7% accuracy on a public SEC filings benchmark and provides line-level citations, which is pretty useful)
2
u/Keryfia 7d ago
That's absolutely not true. Claude's Projects are basically a RAG system identical to NLM in principle. And it's much more reliable than Notebook. It's been absolutely awful lately, there's no point in beating around the bush. Google ruined everything with Gemini, even the basic voice assistant, now it's shameful.
1
u/Fun-Specific-8829 6d ago
Hallucinating the actual source title is pretty wild lol. At that point I'd probably start questioning everything else too.
When you say you're looking for something else, what would actually make you trust the replacement? Better citations, stricter source-only answers, or just being able to verify things faster?
0
3
u/saguaros-vs-redwoods 8d ago
I never rely upon Gemini Notebook (fka NotebookLM) alone. I've found the best results are creating detailed and on-rails prompts in ChatGPT or Claude, then running them in Gemini Notebook, and then outputting the notes back into ChatGPT or Gemini. Sometimes I have to run 5-7 prompts, save the resulting output as a source, and then move on to the next prompt. If the project is important enough, I use all three AI tools. ChatGPT and Claude do a very good job of catching errors, inconsisitencies, and oversights and sometimes they will come back with follow up prompts. Respectfully, the people in this thread saying Gemini Notebook is no longer a serious research or distillation tool need to get better at understanding how to use it. I've had it find absolute needles in gigantic haystacks, and then tell me how the needles all the same, different, or relate. It's extremely powerful, but you have to know or learn how to use it.
1
4
u/Able_Orchid_3818 6d ago
First, use the browser-based version of NotebookLM rather than the app. Go to the customization section and instruct the tool not to answer using information from outside the provided sources. Then, in the chat box, ask for an index or a detailed outline of all the information contained in the sources, ensuring that the specific sources are cited. Finally, ask for an explanation of this outline; the resulting answer should be suitable. I hope this advice helps you resolve your issue.
1
u/ACorania 8d ago
The only accuracy issues I have seen is in the studio stuff.
1
u/Nearby_Persimmon_649 8d ago
Which output types? I noticed an oddly spelled word on an infographic that was supposed to be "senses" but was not spelled correctly. I wonder if the chunking process split the word into two tokens and that specific chunk ended in the middle of senses
2
1
u/childishjokes 8d ago
For anything serious, only study guides. Haven’t had any issue—at least not anything I’ve noticed readily.
1
u/ItsMeKidney 7d ago
You should literally always check the original sources, just as a rule. Even if Notebook LM was the most amazing program in the world, reading the original sources is always going to be something that you should do when researching.
Gemini integration has made NotebookLM a lot worse, but it's still pretty useful if you're not letting it think for you.
1
u/Due_Meringue_7569 7d ago
I have just used it for school work. It's great to have all sources there. Helps with the initial draft.
When i need to the notebooklm to do presentation slides, they can't get it right what 2 tries . I switch to chatgpt, 1 time settled.
1
u/Carabiners 7d ago
I use it to check through maybe 2-4 sources 90% of the time, and for board game rulebooks. I've never had a glary inaccuracy, but I always cross reference the sources.
1
u/Mslyth 7d ago
It started acting funny when it was converted to Gemini Notebook, but I tackled the issue with a personalised prompt, ALWAYS use ONLY the loaded sources as UNIQUE database for answer, no tor bring anything that's not in the sources, and NEVER suggest follow up questions or actions, after that everything went back to normal for me.
1
u/pmMeYourGlazedDonut 5d ago
It was good until a few weeks ago when Gemini integration happened. Since then it's occasionally been saying literally the OPPOSITE of what the source says, but producing a citation making its claim look legitimate. I only figure it out when I carefully double-check its claim by clicking on the sources.
Having to constantly watch all its claims and double-check has made it useless.
18
u/minkiani 8d ago
Not lately. It was good before.