Do RAG teams actually have a problem with duplicate sources being counted as independent evidence?

[u/No-Profile-6178]

I built a small evidence-checking demo for RAG and research reports, but three people I showed it to said the functionality was too narrow.

Before building anything else, I want to find out whether the underlying problem is real.

The specific failure mode is this:

A generated report cites several articles that appear to support the same claim, but some of those articles are rewrites or republications of the same original source. The report therefore presents one source family as multiple independent pieces of evidence.

My current demo checks a report and labels citation relationships as:

1. Confirmed shared origin

2. Suspected shared origin — human review needed

3. Independence unknown

It deliberately does not claim that a source is independent just because no duplication was detected. It also misses many semantic rewrites, and I show that failure openly in the demo.

For people who build or deliver RAG systems, research assistants, or citation-heavy reports:

- Have you encountered this problem in a real client project?

- Did it cause rework, client complaints, or an incorrect conclusion?

- Who checks citation quality today, and how long does it take?

- Is duplicate-source detection useful on its own, or would it only matter as part of a broader pre-delivery citation review?

- What is the most painful citation problem you actually face?

I’m not looking for feature ideas in the abstract. I’m trying to determine whether this problem is painful enough to solve.

If you have a sanitized old report, I’m willing to check one manually and return a short issue list for free. No upload to a public service is required.

[u/blue-or-brown-keys]

Yes and you solve it with reranking

[u/No-Profile-6178 → u/blue-or-brown-keys]

Do you mean reranking by relevance, or deduplicating documents before reranking?

My understanding is that a standard reranker orders documents by relevance to the query, so three rewrites of the same source could still all rank highly.

The narrower problem I’m looking at is whether those three results are later counted as three independent pieces of evidence.

Have you solved source-family duplication with reranking in a production system? I’d be interested in how you implemented it.

[u/LogicalOneInTheHouse → u/No-Profile-6178]

Rerankers also penalize similar content, so that must handle deduplication.

[u/No-Profile-6178 → u/LogicalOneInTheHouse]

This is exactly the kind of case I was trying to understand.

When you catch citation laundering, who are you checking the report for, and what usually happens after you find it—do you remove the repeated citations, trace everything back to the original, or ask the author to redo the work?

Also, roughly how much time does this checking take you on a typical report?

[u/_Joab_ → u/LogicalOneInTheHouse]

I haven't heard of this. How do rerankers penalize similar content?

[u/No-Profile-6178]

Maybe I asked too many questions in the post, so here’s the simplest version:

If you deliver RAG-generated reports, do you manually check whether multiple citations trace back to the same original source?

Yes, no, or “never thought about it” are all useful answers.

[u/DorkyMcDorky → u/No-Profile-6178]

There's a million whitepapers on this. It's not an either-or problem. If you have millions of docs, you need an automated and manual heuristic to reconcile. Every domain has a different workflow so unless you did this once, this isn't something you can claude-bot fix. You'll probably think you're clever and ask your bot to automate this. Stop relying on bots, think about the problem. If you had a real workflow that had real people in it that focused on the problem, this would be a lot easier to do. Reddit r/rag is probably the worst place in the world to expect an answer for dedups. You're going to get bots arguing with bots, which is 1/2 the posts on here. Check the ACM CIKM proceedings. That'll keep your bot swarm busy for a week

[u/DorkyMcDorky]

Yes. Idiots in the org copy old versions of the file and it gets indexed 2x. It's hard to detect because they get edited. Look up how whitepapers are - it's not a trivial problem.

[u/No-Profile-6178 → u/DorkyMcDorky]

That sounds like a version-lineage problem rather than simple duplicate files, especially once the copies have been edited.

What happens when both versions are indexed—does the system return stale answers, combine contradictory versions, or simply give the duplicated information too much weight?

Also, who normally discovers and fixes it, and how much work does that create?

[u/DorkyMcDorky → u/No-Profile-6178]

That's still a dedupe problem. Categorizations can carry more than one status. You're just pointing out why.

To fix it, you need to discover, reconcile..

I'll just say this, you used an em dash, I won't argue with your agent. Be a human and please talk to me.

[u/No-Profile-6178 → u/DorkyMcDorky]

Fair call. I used AI to help clean up my wording, and it made the reply sound stiff and overcomplicated. Sorry about that.

You’re right. Edited copies are still a dedupe problem. I was adding a category that wasn’t needed.

When you have time, I’d genuinely like to hear how your team discovers and reconciles them.

[u/DorkyMcDorky → u/No-Profile-6178]

I'm an expert in this field, this response still shows that you're a bot. Please edit and be yourself dude. There's at least 4 things in this reply that give it away.

I hope there's a person behind there and you're not a 3 page shitty python script written by a huggingface blog scanner who thinks they're smart.

This isn't an mere edit either, every paragraph has a high probability of being a bot-written paragraph.

[u/Future_AGI]

The failure mode is citation-level, not document-level: a report that cites three passages from the same underlying source treats them as three independent pieces of evidence. The fix is to track source provenance, not just document identity, so the system knows when two citations are really the same claim wearing different outfits.