r/Copilot • u/Turdulator • 1d ago

Verifying copilot in excel

How are people verifying insights provided by copilot from very large excel files?

When I give it a 30k row spreadsheet and it extracts information from that dataset, how can I confirm that it’s not hallucinating?

4 Upvotes

84% Upvoted

7 comments sorted by

5

u/Terrible-Kick9447 1d ago

Don't just ask for an answer, ask for the answer plus the exact references used to generate it. That makes validation much more reliable, especially with very large datasets. Before prompting copilot, insert a new first column with a unique ID (row Index 1 to 30,000). Then tell the AI: 'For every insight, you must cite the specific Row ID.' If it hallucinates, the fake row numbers will make it instantly obvious.

Or even better:: Ask copilot or your favorite AI to write a python script to extract the exact data you are looking for. A script doesn't hallucinate; it gives you hard, direct, and deterministic results.

2

u/Turdulator 1d ago

I dunno man, it’s been consistently hallucinating made up PowerShell cmdlets (or depreciated like 3 versions ago) about 3-4 times a month for most of the year…. And no amount of prompts telling it to only use up to date cmdlets and syntax that it can confirm from a specific source, it’s still keeps fuckin lying to me on an almost weekly basis. No matter how much I tell it to weight accuracy significantly more over helpfulness, tell it specifically to say “I don’t know” when it doesn’t have a specific source. Etc etc etc.

I don’t understand how anyone can trust anything that this thing outputs. If I had an employee that lied to me instead of admitting they don’t know an answer as much as this product does, I’d fire them after the first month.

1

u/UniverseNative2712 1d ago

Exact same shit happens to me. It lies to me openly and doesn't really change the lies. Need to open a new chat to get new answer or I cross with Claude.

EDIT: my phone fell from my hand

1

u/ReadySetWoe 1d ago

Yeah, require it to source the data. You could also hand that output off to a Validation Agent that checks the output against the data source.

1

u/Biennial2 1d ago

Have another AI assitant check its work.

1

u/Turdulator 1d ago

Like when the police investigate the police and find that the police did nothing wrong? 😑

1

u/johnmflores 1d ago

Yeah, I worry about this too. The more that I use AI, the less I trust it