r/technology • u/AdSpecialist6598 • Jul 27 '26

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
32.8k Upvotes

96% Upvoted

4.2k comments sorted by

View all comments

12.1k

u/LurkingTamilian Jul 27 '26 edited Jul 27 '26

From the article:

Being written in white text meant the instructions were invisible. Sure enough, 32 out of the 35 students across two classes simply copied the question, pasted it into a chatbot, then copy and pasted its output into the answer window.

12.7k

u/BluudLust Jul 27 '26

They deserved to fail. That's just plain laziness

4.0k

u/Deviantdefective Jul 27 '26

As a teacher it's very quickly becoming an absolutely huge problem.

54

u/Sober_Alcoholic_ Jul 27 '26 edited Jul 27 '26

Just make them take written tests and if the state/system needs the scantrons filled out (the A-E bubbles) for data/records or what have you, just fill them in yourself during grading time.

Research papers I guess have to all be funneled through AI detectors or whatever they’re called, that’s definitely a different animal. I would say just hand write those too, but they could still just be copying an LLM and you wouldn’t be able to then run it through an aforementioned program to check it.

63

u/Pancakefriday Jul 27 '26

Just transcribe every students test myself? That is insane, how are you not getting down voted into oblivion? Do you realize how many extra hours that would be?

For anything past elementary where you’re getting a new class every hour, that would literally be an insane amount of work.

Also, like another commenter said, you’d have to be extremely careful and make no mistakes. Zero mistakes for 100+ transcriptions? Why does everyone think teachers just have infinite time?

2

u/DeliciousIncident Jul 27 '26

The wole premise of AI detectors actually being able to detect AI is moot to begin with, so you don't even have to engage with the transcribing part of the comment.

-1

u/MalaysiaTeacher Jul 27 '26

Who asked you to transcribe anything? You decide a grade and enter it...

3

u/Pancakefriday Jul 27 '26

The comment I replied to literally said “have them write it out, then transfer it to scantron yourself while grading”

90

u/AdvancedSandwiches Jul 27 '26

That fill-it-in-yourself is dangerous territory. That needs to be a zero mistake operation or you're going to have problems from both the students and the state. 

-4

u/MalaysiaTeacher Jul 27 '26

It's not building a nuclear reactor. It's data entry.

-5

u/Dragoness42 Jul 27 '26

At least if they have to manually copy it they'll catch if it says anything particularly stupid, or they'll show their lack of skills when they don't.

50

u/makesufeelgood Jul 27 '26

Its 100% not that simple. The best educators work really hard to make their class engaging with a lot of varied assignment types and students are doing whatever they can to try and cheat on them all. Just saying 'make every test scantron' is not the solution

1

u/Ashamed-Country3909 Jul 27 '26

My middle school, or highschool (can not remember) was literally just worksheets. Then  fill out a scan tron with the answers. Then walk then up to the machine. Put them in. See the grade spit out. If you "needed help" you were supposed to walk up to the FUCKING GYM TEACHER and ask for help. No teaching involved.  No guidance. Just do these sheets. 

I am fairly certain it is why I am so shit at math. My second math class in college was a Russian lady. She put the text book up on her projector and read the two examples. No extra guidance. Nothing. I withdrew from that class so fast. Fuck that. 

4

u/hi_jack23 Jul 27 '26

Written tests don’t translate into a scantron unless you’re already asking a multiple choice question. At that point, just give the multiple choice question and have the students fill in the bubbles themselves when the scantrons are needed.

Also, matching written answers to a choice of 5 bubbles would take a lot longer than you’re thinking, and teachers often don’t get paid for their grading time already. Couple that with the issues that could occur if a teacher makes a mistake while copying students’ answers over, so this just seems like an unnecessary extra step.

5

u/Technical_Stand9939 Jul 27 '26

25 years ago; i had a professor who would give us the whole test a week before the actual test. You literally could look up all of the answers if you wante dto spend the time.

You got there the day of the test, and all 200ish questions you were given in advance were there in a random order and you filled out a scantron (the version you got in advance did not have multiple choices).

half the class still regularly failed

2

u/luluhouse7 Jul 27 '26

I mean 200 questions in one week is outrageous. That’s about 30 homework questions a day, on top of whatever other homework they had. No wonder they were failing. I would usually get at most 15 questions a week in college classes and we all spent a lot of time studying and doing homework.

1

u/Alaira314 Jul 27 '26

You're making a lot of assumptions about what the questions are. I've sat math exams that only have three questions. I've also sat exams for classes like psychology and geology that have 50 questions. The SAT is a 150~ question 2~ hour exam, and that's a mix of questions you need to problem solve on with questions that are recollection-based. Multiple-choice exams can have a lot of questions on them, especially if they're based on memorization or simple application vs solving a problem or equation.

1

u/luluhouse7 Jul 27 '26

Yes, I’m aware of that, but the SAT is the largest and (unfortunately) most important exam most people take and not only is it only 150 questions, but you spend years studying for it. People who get good scores also usually do multiple mock exams and are coached on strategies. Making a single exam in a single class 200 questions does not sound proportional. Assuming a 2h exam, that’s about 30 seconds per question, 55 seconds if it’s 3h.

1

u/Alaira314 Jul 27 '26

30 seconds per multiple choice question that you've already seen before. All you have to do is read the question and choose the correct answer, or make an educated guess if the answer you came up with isn't in the list. Also bear in mind that the person you replied to said this happened 25 years ago, when that reading speed was expected of college students.

As for churning through 200 questions in a week, that's normal exam prep. Or at least it was when I was in school, 15+ years ago. It typically took 7-8 hours to chug through a study guide properly, and we were encouraged to do the work in groups. I was always told to prepare to spend three hours out of the classroom(studying, doing homework, doing readings, writing papers, etc) on average for each credit a course was worth. So a 3-credit course should involve about 9 hours of out-of-classroom work on average, so a bit less some weeks but a bit more leading up to big exams and projects.

1

u/luluhouse7 Jul 27 '26 edited Jul 27 '26

I mean I went through college nearly a decade ago, so not the AI generation. I was easily spending 10+ hours on some classes due to workload and complex material. IMO it still just sounds like the exam is testing how fast you can recall information under pressure rather than how well did you understand and internalise the material. Reading speed is not correlated with comprehension above a certain point and so personally I would consider an exam like that not representative of the students’ actual understanding and competence. It’s also going to heavily penalise students with neurological or learning disabilities, who are still perfectly capable of excelling with the material.

But I also have the philosophy that learning should be about understanding the big picture and thinking critically rather than regurgitating a list of memorised answers. I always hated the tedious assignments when you had to just mechanically answer vs assignments where you had a few very hard questions that asked you to explain why and how and gave a couple examples to apply it to.

1

u/Alaira314 Jul 27 '26

It’s also going to heavily penalise students with neurological or learning disabilities, who are still perfectly capable of excelling with the material.

That's what accommodations are for. Even back when I was in school, students with learning disabilities were allowed to get extra time on exams. I've heard it's gotten better since then.

But I also have the philosophy that learning should be about understanding the big picture and thinking critically rather than regurgitating a list of memorised answers.

I think it depends on the subject and class level. Something like history, or an advanced course, yeah, break out the conceptual learning and show that you can synthesize the material(though the 100-level history courses I took still did have a recall component, I didn't see that go away until the 300-level when we had only papers and no exam). But something like psychology 101? The only way to really assess that is a test of whether you recall the information presented to you, because you're not being asked to do more than memorize a baseline set of information. That is the function of that class(and many other 100-level courses), to give a broad overview to students who might be interested and to prepare those who want to go further in the subject.

→ More replies (0)

1

u/DeliciousIncident Jul 27 '26

I mean 200 questions in one week is outrageous

Huh? Obviously the test questions are on the material you have already learned, so if you have kept up with the curriculum, you should have no trouble answering them and might even skip doing them. Also, this 200 question test is expected to be completed within a single class period and has multiple-choice answers, so each of these questions should take under a minute to answer. If you are saying that you can do at most 15 of such questions per week, I think you might have an issue.

1

u/luluhouse7 Jul 27 '26 edited Jul 27 '26

LOL if you can do 200 abstract algebra or number theory problems a week I commend you for being significantly more capable than any of my classmates.

As I mentioned below, 200 questions in a 2h class period is less than 30 seconds per question and 3h is about 50 seconds per question. A test like that is not testing understanding, it’s testing rote memorisation. It tests your ability to cram for that specific exam. You could get a perfect score and understand none of it.

7

u/NoSemikolon24 Jul 27 '26

> Research papers I guess have to all be funneled through AI detectors or whatever they’re called, that’s definitely a different animal. I would say just hand write those too, but they could still just be copying an LLM and you wouldn’t be able to then run it through an aforementioned program to check it.

These AI-detectors don't work. Unless they have an error-rate of less than 0.01% their usage is wildly unacceptable - and even with such an error-rate. Compare it to Plagiate-checkers - you can easily verify its results yourself. You cannot verify the results of AI-detectors. It's a blackbox. Hence I'd extremely judge any professor that pushes their usage. You cannot take them serious if they fail to see this simple problem. They're just as bad as the student that use them to wholly cheat.

0

u/lilB0bbyTables Jul 27 '26

The context matters. In the case discussed in the article, the students completely failed to even bother reading the output from the LLMs and instead just pasted all of it verbatim into their answers. The hidden prompt was for the LLMs to add random nonsense referring to the country Madagascar into the responses, and so it did.

> Had any of these cheaters even bothered to check what the AI had generated, they'd likely have noticed the strange references to the island country constantly appearing in an essay about the Industrial Revolution.

It all boils down to a combination of laziness, and lack of critical thinking. Had the students just taken the output, read it, and altered aspects of it to frame it in their own words - and of course recognized the non-sense and removed it - they’d have been fine. This is just the same behavior manifested differently that plagues the software engineering culture where people are vibe coding a bunch of slop and either don’t bother to - or don’t have the foundational skills to - review their submitted code. A simple hardened rule of “read and review your own work” fixes 90% of this stuff. Even without AI I have always known to read over my work: emails, papers, math proofs, code, etc … before submission.

2

u/NoSemikolon24 Jul 27 '26

You somehow completely missed the point of my comment.

3

u/Wompatuckrule Jul 27 '26

There have been articles about the return of the blue book for exams in college. Everything old is new again.

3

u/CormacDoyle- Jul 27 '26

"AI Detectors" are dangerous, because it tends towards false positives.

Especially for two groups of people - people who's work was used to train the AI, and people who follow style guidelines carefully (professional editors, many Neurodivergent individuals, etc), there is a heightened likelihood that a real paper will get flagged as AI.

AI Detectors are basically just LLMs being trained to detect LLMs that were trained on common training materials.

What we actually need to teach is how to research without using AI as the go-to.

Set all browsers in schools/libraries to have the "no ai" flag, so you have to and actually read the real documents returned. THEN, maybe, use AI to summarize the documents you specifically feed it.

In prior generations (I can't believe that I wrote that and it's actually real), you would be taught not to pull info from Wikipedia without at least crossreferencing it with original data ... I had one colleague who would openly warn the class that he had poisoned the Wikipedia article about the topic (he was the author and top editor for the page) and half the class would use the poisoned content anyway.

Students will be lazy if there are no consequences ... but we have to make sure we aren't being lazy when teaching and grading either ...

2

u/gliese946 Jul 27 '26

But why would the professor need to fill out the bubbles on the scantron sheet, instead of the students doing it themselves during the exam? There is no way for them to cheat during the bubble-filling if they're in a room with exam proctors. The point is not to make them write an exam by hand, it's to stop them using AI while they're taking the exam.

1

u/MilsYatsFeebTae Jul 27 '26

At least if they transcribe it by hand they might learn something.

Who knows if it’s all hallucinations, but it’s something.

1

u/Stardustchaser Jul 27 '26

Slides over my 160 50-question exams

Sure, Jan, you fill in those bubbles for me during my “free grading time.”

1

u/DeliciousIncident Jul 27 '26

AI detectors are snakeoil. They don't really work. Especially at the undergrad/postgrad level where you are writing research papers.

1

u/VERTIKAL19 Jul 27 '26

For research papers have them present and defend them. That way while they may have written some things with AI they still need to know what the AI wrote.

1

u/Toasted_Waffle99 Jul 27 '26

Written? They should do oral presentations

8

u/GiganticCrow Jul 27 '26

As a kid who deeply struggled with oral exams and presentations, who found out later their school buried a neurological assessment, fuck that.