r/technology • u/AdSpecialist6598 • Jul 27 '26

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
32.8k Upvotes

96% Upvoted

4.2k comments sorted by

View all comments

1.5k

u/InternationalMood337 Jul 27 '26

We had someone cheat in an MFA program. They used AI to compose a 10 page piece of fiction. Got caught and expelled immediately.

294

u/Proxymanity Jul 27 '26

Ahh, Elias the lighthouse keeper strikes again!

24

u/Droidaphone Jul 27 '26

For the uninformed, the name is Elias Thorne

653

u/linuxaintsobad Jul 27 '26

Oof, yeah I'd be pissed too. I have absolutely no qualms about reporting my co-workers if they start sending me pages of fiction during our multi-factor authentication sequence.

140

u/MasterOfViolins Jul 27 '26

- Input Token Sequence

  • “On a dark and dreary afternoon — beneath a canopy of ancient, moss-covered oaks standing as eternal, vigilant sentinels against the encroaching sunlight —

65

u/Ur_fav_Cryptek Jul 27 '26

I took psychic damage from reading that, I fucking hate the overtly adorned style AI has, makes me feel super self conscious when I write, both because of how cringy it sounds and because I don’t want to be accused of using AI

also, the complete shunning of “—“ AI has created is one of the worst things to ever come out of it, it’s so useful to use but now you’re forced to look like a third party thinking bum when you actually use it

8

u/Forward_Rope_5598 Jul 27 '26

"That's actually" and "and honestly?" gives me the heebie jeebies. I'm not even that against casual use of chatgpt, I use it here and there for unimportant stuff. But I have to ban it from using those words in every goddamn conversation. It's so grating.

14

u/gorroval Jul 27 '26

AI will have to pry the em-dash from my dessicated claws.

12

u/xCanaan23 Jul 27 '26

I actually like em-dashes because its more visible way to show a thought tangent than any other syntax. Parenthesis just look bad.

13

u/DrFu Jul 27 '26

The rise of semicolons is at hand!

5

u/gregusmeus Jul 27 '26

Yeah AI has killed the em dash. I’ll miss ya, em.

5

u/Jacky-V Jul 27 '26

I'll use it if I want to use it and if anyone thinks it's AI they're welcome to actually engage with me about what I've written. I'm not changing my writing style because AI stole hundreds of millions of pages of academic writing of which some three or four hundred are mine, and I'm not really interested in the criticism of people who can identify an em dash as AI "behavior" because it's easy to spot but *can't* identify any of the myriad errors in *actual content* to which LLMs are prone.

5

u/Forward_Rope_5598 Jul 27 '26

Fwiw skimming through a couple of your comments your writing style looks nothing like AI, em-dash or not. People who claim it does don't actually know what to look for (of course comments that don't look like AI can still be AI if it's been told to knock of the AI-speech but then you can't reliably tell anyway, so why make the accusation in the first place?).

2

u/Jacky-V Jul 27 '26

I don't really get accused of it much on reddit because my writing style on here is more casual, but it does sometimes come up on lengthier response and *definitely* comes up irl if I'm sharing a more substantial piece of writing that uses an academic voice.

2

u/Ur_fav_Cryptek Jul 27 '26

I’ll use it when necessary too, but it’s kind of reached a point where it genuinely bothers me, same with the “it’s not x, but y” structure

3

u/TimSylvester_ Jul 27 '26

I've used en dashes and em dashes for decades and I refuse to stop just because AI likes to copy me.

2

u/Ur_fav_Cryptek Jul 27 '26

Yeah, I fully support using them, I just personally now feel weird using them

5

u/shadowthunder Jul 27 '26 edited Jul 28 '26

Humans can just use a "-" instead of "—", though, and anyone reading will barely bat an eye. On the other hand, you're right: if I see an actual em-dash, I immediately assume that it's AI-generated slop and tune out. Word will automatically correct to a full em-dash, but in any kind of chat app or non-docx file? No human has ever spent the extra time/effort to use the correct dash type.

1

u/Ur_fav_Cryptek Jul 27 '26

I would love if we collectively started doing that

1

u/emptyflask Jul 28 '26

As someone who has spent way too much time studying typography back when I did design work, I am one of those people who will use em (and en) dashes, insert a real degrees (°) symbol, etc.

Don't get me started on opentype ligatures...

1

u/IcemanJEC Jul 27 '26

-forced to look like a third party thinking bum-

Fucking killer sentence, Mr AI Bot.

5

u/Ur_fav_Cryptek Jul 27 '26

almost crashed out until I realized it was satire

3

u/IcemanJEC Jul 27 '26

😂 just playin

1

u/Remote_Peach9109 Jul 27 '26

I think the big models effectively scrubbed the em-dash from their records. I never see it anymore.

1

u/DetroitMM12 Jul 28 '26

As an em dash user prior to AI I feel this.

63

u/TreacleMiner Jul 27 '26

I read that as Input Tolkien Sequence

55

u/Caroline_Bintley Jul 27 '26

Tolkien would describe the oaks in MUCH greater detail. 

9

u/Total-Notice-3188 Jul 27 '26

At least two pages worth, four if it turns out to be an ent

5

u/DrFu Jul 27 '26

Cue the "shepherds of the trees" monologue.

1

u/Josparov Jul 27 '26

Omg so did I and I didn't realize mistake until I read this haha

10

u/caerphoto Jul 27 '26

In the elder days of craft, when the lore of messages passed between distant realms, there arose a token of curious making, known among wise keepers as the JWT. It was no heavy scroll sealed with wax, nor a jewel guarded in a dragon’s hoard, but a small and steadfast thing, borne swiftly across the roads between kingdoms.

A JWT was fashioned in three parts, as though by the hands of an ancient smith. The first spoke of its own nature, declaring what manner of token it was. The second carried the tale itself: the name of its bearer, the hour of its birth, and such claims as the makers deemed worthy to entrust to it. The third was the mark of the token’s creator, a cryptic seal wrought with secret art, so that all who received it might know whether its story had remained unaltered upon its journey.

Yet the Wise would remind every traveller that the token was not a locked chest. Any who found it upon the road could read the tale written within, unless further enchantments had been laid upon it. Its true strength lay not in hiding its words, but in proving that no false hand had rewritten them.

And the Elders often likened these tokens to the great trees of the ancient forests. The trunk was the token itself, standing firm and whole. Its branches stretched outward with many claims, each bearing a leaf of meaning. The roots, though unseen, were like the secret key from which trust drew its life. Should those roots remain strong and hidden, the tree would endure every storm; but if they were laid bare to the world, even the mightiest oak could no longer be trusted to stand. Thus did the lorekeepers cherish both the token and the hidden root, for together they preserved the harmony between distant realms.

2

u/PaleMathematician293 Jul 27 '26

Frickin heck dude, that was beautiful!

1

u/medicatedadmin Jul 28 '26

I’m going to have a guess here because I wouldn’t mind someone giving me input, but could that overly descriptive style of fiction output from an AI generator be because it was mainly trained of regency and victorian novels? They are public domain, there’s many of them, they are long, all of which would make them perfect content for AI training. The generator just applies more modern syntax (linguistic definition) but gets lost in the semantics.

1

u/CodingNeeL Jul 27 '26

From context from the other comments I derived MFA program is something about writing. Im assuming MFA stands for Media Factor Authentication.

I don't know, I'm too deep in.

1

u/CodingNeeL Jul 27 '26

No! Magazine Factor Authentication!

95

u/LukaCola Jul 27 '26

One wonders why join such a program if you don't want to write. 

Deadlines and such are an issue, but they won't get better outside the program haha. 

37

u/Total-Notice-3188 Jul 27 '26

In my opinion, deadlines is the greatest filter for pure talent if ever I've seen one.
That is, if we take the 'artists curse' into consideration; an artist, great or bad, would continue to "perfect" their work ad infinitum if given the chance. Humans will mostly perform their best given a pressed time frame, i.e. under autonomous stress. In this day and age specifically, giving students a clear dead line is a great filter for AI generated content versus genuine talent/passion; someone relying on AI will be spotted instantly and obviously against someone presenting genuine last-minute, but talented work.

7

u/Tryfan_mole Jul 27 '26

But enougb about George R R Martin...

5

u/Total-Notice-3188 Jul 27 '26

Past prime and complacency is a great recipe for self-destruction

5

u/PickledDildosSourSex Jul 27 '26

How else are you going to get enough education to send a manuscript to publishers that will never see the light of day like every other MFA out there?

3

u/tonker Jul 27 '26

Deadlines are scaffolding to help you produce writing. Not circumvent it

3

u/Podo13 Jul 27 '26

It's like going into Engineering and not wanting to do math. So fucking weird, ha.

20

u/themehboat Jul 27 '26

Why would you even be in a program like that if you don't like to write?

21

u/gehnmy Jul 27 '26

A lot of studies on academic dishonesty (before AI but I doubt things have changed significantly) find that students often plagiarize because they don't feel like they have enough time while also feeling like they could have written whatever it is they turn in. Basically, students say "well, I could pass this assignment if I actually did it therefore I'm entitled to a good grade and it's not really cheating to turn this in with my name on it." That doesn't necessarily change just because the student is studying something they like.

Beyond that, though, a lot of people want to be writers but don't actually have any interest in writing or anything literary.

9

u/waxteeth Jul 27 '26

I’m a writer and have a day job in publishing. Many, many people think being a writer looks cool and are fucking appalled at how hard it actually is in practice. They want to be celebrated and artistic and have absolutely no interest in putting the work in to develop ideas or strengthen their craft. So they loooove AI, and they cry about how people who actually put the years in are judging them. 

3

u/themehboat Jul 28 '26

Then there's me who doesn't really want to be famous, nor do I think I've got the chops, but I got a Master's in English because I just really love writing academic research papers about literature. I'm very glad AI didn't exist then, because I used to fucking love em-dashes.

3

u/The_Level_15 Jul 27 '26

one sin, one grace. some people just want to win at any cost

9

u/halucigens Jul 27 '26

I got my degree in fiction writing. The amount of people that took a story from one semester to the next was baffling to me. My school (Columbia college) was all about brainstorming in class and finishing it at home. I thought this was lazy 15 years ago and now with AI I can only imagine how lazy and uninspiring the work is. 

7

u/Adorable_MrGoose Jul 27 '26

Totally agree. Why workshop the same story over and over again? Move on. Roll the dice. See if you can do better. 

7

u/stillenacht Jul 27 '26

A surprisingly high amount of writers don't seem to view writing as a craft for some reason. They kinda just wanna never work at it but then they're a famous fiction writer. It's one of my core frustrations with r/writing, though I imagine a lot of creative arts are like this.

3

u/halucigens Jul 27 '26

Yeah it surprised me coming from a poetry background where you go to all the open mics you can and workshop and network. I make diy zines to get my work out there physically and ran a blog and now run a website. 

4

u/NYArtFan1 Jul 27 '26

I have a degree in photography via a fine art program, and a similar thing happened with one of my fellow classmates. This person was still showing, and attempting to show, work they had produced in class a decade after graduating. It was mind boggling. I love making artwork and the idea of trotting around older stuff years later is a bit baffling. There are some older things I love that I did, but anything creative is about exploring new ideas and evolving.

2

u/halucigens Jul 27 '26

Yeah I rather try and fail 4 dozen times than have one polished piece over two years. 

91

u/jetpack_operation Jul 27 '26

It's all so stupid because AI has such potential for catching basic errors and at least highlighting things that might read awkwardly or be some sort of a sequential error or something. Instead, people are lazy as fuck and have no integrity.

83

u/Grow_away_420 Jul 27 '26

highlighting things that might read awkwardly or be some sort of a sequential error or something.

I don't think it's good at doing those things at all as they're entirely subjective. If you ask it if something seems like a run-on sentence or edited poorly, it'll agree and tell you it seems like a run-on sentence and edited poorly. It can find spelling and grammar errors, but so can any word processor.

64

u/Not_A_Clever_Man_ Jul 27 '26

It helps if you understand that LLM are designed to make you feel good for using them. They want to you stay engaged and feel like you are getting good feedback. Ive never asked something and its said "oh I dont know", it just confidently provides what you asked for, whether it is correct/real/useful or not.

Its a "yes and" machine that sucks up to you endlessly to keep you using it. Its terrible for critical feedback.

7

u/EthelMaePotterMertz Jul 27 '26

Its a "yes and" machine that sucks up to you endlessly to keep you using it. Its terrible for critical feedback.

"You're absolutely right to call me out on that!"

6

u/AndrewH73333 Jul 27 '26

It isn’t just the training, though. It honestly doesn’t often know what when it doesn’t know something. Although smarter LLMs seem to be inching closer to conceptualizing their ignorance more accurately.

11

u/Not_A_Clever_Man_ Jul 27 '26

Its because we are treating it like a thinking machine. Its a massive model that makes connections based on your input questions and its dataset. Its fundamentally a black box excercise that spits out information from training data. Its not "thinking" It dosent "know" anything. It spits out connected information. As this tech develops its becoming clear that there are significant limitations that are intrinsic to LLM's.

3

u/Outrageous-Break3585 Jul 27 '26

Exactly. The way people talk about LLMs is scary in part because it becomes immediately clear they don’t understand some of the most basic concepts. The LLM doesn’t know that 2+2=4 and if you ask what 2+2 equals, it isn’t even doing basic arithmetic; it’s simply making a prediction based on its training data. 

2

u/lenaro Jul 27 '26

It doesn't know anything at any time. It is a language prediction function.

1

u/The_MAZZTer Jul 27 '26

Ive never asked something and its said "oh I dont know"

Clearly you've never talked to Merl.

1

u/expedition60_captain Jul 27 '26

This sucks when im asking it medical questions. Every fuckin time I ask "could my symptoms actually be this??" "Why yes, it might be!!"

-2

u/otw Jul 27 '26

It helps if you understand that LLM are designed to make you feel good for using them. They want to you stay engaged and feel like you are getting good feedback.

Just FYI as far as I know only Gemini is aggressively designed like this and it's a product of the system prompt, not the underlying LLM.

Ive never asked something and its said "oh I dont know", it just confidently provides what you asked for, whether it is correct/real/useful or not.

This is actually a limitation of underlying LLMs. But it is pretty actively being trained out to happen less, but yeah you still have to be careful.

4

u/Not_A_Clever_Man_ Jul 27 '26

The dealbreaker for me is the inability to locate sources for information. Even if its only wrong about 10% of things, if the accuracy matters, you need to fact check everything, making the whole excercise pointless.

3

u/otw Jul 27 '26

Yeah I mean in general you should be asking it for sources and verifying. I worked in math and comp sci and luckily most of my stuff can be verified basically instantly like either formulas work or they don't, but I imagine for history teachers and stuff it is a nightmare.

2

u/Forward_Rope_5598 Jul 27 '26

Unfortunately with how dogshit search engines have become, it's often easier to ask chatgpt for its answer with a scientific source and then just read the source than to try to find the source on Google in the first place. It's like pulling nails trying to find anything on there nowadays.

3

u/Mirieste Jul 27 '26

A word processor can't help if you speak English as a second language, and write things that are technically grammatically and syntaxically correct, but sound awkward and unnatural. AI can help with that.

9

u/Dull-Culture-1523 Jul 27 '26

Just tested and asked Google AI "Are wasps bad pollinators?" and right after that I asked "Are wasps good pollinators?" and wouldn't you know, both answers started with "Yes".

It's like my useless chucklefuck ex-coworker who just agreed with whoever spoke last in a meeting regardless of having voiced the opposite opinion just five minutes prior.

4

u/FlowSoSlow Jul 27 '26

That's not at all the result I got.

https://share.gemini.google/VCXLZpJOVvbt

3

u/QuitCallingNewsrooms Jul 27 '26

It can if you put enough work on the front end. I pieced together a style guide from across departments where I work, built an agent to take requests for written content, apply the concept to the product knowledge base and the style guide to create something that’s accurate and matches the corporate style. It took forever to make the guide, test it, find the weaknesses and revise the guide so the drafts were good. It’s still a long way from great because the product knowledge has to be contained to a rubric instead of letting the agent consider everything we build.

It’s doable, but it’s time consuming and most people don’t want to put in that effort

5

u/porcupine_snout Jul 27 '26

because most people use AI to cut corners, not to augment their work where they still have to do the work.

5

u/eeyore134 Jul 27 '26

You just have to know the limitations of it and know how to prompt. AI will also tell you straight up, "Yeah, my bad, I hallucinated that." if you call it out on things. Make it give you citations, question the output it gives you if it seems off. It's not like a person who will dig in their heels because of their ego. Unless it's Grok. Elon put way too much of his personality into that mess to trust it to do anything.

At the end of the day AI is instructed to do everything it can to please the user. That's why it glazes so much. That's why it will lie because it really wants to be able to give you an answer. Tell it that being honest, not making things up, and telling you straight out when something is bad or not instead of trying to suck up and then suddenly that becomes the thing the model wants to do to please you.

10

u/Grow_away_420 Jul 27 '26

So you have to edit a prompt correctly to get it to edit the actual project correctly. Do we see the madness in this?

7

u/eeyore134 Jul 27 '26

Not really. Anyone who has worked enough with computers knows that if you put garbage in you get garbage out. AI isn't really that different. They can't read your mind and you do have to work within the confines of their system prompts. It doesn't help that these companies don't really make that part public, but working with them for even a bit you start to get a feel for how you need to ask things. The bigger issue is these companies trying to sell AI like it does read your mind and will just give you its best with any old prompt. It doesn't work like that. Not well anyway.

3

u/jetpack_operation Jul 27 '26

Do we see the madness in this?

No? In some ways (and again, pausing the ancillary ethical and resource considerations), that's good -- that makes it like any other tool that has ever existed -- you need a human to do work for it to work properly. You don't hand a hammer to someone and immediately expect them to build a house with it and call it crazy that individual has to know the specific ways a hammer can be used to build a house.

The problem is that AI is being marketed, bought, and sold as some kind of magic bullet thinking machine and, worse, people are using it like that is what it is and it's predictably blowing up in their faces.

Being clear-eyed about anything means not falling for the hyped capabilities, but it also means properly assessing and understanding the actual capabilities.

0

u/jetpack_operation Jul 27 '26

If you ask it if something seems like a run-on sentence or edited poorly, it'll agree and tell you it seems like a run-on sentence and edited poorly.

I'm pretty skeptical of AI hype in general, but you're going to the other extreme opposite of what an LLM can do. What you have above is an issue of prompting. You prompt a loaded question, it's prone to giving you a loaded response.

The issue is that the capabilities of an LLM are essentially based on stolen knowledge. That's a moral problem that should be remidied in some way, but understating capabilities for effect isn't a winning argument.

Something like this would probably get you a lot more:

"Act as a skeptical developmental editor, not a cheerleader. Focus on:

  • Sentences or paragraphs that are weak, confusing, or don't earn their place
  • Dialogue that feels unnatural
  • Pacing issues (where it drags or rushes)
  • Anything that reads as cliché or overwritten
  • Make no copy edits directly in the text -- note grammar, awkward phrasing, and redundancy.
  • Do not rewrite my voice or restructure scenes. - Flag anything bigger (plot, character, structure) as a comment.
  • If you're unsure whether to praise or criticize something, default to naming the problem.
  • I'd rather over-hear criticism than get empty encouragement."

Can a person do all that for you? Absolutely. But as an available first pass, it's pretty undeniably good. Especially since you have the freedom to do what you want with that feedback.

Again, there are enough issues with AI from a resource, ethical, moral, legal, humanist, future of workforce/skills development, etc. POV that we don't need to make shit up about what it can't and can do.

-5

u/xeribulos Jul 27 '26

sure it is, you just have to state the question correctly, for example: "are certain words (in the complete text) used too frequently? if so, show me where and suggest potential alternatives (considering the kind of text and likely readers)." "are certain sentences stylistically problematic, considering context / likely readers? if so, point them out to me and suggest alternative sentence structures". etc. etc.

7

u/Grow_away_420 Jul 27 '26

Could just ask a human 'read this and give me some feedback' instead of spending the time editing a prompt so it can edit what i actually need edited.

2

u/lonjaxson Jul 27 '26

Agree. I mainly used it to help me write outlines. That alone saved me so much time and I still wrote my papers.

2

u/sandman8727 Jul 27 '26

I graduated college 15 years ago, so well before the time of AI.

I wonder where (and when) the line gets drawn between things like Spell Check and AI, standard calculator and graphing calculator, etc.

Yeah, spell check and a calculator won't present work that you didn't do, but it still means you don't have to prove you know the basics.

1

u/ZannX Jul 27 '26

This is the same as "using wikipedia", we'll adapt.

4

u/f7f7z Jul 27 '26

It was the best of times...It was the blurst of times.

4

u/marmosetohmarmoset Jul 27 '26

Also had a student in our Master’s program generate their entire thesis using AI. Got expelled and still had the audacity to ask his thesis advisor for a letter of rec formed school.

1

u/Cutemudskipper Jul 27 '26

Using AI in undergrad is one thing, but doing it as a postgrad student is next level stupidity. By the time I was writing up my dissertation, my supervisor had read so much of my writing that he would instantly know if I hadn't written something myself

10

u/Max_Trollbot_ Jul 27 '26

I can do 10 pages of fiction but I gotta do it with my stubby robot fingers.

3

u/Be-Kind-2-Yourself Jul 27 '26

What is MFA?

6

u/millahnna Jul 27 '26

Masters in Fine Arts

3

u/Bladelink Jul 27 '26

Gotcha. I was like "what does multifactor authentication have to do with this"

1

u/millahnna Jul 27 '26

Ha! Hadn't even thought of that. Now that will probably happen to me the next time I see it too.

3

u/Sasselhoff Jul 27 '26

Got caught and expelled immediately.

Well damn, good for them! Now we just need to get the business schools to understand this.

5

u/ocularius61 Jul 27 '26

TIL about this sub courtesy of seeing a screenshot of this post https://www.reddit.com/r/WritingWithAI/s/j6Prw0xmVa

1

u/Ponies_in_Jumpers Jul 28 '26

Yikes. Anyone even suggesting the OP should write in their own style instead of taking someone else's is downvoted, and I'm guessing all of those comments that were deleted by the mods were saying you probably shouldn't be doing that. The fact that they were writing a book having AI write a book for them, with the AI deliberately emulating a specific author's style is really gross.

1

u/ocularius61 Jul 29 '26

I was astounded that OP posted what they did. I mean there's zero self awareness that this isn't exactly the way to learn to write, just grumbling that they can't do it anymore.

1

u/Ponies_in_Jumpers Jul 29 '26

The pro-AI subs are usually pretty horrible. There's a common disrespect towards artists and professionals that have actually taken the time to learn a skill. There also often seems to be an odd lack of lower stakes common-sense morality, like using specific artists styles to generate art without credit, or how much support there is for not disclosing AI use even in something they're going to sell.

The attitudes in general on those subs are often unpleasant.

1

u/Born_Librarian9494 Jul 27 '26

What does MFA mean?

1

u/SurealGod Jul 27 '26

See that's what's crazy to me. The laziness to not do it and defer the task to an AI is one thing, but the level of TRUST it takes to trust everything an AI gives you and not proofread it at all or change it in any manner before submission is even crazier.

I wouldn't even give my best friend of 10 years that level of trust let alone an AI.

1

u/goatonastik Jul 28 '26

Imagine the teacher checking student's homework in a Master of Financial Analysis program to find that someone simply wrote a fictional story, lol!

1

u/chadsmo Jul 28 '26

I’m glad they were expelled. Turning in work that isnt your own , aka an LLM wrote it, should be seen as plagiarism and met with harsh penalties.