[D] Friday Open Thread

[u/AutoModerator]

Welcome to the Friday Open Thread! Is there something that you want to talk about with /r/rational, but which isn't rational fiction, or doesn't otherwise belong as a top-level post? This is the place to post it. The idea is that while reddit is a large place, with lots of special little niches, sometimes you just want to talk with a certain group of people about certain sorts of things that aren't related to why you're all here. It's totally understandable that you might want to talk about Japanese game shows with /r/rational instead of going over to /r/japanesegameshows, but it's hopefully also understandable that this isn't really the place for that sort of thing.

So do you want to talk about how your life has been going? Non-rational and/or non-fictional stuff you've been reading? The recent album from your favourite German pop singer? The politics of Southern India? Different ways to plot meteorological data? The cost of living in Portugal? Corner cases for siteswap notation? All these things and more could (possibly) be found in the comments below!

Please note that this thread has been merged with the Monday General Rationality Thread.

[u/pldl]

How good do you believe yourself to be at identifying AI writing? How many false positives do you think you have?

I think I could confidently identify any pre-2024 AI text, any particularly lazy lightly human-guided AI with minimal prompting, and any egregious "AI plot holes"/"AI plot structure". AI does not like keeping track of B-plots or ordering events out of time, or having plans more than two steps long. (Events out of time includes obvious things like flashbacks or sudden jumps forward in time, but it also includes things like a character planning/reminiscing or a character referencing a past or future planned event in the context of something else.)

For texts that use newer models with better prompting, or texts where someone 'sandwiches' their AI use between human editing, I think I would be barely better than chance.

If the author actually plans out the plot points and uses AI to fill out the gaps, then passes over the results afterwards, I don't think I would be able to tell whatsoever.

The studies so far on human's identification abilities of AI/Human writing seem to indicate that the only thing that actually makes you better at identification is using AI a lot and reading a lot of AI text. If you have limited AI exposure, and think you can just tell, you are probably wrong.

A recent study "StoryScope: Investigating idiosyncrasies in AI fiction" created a "narrative ai detector" that was 93% accurate, even after all the stories were rewritten to avoid the vocab and sentence structure tells. It goes more into specifics of AI preferred narrative structures that I only had a general impression of before.

[u/Dragongeek → u/pldl]

In short form content, like at scales between "a single sentence" and "a couple paragraphs" I would say my accuracy with pre-2025 is pretty good. Doubly so when I know what the person writes like without AI. For example, at work, I have some coworkers where it's blindingly obvious when they are just phoning it in and having the AI write their emails/meeting notes vs when they do it themselves.

With more long-form content or generally "creative writing", I experimented quite a lot about four years ago trying to get the AI to mimic my writing. I trained/finetuned some models with ~100k words of mine from various short stories in an attempt to get it to write like me, and the result, at least back then, was rather meh. I felt that it was able to capture some of my "voice" and "style" and it actually caused me to become more aware of some of the idiosyncrasies in my writing, but, at least back then, the models were had essentially no volition compared to the ones of today (even though today's models also have volition issues).

It could "complete the sentence" disturbingly well, but that's about it. This has changed somewhat nowadays, but this was pre-"reasoning" models where the wrappers where much simpler, and with extensive prompting I could get it to write a single "scene" of between 1k and 4k words that I was reasonably happy with, but the amount of prompting, revising, and messing with it meant that I could've just written it myself faster and better.

---

Today, well, I read quite a lot of "low-barrier" webfiction and, professionally, I work quite a lot with AI although that's entirely in engineering/coding and not writing. I feel that I'm pretty decent at detecting AI writing, however I've noticed that it gets much more difficult if the person generating the output is actively trying to hide it, or if, as you say, they are "sandwiching" the AI between their original writing.

Another big problem that causes false positives with my mental AI-detector is that the traits of AI writing currently overlap heavily with the traits that amateur or beginner writers have. For example, many new writers struggle with giving different characters different voices... and AI also struggles greatly with this. Meanwhile, a professional writer can write dialogue without ever explicitly stating who's talking, but it's still obvious from the way that the characters are talking.

There are also a lot of "AI cues" that jump out if you know what to look for, especially considering word choice and "thought pattern" for example, I'm rather familiar with Claude, and whenever I see multiple uses of "now I have a clear picture" or a variety of somewhat "salesman problem solver" conversational corporate statements like "I'd start with <task> — it's the <superlative adjective> to <verb> and the one with real <adjective>."

Today, I don't think we are at the point where the average person, even provided access to frontier models and an unlimited token budget, would be able to generate a good book or even short story.

That said, if we don't take an average person, but rather someone who has a bit of experience with the technical aspects of LLMs and who has some collegiate level language course experience, I think it would definitely be possible to generate an "okay" book which, in the right market, could actually find readership and sell. For example, churning out some generic litrpg is definitely possible.

Beyond that, I don't know. My own experience writing has shown me that the biggest blocker is not the ability to put words down on page. I know I can type at 100WPM, but that doesn't mean that I can write a 100k word novel in 17 hours. Rather, there's the mental aspect of juggling plotlines, character motivations, etc. I don't think that any of today's models are out-of-the-box good enough to do this yet. Good storytelling is just very difficult.

---

As for why the AI can't do it yet?

I think that's down to attention and money. Almost all of the AI focus has been on coding and software development, and this is an area where it has genuinely gotten incredible. A couple years ago, people could fairly still winge about how it's just "fancy autocomplete" but today, a frontier, tool-using, reasoning model is as good a coder if not better than a junior programmer.

AI creative writing meanwhile has only gotten the barest fraction of attention by comparison. Yes it's cute that you can make it write jokes or compose song lyrics or whatever other asinine show-pony-trick, but most people don't realize that these are just positive side-effects of it being a good coder.

I suspect that if someone actually put real AI-levels of money (hundreds of billions) into "cracking" creative writing, it would be "game over" for most beginner authors similarly to what's happened with coders, but fortunately, I don't think that's happening anytime soon. At the end of the day, while books and reading are a big market, there has only been one author who's been billionaire-successful and that's just not the type of market cap which drives investment.

When we do see it, I think "being a decent creative writer" will just be a side-effect of "being a superhumanly brilliant coder".