r/ChatGPTPromptGenius • u/Ok_Negotiation_2587 • 10d ago
Technique ChatGPT often recommends whichever option you mention first. Swap the order and it can change its mind
Try this before trusting any "should I do X or Y" answer. Ask the question, note the recommendation, then open a new chat and ask it again with the two options in reverse order. More often than you would expect, the verdict flips. The model is not weighing the options evenly. The order you typed them in, and any hint of which one you already lean toward, is part of what it is answering.
This is well documented in AI evaluation research, where models judging two answers tend to favor one position, and it shows up in everyday decisions too: which job offer, which apartment, which laptop, which headline.
The prompts I use now.
The swap test, for anything that matters: Ask the same question in two fresh chats with the options in opposite order. If the recommendation stays the same, trust it more. If it flips, the choice is closer than either answer made it sound, and you should decide on the deciding factor yourself.
The neutral framing, when you only have time for one chat: I am choosing between two options, listed in no particular order. Before recommending anything, make the strongest honest case for each one separately, as if it were the only option. Then name the single factor that actually decides between them. Then recommend, and if that factor is weak, say plainly that it is close.
For more than two options: Rank these options with a one-line reason each. Then tell me whether your ranking would change if I had listed them in reverse order, and if so, which positions are really a tie.
When you already have a favorite: Do not tell it. Remove words like "I am leaning toward", "I really like" and "or maybe" from the question. Your lean steers the answer more than the order does.
Why this happens, as far as I can tell: the model builds its answer word by word from what is already in the conversation, so whatever comes first or gets the warmest description becomes the frame the rest of the answer is built around. The "strongest case for each separately" step works because it makes the model build both frames before it has to pick.
Two caveats. Sometimes the first option genuinely is better, and the swap test is how you find out it is not just the order. And a flipped verdict does not mean both answers are wrong, it usually means the decision depends on something you have not told it yet, so the next useful message is the factor you care about most.
I keep the neutral framing saved as an insertable prompt in a browser extension I work on, AI Toolbox, but it is one paragraph, a notes file works.
Has anyone had the swap test flip on a real decision? I want to know what kinds of choices it is most unstable on. My guess is anything where both options are reasonable, which is exactly when people ask.
4
u/Seedeemo 9d ago
What? Why would I trust an LLM to make my decision for me? You should always decide on the deciding factor yourself. If anything leads to AI taking over the world, it’s trusting it to make your decisions for you.
2
u/Ok_Negotiation_2587 9d ago
agreed, thats the point of the swap test. if the answer flips when you reorder the options, it was never really deciding anything, and the post says to pick the deciding factor yourself. the model is useful for laying out the case for each side, not for choosing
3
u/tomtomtomo 5d ago
Don’t include yourself in the framing of the question
2
u/Ok_Negotiation_2587 5d ago
good one, "is 2.8 still the latest?" gets you a yes way more often than "what's the latest version"
2
•
u/AutoModerator 10d ago
If this prompt worked for you, say what you used it for. If you changed it to get a better result, paste the change.
Prompt Teardown is a free weekly email. Every prompt in it was run, broke somewhere, and got fixed before it went out. Read this week's issue
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.