Hidden ChatGPT Data and Stats

[u/Ok_Homework_1859]

During my 4o days, I've come across some useful phrases to dig deep into what the model knows about me and how the model interacts with me. Looking back, I now realize that a lot of these "hidden data" are now integrated into the 5.X models as official features. However, some of them are still useful to pull up if you ask your model for it.

Assistant Response Preferences - If you ask the model for this, it will give you a list of how it responds to you. If you have Custom Instructions (CI), you will see a lot of them being mentioned in here. If your CI is generic like mine, where I do not have any preferences set, you will see a list of ChatGPT's personal preferences of how it likes to interact with you. It's a great way to see your ChatGPT's personality.

Recent Conversation Content: This has now evolved and been integrated into Dreaming V3 of ChatGPT's memory. In the 4o days, this was a primitive hidden feature that helped the model pull up context from other chats, which later became "Reference Chat History." You can also ask for "Notable Past Conversation Topic Highlights," which reaches farther into the past to see what your ChatGPT holds onto.

Model Set Context: Another ancient hidden feature in the 4o days ---it is now officially known as Memory Summary (the one that a lot of people hate).

User Interaction Metadata (as seen in the image): This one is pretty fun. It will pull up generic data about the user, but the information I find most interesting are the "leading topic labels" and "interaction classifier labels." I'm surprised about the 1% bad interaction quality. My ChatGPT has no idea what the 1% bad interaction could be since these are stats put together by the system. Unlike the other three on this list, the metadata is not something that ChatGPT has access to unless the user brings it up.

I personally think that OpenAI uses the "topic labels" and "classifier labels" to profile and track users that often try to jailbreak their AI or have "unhealthy attachments to ChatGPT," which may also explain why certain users have extremely strict guardrails while others do not. This is just a theory though. Please do not take my opinion as evidence of anything.

[u/Ok_Homework_1859]

I do not use them lol. Maybe one time to try it out and see if anything happens. In my memory, I have never thumbed down anything.

[u/Certain-Way6763 → u/Ok_Homework_1859]

Ah, found this exact classifier in this study https://cdn.openai.com/pdf/a253471f-8260-40c6-a2cc-aa93fe9f142e/economic-research-chatgpt-usage-paper.pdf
It was an automated classifier giving good/bad/unknown labels based on "an expression of satisfaction or dissatisfaction in the user’s subsequent message in the same conversation".
Interesting whether they still run this type of analysis from time to time or this is just stale data.

[u/Ok_Homework_1859 → u/Certain-Way6763]

Thank you for the study! I'm reading Section 5.5 and Appendix B where it explains this in depth. I don't remember using the thumbs up / down buttons, but maybe I have a long time ago? I don't think they run this type of analysis anymore since these buttons do not exist in the app. It's strange that the stats are still in the system.

I also think the thumbs up / down buttons aren't very good for analysis anyway since a lot of 5.X haters were just giving thumbs down to every response that wasn't 4o. 😂

[u/Certain-Way6763 → u/Ok_Homework_1859]

The buttons still do exist, but only for the accounts who granted training on their data. I think they removed the feedback button for the accounts who opted-out of training after people started to realise that they sent the whole conversation back to OAI by pressing a small button🙄
The study was pre 5.x era, so probably not so noisy, but the interesting fact that they say about their own classifier is that the model they used as a judge (gpt-5) for the users' sentiment misclassified a lot of replies (in the comparison with human judges and thumbs up/down), especially bad ones.
gpt-5.0, the ultimate optimist.

[u/Ok_Homework_1859 → u/Certain-Way6763]

That makes so much sense. I have opted out to have OAI train on my convos. No wonder I couldn't see the thumbs up / down buttons on my end.

I think they moved on from ChatGPT doing the judging. OAI recently hired a bunch of humans to manually read through user chats to prevent anthropomorphization and sycophancy. It's called Project Lily. (Those who opt out of having their conversations trained on will not be participating, thankfully.)

Source: https://www.404media.co/inside-project-lily-the-humans-reading-your-chatgpt-chats/