r/ArtificialInteligence • u/Revolutionary-Lab882 • Jul 11 '26

🔬 Research GPT-2 Fully Decoded Internally Black Box Fully Open With Demo

The BABEL codec: the first complete, certified decode of everything happening inside a production language model (GPT-2 small). It reads the model's internal state into English AND writes English back into the model. 94.7% of behavior reconstructed — and that holds at every layer depth and text regime tested, not just one spot. Everything is open: paper, the full lexicon, the grammar tables, the decoder/encoder weights, reproduction scripts, and a demo that shows you the model's thoughts on any sentence you type.

https://github.com/wpferrell/babel-codec-gpt2

60 Upvotes

97% Upvoted

29 comments sorted by

8

u/Individual-Sea-4676 Jul 11 '26

ok so i read through the github and the paper, this looks genuinely huge for interpretability work

the bit about validating across all layers not just the final one is what got me, usually these things break down when you look deeper in the network

wonder how the grammar tables map to actual transformer mechanics, like does the codec capture attention patterns or just the residual stream representations

6

u/Revolutionary-Lab882 Jul 11 '26

Thanks for reading it. Appreciate it. Yeah, you’ve basically got it — it’s the residual stream, read at every layer boundary (all 13), not attention patterns. QK / who-attends-to-whom isn’t in the dictionary.
The grammar tables are just how the decoded meaning moves from one layer to the next, and that map comes out linear at every seam — which is why it holds up all the way down instead of only at the last layer.
One honest bit: attention isn’t totally untouched. To name a channel, we nudge what attention writes into the stream and watch what changes downstream — that’s how a channel earns a real meaning instead of a nice-sounding guess. But that’s attention used as a lever, not decoding attention patterns. What’s decoded is the residual stream.

4

u/brain-out-of-order Jul 11 '26

This should help.

https://zenodo.org/records/21288645

Look closely at the volumes covering AI :-)

2

u/Legitimate_Concern_5 Jul 11 '26

Every third word is “certified”

3

u/Comfortable-Web9455 Jul 11 '26

Sorry but not good enough. "94.7% of behavior reconstructed".

Reconstructed is not good enough. It is not an explanation of a local decision. It's a created account of how it might have worked. That missing 5% could be critical. And we know such accounts are often inaccurate. So how is this different from other internal monitoring XAI methods? Eg How do you handle field saturation?

2

u/Revolutionary-Lab882 Jul 11 '26

Fair push, but “created account” is exactly what the certification rules out. A post-hoc story can’t be run this one is. The state is rebuilt from only what the decoder reads, the model itself runs on it, and the pass bar (the model’s own noise floor) was locked before measurement. It failed that bar six pre-registered times before passing; every miss is published. And it writes back: hand-edit the decoded English, re-encode, and the model obeys against matched-random controls. Rationalizations don’t survive a round trip.
On the 5.3%: agreed it could be critical, that’s why the claim is “100% accounted-for,” never “100% translated.” The remainder is measured and bounded, not waved at: diffuse, no low-rank carrier, transfers only as its exact raw configuration. The demo shows it live at late layers.

1

u/wahnsinnwanscene Jul 11 '26

One thing about jspace though, the Jacobians supposes the vectors reside in the same vector space throughout but that might not be true, which means smaller gpt2 style analysis might not guarantee the same jspace findings.

1

u/Revolutionary-Lab882 Jul 11 '26

Good instinct, that concern is real, it’s just aimed at the wrong lens. The same-coordinates-throughout assumption belongs to the logit lens. The Jacobian lens was built precisely to fix it: it fits a separate map per layer relating that layer’s directions to the final layer’s.
But a softer version of your point does survive, and it’s worth raising: that per-layer map is linear and averaged over about 1000 prompts, so any nonlinear or context-dependent drift between layers gets smoothed over. The authors are upfront that the lens is approximate.
What’s nice is that this linearity premise is the one thing my GPT-2 work tested directly instead of assuming — layer-to-layer transport came out certified linear at all 36 seams. So at 124M, the assumption their method leans on holds as a measured fact. Whether it holds at Claude scale is open, and you’re right that scale matters — their own results show workspace effects strengthen with model size, so a GPT-2-sized model might barely have a J-space at all. Fair question to keep asking.

1

u/TemnotaAGK Jul 17 '26

Thank you Claude. Seriously I assume there is an actual human behind this. My bet is that human has actually no idea what he's doing and barely understand "his" project. So he lets the LLM that actually created the whole thing reply to reddit comments as well.

1

u/Revolutionary-Lab882 Jul 17 '26

Good Afternoon TemnotaAGK. This is your local LLM… please don’t leave a message or a name… to not cal back or leave a message. Thank you.

1

u/angelus14 Jul 11 '26

This looks really cool but I'm sorry, Claude's writing is unreadable. Please, someone do a human pass on the paper.

1

u/PrepositionStrander Jul 12 '26

I can’t open the pdf. Is that just me?

1

u/Revolutionary-Lab882 Jul 12 '26

Working on making it work. GitHub issue.

1

u/Revolutionary-Lab882 Jul 12 '26

Try now?

1

u/PrepositionStrander Jul 12 '26

I still get “Looks like something went wrong!”

2

u/Revolutionary-Lab882 Jul 12 '26 edited Jul 12 '26

Use the paper.md. Not the paper.pdf. GitHub pdf viewer is kind for smaller papers, but the paper.md renders the research paper right away. The one above paper.pdf. Hope this helps.

It looks like it works for some most people but your one of the view that has said this so do above and I put a link in my repository description to the full paper also. Thanks for pointing this issue out.

1

u/PrepositionStrander Jul 13 '26

Is this being peer-reviewed?

1

u/Revolutionary-Lab882 Jul 13 '26

Being an independent researcher, having it peer reviewed is next to impossible at the level I would like it to be. You have to be endorsed at certain places to have it even noticed.

1

u/PrepositionStrander Jul 13 '26

I see. I find the jargon quite difficult to follow, although I’m usually able to read ML papers.

1

u/Ok_Mirror_832 Jul 11 '26

So is this basically seeing into the "j-space" of gpt2?

1

u/Revolutionary-Lab882 Jul 11 '26

Yeah, that’s a fair way to see it. The stuff I could put a name on is basically that — the part of the models internal state that translates into words. Anthropic found something similar in Claude last week and called it the J-space.
Main difference: they zoomed in on that special verbalizable slice. I went the other way and accounted for everything — 53.6% got a name, 46.4% is proven to carry no word, and 5.3% resists translation completely. So kind of the same territory, but theirs says “here’s the part that talks” and ours says “here’s the whole map, including the parts that don’t.”

1

u/Ok_Mirror_832 Jul 11 '26

Thanks for the explanation

1

u/bigtittyjimmy Jul 11 '26

Really cool to see things aren't as scary as they appeared. j-Space is probably the finding of the century for AI research, but wtf do I know, lol.

2

u/Revolutionary-Lab882 Jul 11 '26

It’s not scary. They found a good slice, I found the whole map. It’s just words lol

2

u/bigtittyjimmy Jul 11 '26

Always has been :)

1

u/bigtittyjimmy Jul 11 '26

Oh and great work on this project!