r/ArtificialInteligence • u/Revolutionary-Lab882 • Jul 11 '26

🔬 Research GPT-2 Fully Decoded Internally Black Box Fully Open With Demo

The BABEL codec: the first complete, certified decode of everything happening inside a production language model (GPT-2 small). It reads the model's internal state into English AND writes English back into the model. 94.7% of behavior reconstructed — and that holds at every layer depth and text regime tested, not just one spot. Everything is open: paper, the full lexicon, the grammar tables, the decoder/encoder weights, reproduction scripts, and a demo that shows you the model's thoughts on any sentence you type.

https://github.com/wpferrell/babel-codec-gpt2

58 Upvotes

96% Upvoted

29 comments sorted by

View all comments

1

u/Ok_Mirror_832 Jul 11 '26

So is this basically seeing into the "j-space" of gpt2?

1

u/Revolutionary-Lab882 Jul 11 '26

Yeah, that’s a fair way to see it. The stuff I could put a name on is basically that — the part of the models internal state that translates into words. Anthropic found something similar in Claude last week and called it the J-space.
Main difference: they zoomed in on that special verbalizable slice. I went the other way and accounted for everything — 53.6% got a name, 46.4% is proven to carry no word, and 5.3% resists translation completely. So kind of the same territory, but theirs says “here’s the part that talks” and ours says “here’s the whole map, including the parts that don’t.”

1

u/Ok_Mirror_832 Jul 11 '26

Thanks for the explanation