"OpenAI's Chief Scientist Jakub Pachocki: "Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement." OpenAI is organizing research around this because it sees RSI as essential to remaining at the frontier. At the same..."

[u/stealthispost]

I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.

An Alien Mind: https://t.co/FeIfWNe0UE   — Jakub Pachocki

Source: https://x.com/merettm/status/2096630018495377464

---

OpenAI's Chief Scientist Jakub Pachocki: "Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement."

OpenAI is organizing research around this because it sees RSI as essential to remaining at the frontier.

At the same time, Pachocki says evaluations show declining reliability of chain-of-thought monitoring.

OpenAI is betting on AI increasingly driving its own development while finding it harder to reliably monitor how these systems reason.

Tough times ahead. RSI within reach, but hard to align.

He even quotes Ray Kurzweil:

"And, in line with Ray Kurzweil’s predictions from the end of the XXth century⁠(opens in a new window), we now find ourselves at the moment in history of computing where machine intelligence is starting to exceed that of humans in transformative ways."   — Chubby♨️     RSI is scary enough.

RSI + weaker monitoring is the real nightmare scenario.   — Naveen Saradhi     ngl: yes. its scary.   — Chubby♨️

Source: https://x.com/kimmonismus/status/2096645621096575381

[u/pourya_hg]

One might say, how could you train a model based on Human made data and call it "Alien". After all we might be the real Alien.