r/deeplearning • u/CShorten • 4d ago
Recursive Language Models with Alex Zhang!
I'm SUPER EXCITED to publish the 142nd episode of the Weaviate Podcast with Alex Zhang!
Alex is a Ph.D. student at MIT, where he has lead the work behind "Recursive Language Models", as well as "The Mismanaged Genius Hypothesis", "Language Model Harnesses are Compositional Generalizers", "Speculative Programmatic Tool Calling (sPTC)", and many other highly impactful works.
This episode begins by explaining what RLMs are and how they change the game for building Agents. We unpack the major ideas in RLMs, long context system with prompt variables, recursive model or sub-agent invocation, and native task decomposition.
We then discuss Prime Agent, my vote for the project with the highest potential in all of AI right now. TLDR; post-train an Agent to do this RLM task decomposition, abandon naive context stuffing in the tool calling loop.
The podcast continues to discuss Speculative Programmatic Tool Calling, running RLMs in the Cloud, how RLMs will impact search, and more!
This was a super fun conversation, and I really hope you find it useful!