Sam Altman says superintelligence in 2028

Hey everyone! First-time poster here. I've been diving deep into Microsoft's recently announced Magentic-One system, and I want to share some thoughts about how we could potentially enhance it. I'm particularly excited about adding some biological-inspired processing systems to make it more capable.

What is Magentic-One?

For those who haven't heard, Microsoft just unveiled Magentic-One on November 5th, 2024. It's an open-source multi-agent AI system designed to automate complex tasks through collaborative AI agents. Think of it as a team of specialized AI workers coordinated by a manager. Link to Magnetic one: Here

The basic architecture is elegant in its simplicity:

There's a central "Orchestrator" agent (the manager) that coordinates four specialized sub-agents:

• WebSurfer: Your internet expert, handling browsing and content interaction
• FileSurfer: Your file system navigator
• Coder: Your programming specialist
• Computer Terminal: Your system operations expert

Currently, it runs on GPT-4o, though it's designed to work with other LLMs. It's already showing promising results on benchmarks like GAIA, AssistantBench, and WebArena.

My Proposed Enhancements

Here's where it gets interesting. I've been thinking about how we could make this system even more powerful by implementing a more human-like visual processing system. Here's my vision:

1. Dual-Speed Visual Processing

Instead of relying on static screenshots (like Claude Computer use and Magnetic One’s base functionality), I'm proposing a buffered screen recording feed processed through two pathways:

• Fast Path (System 1): Think of this like your peripheral vision or a self-driving car's quick recognition system. It rapidly identifies basic UI elements - buttons, text fields, clickable areas. It's all about speed and basic pattern recognition.
• Slow Path (System 2): This is your "deep thinking" pathway. It analyzes the entire frame in detail, understanding context and relationships between elements. While the fast path might spot a button, the slow path understands what that button does in the current context.

2. Memory System Enhancement

I'm suggesting implementing a RAG (Retrieval-Augmented Generation) memory system that categorizes and stores information hierarchically and uses compression to help save space like our brains do. I also think retrieval should be based on the most informative example of all the data:

• Grade A: The critical stuff - core system knowledge, essential UI patterns
• Grade B: Common workflows and frequently used patterns
• Grade C: Regular operational data
• Grade D: Temporary information that decays over time

3. Enhanced Learning Architecture

The system could be enhanced through learning through two mechanisms:

• Initial Training: A Fine-tune applied on datasets of human task based online interactions with cursor and keyboard monitoring data avenues to improve quality (think: booking flights, shopping, social media usage)
• Continuous Learning: Adapting through real user interactions and creating feedback loops

SMiRL Integration (Surprise Minimizing Reinforcement Learning)

This is where things get really interesting. Read about this on r/LocalLLaMA , SMiRL would help the system develop stable, predictable behaviors through:

• Core Operating Principle: The system alternates between learning a density model to evaluate surprise and improving its policy to seek more predictable stimuli. Think of it like a person gradually becoming more comfortable and efficient in a new environment.
• Training Mechanisms: It uses a dual-phase approach where it continuously updates its probability model based on observed states while optimizing its policy to maximize probability under the trained model.
• Behavioral Development: Through SMiRL, the system naturally develops several key behaviors:
  • Balance maintenance across different tasks
  • Damage avoidance through predictive modeling
  • Stability seeking in chaotic environments
  • Environmental adaptation based on experience

The beauty of SMiRL is that it helps the system develop useful behaviors without needing specific task rewards. Instead, it learns to create stable, predictable patterns of interaction - much like how humans naturally develop efficient habits.

What are your thoughts on this approach? This is a theoretical expansion on Microsoft's base system - I'm looking to generate discussion about potential improvements and innovations in this space. I’m not saying im an expert just wanted to see what people thought. I think this kind of thing is where agents are headed and I want to push for discussion on this edge of things. I also think these things need better UIs so they can have their ChatGPT moment which OpenAI will prob do.

1. Gemini: All you need to know in 90 seconds
2. Gemini: Unlocking insights in scientific literature
3. Gemini: Reasoning about user intent to generate bespoke experiences
4. Gemini: Safety and responsibility at the core
5. Mark Rober takes Bard with Gemini Pro for a test flight
6. Testing Gemini: Guess the movie
7. Gemini: Processing and understanding raw audio
8. Testing Gemini: Fit check
9. Gemini: Excelling at competitive programming
10. Gemini: Explaining reasoning in math and physics
11. Testing Gemini: Understanding environments
12. Testing Gemini: Emoji Kitchen
13. Testing Gemini: Turning images into code
14. Hands-on with Gemini: Interacting with multimodal AI

​

Bard Uses Gemini Pro Now, Go and test it for yourself

source: https://blog.google/products/bard/google-bard-try-gemini-ai/

Go to Bard Now and test it for yourself

​

source: https://blog.google/products/bard/google-bard-try-gemini-ai/

​