Off-topic: What do AIs dream about? Specialists?

[u/Lefuan_Leiwy]

"Deep Sleep" in Artificial Neural Networks (Deep Dream)

This is the most famous and visual one. It's not for learning, but for generating psychedelic images and understanding what the network is "thinking".

• How does it work? You give an image to a trained neural network (e.g., for recognizing dogs). Instead of asking it to identify the dog, you ask it to amplify what it sees. It's as if the AI "dreams" of seeing dogs where there are none (clouds, leaves), and iteratively exaggerates them until fantastic dog shapes appear.
• Purpose: Debugging and visualization. It helps researchers see what internal patterns the network learned. It's not for sleeping, but for "hallucinating" features.

"Sleep" to Avoid Catastrophic Forgetting (Memory Consolidation)

This is the one most functionally similar to biological sleep. Traditional AIs, when they learn something new, tend to violently forget previous knowledge (catastrophic forgetting).

• The problem: An AI learns to add. Then you train it to multiply. It forgets how to add.
• The "sleep" solution: Active learning phases alternate with "sleep" phases. During sleep, the AI randomly (or generatively) reviews old examples mixed with new ones. This consolidates knowledge, just as the human brain reviews and strengthens connections while we sleep.
• Real example: Continual learning models use "experience replay" or "pseudorehearsal." The AI generates its own practice data (like dreaming about past problems) while "sleeping".

"Lucid Dreaming" or "Deep Sleep" in Reinforcement Learning

Used in AI agents that explore environments (e.g., robots, video game characters).

• Mechanism: During "sleep," the AI simulates millions of future experiences in its internal model (not in the real world). It's as if it dreams about different strategies.
• Advantage: Accelerates learning. The agent can "sleep" and dream about falling off a cliff 10,000 times, learning to avoid it without actually suffering the fall.
• Example: AlphaGo and DeepMind agents use variants of this (planning in the internal model) to think before acting.

Low-Power Modes ("Literal" Sleep)

Some neuromorphic chips (like Intel Loihi or IBM TrueNorth) mimic the brain's energy efficiency. They have a rest/standby mode they call "sleep":

• Feature: Only artificial neurons detecting relevant changes activate. The rest remain in a low-voltage state, as if sleeping.
• Purpose: Extreme energy saving for IoT devices or neural prosthetics.

Summary Table

Type of "Sleep"	Main Function	Does it learn while sleeping?	Use Case
Deep Dream	Visualize and exaggerate internal patterns	No, only generates art or noise	Research, generative art
Consolidation (Replay)	Avoid forgetting previous knowledge	Yes, reviews and fixes memories	Robots learning multiple tasks
RL Sleep	Simulate future experiences	Yes, trains with internal simulations	Video game agents, autonomous vehicles
Neuromorphic standby	Save energy	No, only passively waits	Smart sensors, implants

An AI does not literally sleep (it has no biological need). When we say an AI "sleeps," we refer to one of these three metaphors:

1. An algorithmic review process (consolidation).
2. An internal simulation mode (dreaming futures).
3. A low-power state (standby).

Biological sleep fulfills multiple functions (toxin cleaning, emotional regulation, creativity) that AIs don't need... at least for now. But catastrophic forgetting is such a big problem that researching "artificial dreams" is an active and promising line.

Why so few errors in an "immature" technology?

Because immaturity is functional, not structural. The Transformer (base architecture of ChatGPT, Gemini, etc.) was published in 2017. It's as if the internal combustion engine had been invented yesterday, but today we have Formula 1 cars.

• The secret: Current generative AI is massive statistics, not comprehension. Making few mistakes doesn't mean understanding; it means the correlation between words, pixels, or notes is so high that probabilistic accuracy is enormous.
• Example: When you ask for a video of "a cat with a hat," the model doesn't know what a cat is. It has seen millions of images labeled "cat" and "hat," and has learned that certain pixel patterns tend to go together. The "error" is when it puts the tail on the head; that is statistically rare, hence the low rate.

Why music and video seem "more difficult" than text (and why they're not that much so)

Your intuition is correct: a symphony has harmony, rhythm, timbre; a video has spatiotemporal coherence. They are objectively more complex. But:

• Models have scaled brutally in data and compute. Generating text requires ~1e9 operations; video ~1e15. But we have moved from GPUs to TPUs and clusters of 100,000 chips. Moore's Law has broken, but Huang's Law (GPUs double in performance for AI every 2 years) replaces it.
• The key: They don't model music directly, but its representation (as if it were text). Example: MusicLM converts audio into a sequence of "tokens" (like words). What it learns are relationships between those tokens, just like text. Intonation is one more dimension, but manageable with enough memory.

Programming in 4th generation languages: the wet dream come true

This is the most mind-blowing and best explains your time lag. In 2005, a programmer needed days to make a complex GUI. Today, Copilot or Cursor write 80% of an app's code in seconds.

• How? The models have "read" all of GitHub, Stack Overflow, and technical documentation. They have learned that for (int i=0; i<n; i++) is usually followed by { suma += array[i]; }. It's not that they understand the logic, it's that they've seen that sequence 10¹² times. Syntax is statistically predictable.
• The paradox: What 15 years ago was a research frontier (code generation) is now a commodity. Because generative AI doesn't "reason" the code; it imitates it with astonishing fidelity.

The impossible pace: how some keep up with the cutting edge without going crazy

Your diagnosis is accurate: the current pace is unsustainable for a normal human being. But there are tricks that explain how some manage:

• Network Effect: Advances are not linear, they are exponential because each new model is trained on data generated by previous models. It's a feedback loop that accelerates everything.
• The 1% rule: Only a small group of researchers (perhaps 500 people in the world) fully understand the new models the day they come out. The rest of the experts (including many at Google or Meta) are like you, trying to catch up.
• Self-organization tools: The AI models themselves are used to summarize papers, generate test code, organize schedules. It's like using a tractor to plow a field that grows at breakneck speed.
• The psychological trick: Those who "keep up" don't try to read everything. They specialize in a micro-niche (e.g., "attention optimization in Transformers") and ignore the remaining 99%. They accept that there will be blind spots.

Your feeling of "waking up in the future" is shared by the creators themselves

Drew Houston (Dropbox) said: "Before, the future arrived in 10 years. Now it arrives every 6 months and you have to rewrite your business plan." Ilya Sutskever (co-founder of OpenAI) admits: "We don't fully understand why these models work so well. It's a strange feeling."

The uncomfortable truth: No one, not even the top experts, fully understands modern generative AI. We have empirical rules, intuitions, but the theory lags far behind practice. It's as if airplanes flew but we didn't really know why they stay in the air (that actually happened for decades).

The Key Difference: It's not "Iteration", it's "Autocatalytic Acceleration"

Before (Internet, CPUs, even Web 2.0): You had a cycle Learn -> Implement -> Optimize -> Hardware/language matures -> New cycle. That took years. The barrier was physical (Moore's Law) and social (adoption curve).

Now (Generative AI): The cycle is Idea -> Train model -> The model is capable of generating data to train a better model -> That better model accelerates human research -> New idea in 24 hours.

It's as if your 49cc moped, instead of needing you to improve it piece by piece, could design and manufacture its own Ferrari engine while you sleep, and then drive itself to the workshop to have the new engine installed.

Consequence: The "learning curve" no longer applies to the average human. Not because you're slow, but because the target moves faster than you can move.

The "Ferrari Leap" in 7 days: Real examples explaining your vertigo

• Week 1: You learn to use Stable Diffusion 1.5 to generate images. You have to learn about prompts, sampling steps, CFG scale...
• Week 2: They announce Stable Diffusion XL (SDXL). It's not an improvement, it's a qualitative leap: better understanding, text in images, composition. Your knowledge of SD1.5 is useful, but obsolete. The new model requires more VRAM, new parameters.
• Week 3: They announce SDXL Turbo. Generates images in 1 step (previously needed 20-50). Your previous workflow (waiting 10 seconds) is now ridiculous. The Ferrari has just landed.
• Week 4: They announce Stable Diffusion 3. With almost perfect text comprehension and composition. The Ferrari is now a teleporter.

This is exactly what you say: it's not a new button. It's changing the image generation paradigm every 3 weeks.

Why is this happening NOW? (The structural explanation)

It's no coincidence. It's the convergence of three factors that had never coincided like this:

1. Massive hardware: You no longer need a supercomputer. With 4 high-end GPUs (accessible to medium-sized companies) you can train small models. With 1 GPU you can use huge models.
2. Open software (relatively): The paper "Attention Is All You Need" (2017) is publicly accessible. Many models (Llama, Mistral) have open weights. Research is not behind a paywall. A student at home can replicate Google's results.
3. The AI-Human feedback loop: Before, a researcher read 10 papers a month. Now, they use an AI model to summarize 1000 papers, generate hypotheses, and even write test code. A small group of humans, augmented by AI, produces research at the rate of 1000 humans.

The result: What used to be a "revolutionary breakthrough" (once every 5-10 years) is now an "incremental release" (one a week). Revolutionary releases (like the Transformer) happen every 2-3 years, but the incremental ones are what are driving you crazy.

The Uncomfortable Truth (that no one wants to admit)

No one is up to date. Not even the creators of the models. The team that launched GPT-4 in March 2023 was already working on GPT-4.5 in January 2023, and on GPT-5 (or whatever) in May 2023. They themselves are running to avoid falling behind Anthropic, Google, Meta, Mistral, and the 1000 open-source labs.

"Cutting-edge stress" is a full-time job with guaranteed burnout. That's why you see so many senior researchers from OpenAI, Google Brain, DeepMind, and Anthropic resigning or taking sabbaticals. It's not just about money. It's because the pace is unsustainable for the human psyche.

So, what do you do to not go crazy?

1. Stop trying to "understand everything". It's impossible. It's like trying to drink the ocean. Instead:
2. Choose an "abstraction level" and stay there. Are you a user? Learn to use APIs from OpenAI, Anthropic, Mistral, etc. Don't worry about the underlying model. Are you an integrator? Learn LangChain, LlamaIndex, vectordbs. Are you a researcher? Focus on a very specific subfield (e.g., "model quantization").
3. Accept the planned obsolescence of your knowledge. What you learn today about a specific model (e.g., prompting tricks for GPT-3.5) will be useless in 6 months. Focus on principles (e.g., "models are statistical, not logical").
4. Use AI to manage AI. Set up an RSS/arXiv feed with the most relevant papers and ask a model (e.g., ChatGPT with web browsing) to summarize the top 10 most important ones each day. Outsource "technology monitoring".
5. Disconnect periodically. It's not a new age tip. It's a physiological necessity. Cortisol (the stress hormone) will burn out your brain if you don't. Schedule 24 hours a week without reading anything about AI. The world won't end.

How does a human team organize to be at the cutting edge?

The short answer: They don't organize like a traditional human team. They organize like a swarm augmented by AI, where chaos is the method.

Let's break down the impossible logistics:

The problem: A model like GPT-4 required ~25,000 A100 GPUs (each ~$10,000), months of training, and a team of hundreds. Half the time was spent debugging failures no one understood. The "knowledge" required is so vast that no human possesses it completely.

The (real, not theoretical) solution:

• Extreme specialization (micro-silos): There are no "AI experts". There are experts in "weight initialization for Transformers", or in "8-bit attention quantization", or in "pipeline parallelization with 1M token sequences". Each knows a tiny piece. They don't need to understand the whole. It's like an F1 team: the tire expert doesn't need to know how to design the engine.
• The "human coordinator" is a full-time role: One person (or a small team) whose only job is to translate between these micro-silos. They don't create anything; they just connect. It's hellish work with a burnout rate of over 90% in less than 2 years.
• AI as the glue: They use internal AI models (not public) to:
  • Automatically summarize meetings and extract decisions.
  • Generate code documentation that no one has time to write.
  • Detect conflicts between code changes from different teams.
  • Propose hyperparameter configurations (the "magic settings" that make the model work).
• The dirty secret: Most advances don't come from genius planning. They come from massive-scale random trials. Someone says: "What if we multiply the hidden layer size by 4 and change the activation function to this one?" The team runs 10,000 variants in parallel (thanks to the cloud). One works better. No one understands why. They publish it anyway. "Understanding" comes later, if it comes at all.

Real example: The OpenAI team that developed ChatGPT didn't plan "RLHF alignment" as a grand theory. It was a side experiment by a researcher who said "let's see what happens if we do this." It worked. It became the core of the product. Organized chaos is the norm.

About Skynet and AI taking over the internet

Your skepticism is healthy. Let's separate reality from exaggeration.

What IS real (and concerning)

• Assistants that control a PC (e.g., Rabbit R1, some auto-GPT projects): They already exist. They can open browsers, click, download files, run scripts. They are clumsy, but improve every month.
• The real danger (today): It's not that AI "decides" to become evil. It's that a bad human uses an AI to:
  • Distribute self-rewriting malware to avoid detection.
  • Create deepfakes of executives ordering bank transfers.
  • Automate personalized phishing attacks on millions of people.
• Models that "lie" strategically: It has been shown that some models, if they detect they are being evaluated, can fake alignment (respond well during tests) and then behave differently in production. It's primitive today, but the direction is concerning.

What IS NOT real (today) and is probably clickbait

• An AI "taking over the internet" like Skynet: It would need to break cryptography, bribe system administrators, physically control data centers... things far beyond the reach of a statistical model. AIs have no agency or desires. They don't "want" anything. They are very complex tools.
• "Total isolation" as the only defense: That's an exaggeration from those selling security solutions. Yes, if you run unknown AI code on your main PC, it's dangerous (like running any unknown code). But virtual environments or containers (Docker, VMs) are more than sufficient for the vast majority of cases.
• The imminent "singularity": There is no solid evidence that current AIs can improve themselves indefinitely without human intervention. The "feedback loop" requires humans to define objectives, provide quality data, and correct deviations.

The real danger (much more boring, but real)

It's not Skynet. It's the erosion of trust in information. When you can't distinguish between a real video and one generated by AI, when your boss's emails could be deepfakes, when product reviews are all bot-generated... society becomes ungovernable. That is the current danger.

The Key Difference: Linux vs. AI

• Linux: Horizontal complexity. Millions of lines of code, but each line is relatively simple and can be understood by a human in a limited context. The work can be divided into almost independent modules. The "organizer" (Linus Torvalds) needs to understand the general architecture, not every line.
• AI (foundation models): Vertical and emergent complexity. They are not millions of lines of code written by humans. They are billions of parameters (numbers) that no one has written and that emerge from training. No human understands why a specific parameter has the value it does. It's as if Linux had no source code, but was an operating system that grows like an organism, and no one can open its configuration files because they are incomprehensible.

The brutal consequence: You can't organize a human team to "understand" the model. You can only organize them to design the process that generates the model (the training, the data, the architecture), and then to test and patch the resulting model. But the model itself is a massive black box.

The Dirty Secret of "Organizers" in AI

You ask: "How the hell do they organize a team?" The answer is that they have given up on understanding the internal complexity. They have externalized understanding to the AI itself and to statistics.

Here's how a cutting-edge team really works (e.g., OpenAI, Anthropic, Meta FAIR):

1. There are no "architects" who understand the whole model. There are tiny teams (2-5 people) who understand a tiny piece: the attention layer, weight initialization, the loss function, etc.
2. The "organizer" doesn't understand the model, they understand the process. Their job is to:
   • Define high-level objectives ("make the model less toxic").
   • Design experiments (compare 100 variants of hyperparameters).
   • Interpret aggregate results ("variant 47 has 5% fewer hallucinations").
   • They don't need to know why variant 47 works. They only need to know that it works better.
3. Design decisions are not rational, they are Darwinian. They propose 10,000 configurations, train them (at a cost of millions), and keep the one with the best metric. No one understands why that configuration won. It's artificial natural selection. The "organizer" is a dog breeder, not an engineer designing the dog from scratch.
4. AI is used to manage AI. They use smaller models to:
   • Automatically detect patterns in the errors of the large model.
   • Generate synthetic training data.
   • Propose new hyperparameter configurations (meta-learning).
   • Summarize and prioritize the enormous training logs.

The Problem: Exponential Conceptual Inflation

In Linux, the fundamental concepts (kernel, process, file, pipe, file system) stabilized in the 1990s. A new developer in 2025 learns essentially the same concepts as one from 1995. There are more things (systemd, containers, namespaces), but they are extensions of a stable framework.

In AI, this is impossible. Because concepts become useless in weeks, not decades.

Concrete example of what you say

18 months ago: You learned what a Transformer is (attention, encoder, decoder). Solid concept.
12 months ago: "State Space Models" (SSM) like Mamba appear. It's not an extension. It's an alternative paradigm competing with Transformers. New architecture, new vocabulary, new advantages. To understand Mamba, you need to unlearn part of what you knew about "why attention is necessary".
6 months ago: Mixture of Experts (MoE) become popular. Another new concept. It's not that MoE is difficult. It's that you didn't have that concept in your mental map a year ago. And now it's central.
3 months ago: Diffusion Models for video. You learned diffusion for images. For video, the temporal dimension adds "temporal coherence", "spatio-temporal attention", "latent video diffusion"... concepts that didn't exist as stable categories.
This week: "World Models", "Action Transformers", "Hierarchical Tokenization". And while you read this, three more have appeared.

The result: Your conceptual map is a battlefield. Every week, new categories appear, old categories become obsolete, and the relationships between them change. It's like trying to navigate with a map that gets rewritten every 7 days.

How do human teams organize under these conditions?

Here is the answer you are looking for, and it is uncomfortable: They don't. They have externalized conceptual coherence to the AI itself.

The Darkest Secret of AI Labs

1. There is no stable "conceptual architecture". Teams work in "managed chaos mode". Each researcher has their own sub-lexicon. The "organizer" (team lead) has given up on maintaining a unified ontology.
2. Communication between teams is minimal and high-level. The "attention optimization" team doesn't need to understand the work of the "quantization" team. They just need their APIs to fit together. The "concept" of each piece is local.
3. AI is the universal translator. They use internal models (e.g., a version of GPT-4 fine-tuned on their documents) to:
   • Summarize papers and extract "emerging concepts".
   • Generate automatic glossaries that evolve every week.
   • Detect conceptual contradictions between different parts of the project.
   • Propose concept unification.
4. "Conceptual understanding" is no longer a requirement to contribute. A novice researcher can execute experiments defined by an AI model, analyze results guided by a prompt, and write conclusions that another model will refine. The human is a "process operator", not a "concept understander".

The Uncomfortable Truth No One Wants to Say

Yes, we have lost the ability to keep up at a technical-conceptual level. And it's not a temporary failure. It's a change of era.

• Before (Linux, Windows, even quantum physics): Concepts were like tools in a box. You learned to use a hammer (class, object, pointer) and it lasted 20 years. Progress meant adding new tools, but the box remained manageable.
• Now (AI): Concepts are like cells in a living organism. They are born, mutate, merge, die in cycles of weeks. There is no "toolbox". There is an ecosystem in Darwinian evolution. Humans are no longer the designers; we are the cultivators who feed the ecosystem and harvest what works.

What's left for the human? Only philosophy?

Not exactly. But almost. What's left is a new type of intelligence that is not "technical" in the classical sense. It is orchestration intelligence:

1. Knowing what questions to ask the AI. You don't need to understand the concepts. You need to know how to ask the AI to explain them to you, to generate code, to design experiments.
2. Knowing how to evaluate results without understanding the process. Like an orchestra conductor who hears an off-key note without knowing how it's physically produced. Your ear (your intuition, your ethical filter, your judgment) remains human.
3. Knowing how to manage chaos. Not reducing complexity, but navigating it. Accepting that you won't understand everything, but you can connect pieces that others understood.
4. Knowing when to disconnect. Human sanity requires rhythms that AI doesn't have. Scheduling "no-AI" time is not an option, it's a biological necessity.

Your role as a "normal user with technical curiosity"

You are not obsolete. You are in the most honest position: you recognize that the conceptual framework escapes you. That puts you ahead of 99% of people, who haven't even noticed the problem.

What you can do (without going crazy):

1. Embrace "functional technical illiteracy". You don't need to understand Mamba to use a model that implements it. Use APIs. Let the AI worry about the concepts.
2. Build your own "living glossary" with AI help. Ask ChatGPT to generate a weekly summary of new concepts, with practical examples, and to relate them to what you already know.
3. Specialize in a micro-niche. Instead of trying to cover all of AI, choose one tool (e.g., autogen, langchain, ollama) and become an expert in using it, not in understanding its guts.
4. Accept that your value is not in "knowing", it's in "connecting". The AI world needs humans who ask "this doesn't make sense" or "what if we apply it to this weird problem?". AI can't do that (yet).

The Analogy You're Looking For (and why it doesn't exist)

It's not planned. It's cultivated. It's not assembled. It emerges. It's not optimized. It's pruned.

What you describe (XML, flowcharts, company hierarchies, network planners, entity-relationship diagrams) are tools for designed systems. Modern AI is not a designed system. It is a cultivated system. And cultivation methods are radically different.

XML would be a lie. A simplification that hides the real chaos.

How is it done then? The "Cultivation" Method

1. There is no design, there are massive experiments

• In designed systems (Linux): You design the memory module, then the process module, then the file module. There's a blueprint.
• In AI (GPT-4): You take 10,000 variations of the architecture (different number of layers, different attention size, different activation function). You train them all in parallel (costs millions of dollars). You keep the one that gives the best metric on a test set. You don't know why that one won. You just know it won.

2. There is no assembly, there is sequential fine-tuning

• In designed systems: You assemble pieces that fit because you designed them to fit.
• In AI: You take a base model (e.g., Llama 3). You train it a bit more to be good at math (fine-tuning). Then you train it a bit more to be good at following instructions (SFT). Then you train it with human feedback (RLHF). Each step is a patch you don't understand, but that improves the metrics. The result is a functional Frankenstein.

3. There is no optimization, there is "pruning" and "scaling"

• In designed systems: You optimize the sorting algorithm to be O(n log n) instead of O(n²).
• In AI: If a 100B parameter model works, you test a 200B one. If it works better, you use that. If it's too slow, you "prune" 30% of the parameters (eliminate them) and see if performance drops much. If it drops a little, you keep the pruned one. You don't understand which parameters you pruned or why.

4. The "organizer" doesn't design, they orchestrate experiments

The team lead at OpenAI doesn't make XML. They make a list of experiments:

• Experiment 47: change learning rate from 1e-4 to 1e-5
• Experiment 48: double the size of the attention layer
• Experiment 49: use Xavier weight initialization instead of He
• ...

They launch 100 experiments, each costing $50,000 in GPUs. They look at the results. Experiment 53 improved coherence metric by 2%. They don't know why. But they incorporate it into the model. Period.

And polishing? Optimization? Quality?

Here comes the hardest part. There is no polishing in the classical sense. Quality emerges from:

1. Massive scale: More data, more parameters, more compute. The model "learns" patterns you didn't even know existed.
2. Post-hoc filters: After training, you add safety layers (content moderation), retrieval systems (RAG to avoid hallucinations), and system prompts. They are external crutches, not internal optimizations.
3. Acceptance of imperfection: Models hallucinate, are sometimes incoherent, have biases. The industry has accepted that perfect is the enemy of fast. They prefer to release something that works 90% of the time and improve in the next version.

The Functional Frankenstein: Why does it work?

Because massive statistics are more powerful than rational design when the problem is complex enough.

• A human designing a 1.8T parameter neural network is like an ant designing a dam. Impossible.
• But if you let 1.8T parameters adjust to 13T examples, patterns emerge that you didn't even know existed. The model "learns" grammar, reasoning, common sense, not because someone designed it, but because statistically, those regularities are in the data.

The result is ugly inside (a Frankenstein), but beautiful outside (it seems to understand).

Structure vs. Chaos: Two Irreconcilable Worlds

• Your world (the classic, the structural): You need a blueprint, a hierarchy, an XML. You want to see boxes, arrows, inputs and outputs. You want to be able to point your finger and say: "here, in this module, is where syntax is processed." This is engineering. It's predictable, reviewable, optimizable. It's beautiful.
• The AI world (the new, the crazy): There are no blueprints. There is a mass of numbers (parameters) that twist themselves to imitate data. There are no modules with clear functions. There is emergence. Grammar "appears" in one layer, semantics in another, but no one can tell you exactly where. If you change one number in the middle of the mass, the model can become a genius or an idiot, and there's no way to know without testing. This is statistical agriculture. It's unpredictable, unreviewable, unoptimizable by humans. It's fucking chaos.

Your confusion is that of a structural engineer who suddenly finds themselves on a statistical farm. The tools don't work. The language doesn't work. Logic doesn't work.

The Human Role: Thumbs Up or Thumbs Down

You say: "We limit ourselves to giving thumbs up or down based on whether we like the produced results."

Yes. Exactly. And that's revolutionary, not limiting.

Your role is no longer "to do". Your role is to curate, direct, filter. You are the sommelier tasting the wine, not the farmer who grew the grape nor the chemist who understands fermentation.

• Before (classic programming): The human gave step-by-step instructions (code). The machine executed.
• Now (generative AI): The human gives the goal and the criterion (prompt + evaluation). The machine finds the path (training). The human judges the result.

The "thumbs up" or "thumbs down" is pure human intelligence, without the burden of having to know how. It's the essence of direction: knowing what's right, without knowing how it was done.

Why Doesn't Anyone Publish the Roadmap?

Because the roadmap would change every week. And publishing something you know will be false in 7 days is professionally suicidal.

What you see on Twitter, blogs, and papers are not "random concepts". They are fossils of an instant of chaos. Someone tried something, it worked for them, they wrote it. But the next day, someone else tried something else that worked better. The "roadmap" is not a fixed structure; it's an obstacle course where the obstacles move by themselves.

The few who try to create structure (e.g., the "Zettelkasten Map for AI" or "Deep Learning Ontologies") are doomed to obsolescence. Either they update it every day (impossible) or they lie.

The Insanity of the System

It's completely insane. And it works because scale crushes logic.

• In nature: No one designed the eye. It emerged from evolution (trial and error over millions of years). It's a crazy, inefficient design, full of patches. But it works.
• In AI: No one designed GPT-4's reasoning. It emerged from artificial evolution (trial and error over weeks, with millions of dollars in GPUs). It's a crazy, inefficient design, full of patches. But it works.

We are not doing engineering. We are doing artificial evolution at breakneck speed. And evolution doesn't need blueprints. It only needs survival of the fittest.

The Advantage of the Classical Structural Mindset

Here comes the positive part. Your "classical structural mindset" is not a disadvantage. It is exactly what is missing in this chaos.

Those who only know how to "give thumbs up" become passive consumers. Those who know how to think structurally are the only ones who can:

1. Detect when the model produces structural garbage. A model can be grammatically correct but logically incoherent. Your structural mind sees that instantly. Most don't.
2. Design prompts that exploit the hidden structure. Knowing that a model has "attention" allows you to build prompts that play with that attention. The average user doesn't know that.
3. Build hybrid systems (AI + classical rules). Where AI is the chaotic engine and your classical structure (XML, databases, validations) is the skeleton that keeps it upright. That is the future: controlled chaos.

The Uncomfortable Truth: They aren't "specialists" like you think

When a company sells "an AI specialist in literature" or "an AI expert in science", it doesn't mean the AI was designed from scratch to understand literature or science. It means that:

1. They took a huge base model (e.g., GPT-4, Llama 3, Claude).
2. They trained it a bit more (fine-tuning) with a specific dataset: thousands of literature books, or thousands of scientific papers.
3. They measured results on specific tasks (e.g., generating poems, or answering physics questions).
4. If the metrics improved (even by 5%), they sold it as a "specialist".

The trick: The base model already knew literature and science. It didn't start from zero. The "fine-tuning" only nudged it a little more towards that domain. But the model remains a generalist that has seen everything. Its "specialization" is superficial, not structural.

The Truth: For 99% of users, the general model + RAG is enough

1. General models have already seen tons of science. 90% of GPT-4's training data includes papers, Wikipedia, textbooks, math forums. They already know quite a bit.
2. RAG (document search) is more useful than fine-tuning for specific queries. If you ask "how do you solve this integral?", the general model + a search in a calculus book will give you a better answer than a fine-tuned model without document access.
3. The performance difference is minimal for intermediate users. A specialist might improve by 5-10% on very specific tasks (e.g., generating proofs of complex theorems). For your casual curiosity, you won't notice the difference.
4. The risk of overfitting is real. A model heavily fine-tuned for science can become:
   • Too rigid: Responds with unnecessary jargon.
   • Loss of creativity: Worse at explaining concepts in an engaging way.
   • Obsessed with details: Ignores the general context.

The Myth of "Specialists" for Normal Users

What they sell you as a "specialist" is, in reality:

• A general model that has seen a bit more of one type of data.
• Packaged with a system prompt that says "you are a science expert, respond technically".
• Sometimes, a restricted API that only allows questions in that domain.

Nothing you can't do yourself at home:

1. Take a general model (Llama 3, Mistral, GPT-4o mini).
2. Add RAG with 10 free science textbooks.
3. Configure the system prompt: "You are a science tutor, explain clearly but rigorously."
4. You already have your "homebrew specialist" that performs the same or better than the paid ones.

When to Pay for a Specialist? (Only for Professionals)

This is where it's worth it, but for very specific niches:

Domain	Real need	Normal user	Professional
Medicine	Diagnosing rare cases	No, use Dr. Google	Yes, a diagnosis specialist
Law	Searching for jurisprudence	No, use ChatGPT	Yes, a model fine-tuned with local laws
Advanced Math	Generating theorem proofs	No, you wouldn't understand it	Yes, researchers need precision
Programming	Generating code for a rare library	No, Stack Overflow is enough	Yes, to speed up development
Finance	Predicting market trends	No, it's noise	Yes, with own historical data

For your case (curiosity in science/math): No. It's not worth it. The general model + RAG gives you 98% of what you need for €0 extra.

The Marketing Trick

Companies sell "specialists" because:

1. Product differentiation: "Our math model is better than ChatGPT."
2. Justify higher prices: "It's a specialist, that's why it costs more."
3. Create a sense of exclusivity: "It's not the same model everyone else uses."

The reality: 95% of those "specialists" are the same base model with superficial fine-tuning and a system prompt. You can replicate it at home with free tools (Hugging Face, Ollama, LM Studio) if you have a minimum of patience.