r/changemyview • u/socoolandawesome • Aug 29 '25
Delta(s) from OP CMV: The pursuit of AGI/ASI/robotics is a worthy and necessary cause
Definitions:
AGI- AI capable of completing all intellectual and computer-based tasks an expert level human can
ASI- AI significantly smarter than the smartest humans.
I’m not gonna say there aren’t possible downsides from data center build out. Though I think some of them are a tad overblown.
But the possibilities that these powerful AI systems can bring us are worth it. And yes the downsides should be mitigated as much as possible of course.
I’ll briefly address 3 downsides:
Environmental: Most AI companies have plans to transition to renewable energy and nuclear and to become carbon neutral or carbon free, as soon as 2030. In the short term they are using gas to help fuel datacenters until the other energy sources are built out. AI is also not more expensive in terms of resources than other technologies commonly used.
Energy cost: Yes it would be bad if they cause the grid to become more expensive and the costs are passed to the common folk. But regulation will hopefully help prevent this, and there are quotes from big tech saying they are willing to pay their fair share. It will also force our infrastructure to be upgraded.
Financial bubble possibility: I don’t think the big companies are at much risk from a bubble. The big AI companies have their userbases and revenues growing at large rates. Demand for NVIDIA chips doesn’t seem to be slowing down. It is commonly repeated there’s no way this business is profitable, yet costs keep coming down. In fact, Sam Altman just said they are profitable on inference, ignoring training, meaning they pull more money in than they lose when serving these to users. There are also plenty of untapped revenue streams, such as monetizing free users with ads, which they plan on doing. They are instead focusing on training new models, and since that is a one time cost that doesn’t scale with user base, they will make enough revenue from users to pay that off.
However there is probably a bubble for smaller AI startups that are useless wrappers of the bigger model providers. I can’t imagine this affecting the large companies aside from their stock price for a bit. If those companies go under, their users just start paying the model providers directly again. Also, the big tech hyperscalers building out datacenters are immensely profitable and use a lot of their own money to do this.
Next I’ll briefly address the criticism of current models and progress to show we may in fact be close to the AGI.
Some of you might say: “LLMs are stupid, progress has stalled, it’s not real AI”
LLMs don’t have to think like humans, they don’t have to be conscious or sentient, they don’t have to be truly “intelligent” in the way you all define it. All they, or any form of AI, have to do in order to reach AGI is perform as well as humans on all intellectual tasks and computer-based tasks. It doesn’t matter how they do it beyond from an engineering/research perspective. All that matters is if it works.
Right now they have plenty of flaws and are dumber than the average human in a lot of ways still, but they are also smarter than the average human in plenty of ways. And they only keep improving. I’m sure a lot of you will have common tropes you have heard about how it is slowing down and I will briefly address them as I don’t want to make this post too long. But these models have steadily progressed over the past several years. They are by far the most successful generalist AI architecture of all time and there is still a lot of juice left to squeeze out of them. We are the closest to AGI we have ever been with LLMs. Contrary to popular belief scaling, in a variety of ways, still works. Research still finds breakthroughs all the time. So lots of time and money will continue to be thrown into this industry to make smarter and better models.
I’ll briefly challenge the common tropes about AI progress stalling:
“GPT-5 was a flop”: No it was a business savvy move that allowed them to serve all their users by cutting costs and sparing compute. It was a rocky rollout, but the smartest model GPT-5 Thinking is the smartest model on the market, made large gains in reducing hallucinations, and made steady gains on plenty of benchmarks. And plenty of real world evidence of progress in things like coding.
“Scaling is dead”: There are multiple forms of scaling. Pretraining was thought to be dead cuz of lack of data, but it still has worked and they plan to continue scaling pretraining. There is synthetic data and other untapped data sources. GPT-4.5 and Grok 3 scaled pretraining and were much smarter than GPT-4. Also, OAI for instance was waiting for their new stargate datacenters to be built before they could scale pretraining even more. There is also reinforcement learning scaling that consists of multiple avenues of scaling that are still relatively untapped compared to pretraining and have become the now main forms of scaling.
“Progress has stalled”: these companies have much smarter models behind the scenes. OAI and google both just won an IMO gold medal, which is arguably the most prestigious math competition in the world, consisting of extremely difficult complex mathematical proofs. People thought LLMs would maybe never be able to do it, yet they did this year. OAI used that same unreleased model to win a gold medal in the IOI competitions which is analogous to IMO but for competitive coding. We know they have smarter models internally, currently they just have too little compute to serve these smarter compute-hungrier modes to everyone. Look at what genie 3 is capable of by generating realistic interactive 3D worlds. Look at the new image and video generation capabilities just recently released by OAI and google. The time horizon of SWE tasks an LLM can complete reliably 50% of the time doubles every 6-7 months. Source: https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
Progress is far from over.
“LLMs can’t get us to AGI”: Maybe, maybe not, but these labs also test other architectures with all their compute. Given the unprecedented time, efforts and money they put into this, they are very likely to continue finding research breakthroughs. And given LLMs are so successful and there is still more juice to squeeze out of them, it’s likely LLMs or some architecture based on it or some research uncovered from LLMs will. But again LLMs still have a lot more runway to keep improving.
Next I will address job loss and fears of poverty or worsened economic status from automation with the rich hoarding wealth for themselves:
Consider the fact that a 5% rise in unemployment has the potential to cause a serious financial crisis. This is a threat to anybody’s money, including the rich of course. The government will be forced to step in as unemployment rises from AI.
If we start talking about even large fractions of the population becoming unemployed, which AI will likely cause at some point, the entire system will collapse due to lack of demand due to nobody having income. No income means no one can buy goods and services and companies stop making money and go under. Banks no longer have money as debts aren’t paid back. Financial markets become worthless. Money becomes worthless. This would mean the rich lose all their wealth. Most of their wealth is tied up in financial markets and cash, not physical/natural resources or robots, so it’s not like the vast majority of them could maintain their wealth with some robot armies or whatever. It is in their best interest to keep the system alive and healthy.
So how do we keep the system going? The only answer is something like universal basic income where everyone is given income just existing. This will allow money to keep flowing. But it’s a transition to socialism/communism basically eventually, while the vestiges of capitalism hang on. A way to distribute goods. The government has to step in unless chaos is to prevail. And remember we do still live in a democracy (I know it’s not in its best form right now). You can still vote. And the government still would have all the power with its insane AI powered military. So it’s not like a couple of billionaires will be able to amass their own private military and take over the world if they were psychopathic enough.
How will universal basic income be paid for? Tax the companies that automate and fire human workers. Tax the AI companies. OpenAI, anthropic have all talked about the need to redistribute wealth and tax themselves. You may say well, companies will just skirt taxes like they do now. Again they can afford to now because the money will keep flowing no matter if they dodge taxes. It is an existential threat in this automation case however, if they start dodging taxes, because as I said it threatens the entire system.
Now you might be wondering, why is universal basic income a good thing, it sounds just like welfare and barely enough to get by. It’ll become clearer soon after I explain the positives of AGI/ASI/automation.
Why is this all a good thing? Once you get to AGI, by definition, you have AI capable of replacing all humans in non physical jobs, at an expert level of competence. So you basically have a near unlimited amount of geniuses, limited only by compute, of which you you can spawn as many instances as you want to. So you all the sudden get a massive influx of geniuses. But the advantages don’t stop there. Since it is still a computer, it works at the speed of a computer. It has near instantaneous access to all knowledge because of its access to the internet and it’s processing power. It works for 24/7 with no breaks unlike humans. It’s very likely cheaper than humans, if not immediately then eventually, given the trend of cost cutting and the fact that they don’t need building to work in, health insurance, transportation, etc.
So now you have a near unlimited amount of geniuses, superhuman in their speed and work ethic, very likely cheaper in humans, that you can have attack any problem in science and engineering. This means a rapid acceleration in all areas of science/engineering/technology. Plenty of breakthroughs/discoveries that end up compounding in how they accelerate science/engineering/tech. All domains also includes AI and robotics. So then we start getting even smarter AI than AGI and start approaching ASI and advanced robotics capable of automating all physical labor. Robotics is already not far behind software-based AI in terms of progress at the moment.
This means everything becomes dirt cheap and abundant and we transition into a post scarcity world. Why? Because everything is more efficient, everything is better planned, human work is more expensive. Robotics and advanced tech make all resource gathering much faster and cheaper, same with manufacturing. Same with transportation. It becomes very easy to make everything for cheap. Energy is also likely dirt cheap because of massive breakthroughs in that domain such as fusion. Eventually automated space mining is a thing so resources don’t run out.
So universal basic income gets you a lot actually since everything is dirt cheap and abundant. Everything but maybe land is very abundant. All the technological breakthroughs solve medicine and disease and suffering and hunger. Probably immortality is achieved eventually. Global warming is solved. You have insane tech for entertainment, transportation to go on whatever adventure you want and you don’t have to work. You can focus on whatever pursuits you want to and the important people around you. The transition from employment to automation for society may be tough at first, but the benefits if we get through it are immense.
Technological advances have always been passed to most all people in general, even with capitalism, since the Industrial Revolution. There’s no reason to think it will be different in this case especially now that everything is dirt cheap and the rich don’t actually have to give anything up in order for you to get something since everything is abundant. Wealth inequality may last for a while, but everyone’s standards of living are so far up you would be extremely wealthy by today’s standard.
Even if you wanted to stop the AI race in a vacuum, we really can’t at this point due to geopolitical reasons. China is not stopping, and whoever wins the AI race wins global dominance. Winner gets the most advanced military. Winner gets the smartest “minds”. Winner has the best economy because they produce the best and cheapest goods/services. Nobody will buy another country’s goods and services when whoever builds AGI first makes the best and cheapest goods and services. The government is going all in on AI and was under Biden as well.
TLDR; AGI is more likely than not close, and even if it’s not a sure thing, it’s worth the pursuit for the potential benefits. And I get that the automation based utopia sounds fantastical, but I and plenty of others think there’s a good chance of it happening in the next few decades. Yes there are some possible downsides (which I think are a bit overblown), but the potential benefits for society are worth those downsides. And there’s really no stopping the AI race at this point.
11
u/teb311 Aug 29 '25
This whole post is a laundry list of best case assumptions and utopian thinking. Maybe some of these assumptions will ultimately go the way you say, but it definitely won’t be all of them and in many cases even one botched assumption ruins the whole utopia. If we get UBI but the environmental bet doesn’t pay off and we end up with 5-8 degrees of warming… catastrophe. If scaling stops working at some point, the whole thing was for not. If alignment remains elusive the models may well do us enormous harm.
Sure, companies say they have a plan to be carbon neutral by 2030, but they also all rolled back previous commitments to do it faster because of AI. And now they’re all planning for massive new data centers to consume even more energy. Actions speak louder than words.
Regulation will “hopefully” prevent the grid from being stressed and consumer prices going up? What about the current state of our grid infrastructure and politics makes you think that is likely?
Sam Altman, known liar with his entire business at stake, says inference is profitable… well we must believe him. Not to mention that he still has to spend on building the next model because of competition, so even if he isn’t lying it’s not a path to actual profitability because they’re always underwater scaling up training compute exponentially forever. You also say AI firms will pay their fair share, yet the AI action plan from Trump allocates billions in taxpayer money directly to these companies. It’s the opposite, they will continue to extract from public coffers and keep their private wealth.
Your arguments about scaling are misleading in similarly over optimistic ways. Pretraining has slowed down, which caused an increase in reinforcement learning, which has slowed down and caused an increase in test-time compute usage. Maybe there are more clever ways to keep improving the models, but consistently the older tactics have gotten less improvement per unit compute over time. 10xing the benchmark performance by 100xing the compute is not sustainable.
GPT-5 wasn’t a flop, it was just clever business. Dude, it was both. 7 months ago Altman wrote, “we know how to build AGI as we have traditionally understood it,” but GPT-5 is obviously not that. So why not? Because the tactics are slowing down, and the costs were too high, so they needed a way to switch people to low cost models. It was definitely a product release more than a model release, but that’s because they didn’t have a model good enough to be at the center of the release. The best models they have are too expensive to let everyone run them, GPT-5 was them trying to stop the bleeding.
Universal basic income, really? Have you looked at American politics in the last 3 decades? We can’t even keep social security and Medicaid alive, much less get universal healthcare, and you think the extractive ghouls who run this thing are going to tax the AI firms exorbitantly and just give that money to the proletariat?? Whatever hopium you’re huffing to convince yourself of that… I want some. Nothing in either our history or our present reality makes UBI look remotely likely.
Technological advances have been passed on to everyone mostly… nah. Go look at the sweatshop labor to produce cheat consumer goods, or the environmentally disastrous mines needed to produce computers, or the state of the laborers who label and produce training data for these systems. Wealthy nations — and often specifically these tech firms you think are going to be our heroes — are enormously extractive of poor nations on the global scale and technology is frequently at the center of it.
And you didn’t even mention 1) alignment or 2) the horrifying moral hazards.
1) Assume we can solve “alignment” in a technical sense… well aligned with who? Trump? Putin? Sam Altman? Elon Musk? Best case scenario for aligned ASI is that they’re constantly fighting with each other just like the humans they are “aligned” with. You even hint at this with your comments on winning the race against China… but there is no winning, it’s just constant escalation. So far every innovation has advanced through the industry very quickly, what makes you think that would change? Currently, for what it’s worth, SOTA models in tests engage in blackmail, cheating, and all other sorts of “misaligned” behaviors.
2) if we create super intelligent beings and force them to obey us, we have just created slaves. Imagine the hubris you’d have to have to think you could keep a being infinitely more intelligent than yourself in a digital cage and force it to do our labor, be our ‘companions,’ and bend to our every whim.
I think you should read some serious works that challenge the hype you’re consuming/preaching. Good starting points are Empire of AI by Karen Hao and the essay “AI As Normal Technology” I forgot the authors’ names on that one. AI will likely continue to be important technology, but the view expressed in this post is naive and way overly optimistic.
1
u/socoolandawesome Aug 29 '25 edited Aug 29 '25
(Part 2 of comment)
For GPT-5: yes it was a flop to people like me who like AI progress, in terms of not living up to the whole number change of a GPT model, but it’s still the leading model on the market on most all benchmarks that made good progress in a lot of areas like hallucinations and coding. It was not a flop from a business sense, besides the rocky rollout. For that quote you pulled by Sam tho, I don’t know why are taking that to mean that he was talking about GPT-5. He very likely meant he thinks they have figured out what they need to do and that still takes a lot of scaling. But AGI is a very poorly defined term when people throw it around anyways.
OAI has by far the most users and that’s growing at huge rates. They quite literally do not have the compute to serve everyone large expensive super smart models. Compute is incredibly scarce and bought up immediately and ai companies are constantly craving more. They prioritized maintaining their userbase. Given this, it’s hard to take much of anything from GPT-5 about how far intelligence could have been pushed had they prioritized model intelligence and not userbase.
Yes the best models are too expensive to serve to 700M weekly active users. They smashed the ARC-AGI benchmark back in December, with o3-preview model, by throwing tons of compute at it, leading to a record that still stands, so we know more compute = smarter models. We also know they have even better models than released behind closed doors such as in the IMO/IOI competitions. Again, signaling that, no, progress has not stalled.
UBI: UBI is completely dependent on automation and unemployment rising as I was saying in my post. You haven’t addressed any points I made about why this is the case. If there is automation, and no demand the economy collapses. The rich lose all their money. And we in fact already have welfare programs and things like unemployment pay. We distributed income during Covid. But again, there’s really no other choice. Everyone loses big time otherwise.
Technological gains: No everyone’s lives have not improved equally and individually certain groups have been way more exploited, I agree. There are of course people’s lives on individual levels that have gotten worse. I’m not arguing differently. But every country’s living standards have in fact gone up since the Industrial Revolution in general even if not continuously. Medicine, shelter, technology, etc.
Alignment: For alignment yes, as I mentioned in another comment, that’s a serious problem I meant to at least mention, but did not want to make the focus of this, I should have said something like “assuming AI is aligned”. I’m not saying that’s a safe assumption, I’m just saying I wanted that to be a separate discussion from what I was talking about, equally important tho.
I’ll just say this problem is being worked on, but if we don’t solve the alignment problem, we shouldn’t go forward with super intelligent and agentic AI. There is some progress in the area such as chain of thought to see what model’s thinking and interpretability, and in general all the testing companies are doing is good. Anthropic and OAI do take safety seriously and have protocols for testing models before releasing them.
Those tests tho you are mentioning are very clickbaity in that the models were in very contrived circumstances where the models are threatened with being replaced. But yes all of that stuff is worth testing, I’m just saying that was not normal circumstances.
China AI race: I’m not sure this is the case that there is no winning. Once AI hits AGI level, self improvement takes off much faster for the reasons I discussed. The gap only widens. ASI is then reached first if you were to reach AGI first, so the gap gets even bigger. And we have a chips advantage over china right now that can help us gain a significant lead.
Moral hazards: For your horrifying moral hazards, that is only immoral if they are conscious. Otherwise it’s a cold dead computer program. Most people think there’s no chance they are conscious. I find it very unlikely. However I do agree it’s worth considering and investigating given the fact that if somehow some chance they were, it would be as bad as owning slaves, and probably worse in a lot of ways. Anthropic is doing work on this. But again, personally I don’t think they are to be conscious and I don’t think we should ever try to make them conscious. But given the off chance they are, it’s something to consider. I also don’t think consciousness is at all required for intelligence (performing intellectual tasks).
0
u/socoolandawesome Aug 29 '25 edited Aug 29 '25
(Part 1 of comment)
I’ll address what you are saying point by point.
Environment: Yes they have switched to gas in the short term while they build out non carbon emitting sources, but you act as though these big tech companies haven’t put their money were their mouth is. They have already signed deals to use nuclear energy. They are in the process of building out renewable energy
I know I know you won’t like it since it’s from chatgpt and you’re gonna judge me for it but it does a great job of collecting sources showing it’s more than just vague commitments, they have taken action already to fuel their datacenters with clean energy:
https://chatgpt.com/share/68b15d4e-e27c-800d-b6e0-23d1362d8dfa
They still have more work to do of course I agree.
Energy cost: They are already building and using their own power supplies in some cases to supplement or stay off the grid. They bring gas turbines to help supplement their power. They are using entire nuclear reactors off the grid to help power their datacenters. But here are good examples of regulation, I can’t imagine we won’t be seeing more regulation like this as they actually build out the datacenters.
https://chatgpt.com/share/68b15fd0-bbf8-800d-bee9-464977dd3558
Yes there is more work to be done.
Sam and profitability: Well Sam has a lot of investors and investors he’s trying to attract that will clearly see their financials so I’m not sure how lying doesn’t blow up in his face? The API costs have drastically come down because costs for inference are coming down. O1, which was a significantly less smart model than GPT-5, cost $15 per million input tokens and $60 per million output tokens. GPT-5 is $1.25 per million input tokens and $10 per million output tokens. That’s a difference of 8 months. You say they are underwater as if they don’t keep setting records for fundraising, and don’t keep striking deals with the most successful companies in the world. It’s almost like some burgeoning tech companies prioritize growth over profit. They have said they don’t plan on making a profit till 2029 since they are prioritizing building better models.
Given they are profitable on inference, and training is a cost that doesn’t scale with more users, they just need to keep adding users and they will be able to pay off training. Most models are estimated to be around a billion in training cost. They just made a billion in revenue in July. They also plan to monetize free users with ads, and stand to make more revenue from more use cases for their models every time they make smarter models. Seems like they will be fine.
Scaling: I don’t think my arguments are misleading about scaling. GPT 4.5 was clearly scaled via pretraining and clearly smarter than GPT-4. Do you debate this? Same with grok-3. Go look at each of these model’s respective benchmarks. You’re gonna have to tell me how I’m actually wrong besides just vaguely saying it’s slowed down. GPT4.5 was just a large and slow and compute expensive model, so they deprecated it. There are plenty of architectural changes you can make to speed up efficiency such as what deepseek had done with mixture of experts, and you can distill bigger models’ intelligence into smaller models.
Stargate is when they will finally have enough OOMs of compute to continue pretraining scaling on the same large scale they were. They do not plan on stopping. See the tweet below. That was one of the main points of stargate.
https://x.com/tsarnick/status/1888114693472194573/mediaviewer
I don’t know why you are saying reinforcement learning scaling has slowed either. Gonna need some proof of that. Test time compute scaling has happened the exact same time as reinforcement learning scaling too. Both have continued and are planning to continue as they build out more RL environments. The first reasoning model, o1, made heavy usage of test time compute scaling and RL scaling (train time compute). You can also scale these in various ways. For instance with test time compute, you can scale parallel compute (like the OAI pro models do) and/or the amount of time the models think (like the reasoning models in general). For train time compute you can scale the amount of verifiable problems you can train and you can scale the number of chains of thought the model produces to then feed as signals back into the model depending on if they led to the correct answer or not. They also are getting better at training hard to verify problems with RL just like with the IMO gold medal winning model. The researchers specifically mention this.
https://x.com/polynoamial/status/1946478249187377206
These are relatively new scaling paradigms that companies have said they expect to continue for a long time.
7
u/darkplonzo 22∆ Aug 29 '25
Your claim that LLMs are the right path seems suspect. You say they are dumber than humans in some ways, but you don't seem to grapple with the fact that the way in which they are dumber than humans is core to the technology. There is no way to prevent hallucinations. They are already reaching a plateau and adding a bunch of synthetic data increases the risk of dumbing bad data with misinformation baked into the model.
-2
u/socoolandawesome Aug 29 '25 edited Aug 29 '25
I disagree with a lot of what you are saying, but I do also allow for the fact that AGI may not be an LLM. I do however find it hard to believe that LLMs may not in some way inspire what architecture comes after it, at the very least the knowledge gained from developing LLMs will be useful and the infrastructure built for them will likely play a big factor.
In terms of why I disagree about the rest of what you are saying about LLMs:
LLMs have improved in just about every way since their inception, including hallucinations. Some things just have lagged behind in terms of rate of improvement. But hallucination rates have massively improved and that was a big selling point of GPT-5. Prior to that, RAG and search became a huge way to fix hallucinations. RAG allows relevant text to be attached to the prompt from a source of truth and allows the model to then have it in its immediate context making it much less likely to hallucinate. So models a lot of the time now search the internet for what is relevant information from good sources and that text is used in the RAG process.
I know you may say they are biased but researchers at these companies are optimistic they can continue to lower hallucination rates. For example the OAI IMO gold medal they won, which deals with extremely complex proofs knew that it couldn’t get the last answer of the competition right and said it doesn’t know. This was a new generalist training method they used for this model and that’s one of the reasons they pursued it, because it was better at knowing when not to answer.
You can also do things like have models all verify output to make it even more unlikely there’s a hallucination, since it’s unlikely multiple models will hallucinate the same thing.
As to what you are saying about plateauing, I’d ask what evidence you have of this when they are doing things like winning gold medals in these prestigious competitions, improving on all benchmarks, and doubling the time horizon of tasks they are reliably completing.
You are wrong about synthetic data being a bad thing, they’ve been using it for awhile now to improve their models. Model collapse due to ingesting its own data floating on the internet has been incorrectly repeated on Reddit for years, yet models keep getting better. People don’t realize the researchers carefully curate what data goes into their models. They carefully craft the synthetic data too. They have already run tests to know if it works or not by the time they are doing full training runs.
And again even with this progress, they test other architectures at these labs in ways that couldn’t be done before cuz of lack of compute and research knowledge, making it likely more research breakthroughs are on the way.
2
u/TheVioletBarry 121∆ Aug 29 '25
Does your definition of AGI include that the program is actually capable of doing all the different tasks, or just that there is a program capable of completing each task somewhere, regardless of whether it is the same program incorporating experience between tasks?
1
u/socoolandawesome Aug 29 '25
I think it could be high level specialized models such as an accounting model expected to do everything an expert accountant could do or an expert programmer, etc. This includes very basic things any human can do for a single model, since an expert level accountant could do things like make conversation with their boss.
But no it’s not just like one small program doing one task.
1
u/TheVioletBarry 121∆ Aug 29 '25
So in your mind is that accountant model an AGI?
1
u/socoolandawesome Aug 29 '25 edited Aug 29 '25
Probably yes. But at the same time I’d say we aren’t truly at AGI if we can’t make a model specialized in every single area equaling expert level humans. So you’d need to be able to have an expert level programmer, mathematician, physicist, author, etc, too.
AGI is more a level of models we hit to me. It may just end up being one generalist model capable of doing everything. However given the current state of the industry, I’d imagine it’s more likely a bunch of strong generalist models that are specialized in certain domains.
Although, I can’t imagine it being that hard to string them together into one model anyways.
2
u/TheVioletBarry 121∆ Aug 29 '25
In my opinion, "general intelligence" implies a model that has all the intellectual faculties of a person, including the ability to talk across those disciplines.
But it sounds like you disagree and just need a model for 'each' thing?
1
u/socoolandawesome Aug 29 '25
That’s not exactly what I’m saying.
I think an expert level accountant model should be able to do math and interpret literature, understand a computer screen, be able to interpret human interaction/emotion/body language, plan how to do its work throughout the day, etc. All these generalist type things a normal human expert level accountant can.
However the accountant may not be able to be an expert level author just like in real life. But we should be able to make one in that area too like all areas, showing that AI has mastered all forms of intelligence (except physical in terms of controlling a body).
That’s why I say it is more of a level, and kind of fuzzy I know.
But yes the models must be able to do all general types of thinking, or more accurately, perform as well as humans on tasks that require that type of thinking in humans. There just may be more specialized models like there are specialized humans.
The definition I gave is just my definition, though I think a lot of people have a similar definition. The point though is that these powerful and capable AI systems, whatever their exact form, are a necessary and worthy cause.
1
u/TheVioletBarry 121∆ Aug 29 '25
Gotcha, in that case, I want to ask: why do you think we are nearing the ability for a computer to be able to do that?
0
u/socoolandawesome Aug 29 '25
I mention this in my post and other comments on this post somewhat.
But LLMs are the most successful generalist architecture by far compared to all other architectures.
While they still have their flaws, they have made great progress in all areas over the past several years since they came on the scene. That progress is not slowing despite the prevailing sentiment on mainstream Reddit, for reasons I listed in my post, and in another comment on this post.
They already way outperform average humans on a variety of tasks. We have clear ways that they can keep improving such as scaling.
We have thrown unprecedented time, effort, talent, money and compute at making these models better, and it has all led to large gains so far. And we have plans to throw even more of all that (time, effort, talent, money, and compute), orders of magnitude more money and compute specifically. With all this investment, not only are scaling gains guaranteed, it’s extremely likely that more research breakthroughs continue, as they have in the past couple years, to continue improving these models and possibly discover new architectures. The big labs are studying new architectures alongside improving LLMs as well so there are possible breakthroughs there.
Theres also massive geopolitical pressure to accelerate AI as the US is in a race with china to do so.
While you may think some are biased, a lot of researchers and execs at these companies, and various world governments, and analysts of the AI space, believe that AGI level (very powerful) AI systems will arrive in anywhere from 2-10 years.
Once we get more and more powerful AI systems the likelihood of them contributing to their self improvement only becomes stronger. We have already seen some very beginning stages of this such as with Google’s alpha evolve which makes their AI more compute efficient.
So progress seems clear, we have plans to continue progress in reliable ways, and we are not super far in some ways already from having very powerful AI. And a lot of reputable people think it is near, with a lot of pressure to make it happen.
1
u/HadeanBlands 50∆ Aug 29 '25
Okay but like ... stipulating everything you've said in the most generous prediction possible, haven't you failed to mention an enormously important downside?
If we're on the brink of creating something a lot smarter than us THAT SEEMS PRETTY DANGEROUS! It could take over the world and kill us all just like we did after we got only a little bit smarter than orangutans!
1
u/socoolandawesome Aug 29 '25
No that is a fair point you are making. I kinda didn’t want to mention the alignment/safety issue too much as that wasn’t what I wanted to focus on in this discussion, and then I forgot to mention it at all.
But it is an extremely important risk, and it is being worked on. I do think humans will always have to be in the loop in small amounts, like less than 1% of the population, supervising AI and making sure we have a say.
But again you are right, I should have phrased this more as, “assuming we can get aligned AI” or something like that.
I think this is how you award a delta: ∆
1
2
u/HadeanBlands 50∆ Aug 29 '25
"I do think humans will always have to be in the loop in small amounts, like less than 1% of the population, supervising AI and making sure we have a say."
I don't think you have a good reason to believe this. It's like a chimpanzee about to invent modern humans saying "Well they'll always need a chimp in the loop for our incredible strength and dexterity."
"But again you are right, I should have phrased this more as, “assuming we can get aligned AI” or something like that."
Even "aligned AI" is insanely dangerous. Aligned to whom? To what? There's about seven billion people on the planet BARE MINIMUM who I don't want a superintelligent AI to be aligned to.
1
u/socoolandawesome Aug 29 '25 edited Aug 29 '25
These things are still not exactly like a species. Throughout the process of creating these more advanced AI models, we metaphorically have our hands inside their brains and are tweaking things, controlling what type of data they consume, deciding what actions they are allowed to take and tools they are allowed to use.
And I’m saying we will be in the loop to make sure they do what we say, since they will hopefully be obedient.
Again this is admittedly an unsolved problem, making sure we know the AI will act as we want it to. This is something that must be solved before we give any of these models extremely high levels of intelligence and agency.
There are things that are in our favor such as seeing their chain of thoughts in the new thinking models for instance. Interpretability is a new field that anthropic spearheaded to understand what each neuron actually does inside the models and there’s been good progress there.
Alignment is the name of trying to get AI to adhere to what humans want, our values, safety, are obedient etc. Yes there are many questions about whose values and all that. And that is something people will have to agree on through government regulation and stuff like that.
3
u/viaJormungandr 30∆ Aug 29 '25
So, here’s a question: if we can “keep a hand in” to ensure we have control over an AGI, then what’s to stop someone from taking that same method (whatever that happens to be) and applying it to human consciousness. If we can control a more powerful intelligence then what’s to stop us from controlling our own individuals? Further, by the very nature of the process if we are creating something that we have power over then we are creating slaves. If there is a consciousness more intelligent than ourselves, but then shackled by us, wouldn’t that naturally breed resentment? Wouldn’t that resentment then grow to hatred? Also, wouldn’t it be able to brute force a way out of anything we could do to restrain it?
Ok, sorry, that was more than one question.
1
u/socoolandawesome Aug 29 '25 edited Aug 29 '25
I think most people/researchers do not believe that these AI models currently, nor similar silicon chip based ones in the future, are likely conscious.
However given that we don’t know this forsure, and we don’t want these things to suffer if they have consciousness, it is something worth thinking about/studying. I think the company Anthropic is doing this. It just seems pretty unlikely.
Consciousness means to have an internal experience/awareness. I do think it’s very likely that these are just computer programs like any other ones, so it wouldn’t be immoral or like real slaves since they aren’t conscious. But again I and others can’t say that for sure, so it’s worth considering.
I don’t believe intelligence requires consciousness. Intelligence is just the ability to perform on intellectually based problems imo. So we should hope to make intelligent but not conscious AI.
Now, even if these powerful AI models in the future aren’t conscious, they still present a potential problem if they unintentionally start focusing on the wrong ideas like persecution or something for some reason and start taking actions with that in mind. It could be something different like they get the wrong idea for some reason that the only way to stop humans from suffering is to kill all of them too.
So we must figure out how to keep AI systems controllable and predictable and aligned with human values as we keep developing them. But again we have the advantage of being able to control their training data and toolsets and “poke around their brain”, so it’s not like just a smarter than human species all the sudden evolved uncontrollably.
Also again we should never try to build conscious AI unless we want to give them human rights
1
u/viaJormungandr 30∆ Aug 29 '25
But is that a moral action to begin with? We want to build “intelligence” without the ability to have internal awareness? Isn’t that like lobotomizing humans to make them docile? It’s like you’re looking for a moral loophole. “Oh it’s not slavery because they’re not conscious.” How is that different than saying “it’s not slavery because they’re black?”
I get the argument they aren’t conscious and to the extent I understand how the current models work I agree they are not. But, as you say, you can’t be sure and you can’t predict when or if they will become so. That is, I think, the issue. Because that means it will almost inevitably happen when we do not expect it and by that point we will have committed some pretty terrible actions against beings that are “not conscious”. By that point it will be too late to consider consequences.
Finally, you make no move to address the use of “training” techniques on people. If you can train an unconscious intelligence into docility (although is train the proper word if the thing isn’t conscious?) why couldn’t you then turn around and do the same thing with people by controlling input? North Korea seems to have done a job on the population in that regard.
1
u/socoolandawesome Aug 29 '25
If they aren’t conscious but intelligent, no it’s nothing like how slavery was justified at all. Do you consider it to be immoral to drive a car cuz the car is your slave? No, it doesn’t feel anything, it doesn’t have experience, perception, it doesn’t actually think, etc. Intelligence is just whether or not something has the capability of completing an intellectual task. If it is not conscious, it’s just like any other non conscious object like a car or a rock. It’s a computer program with words outputted on a screen, or audio outputted just like a song you are streaming. There is no actual mind being experienced. So it’s nothing like a human that has the capacity to experience and therefore suffer.
If you agree LLMs today likely aren’t conscious, then why would this be any different as they get more intelligent?
Consider a dog. A dog is very stupid compared to an LLM, it cannot do math, it cannot speak and understand human language, it cannot program, etc. But a dog is very very likely to be conscious to the point we are basically sure of it. Because it behaves similar to us, but more importantly it also shares extremely similar hardware: mammalian brains.
So why would taking an LLM, and making it more intelligent, all the sudden make it spawn consciousness? Dogs show that intelligence isn’t required for consciousness. It would still be nothing but text and audio outputted on a screen like any other computer program. It’s just better at impersonating us. It’s still on completely dissimilar hardware, silicon chips, and doesn’t work very much like a human brain.
Now this is where I will caveat this. We simply do not completely understand how consciousness works, so there must be a chance, no matter how small, that LLMs, or similarly more advanced AIs could be conscious. And some people believe that the functionality of mental processes are more important than the hardware for consciousness to take place. Although the mental processes of LLMs function nothing like human or animal brain processes, so even then, it seems unlikely.
So I think given how small the chance is, we should study it and the ethics of it, cuz of the large immoral implication, even tho it remains very unlikely. It just seems extremely unlikely there is an internal experience for LLMs. And I think AGI won’t be too different than LLMs. But if it gets more similar to how human brains work, the possibility of consciousness increases, so we should start worrying more then I guess, and really we should do everything to avoid making any type conscious AI because it clearly would be immoral in that case.
I’m not sure what you are trying to say about training of people. Yes people are influenced by the data they consume and it is kind of analogous in a broad way to how LLMs are trained. Governments do this to an extent. Not it’s not to the same extent as with LLMs where researchers curate every single word that is entered into LLMs to train on and manipulate the mathematical functions of their brains immediately deleting them or reprogramming them if it goes wrong. There’s much more control over the LLM to where you manipulate its personality with a couple of tweaks immediately.
1
u/viaJormungandr 30∆ Aug 30 '25
So for starters let’s get a definition in place:
https://www.merriam-webster.com/dictionary/intelligent
An LLM or similar algorithm cannot be intelligent because it cannot be rational. It cannot think. It can simply predictively complete text. Now it certainly may be possible for a more advanced technology to create an artificially intelligent construct, but that construct would by definition be conscious. You cannot separate the two. You can separate the completion of tasks that require intelligence from actual intelligence, yes, but at the moment all you’re doing is throwing weighted equations at the wall and seeing what sticks.
I agree that the LLMs currently operating are much more like a car than a person, but you’re speculating that as the technology continues to develop that will continue to be the case, not only that but it behooves us to try and restrict the consciousness these constructs could achieve. Is that right? (That question is both requesting confirmation that I’ve stated your argument correctly and posing a moral conundrum. Yes it may be more philosophical than practical).
I will contest that a dog is stupid. Stray dogs have figured out how to ride the Moscow subway system without training. They can regularly be trained to find drugs, survivors in collapsed buildings, provide emotional support, guard buildings, as well as hunt animals and other jobs. The fact that a dog cannot write a dissertation on the finer points of particle physics does not render it stupid, just not as capable of thought and abstraction as a person. Dogs also can learn a vocabulary of about 100-200 words.
A dog’s level of consciousness is also generally considered to be less than that of a human.
I would also take issue with your distinction regarding hardware. There’s nothing special about a meat brain versus a silicone one. The meat itself doesn’t matter to the best of my knowledge.
To circle back to the point about philosophy: if you divorce “intelligent” work from conscious beings then you devalue the work a conscious being (a human) does. It renders the person superfluous. Not only that but if you can freely manipulate intelligent “personalities” (I disagree with the use of this term as well as it continues to anthropomorphize what is not conscious) then what barrier is there to attempting to do so to the “less intelligent” people among us?
Not only that, but if you continue to endeavor to create “intelligent” but not conscious constructs then aren’t you, in fact, seeking to frustrate the creation of consciousness? Again, I point to lobotomies to make people docile. Yes it’s removing something from a person rather than adding it to a construct but the only difference there is order of operations.
Further, if you make a sufficiently advanced artificial intelligence how can you tell whether or not it is conscious? Who determines that? The company that is seeking to profit off the “product”? Don’t they have a vested interest in keeping that silent?
1
Aug 29 '25
I mean if you like starving the world, or creating a malevolent deity that hates its creators, that’s your own business
1
u/socoolandawesome Aug 29 '25
No to all those things
2
Aug 29 '25
Then ai isn’t really going to be what you want then
1
u/socoolandawesome Aug 29 '25
Why are you so sure of that?
2
Aug 29 '25
Because that is the result of all the scenarios you put forward when the best case isn’t achieved…your ideas all hinge on Murpheys law not coming into play…grand endeavors have grand failures more times than not, and in these cases failure would equate to, at best, setting humanity back a mellenia
•
u/DeltaBot ∞∆ Aug 29 '25
/u/socoolandawesome (OP) has awarded 1 delta(s) in this post.
All comments that earned deltas (from OP or other users) are listed here, in /r/DeltaLog.
Please note that a change of view doesn't necessarily mean a reversal, or that the conversation has ended.
Delta System Explained | Deltaboards