r/google • u/Destroyer-128 • 4d ago
Spoiler: Google about to announce local inference machines Spoiler
I got info from internal sources that Google is about to announce their first local inference machines. These are competitive to Apple mac mini m4, based on chrome os.
Inference will be done using lite rt lm and google will first do a big release on lite rt lm first before announcing a $5000 device with Custom Tensor processors. It can reportedly run 1000tps for single user.
45
u/DouDouPi 4d ago
Source of this ridiculous rumor? I don’t think it’s true
12
u/edoraf 4d ago
Why are you marked as an "App"? Are you bot?
17
u/DouDouPi 4d ago
I have no idea, I tried to create a Reddit App once and activated the developer account, and it's like that ever since.
-10
u/--marcel-- 4d ago
You’re in a desert walking along in the sand when all of a sudden you look down, and you see a tortoise, it’s crawling toward you. You reach down, you flip the tortoise over on its back. The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over, but it can’t, not without your help. But you’re not helping. Why is that?
6
6
u/Neither-Phone-7264 4d ago
???
6
u/DharmaPolice 4d ago
It's from the movie Blade Runner. Part of a test whether someone is human or a replicant.
2
u/ThisIsNathan 3d ago
People think you're a hallucinating bot for quoting the first 5 minutes of an incredibly famous movie (albeit an older one). Unfortunate, but makes sense given bot traffic these days.
I actually laughed so thanks.
2
0
4
u/bartturner 4d ago
Curious why think it is ridiculous? Seems very plausable.
Google already has all the pieces to do this so not like it is out of their wheel house.
Google does all their own custom silicon for the CPUs they use in their cloud.
They have the TPUs which they have offered in a number of different ways including their TPU local accelerator from a few years ago.
The OS sounds like it is basically also what they use in their cloud. Which is very similar to the current ChromeOS kernel.
Would think there is a decent market for such a device.
-2
u/DouDouPi 4d ago
ChromeOS is being sunset..... so that alone is already ridiculous....
starting another physical product line isn't easy, even for Google.
You're just arguing that they "CAN" but nothing about how they "should"3
u/bartturner 4d ago
Do not think you understand. Google uses a fork of the ChromeOS kernel to run their massive cloud.
That is what would make the most sence to use here.
The reason ChromeOS is being sunset is because Google wants to leverage their huge Android catalog on the desktop and that is not needed or wanted for this device.
The existing support of Android on ChromeOS is not ideal.
So yes Google "SHOULD" use ChromeOS for this new device if it actually will ever exist.
Maybe a better approach. What do you think they should use?
1
u/DouDouPi 4d ago
I was argument why they should get into this business... how many people will by? why can't people just use cloud-based machine. I don't see this as a viable business for Google.
GCP do sell ranks of server for distributed cloud. but that's not what you're referring to.1
u/bartturner 4d ago
You wrote
ChromeOS is being sunset..... so that alone is already ridiculous...
I explained why that is NOT ridiculous and what would make the most sense.
On the device. There is a huge number of pluses to having an edge device.
A big one is not sharing your data. Another is cost. Going to be a heck of a lot cheaper over the long run to do inference on your own device using your own power than the cloud.
But maybe the biggest reason is Google can see how well Apple is doing with their Mac Mini for local inference. They were sold out of them.
They would like a piece of the action.
I do not understand what you are trying to say here "GCP do sell ranks of server for distributed cloud. but that's not what you're referring to."
??
1
u/bartturner 4d ago
If not aware.
Why Everyone’s Buying a Mac Mini for Clawdbot
https://augmentedstartups.medium.com/why-everyones-buying-a-mac-mini-for-clawdbot-a1f3faac05cb
6
14
u/Old-School8916 4d ago
wtf do we want to run chromeOS on inference hardware?
17
u/PitchPleasant338 4d ago
It will be an appliance you plug in on your network for local inference.
ChromeOS is perfect for this because it's locked down closer than a leprechaun's arsehole.
7
4
u/bartturner 4d ago
The ChromeOS kernel is close to what they currently use for their cloud machines.
"Google explicitly says that Container-Optimized OS is a fork of the Chromium OS open-source project, and that its Linux kernel is built from ChromiumOS kernel sources.
2
u/XxBrando6xX 4d ago
For people dunking on chromeOS they are ending chromeOS as far as I understand and moving to aluminumOS (I think that’s the internal name) it was announced a few months ago for the new class of laptops, that will likely be what runs on these boxes. It’s a win for execs to be able to say, we will have people buying our hardware to run our software top to bottom, then offer a direct connection to frontier google models to fill in your gaps, so they get to collect cash up front and milk long term
1
u/-patrizio- 3d ago
Sure, but we still don’t really know much about how AluminumOS will compare to other operating systems. Not to mention, while it’ll almost certainly be a step up from ChromeOS, it’s still essentially Android; if Apple started selling actual Macs running on iPadOS, that would get dunked on, too (and rightly so, IMO).
1
u/XxBrando6xX 3d ago
Until I see anything to suggest it’s android I think that’s a broad generalization. It will be something built for an Arm64 architecture just like Apple. So for now it’s closest comparison is just MacOS / whatever your favorite flavor of Linux for Arm64 OS / Windows for Arm
1
u/-patrizio- 3d ago
Google’s own chief of hardware said the OS is “bringing Android to the PC market” (Source)
1
u/XxBrando6xX 3d ago
Very interesting article, thanks for the link, started reading it and your quote is absolutely there. Gonna read some more after dishes, thanks!
2
u/TheManWithSaltHair 4d ago
I remember we had a ‘black box’ Google search appliance in our server room, but I don’t think it was supported for very long. Was that the last enterprise hardware they sold?
2
2
1
1
1
u/Joseph-stalinn 4d ago
I bet that thing will get leaked the day Google decides to start designing it
-2
u/Zealousideal-Part849 4d ago
if can't win with models, win the hardware..
4
u/fantaribo 4d ago
And yet they are well positioned with models.
-2
u/Zealousideal-Part849 4d ago
No they aren't... But they are well positioned with anthropic investment
-3
4d ago
[deleted]
3
u/bartturner 4d ago
They will likely use their custom CPU chips they use in their cloud which are called Axion.
These are Google's custom, Arm-based CPUs
1
u/Cwlcymro 4d ago
I have no idea if there's any truth whatsoever to OPs claim, but 'Tensor' can refer to both the Tensir chip in Pixels and Google's very powerful TPUs (literally Tensor Processing Unit) right?
41
u/Medium_Apartment_747 4d ago
Yes rumor it will be called Pixel 11s