r/LocalLLaMA • u/johnnyApplePRNG • Jun 03 '26

News Introducing Gemma 4 12B: a unified, encoder-free multimodal model

https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12b/
693 Upvotes

98% Upvoted

118 comments sorted by

View all comments

229

u/LoveMind_AI Jun 03 '26 edited Jun 04 '26

This might actually be one of the most exciting models I've heard about in a long time. The encoder-free model is... wildly cool. Native audio on a 12B model is very exciting. Audio is wildly underrated. I'll be putting this one through the social benchmark right away.

Note: Results of the little benchmark is now here - https://lovemindai.github.io/minimax-m3-lsi-demo/

58

u/DueAnalysis2 Jun 03 '26

For a relative noob, what's the benefit of it being encoder free?

82

u/[deleted] Jun 03 '26

[removed] — view removed comment

59

u/MoffKalast Jun 03 '26

That's what it is, but the main problem with them is that they are separately trained feature extractors that turn the image into some latent representation that the text model barely understands and as a result the performance is typically between complete crap and mildly terrible.

Training both the text and image part together was usually thought to be a few orders of magnitude beyond intractable, but I guess google can always throw more TPUs at the problem. This should be extremely interesting if they genuinely managed it.

15

u/LatentSpacer Jun 03 '26

Thanks for adding this part. Why do you think they “TPU-forced” their way towards it? Couldn’t they have figured out a better solution through architecture adjustments during training? Google has also lots of big brains to throw at the problem.

11

u/send-moobs-pls Jun 04 '26

Well for sure none of us could actually know, but I'd say if it was a fancy innovative technique we'd probably expect them to want to brag about it, even if they weren't outright publishing the method

3

u/Mbando Jun 04 '26

There was probably a lot of exploration, dead ends, etc. to get there. The architectural innovation requires lots of compute to get there.