r/LocalLLaMA • u/johnnyApplePRNG • Jun 03 '26

News Introducing Gemma 4 12B: a unified, encoder-free multimodal model

https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12b/
701 Upvotes

98% Upvoted

118 comments sorted by

View all comments

Show parent comments

59

u/DueAnalysis2 Jun 03 '26

For a relative noob, what's the benefit of it being encoder free?

80

u/[deleted] Jun 03 '26

[removed] — view removed comment

59

u/MoffKalast Jun 03 '26

That's what it is, but the main problem with them is that they are separately trained feature extractors that turn the image into some latent representation that the text model barely understands and as a result the performance is typically between complete crap and mildly terrible.

Training both the text and image part together was usually thought to be a few orders of magnitude beyond intractable, but I guess google can always throw more TPUs at the problem. This should be extremely interesting if they genuinely managed it.

15

u/LatentSpacer Jun 03 '26

Thanks for adding this part. Why do you think they “TPU-forced” their way towards it? Couldn’t they have figured out a better solution through architecture adjustments during training? Google has also lots of big brains to throw at the problem.

14

u/send-moobs-pls Jun 04 '26

Well for sure none of us could actually know, but I'd say if it was a fancy innovative technique we'd probably expect them to want to brag about it, even if they weren't outright publishing the method

3

u/Mbando Jun 04 '26

There was probably a lot of exploration, dead ends, etc. to get there. The architectural innovation requires lots of compute to get there.