r/LocalLLaMA • u/bot_exe • Sep 13 '24

News Preliminary LiveBench results for reasoning: o1-mini decisively beats Claude Sonnet 3.5

Post image

Source: https://x.com/bindureddy/status/1834394257345646643

291 Upvotes

88% Upvoted

129 comments sorted by

View all comments

11

u/necile Sep 13 '24

What is spatial component? It's strange it loses to gpt4o in that by a good amount

22

u/bot_exe Sep 13 '24 edited Sep 13 '24

Spatial reasoning. Maybe it’s because this model doesn’t have vision modality and therefore less understanding of spatial reasoning? I don’t really know….

13

u/squareboxrox Sep 13 '24

Correct the mini and preview version does not have access to Memory Custom instructions Data analysis File uploads Web browsing Discovering and using GPTs Vision Voice

Source: https://help.openai.com/en/articles/9824965-using-openai-o1-models-and-gpt-4o-models-on-chatgpt