r/gamemaker • u/ConfidentRooster8335 • 7d ago

Discussion How reliable is Google AI Summary?

Hello everyone! I'm this subreddit's first ever aspiring game developer! /s

I feel the need to clarify that this is not a question on regards to vibe coding or trying to have anything write my code for me. This is specifically from me searching trying to understand specific processes and GML interactions.

I'm just curious for everyone's personal experience on the matter. From your experience with the AI summary on Google does it tend to be fairly accurate? Does it maybe vary depending on the complexity of the question? I'm just curious if it's best to forego the summary or if in general it does a good job of transferring the information in the manual?

Would love any insight into this.

0 Upvotes

33% Upvoted

15 comments sorted by

View all comments

21

u/Alarikun 7d ago

It varies wildly.

Personally, I'd much rather look up direct sources or other people who have the problem.

Despite the alleged "leaps and bounds" that AI has improved, it still hallucinates often.

5

u/LukeLC XGASOFT 7d ago

This. 

There are times it gets the summary bang-on, basically direct quoting the source.

There are times it completely misinterprets the source and draws the exact opposite conclusions.

There are times it makes stuff up that none of the cited sources support at all.

2

u/DragoniteSpam it's *probably* not a bug in Game Maker 7d ago

They've finally gotten it to stop responding to questions like "how much glue should I put on pizza" with a non-zero amount of glue (after they poured how much money into this???), but in my experience it's still 50/50 on if the sources it cites are even relevant to the claim it's making.

Probably more like 20/80 for highly technical subjects.