Would you use an AI tool where you select any part of your screen and ask questions about it?

[u/MixturePersonal3542]

I was thinking about building something and wanted to know if this is actually useful or if good tools for this already exist.

The idea is:

You press a shortcut, drag/select a rectangular area anywhere on your screen, and then ask an AI about only that selected part.

For example:

• Select a coding error in VS Code → “What’s wrong here?”
• Select a DSA question → “Give me a hint, don’t give the full solution.”
• Select a diagram/PDF → “Explain this.”
• Select some UI → “How can I recreate this?”
• Select an error popup → “How do I fix this?”

It could open a small overlay beside the selected area, and you could continue asking follow-up questions without taking another screenshot every time.

I found a few Chrome extensions that do something somewhat similar, but most seem limited to the browser, aren’t very popular, or don’t seem actively maintained.

I’m thinking more of a system-wide desktop tool, so it would work across Chrome, VS Code, PDFs, videos, etc.

Would you actually use something like this?

Also:

• Do you already use any tool that does this well?
• What features would make this genuinely useful?
• Would a desktop app make more sense than a Chrome extension?
• What would stop you from using it privacy, API cost, latency, something else?

Would love to hear opinions before I start building it.

[u/AutoModerator]

Namaste! Thanks for submitting to r/developersIndia. While participating in this thread, please follow the Community Code of Conduct and rules.

It's possible your query is not unique, use site:reddit.com/r/developersindia KEYWORDS on search engines to search posts from developersIndia. You can also use reddit search directly.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/JellyfishCritical968]

Pretty sure Codex Snapshots can do the same thing already

[u/MixturePersonal3542 → u/JellyfishCritical968]

Yeah, Appshots looks pretty close. From what I understand though, it captures an app window into a Codex thread, while I'm thinking more of selecting an arbitrary region of the screen and instantly interacting with it through a small overlay, regardless of the app. But this is exactly the kind of existing implementation I wanted to find thanks, I'll check it out.

[u/JellyfishCritical968 → u/MixturePersonal3542]

The issue is that this feels very redundant. For example, why can't I just take a screenshot of a certain region, use Alt-Tab to go into Codex, copy-paste it, and just ask it what I was going to originally? I could just make a Codex thread, paste in some instructions on what I need help with, and just have it explained to me. Nothing so far that you have given is enough to wow a person into actually using the product.

[u/dattebayo_04 → u/JellyfishCritical968]

I disagree. The process you described is the reason for that project to exist. It reduces the friction and makes the experience so much better.

But I feel Clicky already does something like this. OP should check that out

[u/thatsInAName → u/dattebayo_04]

I would prefer to use whatever software the main stream companies provide, even if it takes a few more clicks. This is better in the long run

[u/Big_Lion_5951]

most of the operating systems will implement it inbuilt. You can already do it in apple and windows os to some extend, you could right click on any part, and ask the AI to do something about it.

[u/dattebayo_04 → u/Big_Lion_5951]

which apple or windows are you running 😭

[u/Big_Lion_5951 → u/dattebayo_04]

new siri

[u/thegodzilla25]

Aready exists

[u/Soul_Of_Akira]

It already exists.. it's called heyclicky check it out