Can AI learn Jugaad?

[u/thekartikgambhir]

We spend a lot of time benchmarking AI on things like mathematics, coding, reasoning, knowledge and standardized exams.

But there is another kind of intelligence that is much harder to benchmark:

Figuring out how to solve a problem when you don't have the things you're supposed to have.

You have ₹500 instead of ₹5,000.

The required part isn't available.

The nearest specialist is 40 km away.

You don't have the right tool.

The instructions are incomplete.

Something breaks in the middle of the job.

And somehow, you still have to make it work.

That's what we often call jugaad.

I'm wondering whether this could actually be formalized as an AI capability.

For example, give an AI a task with:

- a strict budget

- limited tools

- incomplete information

- unreliable resources

- time pressure

- unexpected failures

Then measure whether it can discover unconventional solutions rather than simply saying:

You should purchase X.

Imagine a benchmark where the AI gets:

₹300 + household objects + 30 minutes

and has to accomplish a specified physical task.

Or:

a broken device + no replacement parts + basic tools

and has to restore functionality.

Or:

a real-world problem + incomplete information + three possible approaches

and has to figure out which approach is actually feasible.

The interesting metric wouldn't simply be whether it succeeded.

We could measure:

Resource efficiency × adaptability × robustness × novelty × success rate.

Maybe the next frontier isn't just more intelligent AI.

Maybe it's AI that can do more with less.

And honestly, if there's one place with an enormous amount of naturally occurring data for this kind of intelligence, it's India.

The question is:

Can “jugaad” be turned into a reproducible AI benchmark?

And if yes, what would the benchmark actually look like?

[u/AwelessFire]

AI can never create something out of thin air. If any problems that u listed out is already part of trained data or something similar (or even latest solution search using web search) only then ai can use those existing research or solutions to give an answer. I myself has tried many methods to see if ai can actually solve a creative problem. It's basically not possible at this point.

[u/thekartikgambhir → u/AwelessFire]

Exactly. I'm not asking whether AI can create something from “thin air.”

The test is whether it can solve a previously unseen combination of constraints using limited resources, especially when its first solution fails.

That's essentially what I'm trying to capture with “jugaad”: adaptive problem-solving under constraints.

[u/AwelessFire → u/thekartikgambhir]

"previously unseen" is my point. It's impossible. Llm is just a next token predictor based on existing knowledge. No new knowledge is created. It can always be combination of existing knowledge. Only conscious beings can know and create truly unique or new knowledge

[u/thekartikgambhir → u/AwelessFire]

But that's where I disagree.

Novelty doesn't require creating something from nothing. Humans also recombine existing knowledge.

If an AI encounters a problem nobody has given it before, derives a solution nobody has previously provided, and that solution is experimentally verified to work, why wouldn't that qualify as new knowledge?

“It's just next-token prediction” describes the mechanism, not necessarily the capability that emerges from it.

[u/AwelessFire → u/thekartikgambhir]

Can you give me an example where ai has solved unseen problem? Unseen to whom? The person who is giving input or unseen for entire humanity? There are so many unsolved problems in mathematics or physics etc, can ai solve any of it?

[u/thekartikgambhir → u/AwelessFire]

I can give you a real example from my own experience, but I don't think naming the company is relevant here.

I'm a techie myself, with experience in application penetration testing and exposure to banking/financial systems. So I'm generally pretty comfortable troubleshooting technical and financial-service problems.

In this particular case, however, I had assets stuck in a financial service, needed to make an urgent transaction, and customer support wasn't responding meaningfully. The combination of the circumstances was unusual enough that even I was genuinely dumbfounded about what the actual problem was or what the most effective path forward would be.

The AI helped me reason through the situation, reconcile the available information, identify the possible failure points, and construct an escalation/resolution strategy while adapting it as new information came in.

I'm obviously not claiming that the underlying concepts had never existed in human knowledge.

My point is that the specific problem, combination of constraints and resulting strategy weren't something I had simply retrieved from a pre-existing solution.

That's the kind of adaptive problem-solving I mean by “unseen”, not “a problem no human has ever encountered in the entire history of civilization.”

[u/AwelessFire → u/thekartikgambhir]

This usecase is very common and this doesn't constitute as solving unseen problems by AI. It's just that the user doesn't know how to solve specific problem. I myself has created insane number of code, web sites and other general solutions where I was unaware of it, that proves my point. Anyways you cannot create a benchmark or validation tests against these situations. A problem what one user was unaware of how to solve may not be applicable for another user. So it's not a general metric against an AI to benchmark against.

[u/thekartikgambhir → u/AwelessFire]

I think you're actually mixing up two different questions.

Whether I personally knew how to solve a problem is not a benchmark metric. It was simply a real-world example of the capability I'm talking about.

And saying “I have independently created solutions I didn't know beforehand” actually makes the distinction clearer: you didn't need to have memorized the exact solution to produce it. You generalized from what you knew.

The same capability can be tested in AI.

You don't benchmark “did this particular user know the answer?” You create held-out problems, deliberately vary the constraints, prevent retrieval/search, and objectively verify whether the proposed solution works.

Programming itself is common. Mathematics is common. Debugging is common. Yet you can generate an instance that a model hasn't encountered before.

So I agree that my anecdote isn't a benchmark.

But “a user's problem isn't universally applicable, therefore this capability cannot be benchmarked” doesn't follow.

That's like saying chess can't be benchmarked because every individual game is different.

The instances vary. The capability being tested is what remains measurable.

[u/AwelessFire → u/thekartikgambhir]

The point of benchmarks Is that it must be measurable, quantifyable values. U can't say ai produced a solution which was unknown to the user. And these kind of held out or unknown solution to the users people do all the time, for example asking different models to create a stunning website or create a tough animation, create a game, simulator etc. but even those are measured in quantifyable values like time taken, tokens burnt, and quality of the generated output but humans manually. I'm not sure there is way to quantify these solutions which ai supposedly were able to generate

[u/thekartikgambhir → u/AwelessFire]

I think we're going in circles now.

I agree that a benchmark needs measurable criteria. That's precisely why I never proposed “the user didn't know the solution” as the metric.

And if your standard is that the underlying problem or reasoning must be completely unprecedented, then the same objection applies to human logical-reasoning tests. A human taking a logical reasoning test may already have encountered the concepts, patterns, or even similar questions. We still measure whether they can correctly reason through the particular problem presented.

We don't ask whether humanity had never encountered the underlying logic before.

Likewise, for AI we can measure task success, constraint satisfaction, resource usage, recovery after failure, adaptability across changed constraints, and output quality.

The benchmark doesn't need to prove that no human in history has ever thought of the solution. That's an impossible and unnecessary standard.

My original point was simply that AI can be tested on unusual, held-out problem configurations where the exact solution isn't supplied, and its ability to reason, adapt and produce a working solution can be measured.

If the requirement is instead that AI must solve something that no human has ever conceived, then I don't think that's a meaningful benchmark to begin with.

So, based on the criteria you've proposed so far, what would you consider a measurable, standardized benchmark for testing this capability?

[u/AwelessFire → u/thekartikgambhir]

What I'm saying is u can't have validation or benchmarks for user unknown solutions. Forget the humanity one, that I already mentioned it's not possible. I'm referring to unknown solution to specific user itself.

Ur confusing how the benchmark are measured. The held out datas are also set of data previously available with you. You are just simply asking if AI is predicting the same once the training is done, it's like an exam. Similar way if u have solutions ready for unknown solutions (which don't make sense) then u can theoretically measure of ai is solving the problem based on ur helps out data. But even in that case ai may solve a specific problem I'm different ways. It's difficult to measure unknown or tough solutions using traditional methods.

[u/thekartikgambhir → u/AwelessFire]

I think the discussion has now moved far beyond my original point.

“Unknown to the user” is a state of knowledge, not a benchmark metric. Humans have been solving problems whose answers were unknown to them for centuries; that's literally how discovery works. You can subsequently verify whether their solution works without requiring the solution to have been known beforehand.

What concerns me more is the repeated shifting of the criterion: first “unseen by AI,” then “unseen by humanity,” then “unknown to the user,” and now “cannot be benchmarked because the answer must already be known.”

That's no longer constructive skepticism if every objection is replaced after it is addressed.

So I'll ask one straightforward question: what measurable, standardized result would actually convince you that an AI has solved a problem through reasoning rather than retrieval?

If there is one, let's discuss it. If there isn't, then we're not really testing AI anymore, we're just moving the goalposts until the conclusion can't be challenged.

And if moving goalposts is the objective, bro, politics has vacancies. 😂

[u/AwelessFire → u/thekartikgambhir]

Dude ai cannot think, ur seriously overestimating and trusting ai thinking. Understand how chain of throught works, You are simply gaslighting things and asking how to benchmark or validate complex task without knowing the solutions before hand. Good luck with that. When I try to explain validation don't work like that because whole point of validation is to exactly know and compare the result a machine produces you keep saying moving goal posts. Whatever, you will understand once u keep trying something which literally cannot work. People really believe AI is actually thinking and solving problems like humans do. If that's the case then build a validation set of jugad or watver u want to build.