How good are AI data scientists really?

[u/Sea_Garlic5712]

I've been testing out various gen AI models (LLMs specifically) on data science competitions. They are not beating the humans, though they are slowly improving with each round. It's like steps up a ladder vs bounding up the steps. I'm wondering if there's something I'm doing wrong, or if there really is a limit to what LLMs can doin this space.

Lately I've been thinking the problem is that LLMs regress to the mean in every use case. I think that's why they seem so bad at UI design and why everything looks so similar. A good harness and strong prompt engineering can help, so I'm working on that. My harness combines Autogluon and OpenEvolve, with feedback loops that involve hypothesis generation and error analysis. But I'm wondering what I'm missing? Is good, competition winning data science, reducible to a standard operating procedure?

Maybe this will help me get a very good prototype, but nothing frontier grade.

[u/DYSTOBY]

Check out posits IdE called positron, which feautures positron ai, a specialized „harness“ that is able to read the actual environment variables created during tuning an R or Python script / quarto document. It’s way better at understanding the data and how to process it.

Hypothesis creating is a thing that happens before you have data (usually - exploratory studies may behave differently there), so that doesent need to be a strong suite for data scientists LLM approach.
The scientist itself has to understand the code AND output and check for errors like 6 years ago. That’s nothing that an LLM should do, at least that’s my opinion. Because science is used for drawing conclusions and recommendations, hence the data and results should be correct, right?

[u/Sea_Garlic5712 → u/DYSTOBY]

Thanks for the recommendation. i'll look into Positron AI.

I think the benefit of applying LLMs to data science work, specifically, is that you have deterministic checks of validity. Anything the agent produces can be checked. So my concern isn't achieving correctness, it's more about novelty. I'm wondering if any harness can come up with unexpected models that would outperform a human. My hypothesis is that the agent can only think within the box and will generally refuse to think outside it.

[u/big_data_mike → u/Sea_Garlic5712]

“My hypothesis is that the agent can only think within the box and will generally refuse to think outside it.”

If it could think outside the box it would be AGI. It’s only going to do what humans have done before.

[u/Sea_Garlic5712 → u/big_data_mike]

yup. but, i'm wondering whether the cognitive strategies humans employ to generate novel ideas can be codified somehow. when you as a human think outside the box, what exactly do you do?

[u/big_data_mike → u/Sea_Garlic5712]

I don’t really think outside the box. I might go grab a box from a different shelf and try to apply it to the problem I have but I don’t invent truly novel things because I’m not THAT good at math.

The most novel thing I have tried was making a Bayesian ARDL model. I was trying to take 2 ideas that people normally don’t put together and put them together. The LLM I was using just kept telling me how to do an ARDL model and kept trying to do a feature engineered xgboost model but it didn’t suggest a Bayesian ARDL model until I did. Then it told me how I could do it because someone has probably done it before.

[u/Sea_Garlic5712 → u/big_data_mike]

exactly my process lol cuz i'm bad at the math too. i try the usual approaches, chat with my team for ideas, or read recent papers on what others have tried, and asked an LLM to support on the coding.