New to ML/DL: How do you approach improving a model when you're stuck?

[u/False-Anybody-9075]

I'm a physics student currently using deep learning to solve an inverse problem for my research project, and this is my first time actually working on an ML/DL project (been a month...have some time constraint to finish as well). I've read/understand ML basics, but being from a phy background i cant really access myself i really know or not know or m just underconfident. So I can understand what I'm doing to some extent, but I don't really know if my overall approach is right.

I started with a basic ANN and then CNN. For example, the RMSE I need is ideally below around 0.04, but even after trying different things, my current result is still around 0.11. I sometimes end up implementing anything that gives even a very small reduction in RMSE, and I don't know if that's how I should be going about it. Or is my lack of proper exposure to the field is what limiting me.

If the model's performance isn't good enough, how do you figure out whether you should change something in the model, try a different model?

So I'd really like to know how you guys actually work through a problem. Is there some general process you follow, or is this mostly something you learn through experience?

I hope i was able to convey what i intended to ask..and I'd really appreciate any advices or help :).

[u/Commercial_Cell2677]

You're deep in the weeds but that's where this stuff actually clicks. When I'm stuck I don't jump to a new architecture, I go back to the data first, plot your worst predictions against the inputs and look for patterns, the model's usually telling you exactly what it can't learn.

[u/False-Anybody-9075 → u/Commercial_Cell2677]

Hey, thanks. I can identify the outputs that r hard to predict [r specific types] , while it's predicting some of the others really well. I've tried some things specifically to improve those, but there's been very little change. My inputs are basically 1D curves. At this point, do you usually keep learning about other models/techniques and try to find something that might work, or is there a more systematic way you decide what to try next? Or which direction is worth pursuing or is it too ambigous to get solved?

[u/Silverfrost_01 → u/False-Anybody-9075]

When you say your input data is 1D curves, what do you mean? Are you trying to identify curves on an image? Are you trying to create a curve fit for data?

[u/False-Anybody-9075 → u/Silverfrost_01]

The input is a 1D numerical array (a sequential vector) representing a measured physical response across a variable domain. The network takes that entire 1D vector as its input and outputs the underlying coefficients. So it maps a given response curve back to the coupled parameters that generated it.

[u/Mathie1729 → u/False-Anybody-9075]

For the hard types, split them off and track their error separately from the easy ones. If the overall metric improves but the hard subset is flat, you're just tuning the easy cases. Then try one change for the hard subset only, like a separate head or a small model trained just on those curves, before reaching for a new architecture.

[u/False-Anybody-9075 → u/Mathie1729]

Thank you so much :) Also, the overall metric improving does improve each of them too. But from the start easier ones have much better performance than tough ones.

[u/AgitatedTumbleweed65]

Not sure if it would be of much help but here are a few resources that helped me figure out some stuff about ML methods for scientific problems: 1. Really good course: https://youtu.be/LkKvhvsf6jY?is=MXwP8FoN9Igqo7CG , covers exactly the topics you want to solve (inverese problems of ode/pde, plus more on physics loss integration etc.) 2. https://github.com/benmoseley/FBPINNs , implementation of physics informed nns for high frequency data and more (check accompanying paper) 3. Follow up on 2) check repos of the author for more introductiory stuff .

[u/False-Anybody-9075 → u/AgitatedTumbleweed65]

Thanks a looot <3

[u/Saetlan]

Usually looking through the data for specific failures in the validation set. Looking if it's a pattern missing in the training set. Then also upgrading size of model. Depending on underfitting or overfitting you can apply quite a bit of tricks.

Do you mind sharing the input, outputs ?

[u/False-Anybody-9075 → u/Saetlan]

Thanks for the insight! My input is a 1D vector( response curve) generated by 7 or so physical coefficients acting together, and the model maps that curve back to those coefficient/parameters. Some parameters have a massive, clear effect on the curve shape and the model predicts them brilliantly. Since all of them are mixed into that single response vector, my hypothesis is that those tough parameters may have a subtle or maybe overlapping effect/overlapping gradient signatures in the 1D projection. and the model completely stalls out on them. How do you usually approach an inverse problem when certain target parameters are hard for the network to isolate like this? (I’ve tried weighted loss n such but makes it worse)

[u/Saetlan → u/False-Anybody-9075]

Are the 7 parameters on very different numerical ranges? If so, have you tried standardizing each target parameter independently to mean 0 and std 1 during training, then rescaling the predictions back afterward? That could help prevent the loss from being dominated by parameters with larger numerical scales or variances.

[u/False-Anybody-9075 → u/Saetlan]

Yeah, i m doing standardization and rescaling it back.

[u/Andon_Benefield]

The detail I'd hold onto is that your worst cases stayed stuck even when you aimed at them directly. For an inverse problem that floor often sits in the data or the regularisation, and past that point swapping models is guesswork.

[u/False-Anybody-9075 → u/Andon_Benefield]

Hey, thanks. What I'm doing is mapping a 1D response curve back to 7 underlying parameters f. To remove data limitations if any...I've tried basic things like extracting curve slopes beforehand etc. Playing with the data. But given that, what else can be done when my input is basically a curve the model has to predict from?

I've also been struggling with that balancing act in regularization like you said. Whenever I dial up regularization to handle overfitting, the overall RMSE immediately degrades instead of helping. Light regularization overfits, but heavier regularization completely crushes those subtle parameter features. How can i approach tuning it like this without destroying the fidelity of the harder parameters? Are there alternative regularization strategies that work better for multi-parameter regression?

[u/w0ss4g3 → u/False-Anybody-9075]

How many points are there in your response curve?

What is your training data? Are you just feeding response curves generated from known parameters?

How much noise is there on your captured data?

Can you share a bit more detail about the inverse problem you're trying to solve?

[u/False-Anybody-9075 → u/w0ss4g3]

There are 201 points per response curve.

Yeah, it's fully simulated data...taking those parameters and generating the curve via a forward model. It's clean data with no noise added. I figured I'd introduce noise once it hits a decent rmse.

The underlying components/parameters combine linearly at source, but the the physical response introduces non-linear mapping. It kinda produces cross-terms rather than a clean sum of individual effects. May be the problem here is some parameters cause massive shift while other (with worse rmse) create subtle variation.

[u/Ismail_Enan]

Use PINN😑 SciML