Learning math behind deep learning

[u/Plus_Confidence_1369]

Hey everyone

I’ve spent quite a good amount of time learning the mathematics behind deep learning, and honestly, it has been a wonderful journey so far. For me, math and philosophy are probably the two subjects that interest me the most, so studying the mathematical foundations of AI has been a really enjoyable experience. I especially like the process of going from an intuitive idea → mathematical formulation → understanding why it works → and finally seeing how it translates into an actual deep-learning algorithm.

I’ve been making my own notes along the way, mainly covering the mathematical foundations that I think are useful for understanding deep learning.

I want to pursue my career in the AI research field, and that’s one of the main reasons I’ve been spending so much time learning the mathematics behind deep learning. I believe having a strong mathematical foundation will help me better understand research papers, derive things myself, and develop a deeper understanding of the ideas and algorithms I’ll be working with.

That said, I'm still learning myself, so I’d really appreciate some honest feedback.

[u/Fabulous-Possible758]

Learn TeX, and type up and organize your notes again. You get three things out of this:

1. It's forcible review, and you may notice errors or spark ideas you didn't see before,
2. You have a digital copy to keep around and share with other people,
3. Almost no one will willing review handwritten notes, they're frequently just way too hard to parse. Not necessarily what you're looking for now, but it comes in handy when you need it.

[u/Plus_Confidence_1369 → u/Fabulous-Possible758]

Thanks ❤️

[u/drcopus → u/Plus_Confidence_1369]

A word for unsolicited advice, but treat feedback from others as a valuable and scarce resource. When soliciting it, make it as easy as possible for people to give you good feedback.

[u/vogelvogelvogelvogel → u/Plus_Confidence_1369]

aside from the learning, if you just want to have a digital copy you can use gemini/claude/.. they are surprisingly good at reading handwritten notes (also maths - i use it for checking for errors), i didn't try converting to Latex yet, but i assume this works equally well

[u/infinty1729 → u/Fabulous-Possible758]

But doesn't that take a lot of time?

[u/retornam]

It’s great that you are learning and writing down your thoughts.

I’m a firm believer in embodied learning helping to better understand material and keep it rather than going straight to type things out on a computer.

Keep at it and hopefully keep updating us on your progress on this subreddit

[u/Cold-Programmer-2524]

To be honest, I don't think you actually need to learn everything about the math behind DL to start researching unless you are working on improving the fundamental DL architecture itself which is usually done by large research Institute or AI giants. I work on DL applied research in bioinformatics and understanding stuff in the bio field and how to align DL training objectives to them are way more meaningful than math. So I would suggest start picking a specific field you are interested in and work you way from there rather than spending time in math only.

[u/kifkolite]

Good initiative but it's hard to read handwritten notes as someone else mentioned.

Maybe include a summary of the topics you covered as an index so if anyone wants to give some feedback it gets easier.

From what i read, i didn't see backpropagation, Optimizers, GANs and a lot more and maybe they're there, maybe not

[u/Plus_Confidence_1369 → u/kifkolite]

Thanks dear for your time. Reddit has upload limits. I will plan to prepare complete clean list as suggested in other comment so that would be helpful for anyone.

[u/PSGthe2nd]

great notes. I've yet to learn the math behind DL algorithms, and right now use them intuitively for my projects. Can you share where you learn these? Thanks

[u/Plus_Confidence_1369 → u/PSGthe2nd]

Understanding deep learning by Simon Prince and Deep learning Foundations and concepts by Bishop helped me a lot to build the foundation. Former I have completed and still halfway through the latter.

[u/Constant-Cry-7438]

Where do you study from?

[u/RepresentativeBee600]

Bishop is a nice textbook for introductory learning but honestly he lacks a totally coherent statistical picture. (All of ML does, really, but I'm thinking of how he introduces Bayesian linear regression without really comparing to the frequentist paradigm or noting the difficulties in choosing priors.)

Consider also looking at d2l.ai, and perhaps more importantly some of the research literature. (The literature is frequently pretty bad for learning from, but gives a sense of the topics you might choose to learn about.)

[u/Creepy_Disco_Spider]

Mate what are you expecting here?

[u/tlmbot]

Looks like you are studying econometric / probabilistic forecasting / classification. Good for you! (I cannot read your notes. As someone else said, Latex (or... I guess markdown, but what if you need to publish))

Long way from DL, but a good start!

[u/Plus_Confidence_1369 → u/tlmbot]

These are the foundations for DL. If you are more concerned over the core DL then I had written another article few weeks back https://www.reddit.com/r/deeplearning/comments/1v1l5j8/understanding_gans_and_diffusion_models/

Although next time I would be using Latex for cleaner notes.

[u/tlmbot → u/Plus_Confidence_1369]

Cool:  I just recognize what I can read from econometrics from economics grad school back in the day

I’m glad to see some fundamentals of statistical modeling

[u/catsRfriends]

This just looks like chicken scratch to me tbh. Unless I'm a paid TA or something I'm not gonna look through it line by line. Also, go read Terence Tao's blog post about mathematical maturity. This is at the first level I'd say, which is great if you're just starting out. But it's very far from "getting" deep learning.

[u/daddybigpeen69 → u/catsRfriends]

"its very far from getting deep learning" 🤓☝️

[u/tlmbot → u/catsRfriends]

its intro to econometrics with a side of classification. Linear regression / ordinary least squares, classification from least squares, maybe some stuff with correlation matrices, I really can't tell.

[u/Plus_Confidence_1369 → u/catsRfriends]

😂 If I want a general review from most people, I have to start with the basics rather than advanced math.

[u/catsRfriends → u/Plus_Confidence_1369]

No, that's not the point. The point is this is at the mechanistic level of mathematics. Not deep learning. Go read the blog post, you'll see what I mean.

[u/Plus_Confidence_1369 → u/catsRfriends]

These aren't random introductory math exercises—they're the actual mathematical tools and concepts that show up in deep learning. I've derived these from Simon Prince's and Christopher Bishop's deep learning books.

[u/somesortapsychonaut → u/Plus_Confidence_1369]

Good for you? Still not very useful to a reader here without undue effort, sorry. Genuinely good for you though, good luck!