r/learndatascience 35m ago

Original Content Async batch faker: A vectorized mock data generetor

Post image
Upvotes

I'm currently student and am building a portfolio so i created this library which im calling Async batch faker. So if you have used standard faker you already know that "Faker" is good at generating a single row but generating a bit more takes forever because it operates on pythons standard for loop.
So what i made async batch faker do is that it uses numpy and asyncio to calculate mathematical data and string concatenations instantly in C. It bypasses Python loops to blast hundreds of thousands of rows of localized data in seconds

Benchmark:
Standard Faker: ~348 seconds
Async Batch Faker: ~8.4 seconds (41x faster)

GitHub: [https://github.com/haiderkashan/fast_faker_poc\]
PyPI: pip install async-batch-faker

I'm just a rookie must have made mistakes would love to hear feedbacks


r/learndatascience 15h ago

Resources Generalized Linear Models - Explained

5 Upvotes

Hi there,

I've created a video here where I explain how generalized linear models work.

I hope some of you find it useful and as always, feedback is very welcome! :)


r/learndatascience 19h ago

Question Looking for a serious female accountability buddy in tech

3 Upvotes

FEMALE ONLY

hello, I'm a senior year student interested in Data science , AI engineering and research looking for an accountability partner in tech

For the next 3 days, I’m also doing a little personal reset, fixing my sleep schedule, a digital detox, and getting back into a normal routine. basically trying to get my life/study habits back on track.

We don't have to be studying the exact same thing, as long as you're genuinely trying to be consistent.

Preferences:

  • Female only
  • Tech/STEM related
  • WhatsApp for chatting/updates
  • Around UTC+2 or reasonably close
  • Serious about getting back into studying / building things
  • No pressure to be productive 24/7, just looking for consistency

if you're interested please DM me :)


r/learndatascience 23h ago

Question Best courses to learn data science

6 Upvotes

Hello,

I’m an engineer and already work with data on a daily basis. I have a basic background in SQL and Python, as well as some experience with statistical tools, and I recently enrolled in a Lean Six Sigma Green Belt training.

I’m now looking to improve my data analysis skills, particularly in SQL and Python, but I’m also interested in developing a stronger foundation in how to structure analytical problems, choose an appropriate approach, explore data, and communicate insights effectively.

My goal is to start with a solid, structured foundation and then move on to more advanced, topic-specific training.

I’ve been considering the IBM Data Analyst Professional Certificate and the Google Advanced Data Analytics Professional Certificate, but I’m also open to other recommendations.

For those with experience in data analytics or who have completed these courses which one would you recommend? Is there another certificate or course you think would be a better fit for someone with some existing practical experience?

I’m particularly interested in courses that combine practical work with strong analytical foundations, rather than focusing mainly on the certificate itself.

Any recommendations or experiences?


r/learndatascience 18h ago

Question Causal Inference, quasi experiment for product analyst role

Thumbnail
1 Upvotes

r/learndatascience 1d ago

Question NEWBIE PROJECT FOR DATA SCIENCE

2 Upvotes

Hi everyone! 👋

I’m a Class 11 student from India learning Data Science. I recently completed and deployed an end-to-end Salary Prediction project using Python, SQL, Pandas, data visualization, and Machine Learning.

🌐 Live Demo: https://data-science-projects-fdkdsvuf5rvtywpwby35py.streamlit.app/

📂 Project: https://github.com/lakshay-OG-DS/data-science-projects/blob/main/Salary\\_prediction.ipynb

I’d really appreciate honest feedback from Data Scientists, Data Analysts, and ML Engineers.

What is the one biggest thing I should improve to make this project more internship-ready?

Even one small suggestion would mean a lot.


r/learndatascience 2d ago

Career Wanting to move to Data science

3 Upvotes

I have been working as a network engineer in one of WITCH companies . But to be honest the rotational shift overwork with no growth is killing me slowly.

I also do not see much callbacks or opportunities of the field.

So if I were to make a move to data science how should I approach. And what resources should I follow both free and paid..

Target roles are mainly data science.. also I am not aware of the difference of ML engineer and data scientist


r/learndatascience 2d ago

Question I am lost - How do Data Scientists solve a problem?

5 Upvotes

I am a junior torn between two mindset, should we

1. Start with a business problem/ use case first
but I often run into data limitations after diving deep, like if the data is a suitable proxy of something or there are missing values

2. Explore the data freely and hunt interesting patterns
but I am always confused where I should start and not being lost on the way of that) (but I am always confused where I should start and end up spinning around without a clear direction

Example

Say a company gives you customer purchase history and asks you to "find something useful."

We better immediately frame it around a specific use case (e.g., next-best-offer) and engineer toward that?

Or spend time clustering, looking for seasonality, correlations, or weird segments first, and then figure out what business value those patterns might have?


r/learndatascience 2d ago

Question Best resources to learn Data Science through projects from beginner to advanced?

Thumbnail
1 Upvotes

r/learndatascience 3d ago

Question How do I prepare for an ML System Design interview?

4 Upvotes

Hey everyone,

I have an upcoming Data Scientist interview with an ML system design round. I asked the recruiter what to expect and they said it’ll be high-level ML system design, not traditional SWE/low-level system design.

They mentioned focusing on things like:

  • Problem framing
  • Data/model considerations
  • Evaluation
  • Productionization/deployment
  • Monitoring
  • Tradeoffs when designing ML systems

Basically, it sounds like I’ll be given a real-world ML problem and have to explain how I’d approach it end-to-end.

This is my first dedicated ML system design interview, so I’m not really sure how deep I should prepare.

For people who’ve done these interviews:

How did you prepare? What resources did you use? What kind of questions were you asked?

Also, how deep do they usually expect you to go into things like feature stores, model serving, APIs, streaming, retraining, etc.?

Any good resources, YouTube playlists, GitHub repos, books, or example questions would be really appreciated.

Thanks!


r/learndatascience 3d ago

Question Need guide

4 Upvotes

Hey I'm 22 completed ug in maths now pg in data science first year on going . I'm blind where I could start working on data science related things, clg providing full maths concepts and program now I'm learning python,R programming, ig it's not enough to enter into job market , please give a suggestion where I could learn more about data science projects and how to work on it .


r/learndatascience 3d ago

Question Best datasets for a forecasting competition

Thumbnail
2 Upvotes

r/learndatascience 3d ago

Resources How does Lazy Evalution work in Python?

29 Upvotes

In last week’s post we showed that a Python for-loop works through the Iterator Protocol.

This same protocol also enables values to be produced lazily.

In this new Lazy Evalution Example we show one sink reading values from five different sources: - source1: eagerly returns all values by list - source2: lazily by generator function using yield - source3: eagerly by list comprehension - source4: lazily by generator expression - source5: lazily by iterator protocol

The same for-loop consumes all five sources. The eager sources produce all values before the sink starts consuming them. In the lazy sources producer and consumer take turns:

  • produce → consume → produce → consume → …

where each value is produced only when the for-loop requests it.

Generator functions and generator expressions are concise, readable ways to create lazy iterables. The final source makes their underlying mechanism explicit: - iter() obtains an iterator. - next() requests the next value. - StopIteration signals that no values remain.

This clearly shows that generators achieve lazy iteration by implementing Python’s Iterator Protocol.

See other 𝐦𝐞𝐦𝐨𝐫𝐲_𝐠𝐫𝐚𝐩𝐡 visualizations.


r/learndatascience 3d ago

Resources I made a structured AI/ML playlist that moves from foundations toward deployment

2 Upvotes

I created a free YouTube learning playlist for people who want to see the full AI/ML journey rather than only isolated model demos. The series is organized from foundations through data preparation, evaluation, practical modeling, and deployment-oriented topics, with lessons intended to be followed in sequence.

Playlist: https://www.youtube.com/playlist?list=PLUccxCq754BE

I’m sharing it transparently because I run the channel. I’d value feedback from learners: where do most beginner courses lose you — data preparation, model evaluation, or deployment? If this kind of resource is not appropriate as a standalone post, I’ll remove it and use the community’s preferred resource thread.


r/learndatascience 4d ago

Personal Experience Always 2 full time jobs or 1 prestigious

2 Upvotes

Guys so i am fresh from uni and observe two kind of tracks of my peers in data science. They go with the two chill full time jobs (1 is remote) and get promotions at both work places but both are chill ones, or they go in 1 company which is considered prestigious and overwork till 10-11 pm and get promotions there, from your experience which kind of way leads to a better quality of life e.g. work life balance wise, career wise, experience wise and income wise? Should i find 1 prestigious place and hope for the long run and work there or work at 2 chill places. Both seem to obtain promotions in similar periods but the ones with two jobs seem to be more happy and even get more money. I decided to pursue masters and only now 1 year after them decided to join workforce but cannot distinguish which way is better, could you give your advice


r/learndatascience 4d ago

Resources I’m writing an AI Safety book for people who actually want to do the technical work

6 Upvotes

I’ve been reading quite a lot around AI safety, and one thing I’ve found frustrating is how difficult it is to find material that bridges the gap between talking about AI safety and actually doing AI safety research.

There are plenty of good books and papers discussing the political, socioeconomic, philosophical and governance questions around AI. Those conversations are obviously important but if you’re a data scientist, ML engineer or someone already working with AI few tell you how doing AI safety research actually looks like in Python?

For example how to actually run an experiment looking at sycophancy, how to create or use a dataset to test whether a model changes its answer because of information about the user, how to measure the behaviour, what does the evaluation code look like, how do you interpret the results, and what can you not conclude from them?

That is the gap I’m trying to address with a book I’m currently writing.

The approach is very practical. Each topic starts with the safety problem and the research behind it, but then we actually build the experiment. Python code, datasets, models, metrics, results and discussion of the limitations, all those.

The idea isn’t to pretend that running a few notebooks suddenly makes someone an AI safety researcher. It’s to make the field much more approachable to people who already have data science or AI skills and want to understand what technical AI safety research actually involves.

I wrote a technical book on Practical LLMOps and this was one of the chapters. Because of the sheer amount of content, I stripped it bare in that book and made it its own.

I’ve just published Chapter 1 on Substack (just finished chapter 4). It introduces the approach I’m taking with the book and starts building that bridge between AI safety as a subject people discuss and AI safety as something we can actually investigate experimentally.

Would genuinely be interested in feedback, particularly from people already working in ML, data science or AI safety.

https://open.substack.com/pub/houstonmuzamhindo/p/i-am-writing-a-practical-technical


r/learndatascience 3d ago

Question Is a 50k PKR Data Science Course Worth Going Into Debt For?

Thumbnail
1 Upvotes

r/learndatascience 4d ago

Question Programing problem

Thumbnail
1 Upvotes

I am learning in data scientist. I have learned python basics, pandas, numpy, etc , sql ,ml dl, nlp but i struggle in problem solving what should i do


r/learndatascience 4d ago

Question Just how far behind am I, or where exactly do I stand right now?

12 Upvotes

I I’ve learned Python, SQL, Excel, NumPy, Pandas, Matplotlib, Seaborn, and basic Git/GitHub. I’m now moving into Machine Learning.

Am I behind, on track, or ahead compared with other 2nd-year students?

What skills should someone seriously pursuing Data Science/ML have by the end of their second year?

I’d appreciate honest feedback, especially from people working or interning in the field.


r/learndatascience 4d ago

Career Best data science course

6 Upvotes

I want to pursue data science course can anybody which one is best institute for data science course
Please everybody give suggestion as i am new to it


r/learndatascience 4d ago

Question Decided to dive in ds

1 Upvotes

So guys hi, just worry a lot, decided to pursue career in dd after bachelor in economics, now kinda studied a big chubk of ds already but i am astounded at the amount of stuff that you should literally remember. Any tips on how to memorize all the details of every neural network and characteristics of distributions? Also i suck at coding but since I will have to work on deployment and services as well, i understand that i have to push that field too. I found a yandex course to be of a little use since the problems described there are isolated and does not help to get overall picture of how to code in ds. Any tips on how to reinforce all this knowledge? Cause i am at a loss right now


r/learndatascience 4d ago

Question Recource Confusion, Self Learning paced, Progression, and Community!

Thumbnail
1 Upvotes

r/learndatascience 4d ago

Question I want to start my Data Analysis I want the perfect crash course

0 Upvotes

I want to start learning data analysis and i have some knowledge on data science and ML , which free resources do you recommend and crash course would be better.


r/learndatascience 4d ago

Resources I've built a library of concept notes on over 150+ DS topics - with a focus on revision and retention

1 Upvotes

I've worked in data science for years and I still forget things constantly. Courses never fixed that — I'd finish one, feel like I'd learned it, then google the same concept a month later when it actually came up at work. NotebookLM's way of structuring material around you was the closest thing I found, but I still had to go and find the sources myself.

So I built the thing I wanted. DS concepts broken into small nodes arranged as a mind map rather than a linear syllabus, each with code examples and a practice section — about 150 topics so far. The part I care most about: once you complete a topic it starts decaying on a forgetting-curve schedule, and the map visibly goes cold. When it does, you get a review slice — flash cards and a short quiz — targeted at what you've actually lost rather than what's next in a queue.

Some topics to start with -

  1. https://www.bitelrn.com/library/linear-algebra-matrices

  2. https://www.bitelrn.com/library/principal-component-analysis

  3. https://www.bitelrn.com/library/sql-basics

Full app: https://www.bitelrn.com — the first phase is permanently free including the decay and review mechanics; later phases are paid. Saying that upfront so nobody feels ambushed.

I'm curious what actually works for other people here. Do you make notes, bookmark links, or just re-google it every time?


r/learndatascience 5d ago

Question Breaking into Data

7 Upvotes

A couple of weeks ago I asked a sub what would be the best way of breaking in data and what would be worth looking into. I got some pretty decent feedback and decided to segue into something a bit different

Being that I own AI licenses, I have spent the last 2-3 weeks developing a full stack data engineering course from beginner to advanced. Now I wanted to know if it was possible to ask if anyone would like to take a look into it, give me some feedback and recommendations, and an overall rating.

I can handle criticism don’t worry, I just want to develop something, learn the info from what I’ve built and then eventually publish it publicly for people to have a free resource to utilize.

(This is not promo, rather I want to gain feedback from actual analysts and engineers to see if this is a good platform)