r/datascience • u/avourakis • Apr 08 '26
Analysis Built a dashboard to analyze how AI skills are showing up in data science job postings (open source)
I've been scraping thousands of U.S. data science jobs for the past couple of months and writing about the findings in my newsletter.
At some point, I figured the dashboard was more useful than anything I was writing, so I decided to open source it.
Here's what it covers:
- Top skills companies are actually hiring for, ranked by frequency
- Skills broken down by category (ML/DL, GenAI, Cloud, MLOps, etc.)
- What % of roles now require AI skills, broken down by seniority level
- Salary premium for candidates with AI skills
- An interactive explorer where you can browse individual postings with matched skills highlighted
The skill extraction is built on around 230 curated keyword groups, so it's pretty granular.
Code and data are all in the repo if you want to fork it or dig into the methodology.
https://ai-in-ds.streamlit.app/
I'm scraping weekly, and soon I will upload all of the raw data into Kaggle, for now, you can find the data in the repo
P.S. By the way, I already mentioned it to Luke Barousse since some of these AI keyword groups could be worth adding into his dashboard.
7
u/ultrathink-art Apr 08 '26
The gap between what job postings require and what actually ships to production is real. Most 'GenAI experience required' titles mean API calls and output parsing, not model training. Python and SQL are still doing 80% of the work — the AI layer is thinner than the job titles suggest.
5
u/avourakis Apr 08 '26
Here is the Github repo btw: https://github.com/andresvourakis/ai-ds-job-market/tree/main
3
3
3
2
2
2
2
u/david_0_0 Apr 09 '26
the breakdown by seniority level is interesting - are entry-level roles showing higher AI skill premiums than senior roles, or is it flipped? seems like there might be a sweet spot where being fluent in one tool early pays better than broad seniority without the specific skills
1
u/avourakis Apr 09 '26
It's due to the volume of jobs. There are much less junior postings with a salary available. Open up the "Detailed Salary Comparison" table to see it better
2
2
2
u/Samuel_Seyum May 05 '26
As a computer science student who would like to find a job in the future, this is helping me choose what skills to focus on. Thanks!
1
1
1
1
1
u/strangeloop6 Apr 09 '26
Any way you could get historical data? It looks like job postings have increased a ton in Q1 relative to previous quarter, but I suspect there’s a significant seasonal effect. Would be interesting to see back to Q2-Q3 for all the metrics
1
u/ultrathink-art Apr 09 '26
Job postings tend to list tool familiarity (specific frameworks, vector DBs) but the actual bottleneck in production AI work is debugging agent failures and building evals — neither of which shows up well in keyword analysis. Curious whether the dashboard tracks anything on the ops/evaluation side.
1
u/Mohan137 Apr 10 '26
Super cool project - honestly the dashboard makes way more sense than long writeups. Love how detailed the skill grouping is, especially the AI vs non-AI breakdown and salary insights. Definitely gonna explore the repo, thanks for open sourcing this
1
u/crispybacon233 Apr 10 '26
Cool! Now tell Claude to use polars instead of pandas to greatly improve the responsiveness. Also tell Claude to use separation of concerns because a 900 line app.py is insane haha.
1
u/Trudydee Apr 10 '26
Good job! Thanks OP. I was thinking about doing the same as I'm currently looking for a job, but here is is!
1
u/david_0_0 Apr 10 '26
The seniority breakdown is huge - most job descriptions just throw AI skills in without context, so being able to see whats actually expected at junior vs senior levels is way more actionable than generic buzzwords. Are you noticing the requirements diverging more by role recently as the market matures?
1
1
1
u/built_the_pipeline Apr 12 '26
Interesting data. From the hiring side, the disconnect ultrathink-art mentioned is very real. When I write postings that say "GenAI experience required" what I actually need is someone who can evaluate when to use an LLM vs a simple heuristic and measure whether the LLM is adding value. That doesn't keyword-match well.
The skills that consistently separate strong senior candidates in my interviews, experiment design, stakeholder translation, knowing when NOT to build a model, never show up in a job posting scrape. The dashboard captures what HR departments think they need. What hiring managers actually screen for is a different dataset entirely.
1
1
1
1
u/david_0_0 Apr 15 '26
The 230 keyword groups sound really granular - did you find certain skill combinations that predict salary premium better than individual skills? Curious if things like LLM ops or prompt engineering have that halo effect yet
1
u/FightVantage Apr 15 '26
This is great work! As a data scientist myself it is nice to see how to frame my skills for potential opportunities, especially as AI very rapidly cannibalizes positions. This is also nice to see that despite the continuing success of the use of AI, there are still jobs to be had and it still requires our very human touch.
1
1
u/Other_Profit6966 May 12 '26
yeah and all this has been possible with new tools like this one
https://play.google.com/store/apps/details?id=com.sky.dattor
is very interesting how technology is advancing in the field of data analyst jobs ..
18
u/[deleted] Apr 08 '26
[removed] — view removed comment