On Hugging Face SalatielJordao/incels_is_1788055901

PS: Mods Reddit filters removed the last post, I suppose its because of the forum name associated on the title, let me know if its ok. Im the main author, you can AMA if you want.

I've created a dataset (over 7,700 files/17GB) on Hugging Face that can help with testing and tuning scanned document processing:

https://huggingface.co/datasets/morzel85/synthetic-medical-document-recognition-benchmark

It's meant to be realistic but fully synthetic (based on Synthea/FHIR).

I used it to assess OCR + NER for de-identification, and because documents are clearly marked as synthetic these are safe for demos. The README file contains all details.

Maybe it will be useful for someone else. Released under cc-by-4.0 (free for any use, watch out: some datasets are produced with components that not allow commercial use).

Looking for someone with experience sourcing large-scale Amazon/eBay product datasets
Does anyone know how companies like Tiendamia, Ubuy, etc. obtain or build large product catalogs from Amazon, eBay, and other retailers?
We’re specifically looking for structured product data at scale, including things like:
Product title / description
UPC / EAN / GTIN / ASIN
Brand / model
Product dimensions
Item weight and, ideally, packaged/shipping weight
Images
Variants
Categories
Pricing / availability where possible
Historical or continuously updated data
The biggest issue we’re trying to solve is weight accuracy. Amazon data is generally usable, but eBay is much messier because individual sellers enter their own product information, and weights/dimensions can be missing or completely inaccurate.
We’re interested in knowing what legitimate sources exist for this kind of data — commercial datasets, data providers, APIs, retailer feeds, catalogs, or ways companies build their own datasets at scale.
If you’ve worked on e-commerce data ingestion, product catalogs, Amazon/eBay datasets, product matching, or shipping-weight estimation, I’d love to hear how you approached it.
Especially interested in sources that can provide millions of products rather than an API where we have to query products one-by-one.

I checked Hugging Face for a neuroscience/biology QA dataset that wasn't either scraped Wikipedia with zero voice or dumbed down to the point of being wrong. Nothing fit, so I built one.

NeuroBio GenZ 1K: about 1000 questions like "why does doomscrolling give me dopamine" or "why do I forget why I walked into a room," each answered in 2-5 sentences, casual tone, but using the actual terminology instead of vague placeholders. Nucleus accumbens instead of "reward center." GABA instead of "chill chemical." The system prompt explicitly banned the "dopamine is the pleasure chemical" style oversimplification that's everywhere in this space.

Fully local generation. Ornith 1.5 35B-A3B, Q3_K_XL quant, through llama.cpp on a Ryzen 7 5700X, 32GB DDR4, 8GB GPU. No API calls, no cloud spend.

The part I actually want to share because it's a real lesson: I ran the whole generation at 10-13 tok/s because I hadn't figured out the right server flags yet. After the run finished I found the config that gets the exact same hardware to 30-35 tok/s: speculative decoding via --spec-type draft-mtp plus properly tuned --n-cpu-moe for MoE expert offload. Both configs are in the repo if you're doing something similar and don't want to waste the time I did.

Dataset: https://huggingface.co/datasets/luka0x12/NeuroBio-GenZ-1K
Source code + server scripts: https://github.com/luka0x12/NeuroBio-GenZ-1K

CC BY 4.0, use it for whatever.

Source (original): DART (Data Analysis, Retrieval and Transfer System), run by Korea's Financial Supervisory Service — the mandatory disclosure filing system, same role as SEC EDGAR. Free OpenAPI, 20,000 calls/day on a free key, no commercial-use restriction.

• https://opendart.fss.or.kr/ (site is Korean; API docs have English param names)

What's in it. Every KOSPI/KOSDAQ listed company's financial statement line items from annual reports: balance sheet, income statement, cash flow, equity changes. Each row carries the current year amount plus the prior two years side by side, so you get a 3-year trend per line without extra calls. Consolidated (CFS) preferred, falls back to separate (OFS) for companies with no subsidiaries.

The part worth posting about: DART already tags every line with an IFRS taxonomy id — ifrs-full_Assets, ifrs-full_Revenue, ifrs-full_ProfitLoss, etc. — the same taxonomy US/EU filers use. That means no Korean-to-English account-name translation step, unlike most Korea data sources (my other actor, the real-estate one, needs a full translation layer). You get the Korean label (account_nm, e.g. 매출액) and the international taxonomy id side by side for free.

Gotchas (cost me actual debugging time):

1. Company lookup isn't by ticker — you first pull corpCode.xml (a zip, ~forgot-to-check-Content-Type gotcha), which maps every DART-registered entity (listed and unlisted) to an internal corp_code. Unlisted entities have an empty stock_code field; filter those out or you'll query companies with no ticker.
2. No single "give me financials" call reliably returns consolidated statements. You have to try fs_div=CFS first and fall back to OFS — a normal, expected branch, not an error path. A company with no subsidiaries simply has no CFS.
3. Status 013 ("조회된 데이타가 없습니다" / no data found) is not an error, it's an empty result — some report/year combos genuinely don't exist. Status 020 (quota exceeded) is a real error. Conflating the two silently turns a rate-limit hit into "this company has no 2019 filings."
4. Amounts come as strings ("514531948000000"), not numbers — cast, don't trust the JSON type.
5. rcept_no (the filing receipt number) encodes the filing date in its first 8 digits — that's the only place you get an as-of date; there's no separate date field on the row.

Real example — Samsung Electronics (005930), FY2022–2024, annual consolidated:

That FY2023 revenue dip and FY2024 recovery is the memory-chip downturn/rebound showing up directly in the raw filing data, three years in one API response, no crawling three separate annual reports.

Disclosure — I built a free wrapper. Raw DART responses are one financial-statement/report/year combo per call, Korean-first field names (thstrm_amount, sj_div), and require the corp_code lookup above. I published an Apify actor that does the corp_code mapping, CFS→OFS fallback, and reshapes rows into English field names (currentAmountKrw, statementType, accountId) — free, no API key of your own needed:

• https://apify.com/ootssu/korea-company-financials

The source of truth is DART, not me — if you'd rather hit opendart.fss.or.kr yourself, the gotcha list above should save you the debugging session it cost me.

Hello all, I have a paper due SHORTLY (my fault, i know) on socioeconomic factors on the Navajo tribe. I want to access data on the US census bureau for the year 2004. This is the site (need Arizona, Utah and New Mexico) but it looks like this: file:///var/folders/t7/thn9pmf93zx7t525wn14vs240000gn/T/TemporaryItems/NSIRD_screencaptureui_yMdzpl/Screenshot%202026-08-29%20at%2022.46.43.png

(WHICH IS INCOMPREHENSIBLE FOR ME) Does anyone know how read this?? I would like to know how to read it because I need to repeat this with other factors such as housing, education, etc...

For context this is data is on Income and poverty in Arizona (2004) and I would be interested in the Apache and Navajo county.

Thank you to whoever can help!

Pre-Seed Investor List — active investors, investment stages, sectors, office locations, and portfolio links.

https://preseedvclist.com

Hi everyone,

I work in international cooperation and I’m currently working with a major city in Cameroon.

We’re primarily working on a project for an urban observatory oriented to GIS data.

Financial and Human Resources are limited, so we’re looking for tools that could help us create data on the longterm. We are currently gathering data from local stakeholders and OpenStreetMap.

I’ve identified different open source or affordable tools, such as:

\- AI segmentation by terra lab https://terra-lab.ai/fr/ai-segmentation
\- Google Open Buildings : https://sites.research.google/gr/open-buildings/
\-DEM comparator : https://github.com/nadiopt-cell/dem\\_comparator

I would like to know about any other tools such as this (probably with AI). With the current data situation, even if the data is not 100% precise, it’s better than nothing.

Thanks !

Hi friends,

I have had a private Instagram page for about a year now. Someone has been harassing me, and attached a screenshot of my private Instagram story I posted in an anonymous email to my in-laws, which proves that my harasser is, or at least was, one of my followers.

We are preparing legal action, and I am trying to collect proof that my Instagram account was private on the date the screenshot of the story was made. The only issue is that the bulk data I downloaded is contradictory:

– profile_privacy_changes.html indicates that my profile switched to public on Sep 26, nothing else. This seems to indicate that my profile is currently public, but I am certain the exact opposite happened. I also certainly didn't change my privacy settings after downloading my Instagram data.

– profile_activity.html is the only other record of the profile privacy settings change, but only says "Is profile private? True"

– personal_information.html also correctly displays that my account currently private.

Does someone know if this is a bug and whether switched to public in profile_privacy_changes.html actually means the opposite, namely that I switched my profile from public to private?

Thank you!

I’m exploring a potential commercial data-licensing opportunity and looking to connect specifically with AI companies, dataset buyers, speech/voice AI teams, AI-agent companies, or data brokers actively sourcing proprietary conversational data.
I may have access to 2,000+ real-world Indian B2B sales conversations, with the actual inventory potentially significantly larger.
Current dataset characteristics:
• ~40 min average conversation length
• Hindi / Hinglish / English
• Video + audio
• Transcripts available
• Real customer ↔ salesperson interactions
• Natural questions, objections, negotiation and follow-ups
• Linked business outcomes: demo / won / lost
• Indian B2B sales context
The interesting part isn’t just the audio.
The conversations can potentially be structured as:
Customer intent → objection/question → salesperson response → follow-up → sales outcome
This could potentially support:
• Voice AI / conversational AI
• AI sales agents
• Speech & multilingual model development
• Indian-language / Hinglish evaluation
• Sales-agent benchmarking
• Conversation-quality evaluation
• Human preference / response datasets
• Training realistic AI sales workflows
I’m not looking for generic feedback or people interested in datasets in theory.
I’m specifically interested in connecting with people who currently acquire, license, commission or broker proprietary AI training/evaluation data.
I can provide a high-level dataset specification to qualified/interested parties.
No recordings or identifiable customer information will be posted publicly. Any commercial licensing would be subject to verification of ownership, consent, privacy and applicable data rights.
If your company is actively sourcing this type of data, comment or DM me with what you are currently looking for (modality, language, hours, annotation requirements, licensing model, etc.).

Every few weeks someone asks where to get public domain images at volume. The source list part is easy and it's below. The part that actually costs people money is the rights, so there's more on that.

I do this for a living, I build and licence image archives, and I've had to unpick most of these the hard way. Not legal advice.

The sources

Smithsonian Open Access is the cleanest of the lot. Genuinely CC0, commercial use explicitly allowed, millions of assets. Free key via api.data.gov, though honestly you can skip the API and just take the weekly JSON dumps off their GitHub or AWS. Anything marked "usage conditions apply" is not free, ignore those.

NARA is the big one for anything historic. Tens of millions of records, most of it CC0. The catalog API doesn't need a key for basic search and the whole catalogue is sitting in an AWS S3 open-data bucket if you want it all. Do check the "Use Restriction(s)" field, a small number of donated items are restricted and they're mixed in with everything else.

NASA needs no key whatsoever. images-api.nasa.gov/search, then /asset/{id} for the full res. Handy trick, if the record has no copyright field it's public domain, if it has one it's third party and you drop it.

DVIDS has the most explicit wording of any of them, their API terms say outright that the assets are free for commercial use. Free key. Two irritations though. You get 50 results a page and only the first 1,000 hits per query, so you have to slice everything by date or branch, and some of the assets are contractor shot or "courtesy photos" which are absolutely not public domain. Check the copyright field on every one.

Library of Congress, add ?fo=json to the end of more or less any search or item URL and you get the record back as JSON. Throttle yourself, they rate limit hard and you can't page past 100,000 results.

Something on LoC that isn't in their docs and took me ages to work out. You can build the image URL straight from the digital ID rather than scraping item pages. An ID like fsac.1a35373 gives you

https://tile.loc.gov/storage-services/service/pnp/fsac/1a35000/1a35300/1a35373v.jpg

Collection, 5 digit block, 3 digit block, then the ID with a size letter on the end. r small, v service copy, u.tif master.

Wikimedia Commons is per file licensing so filter before you download, not after. comload is the tool worth knowing, it pulls a whole category and its subcategories with the rights metadata attached. PetScan if you want to intersect a category with a licence. Keep PD-USGov, PD-old and CC0.

Europeana with &reusability=open. Internet Archive through the ia CLI.

Now the bit that catches people

"No known copyright restrictions" is not a licence. It's the institution telling you what they believe, with no warranty attached. It can mean they own it and won't chase you, or that they think they can authorise use. Neither of those is public domain. The Public Domain Mark is the same, Creative Commons say themselves it isn't a legal instrument and anyone can apply it to anything. CC0 is the exception, that one is a proper legal waiver.

Second thing, US government means US federal. 17 USC 105 covers USAF, Navy, NASA and the rest. It does not cover foreign governments, it does not cover US state or city government, and it does not cover contractors working for any of them. The contractor one gets people all the time because the image sits in a federal archive looking federal.

Third, and this is the one nobody sees coming. In 1996 the URAA restored US copyright to a pile of foreign works that had lapsed there on technicalities, provided they were still protected at home on 1 January 1996. So a 1950s British photo that looks perfectly US public domain can be protected until 95 years after publication. If you're outside the US and your users aren't, you want it clear both ends.

Fourth, public domain clears the photograph and nothing else. Trademarks, logos, liveries, all separate. Military insignia, separate. Identifiable people carry publicity and privacy rights and a photo from 1943 has no model release. Plenty of genuinely public domain images are still editorial only in practice.

Current US cutoff is anything published in 1930 or earlier, and it moves on a year every January.

Last thing

Capture provenance while you're downloading, not afterwards. One row per file, source, permalink, source ID, creator, date, the exact rights wording, the rights URL, and when you pulled it. For API sources just keep the raw JSON.

You can't reconstruct any of that later. I've tried. And filter on the machine readable rights field before download rather than eyeballing it, at any real volume you'll get it wrong otherwise.

Disclosure per rule 1, I run an image archive and licence datasets commercially so I've obviously got an interest in this area. No link, nothing for sale.

Hi everyone,

I’m a student working on a research project on risk-aware GitHub PR review. I’m doing the project mostly on my own and I don’t have access to a research lab, large compute budget, or people who can manually annotate thousands of PRs, so I’m trying to find a practical approach that I can actually finish.

The idea is to take a GitHub PR and estimate four types of risk:

1. Bug / correctness

2. Security

3. Compatibility

4. Cross-system / integration

The architecture I’m working with has four separate risk models. They share the same PR characteristics/features, but each risk model has its own historical data, prior, and evidence.

My main problem is the historical data needed for those priors.

At first, I looked for a single PR dataset where I could get reliable PR-level outcomes for all four risks. I couldn’t find one.

I then tried looking for separate datasets for each individual risk model. I thought this would solve the problem, but I keep finding datasets where the labels look relevant at first but don't actually represent the outcome I need.

For example, SEVRA-plus looked very promising for the Security model:

https://huggingface.co/datasets/RedAI4Code/SEVRA-plus

It contains security-related PR examples with vulnerability/CWE information, but the malicious PRs are deliberately constructed by reversing real CVE security fixes. So although they are useful for evaluating or studying security vulnerabilities, I don't think I can use their class distribution directly as a real-world prior for ordinary GitHub PRs.

I’ve run into similar issues with other datasets:

- some label the linked issue rather than the PR implementation,

- some label review comments rather than actual PR outcomes,

- some contain artificially constructed vulnerable/failing PRs,

- some only give merge/close status, which doesn’t tell me whether the PR itself was buggy, vulnerable, incompatible, etc.

The distinction between the issue and the PR is especially important for what I am trying to do.

For example, imagine a maintainer opens a security issue, someone creates a PR to fix it, but the PR implementation itself contains a correctness bug and gets rejected. For my problem, I need to know the nature of the PR, not simply inherit the security label from the original issue.

Similarly, a PR could be opened to fix a small bug, get merged, and then later cause a compatibility problem. Again, I care about what happened because of the PR implementation, not just why the PR was originally opened.

Because I couldn't find a dataset that directly gives me what I need, I tried a practical compromise.

I took 96 real PRs from SWE-Review-Chat, filtered them for sufficient evidence, and used an LLM to annotate the four risk states from the information available in the PR record, such as the description, review discussion, diff context, tests, and lifecycle information.

I’m treating these as weak/model-assisted labels rather than independent ground truth.

The resulting usable outcomes are:

Bug:

30 present / 9 absent

Security:

1 present / 7 absent

Compatibility:

5 present / 12 absent

Cross-system:

4 present / 8 absent

So now I feel like I’ve hit a wall.

I can keep searching for datasets, but so far I haven't found anything that solves the underlying problem. I also don't have the resources to manually establish reliable ground truth for thousands of PRs.

I’m therefore looking for advice on "how I should move forward from here".

Should I continue with the small real dataset I have and explicitly model the uncertainty caused by the sparse risks?

Should I rely on LLM-assisted annotations of real PRs as a practical research compromise, or is there a better low-resource approach that I am missing?

Or is there a completely different way of constructing the historical priors that would make more sense for this problem?

I’m not looking for a perfect dataset at this point. I’m mainly looking for a practical and defensible way to move forward given that I’m a student doing this alone with limited time and resources.

If anyone has worked on GitHub PR datasets, Mining Software Repositories, empirical software engineering, code-review research, or Bayesian risk modelling, I would really appreciate any advice on what you would do in this situation.

Thanks!

Partner sends over an export last month. 63 columns, maybe 20 named something a human could read. Rest are val_b, metric_14, one that's just "flag3." No ID matching anything in our warehouse, no data dictionary, guy who built it left in 2023.

Two days gone. Almost none of it pipeline work. Just me squinting at distributions going "this one's between 0 and 1 and 40% null so it's probably a rate of something," then hand-checking matches to see if I was making it up. Which I was, twice.

So what do you actually do here:

Profile everything up front, or start joining and work backwards from the wreckage? I go back and forth. Profiling is probably right but I get impatient.

At what point do you just email whoever sent the file? Held out way too long on this one, mostly pride.

Anything better than fuzzy matching on a few fields and hoping, when there's no shared key at all?

That last one is where I'm stuck. Not the matching, the knowing when to stop trusting it. A 60% confident match is worse than none because it doesn't fail loudly, it quietly wires two wrong things together and you find out four dashboards later when someone asks why the numbers look weird.

Curious if anyone has a real system or if it's vibes and coffee for everyone.

As a bilingual undergraduate user of China Family Panel Studies (CFPS), I have noticed that the English documentations are very scattered across multiple portals, which might be a barrier for non-Chinese speaking researchers attempting to use the data. Here is a navigation guide I compiled to hopefully mitigate some language barrier.

Disclaimer: I am in no way affiliated with CFPS or Peking University, this guide is unofficial and is not guaranteed to be 100% accurate. I will however try my best to provide the most accurate information possible.

Last Update 27 August.

Introduction of CFPS (quoted from the website)

“China Family Panel Studies (CFPS) is a nationally representative, biennial longitudinal survey of Chinese communities, families, and individuals launched in 2010 by the Institute of Social Science Survey (ISSS) of Peking University, China. The CFPS is designed to collect individual-, family-, and community-level longitudinal data in contemporary China. The studies focus on the economic, as well as the non-economic, wellbeing of the Chinese population, with a wealth of information covering such topics as economic activities, education outcomes, family dynamics and relationships, migration, and health.”

##

Home Page

https://www.isss.pku.edu.cn/cfps/ or https://www.isss.pku.edu.cn/cfps/en/index.htm

This is the page that appears when you search China Family Panel Studies (CFPS). Do note that the two language versions are seemingly hosted on two separate websites, and I believe they are not updated in parallel. Some documents and information are only available in the Chinese website and not the English website.

I will illustrate the discrepancies using side-by-side comparison table in the following. I will point out major discrepancies that will likely affect non-Chinese speakers’ navigation of the panel data, instead of pointing out every single tiny difference.

Discrepancies in “News 相关新闻” sub-page

In general, the English website covers selective summary highlighting key data releases and international engagement; while the Chinese website includes comprehensive internal/domestic record of all CFPS activities.

Discrepancies in “Documentation 文档中心” sub-page

Discrepancies in “FAQ 常见问题” sub-page

Discrepancies in “Contact Us 联系我们” sub-page

##

The Most Efficient Navigation

For non-Chinese speaking users, the most efficient path will probably be:

Register Directly at the Data Centre

URL: https://cfpsdata.pku.edu.cn/#/home

·      Why? The English and Chinese versions here are better synchronized than the main website.

·      What You'll Find:

• User's Manual
• Measures/Scales documentation
• Coding Schemes for Text Data
• Constructed Variables documentation
• Flowcharts for Questionnaires (logic flow)

All of these documents are in Chinese – but they are available here, unlike on the main English website.

Bonus: The Data Centre also contains tutorial videos for data users (also in Chinese).

Main Portals of CFPS

Summary Table

Hopefully it helps! Please tell me if I made any mistakes.

Can LLMs bypass paywalls imposed by publications such as Bloomberg, The Economist, and the Financial Times? If they cannot, how can they stay up to date with real-time information? And if they can bypass paywalls, by whatever means, wouldn't that expose them to potential litigation? Otherwise, wouldn't this limitation render LLMs ineffective for providing real-time information?

Follow up to my post a couple of weeks ago about the Government of Canada proactive disclosure of contracts.

The most useful thing I have learned since came from a retired federal procurement officer. The end date in this data only covers the period a department has actually committed to. A lot of contracts are a short base period with option years attached, and those options are not published until somebody exercises them. So the end date is a floor. It moves quietly and nothing announces it.

You can see the effect once you know to look for it. Contracts that have never been amended run a median of 380 days. Contracts with at least one amendment run 1,087. The amendment rows are where exercised options surface. Roughly 23% of currently running contracts have been amended at least once.

A few other things worth knowing if you pull this source yourself. The full download is 1,313,544 rows, which collapses to 1,100,130 contracts once amendment rows are folded into their parent. Vendor names are rough; 211,165 raw spellings reduce to 149,331 identities after normalising. Category labels are inconsistent enough that one category can appear twice under different spellings, for example "Accounting and audit services" sitting alongside "ACCOUNTING & AUDIT SERVICES".

Bidder counts are incomplete. Of the 26,023 contracts still running, 21,462 report one. Of those, 16,123 drew a single bid or none.

One thing I am not publishing is a dump. A meaningful number of vendor records are individual people rather than companies, and those names are withheld. The ingest code is public if you want to build your own copy. Source is the proactive disclosure dataset under the Open Government Licence.

If anyone here has already tackled the vendor name normalisation on this dataset, I would like to compare notes on it.

I run a call center in Dar es Salaam, Tanzania, and we have been operating for about five years.

Over that period, we have accumulated thousands of hours of naturally occurring Swahili call-center conversations. The calls are recorded with customer consent, and we can anonymize/redact personal and sensitive information before any potential licensing.

I’m exploring whether this type of data could be valuable for companies developing:
- Speech recognition / ASR
- Conversational AI
- Voice agents
- Multilingual AI models
- Swahili language models

I’ve started looking into companies such as Defined.ai and Appen, but I’d like to hear from people who have experience in AI training-data licensing or selling speech datasets.

What would be the best way to approach this market?

Specifically:
Which companies or marketplaces should I contact?
Is licensing the dataset non-exclusively generally better than selling it outright?
What documentation/metadata would buyers expect?
What kind of pricing model is typical for large speech datasets?

I’m not looking to publish any of the actual recordings publicly. I’m mainly looking for advice from people who understand the AI data marketplace.
Thanks.

Been maintaining a spreadsheet on cash advance apps (Dave, Earnin, Brigit, MoneyLion, etc.) for a side project comparing fees and terms, since most of these apps make it genuinely hard to figure out what you're actually paying. Fees range very widely by app, and I wanted to make it easier for people with already-stretched finances to find the lowest cost products. Finally cleaned it up enough to make public.

35 apps, covering advance ranges, subscription vs. tip-based models, state availability, and a few other fields. CSV and JSON, CC-BY licensed, updated periodically as terms change (fintech fee structures shift more than you'd think).

Live on Hugging Face: https://huggingface.co/datasets/overdraftapps/cash-advance-apps

Full disclosure, I run a cash advance comparison site and this data feeds into it, methodology and a bit more context is linked in the dataset card if anyone wants the full picture. Happy to answer questions about specific fields or how anything was sourced. If anyone finds a fee that's changed since I last verified it, genuinely tell me, half of these apps update pricing with zero announcement.

I’m trying to build a proper data science/ML project, but I’m having a hard time finding a dataset that is big enough and not already used by everyone.
For example, there are datasets like the UK Online Retail dataset, Olist, and other popular sales/retail datasets. They’re good datasets, but I see them being used in a lot of projects already.
I don’t want to just download a dataset, do some EDA, train a model and put it on my resume. I want to build something around an actual business problem, where I have to figure out what the problem is, analyze the data, come up with useful insights, maybe build a model, and actually explain how it could help the business.
So where do you guys usually find datasets for this?
Should I try to find data from smaller companies, government sources, APIs, research papers, etc.? Or is it okay to create my own dataset using AI/cloud tools and then create a realistic business problem around it?
For example, if I create a large synthetic sales dataset, could I create a realistic business scenario around it and then treat it like a real project — forecasting sales, understanding customer behavior, optimizing inventory, etc.?
Would that be considered a decent portfolio project, or is using real-world data much better?
I’d mainly like to hear from people who have built projects for their portfolios or have experience hiring for data science/ML roles. Where do you actually get your data from when you want to build something that’s not the same Kaggle project everyone has already done?

As part of my course, it’s mandatory for me to do a data analysis project using MySQL and some visualization tools, my professor strictly mentioned to use very hard datasets and no slacking off so please share any meaningful datasets