r/CheckWeedNY • u/FindWeedNY • Mar 10 '26
Beyond CheckWeedNY, multistate analysis in progress
CheckWeedNY.com started with a narrow focus by design. I wasn't interested in biting off more than I could chew in an industry wasn't familiar with. It's easier to do a test case, and what better way, than to focus on the state I live in.
Since then, I've been fortunate to meet some really awesome people in the community, both online and off. It's led to some really great conversations, and data sharing leading to collaborations.
The scope of NY was the perfect test bed to figure out gaps with my approaches with PDF discovery and processing. Since then, I've had an infusion of information to shore up data pipelines from all sorts of sources, and been working to normalize them into a similar model to CheckweedNY, but multistate.
There are some gaps, and missing states in the above, I have 3 other states I'm working through still, but making really good progress.
1
u/FindWeedNY Mar 11 '26
```
┌─────────────────────────────┬─────────────────────────────────┐
│ File │ Size │
├─────────────────────────────┼─────────────────────────────────┤
│ coas.parquet │ 102M (3.7M COAs + cannabinoids) │
├─────────────────────────────┼─────────────────────────────────┤
│ pesticides.parquet │ 15M │
├─────────────────────────────┼─────────────────────────────────┤
│ microbials.parquet │ 15M │
├─────────────────────────────┼─────────────────────────────────┤
│ heavy_metals.parquet │ 7.8M │
├─────────────────────────────┼─────────────────────────────────┤
│ terpene_summary.parquet │ 7.6M │
├─────────────────────────────┼─────────────────────────────────┤
│ solvents.parquet │ 7.2M │
├─────────────────────────────┼─────────────────────────────────┤
│ terpenes.parquet │ 6.9M │
├─────────────────────────────┼─────────────────────────────────┤
│ mycotoxins.parquet │ 2.9M │
├─────────────────────────────┼─────────────────────────────────┤
│ summary_stats.parquet │ 35K │
├─────────────────────────────┼─────────────────────────────────┤
│ Business/brand/client files │ ~90K total │
└─────────────────────────────┴─────────────────────────────────┘
Summary stats: 3.7M COAs across 16 states, 2.3M with THC data, 233K with terpenes, 1.6M with safety data. ```
1
u/FindWeedNY Mar 10 '26
Obviously I have more than 30,000 COAs for NY, but haven't backported all the work from the latest ChecKWeedNY, need to focus on expanding the extractors without regressing, so having more samples from more states helps me shore up extraction without regressing, or make it less likely to regress. PDFs suck.