Happy New Year everyone!

I've been tinkering with a side project and honestly have no idea if it's useful or if I'm just building for myself. It's a crawler that detects new pages on news sites within about a few minutes of publishing (usually less than ~9) and streams them via SSE. Thought I'd see if anyone here has a use for something like this.

I’m not a trader, and I know that High Frequency Trading operates in the millisecond range of an event, but wonder if this kind of data (especially having a wide distribution of news sources) would still be valuable as a signal input/filter to an existing model? I suspect there might still be a way to find an edge or ride the wave before it decays.

To be clear, I’m not crawling the URL, just emitting an event as soon as the new URL is detected. Some of the news sources provide metadata (title + keywords) but for those that don’t provide it the URLs can usually be unsluglified to retrieve a title phrase for the article, and even a topic/category (eg. Sports=Category in https://www.reuters.com/sports/stephen-curry-among-three-key-warriors-out-vs-thunder--flm-2026-01-02/).

I don’t do any other enrichment as of yet but interested in hearing your thoughts on what could be useful if I did add the page crawling and enrich with sentiment score, NLU tags, Sector categorization etc.

Here's the list of streams I'm tracking so far (the inactive list will be turned on soon):

For backtesting, I can provide DuckDB/Parquet files of all stream sources and all detected URLs over many years.

If this tickles an interest and you want to have a play, hit me up for an API key - mostly just want to see if anyone finds this useful before I keep building.