4 Commits

Author	SHA1	Message	Date
Viktor Lofgren	ed373eef61	Refactor crawler and add special logic for some platforms ... * Break apart CrawlerRetreiver * Break apart HttpFetcher into an interface and impl for testing sanity * Add special logic for Lemmy, Mediawiki and Discourse to not waste requests on paths that aren't interesting.	2023-06-27 10:57:54 +02:00
Viktor Lofgren	a9fabba407	Tell experiment runner to only process some domains. ... Updated the experiment runner, as well as the script.	2023-06-20 14:14:01 +02:00
Viktor Lofgren	d0c72ceb7e	Improve experiment runner, convenient start script.	2023-03-30 15:40:31 +02:00
Viktor Lofgren	8f51345a1d	Add experiment runner tool and got rid of experiments module in processes.	2023-03-28 16:58:46 +02:00