161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
74644d59f3
CatgirlIntelligenceAgency/code/features-crawl/crawl-blocklist
History
Viktor Lofgren 74644d59f3 (crawler) Update URL blocklist 
* Don't crawl MDN mirrors
* More mailing list variants
2023-07-10 18:04:43 +02:00
..
src (crawler) Update URL blocklist 2023-07-10 18:04:43 +02:00
build.gradle The refactoring will continue until morale improves. 2023-03-12 11:42:07 +01:00
readme.md The refactoring will continue until morale improves. 2023-03-12 11:42:07 +01:00

readme.md

Crawl Blocklist

Contains tools for blocking links from crawling.

Central Classes