CatgirlIntelligenceAgency/code/features-crawl/crawl-blocklist/readme.md
2023-03-12 11:42:07 +01:00

9 lines
387 B
Markdown

# Crawl Blocklist
Contains tools for blocking links from crawling.
## Central Classes
* [GeoIpBlocklist](src/main/java/nu/marginalia/ip_blocklist/GeoIpBlocklist.java) - country blocking
* [IpBlocklist](src/main/java/nu/marginalia/ip_blocklist/IpBlockList.java) - CIDR-based blocking
* [UrlBlocklist](src/main/java/nu/marginalia/ip_blocklist/UrlBlocklist.java) - URL pattern blocking