dbe9235f3a
... also move some common configuration into the root build.gradle-file. Support for JDK21 in lombok is a bit sketchy at the moment, but it seems to work. This upgrade is kind of important as the new index construction really benefits from Arena based lifecycle control over off-heap memory. |
||
|---|---|---|
| .. | ||
| crawl-blocklist | ||
| link-parser | ||
| readme.md | ||
Crawl Features
These are bits of search-engine related code that are relatively isolated pieces of business logic, that benefit from the clarity of being kept separate from the rest of the crawling code.
- crawl-blocklist - IP and URL blocklists
- link-parser - Code for parsing and normalizing links