161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
f655ec5a5c
CatgirlIntelligenceAgency/code/features-crawl
History
Viktor Lofgren f655ec5a5c (*) Refactor GeoIP-related code 
In this commit, GeoIP-related classes are refactored and relocated to a common library as they are shared across multiple services.

The crawler is refactored to enable the GeoIpBlocklist to use the new GeoIpDictionary as the base of its decisions.

The converter is modified ot query this data to add a geoip:-keyword to documents to permit limiting a search to the country of the hosting server.

The commit also adds due BY-SA attribution in the search engine footer for the source of the IP geolocation data.
2023-12-10 17:30:43 +01:00
..
crawl-blocklist (*) Refactor GeoIP-related code 2023-12-10 17:30:43 +01:00
link-parser (build) Move unit test configuration to root build.gradle 2023-10-04 12:46:22 +02:00
readme.md Yet more restructuring. Improved search result ranking. 2023-03-16 21:35:54 +01:00

readme.md

Crawl Features

These are bits of search-engine related code that are relatively isolated pieces of business logic, that benefit from the clarity of being kept separate from the rest of the crawling code.