161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0553174401
CatgirlIntelligenceAgency/crawl/crawl-job-extractor-process
History
Viktor Lofgren b945fd7f39 A lot of readmes, some refactoring.
2023-03-06 18:32:13 +01:00
..
src Restructuring the git repo 2023-03-04 13:19:01 +01:00
build.gradle Restructuring the git repo 2023-03-04 13:19:01 +01:00
readme.md A lot of readmes, some refactoring. 2023-03-06 18:32:13 +01:00

readme.md

Crawl Job Extractor

The crawl job extractor creates a file containing a list of domains along with known URLs. This is consumed by the crawling-process.