CatgirlIntelligenceAgency/code/crawl/crawl-job-extractor-process/readme.md
2023-03-07 17:14:32 +01:00

4 lines
178 B
Markdown

# Crawl Job Extractor
The crawl job extractor creates a file containing a list of domains
along with known URLs. This is consumed by the [crawling-process](../crawling-process).