CatgirlIntelligenceAgency/code/crawl/crawl-job-extractor-process/readme.md

4 lines
178 B
Markdown
Raw Normal View History

2023-03-06 18:32:13 +01:00
# Crawl Job Extractor
The crawl job extractor creates a file containing a list of domains
along with known URLs. This is consumed by the [crawling-process](../crawling-process).