161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
f1c6525a50
CatgirlIntelligenceAgency/code/tools/crawl-job-extractor
History
Viktor ac1ac3ea57
Move database to a separate module 
* Move database to a separate project, break apart sql file into separate entities.
* Fix front page news listing.
2023-03-25 15:26:17 +01:00
..
src Move database to a separate module 2023-03-25 15:26:17 +01:00
build.gradle Move database to a separate module 2023-03-25 15:26:17 +01:00
readme.md Remove unrelated code, break tools into their own directory. 2023-03-17 16:03:11 +01:00

readme.md

Crawl Job Extractor

The crawl job extractor creates a file containing a list of domains along with known URLs.

This is consumed by processes/crawling-process.