161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

(scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows.

Browse Source
This commit is contained in:
Viktor Lofgren 2023-08-01 22:50:33 +02:00
parent 19402772fc
commit e088eb9ec8
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/crawling.md
View File

@ -40,7 +40,7 @@ A crawl specification file is a compressed JSON file with each domain name to cr
known URLs for each domain. These are created in the `storage -> specifications` view in the operator's gui.
To bootstrap the system, you need a list of known domains. This is just a text file with one domain name per line,
with blanlines and comments starting with `#` ignored.
with blanklines and comments starting with `#` ignored.
Make it available over HTTP(S) and select `Download a list of domains from a URL` in the `Create New Specification`
form. Make sure to give this specification a good description, as it will follow you around for a while.