Viktor Lofgren
|
a5d980ee56
|
(converter) Hook crawl job extractor and adjacencies calculator into control service.
|
2023-07-26 15:46:22 +02:00 |
|
Viktor Lofgren
|
c069c8c182
|
(crawler) Clean up crawl data reference and recrawl logic
|
2023-07-22 18:42:21 +02:00 |
|
Viktor Lofgren
|
f91d92cccb
|
(crawler) WIP
|
2023-07-20 21:05:16 +02:00 |
|
Viktor Lofgren
|
480abfe966
|
(minor) Add limit to pol count in MqPersistence, fix test
|
2023-07-12 18:16:23 +02:00 |
|
Viktor Lofgren
|
dbb758d1a8
|
Minor: Better error handling in crawled domain reader
|
2023-07-10 18:58:43 +02:00 |
|
Viktor Lofgren
|
4fc0ddbc45
|
Improved crawl-job-extractor.
Let crawl-job-extractor run offline and allow it to read domains from file.
Improved docs.
|
2023-06-20 11:37:52 +02:00 |
|
Viktor Lofgren
|
2afbdc2269
|
Adjust the logic for the crawl job extractor to set a relatively low visit limit for websites that are new in the index or has not yielded many good documents previously.
|
2023-06-07 22:01:35 +02:00 |
|
Viktor
|
ac1ac3ea57
|
Move database to a separate module
* Move database to a separate project, break apart sql file into separate entities.
* Fix front page news listing.
|
2023-03-25 15:26:17 +01:00 |
|
Viktor Lofgren
|
2eb972dea1
|
Remove unrelated code, break tools into their own directory.
|
2023-03-17 16:03:11 +01:00 |
|