Viktor Lofgren
a8b9d21f2d
(executor) Refine atag export logic
...
* Remove obviously uninteresting tags
* Omit URL schema for more sensible sorting
* Change the column order to put the source domain last
2023-11-01 13:23:14 +01:00
Viktor Lofgren
23f2068e33
(executor) Actor for exporting anchor tag data from crawl data
2023-10-31 17:32:34 +01:00
Viktor Lofgren
7b5ec6b98f
(executor-service) Embed dist/ in executor-service's docker image
2023-10-19 17:48:34 +02:00
Viktor Lofgren
23526f6d1a
(executor) Executor service now pulls DomainType list for CRAWL on "recrawl"
...
This is an automatic integration with the submit-site repo on github and also
crawl-queue.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
81dd3809e9
(*) WIP Add node affinity to EC_DOMAIN
...
Very messy commit due to fractalline yak shaving
2023-10-19 17:48:34 +02:00
Viktor Lofgren
978550f809
(executor-service) Retire features-convert and move the corresponding packages into the executor service.
2023-10-16 15:43:46 +02:00
Viktor Lofgren
4baf9527d7
(*) WIP Control GUI redesign, executor-service, multi-node mq
...
This turned out to be very difficult to do in small isolated steps.
* Design overhaul of the control gui using bootstrap
* Move the actors out of control-service into to a new executor-service, that can be run on multiple nodes
* Add node-affinity to message queue
2023-10-14 12:08:43 +02:00
