Viktor Lofgren
731afcb864
(qs) Parallel execution
2023-10-23 12:06:03 +02:00
Viktor Lofgren
efb73ff4e7
(qs) Don't blow up if an index node isn't responsive
2023-10-23 11:53:18 +02:00
Viktor Lofgren
2ed2f35a9b
(actor) Rewrite of the actor prototype class using record pattern matching
2023-10-23 10:18:20 +02:00
Viktor Lofgren
119151cad3
(converter) Separtion of concerns
2023-10-22 14:35:33 +02:00
Viktor Lofgren
758f9b5aa5
(converter) Get UUID pips out of the models
...
Rendering concerns shouldn't be in the models, it's poor separation of concerns and very difficult to follow.
2023-10-22 14:24:52 +02:00
Viktor Lofgren
e06a8c1de2
(converter) Put upper limit on number of worker threads.
2023-10-22 14:03:09 +02:00
Viktor Lofgren
29ce8ca0cf
(db) Reduce db pool size
...
This is a temporary thing
2023-10-22 14:03:09 +02:00
Viktor Lofgren
eb4158df0b
(control) Fix start/stop FSM endpoints
2023-10-22 14:03:09 +02:00
Viktor Lofgren
12fda1a36b
(control) Temporarily re-writing the data balancer to get it to work in prod
...
Need to clean this up later.
2023-10-22 14:03:09 +02:00
Viktor Lofgren
e927f99777
(control) JSON serializes Map<Integer> to Map<Double> and Java gets confused
2023-10-21 16:24:20 +02:00
Viktor Lofgren
044bcf55bd
(control) Fix SQL in rebalance actor
2023-10-21 16:13:37 +02:00
Viktor Lofgren
e475af9f49
(control) Initialize controlActorService
2023-10-21 16:06:53 +02:00
Viktor Lofgren
c6abcd91fa
(control) Better use of FS states, fix bug with start/stop actors
2023-10-20 16:37:49 +02:00
Viktor Lofgren
10fc489822
(converter) More robust filename resolution
2023-10-20 14:16:03 +02:00
Viktor Lofgren
d76d926c38
(control/executor) Add new configuration options for node
...
It's now possible to configure prod instance to not retain processed data.
2023-10-20 14:05:19 +02:00
Viktor Lofgren
2b3c167845
(controller) Additional configuration options for node
2023-10-20 13:13:36 +02:00
Viktor Lofgren
1d75b974b5
(loader bugfix) Set DOMAIN_METADATA appropriately
2023-10-20 13:03:27 +02:00
Viktor Lofgren
584bb3a648
(fs) interface cleanup
2023-10-20 12:24:18 +02:00
Viktor Lofgren
7b5ec6b98f
(executor-service) Embed dist/ in executor-service's docker image
2023-10-19 17:48:34 +02:00
Viktor Lofgren
23526f6d1a
(executor) Executor service now pulls DomainType list for CRAWL on "recrawl"
...
This is an automatic integration with the submit-site repo on github and also
crawl-queue.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
c0930ead0f
(doc) Update conceptual-overview.svg
2023-10-19 17:48:34 +02:00
Viktor Lofgren
809b3ee023
(control) Update GUI for crawl specs. They are now less important than they were before.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
23f0c79fba
(control) GUI for data sets/domain types.
2023-10-19 17:48:34 +02:00
Viktor Lofgren
81dd3809e9
(*) WIP Add node affinity to EC_DOMAIN
...
Very messy commit due to fractalline yak shaving
2023-10-19 17:48:34 +02:00
Viktor Lofgren
2bf0c4497d
(*) Tool for unfcking old crawl data so that it aligns with the new style IDs
2023-10-19 17:48:34 +02:00
Viktor Lofgren
93122bdd18
(run) Add two nodes to the demo setup
2023-10-16 17:37:26 +02:00
Viktor Lofgren
978550f809
(executor-service) Retire features-convert and move the corresponding packages into the executor service.
2023-10-16 15:43:46 +02:00
Viktor Lofgren
84fea0fd05
(node) Nodes auto-start their monitor actors.
2023-10-16 15:33:22 +02:00
Viktor Lofgren
2df3e0f881
(node) Nodes auto-configure on start-up instead of requiring manual configuration.
2023-10-16 14:46:35 +02:00
Viktor Lofgren
c98117f69d
(actor) FS monitor should pick up stuff in BACKUP as well.
2023-10-16 14:37:36 +02:00
Viktor Lofgren
ede5d1f890
(actor) Give process spawners more easily recognizable names.
2023-10-16 14:19:00 +02:00
Viktor Lofgren
39911e3acd
(control) Fix incorrect storage base and clean up GUI for data
2023-10-16 13:30:26 +02:00
Viktor Lofgren
3d1c15ef99
(client) Refactor liveness monitor
2023-10-16 12:34:01 +02:00
Viktor Lofgren
f718482e98
(client) Fix tests
2023-10-16 12:12:16 +02:00
Viktor Lofgren
8dafd13cd7
(client) Fix executor tests
2023-10-16 12:02:57 +02:00
Viktor Lofgren
0b19b28a64
(file-storage) Delete unused code
2023-10-16 12:02:57 +02:00
Viktor Lofgren
c245f7ce3a
(control) Bootstrapify review-domains and search-to-ban views.
2023-10-15 22:04:23 +02:00
Viktor Lofgren
607d647483
(control) Remove services listing view
2023-10-15 21:48:55 +02:00
Viktor Lofgren
9a38a455c9
(control/exec) File listings in control GUI
2023-10-15 19:15:44 +02:00
Viktor Lofgren
16e0738731
(*) Get multi-node routing working.
2023-10-15 18:38:30 +02:00
Viktor Lofgren
eacbf87979
(control) New list and form for index nodes.
2023-10-14 21:46:52 +02:00
Viktor Lofgren
108b4cb648
(service) Keep disabled multi-noded services dormant when they are configured to be disabled.
2023-10-14 20:58:55 +02:00
Viktor Lofgren
a9dff407a1
(config/db) Clean up migrations
2023-10-14 20:34:03 +02:00
Viktor Lofgren
9e26109e36
(reverse-index) Don't always POST
2023-10-14 16:48:29 +02:00
Viktor Lofgren
6308a8dfcd
(control) Node configuration
2023-10-14 16:47:52 +02:00
Viktor Lofgren
4baf9527d7
(*) WIP Control GUI redesign, executor-service, multi-node mq
...
This turned out to be very difficult to do in small isolated steps.
* Design overhaul of the control gui using bootstrap
* Move the actors out of control-service into to a new executor-service, that can be run on multiple nodes
* Add node-affinity to message queue
2023-10-14 12:08:43 +02:00
Viktor Lofgren
199c459697
(*) Add node-affinity to services, processes and file storage.
2023-10-10 12:32:22 +02:00
Viktor Lofgren
61288c5e68
(service, client) First steps towards multiple nodedness
2023-10-09 22:13:27 +02:00
Viktor Lofgren
8375237de5
(converter) Add special keyword for websites with a tilde url.
2023-10-09 17:02:32 +02:00
Viktor
c8d820c17b
Merge pull request #53 from MarginaliaSearch/standalone-index
...
Move ranking to the index-service, and query parsing to a new query-service; separate out the search-service
2023-10-09 15:42:06 +02:00
