Viktor Lofgren
8ae1f08095
(WIP) Implement first take of new query segmentation algorithm
2024-03-12 13:12:50 +01:00
Viktor Lofgren
57e6a12d08
(registry) Correct registerMonitor() behavior
...
The previous behavior would listen to too many changes, and based on zookeeper and not curator assumptions about behavior, add an additional monitor on each invocation of each monitor, (which always trigger on service state changes), leading to each monitor re-registering and effectively doubling monitors in numbers whenever a service stopped or started, which in turn meant a lot of bizarre thrashing behavior even on changes in services that don't explicitly talk to each other.
This re-registering behavior is no longer done.
2024-03-06 12:22:15 +01:00
Viktor Lofgren
46423612e3
(refac) Merge service-discovery and service modules
...
Also adds a few tests to the server/client code.
2024-03-03 10:49:23 +01:00
Viktor Lofgren
29bf473d74
(encyclopedia) Add URLencoding to path element
...
This prevents corruption of the links to the sideloaded encyclopedia data when the article path contains characters that are not valid in a URL.
2024-03-01 17:28:09 +01:00
Viktor Lofgren
9689f3faee
(domain-info) Fix incorrect array indexing
2024-02-29 18:56:09 +01:00
Viktor Lofgren
93fa58c93d
(domain-info) Fix incorrect array indexing
...
Using the id instead of idx when addressing the ranksArray caused exceptions.
2024-02-29 17:54:23 +01:00
Viktor Lofgren
186a98cc99
(doc) Fix wonky bullet lists
2024-02-28 17:43:05 +01:00
Viktor Lofgren
9993f265ca
(doc) Remove irrelevant text
2024-02-28 17:40:05 +01:00
Viktor Lofgren
144f967dbf
(misc) Tweak pool sizes
2024-02-28 16:23:02 +01:00
Viktor Lofgren
b31c9bb726
(docs) Update process docs
2024-02-28 15:21:33 +01:00
Viktor Lofgren
c0820b5e5c
(docs) Update service docs
2024-02-28 15:19:31 +01:00
Viktor Lofgren
65b8a1d5d9
(grpc) Reduce error spam
2024-02-28 14:44:48 +01:00
Viktor Lofgren
a0648844fb
(grpc) Reduce error spam
2024-02-28 14:35:29 +01:00
Viktor Lofgren
c4a27003c6
(docs) Fix formatting
2024-02-28 14:22:57 +01:00
Viktor Lofgren
41abd8982f
(math) Clean up error handling
2024-02-28 14:19:50 +01:00
Viktor Lofgren
86bbc1043e
(service) Clean up thread pool creation
2024-02-28 14:06:32 +01:00
Viktor Lofgren
9a045a0588
(index) Clean up index code
2024-02-28 13:09:47 +01:00
Viktor Lofgren
9415539b38
(docs) Update docs
2024-02-28 12:25:19 +01:00
Viktor Lofgren
84bab2783d
(docs) Fix fake news in docs
2024-02-28 12:16:45 +01:00
Viktor
0d6e7673e4
Merge pull request #81 from MarginaliaSearch/service-discovery
...
Zookeeper for service-discovery, kill service-client lib, refactor everything
2024-02-28 12:15:25 +01:00
Viktor Lofgren
d78e9e715f
(misc) Fix broken tests
2024-02-28 12:12:43 +01:00
Viktor Lofgren
a8ec59eb75
(conf) Add migration warning when ZOOKEEPER_HOSTS is not set.
2024-02-28 12:09:38 +01:00
Viktor Lofgren
20fc0ef13c
(gradle) Add task alias 'docker' for 'jibDockerBuild'
...
The change also moves the jib boilerplate to an include.
2024-02-28 11:59:15 +01:00
Viktor Lofgren
37ae8cb33c
Migrate the docker compose files
2024-02-28 11:48:16 +01:00
Viktor Lofgren
9f1649636e
Clean up documentation and rename domain-links to link-graph
2024-02-28 11:40:39 +01:00
Viktor Lofgren
3a65fe8917
Add offload executor to GrpcChannelPoolFactory
2024-02-27 22:08:39 +01:00
Viktor Lofgren
99a6e56e99
(index-client) Increase thread count in index client
...
This should be a fair bit larger than the number of index nodes
2024-02-27 22:00:29 +01:00
Viktor Lofgren
e696fd9e92
(docs) Begin un-fucking the docs after refactoring
2024-02-27 21:22:21 +01:00
Viktor Lofgren
c943954bb4
(domain-info) Reduce memory usage
2024-02-27 21:22:21 +01:00
Viktor Lofgren
eaf836dc66
(service/grpc) Reduce thread count
...
Netty and GRPC by default spawns an incredible number of threads on high-core CPUs, which amount to a fair bit of RAM usage.
Add custom executors that throttle this behavior.
2024-02-27 21:22:21 +01:00
Viktor Lofgren
dbf64b0987
(logs) Add the option for json logging
2024-02-27 21:22:20 +01:00
Viktor Lofgren
8d0af9548b
(search) Bot mitigation
...
Add the ability to indicate to the search service that a request is malicious, and to poison the results by providing randomly reorered old results instead.
2024-02-27 21:22:19 +01:00
Viktor Lofgren
67aa20ea2c
(array) Attempting to debug strange errors
2024-02-27 21:22:18 +01:00
Viktor Lofgren
5604e9f531
(query) Bump query length, see what happens :P
2024-02-27 21:22:17 +01:00
Viktor Lofgren
1a51ec2d69
(index) Index optimization
2024-02-27 21:22:17 +01:00
Viktor Lofgren
3eb0800742
(index) Improve granularity of candidate queue polling
2024-02-27 21:22:17 +01:00
Viktor Lofgren
427f3e922f
(index) Retire count operation, clean up index code.
2024-02-27 21:22:17 +01:00
Viktor Lofgren
823ca73a3f
(domain-ranking) Fix a crash during ranking the edges of the similarity graph doesn't quite match the vertices of the link graph.
2024-02-27 21:22:17 +01:00
Viktor Lofgren
7fc0d4d786
(index) Observability for query execution queues
2024-02-27 21:22:17 +01:00
Viktor Lofgren
b8e336e809
(index) Reduce time allocation a bit
2024-02-27 21:22:17 +01:00
Viktor Lofgren
9429bf5c45
(index) Clean up
2024-02-27 21:22:17 +01:00
Viktor Lofgren
f7f0100174
(build) Make docker image registry and tag configurable in root build.gradle
2024-02-25 11:08:49 +01:00
Viktor Lofgren
fc00701a1e
(index) Experimental refactoring of the indexing functionality
2024-02-25 11:05:10 +01:00
Viktor Lofgren
09447f2ad2
(process service) Inherit parent's assertion status
2024-02-24 18:32:37 +01:00
Viktor Lofgren
ff0ef1eebc
(cleanup) Minor cleanups
2024-02-24 15:33:56 +01:00
Viktor Lofgren
1d34224416
(refac) Remove src/main from all source code paths.
...
Look, this will make the git history look funny, but trimming unnecessary depth from the source tree is a very necessary sanity-preserving measure when dealing with a super-modularized codebase like this one.
While it makes the project configuration a bit less conventional, it will save you several clicks every time you jump between modules. Which you'll do a lot, because it's *modul*ar. The src/main/java convention makes a lot of sense for a non-modular project though. This ain't that.
2024-02-23 16:13:40 +01:00
Viktor Lofgren
56d35aa596
(refac) Move execution API out of executor service
2024-02-23 13:26:11 +01:00
Viktor Lofgren
2201b1a506
(refac) Clean up code issues
2024-02-23 11:39:19 +01:00
Viktor Lofgren
5cdb07023b
(refac) Clean up unused imports
2024-02-23 11:27:20 +01:00
Viktor Lofgren
6154e16951
(refac) Remove "distPath"
2024-02-23 11:22:02 +01:00
