0307c55f9f
To avoid having to either hard-code or manually configure service addresses (possibly several dozen), and to reduce the project's dependency on docker to deal with routing and discovery, the option to use [Zookeeper](https://zookeeper.apache.org/) to manage services and discovery has been added. A service registry interface was added, with a Zookeeper implementation and a basic implementation that only works on docker and hard-codes everything. The last remaining REST service, the assistant-service, has been migrated to gRPC. This also proved a good time to clear out primordial technical debt from the root of the codebase. The 'service-client' library has been taken behind the barn and given a last farewell. It's replaced by a small library for managing gRPC channels. Since it's no longer used by anything, RxJava has been removed as a dependency from the project. Although the current state seems reasonably stable, this is a work-in-progress commit. |
||
|---|---|---|
| .. | ||
| src | ||
| build.gradle | ||
| readme.md | ||
Index Journal
The index journal contains a list of entries with keywords and keyword metadata per document.
This journal is written by processes/loading-process and read when constructing the forward and reverse indices.
The journal format is a file header, followed by a zstd-compressed list of entries, each containing a header with document-level data, and a data section with keyword-level data.
The journal data may be split into multiple files, and the journal writers and readers are designed to handle this transparently via their Paging implementation.