diff --git a/code/readme.md b/code/readme.md index c10d1c8e..9d59ac42 100644 --- a/code/readme.md +++ b/code/readme.md @@ -12,14 +12,24 @@ A map of the most important components and how they relate can be found below. ![image](../doc/diagram/conceptual-overview.svg) +The core part of the search engine is the index service, which is responsible for storing and retrieving +the document data. The index serive is partitioned, along with the executor service, which is responsible for executing +processes. At least one instance of each service must be run, but more can be run +alongside. Multiple partitions is desirable in production to distribute load across multiple physical drives, +as well as reducing the impact of downtime. + +Search queries are delegated via the query service, which is a proxy that fans out the query to all +eligible index services. The control service is responsible for distributing commands to the executor +service, and for monitoring the health of the system. It also offers a web interface for operating the system. + ### Services -* [core services](services-core/) "macroservices", stateful, memory hungry doing heavy lifting. +* [core services](services-core/) Most of these services are stateful, memory hungry, and doing heavy lifting. * * [control](services-core/control-service) * * [query](services-core/query-service) * * [index](services-core/index-service) * * [executor](services-core/executor-service) * * [assistant](services-core/assistant-service) -* [application services](services-application/) "microservices", stateless providing additional functionality and making an application out of the search engine. +* [application services](services-application/) Mostly stateless gateways providing access to the core services. * * [api](services-application/api-service) - public API * * [search](services-application/search-service) - marginalia search application * * [dating](services-application/dating-service) - [https://explore.marginalia.nu/](https://explore.marginalia.nu/) @@ -28,7 +38,8 @@ A map of the most important components and how they relate can be found below. ### Processes -Processes are batch jobs that deal with data retrieval, processing and loading. +Processes are batch jobs that deal with data retrieval, processing and loading. These are spawned and orchestrated by +the executor service, which is controlled by the control service. * [processes](processes/) * * [crawling-process](processes/crawling-process)