CatgirlIntelligenceAgency/code
Viktor Lofgren 23526f6d1a (executor) Executor service now pulls DomainType list for CRAWL on "recrawl"
This is an automatic integration with the submit-site repo on github and also
crawl-queue.
2023-10-19 17:48:34 +02:00
..
api (*) WIP Add node affinity to EC_DOMAIN 2023-10-19 17:48:34 +02:00
common (executor) Executor service now pulls DomainType list for CRAWL on "recrawl" 2023-10-19 17:48:34 +02:00
features-convert (build) Move unit test configuration to root build.gradle 2023-10-04 12:46:22 +02:00
features-crawl (build) Move unit test configuration to root build.gradle 2023-10-04 12:46:22 +02:00
features-index (reverse-index) Don't always POST 2023-10-14 16:48:29 +02:00
features-qs (refactor) Remove features-search and update documentation 2023-10-09 15:12:30 +02:00
features-search (refactor) Remove features-search and update documentation 2023-10-09 15:12:30 +02:00
libraries (*) WIP Add node affinity to EC_DOMAIN 2023-10-19 17:48:34 +02:00
process-models (*) WIP Add node affinity to EC_DOMAIN 2023-10-19 17:48:34 +02:00
processes (*) WIP Add node affinity to EC_DOMAIN 2023-10-19 17:48:34 +02:00
services-application (*) WIP Control GUI redesign, executor-service, multi-node mq 2023-10-14 12:08:43 +02:00
services-core (executor) Executor service now pulls DomainType list for CRAWL on "recrawl" 2023-10-19 17:48:34 +02:00
tools (*) Tool for unfcking old crawl data so that it aligns with the new style IDs 2023-10-19 17:48:34 +02:00
readme.md (refactor) Remove features-search and update documentation 2023-10-09 15:12:30 +02:00

Code

This is a pretty large and diverse project with many moving parts.

You'll find a short description in each module of what it does and how it relates to other modules. The modules each have names like "library" or "process" or "feature". These have specific meanings. See doc/module-taxonomy.md.

Overview

A map of the most important components and how they relate can be found below.

image

Services

Processes

Processes are batch jobs that deal with data retrieval, processing and loading.

Tools

Features

Features are relatively stand-alone components that serve some part of the domain. They aren't domain-independent, but isolated.

Libraries and primitives

Libraries are stand-alone code that is independent of the domain logic.

  • common elements for creating a service, a client etc.
  • libraries containing non-search specific code.
    • array - large memory mapped area library
    • btree - static btree library