CatgirlIntelligenceAgency/code/features-index/readme.md

25 lines
1.2 KiB
Markdown
Raw Normal View History

2023-03-04 13:19:01 +01:00
# Index
These are components that offer functionality for the [index-service](../services-core/index-service).
## Indexes
There are two indexes with accompanying tools for constructing them.
2023-03-20 16:27:37 +01:00
* [index-reverse](index-reverse/) is code for `word->document` indexes. There are two such indexes, one containing only document-word pairs that are flagged as important, e.g. the word appears in the title or has a high TF-IDF. This allows good results to be discovered quickly without having to sift through ten thousand bad ones first.
* [index-forward](index-forward/) is the `document->word` index containing metadata about each word, such as its position. It is used after identifying candidate search results via the reverse index to fetch metadata and rank the results.
2023-03-04 13:19:01 +01:00
2023-03-06 18:55:28 +01:00
These indices rely heavily on the [libraries/btree](../libraries/btree) and [libraries/array](../libraries/array) components.
## Algorithms
* [domain-ranking](domain-ranking/) contains ranking algorithms.
2023-03-04 13:19:01 +01:00
# Libraries
2023-03-04 13:19:01 +01:00
* [index-query](index-query/) contains structures for evaluating search queries.
* [index-journal](index-journal/) contains tools for writing and reading index data.
* [lexicon](lexicon/) contains a mapping between words' string representation and an unique integer identifier.