CatgirlIntelligenceAgency/code/features-index
Viktor Lofgren eb59ac8535 (index-ranking) Adjust the BM25P factors a bit
Since the bleed-flags set by the anchor tags logics have been changed to Site and SiteAdjacent, give them a bit of more importance when set together with ExternalLink.

UrlDomain and UrlPath are also only more consistently only rewarded once.
2024-01-30 21:27:29 +01:00
..
domain-ranking (*) Replace EC_DOMAIN_LINK table with files and in-memory caching 2024-01-08 15:53:13 +01:00
index-forward (array) Remove unused 'madvise' code and 3rd party dependency on 'uppend' 2024-01-22 13:01:57 +01:00
index-journal (loader) Update the size of the keyword files created by the loader 2024-01-10 17:09:19 +01:00
index-query (search) Add query strategy requiring link 2024-01-03 16:23:00 +01:00
index-reverse (minor) Reduce log severity for getTermMeta miss 2023-10-26 15:41:52 +02:00
result-ranking (index-ranking) Adjust the BM25P factors a bit 2024-01-30 21:27:29 +01:00
readme.md (refactor) Remove features-search and update documentation 2023-10-09 15:12:30 +02:00

Index

These are components that offer functionality for the index-service.

Indexes

There are two indexes with accompanying tools for constructing them.

  • index-reverse is code for word->document indexes. There are two such indexes, one containing only document-word pairs that are flagged as important, e.g. the word appears in the title or has a high TF-IDF. This allows good results to be discovered quickly without having to sift through ten thousand bad ones first.

  • index-forward is the document->word index containing metadata about each word, such as its position. It is used after identifying candidate search results via the reverse index to fetch metadata and rank the results.

These indices rely heavily on the libraries/btree and libraries/array components.

Algorithms

Libraries

  • index-query contains structures for evaluating search queries.
  • index-journal contains tools for writing and reading index data.