History

Viktor Lofgren 3101b74580 (index) Move to a lexicon-free index design This is a system-wide change. The index used to have a lexicon, mapping words to wordIds using a large in-memory hash table. This made index-construction easier, but it also added a fairly significant RAM penalty to both the index service and the loader. The new design moves to 64 bit word identifiers calculated using the murmur hash of the keyword, and an index construction based on merging smaller indices. It also became necessary half-way through to upgrade guice as its error reporting wasn't quite compatible with JDK20.		2023-08-28 14:02:23 +02:00
..
src/main/java/nu/marginalia/term_frequency_dict	(index) Move to a lexicon-free index design	2023-08-28 14:02:23 +02:00
build.gradle	Upgrade antique lombok plugin	2023-08-23 14:34:32 +00:00
readme.md	Remove unrelated code, break tools into their own directory.	2023-03-17 16:03:11 +01:00

Term Frequency Dictionary

This dictionary is used by various parts of the system to evaluate for example the TF-IDF score of a keyword.

Central Classes