History

Viktor Lofgren b945fd7f39 A lot of readmes, some refactoring.		2023-03-06 18:32:13 +01:00
..
src	A lot of readmes, some refactoring.	2023-03-06 18:32:13 +01:00
build.gradle	A lot of readmes, some refactoring.	2023-03-06 18:32:13 +01:00
readme.md	A lot of readmes, some refactoring.	2023-03-06 18:32:13 +01:00

readme.md

Lexicon

The lexicon contains a mapping for words to identifiers. This lexicon is populated from a journal. The actual word data isn't mapped, but rather a 64 bit hash.

The lexicon is written by crawl/loading-process and read when services-core/index-service interprets queries.

Central Classes

KeywordLexicon
KeywordLexiconJournal
DictionaryMap comes in two versions
- OnHeapDictionaryMap - basically just a fastutil Long2IntOpenHashMap
- OffHeapDictionaryHashMap - a heavily modified trove TLongIntHashMap that uses off heap memory