CatgirlIntelligenceAgency/code/functions/index/index-journal

Index Journal

The index journal contains a list of entries with keywords and keyword metadata per document.

This journal is written by processes/loading-process and read when constructing the forward and reverse indices.

The journal format is a file header, followed by a zstd-compressed list of entries, each containing a header with document-level data, and a data section with keyword-level data.

The journal data may be split into multiple files, and the journal writers and readers are designed to handle this transparently via their Paging implementation.

Central Classes

Model

• IndexJournalEntry
• IndexJournalEntryHeader
• IndexJournalEntryData

I/O

• IndexJournalReader
• IndexJournalWriter