8e1abc3f10
* (index-reverse) Parallel construction of the reverse indexes. * (array) Remove wasteful calculation of numDistinct before merging two sorted arrays. * (index-reverse) Force changes to disk on close, reduce logging. * (index-reverse) Clean up merging process and add back logging * (run) Add a conservative default for INDEX_CONSTRUCTION_PROCESS_OPTS's parallelism as it eats a lot of RAM * (index-reverse) Better logging during processing * (array) 2GB+ compatible write() function * (array) 2GB+ compatible write() function * (index-reverse) We are logging like Bolsonaro and I will not have it. * (reverse-index) Self-diagnostics * (btree) Fix bug in btree reader to do with large data sizes |
||
|---|---|---|
| .. | ||
| array | ||
| big-string | ||
| blocking-thread-pool | ||
| braille-block-punch-cards | ||
| btree | ||
| easy-lsh | ||
| guarded-regex | ||
| language-processing | ||
| message-queue | ||
| next-prime | ||
| random-write-funnel | ||
| term-frequency-dict | ||
| LICENSE.txt | ||
| readme.md | ||
Libraries
These are libraries that are not strongly coupled to the search engine's business logic. These libraries may not depend on features, services, processes, models, etc.
NOTE: These libraries are co-licensed under the MIT license.
Libraries
- The array library is for memory mapping large memory-areas, which Java has bad support for. It's designed to be able to easily replaced when Java's Foreign Function And Memory API is released.
- The btree library offers a static BTree implementation based on the array library.
- language-processing contains primitives for sentence extraction and POS-tagging.
- The message-queue library.
Micro libraries
- easy-lsh is a simple locality-sensitive hash for document deduplication
- guarded-regex makes predicated regular expressions clearer
- big-string offers seamless string compression
- random-write-funnel is a tool for reducing write amplification when constructing large files out of order.
- next-prime naive brute force prime sieve.
- braille-block-punch-cards renders bit masks into human-readable dot matrices using the braille block.