CatgirlIntelligenceAgency/third-party/encyclopedia-marginalia-nu
Viktor Lofgren 27ffb8fa8a (converter) Integrate zim->db conversion into automatic encyclopedia processing workflow
Previously, in order to load encyclopedia data into the search engine, it was necessary to use the encyclopedia.marginalia.nu converter to first create a .db-file.  This isn't very ergonomic, so parts of that code-base was lifted in as a 3rd party library, and conversion from .zim to .db is now done automatically.

The output file name is based on the original filename, plus a crc32 hash and a .db-ending, to ensure we can recycle the data on repeat loads.
2024-01-19 13:59:03 +01:00
..
src/main/java/nu/marginalia/encyclopedia (converter) Integrate zim->db conversion into automatic encyclopedia processing workflow 2024-01-19 13:59:03 +01:00
build.gradle (converter) Integrate zim->db conversion into automatic encyclopedia processing workflow 2024-01-19 13:59:03 +01:00
readme.md (converter) Integrate zim->db conversion into automatic encyclopedia processing workflow 2024-01-19 13:59:03 +01:00

This package contains a severely stripped down version of the codebase from encyclopedia.marginalia.nu.

The extracted code is a ZimFile reader and WikiHTML cleaner. It is used by the encyclopedia side-loader.