History

Viktor Lofgren ebc84c22fb Upgrade antique lombok plugin This permits tests to run on JDK20 environments.		2023-08-23 14:34:32 +00:00
..
src/main/java/nu/marginalia/tools	(language) fasttext based language filter	2023-08-16 15:48:12 +02:00
build.gradle	Upgrade antique lombok plugin	2023-08-23 14:34:32 +00:00
readme.md	Remove unrelated code, break tools into their own directory.	2023-03-17 16:03:11 +01:00

readme.md

Term Frequency Extractor

Generates a term frequency dictionary file from a batch of crawl data.

Usage:

PATH_TO_SAMPLES=run/samples/crawl-s
export JAVA_OPTS=-Dcrawl.rootDirRewrite=/crawl:${PATH_TO_SAMPLES} 

term-frequency-extractor ${PATH_TO_SAMPLES}/plan.yaml out.dat

readme.md

Term Frequency Extractor

See Also