History

Viktor Lofgren aa0d256d6a Upgrade code to Java 20. * Change language version * Upgrade Lombok to a JDK20 compatible version		2023-08-23 13:37:49 +00:00
..
src/main/java/nu/marginalia/tools	(language) fasttext based language filter	2023-08-16 15:48:12 +02:00
build.gradle	Upgrade code to Java 20.	2023-08-23 13:37:49 +00:00
readme.md	Remove unrelated code, break tools into their own directory.	2023-03-17 16:03:11 +01:00

readme.md

Term Frequency Extractor

Generates a term frequency dictionary file from a batch of crawl data.

Usage:

PATH_TO_SAMPLES=run/samples/crawl-s
export JAVA_OPTS=-Dcrawl.rootDirRewrite=/crawl:${PATH_TO_SAMPLES} 

term-frequency-extractor ${PATH_TO_SAMPLES}/plan.yaml out.dat

readme.md

Term Frequency Extractor

See Also