161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

(big-string) Make big-string disable:able

Browse Source
This commit is contained in:
Viktor Lofgren 2023-07-21 19:50:35 +02:00
parent 58f2f86ea8
commit 995657c6ce
3 changed files with 11 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
code/libraries/big-string/src/main/java/nu/marginalia/bigstring/BigString.java
View File

@ -1,8 +1,11 @@
package nu.marginalia.bigstring;
public interface BigString {
boolean disableBigString = Boolean.getBoolean("bigstring.disabled");
static BigString encode(String stringValue) {
if (stringValue.length() > 64) {
if (!disableBigString && stringValue.length() > 64) {
return new CompressedBigString(stringValue);
}
else {

6
doc/system-properties.md
View File

@ -29,3 +29,9 @@ These are JVM system properties used by each service
|lexiconSizeHint| 800000000 | The default size of the lexicon |
|local-index-path| /some/path | Selects the location the loader will write index data |
|crawl.rootDirRewrite|/some/path|Sets the base directory of a crawl plan |
## Other
|flag| values | description |
|---|------------|---------------------------------------------|
|bigstring.disabled| true/false | Disables transparent big string compression |

2
run/env/service.env vendored
View File

@ -1,4 +1,4 @@
WMSA_HOME=run/
CONTROL_SERVICE_OPTS="-DdistPath=/dist"
CONVERTER_OPTS="-ea -Xmx16G -XX:-CompactStrings -XX:+UseParallelGC -XX:GCTimeRatio=14 -XX:ParallelGCThreads=15"
CRAWLER_OPTS="-Xmx16G -XX:+UseParallelGC -XX:GCTimeRatio=14 -XX:ParallelGCThreads=15"
CRAWLER_OPTS="-Dbigstring.disabled=true -Xmx16G -XX:+UseParallelGC -XX:GCTimeRatio=14 -XX:ParallelGCThreads=15"