Viktor Lofgren
|
c68d17d482
|
(keyword-extraction) Fix bug leading to position data missing on some keywords.
This was due to a discrepancy between the KeywordPositionBitmask and WordsTfIdfCounts' concept of a keyword.
|
2023-09-02 14:48:55 +02:00 |
|
Viktor Lofgren
|
2b00cd632d
|
(process) Propagate environment JVM params to the index constructor
|
2023-09-01 15:39:42 +02:00 |
|
Viktor
|
bdcbfb11a8
|
Merge pull request #42 from MarginaliaSearch/no-downtime-upgrades
Zero downtime upgrades, merge-based index construction
|
2023-08-29 17:05:48 +02:00 |
|
Viktor Lofgren
|
fa87c7e1b7
|
(process) Automatic flightrecorder runs for processes when run in docker.
|
2023-08-29 14:12:51 +02:00 |
|
Viktor Lofgren
|
194a6057dd
|
(index,control) Recoverable index backups
|
2023-08-25 14:57:43 +02:00 |
|
Viktor
|
229c63c46d
|
Update readme.md
|
2023-08-24 13:27:24 +02:00 |
|
Viktor Lofgren
|
b958acb76a
|
(file-storage) New File Storage type for linkdb
|
2023-08-24 09:06:13 +02:00 |
|
Viktor Lofgren
|
e8c0648e04
|
Fix missing vol/ss dir in setup.sh
|
2023-08-23 17:59:40 +02:00 |
|
Viktor Lofgren
|
8bd9a00c38
|
Amend setup instructions with command
|
2023-08-23 14:02:21 +00:00 |
|
Viktor Lofgren
|
972d03efdf
|
Fix error in run/readme where it suggested local dev environment uses HTTPS
|
2023-08-23 13:47:39 +00:00 |
|
Viktor Lofgren
|
2656fcfe2c
|
(conf) Remove unnecessary JVM flags for processes
|
2023-08-17 17:42:47 +02:00 |
|
Viktor Lofgren
|
46d761f34f
|
(language) fasttext based language filter
|
2023-08-16 15:48:12 +02:00 |
|
Viktor Lofgren
|
c56ee10185
|
(control) Separate [Process] and [Process and Load] actions for crawl data; all SLOW data is deletable.
|
2023-08-13 13:39:59 +02:00 |
|
Viktor
|
69b28fd07d
|
Update readme.md
|
2023-08-12 18:58:21 +02:00 |
|
Viktor
|
99884c2c7e
|
Update readme.md
|
2023-08-12 15:39:28 +02:00 |
|
Viktor Lofgren
|
a42f707b2d
|
(docs) Update readme with up to date instructions
|
2023-08-11 13:43:00 +02:00 |
|
Viktor Lofgren
|
eef37927ba
|
(docs) Update readme with up to date instructions
|
2023-08-11 13:42:14 +02:00 |
|
Viktor Lofgren
|
cdfe284f9a
|
(file storage) File Storage Type for EXPORT data
(file storage) File Storage Type for EXPORT data
|
2023-08-05 14:45:03 +02:00 |
|
Viktor Lofgren
|
ba724bc1b2
|
(scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows.
|
2023-08-01 22:47:37 +02:00 |
|
Viktor Lofgren
|
483c2dbb44
|
(conf) Change default user-agent to not associate it with the project; remove unused disks.properties file.
|
2023-08-01 17:34:25 +02:00 |
|
Viktor Lofgren
|
58556af6c7
|
(db) Use flwyay for database migrations.
|
2023-08-01 17:08:42 +02:00 |
|
Viktor Lofgren
|
9786f82220
|
Fix environment variables to processes so jmc works
|
2023-07-31 10:32:23 +02:00 |
|
Viktor Lofgren
|
6ff7e9648f
|
(crawler) Use and pass the proper environment variables to the processes.
|
2023-07-30 16:54:02 +02:00 |
|
Viktor Lofgren
|
a56953c798
|
(converter, WIP) Refactor converter to not have to load everything into RAM.
|
2023-07-24 15:25:09 +02:00 |
|
Viktor Lofgren
|
995657c6ce
|
(big-string) Make big-string disable:able
|
2023-07-21 19:50:35 +02:00 |
|
Viktor Lofgren
|
f91d92cccb
|
(crawler) WIP
|
2023-07-20 21:05:16 +02:00 |
|
Viktor Lofgren
|
d7ab21fe34
|
(*) Refactor Control Service and processes
|
2023-07-17 21:20:31 +02:00 |
|
Viktor Lofgren
|
8b74e3aa0d
|
(*) File Storage WIP
|
2023-07-14 17:08:10 +02:00 |
|
Viktor Lofgren
|
7087ab5f07
|
(run) Reduce nginx access log noise for local setup
|
2023-07-11 23:11:34 +02:00 |
|
Viktor Lofgren
|
77261a38cd
|
(control, WIP) MQFSM and ProcessService are sitting in a tree
We're spawning processes from the MSFSM in control service now!
|
2023-07-11 17:08:43 +02:00 |
|
Viktor Lofgren
|
2283ceb77d
|
(control) WIP control service
|
2023-07-10 18:58:43 +02:00 |
|
Viktor Lofgren
|
62cc9df206
|
Embryo of new control process
* New events and heartbeat tables in mariadb
* Refactored to a cleaner Service interface
|
2023-07-03 10:40:32 +02:00 |
|
Viktor Lofgren
|
a9fabba407
|
Tell experiment runner to only process some domains.
Updated the experiment runner, as well as the script.
|
2023-06-20 14:14:01 +02:00 |
|
Viktor
|
f1c6525a50
|
Update setup.sh
|
2023-04-02 14:44:43 +02:00 |
|
Viktor Lofgren
|
d0c72ceb7e
|
Improve experiment runner, convenient start script.
|
2023-03-30 15:40:31 +02:00 |
|
Viktor Lofgren
|
8f51345a1d
|
Add experiment runner tool and got rid of experiments module in processes.
|
2023-03-28 16:58:46 +02:00 |
|
Viktor Lofgren
|
862e925d7c
|
"-Dsmall-ram=TRUE" no longer does anything. Remove references to the flag, which previously reduced the memory footprint of the loader and index service.
|
2023-03-26 21:37:11 +02:00 |
|
Viktor Lofgren
|
964014860a
|
Get suggestions working again
|
2023-03-22 15:11:22 +01:00 |
|
Viktor Lofgren
|
d82532b7f1
|
More restructuring, big bug fixes in keyword extraction.
|
2023-03-13 17:39:53 +01:00 |
|
Viktor Lofgren
|
347f16939c
|
Fix broken setup script
|
2023-03-12 12:21:37 +01:00 |
|
Viktor Lofgren
|
8b8fc49901
|
The refactoring will continue until morale improves.
|
2023-03-12 11:42:07 +01:00 |
|
Viktor Lofgren
|
02db999762
|
Enable assertions in reconvert script.
|
2023-03-09 17:26:08 +01:00 |
|
Viktor Lofgren
|
2a25b5e8a9
|
Placeholder screenshots when the domain is missing from the database entirely.
|
2023-03-08 18:36:41 +01:00 |
|
Viktor Lofgren
|
5c1a59257c
|
Reconvert script broken when code/ moved.
|
2023-03-08 17:18:04 +01:00 |
|
Viktor Lofgren
|
2d066af5b9
|
More documentation...
|
2023-03-06 18:45:01 +01:00 |
|
Viktor Lofgren
|
fe0d754f2c
|
Make the code run properly without WMSA_HOME set, adding missing test assets.
|
2023-03-05 14:12:13 +01:00 |
|
Viktor Lofgren
|
fd1b56dbad
|
Make the code run properly without WMSA_HOME set, adding missing test assets.
|
2023-03-05 13:47:40 +01:00 |
|
Viktor Lofgren
|
9a0d1d5d4e
|
Setup readme
|
2023-03-04 16:14:03 +01:00 |
|
Viktor Lofgren
|
592766ba65
|
Setup readme
|
2023-03-04 16:12:37 +01:00 |
|
Viktor Lofgren
|
036adcbe1f
|
Setup readme
|
2023-03-04 16:06:54 +01:00 |
|