Commit Graph

1083 Commits

Author SHA1 Message Date
Viktor Lofgren
a42f707b2d (docs) Update readme with up to date instructions 2023-08-11 13:43:00 +02:00
Viktor Lofgren
eef37927ba (docs) Update readme with up to date instructions 2023-08-11 13:42:14 +02:00
Viktor Lofgren
7440da240d (blacklist) Fix broken SQL migration 2023-08-11 13:33:35 +02:00
Viktor
d0239368e2
Merge pull request #39 from MarginaliaSearch/master-control-program
Message Queue, State Machine, and Control Service
2023-08-10 15:42:58 +02:00
Viktor Lofgren
4f8048be31 (blacklist) Blacklist management 2023-08-10 15:40:07 +02:00
Viktor Lofgren
807fb2d052 (service) Task heartbeat creates event log entries 2023-08-09 15:15:16 +02:00
Viktor Lofgren
ce293029c7 (converter) Treat adtech tracking as advertisement. 2023-08-09 14:23:53 +02:00
Viktor Lofgren
b5ed21be21 (mq) MqPersistence no longer relies on autoCommit being enabled 2023-08-09 14:23:22 +02:00
Viktor Lofgren
251fc63b42 (*) Fix merge gore 2023-08-09 13:33:28 +02:00
Viktor Lofgren
47f3855a4b (control) More informative readme.md 2023-08-09 12:42:23 +02:00
Viktor Lofgren
71dfe9f33e (control) Clean up the ControlService, move mq-related endpoints to MessageQueueService. 2023-08-09 12:42:01 +02:00
Viktor Lofgren
afad4f5ebb (*) last touches 2023-08-07 12:59:33 +02:00
Viktor Lofgren
4ab1cd9502 (*) last touches 2023-08-07 12:57:44 +02:00
Viktor
52e2ab45bf
Merge branch 'master' into master-control-program 2023-08-07 12:53:43 +02:00
Viktor Lofgren
be444f9172 (control) New actions view, re-arrange navigation menu 2023-08-05 14:45:04 +02:00
Viktor Lofgren
715d61dfea (mq) Fix bug in notice handling where they were registered on the wrong name 2023-08-05 14:45:04 +02:00
Viktor Lofgren
bf37a3eb25 (search-service) Make flushCaches endpoint a notice and not a request 2023-08-05 14:45:04 +02:00
Viktor Lofgren
c2b45bec8d (mq) Rename notify to sendNotice to avoid name clash with the java object function 2023-08-05 14:45:04 +02:00
Viktor Lofgren
cdfe284f9a (file storage) File Storage Type for EXPORT data
(file storage) File Storage Type for EXPORT data
2023-08-05 14:45:03 +02:00
Viktor Lofgren
08eed17e66 (api-service) Mq endpoint for flushing caches 2023-08-05 14:42:16 +02:00
Viktor Lofgren
00eb8b90dc (control) Message Queue GUI 2023-08-04 22:05:29 +02:00
Viktor Lofgren
912129311d (control) Message Queue GUI 2023-08-04 17:54:18 +02:00
Viktor Lofgren
624b78ec3a (heartbeat) Task heartbeats 2023-08-04 14:40:06 +02:00
Viktor Lofgren
1d0cea1d55 (converter) GUI for dealing with user complaints 2023-08-03 17:59:57 +02:00
Viktor Lofgren
f01f608474 (blacklist) Support blacklists with subdomain 2023-08-03 17:58:52 +02:00
Viktor Lofgren
c22feaf42e (crawl) Make crawler limiter request a GC when throttling 2023-08-03 17:58:18 +02:00
Viktor Lofgren
63e857f7cd (control) Add basic api key management 2023-08-02 20:14:03 +02:00
Viktor Lofgren
9979c9defe (search/index) Add blogosphere filter 2023-08-02 20:13:30 +02:00
Viktor Lofgren
7763df0715 (docs) Add control-service to the main readme.md 2023-08-01 22:52:41 +02:00
Viktor Lofgren
e088eb9ec8 (scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows. 2023-08-01 22:50:33 +02:00
Viktor Lofgren
19402772fc (scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows. 2023-08-01 22:50:05 +02:00
Viktor Lofgren
ba724bc1b2 (scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows. 2023-08-01 22:47:37 +02:00
Viktor Lofgren
8de3e6ab80 (control) Fix bug where CrawlActor and RecrawlActor would steal each others' mail 2023-08-01 22:33:30 +02:00
Viktor Lofgren
659d2134ba (file-storage) Deprecate mustClean flag 2023-08-01 22:32:30 +02:00
Viktor Lofgren
867410c66b (file-storage) Automatic file storage discovery via manifest file 2023-08-01 18:05:43 +02:00
Viktor Lofgren
483c2dbb44 (conf) Change default user-agent to not associate it with the project; remove unused disks.properties file. 2023-08-01 17:34:25 +02:00
Viktor Lofgren
e5c9791b14 (crawler) Fix rare ConcurrentModificationError due to HashSet 2023-08-01 17:28:29 +02:00
Viktor Lofgren
58556af6c7 (db) Use flwyay for database migrations. 2023-08-01 17:08:42 +02:00
Viktor Lofgren
2e29038ecd (db) Fix broken insert statement, move file storage defaults to a separate file. 2023-08-01 15:50:08 +02:00
Viktor Lofgren
36a23707c1 (control) Control service should be a core service. 2023-08-01 15:49:50 +02:00
Viktor Lofgren
c1ea60b399 (db) Default values for storage base 2023-08-01 15:05:04 +02:00
Viktor Lofgren
b08e302dd5 (lexicon) Optimize lexicon by using Murmur3_128's hash function 2023-08-01 15:02:13 +02:00
Viktor Lofgren
ea66195b97 (loader) Optimize loader by using zstd's direct streaming writer and the Murmur3_128 string hash 2023-08-01 15:02:13 +02:00
Viktor Lofgren
86a5cc5c5f (hash) Modified version of common codec's Murmur3 hash 2023-08-01 14:57:40 +02:00
Viktor Lofgren
8f0cbf267b (loader) Perform instruction reads in a separate thread for extra vroom vroom 2023-07-31 14:24:08 +02:00
Viktor Lofgren
2f8488610a (loader) Fix bug where trailing deferred domain meta inserts weren't executed 2023-07-31 14:23:23 +02:00
Viktor Lofgren
d95f01b701 (control) Reduce log spam in control svc 2023-07-31 14:21:06 +02:00
Viktor Lofgren
c9d7635370 (control) Aborting an actor that waits on a process request terminates the running job.
(control) Aborting an actor that waits on a process request terminates the running job.
2023-07-31 14:21:06 +02:00
Viktor Lofgren
6b5fb0f841 (control) Disable the start button for actors that aren't directly initializable.
(control) Disable the start button for actors that aren't directly initializable.
2023-07-31 14:21:00 +02:00
Viktor Lofgren
12bd74d4f3 Clean up ProcessService 2023-07-31 10:56:16 +02:00