Viktor
|
d0239368e2
|
Merge pull request #39 from MarginaliaSearch/master-control-program
Message Queue, State Machine, and Control Service
|
2023-08-10 15:42:58 +02:00 |
|
Viktor Lofgren
|
4f8048be31
|
(blacklist) Blacklist management
|
2023-08-10 15:40:07 +02:00 |
|
Viktor Lofgren
|
807fb2d052
|
(service) Task heartbeat creates event log entries
|
2023-08-09 15:15:16 +02:00 |
|
Viktor Lofgren
|
ce293029c7
|
(converter) Treat adtech tracking as advertisement.
|
2023-08-09 14:23:53 +02:00 |
|
Viktor Lofgren
|
b5ed21be21
|
(mq) MqPersistence no longer relies on autoCommit being enabled
|
2023-08-09 14:23:22 +02:00 |
|
Viktor Lofgren
|
251fc63b42
|
(*) Fix merge gore
|
2023-08-09 13:33:28 +02:00 |
|
Viktor Lofgren
|
47f3855a4b
|
(control) More informative readme.md
|
2023-08-09 12:42:23 +02:00 |
|
Viktor Lofgren
|
71dfe9f33e
|
(control) Clean up the ControlService, move mq-related endpoints to MessageQueueService.
|
2023-08-09 12:42:01 +02:00 |
|
Viktor Lofgren
|
afad4f5ebb
|
(*) last touches
|
2023-08-07 12:59:33 +02:00 |
|
Viktor Lofgren
|
4ab1cd9502
|
(*) last touches
|
2023-08-07 12:57:44 +02:00 |
|
Viktor
|
52e2ab45bf
|
Merge branch 'master' into master-control-program
|
2023-08-07 12:53:43 +02:00 |
|
Viktor Lofgren
|
be444f9172
|
(control) New actions view, re-arrange navigation menu
|
2023-08-05 14:45:04 +02:00 |
|
Viktor Lofgren
|
715d61dfea
|
(mq) Fix bug in notice handling where they were registered on the wrong name
|
2023-08-05 14:45:04 +02:00 |
|
Viktor Lofgren
|
bf37a3eb25
|
(search-service) Make flushCaches endpoint a notice and not a request
|
2023-08-05 14:45:04 +02:00 |
|
Viktor Lofgren
|
c2b45bec8d
|
(mq) Rename notify to sendNotice to avoid name clash with the java object function
|
2023-08-05 14:45:04 +02:00 |
|
Viktor Lofgren
|
cdfe284f9a
|
(file storage) File Storage Type for EXPORT data
(file storage) File Storage Type for EXPORT data
|
2023-08-05 14:45:03 +02:00 |
|
Viktor Lofgren
|
08eed17e66
|
(api-service) Mq endpoint for flushing caches
|
2023-08-05 14:42:16 +02:00 |
|
Viktor Lofgren
|
00eb8b90dc
|
(control) Message Queue GUI
|
2023-08-04 22:05:29 +02:00 |
|
Viktor Lofgren
|
912129311d
|
(control) Message Queue GUI
|
2023-08-04 17:54:18 +02:00 |
|
Viktor Lofgren
|
624b78ec3a
|
(heartbeat) Task heartbeats
|
2023-08-04 14:40:06 +02:00 |
|
Viktor Lofgren
|
1d0cea1d55
|
(converter) GUI for dealing with user complaints
|
2023-08-03 17:59:57 +02:00 |
|
Viktor Lofgren
|
f01f608474
|
(blacklist) Support blacklists with subdomain
|
2023-08-03 17:58:52 +02:00 |
|
Viktor Lofgren
|
c22feaf42e
|
(crawl) Make crawler limiter request a GC when throttling
|
2023-08-03 17:58:18 +02:00 |
|
Viktor Lofgren
|
63e857f7cd
|
(control) Add basic api key management
|
2023-08-02 20:14:03 +02:00 |
|
Viktor Lofgren
|
9979c9defe
|
(search/index) Add blogosphere filter
|
2023-08-02 20:13:30 +02:00 |
|
Viktor Lofgren
|
7763df0715
|
(docs) Add control-service to the main readme.md
|
2023-08-01 22:52:41 +02:00 |
|
Viktor Lofgren
|
e088eb9ec8
|
(scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows.
|
2023-08-01 22:50:33 +02:00 |
|
Viktor Lofgren
|
19402772fc
|
(scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows.
|
2023-08-01 22:50:05 +02:00 |
|
Viktor Lofgren
|
ba724bc1b2
|
(scripts|docs) Update scripts and documentations for the new operator's gui and file storage workflows.
|
2023-08-01 22:47:37 +02:00 |
|
Viktor Lofgren
|
8de3e6ab80
|
(control) Fix bug where CrawlActor and RecrawlActor would steal each others' mail
|
2023-08-01 22:33:30 +02:00 |
|
Viktor Lofgren
|
659d2134ba
|
(file-storage) Deprecate mustClean flag
|
2023-08-01 22:32:30 +02:00 |
|
Viktor Lofgren
|
867410c66b
|
(file-storage) Automatic file storage discovery via manifest file
|
2023-08-01 18:05:43 +02:00 |
|
Viktor Lofgren
|
483c2dbb44
|
(conf) Change default user-agent to not associate it with the project; remove unused disks.properties file.
|
2023-08-01 17:34:25 +02:00 |
|
Viktor Lofgren
|
e5c9791b14
|
(crawler) Fix rare ConcurrentModificationError due to HashSet
|
2023-08-01 17:28:29 +02:00 |
|
Viktor Lofgren
|
58556af6c7
|
(db) Use flwyay for database migrations.
|
2023-08-01 17:08:42 +02:00 |
|
Viktor Lofgren
|
2e29038ecd
|
(db) Fix broken insert statement, move file storage defaults to a separate file.
|
2023-08-01 15:50:08 +02:00 |
|
Viktor Lofgren
|
36a23707c1
|
(control) Control service should be a core service.
|
2023-08-01 15:49:50 +02:00 |
|
Viktor Lofgren
|
c1ea60b399
|
(db) Default values for storage base
|
2023-08-01 15:05:04 +02:00 |
|
Viktor Lofgren
|
b08e302dd5
|
(lexicon) Optimize lexicon by using Murmur3_128's hash function
|
2023-08-01 15:02:13 +02:00 |
|
Viktor Lofgren
|
ea66195b97
|
(loader) Optimize loader by using zstd's direct streaming writer and the Murmur3_128 string hash
|
2023-08-01 15:02:13 +02:00 |
|
Viktor Lofgren
|
86a5cc5c5f
|
(hash) Modified version of common codec's Murmur3 hash
|
2023-08-01 14:57:40 +02:00 |
|
Viktor Lofgren
|
8f0cbf267b
|
(loader) Perform instruction reads in a separate thread for extra vroom vroom
|
2023-07-31 14:24:08 +02:00 |
|
Viktor Lofgren
|
2f8488610a
|
(loader) Fix bug where trailing deferred domain meta inserts weren't executed
|
2023-07-31 14:23:23 +02:00 |
|
Viktor Lofgren
|
d95f01b701
|
(control) Reduce log spam in control svc
|
2023-07-31 14:21:06 +02:00 |
|
Viktor Lofgren
|
c9d7635370
|
(control) Aborting an actor that waits on a process request terminates the running job.
(control) Aborting an actor that waits on a process request terminates the running job.
|
2023-07-31 14:21:06 +02:00 |
|
Viktor Lofgren
|
6b5fb0f841
|
(control) Disable the start button for actors that aren't directly initializable.
(control) Disable the start button for actors that aren't directly initializable.
|
2023-07-31 14:21:00 +02:00 |
|
Viktor Lofgren
|
12bd74d4f3
|
Clean up ProcessService
|
2023-07-31 10:56:16 +02:00 |
|
Viktor Lofgren
|
37c4cc68ed
|
TODO
|
2023-07-31 10:34:42 +02:00 |
|
Viktor Lofgren
|
1c948eb3d8
|
(minor) Alter DumbThreadPool in Converter to not claim the threads are crawlers.
|
2023-07-31 10:33:15 +02:00 |
|
Viktor Lofgren
|
cd90ca820f
|
YAGNI filter over ConverterDomainTypes
|
2023-07-31 10:32:47 +02:00 |
|