161sh/CatgirlIntelligenceAgency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
485 Commits 19 Branches 14 Tags 14 MiB
1b53a5389d
Commit Graph

485 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
vlofgren
971089bad3 Cleaning up. 2022-09-11 11:58:39 +02:00
vlofgren
eaef93f4ae Cleaning up and adding better error messages. 2022-09-11 11:31:22 +02:00
vlofgren
fbe17b62ed Giga-refactor of the index query logic 2022-09-10 20:28:45 +02:00
vlofgren
c6976acdfc WIP Loading 2022-09-05 17:51:49 +02:00
vlofgren
c912d3127d Better hints. 2022-09-03 18:35:04 +02:00
vlofgren
2e3d95bcb1 Refactoring and cleanup 2022-09-03 17:32:53 +02:00
vlofgren
5a4d41d414 Refactoring and cleanup, WIP 2022-09-03 15:20:26 +02:00
vlofgren
26e0cfec3a Preparation for conversion 2022-09-02 17:45:03 +02:00
vlofgren
ccf79f47b0 Preparation for conversion 2022-09-02 14:51:11 +02:00
vlofgren
a04d27692e Merge branch 'master' into experimental-22-08 2022-09-02 11:29:30 +02:00
Viktor Lofgren
eb3e996c71 Merge pull request 'CSS tweaks for search.' (#107) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/107
 2022-09-02 11:21:50 +02:00
Viktor Lofgren
ca7a1ceaf0 Merge branch 'release' into master 2022-09-02 11:21:40 +02:00
vlofgren
578ecfb27d CSS tweaks for search. 2022-09-02 10:58:07 +02:00
vlofgren
3fd48e0e53 Cleaning the code a bit, fix URL loading bug with multiple fragments in URL 2022-09-02 10:41:02 +02:00
vlofgren
5dd61387bf Merge branch 'master' into experimental-22-08 2022-09-02 09:39:20 +02:00
Viktor Lofgren
5eb16fc643 Merge pull request 'Fix index.hdb errors from copying' (#106) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/106
 2022-09-02 09:36:15 +02:00
Viktor Lofgren
646e39f51b Merge branch 'release' into master 2022-09-02 09:36:02 +02:00
vlofgren
5b8dc18d81 Fix copy errrors in index.hdb 2022-09-02 09:35:19 +02:00
vlofgren
9270230065 WIP logic for detecting significant images in the body of a website. 2022-09-02 09:35:19 +02:00
vlofgren
5f993c72dd Tweaks for search result relevance 2022-09-02 09:34:20 +02:00
vlofgren
813399401e Tweaks for search result relevance 2022-08-29 18:01:07 +02:00
vlofgren
3f2854a5e9 WIP n-gram loader 2022-08-27 20:30:18 +02:00
vlofgren
0282156979 WIP n-gram loader 2022-08-27 19:19:16 +02:00
vlofgren
c865d6c6b2 Change TF-IDF normalization to reduce the amount of not-so-relevant matches. 2022-08-27 11:38:29 +02:00
vlofgren
f4ad7aaf33 Remove accidental import of an unused library, 
fix build on jdk18-systems.
 2022-08-26 20:48:44 +02:00
vlofgren
3200c36072 Experimental changes for 22-08/09 update. 2022-08-26 16:08:46 +02:00
vlofgren
db056be06a WIP logic for detecting significant images in the body of a website. 2022-08-24 22:05:32 +02:00
Viktor Lofgren
be56852c19 Merge pull request 'Crawler fixes, better stylesheet for search' (#104) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/104
 2022-08-24 19:30:50 +02:00
Viktor Lofgren
c8bbb37de1 Merge branch 'release' into master 2022-08-24 19:30:40 +02:00
vlofgren
c6db2aad48 Fixed stylesheet for search to make random websites button more prominent. 2022-08-24 19:29:00 +02:00
vlofgren
69b9f93dc6 Fixed stylesheet for search to make random websites button more prominent. 2022-08-24 19:28:06 +02:00
vlofgren
9cf78d6929 Bugfixes for the crawler: Better charset support, better 429 handling, better error handling, fixed resource leak. 2022-08-24 19:27:46 +02:00
Viktor Lofgren
fd9083c61f Merge pull request 'Use links index for site suggestions.' (#103) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/103
 2022-08-24 04:42:03 +02:00
Viktor Lofgren
6c6b2ca19f Merge branch 'release' into master 2022-08-24 04:41:54 +02:00
vlofgren
407ec39c0c Use links index for site suggestions. 2022-08-24 04:41:26 +02:00
Viktor Lofgren
be1f5f076b Merge pull request 'Use links index for site suggestions.' (#102) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/102
 2022-08-24 03:50:39 +02:00
Viktor Lofgren
e6549ed30d Merge branch 'release' into master 2022-08-24 03:50:30 +02:00
vlofgren
e1a726babf Use links index for site suggestions. 2022-08-24 03:50:08 +02:00
Viktor Lofgren
556aa31452 Merge pull request 'Use links index for site suggestions.' (#101) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/101
 2022-08-24 03:45:41 +02:00
Viktor Lofgren
8f3eedf0ee Merge branch 'release' into master 2022-08-24 03:45:32 +02:00
vlofgren
4c8c8f5140 Use links index for site suggestions. 2022-08-24 03:45:09 +02:00
Viktor Lofgren
204635e1c7 Merge pull request 'Serve assets from search service instead of resource-store,' (#100) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/100
 2022-08-24 00:41:51 +02:00
Viktor Lofgren
54a24294b9 Merge branch 'release' into master 2022-08-24 00:41:42 +02:00
vlofgren
961ef2a930 Serve assets from search service instead of resource-store, 
dynamically render index for future goodies,
css tweaks.
 2022-08-24 00:41:20 +02:00
Viktor Lofgren
2f8cab7f0e Merge pull request 'master' (#99) from master into release 
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/99
 2022-08-24 00:39:10 +02:00
Viktor Lofgren
5e7c706802 Merge branch 'release' into master 2022-08-24 00:39:00 +02:00
vlofgren
ee0580273e Serve assets from search service instead of resource-store, 
dynamically render index for future goodies,
css tweaks.
 2022-08-24 00:35:22 +02:00
vlofgren
db4cf70784 Reduce resource consumption during crawling, 
reduce TIME_WAIT sockets with a custom socket
factory.
 2022-08-23 13:26:37 +02:00
vlofgren
6fc72b3eb8 Clean up feature extraction, fix misidentification of 'application/ld+json' as javascript. 2022-08-23 00:48:48 +02:00
vlofgren
6e2fdb7a77 Reduce crawling memory consumption, 
Increase crawling threads,
Dynamically adjust crawling rate.
 2022-08-23 00:35:45 +02:00