Commit Graph

886 Commits

Author SHA1 Message Date
vlofgren
7749ce645a Further more cleaning 2022-09-12 10:39:02 +02:00
vlofgren
971089bad3 Cleaning up. 2022-09-11 11:58:39 +02:00
vlofgren
eaef93f4ae Cleaning up and adding better error messages. 2022-09-11 11:31:22 +02:00
vlofgren
fbe17b62ed Giga-refactor of the index query logic 2022-09-10 20:28:45 +02:00
vlofgren
c6976acdfc WIP Loading 2022-09-05 17:51:49 +02:00
vlofgren
c912d3127d Better hints. 2022-09-03 18:35:04 +02:00
vlofgren
2e3d95bcb1 Refactoring and cleanup 2022-09-03 17:32:53 +02:00
vlofgren
5a4d41d414 Refactoring and cleanup, WIP 2022-09-03 15:20:26 +02:00
vlofgren
26e0cfec3a Preparation for conversion 2022-09-02 17:45:03 +02:00
vlofgren
ccf79f47b0 Preparation for conversion 2022-09-02 14:51:11 +02:00
vlofgren
a04d27692e Merge branch 'master' into experimental-22-08 2022-09-02 11:29:30 +02:00
Viktor Lofgren
eb3e996c71 Merge pull request 'CSS tweaks for search.' (#107) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/107
2022-09-02 11:21:50 +02:00
Viktor Lofgren
ca7a1ceaf0 Merge branch 'release' into master 2022-09-02 11:21:40 +02:00
vlofgren
578ecfb27d CSS tweaks for search. 2022-09-02 10:58:07 +02:00
vlofgren
3fd48e0e53 Cleaning the code a bit, fix URL loading bug with multiple fragments in URL 2022-09-02 10:41:02 +02:00
vlofgren
5dd61387bf Merge branch 'master' into experimental-22-08 2022-09-02 09:39:20 +02:00
Viktor Lofgren
5eb16fc643 Merge pull request 'Fix index.hdb errors from copying' (#106) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/106
2022-09-02 09:36:15 +02:00
Viktor Lofgren
646e39f51b Merge branch 'release' into master 2022-09-02 09:36:02 +02:00
vlofgren
5b8dc18d81 Fix copy errrors in index.hdb 2022-09-02 09:35:19 +02:00
vlofgren
9270230065 WIP logic for detecting significant images in the body of a website. 2022-09-02 09:35:19 +02:00
vlofgren
5f993c72dd Tweaks for search result relevance 2022-09-02 09:34:20 +02:00
vlofgren
813399401e Tweaks for search result relevance 2022-08-29 18:01:07 +02:00
vlofgren
3f2854a5e9 WIP n-gram loader 2022-08-27 20:30:18 +02:00
vlofgren
0282156979 WIP n-gram loader 2022-08-27 19:19:16 +02:00
vlofgren
c865d6c6b2 Change TF-IDF normalization to reduce the amount of not-so-relevant matches. 2022-08-27 11:38:29 +02:00
vlofgren
f4ad7aaf33 Remove accidental import of an unused library,
fix build on jdk18-systems.
2022-08-26 20:48:44 +02:00
vlofgren
3200c36072 Experimental changes for 22-08/09 update. 2022-08-26 16:08:46 +02:00
vlofgren
db056be06a WIP logic for detecting significant images in the body of a website. 2022-08-24 22:05:32 +02:00
Viktor Lofgren
be56852c19 Merge pull request 'Crawler fixes, better stylesheet for search' (#104) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/104
2022-08-24 19:30:50 +02:00
Viktor Lofgren
c8bbb37de1 Merge branch 'release' into master 2022-08-24 19:30:40 +02:00
vlofgren
c6db2aad48 Fixed stylesheet for search to make random websites button more prominent. 2022-08-24 19:29:00 +02:00
vlofgren
69b9f93dc6 Fixed stylesheet for search to make random websites button more prominent. 2022-08-24 19:28:06 +02:00
vlofgren
9cf78d6929 Bugfixes for the crawler: Better charset support, better 429 handling, better error handling, fixed resource leak. 2022-08-24 19:27:46 +02:00
Viktor Lofgren
fd9083c61f Merge pull request 'Use links index for site suggestions.' (#103) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/103
2022-08-24 04:42:03 +02:00
Viktor Lofgren
6c6b2ca19f Merge branch 'release' into master 2022-08-24 04:41:54 +02:00
vlofgren
407ec39c0c Use links index for site suggestions. 2022-08-24 04:41:26 +02:00
Viktor Lofgren
be1f5f076b Merge pull request 'Use links index for site suggestions.' (#102) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/102
2022-08-24 03:50:39 +02:00
Viktor Lofgren
e6549ed30d Merge branch 'release' into master 2022-08-24 03:50:30 +02:00
vlofgren
e1a726babf Use links index for site suggestions. 2022-08-24 03:50:08 +02:00
Viktor Lofgren
556aa31452 Merge pull request 'Use links index for site suggestions.' (#101) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/101
2022-08-24 03:45:41 +02:00
Viktor Lofgren
8f3eedf0ee Merge branch 'release' into master 2022-08-24 03:45:32 +02:00
vlofgren
4c8c8f5140 Use links index for site suggestions. 2022-08-24 03:45:09 +02:00
Viktor Lofgren
204635e1c7 Merge pull request 'Serve assets from search service instead of resource-store,' (#100) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/100
2022-08-24 00:41:51 +02:00
Viktor Lofgren
54a24294b9 Merge branch 'release' into master 2022-08-24 00:41:42 +02:00
vlofgren
961ef2a930 Serve assets from search service instead of resource-store,
dynamically render index for future goodies,
css tweaks.
2022-08-24 00:41:20 +02:00
Viktor Lofgren
2f8cab7f0e Merge pull request 'master' (#99) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/99
2022-08-24 00:39:10 +02:00
Viktor Lofgren
5e7c706802 Merge branch 'release' into master 2022-08-24 00:39:00 +02:00
vlofgren
ee0580273e Serve assets from search service instead of resource-store,
dynamically render index for future goodies,
css tweaks.
2022-08-24 00:35:22 +02:00
vlofgren
db4cf70784 Reduce resource consumption during crawling,
reduce TIME_WAIT sockets with a custom socket
factory.
2022-08-23 13:26:37 +02:00
vlofgren
6fc72b3eb8 Clean up feature extraction, fix misidentification of 'application/ld+json' as javascript. 2022-08-23 00:48:48 +02:00