vlofgren
|
0282156979
|
WIP n-gram loader
|
2022-08-27 19:19:16 +02:00 |
|
vlofgren
|
c865d6c6b2
|
Change TF-IDF normalization to reduce the amount of not-so-relevant matches.
|
2022-08-27 11:38:29 +02:00 |
|
vlofgren
|
f4ad7aaf33
|
Remove accidental import of an unused library,
fix build on jdk18-systems.
|
2022-08-26 20:48:44 +02:00 |
|
vlofgren
|
3200c36072
|
Experimental changes for 22-08/09 update.
|
2022-08-26 16:08:46 +02:00 |
|
vlofgren
|
db056be06a
|
WIP logic for detecting significant images in the body of a website.
|
2022-08-24 22:05:32 +02:00 |
|
Viktor Lofgren
|
be56852c19
|
Merge pull request 'Crawler fixes, better stylesheet for search' (#104) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/104
|
2022-08-24 19:30:50 +02:00 |
|
Viktor Lofgren
|
c8bbb37de1
|
Merge branch 'release' into master
|
2022-08-24 19:30:40 +02:00 |
|
vlofgren
|
c6db2aad48
|
Fixed stylesheet for search to make random websites button more prominent.
|
2022-08-24 19:29:00 +02:00 |
|
vlofgren
|
69b9f93dc6
|
Fixed stylesheet for search to make random websites button more prominent.
|
2022-08-24 19:28:06 +02:00 |
|
vlofgren
|
9cf78d6929
|
Bugfixes for the crawler: Better charset support, better 429 handling, better error handling, fixed resource leak.
|
2022-08-24 19:27:46 +02:00 |
|
Viktor Lofgren
|
fd9083c61f
|
Merge pull request 'Use links index for site suggestions.' (#103) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/103
|
2022-08-24 04:42:03 +02:00 |
|
Viktor Lofgren
|
6c6b2ca19f
|
Merge branch 'release' into master
|
2022-08-24 04:41:54 +02:00 |
|
vlofgren
|
407ec39c0c
|
Use links index for site suggestions.
|
2022-08-24 04:41:26 +02:00 |
|
Viktor Lofgren
|
be1f5f076b
|
Merge pull request 'Use links index for site suggestions.' (#102) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/102
|
2022-08-24 03:50:39 +02:00 |
|
Viktor Lofgren
|
e6549ed30d
|
Merge branch 'release' into master
|
2022-08-24 03:50:30 +02:00 |
|
vlofgren
|
e1a726babf
|
Use links index for site suggestions.
|
2022-08-24 03:50:08 +02:00 |
|
Viktor Lofgren
|
556aa31452
|
Merge pull request 'Use links index for site suggestions.' (#101) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/101
|
2022-08-24 03:45:41 +02:00 |
|
Viktor Lofgren
|
8f3eedf0ee
|
Merge branch 'release' into master
|
2022-08-24 03:45:32 +02:00 |
|
vlofgren
|
4c8c8f5140
|
Use links index for site suggestions.
|
2022-08-24 03:45:09 +02:00 |
|
Viktor Lofgren
|
204635e1c7
|
Merge pull request 'Serve assets from search service instead of resource-store,' (#100) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/100
|
2022-08-24 00:41:51 +02:00 |
|
Viktor Lofgren
|
54a24294b9
|
Merge branch 'release' into master
|
2022-08-24 00:41:42 +02:00 |
|
vlofgren
|
961ef2a930
|
Serve assets from search service instead of resource-store,
dynamically render index for future goodies,
css tweaks.
|
2022-08-24 00:41:20 +02:00 |
|
Viktor Lofgren
|
2f8cab7f0e
|
Merge pull request 'master' (#99) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/99
|
2022-08-24 00:39:10 +02:00 |
|
Viktor Lofgren
|
5e7c706802
|
Merge branch 'release' into master
|
2022-08-24 00:39:00 +02:00 |
|
vlofgren
|
ee0580273e
|
Serve assets from search service instead of resource-store,
dynamically render index for future goodies,
css tweaks.
|
2022-08-24 00:35:22 +02:00 |
|
vlofgren
|
db4cf70784
|
Reduce resource consumption during crawling,
reduce TIME_WAIT sockets with a custom socket
factory.
|
2022-08-23 13:26:37 +02:00 |
|
vlofgren
|
6fc72b3eb8
|
Clean up feature extraction, fix misidentification of 'application/ld+json' as javascript.
|
2022-08-23 00:48:48 +02:00 |
|
vlofgren
|
6e2fdb7a77
|
Reduce crawling memory consumption,
Increase crawling threads,
Dynamically adjust crawling rate.
|
2022-08-23 00:35:45 +02:00 |
|
Viktor Lofgren
|
c97cdfcc25
|
Merge pull request 'Revert the previous change as my IP got kicked back to ol' reliable '81.170.128.52'' (#98) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/98
|
2022-08-22 17:34:22 +02:00 |
|
Viktor Lofgren
|
f48e92630e
|
Merge branch 'release' into master
|
2022-08-22 17:34:13 +02:00 |
|
vlofgren
|
fc9d9d1bad
|
And revert the previous change as my IP got kicked back to ol' reliable '81.170.128.52'
|
2022-08-22 17:32:56 +02:00 |
|
Viktor Lofgren
|
f498b55301
|
Merge pull request 'Update crawler IP file to reflect the fact that the IP changed.' (#97) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/97
|
2022-08-22 13:04:50 +02:00 |
|
Viktor Lofgren
|
e9dc01dfc5
|
Merge branch 'release' into master
|
2022-08-22 13:04:40 +02:00 |
|
vlofgren
|
087ad0124d
|
Update crawler IP file to reflect the fact that the IP changed.
|
2022-08-22 13:04:07 +02:00 |
|
Viktor Lofgren
|
bbcedb7e9d
|
Merge pull request 'Tweak CSS a tiny bit to add more padding to the right of info cells.' (#95) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/95
|
2022-08-19 16:07:52 +02:00 |
|
Viktor Lofgren
|
475b7fe5e0
|
Merge branch 'release' into master
|
2022-08-19 16:07:44 +02:00 |
|
vlofgren
|
095ed7c6c4
|
Tweak CSS a tiny bit to add more padding to the right of info cells.
|
2022-08-19 16:07:26 +02:00 |
|
Viktor Lofgren
|
7cf2be356c
|
Merge pull request 'Add recipe selector to list of updates.' (#94) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/94
|
2022-08-19 15:55:49 +02:00 |
|
Viktor Lofgren
|
09f3e0c97e
|
Merge branch 'release' into master
|
2022-08-19 15:55:38 +02:00 |
|
vlofgren
|
2adbe5f74c
|
Update publicity roll.
|
2022-08-19 15:55:01 +02:00 |
|
Viktor Lofgren
|
a671341c5c
|
Merge pull request 'Update publicity roll with Deutschlandfunk article' (#93) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/93
|
2022-08-19 15:51:11 +02:00 |
|
Viktor Lofgren
|
8de6b52e1e
|
Merge branch 'release' into master
|
2022-08-19 15:51:01 +02:00 |
|
vlofgren
|
56987f6664
|
Update publicity roll.
|
2022-08-19 15:50:15 +02:00 |
|
vlofgren
|
7567890708
|
Update publicity roll.
|
2022-08-19 15:49:52 +02:00 |
|
Viktor Lofgren
|
f708fa643b
|
Merge pull request 'master' (#92) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/92
|
2022-08-18 20:45:28 +02:00 |
|
Viktor Lofgren
|
2a72f1da78
|
Merge branch 'release' into master
|
2022-08-18 20:45:19 +02:00 |
|
vlofgren
|
ede62f2515
|
Retain cookies for domain.
|
2022-08-18 20:44:44 +02:00 |
|
vlofgren
|
a1eb8375a2
|
Exclude wp-content/uploads from crawling
|
2022-08-18 19:05:07 +02:00 |
|
Viktor Lofgren
|
014068ace5
|
Merge pull request 'Don't try to fetch text/css and text/javascript-files. Refactor fetcher to separate content type sniffing logic. Clean up crawler a smidge.' (#91) from master into release
Reviewed-on: https://git.marginalia.nu/marginalia/marginalia.nu/pulls/91
|
2022-08-18 18:41:22 +02:00 |
|
Viktor Lofgren
|
4e3a977049
|
Merge branch 'release' into master
|
2022-08-18 18:41:13 +02:00 |
|