66c1281301
Cleaning out a lot of old junk from the code, and one thing lead to another...
* Build is improved, now constructing docker images with 'jib'. Clean build went from 3 minutes to 50 seconds.
* The ProcessService's spawning is smarter. Will now just spawn a java process instead of relying on the application plugin's generated outputs.
* Project is migrated to GraalVM
* gRPC clients are re-written with a neat fluent/functional style. e.g.
```channelPool.call(grpcStub::method)
.async(executor) // <-- optional
.run(argument);
```
This change is primarily to allow handling ManagedChannel errors, but it turned out to be a pretty clean API overall.
* For now the project is all in on zookeeper
* Service discovery is now based on APIs and not services. Theoretically means we could ship the same code either a monolith or a service mesh.
* To this end, began modularizing a few of the APIs so that they aren't strongly "living" in a service. WIP!
Missing is documentation and testing, and some more breaking apart of code.
|
||
|---|---|---|
| .. | ||
| src | ||
| build.gradle | ||
| readme.md | ||
Domain Ranking
Contains domain ranking algorithms. The domain ranking algorithms are based on the JGraphT library.
Two principal algorithms are available, the standard PageRank algorithm, and personalized pagerank; each are available for two graphs, the link graph and a similarity graph where each edge corresponds to the similarity between the sets of incident links to two domains, their cosine similarity acting as the weight of the links.
With the standard PageRank algorithm, the similarity graph does not produce anything useful, but something magical happens when you apply Personalized PageRank to this graph. It turns into a very good "vibe"-sensitive ranking algorithm.
It's unclear if this is a well known result, but it's a very interesting one for creating a ranking algorithm that is focused on a particular segment of the web.
Central Classes
- PageRankDomainRanker - Ranks domains using the PageRank or Personalized PageRank algorithm depending on whether a list of influence domains is provided.
Data sources
- LinkGraphSource - fetches the link graph
- InvertedLinkGraphSource - fetches the inverted link graph
- SimilarityGraphSource - fetches the similarity graph from the database
Note that the similarity graph needs to be precomputed and stored in the database for the similarity graph source to be available.
See Also
- result-ranking - Ranks search results