f58a9f46be
This behavior is an old vestige from the days of only having a single loader process. We'd truncate the links table because doing inserts/updates was too slow. This was also important because we had 32 bit ID, and there's a lot of links between domains to go around... Instead we delete the rows associated with the current node with a stored procedure PURGE_LINKS_TABLE. We also update the PRIMARY KEY to a BIGINT. We'll need to load the data in excess of billion times to hit an ID rollover, so it'll be fine. |
||
|---|---|---|
| .. | ||
| src | ||
| build.gradle | ||
| readme.md | ||
DB
This module primarily contains SQL files for the URLs database. The most central tables are EC_DOMAIN, EC_URL and EC_PAGE_DATA.
Flyway
The system uses flyway to track database changes and allow easy migrations, this is accessible via gradle tasks.
flywayMigrateflywayBaselineflywayRepairflywayClean(dangerous as in wipes your entire database)
Refer to the Flyway documentation for guidance. It's well documented and these are probably the only four tasks you'll ever need.
If you are not running the system via docker, you need to provide alternative connection details than the defaults (TODO: how?).
The migration files are in resources/db/migration. The file name convention incorporates the project's cal-ver versioning; and are applied in lexicographical order.
VYY_MM_v_nnn__description.sql
Central Paths
- migrations - Flyway migrations
See Also
- common/service implements DatabaseModule, which is from where the services get database connections.