-
Task
-
Resolution: Fixed
-
Normal
-
None
-
None
The postgres extension we use to do live(~hourly) database replication feeds is a 20 year old tool. The original project was a precursor to Slony and to the actual replication in PG.
There are issues with cascading triggers that broke replication in 2017; the MB team had to work around in musicbrainz-server, but it revealed an underlying issue.
Workarounds (f.e. using BEFORE trigger instead of AFTER) feel uncomfortable.
Fixing DBMirror also has many drawbacks
Enter dbmirror2! https://github.com/metabrainz/dbmirror2
Written by bitmap in PL/pgSQL (native postgres language), stores data as JSON, and more critically allows to determine if operations are out of order, thus getting rid of the cascading triggers issue.
Could possibly improve JSON dumps efficiency, and also using those as cache for the webservice
Replication data would possibly give us a great way to have historical data of all edits!
See also the presentation here: https://drive.google.com/file/d/1_ePGtA5_KAkJTxuZzh9iGQzgeJG1s4Hm/view?usp=sharing
- depends on
-
MBS-12250 Create dbmirror2 schema on production and mirror servers
- Closed