Uploaded image for project: 'MusicBrainz Server'
  1. MusicBrainz Server
  2. MBS-12107

Replication with dbmirror2

XMLWordPrintable

      The postgres extension we use to do live(~hourly) database replication feeds is a 20 year old tool. The original project was a precursor to Slony and to the actual replication in PG.

      There are issues with cascading triggers that broke replication in 2017; the MB team had to work around in musicbrainz-server, but it revealed an underlying issue.
      Workarounds (f.e. using BEFORE trigger instead of AFTER) feel uncomfortable.
      Fixing DBMirror also has many drawbacks

      Enter dbmirror2! https://github.com/metabrainz/dbmirror2
      Written by bitmap in PL/pgSQL (native postgres language), stores data as JSON, and more critically allows to determine if operations are out of order, thus getting rid of the cascading triggers issue.

      Could possibly improve JSON dumps efficiency, and also using those as cache for the webservice
      Replication data would possibly give us a great way to have historical data of all edits!

      See also the presentation here: https://drive.google.com/file/d/1_ePGtA5_KAkJTxuZzh9iGQzgeJG1s4Hm/view?usp=sharing

            bitmap Michael Wiencek
            ApekattQuest, MonkeyPython MonkeyPython
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:

                Version Package
                Schema change, 2023 Q2