ExportXMLWordPrintable

    • Type: Task
    • Resolution: Unresolved
    • Priority: High
    • Component/s: Nagios
    • None

      I'm not sure how easy this would be, but there are some pieces of data which would make this possible:

      The incremental JSON dumps work by consuming replication packets.

      The current_replication_sequence is the number of the latest replication packet which was produced by MB:

      ssh -A queen 'docker exec postgres-slave psql -U musicbrainz -c "select current_replication_sequence from musicbrainz.replication_control;" musicbrainz_db' | head -n 3 | tail -n 1 | awk '{$1=$1};1'

      The last_processed_replication_sequence is the last sequence which the JSON dumps have processed and generated an incremental dump for:

      ssh -A williams 'docker exec postgres-williams psql -U musicbrainz -c "select last_processed_replication_sequence from json_dump.control;" musicbrainz_json_dump' | head -n 3 | tail -n 1 | awk '{$1=$1};1')

      It would be nice to have flashing red alert for when:

      last_processed_replication_sequence < (current_replication_sequence - 1)

      It would also be nice to alert if the last processed sequence can't be fetched from the API (if this request 404s):

      curl -I "https://metabrainz.org/api/musicbrainz/json-dumps/json-dump-$((last_processed_replication_sequence - 1))/artist.tar.xz?token=$TOKEN"

      For the full JSON dumps, we should alert if the date in LATEST is out of date. The full dumps run every Wednesday and Saturday at 00:10:00 UTC, and usually finish by 05:00:00 (but sometimes a couple hours later, depending on server load I guess), then sync to FTP. So perhaps a check that occurs every Wednesday and Saturday at 12:00:00 UTC that alerts if LATEST is not equal to the current day.

            Assignee:
            Zas
            Reporter:
            Michael Wiencek
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:

                Version Package