-
Improvement
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
We have some users who have reported that some submissions have had no highlevel features generated: https://community.metabrainz.org/t/high-level-data-not-calculated/407196
I tried to run the highlevel extractor with these lowlevel submissions and they successfully generated data. I suspect that there may have been a small time (during the site migration maybe?) where data was incorrectly generated and we didn't correctly write it to the database.
We should add some features to the AB management commands to help us understand this:
- Make a command that checks for rows that exist in `highlevel` but have no `highlevel_meta` row (Failed jobs: https://github.com/metabrainz/acousticbrainz-server/blob/278c1d47cd0442a9fa335cf45766ff9c2acc4b33/db/data.py#L320-L321 ). This could be either a manage.py command, or something in the admin section?
- Make a command to delete `highlevel` rows for these items (one at a time, or in bulk?) so that we can re-generate these highlevel files
- (optional): Add more reporting so that we can track failed highlevel calculations. We should include the output of the extractor. This could be a sentry message, or some database rows for "failed highlevel", or a logfile, or just sending stuff to redis temporarily... Any feedback on this would be welcome