The entire UI/UX around submitting fingerprints to the AcoustID database hides way too much information. It is estimated that 1% of all AcoustIDs are attached to the wrong recording MBIDs...

https://community.metabrainz.org/t/picard-thinks-some-early-1990s-jungle-techno-sounds-like-m-peoples-how-can-i-love-you-more/547631/19?u=foxgrrl

... but that report referenced above is very limited in what it checks, and I am still finding and correcting incorrect AcoustID->MBID links as I sort out my Psytrance collection. I mean, I guess now that AcoustID has similarity clustering on the server, it may be possible to automatically find the extreme outliers in MBID links. (Like Psytrance recordings linked to folk music MBIDs... I just disabled one exactly like this.) 

For many years, there was no way to know just what exactly happens when you click the "Submit AcoustIDs" button in Picard.  (If you started Picard in Debug mode, you could see what you submitted to the AcoustID server, after the fact.)

An errant drag and drop, unnoticed by the user, or Picard helpfully trying to match audio files to albums with some kind of audio time duration query, and a fuzzy match of some string in an existing tag... and the user has no idea Picard has associated an audio file with the wrong MBID on the right pane. 

When a user has a large collection of files they are tagging, they're probably getting fatigued, and working in batches, which they don't spend much, or any, time really examining the details before hitting the illuminated "Submit AcoustIDs" button. Just trusting that whatever Picard is doing automatically must be the right thing to do.

This was a step in the right direction...

https://tickets.metabrainz.org/browse/PICARD-34

https://github.com/metabrainz/picard/pull/1431

But,

1. The user needs to know to right-click on the "Title|Length|Artist|etc." heading of the right panel, and select "Fingerprint status".
2. I have perfect color vision, and even I have difficulty distinguishing a light-gray colored swirl on a white background, from a red colored swirl on a white background.

The glowing fingerprint icon needs to be much more visible... I mean, not just visibly distinguishable, but also easy for the user to find – perhaps even enabled by default.

Additionally, even having a total count of the number of fingerprints which will be submitted, prior to and if the "Submit AcoustIDs" button gets clicked, I think will go a long way towards catching yourself before submitting a bunch of bogus fingerprints. 

That is, if I'm adding a new album with nine tracks, and I've generated fingerprints (or "Scanned") the nine audio files, put them in the right place, etc. I should expect to see, prominently displayed in the Picard UI, that there are exactly "9", and only nine AcoustID fingerprints ready to go to the big AcoustID server in the sky. If instead, I see that there are "69" AcoustID Fingerprints ready to be submitted, I will realize that there are sixty audio files hidden under a collapsed Album in the right pane, which were from a previous Scan/Lookup of a dozen other albums, and which, the user, hasn't looked at yet, or even realized that Picard thinks these are new AcoustID fingerprint to MBID Recording links to make! 

User Experience

I tend to work on tagging audio in batches of a dozen or so albums at a time. Frequently, I'll try "Scan" first... and then wander off away from my computer for a while... When I later return, I try to remember what I was doing, and then try to figure out what Picard did... Most of the time, Audio files were associated with (MBID) recordings attached to Various Artists compilations. I'm tagging the full album by a specific artist, so I drag all of those files from the right pane, back over to the "Unclustered Files" on the left.

Usually, I then need to actually go to MusicBrainz in a web browser, and add that album (import script from Bandcamp or Discogs or worst case Beatport), add genre tags, import cover art, and then I cut and paste the URL of this new release into the search box in the upper right corner of Picard, hit return. New album appears in the right pane, drag audio files over to new album. Individually "down arrow" select each track and squint at the "Original Value" vs. "New Value" pane, for each track, to make sure it's all correct (and investigate each case if not, etc.) and then.... I click "Submit AcoustIDs" (and then "Save", and now "Submit Acoustic features").

If the earlier matches from the initial "Scan" operations, also not dragged and dropped from the right pane back onto the left pane, are those also (re)-submitted to AcoustID along with the newly created album? 

Sometimes the tiny fingerprint icon is red, after only performing a "Scan" on a cluster in the left pane. Is this a NEW AcoustID, or is this an OLD AcoustID, as it was successfully looked up... (but possibly attached to the wrong album.) I would like to know exactly what is about to happen before I click on "Submit AcoustIDs"

(For safety, I've been moving everything I'm not immediately working on back over to the left pane, because when I want to submit AcoustIDs for my newly added album, I don't want to be messing up AcoustIDs somewhere else!) (Could I have been submitting bad AcoustID->MBID links without realizing it?)

But... there are some more weird situations...

Sometimes, after hitting "Scan", an audio file shows up in a completely random, unrelated, album in the right side pane. Not a Various Artists compilation, but an album of folk music, or chapter 42 of an audio book...

These situations involve looking up that exact recording MBID in a web browser, and then for each of the attached fingerprints (it's almost always at least two) looking each up on the AcoustID website, in a web browser. And then trying to decipher which one(s) are in error. Usually it's pretty obvious if there's 300 submissions with a specific Artist/Title, and one for your AcoustID.

Sometimes, there's only one submission for each possibility, so it's a 50/50 split... but it's still obvious because you know that your audio file (and AcoustID) is the correct one, so the other linked recording must be wrong... but you can't disable it just yet... because you need to be really certain... or at least, unable to get the two mixed up in your head while trying to memorize enough of the UUIDs while looking back and forth between the multiple browser tabs you have open at this point. (Or eyeballing the two bitmaps side-by-side.)

Because... sometimes, the correct audio file you have, that you know is correct, is not actually attached to anything....

So, I need to add a new release, search for it in Picard, drag and drop the audio file into the right spot, on the right, eyeball it for correctness, and THEN click "Submit AcoustIDs", and THEN go back to the browser, refresh all the tabs related to the AcoustID fingerprints... And now I know which links are correct and which aren't, and I can finally "Disable" the incorrect one (after logging into the AcoustID server.)

Whew....

Oh, but wait, there's more...

Sometimes, MusicBrainz does have the album I'm looking for, but there's no AcoustIDs attached to the recordings yet...

So, like most users, I hit the "Lookup" 'magic wand' button, with everything selected, and then walk away from my computer and come back an hour later to try to figure out what happened...

So... Here's a funny thing that happens...

If the user does a "Scan" of all these audio files first, and nothing is found...

and then does a "Lookup"...

Picard will move the audio files over into the right pane, with whichever the first, "closest", search hit was on MusicBrainz, based on whatever minimal clues are present in the audio file itself (and maybe the filename and directory name too?) 

Whatever this match is, may not necessarily be the correct one. (In fact, I've known for certain that MusicBrainz has the correct release, because I added it myself, but the "Lookup" button will not find it and gives me the completely wrong release entirely. I need to manually cut and paste the URL from the release sitting in my web browser, into the search box in Picard, and get it that way. (If the green tagger button is gone, which is usually is by now.))

So... users (who are not me), may just trust that whatever Picard has given them is the correct data, and not spend the time tediously examining each and every track for correctness. (They just want all their stuff tagged in a hurry.) And after all... why would the user ever expect Picard to lie to them? 

Anyway... as I was saying... If you "Scan" first, and then "Lookup", and the audio tracks are on the wrong looked-up release... Picard will think: "Oh, these recordings have new AcoustIDs which are not on the AcoustID server, or linked to any MBID recordings!" and enables the "Submit AcoustIDs" button to link these audio file fingerprints to wherever the files landed in the right side pane. And if the user has a hundred albums, collapsed, in the right pane, they're not going to know or realize that they're submitting bad data when they click that button.

And since clicking the button seems to submit EVERYTHING, even if you are intending on just submitting the one new album you just added... you will be completely unaware of the mess being made.

There is https://picard-docs.musicbrainz.org/en/tutorials/acoustid.html

> The point I was trying to make is that if a recording may have had a rip error (like a CD skip), I assume that can make the fingerprint being looked up not match the recording in the database.

If AcoustID says it does not know the fingerprint you don't know whether this is because your file has an error and hence produces strange fingerprints. The only thing you know is that AcoustID does not know about this fingerprint. Even if the fingerprint is slightly off due to a ripping error there is a big chance AcoustID might still assign it to the correct AcoustID once you submit it. It all depends on how audible this error is and how much influence this takes on the fingerprint. In any way, I don't see a reason to not submit the fingerprint. AcoustID was designed to allow quality differences, e.g. you can also get similar issues with a low bitrate MP3.

> To clarify, I was saying there appears to be no way for me (or any user) to be able to determine if the fingerprint column is red after a scan (indicating to-be-submitted) because the fingerprint lookup request was successful but a match wasn't found, or just because the lookup request failed. 

Yes. Picard will only disable fingerprint submission in two cases:

1. It found a matching recording. In this case the file gets moved to the match automatically. If it would then resubmit the fingerprint for this audio it would just emphasize the match, even if it was invalid. You can however submit the fingerprint if you match the file to another recording or if you run "Generate fingerprints" on the matched file again (both are explicit user actions).
2. After submission: The fingerprint / recording pair will be marked as submitted after submission. Again you can move the file to a different recording or use "Generate fingerprints" to have it back in the submission queue.

So from this point of view there is no difference between AcoustID had no results or AcoustID server could not be reached, the end result is the same (no match).

I agree that we should have better error feedback, currently it is all in the logs. We have PICARD-119 and PICARD-286 for that. But in any case that won't change if a fingerprint can be submitted or not.

When I was asking if there was documentation, I meant something official, like on https://picard-docs.musicbrainz.org/ where others would be likely to find it, not just hidden in a closed ticket.  I think it would be helpful if there was some explanation of these concerns in the official documentation.

I didn't realize that the number of times a fingerprint was submitted for a recording was tracked and helped in the ranking.  Thanks for clarifying that.

> There are two cases: ...

The point I was trying to make is that if a recording may have had a rip error (like a CD skip), I assume that can make the fingerprint being looked up not match the recording in the database.  Or, if one track may have been edited differently on a particular release, it may not match when all the other tracks on the same release do, and really I should see if there is a different release where they all do match.  In both of these cases, it would seem detrimental to submit the fingerprints because the user is potentially polluting the database with either: 1. a bad fingerprint for a bad rip/corrupted file, or 2. a wrong fingerprint that misidentifies one recording as another recording.  But the user won't know that without the acoustid's being looked up, which is only done if a scan was done.

> No, red means Picard has the fingerprint for the file but it is available for submission. If you do a scan on a file and it has a match the fingerprint will be gray, because Picard will not submit fingerprints for files where it got a match (to avoid self-fullfilling prophecies). But if you move this file to another track Picard will put this fingerprint into its submission queue. Fingerprints are also always available for submission if you use the "Generate fingerprints" action.

To clarify, I was saying there appears to be no way for me (or any user) to be able to determine if the fingerprint column is red after a scan (indicating to-be-submitted) because the fingerprint lookup request was successful but a match wasn't found, or just because the lookup request failed.  The problem I was describing was happening with the exact same recordings matched to the exact same tracks in the exact same release, where some tracks would be reporting as matching sometimes, and reported as not matching other times, which I can only assume was because there was some transient failure in the lookup, but that is not transparent/indicated to the user.

> Is there some documentation on exactly how Picard (and the databases it relies on) uses fingerprints vs AcoustID's, what the limitations are, and why?

I tried to explain it in PICARD-2115, there is also https://picard-docs.musicbrainz.org/en/tutorials/acoustid.html?highlight=acoustid

It is also not an either or decision. Both the AcoustID fingerprint and the AcoustID ID are part of the same audio identification system. The fingerprint in itself is not very useful, what makes it useful is being able to compare it to other fingerprints for similarity. To do this you need a database of fingerprints and a system that can do lookups in this database. This is what  the AcoustID service is. It groups fingerprints by what it assumes are fingerprints for basically the same recording. This grouping is what is called an AcoustID. If you give the AcoustID server a fingerprint it can tell you the AcoustID this belongs to, if it knows about the fingerprint.

> The menu item in Picard ("Generate AcoustID fingerprints) seems to suggest they are the same thing.

It's actually explicitly saying "fingerprints" here because it only generates the fingerprints

> But for me, it also seems like most of the functionality of AcoustID's can be provided by Musicbrainz' RecordingID, in that if it's known it is easy to look up the other related info, and if it's not known, it needs to be generated (recalculated) and looked up to find the corresponding RecordingID(s). So why use the AcoustID at all if it's insufficient to uniquely (enough) identify the track?

You are right in so far as that an AcoustID loosely matches the concept of a MB recording. That's not too surprising, because the AcoustID system was explicitly designed to tell apart different recordings. The difference is still that AcoustIDs are separated purely on similarity based on the acoustic analysis, while MB recordings are separated by formal properties defined in guidelines. You can see that this does not always match, as there are both cases where multiple recordings share the same AcoustID and where a single recording has multiple AcoustIDs.

You could imagine a system where fingerprints are just attached to MB recordings, and the system only does allow manual submission to a MB recording and then does a static query on those.

But while the AcoustID system had been designed to closely work with MusicBrainz, it is at the same time independent. It can be used without using MusicBrainz data at all, it will still identify similar fingerprints and group them. Picard does not allow that, but you can submit fingerprints without matching them to specific MB recordings. Also the AcoustID on itself can automatically say that multiple fingerprints identify basically the same recording, and I think that is kind of the core functionality of an audio identification service.

> Right now if I generate ID's, it looks like they are all "unknown" and will all be submitted. 

This is by design. It is basically why the "Generate AcoustID fingerprints" function exists: People wanted to be able to generate the fingerprints on already matched files on the right in order to submit them. There is also nothing wrong with submitting a fingerprint already attached to a recording again. AcoustID keeps track of how often a fingerprint - MB recording combination has been submitted, and this count goes into the evaluation when Picard checks for the best match.

It's also kind of pointless and wasteful to issue an extra request per fingerprint to the AcoustID server to ask it if it already knows this combination. What do you do with this information? There are two cases:

1. It tells you it does not yet know about this combination. In this case you want to submit it, right?
2. It tells you it already knows about this combination? In this case you should also submit it. It helps improve the service. You don't need to of course, but what's the point

Submission to the AcoustID server is an easy single request. If you want to submit 20 fingerprints we can just submit them. No need to do another 20 requests before to get information that will not give any useful information.

What we could have is kind of a submission Wizard, that let's you review the fingerprints that would be submitted together with details to which recording they are matched. Because this is the only thing that is important: You should submit fingerprints only if you have matched them, to the best of your knowledge, to the correct MB recording.

> But if only one or two tracks doesn't match, I have no way to know if that might just be a track that might be a different version (recording) and maybe I just have the wrong release, or maybe I have a rip error that I haven't noticed yet, either of which means I probably don't want to submit them.

You cannot use AcoustID to tell you exactly which release you have. It can only identify on the recording level. If the same recordings are used on different releases AcoustID cannot help you identify that.

> I often find that sometimes a track matches (shows a gray print icon) and sometimes it doesn't (shows as red), which I can only assume is because there was a transient error looking it up. 

No, red means Picard has the fingerprint for the file but it is available for submission. If you do a scan on a file and it has a match the fingerprint will be gray, because Picard will not submit fingerprints for files where it got a match (to avoid self-fullfilling prophecies). But if you move this file to another track Picard will put this fingerprint into its submission queue. Fingerprints are also always available for submission if you use the "Generate fingerprints" action.

> I use the terms "AcoustID" and "fingerprint" interchangeably

This are two completely different things. The AcoustID is a number as you can find it on the MB website in short form like "dd6977".
The full AcoustID would be dd69777f-3431-4d36-822b-0a8192b7ce96 and links to this track on the external website AcoustID.org https://acoustid.org/track/dd69777f-3431-4d36-822b-0a8192b7ce96

Acoustid.org partners with MusicBrainz and combine the acoustic fingerprint of a song with the metadata around this song from MusicBrainz.

The above dd6977 track - for example - has currently 3 fingerprints attached: The 1st with 153 sources, 2nd with 79 sources and 3rd with 1 source. There is a big chance that the 3rd fingerprint is wrongly attached to this track. If 153 different submissions says THIS is the correct fingerprint it is a good indicator.

If you click on the 1st fingerprint https://acoustid.org/fingerprint/17108675 you see the graphical representation of the fingerprint. This fingerprint will be calculated by the cmd-line-tool fpcalc.exe on Windows. If you execute fpcalc.exe on a mp3 track, you will get a very long string looking like this:

FINGERPRINT=AQADtEmSRYmThAz640xwRkfJ73BfQTzz4Ul3XNnR_MdzHG6T40mUByIn6vginBYFa0kWHDqvYNKVIsyqPMWPm8eX41ya4Udz9Dl-5FGgXUeYsAl-43qMKXrEoyLx4dzRLLxwqUTcygGTZGIcQs9ylIe7Qg-yBA8JJ3F7_HnwabhKPBWDeDz0R0hLHV8S4s-S4_mxSyGaPyh5PMlVnE6DRgih-0FcBaVuPMWfwDcmNgqeJDveo_kFRmHCB7Gkg83hB8cFN83RTKtRpj--BFrCJEtQG0-P_MGUyDv65MFxHxfDBr.....

This is already a compressed form of the fingerprint. The raw fingerprint looks like this:

FINGERPRINT=3296468383,3292290447,3292517775,3309221775,1165403807,1207083711,1324462749,1316000605,1311801373,237015085,505454637,505471021,522413069,421713933,959875631,1006537263,985370223,985182927,985174735,979928013,975946649,2032820904,2016027304,1480271528,1484472232,1555570088,3703053800,3703061976,3702007240,3433571656,3433567049,3299361627,3299591978,3303720234,3295265898,1143751902,1143734414,1143881870,1278136718,1283388094....

As @phw already wrote you can always re-calculate this fingerprint locally, without any external help. But if you have to do that for many thousands of your songs, you can think about the time you save, if you have to do that only once and save the calculated fingerprint inside your song - if you don't care that every song will increase it's size by some KB of metadata.

> The AcoustID is not the fingerprint. Having an AcoustID in the tags is mostly unrelated to this...

Is there some documentation on exactly how Picard (and the databases it relies on) uses fingerprints vs AcoustID's, what the limitations are, and why?  All I ever see in Picard is AcoustID's, so I guess I didn't really even realize that the fingerprint is not (as far as Picard's use) essentially the same thing.  The menu item in Picard ("Generate AcoustID fingerprints) seems to suggest they are the same thing.

I use the terms "AcoustID" and "fingerprint" interchangeably, since I don't know what the distinction is as far as how they are used in Picard and Musicbrainz and the databases they use.  In fact, when I click on the "fingerprints" tab on a recording in Musicbrainz, the page I am taken to is titled "Associated AcoustIDs", so even Musicbrainz seems to consider them interchangeable terms.

Personally, I'm fine with having to generate the fingerprint from the audio data rather than storing them as tags in the files, that's easy enough to do and not even that much slower.  But for me, it also seems like most of the functionality of AcoustID's can be provided by Musicbrainz' RecordingID, in that if it's known it is easy to look up the other related info, and if it's not known, it needs to be generated (recalculated) and looked up to find the corresponding RecordingID(s).  So why use the AcoustID at all if it's insufficient to uniquely (enough) identify the track?

I think it would be very good to have some way to (easily) check if a particular fingerprint (from a local file) is already known/associated with a particular track/recording, and also how many fingerprints are already associated with that recording.  Perhaps the submit button should become a two-step process that first checks if the fingerprint is already known for that recording or not (and also report if any of the lookups failed) and only then allows the user to submit them.

Right now if I generate ID's, it looks like they are all "unknown" and will all be submitted.  If the album doesn't have any associated fingerprints set yet, I would almost certainly want to submit them.  But if only one or two tracks doesn't match, I have no way to know if that might just be a track that might be a different version (recording) and maybe I just have the wrong release, or maybe I have a rip error that I haven't noticed yet, either of which means I probably don't want to submit them.  Not having a better indicator of the status of the fingerprints encourages users to just submit them always, which would seem to have the result of potentially polluting the database with possibly bad AcoustID's.

There should also be some indication if there were any errors from the AcoustID lookup, as I often find that sometimes a track matches (shows a gray print icon) and sometimes it doesn't (shows as red), which I can only assume is because there was a transient error looking it up.  I also am guessing that fingerprint lookups often fail, since they seem to have a pretty low limit on how many queries you can perform in succession before they start throttling your lookups.  When albums have lots of tracks, it seems like it's almost impossible to get all the lookups to succeed, but there's also no way that I know of to cause any failed ones retry the lookup.

Just to be clear on a few things here:

• The fingerprint column only is about the fingerprints. It will be filled if a fingerprint is available for a file, which involves running fpcalc on that file.

• Generate fingerprints currently is intentionally a purely local operation. It only runs fpcalc. Enabling of submission is also kind of intentional, as some users wanted to have a way to explicitly be able to mark fingerprints for submission.

• Once the fingerprint has been calculated it is unknown whether this fingerprint has already been submitted or not. To know whether AcoustID already knows about this fingerprint will require an additional web service request per fingerprint.

• Using Scan does generate the fingerprint and does a lookup. If it finds a match it selects the best matching recording (based on metadata) as the result. Only for this result (pair of fingerprint + matched recording ID) it disables submission. This could be expanded to mark all recordings linked to this fingerprint as already submitted.

• A fingerprint, somewhat interestingly, can sometimes match more than one AcoustID. A score is given by the AcoustID server indicating how well the AcoustID matches. Picard considers this when selecting the best match.

• The AcoustID is not the fingerprint. Having an AcoustID in the tags is mostly unrelated to this, except that it somewhat indicates that AcoustID should in theory already know about the file's fingerprint. It doesn't indicate if the AcoustID has been already matched to any specific track. Getting this information again requires a call to the AcoustID service, either by AcoustID or by fingerprint. Fingerprint probably would be the safer. In this case the AcoustID tag does not really provide any usable information and fpcalc + lookup have to be run anyway

• There is a tag to store the fingerprint in files, if present Picard reads it. Again at that point the fingerprint is present, but status is unknown and would require a web request.

• Picard does not write the fingerprint tag, as it was deemed unnecessary as the fingerprint at any time can be calculated from the audio. It also is several kB in size. But I would like to have the option in Picard to also store this optionally. Some users frequently generate fingerprints for their files for submission, and caching the generated fingerprint in the file saves some time (especially if the file resides on a slow storage).

Adding an indicator for if fpcalc fails (as was mentioned in https://tickets.metabrainz.org/browse/PICARD-119 would also be good).