Uploaded image for project: 'Picard'
  1. Picard
  2. PICARD-840

Warn people who try to load very large amounts of files

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • None
    • User Interface

      Fairly often we get messages from users like "I loaded 50k files on Picard and it's going very slowly". We should have some sort of alert for these cases, possibly warn when someone tries to load more than 5k or 10k files in one go and ask "Are you sure you want to load so many files? Picard might run very slowly" or whatnot.

          [PICARD-840] Warn people who try to load very large amounts of files

          I'd be in favor of closing this as I think PICARD-2671 and PICARD-2676 provide a better solution.

          Philipp Wolfer added a comment - I'd be in favor of closing this as I think PICARD-2671 and PICARD-2676 provide a better solution.

          Invisible Man added a comment - - edited

          I would add another reason, why such a warning should be introduced:

          https://community.metabrainz.org/t/this-app-trashed-weeks-of-tedious-exacting-work/642937

          People blaim Picard not only because of slow loading and processing large number of tracks, but also because they don't understand how the process works to recognize a (perfect matching) release.

          In addition to @Aerozol's popup text, I suggest to also show something like this:

          You try to recognize a lot of different tracks from different albums. Picard is designed to recognize files grouped by album. We recommend to follow this workflow instructions for best results:

          https://picard-docs.musicbrainz.org/en/workflows/workflows.html

           

          The mentioned workflow documentation page also includes "...strongly recommended that you make a backup copy of the files being processed and initially process a copy of your music files."

           

           

          Invisible Man added a comment - - edited I would add another reason, why such a warning should be introduced: https://community.metabrainz.org/t/this-app-trashed-weeks-of-tedious-exacting-work/642937 People blaim Picard not only because of slow loading and processing large number of tracks, but also because they don't understand how the process works to recognize a (perfect matching) release. In addition to @Aerozol's popup text, I suggest to also show something like this: You try to recognize a lot of different tracks from different albums. Picard is designed to recognize files grouped by album. We recommend to follow this workflow instructions for best results: https://picard-docs.musicbrainz.org/en/workflows/workflows.html   The mentioned workflow documentation page also includes "... strongly recommended that you make a backup copy of the files being processed and initially process a copy of your music files."    

          Sophist added a comment - - edited

          I am a user of MediaMonkey which is to media players what Picard is to media taggers, and MediaMonkey maintains a (SQLite) database of my music library and its metadata, and it doesn't take ages to load the whole library.

          But fundamentally, MediaMonkey serves a different purpose to Picard. MediaMonkey needs to know about every file in the library so that you can choose what to play, but it doesn't need to re-read the metadata to check that it hasn't changed since it last indexed it; Picard's purpose is solely to tag specific files from your library that you choose to retag, and it has to read the metadata every time to see whether the new MB metadata has changed.

          I suppose that Picard could maintain a parallel database files (location and timestamp and possibly filesize) and the metadata contained therein, and then it would only need to read the metadata afresh from the music files if the file is new or if the timestamp/size are different, using the metadata in the database to populate the records for all music files it already knows about. I would expect this to be substantially faster to load files.

          I am unclear personally whether there is really a need for this sort of enhancement, but if so this would be a non-trivial development, and it would therefore need A) agreement from the primary Picard developers that such an enhancement would be merged, and B) one or more volunteers to code it.

          Sophist added a comment - - edited I am a user of MediaMonkey which is to media players what Picard is to media taggers, and MediaMonkey maintains a (SQLite) database of my music library and its metadata, and it doesn't take ages to load the whole library. But fundamentally, MediaMonkey serves a different purpose to Picard. MediaMonkey needs to know about every file in the library so that you can choose what to play, but it doesn't need to re-read the metadata to check that it hasn't changed since it last indexed it; Picard's purpose is solely to tag specific files from your library that you choose to retag, and it has to read the metadata every time to see whether the new MB metadata has changed. I suppose that Picard could maintain a parallel database files (location and timestamp and possibly filesize) and the metadata contained therein, and then it would only need to read the metadata afresh from the music files if the file is new or if the timestamp/size are different, using the metadata in the database to populate the records for all music files it already knows about. I would expect this to be substantially faster to load files. I am unclear personally whether there is really a need for this sort of enhancement, but if so this would be a non-trivial development, and it would therefore need A) agreement from the primary Picard developers that such an enhancement would be merged, and B) one or more volunteers to code it.

          Aerozol added a comment -

          A note on:

          "It seems to be rather common that people discover a new tagging tool, load their entire library into it and then just hit save, trusting it blindly without even knowing what the tool can do "

          That's obviously kinda crazy (especially the hitting 'save' button), but to be honest I don't have any other software on my computer that expects me to load or do things in batches. All my players, streaming services etc - would be more likely to complain if I didn't load it all at once. Because we know the ins and outs of tagging we know it's crazy to expect the same to happen there - but I have to say that Picard (or, more accurately, the task it's doing) is the outlier here, rather than the users.

          Aerozol added a comment - A note on: "It seems to be rather common that people discover a new tagging tool, load their entire library into it and then just hit save, trusting it blindly without even knowing what the tool can do  " That's obviously kinda crazy (especially the hitting 'save' button), but to be honest I don't have any other software on my computer that expects me to load or do things in batches. All my players, streaming services etc - would be more likely to complain if I didn't load it all at once. Because we know the ins and outs of tagging we know it's crazy to expect the same to happen there - but I have to say that Picard (or, more accurately, the task it's doing) is the outlier here, rather than the users.

          Aerozol added a comment - - edited

          I had made a ticket and some mockups for a 'new user mode' here: PICARD-2315
          I'm not 100% sure the improvements couldn't be incorporated into Picard default, but it was a good exploration of what could be done.

          In the meantime, a simple popup like:

          "You are loading a large amount of files.
          You may experience slow down when performing certain tasks - we recommend working with smaller batches.

          [cancel] [ignore]
          don't show again [ ]"

          Would immediately knock out a significant portion of the complaints/negative feedback we get.

          Having the warning display upon lookup/scan/save etc would be more technically accurate though, as that's where the issues arise*, but we may as well nip it in the bud. Potentially it wouldn't hurt to double insulate by displaying the popup again when people click on lookup/scan/save, on a ridiculous amount of files. Would also guard against accidental clicks - for experienced users too!

          *I just test loaded TB+ collection into Picard, beautiful work tīmu, smooth as.

          Aerozol added a comment - - edited I had made a ticket and some mockups for a 'new user mode' here: PICARD-2315 I'm not 100% sure the improvements couldn't be incorporated into Picard default, but it was a good exploration of what could be done. In the meantime, a simple popup like: "You are loading a large amount of files. You may experience slow down when performing certain tasks - we recommend working with smaller batches. [cancel] [ignore] don't show again [ ]" Would immediately knock out a significant portion of the complaints/negative feedback we get. Having the warning display upon lookup/scan/save etc would be more technically accurate though, as that's where the issues arise*, but we may as well nip it in the bud. Potentially it wouldn't hurt to double insulate by displaying the popup again when people click on lookup/scan/save, on a ridiculous amount of files. Would also guard against accidental clicks - for experienced users too! *I just test loaded TB+ collection into Picard, beautiful work tīmu, smooth as.

          Sophist added a comment - - edited

          Perhaps what we need is to have two UI modes - a New User mode which simplifies the UI and has some limitations (e.g. on numbers of files) perhaps with some sort of Wizard or pop-up tips to help people learn the basics, and a Standard mode i.e. as now.

          Edited: to add a ^5 to @Aerozol who had created PICARD-2315 with this idea over 18 months ago.

          Sophist added a comment - - edited Perhaps what we need is to have two UI modes - a New User mode which simplifies the UI and has some limitations (e.g. on numbers of files) perhaps with some sort of Wizard or pop-up tips to help people learn the basics, and a Standard mode i.e. as now. Edited: to add a ^5 to @Aerozol who had created PICARD-2315 with this idea over 18 months ago.

          > If Picard is capable of loading large numbers of albums, then IMO no warning is needed.

          That was also my stance on this for a long time. But actually I realized this is not only about technical limitations. There was great progress in allowing Picard to handle larger files (even though there are still a few performance bottlenecks, but overall you can if you want load thousands of files). But the other issue is usage by new users.

          It seems to be rather common that people discover a new tagging tool, load their entire library into it and then just hit save, trusting it blindly without even knowing what the tool can do So maybe some warning message for this use would be useful. Definitely with a "don't show me this again" checkbox.

          Philipp Wolfer added a comment - > If Picard is capable of loading large numbers of albums, then IMO no warning is needed. That was also my stance on this for a long time. But actually I realized this is not only about technical limitations. There was great progress in allowing Picard to handle larger files (even though there are still a few performance bottlenecks, but overall you can if you want load thousands of files). But the other issue is usage by new users. It seems to be rather common that people discover a new tagging tool, load their entire library into it and then just hit save, trusting it blindly without even knowing what the tool can do So maybe some warning message for this use would be useful. Definitely with a "don't show me this again" checkbox.

          Sophist added a comment -

          If Picard is capable of loading large numbers of albums, then IMO no warning is needed.

           

          However, if the queue of things (file loads or network tasks) is going to take a long time, then an ability to cancel the queue in a clean and elegant way might be useful. 

          Sophist added a comment - If Picard is capable of loading large numbers of albums, then IMO no warning is needed.   However, if the queue of things (file loads or network tasks) is going to take a long time, then an ability to cancel the queue in a clean and elegant way might be useful. 

          Aerozol added a comment -

          I think this is high priority - this still causes trouble and lack of faith in Picard on a much bigger scale than most other issues (e.g. PICARD-2644). There's no reason for people to assume that Picard wouldn't handle adding am entire 'music' folder gracefully, or that that's not what they're expected to do.

          Aerozol added a comment - I think this is high priority - this still causes trouble and lack of faith in Picard on a much bigger scale than most other issues (e.g. PICARD-2644 ). There's no reason for people to assume that Picard wouldn't handle adding am entire 'music' folder gracefully, or that that's not what they're expected to do.

          Jason Condo added a comment - - edited

          Can you add a cancel current task option so that if it becomes hung on lookup or clustering of a large number of files, the process can be interrupted and th user can select a smaller number of files to execute the scan on?

          Jason Condo added a comment - - edited Can you add a cancel current task option so that if it becomes hung on lookup or clustering of a large number of files, the process can be interrupted and th user can select a smaller number of files to execute the scan on?

            samj1912 Sambhav Kothari
            reosarevok Nicolás Tamargo
            Votes:
            11 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:

                Version Package