The next release will be Picard 2.3.0 beta 1. I still want to get this done in January

Thank you for this update, @Philipp Wolfer

Which is the next Picard version including this changes?

With PICARD-1625 we fix some freezing of the UI when clustering and moving files in the UI. This was especially noticable when moving a lot of files to hige releases with many tracks (such as the "Barock masterpieces"), which could easily freeze the UI for minutes. This should fix the issue you reported for step d).

I'll close this issue as it no longer provides much value. I think we have addressed the specific issues here. That's not saying all performance issues are resolved, but we have a couple of several other performance related tickets (mostly around saving files) and if there is a specific bottle neck detected we should have a new ticket for this (e.g. we have one ticket specifically about improving clustering performance, see PICARD-1432).

Thanks a lot for retesting, this is reassuring. Also a bit embarrassing that we lost that code optimization in the Windows and macOS builds and did not notice

We should be able to do something about the responsiveness during d), I will take a look (PICARD-1625).

Todays result with version "picard-setup-2.2.2.dev1_python3.7.4_20191002195345" and "Baroque Masterpieces (60 CD Set BOX)". All files are stored locally on a internal SSD. Windows 10 is running on the same internal SSD.

a) Drag & Drop 1'449 files in 122 folders from windows file explorer into the upper left (unclustered) part of Picard:
20 seconds (= -10 Seconds to my february test).

b) Press Cluster-Button to get 60 "clusters" or 60 subfolders with the content of the 60 CDs:
2 seconds (-5 Seconds, feels much faster then before!)

Copy & paste 44f53f7b-7a0c-4796-b06c-1077327b13ff in the upper right album text search box

c) Press Enter to get the release "Baroque Masterpieces (0/1142; 0 images)": 4 seconds (Unchanged)

d) Drag & Drop the 60 clustered subfolders from the left to the right side: 3min10sec!!! (What a great boost!)

It seems that this fix really speed up things!

There is one remaining big point: 

You can't see any progress during the step d). You only see a completely frozen GUI, the windows 10 taskmanager shows:

That would be great. The currently latest build with all changes included is available at https://ci.appveyor.com/project/MetaBrainz/picard/build/job/wqyrpefdnaoheks5/artifacts

I hope this will improve performance for you, I'm curious about your feedback. It probably has the biggest impact while clustering, but also the general matching should be affected. This should not make Picard suddenly be super fast in processing your specific collection, there are other bottlenecks in place. But I hope for better responsiveness and some improvement on processing speed.

Just let me know where I can download a build including this very reasonable fix and test it.

We just found out that the Windows and macOS packages did not include the C version of Picard's astrcmp module used for string comparison. This is heavily used while clustering, but also at other places where metadata gets compared.

If this C module does not exist Picard falls back to a much slower Python implementation. This has two effects:

1. It hurts plain execution time, because the Python version is multiple times slower then the C version
2. It hurts responsiveness, because only a single thread can execute Python code at a time due to the GIL. And the Python version obviously runs completely in the Python interpreter, while the C version tries to do as much as possible outside a Python context.

This will be fixed with the next release, see PICARD-1618

This can partially explain the performance issues you experienced and why I could not fully reproduce them in my development builds. There might be more to it, hence I keep this issue open for now. But I consider it partially solved by PICARD-1618

Another big collection to see how slow the GUI can be:

https://beta.musicbrainz.org/release/d572e17d-10ef-47f1-bd39-9d1406a252d0

Can you give me some details on the existing metadata of your files?