@phw

I cannot disagree with any of the points you make.

In the end, open source projects like Picard only get the improvements that can be resourced, and that certainly seems to be a limiting factor. Without MeB committing some of its resources to Picard, we are as ever reliant on volunteers and thus only the things that volunteers are keen to work on are likely to get done. And as you say GSoC soaks up mentorship time, so a bit swings and roundabouts.

The V3 release page (which I hadn't previously been aware of - so thanks for pointing it out) already seems to have more things on it than can currently be resourced, and so @aerozol 's UX desired improvements seem to be at best long-long-term.

So on this basis, it would seem that we should (as Philipp suggests) move new items to a V4 release.

The security point I made above is already kinda included in V3 because the new Plugin Management System in PICARD-1861 already has these points covered in the Wiki page it refers to, though I have enhanced my contribution in the Wiki page to give a little more detail.

If there is now consensus on this, we can change the title of this ticket to refer to v4.

Just a few random general notes and some comments on specific things.

First generally I think this ticket is more a Picard v4 scoping discussion. The scope for Picard v3 that we had decided last year is pretty much the Qt6 transition with hopefully some improvements around configuration management and plugin structure (needs to be discussed in detail) [1]. The major version bump is required as Qt6 both requires updates to existing plugins (even though it should be just replacing Qt5 with Qt6 in the imports for most cases) and raises the min. system requirements as discussed above. All the plugins in the picard-plugins repository will be ported over.

Since the release of Picard 2.0 we have done frequent releases, and I'd like to continue this. Version 2.9 is a bit overdue currently, but we'll do it soon. Afterwards I'll want to get the Qt6 branch merged and focus on Picard 3 features. Again, it's not the plan to go into a 2 years hiatus, but I'd like to get a pre-release out timely. Also it is definitely the plan to provide a stable release updated to a newer technology stack. Limiting the scope enables this.

I definitely don't see the multi-process work in v3 (this is a pretty vague thing anyway), and I definitely would like to see any major UI changes being done after a stable release ported to Qt6. Everything else requires someone to commit doing the work in time. We include features we take in if they get ready, as always. There are always some contributions happening from people just getting interested in a specific thing.

So if someone wants to see X in Picard and says they will work on X that's great.

Always keep in mind that this release is not the end. Things discussed above can be done in future releases.

So v3 is the opportunity for bigger changes that may (temporarily) reduce the stability or which may need more  than one iteration to get optimally right.

The idea was to provide a stable version on an updated technology stack, not an experimental one.

I think we need these to understand just how important it will be to be able to back-port improvements from v3 to v2.9, which will potentially be much more difficult if the code base changes in a major way due to rearchitecture.

We will do patch 2.9.x releases as necessary. The plan is definitely not to keep the 2.x branch of Picard on feature parity with 3.x. But if we do 2.x releases and patches are easy to take over there is nothing speaking against doing this. We don't need to or even can decide this now. We will see when we are doing the releases.

The only thing I can say for sure is that we will get requests about users asking about running Picard on Windows 7 or macOS 10.x. And the only thing decided (because I have agreed to do the work) is that we will maintain the 2.9 release for a while if we can continue to support the build infrastructure. At some point this will not be feasible anymore. E.g. once Python 3.8 runs out of support.

Invite participants for Picard v3 as part of GSoC.

We had GSoC contribution last year. This year we also had project ideas, but were not quite convinced by the applications. Also keep in mind that this always also requires time for mentorship. This year I couldn't fully commit to this (but would have co-mentored with zas).

–

[1] See also https://tickets.metabrainz.org/projects/PICARD/versions/12322

Agreed. That's also how I understood your initial comment (just to make sure there is no conceived misunderstanding).

In my experience over the last decade or so, I cannot say that I have experienced any limitation of vision of the main Picard Devs, just:

A) A focus on maintaining the quality of the released application; and

B) A lack of time to do all the things they might like to.

When you have thousands of users and an online update checker in your app, maintaining quality of full releases is essential, and the Picard Devs do a great job of this. If you are going to do some more risky development, a new major version and extensive betas are the way to go. 

@Sophist:

"…rather than limit our vision by only doing what @phw has the time and passion for."

I just wanted to say that I have been very happy and very impressed with how good and stable Picard has become under what seems to be implied here as being outsidecontext's 'limited vision'.

@phw Fair enough.

I think we need to understand A) the new minimum system requirements,

The key factors are upgrading Qt and to a lesser extend Python. For Python for 2.x we currently stay with Python 3.8 to maintain Windows 7 support.

Regarding Qt6 it currently looks like that we will use Qt 6.5 as the basis for our binary builds for macOS and Windows. That means the minimum requirements are what is documented at https://doc.qt.io/qt-6/supported-platforms.html . That means Windows 10 (1809 or later) and Windows 11; macOS 11, 12, 13. See also https://github.com/metabrainz/picard/pull/2154 for some details.

I guess the drop of Windows 7 is justifiable. Also Apple always pushes for newer versions and cares less about older ones. But still there will be users on both platforms using older OS versions.

wonder whether we should support other ARM platforms e.g. RPi 400, Windows.

It's specifically about macOS because all the new Apple machines are ARM based. So people are explicitly asking for support, as currently Picard can only run under emulation on those. Still I can't say for sure how tricky it will be to support, as I don't have access to an Apple ARM machine myself. But theoretically it will "just work" if we use the universal packages for Python 3, PyQt6 and PyInstaller.

RPi is not a big deal, because if you are running it with Linux (which most will do) you already have access to run Picard on it. Either through the distributions package repository or via the Snap package (https://snapcraft.io/picard). Both 32 and 64 bit ARM are supported. Windows ARM would be theoretically possible, as it's one of the platforms supported by Qt6 (see link above). But there are no PyQt6 binary builds available currently, and I don't think the platform is relevant enough to warrant the effort currently.

@zas - Just guessing, but using typical techniques for such things ...

1. Picard Plugins hosted - using public-key cryptographic code signing and hashing techniques (where the private key is held secret by MeB).
2. External plugins - either refuse to load unsigned / wrong hash plugins, or give a BIG red bold underscored FLASHING warning box on load of any unsigned or incorrect hash plugins.

"Security - I am unclear whether we need security improvements to protect against malicious plugins i.e. plugins not hosted by the curated Picard-Plugins repo, code externally injected into plugins downloaded from the Picard-Plugins repo, but we should at least consider whether there is a need."

How is this supposed to work exactly?

• PICARD-2331 - I agree
• PICARD-2192 - I agree, but wonder whether we should support other ARM platforms e.g. RPi 400, Windows.
• PICARD-1861 - I agree. But this will mean that existing plugins will need to have v3 versions created by a central team if we are to avoid users being unable to upgrade because a plugin is not available.
• PICARD-2227 - I have no views about use of YAML, but I agree that a stable, common and O/S standard ways of storing settings including PICARD-522 should be done. Format of a settings file is IMO less important - I would personally vote for whatever has the best programming support libraries.

A couple more ideas for the pot:

• Security - I am unclear whether we need security improvements to protect against malicious plugins i.e. plugins not hosted by the curated Picard-Plugins repo, code externally injected into plugins downloaded from the Picard-Plugins repo, but we should at least consider whether there is a need.
• Separation of core MB download and release choice code - so that it can e.g. easily be reused (in a completely consistent way) in plugins for (or included by developers in) other software (e.g. Plex, MediaMonkey etc.)  

@aerozol will undoubtedly be pushing for UX improvements - whilst I am personally perfectly happy (as a reasonably expert user) with Picard as it is now, I think that many of us already know that UX improvements would make it much easier for newer and less technical users. Obviously, small and independent UX tweaks may not require restructuring, but if the UX improvements are more global/wholesale/significant and require internal restructuring, then my worry with not including them in V3 is that they will need to wait for V4 which could then be several years away.

(@aerozol and I had a brief discussion a couple of hours ago about what UX issues exist and what the solutions might need to be - it was curtailed because of time-zone differences, but some of these were pretty significant, and I hope that these will get some visibility in the not too distant future. But from this discussion I do have a feeling that @aerozol, as the MeB UX expert, will be pushing for significant UX changes rather than minor ones.)

Regarding your wish to continue supporting 2.9 for some time, I think we need to understand A) the new minimum system requirements, and B) how these will impact the existing user base. Do we get USER-AGENT stats from MB that can tell us what platforms, O/S versions, python versions, package vs. source code instances etc. are for the existing user base that can inform us how many users are on old hardware that cannot be supported by Qt6? I think we need these to understand just how important it will be to be able to back-port improvements from v3 to v2.9, which will potentially be much more difficult if the code base changes in a major way due to rearchitecture.

(In the absence of these stats, my view is that aside from bug fixes, v2.9 should be functionally stabilised once we start on the V3 development. The move to Qt6 is presumably due to Qt5 support coming to an end, and presumably many of the underlying technologies supported by Qt5 are also coming to the end of their support. I am unclear why we should continue to support older unsupported environments with new functionality when A) those users should be considering upgrades for other reasons and B) those users can continue to use Picard 2.9 for an open ended period,) 

OTOH, the more major changes we include in V3, the longer it will take and the greater the risk that it doesn't make the light of day. Then again, the more architectural decoupling that can be included, the easier it will be to make significant changes later in V3 without requiring a V4.

Finally as with all open source projects that are delivered by volunteers, whatever is decided needs to have people committing to put the effort in to make it happen - and any great ideas which don't get a volunteer commitment to delivery will have to be left for another day. And this will particularly be  the case if the only v3 dev is @phw. BUT... @aerozol does have a point about a UX so perhaps we should be trying to gather additional volunteer development resource (ideally with Picard dev experience) to deliver a significant new functionality rather than limit our vision by only doing what @phw has the time and passion for.

To be honest I would like to see Picard 3 not getting too big. Rather setting the technical basis for future changes. Apart from usual improvements and smaller features I think we should look into the following:

• PICARD-2331: Update to PyQt6. This is mandatory and actually the reason for the major version bump.
• PICARD-2192: Native ARM builds for macOS. This requires the PyQt6 update.
• PICARD-1861 and related: Update the plugin system. As the switch to PyQt6 will break many plugins and require them to be updated anyway it would be the perfect time to do some needed changes to the plugin system overall.
• PICARD-2227: YAML based configuration files. Not absolutely mandatory, but as we had some trouble with QSettings and because it is a breaking change it would be good to include this. Also it must consider PICARD-522 . So have one major update handling all this and be done.

The above is something we can manage in a timely fashion, which is required to continue supporting newer OS versions. One big downside is that the Qt6 upgrade raises minimum system requirements. So I'd like to see Picard 2.9 continued to be supported with fixes at least, maybe even receive new features if they can be backported.

Apart from that I think we should not necessarily put much more larger changes into this release. As always we can include whatever gets ready. But I don't think we should delay the release for finishing larger tasks. I'm especially against a longer unstable period here. Let's try to get a technically updated Picard 3 and afterwards we can do all kinds of stuff. It would be great to have at least a pre-release available this year.

But of course, if someone starts working on any UI rework and things get ready in time let's include it

Here is my own starter list:

1. Improved UX - as discussed in PICARD-2315 and (less appropriately) in PICARD-2644. The scope of this is somewhat nebulous - at one end of the scale it could be a complete redesign of the UX, and at the other a set of independent minor improvements. I nominate @aerozol to lead this discussion.
2. Multi-process architecture - to get around the limitations of utilising multi-CPU hardware due to the GIL. (Note: CPython long term strategy is to eliminate the GIL - a PoC has already been demonstrated.)
3. Alternative Python implementations - switch to a different Python implementation or Mojo to eliminate performance bottlenecks. (Note: I am not clear whether CPU is actually a bottleneck for most users rather than disk I/O or memory or network API calls.)
   Note: continuing to support dynamic / external plugins might prevent this.
4. Plugin architecture - try to divide Picard into a core kernel and a series of internal plugins. So, e.g. different file types might be defined in plugins, etc. Possible benefits - looser code dependence, possible enabler for multi-process architecture. (This idea comes from personal experience of how other more recently developed (Python) tools are architected.)