Continuing on from the open questions raised in MBH-512

MetaBrainz has no policy on when to upgrade the base docker images used in all of our projects (covering new versions of ubuntu, dependent packages in the baseimage, security updates in ubuntu packages), or when to upgrade direct python/js dependencies (security issues, new major versions)

In our case this means quite a few things:

• Ubuntu security updates and changes to the phusion baseimage: https://github.com/metabrainz/base-image go unapplied. For ubuntu packages this means that some security fixes may not be applied, however the majority of our dependencies are installed directly by pip/npm, and so this might not be a very important point to consider
• If we updated ubuntu packages more frequently, this has the possibility to cause part of the metabrainz stack to break (e.g. imagine for some reason a new version of runit is released in a new package and a bug in our config files causes all of the services to not start... [a pretty extreme case, but perhaps it could happen?])
• We build our own version of python (https://github.com/metabrainz/docker-python, I guess so that we can have a more up-to-date version of python than is available in the ubuntu LTS), but we've not been keeping up to date with python releases. We should consider if we want to track these point releases and update them as soon as possible. When making this update, should we immediately update all metabrainz apps that use this version of python, or wait until the maintainer has time? Should there be a list somewhere of "out of date" apps that are not on the current point-release? How will we make sure that this list itself doesn't become out of date?
• Github has started to give security alerts for python and js packages. Should we have a policy of updating these packages as soon as possible? Sometimes the version which fixes a bug could be a new major version, which means we may have to be careful when applying it
• When should we upgrade other out-of-date python and js packages? Should we be using a tool to tell us immediately when there is a new version released, or only update these packages periodically (every 6 months, or when needed)? What happens when it's a major version update? For example, we've seen issues twice in AB in the last few months regarding upgrades to Flask (https://github.com/metabrainz/acousticbrainz-server/pull/313) and to Click (https://github.com/metabrainz/acousticbrainz-server/pull/318). This means that we should have a tradeoff of upgrading as soon as possible, but not leaving the upgrade for so long that we get behind on (for example) security updates to our dependencies and then having to spend a major effort upgrading packages.
• Should we upgrade to new versions of python or node as soon as they are released, or only when we see a need to upgrade? What does that mean for project sites that are still on python 3.5? Should we upgrade all projects to python 3.6 or 3.7 at the same time? AB is using node 8 because node10 has issues with one of its dependencies. Should we be happy with node8 (which is LTS and so still receiving security updates), or prioritise upgrading our tools so that we can install the latest and greatest node?

Lots of complex questions. I'm happy to talk about ideas!