-
Improvement
-
Resolution: Unresolved
-
Low
-
None
-
None
-
None
-
Windows or Linux
According to https://www.mirantis.com/blog/improve-performance-python-programs-restricting-single-cpu/ , because of the GIL Python performs better when restricted to a single processor. The detailed explanation for why can be found in http://www.dabeaz.com/python/UnderstandingGIL.pdf .
In essence:
a. The GIL prevents a python instance executing on multiple processors at the same time even if you are multi-threading.
b. Multiple threads perform worse on a multiprocessor system than on a single processor because threads compete for the GIL.
c. Even a single threaded python app performs worse because it moves from core to core invalidating the processor instruction cache.
By setting processor affinity to a single core, you can improve performance.
(Note: Python 3.2 and later is better at thread switching, but c. still applies.)
My proposal is to use the "affinity" PyPi package (https://pypi.python.org/pypi/affinity , https://github.com/apoikos/affinity ) to restrict Picard to running on a single core (for Windows and Linux).
NOTE 1: Given impending development of Picard2 on Py3.6 with revamped multi-threading (see PICARD-975) the relevance of this should be evaluated.
NOTE 2: On Linux "affinity" uses a C module - not sure what issues that would create for a packaged executable version of Picard.