Uploaded image for project: 'Picard'
  1. Picard
  2. PICARD-1038

Because of Python GIL, set processor affinity to improve performance

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Low Low
    • None
    • None
    • None
    • Windows or Linux

      According to https://www.mirantis.com/blog/improve-performance-python-programs-restricting-single-cpu/ , because of the GIL Python performs better when restricted to a single processor. The detailed explanation for why can be found in http://www.dabeaz.com/python/UnderstandingGIL.pdf .

      In essence:

      a. The GIL prevents a python instance executing on multiple processors at the same time even if you are multi-threading.

      b. Multiple threads perform worse on a multiprocessor system than on a single processor because threads compete for the GIL.

      c. Even a single threaded python app performs worse because it moves from core to core invalidating the processor instruction cache.

      By setting processor affinity to a single core, you can improve performance.

      (Note: Python 3.2 and later is better at thread switching, but c. still applies.)

      My proposal is to use the "affinity" PyPi package (https://pypi.python.org/pypi/affinity , https://github.com/apoikos/affinity ) to restrict Picard to running on a single core (for Windows and Linux).

      NOTE 1: Given impending development of Picard2 on Py3.6 with revamped multi-threading (see PICARD-975) the relevance of this should be evaluated.

      NOTE 2: On Linux "affinity" uses a C module - not sure what issues that would create for a packaged executable version of Picard.

            Unassigned Unassigned
            sophist Sophist
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:

                Version Package