Uploaded image for project: 'Picard'
  1. Picard
  2. PICARD-1878

Inefficient reading of tracks leads to slow saving on some file systems and network shares

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: 2.3.2, 2.4.0b2
    • Fix Version/s: 2.4
    • Component/s: None
    • Labels:
      None
    • Environment:
      Windows 10 with slower (non SSD) data access

      Description

      I have seen this problem for years and I think I have finally tracked it down.

      In the process of updating the metadata and saving/moving the tracks, it starts by reading the original file backwards in 16KB chunks (FLAC in my tests).

      This causes very slow behavior in the following conditions:
      1. Mechanical disk (e.g., slow seek times).
      2. Files not cached.

      Since it is reading the files backwards, the natural read ahead on the file systems does not help. Analyzing one file, it started by reading backwards and every 16KB chunk was taking ~8mSec. After it finished that read, it wrote the file extremely quickly (probably limited by disk speed).

      If I manually copy the tracks (to force them in the cache), or I put them on an SSD, the rewrite speeds are good.

      On a local disk, I am seeing ~3MB/sec read speeds when this is happening. I see the same behavior if the backward reading is done over a network connection (SMB) to mechanical disks without cache as well.

      Can we update Picard to read the file forwards?

        Attachments

        1. compiled_save.png
          compiled_save.png
          562 kB
        2. fallback_move_2MB_BUFFER_SIZE.png
          fallback_move_2MB_BUFFER_SIZE.png
          172 kB
        3. image-2020-07-11-16-59-40-676.png
          image-2020-07-11-16-59-40-676.png
          119 kB
        4. image-2020-07-11-22-18-36-816.png
          image-2020-07-11-22-18-36-816.png
          465 kB
        5. image-2020-07-15-21-37-37-895.png
          image-2020-07-15-21-37-37-895.png
          95 kB
        6. image-2020-07-15-21-40-59-109.png
          image-2020-07-15-21-40-59-109.png
          91 kB
        7. InitialFetch.png
          InitialFetch.png
          64 kB
        8. Procmon.png
          Procmon.png
          495 kB
        9. SaveEnd.png
          SaveEnd.png
          143 kB
        10. SaveStart.png
          SaveStart.png
          152 kB

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              thebradleys Jami Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:

                  Packages

                  Version Package
                  2.4