Uploaded image for project: 'BrainzUtils'
  1. BrainzUtils
  2. BU-4

Don't hash cache keys before inserting them

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Normal Normal
    • None
    • Python
    • None

      When we used memcache as a cache backend we took the sha1 of any key and inserted that value into the cache. This is because memcache had a very small size limit on key length and we often ran over it.

      Redis doesn't have this limitation, in fact during the changeover we also considered if it was necessary: https://github.com/metabrainz/brainzutils-python/blob/700bce44255f22c5492e811b7794c89acd71d40e/brainzutils/cache.py#L256

      I believe that going back to plain strings for keys is a good idea because it can help with debugging, and I don't think we are using any extremely large keys in any of our apps.

      To verify that this is the case we could either:

      • Inspect code which uses the cache and come up with an estimate of the largest key size
      • Make a temporary patch tot the cache to add a separate item to the cache on set() that stores the max key length, so that we can inspect it.

      Investigation into the speed of redis seems to indicate that there's not a huge performance penalty with long keys as long as we keep them below ~1000 characters (http://adamnengland.com/2012/11/15/redis-performance-does-key-length-matter/)

            kartik1712 amCap1712
            alastairp Alastair Porter
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:

                Version Package