Uploaded image for project: 'MusicBrainz Search Server'
  1. MusicBrainz Search Server
  2. SEARCH-161

Search server should consult ratelimit-server

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2011-12-24
    • Labels:
      None

      Description

      AS A WFE sysadmin
      I WANT the search server to natively know how to check the ratelimit-server
      SO THAT I don't have to have a separate proxy to perform the checking

      When the search server receives a request via http://search.musicbrainz.org/ , it should perform rate-limiting. When it receives a request via musicbrainz-server, it should not perform rate-limiting.

      Here's how:

      Headers

      Requests via http://search.musicbrainz.org/ will arrive at the search server with two extra headers,

        X-Apply-Rate-Limit: yes
        X-MB-Remote-Addr: 193.195.43.199
      

      i.e. the client's IP address.

      Requests via musicbrainz-server will not say "X-Apply-Rate-Limit: yes" (the header will either be missing, or say "no"). The X-MB-Remote-Addr header may be present.

      Config

      The search server config will need to include the ratelimit-server endpoint address: ratelimitserver.host and ratelimitserver.port.

      Processing Logic

      When the search server receives a search request, first determine whether or not we will be applying rate limiting:

      • if ratelimitserver.host and/or ratelimitserver.port are not set, then skip rate limiting
      • otherwise, read the X-Apply-Rate-Limit header; if it's missing or anything other than "yes", then skip rate limiting
      • otherwise, read the X-MB-Remote-Addr header (it should be a dot-quad IP address, e.g. 1.2.3.4); if it's missing or malformed then skip rate limiting
      • otherwise, we will apply rate limiting

      Next, we apply rate limiting (unless of course we're skipping it):

      • construct the ratelimit key, which should be: "search ip=x.x.x.x" (from X-MB-Remote-Addr header)
      • test the ratelimit (i.e. ask the ratelimit-server "over_limit search ip=x.x.x.x")
      • if the response was "Y" (over limit), then reject with a 503 response, ideally including the current rate / max rate / period in the response somewhere
      • otherwise, continue

      Next (unless we've already 503'd), serve the search request as normal.

      See attached for how to talk to the ratelimit-server.

        Attachments

          Activity

            People

            • Assignee:
              ijabz Paul Taylor
              Reporter:
              djce Dave Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Packages

                Version Package
                2011-12-24