• Icon: New Feature New Feature
    • Resolution: Fixed
    • Icon: Normal Normal
    • 2011-12-24
    • None
    • None
    • None

      AS A WFE sysadmin
      I WANT the search server to natively know how to check the ratelimit-server
      SO THAT I don't have to have a separate proxy to perform the checking

      When the search server receives a request via http://search.musicbrainz.org/ , it should perform rate-limiting. When it receives a request via musicbrainz-server, it should not perform rate-limiting.

      Here's how:

      Headers

      Requests via http://search.musicbrainz.org/ will arrive at the search server with two extra headers,

        X-Apply-Rate-Limit: yes
        X-MB-Remote-Addr: 193.195.43.199
      

      i.e. the client's IP address.

      Requests via musicbrainz-server will not say "X-Apply-Rate-Limit: yes" (the header will either be missing, or say "no"). The X-MB-Remote-Addr header may be present.

      Config

      The search server config will need to include the ratelimit-server endpoint address: ratelimitserver.host and ratelimitserver.port.

      Processing Logic

      When the search server receives a search request, first determine whether or not we will be applying rate limiting:

      • if ratelimitserver.host and/or ratelimitserver.port are not set, then skip rate limiting
      • otherwise, read the X-Apply-Rate-Limit header; if it's missing or anything other than "yes", then skip rate limiting
      • otherwise, read the X-MB-Remote-Addr header (it should be a dot-quad IP address, e.g. 1.2.3.4); if it's missing or malformed then skip rate limiting
      • otherwise, we will apply rate limiting

      Next, we apply rate limiting (unless of course we're skipping it):

      • construct the ratelimit key, which should be: "search ip=x.x.x.x" (from X-MB-Remote-Addr header)
      • test the ratelimit (i.e. ask the ratelimit-server "over_limit search ip=x.x.x.x")
      • if the response was "Y" (over limit), then reject with a 503 response, ideally including the current rate / max rate / period in the response somewhere
      • otherwise, continue

      Next (unless we've already 503'd), serve the search request as normal.

      See attached for how to talk to the ratelimit-server.

          Loading...

            • Icon: New Feature New Feature
            • Resolution: Fixed
            • Icon: Normal Normal
            • 2011-12-24
            • None
            • None
            • None

              AS A WFE sysadmin
              I WANT the search server to natively know how to check the ratelimit-server
              SO THAT I don't have to have a separate proxy to perform the checking

              When the search server receives a request via http://search.musicbrainz.org/ , it should perform rate-limiting. When it receives a request via musicbrainz-server, it should not perform rate-limiting.

              Here's how:

              Headers

              Requests via http://search.musicbrainz.org/ will arrive at the search server with two extra headers,

                X-Apply-Rate-Limit: yes
                X-MB-Remote-Addr: 193.195.43.199
              

              i.e. the client's IP address.

              Requests via musicbrainz-server will not say "X-Apply-Rate-Limit: yes" (the header will either be missing, or say "no"). The X-MB-Remote-Addr header may be present.

              Config

              The search server config will need to include the ratelimit-server endpoint address: ratelimitserver.host and ratelimitserver.port.

              Processing Logic

              When the search server receives a search request, first determine whether or not we will be applying rate limiting:

              • if ratelimitserver.host and/or ratelimitserver.port are not set, then skip rate limiting
              • otherwise, read the X-Apply-Rate-Limit header; if it's missing or anything other than "yes", then skip rate limiting
              • otherwise, read the X-MB-Remote-Addr header (it should be a dot-quad IP address, e.g. 1.2.3.4); if it's missing or malformed then skip rate limiting
              • otherwise, we will apply rate limiting

              Next, we apply rate limiting (unless of course we're skipping it):

              • construct the ratelimit key, which should be: "search ip=x.x.x.x" (from X-MB-Remote-Addr header)
              • test the ratelimit (i.e. ask the ratelimit-server "over_limit search ip=x.x.x.x")
              • if the response was "Y" (over limit), then reject with a 503 response, ideally including the current rate / max rate / period in the response somewhere
              • otherwise, continue

              Next (unless we've already 503'd), serve the search request as normal.

              See attached for how to talk to the ratelimit-server.

                    ijabz Paul Taylor
                    djce Dave Evans
                    Votes:
                    0 Vote for this issue
                    Watchers:
                    0 Start watching this issue

                      Created:
                      Updated:
                      Resolved:

                        Version Package
                        2011-12-24

                          ijabz Paul Taylor
                          djce Dave Evans
                          Votes:
                          0 Vote for this issue
                          Watchers:
                          0 Start watching this issue

                            Created:
                            Updated:
                            Resolved:

                              Version Package
                              2011-12-24