AS A WFE sysadmin
I WANT the search server to natively know how to check the ratelimit-server
SO THAT I don't have to have a separate proxy to perform the checking

When the search server receives a request via http://search.musicbrainz.org/ , it should perform rate-limiting. When it receives a request via musicbrainz-server, it should not perform rate-limiting.

Here's how:

Headers

Requests via http://search.musicbrainz.org/ will arrive at the search server with two extra headers,

  X-Apply-Rate-Limit: yes
  X-MB-Remote-Addr: 193.195.43.199

i.e. the client's IP address.

Requests via musicbrainz-server will not say "X-Apply-Rate-Limit: yes" (the header will either be missing, or say "no"). The X-MB-Remote-Addr header may be present.

Config

The search server config will need to include the ratelimit-server endpoint address: ratelimitserver.host and ratelimitserver.port.

Processing Logic

When the search server receives a search request, first determine whether or not we will be applying rate limiting:

• if ratelimitserver.host and/or ratelimitserver.port are not set, then skip rate limiting
• otherwise, read the X-Apply-Rate-Limit header; if it's missing or anything other than "yes", then skip rate limiting
• otherwise, read the X-MB-Remote-Addr header (it should be a dot-quad IP address, e.g. 1.2.3.4); if it's missing or malformed then skip rate limiting
• otherwise, we will apply rate limiting

Next, we apply rate limiting (unless of course we're skipping it):

• construct the ratelimit key, which should be: "search ip=x.x.x.x" (from X-MB-Remote-Addr header)
• test the ratelimit (i.e. ask the ratelimit-server "over_limit search ip=x.x.x.x")
• if the response was "Y" (over limit), then reject with a 503 response, ideally including the current rate / max rate / period in the response somewhere
• otherwise, continue

Next (unless we've already 503'd), serve the search request as normal.

See attached for how to talk to the ratelimit-server.