-
Improvement
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
We should implement some kind of adaptive rate limiting, so clients do not have to hardcode the 1 request per second rule.
One way to do this is by using a few HTTP headers to inform a client how many requests they have remaining, e.g.:
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 59
This seems fairly common (see: http://developer.github.com/v3/#rate-limiting , https://dev.twitter.com/docs/rate-limiting/1.1 , https://www.discogs.com/developers/accessing.html , etc..), and most of these APIs use some kind of limit/remaining scheme. But there are probably alternative ways to communicate a rate limit to a client through HTTP headers, we should probably look at some alternatives as well.
Being able to dynamically tweak these values could help us deal with temporary heavy loads (e.g. enforce a temporary decrease in request rates when one or more servers are out of rotation, etc..).
We should implement these headers ASAP, even if they'd be largely dummy values at first. Because clients are slow to update their software and get the updated versions in the hands of users, we should encourage them to implement using these headers now so we can benefit from it when we do a proper implementation of them in the server later on.
[MBS-5874] Implement adaptive rate limiting
We must also specify the time (it seems to be with that `X-RateLimit-Reset: 60` thing).
If I translate the 1 request per second, isn't it giving the following ?
X-RateLimit-Remaining: 0 X-RateLimit-Limit: 1 X-RateLimit-Reset: 60
Looks rather binary, though...
Maybe I write nonsense...
I'm trying to understand how it works (as a user script writer point of view).
I like it if we have an easy soft way to manage those limits, instead of hard coding but this system I don't understand fully yet.
I thought a simple `next-request-ok-in: X ms` header would be more stratightforward for us script users (together with the retry-after too).
BTW 1 req/sec is rather slow.
We've discussed this before - could you find and link prior discussions?
I doubt we can change the return code until /ws/3, but yes – 429 should be used: MBS-5827
The response to a request which is denied by the rate limiter should be 429 and have a Retry-After header (like http://developers.app.net/docs/basics/rate-limits/ ).
previous irc discussions related to this ticket:
http://chatlogs.musicbrainz.org/musicbrainz-devel/2012/2012-09/2012-09-26.html#T19-00-18-381762
http://chatlogs.musicbrainz.org/musicbrainz-devel/2013/2013-02/2013-02-14.html#T06-46-05-257922