Uploaded image for project: 'MetaBrainz Hosting'
  1. MetaBrainz Hosting
  2. MBH-261

"Advisory rate-limit" for python-musicbrainz-ngs

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Normal Normal
    • mrtg / webstats
    • None

      Currently we have several of what I'll call "advisory rate-limits" – e.g. http://stats.musicbrainz.org/mrtg/drraw/drraw.cgi?Mode=view;Template=1262469754.29207;Base=%2Fvar%2Fwww%2Fmrtg%2F%2Flenny_ratelimit-default-picard-rate.rrd for picard. python-musicbrainz-ngs would be a nice one to add, especially since headphones is using it nowadays, and they're back to 33% of our traffic with it (and another 14% with the old UA string): http://stats.musicbrainz.org/webstats/zaphod/musicbrainz-full/usage_201207.html#TOPAGENTS

      python-musicbrainz-ngs has a UA string of the format: "<specific usage details> python-musicbrainz-ngs/<version> ( <url/email for the library using pymbngs> )" or "<specific usage details python/musicbrainz-ngs/<version>" (the latter when no contact info is provided).

      Other client libraries may also fit into this category, e.g. libmusicbrainz4/5 (with UA format "<details> libmusicbrainz

      {4,5}

      /<version> <neon version details>").

      Just a wishlist item for me, but it could be useful!

          [MBH-261] "Advisory rate-limit" for python-musicbrainz-ngs

          Ian McEwen added a comment -

          Unsure how this is supposed to play out – especially given the '-' in the name. I'm seeing data but incorrect titles and such at http://stats.musicbrainz.org/mrtg/drraw/drraw.cgi?Mode=view;Template=1262469543.28951;Base=%2Fvar%2Fwww%2Fmrtg%2F%2Flenny_ratelimit-default-python-musicbrainz-ngs-count.rrd

          Ian McEwen added a comment - Unsure how this is supposed to play out – especially given the '-' in the name. I'm seeing data but incorrect titles and such at http://stats.musicbrainz.org/mrtg/drraw/drraw.cgi?Mode=view;Template=1262469543.28951;Base=%2Fvar%2Fwww%2Fmrtg%2F%2Flenny_ratelimit-default-python-musicbrainz-ngs-count.rrd

          Ian McEwen added a comment -

          I think this is set, will check back if something's looking funky.

          Ian McEwen added a comment - I think this is set, will check back if something's looking funky.

          Dave Evans added a comment -

          See syswiki, RateLimit and MrtgSetup.

          Come back with more questions as and when you have them, of course!

          Dave Evans added a comment - See syswiki, RateLimit and MrtgSetup. Come back with more questions as and when you have them, of course!

          Ian McEwen added a comment -

          Assigning to Dave for question-answering

          Ian McEwen added a comment - Assigning to Dave for question-answering

          Ian McEwen added a comment -

          The new config is added to git and is on lenny/carl, but:

          Issues:

          a.) I suppose I need to restart something on lenny, but I'm hesitant to svc -t or svc -h the ratelimit-server without knowing that's the correct thing to do (or which is the correct thing to do, for that matter)
          b.) MrtgSetup talks about a RefreshHost command, but it has it in /etc/mrtg, not where it is in /etc/mrtg/bin. To boot, running it as ./RefreshHost from within the bin directory fails at trying to find the lenny config; running it from /etc/mrtg (as 'sudo ./bin/RefreshHost lenny') complains about being unable to find another script that's in bin.

          Ian McEwen added a comment - The new config is added to git and is on lenny/carl, but: Issues: a.) I suppose I need to restart something on lenny, but I'm hesitant to svc -t or svc -h the ratelimit-server without knowing that's the correct thing to do (or which is the correct thing to do, for that matter) b.) MrtgSetup talks about a RefreshHost command, but it has it in /etc/mrtg, not where it is in /etc/mrtg/bin. To boot, running it as ./RefreshHost from within the bin directory fails at trying to find the lenny config; running it from /etc/mrtg (as 'sudo ./bin/RefreshHost lenny') complains about being unable to find another script that's in bin.

          Dave Evans added a comment -

          Go for it.

          git.musicbrainz.org/sysadmin/ratelimit-server/

          If you just want to collect stats without applying any actual limits, see keep_stats_only in RateLimitServer.pm. (It has tests, too! t/*.t).

          If you want to expose those stats via snmp (so that they can appear on the graphs), then add something to snmp-agent.pl (search e.g. for picard).

          Deploy those changes to carl/lenny using git push / sudo -E git pull, as usual.

          Then remodel lenny's mrtg config (see MrtgSetup for how).

          (I think there's a bug here - perhaps the ratelimit stats are only collected if lenny is the active host).

          Dave Evans added a comment - Go for it. git.musicbrainz.org/sysadmin/ratelimit-server/ If you just want to collect stats without applying any actual limits, see keep_stats_only in RateLimitServer.pm. (It has tests, too! t/*.t). If you want to expose those stats via snmp (so that they can appear on the graphs), then add something to snmp-agent.pl (search e.g. for picard). Deploy those changes to carl/lenny using git push / sudo -E git pull, as usual. Then remodel lenny's mrtg config (see MrtgSetup for how). (I think there's a bug here - perhaps the ratelimit stats are only collected if lenny is the active host).

            ianmcorvidae Ian McEwen
            ianmcorvidae Ian McEwen
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved:

                Version Package