Asks 'Discogs' how many requests are left in the current window. The check is a request itself and counts towards the limit.
Value
A tibble with one row and the columns limit, used,
remaining and authenticated, or NULL invisibly if the API cannot
be reached.
Details
'Discogs' allows 60 requests a minute with a token and 25 without. Its documentation describes a moving average over a 60 second window. In tests made for tidycrates, the count followed clock minutes and was
where current is the number of requests so far in this clock minute,
previous the number in the minute before, and seconds the time since
this minute began. Requests of one minute therefore fade out over the
next, and a burst counts for up to two minutes. Refused requests were
counted too.
tidycrates keeps the same count of its own requests. A request is sent at once while the count is 58 or less with a token, or 23 or less without, so up to 59 or 24 requests go out in a minute. Later requests wait until the count has room again, and no longer. If the response headers give another limit, the count may reach two below it. If 'Discogs' reports a higher count, for example because other applications use the same token, the difference is added. Requests are retried after HTTP 429 and 503 responses. A message says how long each wait of a second or more takes; when several items are fetched, the progress bar counts the wait down instead and its ETA includes the waits still to come. The limit only needs checking when other applications share the same token or IP address.
The count uses the clock of your computer. A clock that is off by a second changes it by about one request, which the margin of two covers. A larger error can lead to refused requests until the count reported by 'Discogs' corrects it.
Why tidycrates throttles requests itself
httr2 has a throttle of its own, httr2::req_throttle(), but it cannot
follow the count of 'Discogs'. Its bucket allows a burst and then
refills at a steady rate, and it does not know that a burst counts for
up to two minutes. A bucket that allows a full minute of requests at
once sends more requests than 'Discogs' allows. They are refused, still
counted, and retried after waits of 10 seconds or more, so long batches
slow down and can fail. A bucket small enough to stay within the limit
either spaces all requests evenly, so every small batch waits about a
second for each request, or refills more slowly, so long batches take
up to twice as long.
Keeping the same count as 'Discogs' avoids both problems. Requests go out at once whenever 'Discogs' has room, wait only as long as needed when it has none, and are not refused. In a simulation with 0.4 seconds for each request, 46 requests took about 18 seconds this way and about 46 seconds with even spacing, and 300 requests took about as long with both.
