Answer provided by Felix Bonkoski:
You can only fetch X many followers per minute with the API. This is represented in the x_ratelimit_api_... headers. These fields in the header inform you what the limit is, how many more calls to the API you can make, and when the limit will expire so that you can start using that API endpoint again. This is not in the documentation, but please see the Tumblr API Discussion Group where there are posts dealing with this issue.