12 Ways to Limit an API

John Musser
Apr. 02 2007, 12:10AM EDT

The vast majority of the over 400 open APIs listed here have imposed some limitations on how much they can be used, certainly in the free use model. There are good reasons for this ranging from preventing abuse, controlling costs, or other business-driven reasons. Just over a year ago, in 7 ways to limit API use we looked at some of these. With twice as many APIs now listed it's a good time to check back and see what other ways APIs get throttled. As a refresher, here's the original list:

  • Time based limits: 1 call per second, Last.fm
  • Call Volume by Address: 5,000 queries per IP per day, Yahoo! Image Search
  • Call volume per-application: 10,000 queries per application per day MSN Search
  • Return results volume: 10 results per query, the now deprecated Google Search, or 100 items returned per call, Tailrank, or 100 blogs per map FeedMap
  • Data Transmission Volume: 120 packets of 1.6KB per minute, MSN Messenger
  • Formula: Monthly quotas based on various factors, Google AdWords
  • Kindness of strangers: "Please be gentle with Simpy's server", Simpy

There are over 40 variations of the above list, mostly differing in exact size of the limit. What happens if you hit one of these limits? It depends. Most return an error code and/or message, some unfortunately leave it undefined.

And here are a few more to keep in mind, some are more exact than others...

  • Per second and per month limits: Up to 1 call per second for up to 60,000 requests per month, Amazon Historical Pricing
  • Login limits: 250,000 logins per day or 2 million per month, AOL Instant Messenger, AIM
  • Varies within a range: determined by overall system load, GeoCoder.ca
  • As needed: "The servers will block excessive requests", ISBNdb
  • Heads-up: not so much a limit as a request, "Let me know if you're going to hit it hard", Where's Tim API
John Musser

Comments

Comments(9)

User HTML

  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.

[...] John Musser over at ProgrammableWeb.com has summarized the various ways (he&#39;s outlined 12) API providers can control access to their web services. As John says, &nbsp;&quot;There are good reasons for this ranging from preventing abuse, controlling costs, or other business-driven reasons.&quot;John triggered a few things in my mind with this post...Thinking out loud now...7 Partially Random Thoughts on the Business of API ThrottlingWeb APIs&nbsp;will be a key driver of future of the networked, services-based&nbsp;economiesThe science of web API throttling&nbsp;is set to&nbsp;become a&nbsp;core business competency - not just a technical competency -&nbsp;for (hundreds of) thousands of businessesBusinesses need to learn about the business of API-based business models (an&nbsp;area where there is some experimentation and learning is going on&nbsp;today, but not enough)Business expertise in designing and evolving API-based business models will be a highly valued skill / commodityStandardized business models need to (and will) emerge regarding&nbsp;web APIs, similar to how there are standardized business contracts todayTechnical competency&nbsp;pertaining to API throttling&nbsp;will become exponentially more complex to execute and operateA significant number of small and medium-sized business (the long tail of today&#39;s economies) that want to&nbsp;thrive in the networked economy will&nbsp;need to make a&nbsp;decision in the near future: &quot;do we spend large amounts of resources in the technical operation of&nbsp;API service&nbsp;delivery&nbsp;and throttling expertise in-house, or should we outsource this?&quot; Posted: Friday, April 06, 2007 6:21 AM by alexbarnett Filed under: Web 2.0, webservices, APIs, SOA, enterprise2.0, SaaS, BungeeLabs [...]

Thanks for helping Mashery with our product roadmap, John!

Mashery's clients can currently throttle the APIs we manange for them according to most of these techniques, and the rest are on our development roadmap for release in the near future.

Most of these presuppose the issuance of a developer key, which is passed with the API request to identify their calls. What goes hand-in-hand with the throttling techniques are the means of authentication that some APIs apply to API requests to gain access in the first place. These range from limiting requests to certain IP addresses, geographies, user agents or referring domains, to requiring username/password authentication to be passed, to implementing standards-based authentication such as X509. We're already seeing requests for some of these, and expect that as more sensitive or strategically important data and services are made available through APIs, stronger authentication will become more commonplace.