Twitter API Calls Doubled Since April: Now Serving 70,000 Every Second

Curtis Chen
Sep. 16 2010, 03:05PM EDT

Have you ever wondered how much traffic Twitter handles in a given day, or what software sits behind the curtain of the popular service?  A recent presentation reveals some of the answers. Twitter's incredible growth becomes obvious when you compare the recent numbers to those announced at Chirp.

On September 9th, Twitter's university recruiting team stopped by UC Berkeley to talk about the company and what it does. The slides from platform engineer Raffi Krikorian's talk, Twitter by the Numbers, are now online, and they disclose some fascinating technical details about the social media giant's operations.

Twitter serves over 70 million tweets per day, totaling over 12GB of tweet text alone.  Many of those messages are delivered to client apps and web sites through the Twitter API to the tune of six billion API calls per day (double what was announced at Chirp in April), or about 70,000 API calls per second.  All told, the service generates 8TB of data every day, which is eight times more than the New York Stock Exchange.

Pop quiz, engineers: Your web service needs to deliver real-time message traffic to an asymmetric digraph of over 150 million users.  What database do you use?  WHAT DATABASE DO YOU USE?

    • Shoot the hostage
    • Oracle
    • MySQL
    • Write your own database

      If you're Twitter, the correct answer is #4:  Create your own database software, call it FlockDB, and release it on github.  (By the way, if you picked option 1, maybe software engineering isn't the right career choice for you.)

      FlockDB is just one of the home-grown, high-performance software systems Twitter uses to support its tremendous growth.  Others include:

      • hosebird, a "near real-time" streaming API back-end (instead of REST, which is only "pseudo real-time"); and
      • snowflake (also on github), a network service to generate unique IDs at high scale (MySQL couldn't keep up, and was a single point of failure).

      With a stated goal of supporting "half the world and all its devices," Twitter faces many engineering challenges.  This peek under the hood (full slides are embedded below) shows that they're aware of the potential problems, and are working hard to steer clear of the fail whale.

      Via Delyn Simons

      Curtis Chen Once a software engineer in Silicon Valley; now a science fiction writer and puzzle hunt maker near Portland, Oregon. You may have seen his "Cat Feeding Robot" Ignite presentation. Curtis is not an aardvark.

      Comments

      Comments(8)

      User HTML

      • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
      • Lines and paragraphs break automatically.
      • Web page addresses and e-mail addresses turn into links automatically.

      [...] C. Chen / ProgrammableWeb:Twitter API Calls Doubled S&amp;#1110&amp;#1495&amp;#1089&amp;#1077 April: Now Serving 70,000 E&amp;#957&amp;#1077r&amp;#1091 Se...&nbsp; &mdash;&nbsp; H&amp;#1072&amp;#957&amp;#1077 &amp;#1091&amp;#959&amp;#965 &amp;#1077&amp;#957&amp;#1077r wondered &amp;#1211&amp;#959w [...]

      [...] Last week, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That&#8217;s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      [...] Last week, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That&#8217;s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      [...] Yesterday, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That&#8217;s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      [...] Twitter Seeing 6 Billion API Calls Per Day, 70K Per Second Yesterday, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That&#8217;s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      [...] Yesterday, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That&#8217;s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]