How Dropbox Scaled and Secured their API

DropBox veterans Leah Culver and Chris Varenhorst recently sat down with Gordon Wintrob over at getputpost.io to reveal how they built their APIs from primitive beginnings to handling over 500 billion calls a year.

The Dropbox API lets you work with files, do full-text search and manage file and folder sharing. Version zero was built back in 2010 off the back of the company’s first mobile apps. Leah and Chris revealed that the team built the early mobile APIs purposely in a general way with the idea to use them again for the public APIs farther down the road. 

From the beginning, the team was conscious of the importance of scaling the API well. While more new features are great for users, they can be a nightmare for API clients when the new features call for new API versions and problems for clients using legacy APIs. As a result, for its very first API, the team waited until the core functionality, uploading files and listing folders for example, was well established and unlikely to change much. This was part of a policy to only add new features or make major changes very incrementally and any major features were to be supported for a long time.

The team did later build a second version to overcome shortcomings of the first version. For v2, the team decided to change the API to simply receive and return JSON. The Dropbox duo state that a REST API was simply not an option for them. REST is great for doc retrieval but it’s terrible for things like useful error messages. The basic HTTP error codes simply didn’t fit most Dropbox API problems. JSON was much better since you could include a specific error identifier in the returned message. 

As for security, in building the second version Leah was weary of DDoS attacks and arbitrary queries designed to steal data. To handle potential malicious traffic, the team set up NGINX servers to handle the incoming requests and then many tasks would get thrown on a queue via Amazon SQS to be handled asynchronously. 

Original Article

Scaling and securing the Dropbox API

Seamus Holland

Comments