FoundationDB has released a new SQL layer available with API access. CEO and co-founder Dave Rosenthal spoke with ProgrammableWeb about how the new feature is the start of an open source ecosystem built on top of its database engine.
FoundationDB is a distributed database storage engine, using key-value stores at the center of a NoSQL database design.
Now it has released an abstraction layer so that customers can access data via SQL queries, while the data is still stored below the abstraction layer in the NoSQL key-value store engine.
“The exciting thing for us is we have had a key-value store out in the wild, but we have always talked about an ecosystem on top of that," Rosenthal explains. "Now we are adding SQL capability to our engine via a free, open source layer. This is not just another project; we are really delivering on the idea of an open source ecosystem on top of our database engine.”
Rosenthal says this has been the goal of FoundationDB since day one, and he is excited that the SQL layer represents the first opportunity to demonstrate this business vision becoming a reality.
The new technology stack is expected to be of immediate use to database developers working in consumer Internet fields where the ability to manage transactions in real time while maintaining stores of more static customer data is essential. Rosenthal confirms the industry verticals likely to get the most use of this feature:
An obvious targeted application is e-commerce. In general, the SQL layer is built to excel at OLTP, usually to drive some user experience that is unfolding in real time.
So consumer Internet is one of the obvious examples where you need scale because of the number of uses. Real-time messaging platforms, booking tickets, e-commerce — all of those things will need the real-time read-write performance that we can provide.
The main thing we have been working on is zero latency. This is the biggest challenge in a distributed database like ours, so what we had to do with our SQL database technology is reinvent how plans are executed. We built a really cool pipelining technology so that the SQL layer itself can take advantage of the concurrency layer.
For example, when doing a query that looks at a thousand things in the database, the database will send the search results asynchronously and then can rearrange them in the correct order to display the result, so that can drastically reduce latency.
At a technical level, our main focus for the past year has been on building that technology.
The most obvious thing about this architecture is the scalability that we can offer. It turns out that what we have seen from customer feedback is that scalability is the icing on the cake, but the most interesting for them is that the SQL layer is super-fault-tolerant.
You would normally have to go and buy something like Oracle RAC for this sort of robustness. Now this super-tolerant SQL database is being brought down to an open source layer that runs on top of our key value store.
The new SQL layer features are all available via a REST API.
“You can access those objects through the SQL interface or through a REST API, which returns JSON objects, which is the same data you are getting at,” confirms Rosenthal.