A new API tool that transforms open datasets so they can be accessed by a REST API is capturing the attention of local governments interested in implementing smart cities policies. The DataTank is a data publishing Platform that can also be used as a plug-in with the CKAN open data platform. It is created and managed by the Belgium chapter of the Open Knowledge Foundation, which operates as a social enterprise startup. ProgrammableWeb spoke with The DataTank's Technical Lead Jan Vansteenlandt about the new open data/API tool and how it can be used to drive the smart cities agenda.
2014 may well be the year that the idea of 'smart cities' reaches maturity and moves beyond one-off, showcase projects and - with the help of APIs - becomes the way city authorities and local governments manage their operations. Smart cities is often the term used to refer to the twin policy goals of using sensor technologies and open data strategies to better coordinate a city's urban form, foster civic participation, manage resources and heighten metropolitan livability.
Now, the new tool The Datatank is poised to help local governments make smart cities more of a reality - at least on the open data front. The DataTank is a standalone open data publishing platform that makes data machine-readable via an API. It can also be used as an extension of the CKAN platform. CKAN is an open source tool used by governments and companies to publish their datasets on an open data platform. The DataTank can be used standalone or in conjunction with CKAN to turn datasets into REST APIs. CTO Jan Vansteenlandt explains:
"The DataTank purpose is to make data readable via API. Local governments and other open data providers can add their data directly to the DataTank platform. For those using CKAN, CKAN records the metadata associated with a dataset. So the DataTank identifies which ones are machine-readable and adds them to the DataTank platform. For example, if a dataset on the CKAN platform notes that it is a JSON file, the URI is normally included in the metadata, and that's where CKAN stops, it just stores the metadata. That's where The DataTank comes in. It takes the data, reads it into HTTP and can transform it into any format you want."
To date, the DataTank works with a variety of formats including JSON, XML, CSV and Excel files, enabling them to be accessed via a REST API.
A sustainable business model
While managed by the Belgium chapter of the Open Knowledge Foundation, the group uses a social enterprise startup-type model to ensure their long term sustainability. Currently, they are funded by an innovation grant that has paired them with a research unit to continue developing the tool as well as undertake a related semantic data project. Vansteenlandt and founder Pieter Colpaert are leveraging the funding to create a business model based on providing fee-basesd service level agreements (SLAs) to end customers.
"Our aim is to be sustainable over a long period of time, so we have been drafting a biannual SLA that documents what we offer and agrees to maintain the features and fix anything that is needed, and a non-hosted SLA where we offer a similar service, but customers will need to migrate to upgrades themselves. We are looking at probably a biannual SLA model. We have done a large release presentation in December and are currently following up with some queries where customers may want to incorporate the DataTank with their SaaS or PaaS solutions, so we are waiting on feedback from that," Vansteenlandt said.
Smart cities and local governments
One of the key sectors interested in using the tool have been local governments. Vansteenlandt believes city authorities may be more 'open data ready' to take up using the API tool than many national governments, as they often have faster decision-making processes.
"Local governments have come to us and asked for support to set up using the DataTank. They do a lot of hackathons and they are always amazed at the ideas that come out of them. The DataTank lets competitors access open data endpoints in their app creations, the benefit is that they don't have to Parse a city's dataset on an entire CSV to access the data. We take away that whole troublesome parsing part and local governments are really happy with how easy it is to set up access to their data via our API tool. It means that apps developed in hackathons can continue working after the competition. For example, we have one app that was created in a hackathon two years ago that is still receiving 3,000 API calls a day."
The current Apps For Europe competition also makes use of the DataTank tool to provide open data sets via API to the competitors entering hackathons across European locations over the next month. "Developers [competing in hackathons] can do a lot more with a JSON or XML representation of a CSV file than just with the CSV file itself," Vansteenlandt said.
At present, many of the apps being developed at these civic hackathons are working with location-based data to create mapping solutions: things like locations of the nearest public toilet or pharmacist. It is hoped that with tools like the DataTank becoming more accessible, more sophisticated place-based and contextual smart cities applications can be created that offer real value for local citizen life.
Local governments will be a key target sector for the DataTank's plans to build a viable business model for its open data tool. Creating a scalable, viable business model for open data platforms has been a key sticking point in recent years with previous providers like InfoChimps needing to pivot away from a platform product in order to generate sufficient market income. While Vansteenlandt believes the SLA model is the solution for their product, the DataTank also continues to make itself available for one-off consultancy, custom projects in the meantime.
Open Data Provider-Customers
Other sectors that have shown interest include private industry and cultural institutions, but each comes with a unique set of problems:
"Open data is still something of a conceptual thing in the private sector," Vansteenlandt said. "We have only worked with one purely private sector client and that was an energy distributor. The problem for private companies is that open data can be profitable for customers to use, but it can also be profitable for competition to access.
"Another sector showing interest is the cultural sector: museums and such. They are keen for applications to be built on top of their cultural institution, they have a lot of data to share and their goal is often to inform people about certain arts or artists, but they often have problems with copyrights: you can't just publish a picture of a painting in an app or even a photo of the artist next to a painting."
Mini tutorial: Accessing the DataTank demo data via API
Developers can trial using the REST APIs for many of the demo datasets provided on the DataTank website. For example, there are open data location datasets provided listing police stations, ATMs, European institutions and parks in Brussels.
Developers can click on the dataset and then select the dark blue "View as JSON" option on the right hand side.
This now reveals the URL that can be used to make API requests to map the data or begin experimenting with smart cities app solutions based on city datasets.
By Mark Boyd. Mark is a freelance writer focusing on how we use technology to connect and interact. He writes regularly about API business models, open data, smart cities, Quantified Self and e-commerce. He can be contacted via email, on Twitter, or on Google+.