In a previous how-to, we created a RESTful API to search videos, performing the actual search through YouTube Data API. To provide additional support for languages and to demonstrate that we are stating general principles that can be implemented using (almost) any programming language/technology, in this report we will explain how to implement the same interface using node.js. As you’ll see, using a couple of Node modules, the job is even easier than with Google App Engine.
Here is a list of features we will implement:
We’ll need to route three different paths, as we did with our Python implementation for Google App Engine. However, while the Python implementation required that we get into the details of regular expressions to define the rules to match those paths, with the Express module for Node we will be able to effortlessly catch the parameters in the URLs.
Setting Up Node.js
If you have not already installed node.js on your machine, you can download the installer here.
Now you can create a folder to contain your project. For example, I created one named “pweb.”
Enter your folder name, and create a new file named package.json. This configuration file will be needed if you export your module.
Open package.json and insert the following text:
You can replace the name and the version with consistent values for your app, and the version of node and npm that you’ll retrieve by entering node –version and npm –version in your terminal. (To discover more about versioning conventions, check https://www.npmjs.org/doc/misc/semver.html and http://semver.org/.)
Now, in your terminal, navigate to the folder containing your project, and then enter the following commands to install the Express and GoogleApis modules. (The -d option tells npm to install/update all dependencies, and the --save option automatically updates the dependencies field inside your project’s package.json.)
Express is a framework for node.js, and, although it is a very lightweight library, it is extremely flexible and allows users to easily serve static files (or, just as easily, to create a dynamic application server).
To be fair, there are several different options that can be leveraged to solve this problem. These include using other node.js modules, and even manually assembling the query address and sending the GET request to YouTube servers. However, these options could, in time, require more maintenance. If Google releases a new API version or is forced to change something in current API, you’ll probably need to modify your code to reflect those changes.
As stated on its page, instead,
So, if you are going to develop a commercial product, it’s best to avoid depending on “some guy over the Internet’s module” because you can’t rely on her/him to provide maintenance and support over time.
Of course, it’s not possible to completely avoid dependencies. (I just instructed you to install two non-core modules in this very same post, after all). However, you should try to minimize them and to depend only on reliable sources (especially if, as in this case, the module you use depends in turn on another service).
Step by Step, Line by Line
To get started, import the modules mentioned in the previous section, express and googleapis. We created an object for our server app, using the method exported by express, and set some sys variables.
Next, set up a static server on your server’s root so you can add a static “error” page with a brief manual, as we did on Google App Engine. (This is an optional step.)
Set the paths that your Web server will handle, and associate each of them with the helper function route, which we will introduce shortly.
You can see how easy it is to capture parameters in the URL path: Just prepend a colon to the name of the parameter, and you’ll find a field inside the request object with that same name. (For example, :keywords is mapped to req.params.keywords).
GET parameters, instead, are automatically parsed for you by express, and mapped in the ‘query’ field of the request wrapper object (req, above).
In our tests, once we finished setting paths, we had to actually start the server, which is done using its ‘listen’ method (passing the port number of the port we wanted to listen to).
To be able to use the YouTube Data API, we still had to “discover” the API. In other words, we needed to query Google API Discovery Service to get a list of the methods available for this particular API.
The discovery step is asynchronous, and we basically had two choices as to when it would be performed: We could do it once and for all, and save a reference to the client returned, or we could rediscover the API at each request. (Googleapis caches locally the result of discovery calls for 5 minutes, so your server won’t actually issue an HTTP call every time you call it, but only once every 5 minutes.)
The latter will ensure fresh data, but will increase latency and diminish availability. (If, for any reason, Google Discovery Service is down but YouTube API isn’t, you won’t be able to answer the request you received anyway.) Since the signature for this API likely won’t change often, the former solution made more sense for us. We performed the discovery once when we started our server, encapsulating the code above into the asynch call to discover:
Before getting into the details of search, we defined the helper functions we’ll use to validate the parameters for our API:
- validatePositiveNumber: checks that a parameter is (or can be converted to) a positive integer, and otherwise return a defaultValue;
- validateOrderParameter: validates the “order” parameter, accepting only those values recognized by YouTube API;
- parseData: transforms the JSON entry for a video as received from YouTube API into a more convenient, less redundant format;
- route: handles all of the different paths that are considered valid by our Web server; defines the callbacks that will handle errors and data retrieved from the API remote call; and, finally, once it has checked that at least the keywords parameter has been correctly passed to our API, gathers all the parameters for the API in a single object, computes an acceptable value for the first_result parameter, and then calls the search function.
This is the last piece of the puzzle.
We had to correctly set the params object with the search option for the Data API. Since we wanted to keep the default value for optional parameters for which the user hasn’t set any value, we checked each of them inside a loop to keep the code “DRY” (Do not Repeat Yourself) and set only those that were actually passed by the user.
We could then run the query, using the search.list method.
With googleapis, the list command sets the parameter for the API. (It takes them as parameter in the form of an object literal.) With that done, we needed to set our API developer key, using the withApiKey method. We could also use OAuth 2 authentication, but this wouldn’t be appropriate in our case. It would require the user to authenticate, so an extra step would be needed. Moreover, we were not going to access users’ personal information, so we didn’t really need OAuth.
These methods set the parameters only for the search, returning an object that “wraps” the query itself (so we could save it and run multiple times). To actually perform the query, we need to call the execute method on this object, which takes a callback function as a parameter.
To see the application in action, go to Heroku, where I have uploaded it. The complete code for this example is hosted on gist, so you can easily download and test it.
In the version hosted on gist, I also added support for the first_result parameter. This time, it is a full support—that is, first_result and max_results are unrestricted (in particular, max_results can even be greater than 50, which is the limit for the Data API)--and results will be retrieved even across multiple pages.
The implementation leverages v3 API’s page tokens mechanism, although in a questionably
inefficient way: It just downloads all the results between the first and the one at index first_result + max_results. By using memcached (installing an npm module, like this one), however, this weakness can be turned into strength because we can cache the results for popular and/or recent queries, and save API calls. As an alternative, we could cache the page tokens retrieved from YouTube queries: They stay the same across different queries, so we can leverage this fact, cache only the tokens corresponding to the indices, and spare the download of intermediate results.
The main goal of this post was to show you how the same principles that apply when creating a RESTful API also apply with different frameworks and languages. It was as easy for us to develop an API using Python and Google App Engine as it was to create an equivalent interface with Node.js.
The key was to keep the guidelines focused mainly on the design of the interface and on making it simple and easy to use. The need for ease of use cannot be overstated: It must be your first concern when creating an API or an interface of any kind. In our demonstration, the technology underneath acted as a means to an end. When building an API, think about what you can do to make development quick, easy, aligned with existing skills and including as few bugs as possible.
If you aren’t familiar with Node.js, this post is a good starting point. Hopefully, you have seen how easy it can be to develop even complex applications using Node.