How to Access Any RESTful API Using the R Language

R is an excellent language for data analytics, but it's uncommon to use it for serious development. This means that popular APIs don't offer software development kits (SDKs) or how-to guides for analysts working in R the way they do for other more popular languages like Python or Objective-C (for Apple's iOS). This is a how-to guide for connecting to an API to receive stock prices as a data frame when the API doesn't have a specific package for R. For those of you not familiar with R, a data frame is like a spreadsheet, with data arranged in rows in columns. You can then use these same techniques to pull data into R from other APIs.

R is missing from Google's list of API SDKsFigure 1. R is missing from Google's list of API SDKs, a common problem for R users working outside of Google Analytics.

Getting Started With APIs in R

If you're working with an API that has pre-built SDKs for R, accessing data with R is simple. For example, to make your job easier, the Google Analytics API has well-documented packages, such as RGA or RGoogleAnalytics. If you're working with an API that doesn't have R packages (such as Stripe or Intercom), you'll need to know how to access the API on your own. An API can automate your data collection, so it's well worth the effort.

This tutorial assumes you have a basic working knowledge of R and are comfortable scripting with RStudio or working with the Rstudio console. These examples will work on Mac or PC as long as you have an internet connection and an up to date version of R installed on your computer (3.2 or newer).

Here are the basic steps:

  • Install the "httr" and "jsonlite" packages
  • Make a "GET" request to the API to pull raw data into your environment
  • "Parse" that data from its raw form through JavaScript Object Notification (JSON) into a usable format
  • Write a loop to "page" through that data and retrieve the full data set
  • Apply the same methodology to other APIs

A good way to follow along with this how-to guide is to copy each line of code into a script in RStudio. This will enable you to run each line of code individually so you can see it working and then to run them all at once at the end. You can also enter them line by line from the R console.

Install the "httr" and "jsonlite" Packages

Start your script off by installing the httr package:

install.packages("httr")

#Require the package so you can use it
require("httr")

This package makes requesting data from just about any API easier by formatting your GET requests with the proper headers and authentications. Next, install jsonlite in your script:

install.packages("jsonlite")

#Require the package so you can use it
require("jsonlite")

When the data comes back from many APIs, it will be in JSON format. If you're like most R users, you'll want to convert the JSON from its native nested form to a flat form like a data frame so it's easier to work with. The jsonlite package makes this easy.

These two simple packages make it possible for R to work with many APIs even if a prebuilt R SDK doesn't exist for those APIs.

Installing the httr package to make GET requests and the jsonlite package to parse the JSON responses

Figure 2. Installing the httr package to make GET requests and the jsonlite package to parse the JSON responses

Making a "GET" request in R

Pulling data from a RESTful API often requires an API password, an API username, or both, as well as a properly formatted URL and header. The URL (which is technically known as the address of the API's "endpoint") tells the API what data you're looking for and the username/password is for APIs that have what's called "basic authentication" (not all do, but the example assumes this to be true). The headers are often used to negotiate other parameters that enable the application to communicate with the API successfully. For example, they may describe the formatting of the data payload.

Andrew Carpenter I'm an author, I/O Psychologist, and budding data analyst. When I'm not living in a fantasy world, I'm at Intrinio trying to redefine HR Analytics. Intrinio's mission is to make financial data affordable and accessible so investors can save money and make time to build something meaningful. You can view examples of books I've written here: https://www.amazon.com/Roll-Honey-Andrew-James-Carpenter/dp/1537174711?_encoding=UTF8&psc=1 https://www.amazon.com/Blood-Born-Saga-Book-ebook/dp/B00GM8WYIK/ref=sr_1_1?ie=UTF8&qid=1495128022&sr=8-1&keywords=the+blood+born+andrew+carpenter
 

Comments