How to Create a Web Application that Visualizes Twitter Updates Over Time

Guest Author
Nov. 07 2013, 07:14PM EST

Eric Rowell is the creator of KineticJS, the founder of MeteorCharts, the founder and chief editor of Html5CanvasTutorials.com, the author of HTML5 Canvas Cookbook, and a senior web developer at LinkedIn. He really likes robots, and dreams of one day building his own Megazord.

As companies, organizations and professionals become more and more interconnected via social networks such as Twitter, LinkedIn and Facebook, a new trend is beginning to emerge—social network data visualization. So let’s make a data visualization of our own.

In this tutorial, we are going to create a simple web application that graphs a user’s number of tweets per day. It is written for someone with intermediate skills in web application development and requires some knowledge of both the Javascript and PHP programming languages as well as how to install downloadable libraries to both environments. Our goal is to render something like this chart:

Let’s get started!

Registering an Application with Twitter

First things first. Before we can access the Twitter API, we must first register our application. Head on over to https://dev.twitter.com/apps and click on the “Create a new application” button. You will be asked to provide your application name, description and website address.

Connecting to the Twitter API

Once you’ve registered your application, Twitter should provide four important pieces of information:

  1. Consumer key
  2. Consumer key secret
  3. Access token
  4. Access token secret

Twitter uses the OAuth authentication protocol to enable Twitter users to access Twitter data using your web application. Therefore, the next step is to create a server-side proxy, which establishes an OAuth connection with Twitter. Depending on your server-side language of choice, you can head over to https://dev.twitter.com/docs/twitter-libraries and download an OAuth library that manages the connection for you. For this article, I’ve chosen to go with Abraham Williams’ PHP twitteroauth library, which you can find here: https://github.com/abraham/twitteroauth/tree/master/twitteroauth.

The twitteroauth library includes twitteroauth.php and OAuth.php. You’ll need both. Once you’ve downloaded these files to your server, you’ll need to create an endpoint that your web application can hit to get the Twitter data. I called this file get_tweets.php, which looks like this:


get("https://api.twitter.com/1.1/statuses/

user_timeline.json?screen_name=".$twitteruser."&count=".$notweets);

echo json_encode($tweets);

?>

This file should serve as a template for your endpoint. You need to provide your key, token and secrets, and then construct the Twitter API URL that’s passed into the get() method. This particular Twitter endpoint retrieves the last 30 tweets from the user @ericdrowell (that’s me).

To make sure that the endpoint is working, you can go to your-website/some/path/get_tweets.php. The endpoint should return a blob of JSON.

Visualizing the Data with MeteorCharts

OK, now for the fun part. Now that we have data, we need to aggregate the tweets, then feed the result into a charting library like MeteorCharts, which is a high-performance HTML5 Canvas charting library that works great on desktops, tablets and mobile devices. You can find more info about it here: http://meteorcharts.com/

At the JavaScript layer, we need to do three things:

  1. Hit the endpoint with AJAX and get JSON data.
  2. Roll up the data by tallying up the number of tweets per day.
  3. Instantiate a MeteorCharts line chart by passing in the rolled up data.

To hit the endpoint with AJAX using jQuery, we can do this:


$.ajax({

url: 'path/to/get_tweets.php',

success: render

});

Before we roll up the data, we’ll need a function that converts the tweet creation times provided by the Twitter API into a unix time stamp.



function getTimestamp(createdAt) {

return Date.parse(createdAt.replace(/( \+)/, ' UTC$1'));

}

Now we’re ready to roll up the tweets. We can define a new return array, loop through each tweet object, then add up the number of tweets per day. Each time a new day is detected, we can add the previous aggregation object to the return array. After we’ve gone through every tweet in the original JSON data, we return the new array which contains the aggregated tweets. This routine depends on moment.js, which is a library that elegantly handles date formatting and calculations. You can read more about it here: http://momentjs.com/.


        function rollupTweets(tweets) {
          var len = tweets.length,
              rTweets = [],
              index = -1,
              n, tweet, tdate, lastDay, day;

          for (n=0; n day.unix()) {
              // fill zeros
              if (lastDay !== undefined) {
                while(lastDay.subtract('days', 1).unix() > day.unix()) {
                  rTweets.push({
                    x: lastDay.unix(),
                    y: 0
                  });
                  index++;
                }
              }

              rTweets.push({
                x: day.unix(),
                y: 1
              });
              index++;
            }
            else {
              rTweets[index].y++;
            }

            lastDay = day.clone();
          }

          return rTweets;
        }

Finally, we can instantiate a MeteorCharts line chart by defining a container DOM element and a model object, which contains the roll up data created with the rollupTweets() function. The container DOM element should be a div inside your HTML document.

 function render(tweets) {

var lineChart = new MeteorCharts.Line({

container: 'meteorcharts-container',

model: {

title: 'Tweets per day from @ericdrowell',

series: [{

title: 'Number of Tweets',

points: rollupTweets(tweets).reverse()

}]

}

});

 }

In this mashup, we’re using the default MeteorCharts skin, but you could style the chart any way you like with the view config. You can find more information about MeteorCharts styling here: www.meteorcharts.com/docs

Relevant Links

Guest Author

Comments

Comments(1)