D3.js

Dec. 7th, 7pm at Fabelier by Raphael Velt

A tutorial to create a bubble chart of Twitter Hashtags

Step 1: Setting up your HTML page

Open your favourite Development Environment and create a new project and/or directory.

Download D3.js from https://github.com/mbostock/d3/downloads

Have a look at the “examples” directory inside the downloaded file, you'll find a lot of inspiration there.

For the beginning, we'll only need the d3.min.js file, copy it to your project.

Here is a sample HTML template you can copy :

<!DOCTYPE html>
<html lang="en">
    <head>
       <meta charset="utf-8" />
       <title>D3.js Tutorial</title>
       <style type="text/css">
           <span style="color: red;">&lt;!-- This is where we'll put style information --></span>
       </style>
       <script type="text/javascript" src="d3.min.js"></script>
    </head>
    <body>
       &lt;div id="visu">
           <span style="color: red;">&lt;!-- Our data visualization will be displayed here --></span>
       </div>
       <script type="text/javascript">
           <span style="color: red;">&lt;!-- We'll type our code here --></span>
       </script>
    </body>
</html>

Step 2: Get the data from Twitter

We'll use the Twitter Search API, which works with the following syntax :

http://search.twitter.com/search.json?q=<search request>

If we request a hashtag, such as #ows, we'll have to escape the “#” character (which becomes %23).

We can request up to 100 results, by adding the “rpp” parameters. Our request URL now becomes :

http://search.twitter.com/search.json?q=%23ows&rpp=100

D3.js has a function called .json() to request JSON data through Ajax. Unfortunately, Ajax has a policy preventing cross-domain requests.

Therefore, we'll use the “JSONP” (JSON with padding) trick, using a callback function.

function myCallback(json) {
    <span style="color:red">/* This is where we'll write the data visualization code */</span>
}
var jsonpElement = document.createElement('script');
jsonpElement.type = "text/javascript";
jsonpElement.src = "http://search.twitter.com/search.json?q=%23ows&rpp=100&callback=myCallback";
document.head.appendChild(jsonpElement);

Step 3: Process the data to extract hashtags

The code listed below has to be put inside the callback function.

We'll use an associative array (also known to JS developers as “object”) to count how many times hashtags have been used

var counter = {};

In the data returned by the Twitter search API, tweets are listed under the “results” key, as an array. We'll iterate through this array.

for (var i = 0; i < json.results.length; i++) {

We'll get the text field of the tweet, and convert it to lower case, so we count “#ows” and “#OWS” as being the same hashtag

var text = json.results[i].text.toLowerCase()

We'll extract hashtags with a regexp. A hashtag is a hash followed by any number of word characters (\w)

var hashtable = text.match(/#\w+/g);

We'll now parse this table to add hashtags to the counting array

for (var j = 0; j < hashtable.length; j++) {
  counter[hashtable[j]] = 1 + counter[hashtable[j]] || 0;
}

End of the array loop :

We now have an associative array that looks like { “#ows” : 101, “#p2” : 67, “#revolution” : 52 }.

D3 cannot process associative arrays, we'll have to convert it to a simple array.

D3 provides a convenient function, called .entries() for this :

var hashtags = d3.entries(counter);

Our data now looks like [ { “key” : “#ows”, “value” : 101 }, { “key” : “#p2”, “value” : 67 }, { “key” : “#revolution”, “value” : 52 } ]

We'll now filter our data, to retain only hashtags occurring at least twice :

hashtags = hashtags.filter( function(entry) { return entry.value >= 2 } );

We can also sort the array by decreasing value :

hashtags = hashtags.sort( function(a, b) { return b.value - a.value } );

And retain only the first 12 occurrences :

hashtags = hashtags.splice(0,12);

Step 4: Build our first D3.js visualization (with HTML DIVs)

We'll start by defining where we want to put our visualization.

d3.select() selects DOM elements, like JQuery's $() does, or in native JS on recent browsers, document.querySelector().

var chart = d3.select("#visu");

D3 works by associating DOM elements (HTML markups or SVG vector graphics elements) with entries in a data table.

We'll now associate our table with divs that have the hashtag class:

chart.selectAll("div.hashtag")
  .data(hashtags)

As you've probably noticed, we still don't have any div.hashtag in our page ! But don't be afraid, as D3 provides a function called .enter() that returns a placeholder for data items for which there is no DOM element. (The opposite function is .exit(), often used after removing data, and returns DOM elements that don't have data anymore)

Note that, after this operation, we only get a placeholder. We'll have to append the elements we need, with .append() and specify their characteristics.

Our code is now :

chart.selectAll("div.hashtag")
  .data(hashtags)
  .enter()
  .append("div")
  .attr("class", "hashtag")

We only use .append(“div”) once, but this appends a div element for each placeholder returned by .enter() !

In the same logic, attr(“class”,“hashtags”) has given the hashtag class to all the div elements created by .append(“div”)

You'll have to be careful with D3's powerful chaining syntax. Some functions, like .attr() return the same objects they apply to, but some, like .append(), or .enter(), return new objects.

To see the results, let's throw in some style in the stylesheets :

div.hashtags { height: 20px; padding: 2px; margin: 2px; border: 1px solid #999; }

At the moment, we only have empty, full-width rectangles.

We want them to be sized according to the number of occurrences of each hashtag, and to contain the text of the hashtag.

As we've seen, using functions like .attr() to do so will loop over the div elements and apply the same attributes to each. To make sure the attributes will be different for each entry, we'll replace the static attributes by a function that operates on data items.

We'll use the .style() function, that takes CSS style arguments, to change the width, and the .text() function for the contents

d3.selectAll("div.hashtag")
  .style("width", function(d) { return 8 * d.value + "px" })
  .text("width", function(d) { return d.key });

To be continued…

Fabelier

User Tools

Site Tools

Table of Contents