Program a map to display frequency of data exchange, every thousand megabytes a single pixel on a very large screen. Manhattan and Atlanta burn solid white. Then they start to pulse, the rate of traffic threatening to overload your simulation. Your map is about to go nova. Cool it down. Up your scale. Each pixel a million megabytes. At a hundred million megabytes per second, you begin to make out certain blocks in midtown Manhattan, outlines of hundred-year-old industrial parks ringing the old core of Atlanta...
William Gibson, Neuromancer
Last week I hooked a computer up to the Twitter data streaming API and, over the course of a day and a bit, grabbed every tweet that had geographic coordinates. I wrote a Python script to parse the 2GB of JSON files and used Matplotlib with the Basemap extension to animate 25 hours of data on a world map. The resulting animation plots almost 530,000 tweets — and remember these are just tweets with geo-coordinates enabled.
I recommend you full-screen this video, turn scaling off and high definition on.
[vimeo width=”651″ height=”391″]http://vimeo.com/17178821[/vimeo]
The animation begins at 5am on November 18, Greenwich Mean Time (United Kingdom). This corresponds to midnight Eastern Standard Time, 9pm Pacific Time (Nov 17) and 6pm in New Zealand (Nov 18).
There are many interesting things to notice. Here are a few:
- It is possible to infer the passage of the sun across the map as data begins to stream out of mobile phones and desktops and previously dark patches of the map begin to glow white.
- At 8:00, 9:00 and 10:00 GMT waves of tweets pass across the United States from East to West. This is an automated Twitter service that tweets local news for specific ZIP codes.
- Turn your attention to Indonesia. Jakarta glows as brightly as New York and San Francisco.
- Note the black spots. With the exception of a few cities, such as Lagos and Johannesburg, Africa remains the dark continent.
Each frame of the animation represents one minute of tweets. The animation runs at ten frames per second. I represent each tweet as a small white circle at two percent opacity. At the moment a tweet occurs I plot it at ten point size. Every minute that passes I drop the marker size by one point until it disappears.
Many thanks to Pierre Roudier for his sage counsel and bug spotting skills. I will post videos focusing on particular parts of the world over the coming days.