Mapping a Day in the Life of Twitter

By Chris McDowall 25/11/2010 22


Program a map to display frequency of data exchange,
every thousand megabytes a single pixel on a very large screen.
Manhattan and Atlanta burn solid white.
Then they start to pulse, the rate of traffic threatening to overload your simulation.
Your map is about to go nova.
Cool it down. Up your scale.
Each pixel a million megabytes.
At a hundred million megabytes per second, you begin to make out certain blocks
in midtown Manhattan, outlines of hundred-year-old industrial parks
ringing the old core of Atlanta...

William Gibson, Neuromancer

Last week I hooked a computer up to the Twitter data streaming API and, over the course of a day and a bit, grabbed every tweet that had geographic coordinates. I wrote a Python script to parse the 2GB of JSON files and used Matplotlib with the Basemap extension to animate 25 hours of data on a world map. The resulting animation plots almost 530,000 tweets — and remember these are just tweets with geo-coordinates enabled.

I recommend you full-screen this video, turn scaling off and high definition on.

The animation begins at 5am on November 18, Greenwich Mean Time (United Kingdom). This corresponds to midnight Eastern Standard Time, 9pm Pacific Time (Nov 17) and 6pm in New Zealand (Nov 18).

There are many interesting things to notice. Here are a few:

  • It is possible to infer the passage of the sun across the map as data begins to stream out of mobile phones and desktops and previously dark patches of the map begin to glow white.
  • At 8:00, 9:00 and 10:00 GMT waves of tweets pass across the United States from East to West. This is an automated Twitter service that tweets local news for specific ZIP codes.
  • Turn your attention to Indonesia. Jakarta glows as brightly as New York and San Francisco.
  • Note the black spots. With the exception of a few cities, such as Lagos and Johannesburg, Africa remains the dark continent.

Each frame of the animation represents one minute of tweets. The animation runs at ten frames per second. I represent each tweet as a small white circle at two percent opacity. At the moment a tweet occurs I plot it at ten point size. Every minute that passes I drop the marker size by one point until it disappears.

Many thanks to Pierre Roudier for his sage counsel and bug spotting skills. I will post videos focusing on particular parts of the world over the coming days.


22 Responses to “Mapping a Day in the Life of Twitter”

  • Would be interesting to see if one can see the rush hour traffic movements by plotting directions and speed of tweeters. Who is the fastest moving tweeter in the world!

  • @Julian That’s a neat idea. Classifying the tweets could be achieved using a service like the Evri sentiment API – well the English tweets at least. I am not sure which languages it handles.

    It would be tough wrestling with scale and granularity issues. I imagine there would be tricky trade-offs between showing detail and making it comprehensible.

    Hmmmm … maybe over the Christmas break …

  • Wow.

    I do wonder what that sudden flare-up of Tweets going roughly East-West in America is at 23-25 seconds in (09:00 GMT).

    This is beautiful. Thank you.

  • Brilliant! Very insightful and inspiring. It would also be interesting to compare peak Twitter periods across the world over a year or longer to map increasing adoption in the dark areas.

    Thanks for sharing!

  • Wow, so beautiful. I’d love to know what those sudden flares (a few strong ones shot straight across the US) were about.

  • @Kai & @Maria Those flare-ups are early morning tweets from a news service with accounts targeting particular US ZIP codes. It sends out the tweets in three bursts (8:00, 9:00 & 10:00 GMT) with appropriate geo coordinates.

    Aside from news bots there are some pretty interesting patterns that look like geo-tweet spam bombs. They are not apparent in this video but I will try to point them out on the version I’ll produce focused on North America.

  • Beautiful Chris! Would be interesting to see with a band of light moving across the screen to visually represent the various day/night segments. Or could have check boxes for differing hourly sets ie /5am->9/ 9->5pm/night owls hours/ etc to show different countries(cultural) tweeting patterns? Julian Carvers idea could be aided by http://www.wefeelfine.org/methodology.html
    Very apt quote too :0)

  • Wow, thanks for the post! It’s amazing how, with all the not-so-beautiful things going on in the world, from a distance (at the risk of sounding Bette-Middler-like 😉 ) it’s really quite a beautiful and fascinating sight.

  • Thanks so much for the explanation. (I had been kind of hoping they were some kind of national gossip-stampede! I admit it.)

  • @greer Would love to do the band of light. I can see it so clearly… just need to sit down and do some math…

  • @nikhil I love it! This is really neat. Are you going to continue working on it? I would love to see where this goes.

  • it is beautiful. feels like a satellite flyover at night. would be interesting to see a version with timegrid (& day/night overlays as @greer termed “band of light”).

Site Meter