NZ Open Data Day

By Aaron Schiff 28/03/2015

I had a great time at the NZ Open Data Day today.* Many thanks to Caleb and Harkanwal for organising the day, and to the NZ Herald for hosting and lunching us.

I won’t attempt to summarise the whole day. Instead, here are a few snippets of things that I learned:

Kim Ollivier has done the hard work of converting the 2013 Census meshblock dataset into an SQLite database that you can query instead of fishing around in CSV files. You can download it for free here.

DigitalNZ has a really great API that provides access to their incredible database of New Zealand content.

DigitalNZ has also released their web scraping technology as an open source project called Supplejack. It looks to be highly capable at turning messy data from a variety of sources into something structured and useful.

XPath is a natty query language for pulling things out of XML documents. It is used by Supplejack.

MongoDB is an easy to use and flexible database, and Caleb has imported NZ Census and other geographic boundaries into it — see here.

The LINZ Data Service is making really high resolution imagery available, with more to come soon I believe.

Also on the LINZ Data Service (and Koordinates), you can view the history of many of the data files, including changes made to them over time.

Tablua is a nifty tool for extracting data tables from PDFs, and LiveReload will save your browser’s reload button from wearing out.

Last but not least, the person from StatsNZ in attendance graciously listened to our collective whinging about NZ.Stat, and proved once again that StatsNZ is populated by awesome and helpful people.

* Editor’s note: we continue to work through Aaron’s excellent back-catalogue of posts.