Tonight I will be talking about the aurora on Bryan Crump’s radio show (Nights, Radio New Zealand National, 8.42pm, Thursday August 26th).  I won’t spend much time here explaining the underlying physics of the effect, but take a look at the beautiful infographic posted by Peter Griffin.

Now although I didn’t see any sign of the predicted aurora a few weeks ago (did anyone with clear skies that night see anything?), I did regularly encounter aurora during the early years of my PhD studies at the University of Alberta in Edmonton, Canada. In fact, I managed to take a girl out to see an absolutely spectacular display on our first date.

“Ah, physics …” I said, as we stared up at the blue, red and green streaks shimmering in the sky. Not the greatest line ever used but good enough that the girl in question married me a few years later.

Now  you might guess that my apparent good planning was down to some quick back of the envelope calculation. Not so – although theoretical physicists have many powers (and lets face it, they have special need of them when romance is called for), predicting the time and place for watching aurora is not yet one of them.    

When to see an aurora

Both the northern and the southern aurora are caused by electrons and protons (known as a plasma) from the sun colliding with nitrogen and oxygen in the Earth’s upper atmosphere.  This flow of plasma from the sun, known as “space weather”, is concentrated by the Earth’s magnetic field at the poles so that you are more likely to see an aurora in Edmonton or Dunedin than Wellington.

Like terrestrial weather, space weather can be quite irregular, which is why aurora can be relatively special events.  As the plasma coming from the sun consists of charged protons and electrons, its flow from is strongly affected by the magnetic activity of the sun.  At periods of high solar magnetic activity, it turns out that the sun is more likely to be eject plasma. 

Solar magnetic activity cycles over a period of just under 11 years for reasons that are not yet well understood.  We are entering a period of maximum solar activity now, so the flow of plasma from the sun should be relatively strong.  Yet if I look at when I took my date out to see the aurora (May 1994), the sun was actually in a period of minimum activity.  And since 2009, the sun has been in an active period, yet we are only just starting to see aurora at our latitudes now. 

Picking a good night for an aurora is obviously not so straightforward. 

The trouble with space weather

So what’s the problem? Well, the flow of a plasma is a complicated thing:  a flowing plasma generates a magnetic field, but the flow itself depends on the magnetic field.  So to understand a plasma you need to work out both the magnetic field and the flow at the same time: it’s the physicists’ version of the chicken and the egg problem.      

In fact, doing this is so difficult that it gets its own sub-field of physics known as magnetohydrodynamics, or MHD for short.  The equations that are used to model MHD are nonlinear and difficult to solve, requiring large supercomputers for even simple flows.  In the case of the aurora, part of the difficulty seems to lie in the way in which the magnetic field of the sun, the Earth and the plasma itself interact to direct the plasma towards the atmosphere. 

Now knowing when plasma from the solar wind is going to hit the atmosphere is not just important for the love lives of physicists.  Intense flows of plasma can cause geomagnetic storms, which can knock out power grids and satellites. 

For this reason NASA launched the THEMIS space mission a few years ago to put in place a collection of satellites that can monitor the space weather.  Data from this mission will be used to help improve the models we use for the flow of plasma from the sun and may even allow scientists to forecast the solar weather in the way terrestrial weather is forecast today. 

And if the space and terrestrial weather were to cooperate to put on a show in Wellington then I will get a chance to come up with a better line than the one I used in 1994. Suggestions in the comments section please …

It seems to have become received wisdom recently that New Zealand must pick winners with its public science investment.  In this post, I argue that this is not new:  we picked our winners a long time ago, with a strong focus on agricultural and environmental sciences.  So what are the pros and cons of backing the same winners decade after decade?  You’ve got to get lucky sooner or later, right? 

A matter of scale?

A few weeks ago, I went to another Philip McCann seminar, this time at the Treasury.  This talk was again based on his paper [1] in the New Zealand Economic Papers.  Readers will be familiar with McCann’s ideas about the New Zealand economy from my series of posts on New Zealand’s Productivity Paradox.  There were a few additions to a talk I saw earlier this year at the Reserve Bank, but his underlying message was still the same.

McCann argues that one of the key issues affecting New Zealand is a lack of scale.  As I have discussed on this blog, there is a lot of evidence that scale in important for innovation.  Big cities are the drivers of innovation –  scale matters when it comes to generating new knowledge.

An apparent corollary is that as a small country, New Zealand will only be able to achieve scale on the international stage in a few areas.  Indeed, many people, including McCann and Sir Peter Gluckman, the Prime Minister’s Chief Science Advisor, take this to mean that New Zealand must pick winners if it wants its innovation spending to have any impact.

This is nothing new.  New Zealand has always prioritised its science spending.  A glance at our New Zealand science tag cloud for 2009 shows that we are strongly focused on agricultural, environmental and medical sciences.  If we have scale, then it lies in these disciplines.

Diminishing returns? 

So should we simply invest more in these areas to take advantage of scale?

Not necessarily.  Not all areas of science are going to produce the same returns.  In fact, a recent study of several scientific fields by Harvard University scientist Samuel Arbesman [3] found that the new knowledge discovered in these fields each year decayed exponentially (HT: Nicola).  The corollary here is that to maintain the same rate of discovery in a mature field, investment must increase each year. 

Indeed, there is an interesting empirical study (“Has New Zealand benefited from its investments in R&D?” [2]) that finds New Zealand’s sectoral public investments in R&D do not correlate with sectoral productivity growth.  Relative to other OECD countries, for instance, our productivity in agriculture is slipping.

Is it possible that the resources required to maintain competitiveness in a mature field such as agriculture have grown beyond what New Zealand can muster?  The Netherlands alone has more agricultural scientists than New Zealand has scientists.  How can we compete?

A way forward 

Last time I checked, science budgets were not increasing exponentially, so how do scientists make any progress at all?  As I have seen from my studies, they collaborate more, work in bigger teams and become more specialised.  Some of the tools scientists use, such as computing and genomics, are becoming exponentially more powerful each year.  And blue skies research can open up new fields of discovery where the going is easier, at least for a time.

So does our science funding system facilitate or foil scientists’ attempts to beat these diminishing returns?

There are some hopeful signs on the horizon.  New Zealand scientists do collaborate through mechanisms such as the Centres of Research Excellence (the CoREs), and increases in non-contestable funding for the CRIs may be able to break down traditional institutional barriers.  Furthermore, the government has also shown a willingness to invest in key scientific infrastructure, including advanced genomics capability and high performance computing.

However, continued micromanagement of the contestable funding pool for science is likely to limit the size of science teams in Universities and skew the balance of this portfolio towards well-established, low pay-off areas.

The CoRE selection process was one of the few occasions in New Zealand where large teams were picked on merit from across the sciences.  The outcomes did not align well with FRST portfolios.

Further merit-based investment in research networks is needed.  Such investment not only offers the possibility of uncovering new under-exploited areas of knowledge, but couples this with the opportunity to rapidly build scale in a way that blue skies funding, like Marsden, cannot.   

Finally, we should not labour under the pretence that decades of flat-line funding of science will keep our industries competitive.  In areas of sustained national economic importance, we have no choice but to grapple with diminishing returns of knowledge by further investments in infrastructure and research networks.  

[1] McCann, P. (2009). Economic geography, globalisation and New Zealand’s productivity paradox New Zealand Economic Papers, 43 (3), 279-314 DOI: 10.1080/00779950903308794

[2] Johnson, R., Razzak, W., & Stillman, S. (2007). Has New Zealand benefited from its investments in research & development? Applied Economics, 39 (19), 2425-2440 DOI: 10.1080/00036840600707308

[3] Samuel Arbesman (2009). Quantifying the Ease of Scientific Discovery Scientometrics arXiv: 0912.1567v3

By popular request our intern has put together a subject area tag cloud for Australia from their 2009 publications in the ISI database.  As she observed, Australia is poorly designed.  So much so that it is hard to squeeze Canberra’s tag cloud in between those of Sydney and Melbourne.  In fact, you’ll see in the map below it has drifted out to the southern coast of New South Wales in a most aesthetic manner.  It may be in fact that many of the residents of Canberra would be in favour of such a move …

Medicine and medical sciences dominate the clouds over Sydney, Melbourne, Perth and Adelaide.  Hobart looks a lot like Wellington with its emphasis on oceanography and marine biology.  Canberra seems to have a broader focus albeit with a strong contribution from the physical sciences and engineering.  Brisbane stands out with a very strong signal from biochemistry.

Our talented intern from MIT has produced another tag cloud.  This time she has taken a look at who we collaborated with in 2008 based on our co-publication preferences in the ISI database.   The resulting map is shown below:

It’s clear we like working with Australians.  Those in Auckland, Palmerston North and Christchurch prefer to work with Sydneysiders, while those of us in Wellington prefer Victorians.  Hamiltonians have more exotic tastes with a clear preference for Californians.  And although Dunedin is often said to be the Edinburgh of the South, our southern scientists show a strong preference for London.

What science are New Zealanders working on?  To help me answer this question, I have an intern from MIT here for her summer break.  Luckily for me, she hadn’t heard about Wellington’s winter.  (Not that our spring or summer are up to much either, although we can put on a decent autumn.)

She is a very bright cookie, and she mastered the ISI bibliometric database and our network analysis software in no time at all.  She is mainly studying the bibliometric performance of the Centres of Research Excellence (CoREs), but she has found time to look into other aspects of New Zealand’s bibliometric record.

Inspired by visualisations of the Twitter universe (such as trendsmap), last week we produced a “tag cloud” of subject areas Kiwis are publishing in across the main centres.  We picked the top five ISI subject areas in each of the main centres, scaling the text by how often it occurred (i.e. by the total volume of papers published in each subject area).  The 2009 cloud is shown below:

In Auckland and Dunedin, pharmacology dominates, presumably due to their university medical schools.  In Christchurch and Hamilton, environmental science dominates; in Wellington, it is marine biology; and in Palmerston North, it is veterinary science.

The map clearly shows New Zealand’s strong specialisation in health sciences, the environment, and food and agriculture.  As I pointed out in a previous post, the proportion of articles that Kiwis publish in the health sciences is similar to the rest of the world.  Where we differ from the international norm is the high priority we give agricultural and environmental science and the low priority we assign to the physical sciences.

In my job as deputy director of the MacDiarmid Institute, I regularly get to recount the story behind our Institute to all sorts of visitors.  Last week I hosted a group of US scientists on Wednesday and a small Iranian science and technology delegation on Thursday.  On Thursday this week I had the opportunity to introduce some of the members of Alan MacDiarmid’s family to the Institute after many had travelled to join us at the opening of the Alan MacDiarmid building at Victoria University.

Our patron

Alan MacDiarmid was New Zealand’s most recent Nobel Laureate.  He was born in Masterton on 14 April 1927, and although he spent most of his career was spent in the United States, he maintained strong links with New Zealand.  He attended school in the Hutt Valley near Wellington and took a Masters degree in Chemistry at Victoria University of Wellington.  However, the majority of his professional life was spent at the University of Pennsylvania, after PhDs at Wisconsin and Cambridge.  Sadly Alan passed away in Philadelphia on the 7th of February 2007, just days before he was due to travel to New Zealand to attend one of our conferences.

Alan’s Nobel prize was awarded in 2000 for his part in the discovery of polymers that conduct electricity.  Most of us take it for granted that polymer-based materials like plastics are good electrical insulators.  This is a pretty good assumption unless they are made from some of Alan MacDiarmid’s conducting polymers.  Many of the new smart phone active display technologies now rely on conducting polymers for instance.

After Alan won his prize, he embarked on a New Zealand lecture tourin 2001.  Alan was a superb public speaker and he drew crowds at every venue he spoke at around the country.  This was timely reminder to the public that Kiwis could do world beating science.  Alan’s story of hard work, collaboration and a little bit of luck was also an inspiration to many scientists.

The Centres of Research Excellence

In 2001, the government decided to experiment with a new way of funding research at universities.  At the time New Zealand was widely regarded as having one of the most competitive systems for funding research in the world.  In a new approach, the Centres of Research Excellence (CoREs) were set up to try to encourage collaborative research between institutions.

Late in 2001, the Royal Society of New Zealand was asked to run a competitive tender process to select the CoREs.  I was associated with two initial proposals, one led by Richard Blaikie at the University of Canterbury and another led by Paul Callaghan (now Sir Paul Callaghan) at Victoria University of Wellington.  However, only a year out of my post-doc at IRL, I was not sophisticated enough to see that these two proposals should be combined.  Luckily, the Royal Society called first for expressions of interest, and then published these on line, allowing wiser heads to put two and two together before the final selection process began.

The MacDiarmid Institute was born out of the union of these two proposals and today this gives the Institute a multi-institutional character unmatched by any of the other CoREs.  Paul was the founding director of the Institute, serving from 2002-8, while Richard, who had been deputy, took over in 2008.  Alan MacDiarmid played a key role as the Institute’s patron in our early years; his presence at our first two conferences in 2003 and 2005 turned them into major international events.

Has it worked?

Well, yes, but I guess I would say that wouldn’t I?  Actually, my interest in collaborative networks was sparked by some work by Sally Davenport and Urs Daellenbach from Victoria’s School of Management who decided to look at how successful a delocalised “Centre” could be.   One of the things they did was to construct co-authorship diagrams which showed that not only did the Institute’s productivity climb sharply, but that we were collaborating more widely with one another.  This is something I have picked up with my studies of co-authorship and co-invention.  The figure on the right shows the Institute’s co-authorship network from 2002-8 of the 2008 cohort of Principal Investigators.

There are many other measures that we have seen improve, including our relative citation impact and our external (non-CoRE) research income.  In fact our citation impact today places us up with some of the very best research institutes in the world.  As I see it now, the Institute brings scale and scientific excellence to materials science and nanotechnology in New Zealand.

Why did it work?

I think there are many things responsible for the improved performance of researchers in the Institute.  Most important was the example set by Paul and Richard in working so effectively across institutions.  In particular, Paul was an inspirational founding leader who was able to unite forty principal investigators from seven different institutions around the country in a common purpose.

By allowing the Institute to carry his name and by taking such an interest in our activities, some of Alan MacDiarmid’s mana rubbed off on us – this also helped break down the institutional barriers that had come to characterise New Zealand science in the 1990s.  People in the Institute were proud to put the MacDiarmid Institute as their primary affiliation – I remember how good it felt as a young researcher to give talks overseas with the MacDiarmid Institute logo on my powerpoint slides.

There are a number of other factors I think were important.  I will highlight a few here:

• Our two capital injections enabled us to purchase world class shared equipment that would have been very difficult for individual institutions to afford.  Several of our main collaborative nodes seem to be based on particular pieces of equipment.
• Alan’s success in communicating science to the public was an inspiration to Paul, and his job as director gave him the mandate and the resources to pick up where Alan had left off.  Paul describes it as the start of the science communication business in New Zealand.  Not surprisingly, scientists like working for organisations that have good public profiles, and the profile that Paul built for the Institute made us all proud to be part of it.
• In 2005, we created our Science Executive committee to make executive decision making within the Institute more collegial.  Most of our scientists get to serve on this committee at some stage. It helps bring institutional balance to our decision making, something that is so important for a distributed research centre.
• We make sure that we scrutinise our own performance as closely as possible.  Continued membership of the Institute is not guaranteed.  Every three years the Science Executive reviews each of our scientist’s performance both on measures of scientific excellence and productivity, but also on their wider contributions through outreach or commercialisation for instance.  We have also held two science reviews by panels of international experts who put our performance in an international context.

Importance to New Zealand

In think the MacDiarmid Institute’s success will prove very important to New Zealand in the long run.  Some of this benefit will come from the companies we spin out and the talented graduate students we produce of course.  But perhaps more importantly I think that the Institute has exemplified a new way of doing things in New Zealand.  By assembling teams of scientists on the basis of merit and skill rather than geography or institution, New Zealand can create scientific research institutes which compete with the best in the world.

On Thursday, I will be back on Bryan Crump’s radio show (Nights, Radio New Zealand National, 8.42pm, Thursday July 15th).  This week, we will continue our discussion of transistors, several billion of which are currently helping you read this article.  Last time, we talked about how quantum mechanics allows transistors to work as electronic switches.  This week, Bryan wants to discuss how transistors became so embedded in so many of the technologies we rely on in the modern world, and what exactly they are doing there!

A valley of silicon?

Although the idea had been around since the 1920s, the first transistor was made by John Bardeen and Walter Brattain at Bell Labs in New Jersey in 1947.  It was made out of germanium, a semiconducting material similar to silicon, and was about the size of something you might put on your mantelpiece.

However, it was their boss, William Shockley, who tried to commercialise the transistor.

Bell Labs has been credited with pretty much inventing the modern world (just take a look at this list of its inventions).  It has been criticised, however, for stifling the commercialisation of its inventions.  Indeed, Shockley didn’t get an opportunity to try to turn a buck from the transistor at Bell Labs.  Rather, he founded Shockley Semiconductor Laboratory on the opposite coast of the US:  in Mountain View, California.

Why there?  Well, the San Francisco Bay area already had a healthy electronics industry, which supplied components to the US military.  This meant that there was a supply of skilled electronics workers.  Nearby Palo Alto was home to Stanford University, which had set up the Stanford Industrial Park to encourage the development of high-tech industries such as Hewlett-Packard in the region.  And importantly, Palo Alto was also home to William Shockley’s aging mother.

Silicon Valley was born.

Creative destruction

Putting together a team of talented physicists and engineers, Shockley immediately set to work on developing silicon transistors.  But Shockley was a terrible manager.  Within a few years, Shockley Semiconductor was haemorrhaging its best young staff, including Gordon Moore (of ‘Moore’s Law’), who would later go on to co-found Intel.  The firm was not well placed to react to the invention of the integrated circuit by Texas Instruments in Dallas in 1958.

The integrated circuit revolutionised the manufacture of electronics.  Instead of making individual components, like transistors, separately, and then assembling them one by one on a circuit board, Jack Kilby developed a multi-step technique to fabricate the components and the circuit on a sheet of germanium all in one go.  This tremendously sped up mass production, and led to cheap, light-weight electronic devices.

However, a Bay Area company that had been founded by disgruntled Shockley employees was not far behind Texas Instruments in making integrated circuits.  In fact, Robert Noyce at Fairchild Semiconductor produced the first silicon integrated circuit six months after Kilby.  And in the end, it was Noyce’s design that prevailed.

The military-industrial complex

From its invention until the mid-1960s, the Apollo program and the US military bought almost every integrated circuit built. Costs fell dramatically as production volumes increased and companies like Fairchild began to outsource to Asia.

By the end of the decade, however, pressures on the US military budget meant that the gravy train began to dry up, and the semiconductor industry had to develop new consumer markets.  Today, you’ll find integrated circuits in cell phones, computers, and many other digital appliances.

Fabrication

So how are integrated circuits made?  The process, known as photolithography, is actually a bit like taking a photograph using film.

The layout of the circuit is defined by a light shining it through a cut-out template, known as a mask, onto a wafer of silicon.  The wafer will be covered in film of light sensitive chemicals called photo resist, which ‘cure’ when exposed to light.  Regions that are shaded by the template don’t undergo this curing process, and chemical treatments can then be used to etch these regions away, engraving a pattern defined by the template into the silicon wafer underneath the resist.

I like traffic lights

So what can be done with a circuit full of transistors?

In my previous article on transistors, I explained that a transistor is an electronic switch:  the current that flows through a transistor is turned on and off by applying a voltage at what is called its ‘gate’.

Transistors can be assembled into logic devices.  A traffic light is a type of logic device, for instance:  if the light is green light is on, the red light should be off.  We could ensure this always happened using just a single transistor.

Imagine we set up the circuit that supplies electricity to the red light so that it can be short-circuited by a transistor. The transistor will now act as an inverter: the red light will switch on when the transistor is off but will switch off when the transistor is on.

Now by allowing the transistor to be switched on and off by the circuit that supplies current to the green light, then we ensure the red light will never be on when the green light is on.

More complicated logic operations can be performed if we assemble more transistors.  If we have traffic lights running north-south and east-west, we could use the transistor that shorts the north-south red light to switch a transistor that short-circuits the green east-west light.  Thus, when the north-south green light is on, it switches off the east-west green light and so on …

Unless your cell phone is modelled on something out of the original Star Trek, it probably doesn’t work by switching red and green lights off and on.  Rather, it is adding and multiplying many, many ones and zeroes (“ons” and “offs”) using arrays of transistors assembled for the purpose.

Silicon Valley

Today, of course, Silicon Valley is a hub not only for electronics, but also for software and biotechnology.  This is partly due to the fact that those early semiconductor companies not only invented the integrated circuit, but being far from the traditional sources of finance on the east coast, also had to pioneer the modern venture capital industry.  William Shockley’s decision to set up in Mountain View and his subsequent mismanagement had far reaching implications indeed.

MoRST has released its national bibliometric report covering papers published during the years 2002-7. The team from MoRST and the Royal Society that put the report together used the Scopus database.  The major findings include:

• The rate and impact of New Zealand publications has increased during the period 2002-2007. This is especially so in the Tertiary Education sector, which appears to be associated with changes to Tertiary Sector research funding.

I looked at the rate of publication in a previous post (New Zealand’s recent bibliometric productivity) and have recently had a paper published in the New Zealand Association of Scientists’ journal, the New Zealand Science Review, “New Zealand’s bibliometric record in research and development: 1990-2008” [1], available here.

The MoRST report reaches similar conclusions to mine, except that I was not as confident that the introduction of the Performance Based Research Fund for the universities has driven the change in citation impact, as it seems to have occurred in the CRIs as well.

• While the impact of New Zealand publications is generally average for an OECD nation, there are certain disciplines (especially in the medical sciences) where New Zealand research has a higher than average impact. This is the same as in previous bibliometric findings.

Here are the top five subjects by citation impact relative to the OECD according to the report:

This type of calculation is a bit more difficult for me to do, because I have access to a different type of data set than the researchers at MoRST and the Royal Society.  Nonetheless, you can do something similar using the Scimago website.  For instance, if you are pleasantly surprised that Physics and Astronomy make the top five, you can use Scimago to see where New Zealand ranks in Physics and Astronomy by sorting by citations per document in a country comparison of papers.  As you can see below, New Zealand ranked ninth in citations amongst countries which publish more than 100 per year for Physics and Astronomy papers published between 1996-2008:

It’s always nice to be ahead of Australia!

• New Zealand is a cost effective place to do research. It has a comparatively high rate of publication per dollar of R&D expenditure.

This is also consistent with my findings on patents, where New Zealand appears to produce more patents per dollar than a number of other countries.  The figure below (taken from the report) shows that Kiwis are very cost effective indeed, although productivity per researcher FTE is middle of the road.  

My data shows that neither of these measures has changed much in New Zealand over the last twenty years.  Productivity (in papers per FTE) and publications per dollar have remained static.  As I suggest here and in my NZ Science Review article, the national increase in publication rates has been driven by increases in researcher FTE in the tertiary sector.

There is plenty of other material in the report to talk about, including some nice network diagrams illustrating collaborations between institutions.  Also, the authors have made some institutional comparisons of citation impact.  I will comment on that in a future post.

[1] S. C. Hendy “New Zealand’s bibliometric record in research and development: 1990-2008”, New Zealand Science Review 67, 56-59 (2010).

The Black Swan by Nicholas Taleb (Random House, 2007, 366 pages).
The Wisdom of Crowds by James Surowiecki (Anchor, 2005, 336 pages).

Do markets work?  Writing from perspectives that precede the recent financial crisis, Nicholas Taleb, Wall Street trader turned academic, and James Surowiecki, economics correspondent for the New Yorker, offer insights into the strengths and weaknesses of market forces. 

Surowiecki opens with an account of a pre-industrial version of that school gala staple, guess the number of jelly beans in the jar.  While Surowiecki’s seventeenth century protagonists are interested in the weight of an Ox, rather than a quantity of sweets, he notes that the average guess of a crowd in either case is typically very close to the actual answer.  The first half of Surweicki’s book is devoted to understanding under what conditions crowds are able to perform such feats of wisdom. 

What has this got to do with markets?  Well, for a market to allocate goods efficiently, the agents trading in such a market must act rationally to maximise their own well being.  However, experiment after experiment has shown that people do not always behave rationally, and while there is only one way to act rationally, while there are a myriad of ways to act irrationally.

Surowiecki’s wise crowds offer a possible way out: in the same way that the average guess of a crowd can get close to the true weight of the ox, maybe a crowd of irrational agents can, on average, behave rationally.      

In the second half of the Wisdom of Crowds, Surowiecki investigates under what conditions groups might fail to be wise.  For instance, he suggests that individuals in a crowd must be able to make their decisions independently, without influencing others.  If this condition is not satisfied, bubbles can grow and pop in a market as individuals follow others rather than make their own decisions.     

In fact, it is the failure of crowds, and in particular markets, that concerns Nicholas Taleb.  In The Black Swan, Taleb is scathing of the modern financial system that systematically underestimates the risks of rare but high-impact events.  For instance, the models used by traders to value complex financial products assume that stock prices will fluctuate as if they were undergoing a ‘random’ walk, precluding the possibility of rare but high-impact events.

How do such high-impact events occur? Under Surowiecki’s conditions, where the decisions of a large ‘crowd’ of traders are independent, the fluctuations in stock prices may be well approximated by a random walk as the financial models assume.  The crowd acts as if wise and markets operate efficiently.

However, if the decisions of traders become correlated, if traders start to follows the decisions of others, then the central theorem no longer applies – stock prices will cease to fluctuate according to a random walk.  Under such conditions bubbles can form, leading to rare but high-impact collapses such as that experienced in the recent financial crisis. 

Indeed, stock market data shows that just prior to, and during a crash, the decisions of traders become highly correlated.  Surowiecki devotes several chapters to discussing the consequences of such “group-think”, including an in depth post mortem of the dangerous consensus that developed within NASA’s management team that lead to the Challenger disaster.

Ultimately, Taleb is concerned with more than just bubbles and group-think. His ‘Black Swans’ include any low-probability but high-impact event, including the arrival of far-reaching innovations like the internet, wars or terrorist attacks. As I have discussed previously, such rare events seem to be important for progress in science; the ‘Black Swan’ of science lead to scientific revolutions.

Can crowds ever be wise in the face of ‘Black Swans’?  Taleb is pessimistic; Surowiecki would offer a conditional maybe.  Many of us (although not all) would agree that stockpiling supplies of was a prudent response to the possibility of a high mortality flu pandemic that threatened in the winter of 2009.  Yet to perform a cost-benefit analysis of this precaution of the type used by financial markets to hedge against risk is clearly impossible.  If one were charitable, one might say that governments fell back on a precautionary principle; Taleb would argue that such principles could be usefully applied to markets.             

Both books offer insights into the way people interact and make decisions.  Despite being written prior to the events of late 2008, both offer insight into the recent financial crisis.  As Taleb notes, we have a compulsion to invent post hoc narratives for each ‘Black Swan’ and the recent crisis is no exception.  It is quite possible a reading of these two books will leave you with a deeper insight into recent events than anything that has been written in their aftermath.    

Having reached my fiftieth post, I will indulge myself a little by reflecting on some of the posts that have been and gone. Writing this blog has been both a lot of fun and a lot of work. Thanks to google analytics I can see that people do read it, although some articles attract considerably more readers than others.

The top five articles by page views are listed below:

1. Is superconductivity the wrong science for New Zealand?
2. Kiwi superconductivity overcomes resistance
Superconductivity reigns supreme and indeed the topics for both these articles were suggested by readers.  The first generated a lengthy discussion in the comments section, and as a result this article has the most page views by a wide margin.  However it still leads even by unique page views.  If you try to ‘pick winners’, you will to generate a vigorous debate on what those winners will be.

3. The New Zealand skills deficit
This post was only my seventh, but the issues raised have continued to come up in later posts. It will be worthwhile continuing to track the numbers of graduate by field. In particular, I hope that the numbers of physical sciences PhD graduates in New Zealand does not continue to fall. This is also the only data-driven post in the top five.  The rule of thumb that every equation halves your readership seems to extend here to graphs and charts!  This probably shouldn’t surprise me.

4. New Zealand’s productivity paradox
This was a six-part series and the ranking here is based on the average number of page views across the series.  Individual posts in that series would rank higher in this list. The idea for this series came about after several astute readers noted the overlap between my patent analyses and Philip McCann’s work on New Zealand’s economic geography.  Actually, the link between Philip’s work and mine was first made last year by Howard Fancy from Motu after he came to a talk I gave on networks of inventors at MoRST.  I finally decided to launch into the series after seeing a brilliant talk by Philip at the Reserve Bank earlier this year.  

5. Giving a great scientific talk
I originally wrote this with my research group in mind (see below – you will see that a couple of them might also benefit from a post on “how to show due respect to your supervisor so that he will write you a letter of recommendation that will actually get you a job …”), but realised that it might have a wider readership.  Some of the advice is based on a Physics Today article from the early 1990s:  in those days, the most important tip for giving a scientific talk was to avoid scattering your transparencies over the lecture theatre on the way to the podium.  

After fifty posts, I would also like to thank some of the people that have helped me with this blog.  Most importantly, I want to thank my wife, who edits many of these posts over breakfast and tells me when I am not making sense.  If you spot a typo in a post or I seem more obtuse than usual, it likely means that she is overseas for work (or that I forgot to take the rubbish out).

Also, thanks to the hard-working guys at the Science Media Centre, who have been a pleasure to work with on this and other projects.  Finally, thanks to the many readers who have sent me material to blog about.  I don’t always have time to follow up with a post on everything that is sent in but it is typically very stimulating.