As discussed in an earlier post, there are a number of sources for bibliometric data. Scimago Journal and Country Rank is a freely accessible bibliometric analysis site developed by a Spanish research group using Elsevier’s Scopus bibliometric database, which holds country and journal summary information. For example, there is a New Zealand summary statistic page that has data on publications each year since 1996.
The first thing that leaps out at you on the NZ summary page is the large increase in publications per year evident from 2003, as I have replotted on the right. This increase is substantial: NZ has gone from publishing 5000 scientific articles per year to more than 8000.
Actually, 2003 was the year in which the first performance-based research fund (pbrf) assessment round was held. This was part of a change in the way university funding was allocated, from a system where funding levels were set largely by full-time student numbers, to a system where levels are partially determined by research performance. The performance measures used were based on the quantity and quality of research performed by individual researchers, with assessments taking place in 2003 and 2006.
It is tempting to attribute the growth in annual publication to the pbrf exercise, with researchers responding to this assessment by increasing their output. However I mentioned in an earlier post that Statistics NZ provides an estimate of the number of full-time equivalent (FTE) researchers in the university, government and business sectors every two years. This allows us to calculate the number of papers per FTE researcher, which is a measure of researcher productivity. On the left I’ve plotted the number of university and government FTE researchers (including post-grads), and the productivity in papers per FTE researcher from 1996-2006.
This shows that while total publication output has increased significantly, so has the number of FTE researchers, leaving productivity in papers per FTE surprisingly static. Most of this increase in FTE researchers comes from a large expansion in post-graduate student numbers. In many disciplines, we are now training more post-graduate students than ever before. This is good news (especially given the discussion here), but as I’ll discuss in a later post, this growth in post-grad numbers is not uniform across the disciplines.
Thus on the face of it, the introduction of the performance-based research fund has not led to an increase in bibliometric productivity. However there are claims that the pbrf has led to an increase in research quality, as measured by citations. One way to test this is to compare university citations with those from the CRIs — this will be the subject of a later post.