The old PBRF blues.

By Peter Dearden 15/02/2012

Peter K. Dearden


We’re back

Sorry we have been away

2011 was a colossal year for us, and 2012 doesn’t look much better.

One of the key things that has been occupying our attention, and that of most of the rest of the University community, is PBRF; the Performance Based Research Fund. Every several years we are assessed as to the quality of our research, and because there is an arbitrary deadline, we all rush to get our research published before the deadline. I failed, and now have a bunch of cool papers to write that won’t count.

Part of the PBRF process is about what papers we get published and their quality. Unfortunately there is no good single measure of quality of a publication, so we put them all in. Journal impact factors, rankings, citations etc, all get quoted as evidence of how wonderful and important our work is. In the years leading up to PBRF we spend a great deal of time trying to get our papers into journals that will look good at PBRF time. What that generally means is writing up a manuscript, then sending it to a journal with a high impact factor, getting it rejecting, sending it somewhere else, rejection, somewhere else, rejection; all the while sliding down the journal rankings, until it gets into a journal. Often it gets into the journal you think it would naturally go into, but on the way it has been reformatted for each journal, peer reviewed continuously, and edited to deal with reviews and editors comments.

But PBRF isn’t just about our papers, we also get judged on Peer-Esteem and Contributions to Research Environment. To deal with this we have to develop a portfolio providing evidence of how much our colleagues like us, and how much we do to support research. Apparently the number of friends you have on facebook isn’t ‘evidence of peer esteem’.

So here at the University of Otago we have just had to submit our PBRF portfolio, and so for the last two weeks we have all been looking up our H-indices, our average cites and the rank positions of the journals we publish in. To tell the truth we have probably also been looking at everyone else’s too, just to try and work out where we might fit in the PBRF scales.

But though it changes the way we work, and writing the portfolio is a chore we could do without, I quite like PBRF. At this University, PBRF has made a huge difference, it has raised research to be a key driver of the University. Support for research is high, and the quality of research is increasing. All our senior management are required to carry out research. This is awesome. For a guy like me who loves research, working in a place where that is seen as valuable is great.

So while a number of sleepless nights and painful days have gone into the process, I am happy to be judged on what I do. And happy that the tax payers, who foot the bill for my research, might get some measure of its quality and usefulness.

Long live PBRF (now I duck for cover…)

0 Responses to “The old PBRF blues.”

  • Bang bang….
    (I’ve nearly completed mine…).
    I too am happy to be judged on what I do …. I am done so continually by grant funding bodies and internally within the Uni. As most of the grant funding bodies and the Uni are public institutions I think the tax payers get plenty of feedback on my quality. What PBRF is about is a ranking system allowing govt to justify giving more money to some institutions. I don’t think it needs a mega million dollar exercise involving every academic in the country to do that.
    (ps. I think the PBRF scores for individuals are kept confidential…so it is not as if the tax payer can see how you measure up).

    • True- you never get to see a personal score, but you can see the university score, and possibly the academic unit score. I guess the key thing is justifying sending sums of tax payers money to specific institutions for research. You at least have some indication that the money is doing things.

  • I think we can choose to see PBRF as a real pain in the back or as the easiest way to get funding for our university (odds are much better than for any grant application process: one is always going to get some money). This time I chose the latter; it works wonders for my mental health while filling out forms.

  • I’m with you. I don’t see PBRF as being a bad thing. I’ve spent several years overseas where I often saw groups of very talented people who rarely produced any measurable output. There was no motivation for them to do so.

  • On the other hand….
    The PBRF is the work of the devil. It is about the worst piece of policy I have seen in the last 20 years.
    Only about 25% of university activity is research, yet the PBRF drives the way the universities manage themselves. This is because it is the only means available for inter-university comparison. The universities then put a spin on the results for public consumption. (‘Five of our departments were ranked in the top three…etc etc ‘)
    It is loved by the VCs who believe their own university will shine, and it gives the VCs some money which they have a lot of discretion over. It’s also popular among those academics to whom research is much more important than teaching. The grading of individuals (unique to NZ) plays to the self-indulgent element in university culture.
    If you look at the growth of ‘A’ grade staff numbers by subject area since the PBRF began, you will find strong growth in areas which have scant relevance to NZ’s needs. I could go on….

    It might have been possible to design a holistic performance measurement system which captured all the factors which make a university great, but the opportunity was missed.
    There are straws in the wind that the govt is going to fix things at long last (read the briefing from the MoE to the Minister of Tertiary Education). I predict that the 2012 round will not happen- although there may be something else in its place.