Academic profiling – unwise, unfair, unethical, but common?

By - Wayne Linklater 29/10/2013 5


University rankings can be useful. They can also be used inappropriately to discriminate inaccurately and unjustly.

Dr. Mark Hauser at Harvard University – consistently amongst the world’s highest ranked universities – was found guilty of scientific miscounduct, fraudulent science in 2012 – an example of why the university a person is employed at is a poor metric of their credibility or future performance (source: www.nytimes.com)

To students making enrolment decisions, or governments deciding how to invest in institutions, a university’s rank represents the relative and average quality of its measured parts – staff or student achievement, research output and influence, teaching quality, or resources. It is a population- or institution-level metric only, not a measure of any of its individuals – duh, right?!

But I have observed a rise in academic profiling – university ranks sometimes also being used to discriminate amongst individuals.

The differences between the ranks of universities are small compared to the range in the quality of their parts. Some departments, staff, and students perform poorly while others outperform at all institutions. A university’s rank, therefore, is a poor measure or predictor of the quality of any individual that works or studies there.

Why, then, have I developed the impression that university rankings are increasingly being used to evaluate the merits of individuals and the value of their supporting referees by committees awarding scholarships to students, or sabbatical leave and promotion to academics?

Sir Paul Callaghan GNZM FRS FRSNZ (1947 – 2012) had his DPhil from Oxford University – ranked 2nd in the world – but chose to work in New Zealand at Victoria University. Victoria University’s world ranking dropped 28 places this year to 265th. Amazing people choose to work in much less ‘amazing’ universities.

At first encounter this practice seemed reasonable. Highly ranked universities generally employ and enrol more talented staff and students, are better funded, and maintain better reputations. It seems useful, therefore, to value individuals at or from those institutions relatively highly.

BUT… using a university’s rank to evaluate an individual’s application or support is like racial or sexual profiling (see text box below).

A person can be at a lower ranked university but be an international leader. Academics at some of the world’s best universities do some very bad work. Many graduates from the world’s top universities perform poorly, despite their greater opportunities. Some very talented people graduate from the world’s lowest ranked universities.

Sir Don McKinnon ONZ GCVO – alumni of one of New Zealand’s lowest ranked universities (Valuation and Farm Management, Lincoln University,1960-61) but also previously Nobel Peace Prize nominee, Secretary-General of the Commonwealth, and Deputy Prime Minister of New Zealand. How unimportant is the rank of one’s university?

Most people choose to work or study at a university largely for reasons other than the institution’s rank. Many choose universities near their families, or in their communities or home countries. People all over the world move from higher to lower ranked universities – actively choosing, for example, to make a contribution to the future of their country or community of birth, or raise their families in places familiar. Many choose or return to lower ranked universities because they aspire to achieve in ways that cannot be measured by the rank of the institution they work or study at.

I understand that committees formed to award student scholarships or academic promotion have a difficult job. They are deciding the winners and losers with far-reaching consequences for individuals. Using a rank of the university that a referee is writing from, or student graduated from, to judge their value simplifies their decision. The contemporary Performance-based Research Fund, amongst other ranking systems, and the competitive culture developing amongst institutions encourages the practice.

But using an institutional address to evaluate an individual is academic profiling. Academic profiling is not a reliable predictor of individual performance, and it is unethical. It is also bizarre, or perhaps just laziness, because applicants provide an extraordinary level of individual detail for committees to evaluate their suitability and merit. The university address they, or their references, come from is rendered redundant.

I hope my impression that academic profiling is on the increase is incorrect or, at least, exceptional. I’d be interested in hearing from colleagues about how often they encounter it. I have encountered two examples in as many months.

Next time someone says to you an applicant’s university isn’t good enough, or a reference less valuable because it comes from someone at a lower-ranked university, suggest politely that they do their job and evaluate people, not their address.

profiling text box

Postscript – written from the university ranked 8th in the world while on sabbatical from the university ranked 265th. I look forward to returning to my home university at the end of my sabbatical for reasons unrelated to its rank.

Post-script (15 Nov. 2016): For a recent review and account of metrics role in higher education see article in Times Higher Education ‘Metrics: how to handle them responsibly’ here.

 

 


5 Responses to “Academic profiling – unwise, unfair, unethical, but common?”

  • Wayne
    You should have been much tougher than this. Very few ranking systems that I have seen stand up to the slightest scrutiny.
    The university rankings are no exception. There is an overpowering urge to aggregate the results of a variety of critieria to come up with a single, simple league table. This is done quite cheerfully despite all the particpants knowing it’s absurd.
    The most you can say is that some universities seem to be good at some things, some of the time.

    • Thank you for your comment.
      I suspect you are right. There is very much to be written about the fallacy of ranking such complex institutions with multi-faceted objectives and roles. Nevertheless, in this instance I just wanted to focus on the absurdity of using an institutional rank, of whatever ‘flavour’, to evaluate individuals. In my experience the practice is growing and I am concerned it is counter-productive.
      Regards,
      Wayne.

  • Wayne,

    I’m a little unclear as to when you think this profiling is taking place.
    Is it the university where one graduates from, as implied by your Don McKinnon example, or the university where an academic is currently employed?
    I think you will find that quite a few “high flying” academics do actually move to higher ranked universities – for example, Prof Callaghan was one of my lecturers at Massey University, a university with a lower ranking than Victoria. I know several very high flying academics who have worked their way from lower ranked universities to those with a higher rating.
    However, I also agree that other academics certainly consider other factors such as family needs in deciding where they work – from what I’ve seen movement to higher ranked institutions often occurs when academics have no, or very young children, or later in their careers when their children have moved on.
    Higher ranked universities will also actively “court” high flying academics.
    My impression is that those who understand academia are more likely to judge people in their publications than their institution.

    • Thank you Michael for your comment.
      Academic profiling occurs wherever a population- or institutional-level metric is used to evaluate an individual. Examples might include, but not be limited to:
      a. using the rank of the university a student studied at as a measure of their merit compared to other students,
      b. relegating the value of a professional relationship with individual academics at other institutions based on the latter’s rank,
      c. using the university address of the author, rather than their individul status, when deciding how much value to place in their letter of reference for a student or academic.
      Thus, in answer to your first question, it can be both and occur in other contexts as well.
      The key problem with academic profiling is that institutional-level ranks do not represent individuals any more than other population-level rankings, like race or sex, do. They will, therefore, result in poorer decision-making.
      My recent observations of academic profiling lead me to be concerned that it occurs and might be increasingly common.

  • University rankings are much loved by university central administrators who increasingly come from non academic positions to “rule” and prefer an undemocratic management style. In such corporate universities the central admin people don’t have the skills to evaluate their employees, and so external ranking systems, and access to them at an employee level, are essential. Nevertheless, there is among them a tension between rankings at a university level and a desire to simply manage for financial “success”.