Appraisal doesn’t equal Evaluation

By Marcus Wilson 25/02/2010

I spent yesterday afternoon in a seminar discussing how my teaching can be analyzed for its effectiveness. One much used word is ‘appraisal’.   Students may recognize that as meaning those annoying questionnaires that get thrown in front of them in the last two minutes of the final lecture of the year, in which they need to answer questions on the content of the paper and the performance of the lecturer. There’s a vague implication that the university wants to know from its students how they feel about the teaching they have received, but it’s not often made clear.

It was very interesting to learn yesterday exactly how different lecturers use the appraisals.  Students might be in for a shock here. It depends very much on the lecturer.  Some lecturers carefully go through every form, pull out the major themes from student comments, think how they can improve and make changes for next year’s teaching.  Then, next year, they pick their appraisal questionnaire questions carefully so they can assess whether their changes have been effective.

But some lecturers do nothing with them (other than maybe check their overall score isn’t too bad.)   All that time the poor student has spent identifying three ways in which the lecturer could improve, etc – the lecturer may choose not even to read it, let alone do anything about it.

Also, something that struck me this morning, is that the appraisal questionnaires that we gave our students here at Waikato in 2009 are not very much different from those I had to fill out as a student in Cambridge in 1989.   Twenty years have passed since I first sat in lectures and, on the face of it, not much has actually changed in terms of how the quality of teaching is reviewed (or not).  I now wonder what happened to the forms I did back then – did anyone take notice?  

But there is actually, as I’m learning, a huge amount of research on how to evaluate your own teaching effectively.  So why do we seem to focus so much on those questionnaires? 

0 Responses to “Appraisal doesn’t equal Evaluation”

  • Because the drive in academia is to find more mindless ways to measure what we do, knowing that for the most part the data will sit on some hard drive or database to never see the light of day ever again. Or if you prefer, we seek data in ever increasing resolution without defining what it will be used for and independent of any incentives we have to use it.

    The problem is that questionnaires are often low quality information, respondent fatigue sets in quickly, it’s sensitive to class size, proximity to tests, how well they did in assessments etc. Some students like the examples you use, some students hate them.

  • Sounds like you’ve had a few bad experiences here…

    I often find it a real struggle to extract opinions out of my students. Most physics and engineering students don’t say very much – in tutorials I frequently get barraged by total silence. Of course, there is always the odd vocal student who will make his feelings clear, but these are not necessarily the feelings of the whole class. Tricky.

    Anyway, this year I’m working on some sneaky tricks by which (I hope) the students will be giving me feedback and may not even know they’re doing it. All without an appraisal form in sight.

  • I don’t think it’s exactly a matter of bad experiences 🙂 Just a lot of information-gathering takes place with not much recognition of whether the incentives or structures exist to utilise that information.

    In general I do consistently well on student appraisals (& one of the side effects of that is often getting reassigned to papers to rebuild numbers). 🙂

    But student appraisals can & do get influenced by a lot of factors. So extracting useful information is not always easy. And in recent times, there’s simply not the structures to deal with the problems. If my paper budget gets cut, then I have to do more with less, and get a lower appraisal. Everybody knows what the problem is- but we can’t fix it because the budget’s the binding constraint. Fewer and bigger tutorials aren’t good, slower turn around on assignments because of less employed markers etc- become negatives.

    Nonetheless, there is one area where I think we are starting to get good information- and that’s from the chat that goes on in the online forums we use.