Ivan Snook: Assessing teachers - a plea for caution
OPINION: The recent report by the New Zealand Initiative (" the report" ) is, in one respect, a significant step in the right direction; it may also be a step too far.
In recent discussion of teaching in New Zealand it has been assumed that the achievement of students and schools can be directly attributed to the work of teachers.
In its most naïve form, the claim is made that "good teachers" (that is those whose students achieve good grades) should be singled out (and somehow rewarded) and those who do not should be identified (and somehow punished).
The report points out how wrong-headed this proposal is since it takes no account of the nature of the students or the progress they may make over a period of time.
READ MORE: NZ Initiative calls for better measures for schools to help parents make decisions
Thus it would mask the good work of schools with students from disadvantaged backgrounds and allow schools with students from wealthy backgrounds to "cruise".
The point has been reinforced by international studies. For example, in a study of 25 school systems, the OECD found that "the first and most solidly based finding is that the largest source of variation in student learning is attributable to differences in what students bring to school – their abilities and attitudes, and family and community background (OECD, 2005).
The OECD, however, goes on to make the additional point that most work on teacher quality has been too narrow, focusing on students' test scores and teacher characteristics that are easily measured.
The teacher characteristics that are harder to measure, but which can be vital to student learning, include the ability to convey ideas in clear and convincing ways; to create effective learning environments for different types of students; to foster productive teacher-student relationships; to be enthusiastic and creative; and to work effectively with colleagues and parents.
The report makes just this mistake. Most of these qualities will be overlooked by the assessment procedures suggested and teachers will be categorised on a narrow range of test scores in relation to the school's intake. The approach it advocates has been labelled as "Value Added Measurement" (or VAM for short) and various experiments have been carried out in some American states and, for a time, in England.
In general, VAM scores have been very unstable, with dramatic fluctuations from year to year and high rates of measurement error.
It is true that some econometric studies find important connections between teachers and student achievement. However, these studies face serious statistical and analytical problems and yield serious error rates.
It is calculated that if there were 100,000 teachers in a system and three years of student achievement data were used, at least 5000 underperforming teachers would be rated as superior (and hence be
worthy of performance pay or other rewards) and at least 5000 teachers who are superior would be identified as underperforming (facing censure or remedial work).
Researchers have also found that the results of such teacher appraisals are very unstable: one study found that only one-fifth of the improvement remained after one year and only one-eighth after two years. These schemes are, to say the least, notoriously unreliable and their use for teacher assessment grossly unfair.
It is a major irony that the report advocates the use of a variant of deciles at the very time when the government plans to implement a different method for funding schools. Deciles (however, they are generated) have their use but they are not without problems.
It is instructive that a scheme very similar to that suggested by the report (called Contextualised Value Added) was implemented in England by the Labour government early this century but was abandoned by the Coalition government on election in 2010.
According to the press release, Education Minister Nikki Kaye says she is "very interested in looking at what can be done to more accurately reflect the impact that schools are having on their students, which means being able to better measure the growth that students are making".
It is to be hoped that the minister, who has expressed strong reservations about the use of school deciles, will have second thoughts about using a variation of deciles to "measure the growth that students are making" or the quality of their teachers.
Ivan Snook is Emeritus Professor of Education at Massey University and the author of The Ethical Teacher (Dunmore Press 2003), and several books on values in education. He is the first author of a report The Assessment of Teacher Quality (Massey University 2013).
- The Dominion Post