Understanding National Standards results
Is the publication of results:
(a) A disaster.
(b) A turning point.
(c) A triumph.
(d) All of the above.
Don't fret. There's no easy answer.
Four years of fury and conjecture over the Government's controversial, but popular, national standards policy came before today. Now, for the first time, the results are public.
We can see, for example, that about 8 per cent of the nation's primary and intermediate age pupils have been placed well below the standard in writing. We can see that about 35 per cent were put above the standard for reading - better than the 22 per cent of Pacifica pupils and the 25 per cent of Maori who made the same grade.
Parents will, inevitably, want to know where their child's school fits in to the national picture. But how much store should be put in the figures? Does the national standards data prove anything at all about the quality of a school?
There is a unanimous expert opinion - even among those championing the potential of the National Standards - that it would be very foolish indeed to make judgments about any school on the basis of their results. Not yet, anyway.
The standards, which were rushed in to law as one of the first acts of the National-led Government in 2009, are in to their third year of use for children in years 1 to 8. Since 2010, schools have been required to report to parents at least twice a year on the progress of pupils against the standards, but it was only in May this year that school-level results had to be sent to the Education Ministry.
The Education Review Office (ERO) estimates that about 72 per cent of schools had taken up the standards by the end of last year - an improvement on 60 per cent the year before.
But just using the National Standards is not a simple matter. Teachers don't just open a book marked "National Standards: Maths, Year One" and run a bunch of tests from the book on their class.
When the standards were developed, experts successfully lobbied the Government not to base them on the national testing that is used in other countries.
Instead, the National Standards results have been decided by "Overall Teacher Judgements" (OTJs).
A whole mixture of things goes in to any one OTJ.
Informal assessment by watching the daily activities of a pupil is one. Figuring out what a pupil knows and their ability to express themselves by talking to them is another. And weight is also given to results from more formal assessments, including standardised tests like PAT, STAR and e-asTTle.
Dr Jenny Poskitt, who devised a system to help teachers make OTJs on the National Standards, says teachers need a range of skills and some pretty in-depth knowledge to make solid OTJs. First, they must know the curriculum very well; they must also understand a fair bit about child development; and they have to know what counts as robust assessment and what doesn't.
Getting all of those skills together for every teacher in the country - and marshalling them for use specifically on National Standards - takes time and money.
"What we've learnt from overseas is that reliability, that consistency of judgment takes a number of years of professional development and lots of time spent in teacher discussions to get that consistency," Dr Poskitt says.
"We are not there yet and we are unlikely to be there for three to five years."
Up to now, only a few schools have got to the point of checking their use of the National Standards against others. There has been no formal moderation between schools, and the support for teachers appears to have missed the mark for some.
The first of the four independent evaluation reports into National Standards implementation, carried out by Maths Technology Ltd (MTL) and published last year, found only one-third of teachers were using current assessment evidence for reading and maths OTJs, and a little over half for writing OTJs.
"Most principals described themselves as minimally supported or unsupported by the Ministry of Education. The areas in which principals felt most supported were making OTJs, and reporting to families and whanau, while they felt least supported to moderate OTJs," the report found.
Glennis Vinton, principal of Stanley Bay School in Auckland says schools are reading ministry guidelines for the standards "in very different ways".
"Some are being very lenient in their decisions and some, like us, being very conservative," she says.
Even if the standards reach a point where they are considered more robust, many schools still have doubts about their usefulness and concerns about their potential to cause harm.
"Data hides a lot of stuff," Edendale School principal David McKenzie says.
"Numbers conceal human beings with all the events that shape and make up their lives."
McKenzie notes the untold impact on National Standards results of things like attendance, the amount and quality of pre-school learning and transience.
"The children that we assess from one year to the next can be quite different," he says.
"As a general rule we work on the measure that each shift [of school] impacts a child by negative one term. So if a child is onto their fourth school they have lost four terms which equals one year's worth of learning. Therefore, the reason a child may be below has little to do with the quality of teaching and learning systems within a school, but more to do with the level of instability that is being experienced within the child's home life."
But McKenzie is not wholly against the standards.
Like hundreds of other schools that gave their National Standards data to Fairfax, Edendale School plans to use its data to help pick its priorities for the following year.
"We use the data to help us consider the areas of our school where there are learning needs for the children. We then will set goals for the following year based upon what the data is showing," McKenzie says.
That probably qualifies McKenzie as something like a model student.
Both Education Minister Hekia Parata and ERO chief review officer Graham Stoop are most enthusiastic about the potential for the National Standards data to drive powerful changes in what schools do for their pupils.
Stoop admits it's "early days", but he says the standards are already showing signs of potential.
"If schools are finding on the basis of their National Standards information that there is a group of students that aren't meeting the standards, the question that ERO wants to know is, well, what are you doing about that, what plans have you got in place? What self-review mechanisms have you got in place to help you with making sure that in time those students do meet the standards," Stoop says.
"Potentially, as they're developed further, there will be some good conversations that we'll be able to have about how they're going to use that National Standards information to do something different, innovative perhaps, systematic with the students who need that assistance."
This well-recognised and highly-regarded concept of "assessment for learning" as opposed to just "assessment of learning" has the support of the Government's academic favourite, Professor John Hattie.
In his well-known synthesis of over 800 meta-analyses relating to achievement, Hattie said the best schools boast pupils that are "assessment capable". Kids who know how well they are doing and where they are going to next will invariably get there, Hattie found.
This is perfectly possible without National Standards, but Stoop insists they have given it an extra push.
"In the past, we often found the schools didn't have that [assessment] information or they had it, but they weren't doing anything with it," he says.
"With National Standards, all schools have data and what we're interested in is what schools are doing with that data."
Perhaps most significantly, if they are successful, the standards should improve the links between home and school.
A focus-group study conducted by Colmar Brunton for the Education Ministry in July found a perception among parents that school management tended to only tell them about successes. Parents wanted to feel confident that achievement reported as good or improving was genuine and they could ask the right questions if it was not, the study found.
Poskitt, a former primary school teacher, says schools have historically struggled to give it to parents straight when a child is struggling.
"It's often been coloured or mystified and parents haven't understood," Poskitt says.
"The standards have the potential to give that clarity of telling parents earlier - look, your child is behind."
Once they know what's needed, most parents will move mountains to help their kids succeed, she says.
And there are few things to match the potency of well-engaged parents for a child's learning.
Given the weaknesses of the National Standards data, however, parents are being actively encouraged in a new campaign by ERO to look at a whole range of things when searching for the right school for their children.
Its website has had a revamp and includes advice to parents on "the six dimensions of a successful school".
According to ERO, parents should probe primary schools with questions like what exactly "on track" to meet a standard means, or how much work their child might need to do to catch up.
"We want parents to be increasingly involved in the education of their children, knowing about the school and being able to make informed decisions about which school is best for their children or child," Stoop says.
The importance of one of the six dimensions - school culture - is shown in a 2010 survey of 1168 Kiwi school children in school years 4-8. The study, published in the Educational Research journal, found involvement in bullying was related to empathy and classroom climate, with bullies and their victims having the lowest connection to school and the poorest relationships with their teachers.
Hattie suggests parents watch the playground atmosphere for cues.
"Do the students look each other in the eye? Or do they avoid each other, or sit in cliques," he suggests parents ask.
That's not the kind of thing parents will typically read in an ERO report. While Stoop insists the ERO is improving the parent-friendliness of its reports, Dr Poskitt has an even better idea.
"Nothing beats going in to a school and meeting a principal, walking around the playground and getting a sense for the way of life at that school," Poskitt says.
"How friendly they are, how at ease you feel and if you feel that you could go in with any concerns about your child and be listened to and have them do their best for you and your child.
"Those are really important things that an ERO report can't tell you and nor can National Standards results."
HOW TO RECOGNISE A GOOD SCHOOL
Professor John Hattie suggests 10 indicators for parents:
1. In the playground, do the students look each other in the eye? Or do they avoid each other, or sit in cliques.
2. Diversity breeds fresh thinking. Can they show you genuine evidence it is encouraged?
3. How do they measure success? By the achievements of the few or of the many?
4. Ask to meet the best teacher. If they tell you they're all good, they're not thinking clearly.
5. Who do students turn to? Every student should have someone who knows how they are doing and will spend time with them.
6. Do new students make friends in the first month? It is a critical indicator for success: how does the school make sure it happens with all students?
7. Do they like mistakes? Learning starts from not knowing, so do they embrace that? Do students feel confident enough to talk about errors or not knowing something?
8. Are students "assessment capable" in this school. Can they talk about how well they are doing, where they are now and going next?
9. Do they use acceleration for all? Are students enabled to learn at different speeds?
10. What feedback do students get? Ask - "what did you get told about your work today"?