How effective are expert witnesses?
In a Christchurch courtroom in May, prosecutor Brent Stanaway reminded those tasked with deciding the guilt or innocence of murder-accused George Gwaze that this was not a trial by experts but a trial by jury.
They could have been forgiven for thinking otherwise, and for being bamboozled by the complex scientific and medical argument advanced by both sides. Did the Zimbabwean vet's -year-old niece, Charlene Makaza, die from asphyxiation or overwhelming HIV infection and systemic shock following massive diarrhoea-related fluid loss? Did the traces of Gwaze's sperm get into Charlene's underpants by sinister means, or was it transferred in the wash?
The Crown called a slew of doctors and pathologists who maintained the only explanation for Charlene's death was sodomy and suffocation. The defence countered with four international experts supporting its contention she died from the HIV ravaging her body.
The one critical question the jury did not have to decide, was how 12 people with no scientific credentials could be expected to weigh conflicting evidence and agree on a conclusion when experts could not. And what would have happened to Gwaze had the defence not doggedly questioned evidence often regarded as unimpeachable?
Just a month later, in the trial of Ewen Macdonald for the murder of his -in-law Scott Guy, the quality and objectivity of expert evidence again came under scrutiny. There were many powerful images that emerged at the High Court at Wellington – of weeping witnesses and defence lawyer Greg King in full flight. But perhaps the one that defined the collapse of the Crown case was that of ESR forensic scientist David Neale, under King's cross-examination, counting the rows of wavy lines on the fore-foot of a Pro Line dive boot. That simple measurement – included in Neale's original working notes but not in his written evidence – served to show that the only piece of physical evidence linking Macdonald to the scene did not match the size 9 boots the police said the accused once owned.
In his closing address an angry King accused Neale of fudging the evidence. "An expert witness is entitled to do what witnesses of fact can't do, and that is to express an opinion. But with that comes responsibility ... He didn't even bother to count the rows on a size 9. It's a murder trial. My client deserves better. And you, having the burden of the Crown saying that you should convict him, deserve better."
IMPARTIALITY IN A PARTISAN SYSTEM
"Having introduced their written evidence with the protestation that they will be independent and unbiased, experts routinely go on to give evidence that is anything but," senior lawyer and former judge Robert Fisher, QC, told an expert witness conference last year. "The boilerplate declaration of impartiality has as much impact as the staple at the top of the page."
Unlike most witnesses, who can only give evidence on questions of fact, expert witnesses are allowed to draw inferences and give opinions based on their experience, exposing them to accusations of bias. That danger is exacerbated by an adversarial system pitting one set of experts against the other, and the fact experts are engaged and paid by one side (ESR forensic scientists are paid by police and defence experts are paid by Legal Aid, via defence lawyers).
In two recent cases (see On The Case) judges have berated both Crown and defence experts for failing to have an open mind.
The potential implications of biased expert evidence are obvious. Almost every criminal trial of any complexity includes the evidence of at least one expert witness, ranging from pathologists giving autopsy reports and ballistics experts matching ammunition types to paediatricians weighing whether a child's injury is accidental or the result of child abuse. The often-touted "CSI effect" suggests juries place a heavy weight on forensic evidence, and it is not uncommon for criminal trials to boil down to a tussle between experts.
A Law Commission discussion paper on expert evidence noted "injustice sometimes results when experts interpret their role as being to support the case of the party who hired them and only to bring to light evidence supporting the other party when they are specifically asked".
Independent forensic science consultant and The Forensic Group director Anna Sandiford, who gave evidence about bloody sockprints in the retrial of David Bain, says there is huge variability in the degree of impartiality among the scientists whose work she reviews. In about 70 per cent of cases she agrees with the Crown expert's view.
"When I get a report I can look at the names of the individuals involved and get an idea of whether there is likely to be any problem. Some scientists are far fairer than others ... Which is why it's always important to at least assess the science in every case. If people know there's a good chance their work's going to be examined, then they're more likely, even if it's subconsciously, to remain less partial."
Sandiford says New Zealand defence lawyers seem less inclined to question scientific evidence than those in Britain – where she worked previously – based on the misunderstanding that science is unimpeachable. That failure to review scientific evidence inevitably results in miscarriages of justice.
Criminal trials, she says, should be covered by the same expert code of conduct used in civil trials, which reminds witnesses their duty is to the court, not the party paying them.
"Juries can put a massive amount of weight on an expert witness, so you really have to be careful about what you say and always keep to the fore of your mind that our duty is to the court not to the people instructing us."
Under cross-examination in the Macdonald case, Neale said that, discounting the number of waves, the impressions at the crime scene matched a Pro Line boot, or similar.
"Why would you discount the rows of waves? Why would you discount it?" King asked, incredulous.
And why would you, except to fit the evidence to a predetermined scenario, rather than using the evidence to determine the truth?
ESR did not respond to requests to interview Neale, but Your Weekend put that question to ESR forensics general manager Keith Bedford, who reviewed Neale's evidence and trial transcripts, following King's criticism.
"Looking at David's evidence it would be quite fair to describe it as inconclusive, and Greg King highlighted that. I'm satisfied that he was not pressured into trying to fit his evidence to the prosecution case."
Bedford acknowledges the risk of bias when scientists only ever provide evidence for one side. That risk is managed by training, clear ethical standards and reviews of court performance.
All expert evidence is also peer reviewed by another ESR scientist, and witness statements are prepared independently of police and prosecutors, he says. To knowingly suppress information of benefit to the defence case would be deemed serious professional misconduct.
"We will not allow evidence to be put in our mouths ... That said, prosecution or interested parties may ask for a clarification or expansion on something that's not clear."
Auckland prosecutor Simon Moore, QC, says it's not uncommon for ESR witnesses to provide information that is both helpful and prejudicial to his case, suggesting they are impartial.
Paediatricians at Auckland's Starship Children's Hospital's child protection unit have also been criticised for being too quick to interpret injuries as child abuse (see On The Case). That and the Gwaze case prompted lawyers to suggest that treating doctors should not give opinion evidence.
But child protection unit clinical director Patrick Kelly says cases usually only reach him after several other doctors have flagged concerns. And most cases never make it to court.
"The first problem you face is a diagnostic one – is this most likely to be an inflicted injury? If I get that wrong upfront that can potentially result in a child being unjustly removed from their family, or a child being left in a dangerous situation and being further injured. So it's hugely important that I'm objective from the moment I walk into a room with a child. To suggest that somehow it's more important that I'm objective in a courtroom, where only a minority of my children end up, is, I think, based on a misunderstanding."
With New Zealand's specialist paediatric head-injury knowledge concentrated at Starship, it would be practically unworkable to have all evidence reviewed outside the unit, Kelly says.
While he has no philosophical problem with giving evidence for the defence, Kelly says he can't spare the week it would take to review a difficult case, so turns down three or four defence requests a year. He also declines Crown requests for the same reason. "If you do this kind of work as a crusader, wanting a particular outcome in a courtroom, you are inevitably going to come a cropper.
GUNS FOR HIRE
While experts who work solely for the prosecution draw allegations of potential institutional bias, experts appearing primarily for the defence risk being dubbed guns for hire.
It is a criticism often levelled at -abuse expert and Auckland University general practice professor Felicity Goodyear-Smith, who acted as medical adviser in the Gwaze case, advising defence lawyer Jonathan Eaton how to cross-examine Crown witnesses, and finding international HIV experts to test the Crown evidence.
Goodyear-Smith began giving expert evidence in 1981, as a sexual-abuse examination doctor for police. She appeared hundreds of times for the Crown before becoming concerned that Doctors for Sexual Abuse Care (DSAC), of which she was an honorary life member, was acting as an advocacy group rather than an impartial forensic organisation. "By suggesting that some accusations may not be genuine, I seen as a heretic."
Goodyear-Smith's position was further undermined by her later links to Centrepoint, having married commune guru and paedophile Bert Potter's son. She initially resisted giving evidence for the defence, realising "I wasn't the ideal person to be waving the flag". But -one else would do the work.
"I know DSAC accuse me of being a hired gun, but you wouldn't do this for money. It took a huge toll on me. It was very unpleasant to stand up; you are not treated well. In the end I almost had nothing to lose. Whereas some of my other colleagues said 'We can't stand up against this, we've got families, we've got jobs, we could lose them'."
Defence lawyers are also accused of expert shopping – ringing around until you find someone to say what you want. One expert says defence lawyers ask him not to contact the Crown witness whose opinion he is reviewing, so that if he agrees with their view the Crown will not know the defence sought a second opinion.
Helen Cull, QC, who defended David Bain and John Barlow, rejects the idea that defence witnesses are hired guns, and the criticism of expert shopping.
Self-proclaimed experts sometimes offer their services in high-profile cases, Cull says. But no defence lawyer would call a shonky scientist, because they would never stand the scrutiny of -examination. "You cross-examine them to make sure you are not calling a flake. You don't want to be caught red-handed ... Most experts take their evidence extremely seriously and will not be pushed beyond a particular boundary."
Both Goodyear-Smith and DNA expert and Independent Forensic Practitioners Institute president Arie Geursen, who also worked on the Gwaze case, are giving up as expert witnesses. Geursen, who received a 2011 New Year's Honour for services to forensics, calls the adversarial system "stressful" and "brutal".
FACT OR OPINION?
The Gwaze trial turned largely not on physical evidence, but on what that evidence meant. Contrary to popular belief, science is often a matter of interpretation. Which is how you end up with experts proffering unreconcilable explanations for the same facts. The question is, do juries understand the difference?
"We refer to it as the CSI effect," explains Geursen. "People have grown up in this culture where we think forensic science can answer every question in a case. It does run the risk of being given much greater weight by a jury than it perhaps deserves. Juries perceive it as not being vulnerable to human frailty."
Geursen says that once you introduce mixed samples and incomplete data, juries should be warned about the uncertainty of interpretations of that evidence.
In Gwaze, the key question was not whether it was Gwaze's sperm on Charlene's underpants, but how it got there. Geursen pointed to Canadian research showing sperm could be extracted from never-worn underpants washed with semen-soiled underwear.
"Science, forming an expert opinion, is a human endeavour just like everything else. I don't think it's appropriate for that to be just accepted hook, line and sinker without being checked to see that it stands the scrutiny of its peers," Geursen says.
And internal peer review is not always good enough. Oregon lawyer Brandon Mayfield was held for two weeks on suspicion of involvement in the 2004 Madrid bombing, which killed 191, after a scene fingerprint was misidentified. Mayfield – a convert to Islam – accused the FBI of religious bias. A report faulted the FBI's peer review, concluding "to disagree was not an expected response".
Fingerprint matching is now internationally accepted as opinion rather than fact, says Sandiford. So is that how it is presented in court?
"No, and that's also not people's general understanding of what fingerprints mean ... You have to retrain people in how to approach the interpretation, and that's a big challenge."
Bedford says it is the expert's responsibility to make clear what is interpretive opinion and what is -and-white.
the most scrupulously fair expert can fall victim to subconscious bias. Research by University College London psychologist Itiel Dror reveals experts can change their opinion depending on context.
In one study, five fingerprint examiners were given print pairs to compare and told the FBI had mistakenly identified them as coming from the same person. The samples were, in fact, pairs the experts had themselves judged as matching five years earlier, but only one expert reached the same conclusion as their earlier self. Subconscious bias is "potentially very dangerous" and should be minimised by judging every piece of evidence in isolation, Dror says.
Bedford says Dror's research is powerful, and ESR is looking at ways to limit subconscious bias. Take the system for peer review of tool-mark matching. A second opinion is sought only if the first examiner thinks the mark matches that at the crime scene. "We need to be mixing in some exclusions, so the second examiner doesn't always assume he's being asked to confirm what somebody else has already seen to be a match."
To ensure objective analysis, ESR replaces any identifying details with barcodes on the 40,000 samples processed annually by its DNA labs. But some context is essential to interpret DNA results, Bedford says.
EQUALITY OF ARMS
Despite the system's vulnerabilities, Cull believes it works as long as both sides have equality of arms. That means both equality of access to experts and funding parity.
With ESR scientists contracted solely to the Crown, and just 34 listed members of the Independent Forensic Practitioners Institute, defence lawyers struggle to find experts to give second opinions. That presents two problems – either they use a jack of all trades, or they look to international experts whose evidence can cost anything from $5000 for a simple review, to more than $20,000 to give evidence in person.
"The biggest vulnerability for expert witnesses is when they go out of their area of expertise," Cull says. "They may well be a forensic scientist but they may not be a ballistics expert. In New Zealand, with a limited pool and limited resources, often experts are asked to go beyond where their expertise really ends." Take child abuse – with the expertise concentrated at Starship, and Starship doctors already giving evidence for the Crown, defence lawyers must look overseas, often to Adelaide specialist Terry Donald. Donald is presently reviewing five New Zealand cases, and while he does not strongly disagree with those Crown opinions, he believes defence lawyers need better access to qualified experts. "The most important issue is that more thought is given to how the defence might get useful high-level opinions in cases where there is a serious concern about a child being seriously physically assaulted."
An expert's evidence for the Crown is usually only part of the puzzle, but the defence might call only one expert, making access to good-quality opinions even more crucial, Donald says.
"You have an absolutely amazing level of responsibility."
Donald points out that payment for defence experts is also "incredibly slow". "You wouldn't want to be trying to earn a living out of it, I can tell you that."
Cull advocates more funding for both sides, to enable access to the best international expertise. And greater parity – in a recent complex insurance case, Cull was offered funding for only one expert to rebut the Crown's eight.
Goodyear-Smith says Gwaze's defence team had to fight for funding for its four international experts, filing a 50-page report to justify its request.
A BETTER WAY?
"I'm just not sure that experts squabbling in front of a jury is particularly helpful if you want to get to the truth of the matter," says Geursen. "How will juries get on top of that evidence? I can tell you: what they do is they decide whether they like the defence experts or whether they like the Crown's experts."
Prosecutor Simon Moore, who acted for the Crown in the Chris Kahui trial, agrees trials can become a battle of persuasive presentation, rather than an objective weighing of the science.
"The trial process is intrinsically human. That's why I think it works relatively well ... A group of people from all different walks of life apply their collective experience and wisdom to an assessment of the various players."
Cull also has faith in juries, after jury questions in the Bain retrial showed they were making sense of expert evidence.
She argues it is the cross-examiner's job to test the science and expose the expert who sounds credible but does not stand scrutiny.
But is there a better way? "It's a very important question to ask and keep on asking," says Bedford, who rejects the idea that cross-examination is like the alchemist's refiner's fire, burning away the dross and leaving the gold. "It is often as much about the ability of the expert to present themselves, rather than the true weight of their evidence."
One option is pre-trial conferences, where defence and crown experts agree areas of agreement and disagreement. It is an option Moore favours, and one he says can lead to experts changing their opinions. Any mechanism to make expert evidence less adversarial would be a good thing, he says.
Another alternative is hot-tubbing – an inquisitorial process used in some civil and coronial cases, involving lining up experts in a courtroom, or pre-trial, allowing lawyers and judges to quiz them, and enabling experts to comment on colleagues' evidence.
In the case of the Kahui twins murder case, the scientific evidence was weighed differently in the criminal trial and coroner's inquest. Patrick Kelly, who gave evidence at both, found the hot-tubbing coronial approach useful, providing instant peer review.
"Everyone who answered those questions knew they were surrounded by colleagues who were perfectly competent to pick them up if they said something outlandish."
While pre-trial expert conferences to establish common ground are useful in principle, in practice they require a skilled facilitator, Kelly says.
Kelly says the system works OK. But he asks if OK is good enough.
"It's a human system and it's fallible and it has upsides and downsides. Why not pilot expert witnesses meeting before court with a court-appointed facilitator as a matter of routine? Or trial hot-tubbing and see if it makes a difference. If it doesn't, you haven't lost anything."
The Dominion Post