Lies, damned lies, and statistics

There are three kinds of lies," thundered former British Prime Minister Benjamin Disraeli. "Lies, damned lies, and statistics."

It is often in the world of politics that the worst of the three lies is most often used. Given that it is election year, it is worth looking at the use of statistics to justify what seems, with the benefit of hindsight, to have been little more than heat, noise and little else.

In a recent newsletter by the New Zealand Institute for the Study of Competition and Regulation, an article highlights some of the problems of oft-quoted statistics and their impact on Government and business activities.

As the authors, Glenn Boyle and Bronwyn Howell, note, the Organisation for Economic Cooperation and Development statistics related to broadband penetration have become something of a "gold standard" in recent times.

Government agencies and interest groups have been grumbling about New Zealand's "low" ranking in terms of broadband penetration and have offered and prescribed any number of policies to boost our national standing.

Our standing of 21st out of the 30 countries in the OECD has led to dire warnings about being left behind in the development of the digital economy and such like.

It has made rather good press for those complaining, but has not been particularly easy for those on the other side, such as Telecom, which has often been portrayed as the guilty party holding New Zealand back from this "inevitable" digital nirvana we must share in.

One could even argue that some of the Government's policy related to the splitting of Telecom into a three-headed beast is a direct result of wanting to boost our international standing as far as broadband penetration goes.

Of course, everyone wants to be a winner and there is cold comfort in knowing that our rival Australia is five places ahead of us at 16th. We want to be in the top tier (and beating Australia), and the Government and interest groups want to get us there.

If you cast your mind back to 1999, the Government was all for returning New Zealand to the top half of the OECD in terms of gross domestic product per capita. This resolve was quietly dropped when it became clear the goal didn't exactly gel with the Government's redistributive tendencies.

We might not get back to the top tier in gdp terms so we might as well have a go at getting into the top tier with something as simple as broadband penetration, right?

And this is where the tricky issue of statistics comes into play.

For the OECD, making comparisons across countries is often best achieved using "per capita" statistics as it takes away a lot of distortions that can creep into the numbers. But it can also lead to other distortions creeping in.

Think about it in these terms: the statistics used to compare us to the rest of the world are based on broadband penetration per capita.

But broadband is sold on the basis of connection to a business or home. See the problem? The statistics are using a different measurement base compared with what actually happens; apples are not being compared with apples.

The implication is that using a per capita measure of broadband penetration is not going to get you where you want to go. But it is the "gold standard" after all.

In April 2007, academic George Ford highlighted the problem of quoting the OECD broadband statistics without understanding the measurement basis. In a presentation to the United States House of Representatives Energy and Commerce Committee on the Digital Future of the US, he asked a simple question: what would the rankings look like if every country in the OECD had 100 per cent broadband penetration into homes and businesses?

The answers aren't going to please any Government politician or the "self" interest groups. New Zealand would charge up the rankings to 16th – but still be in the bottom half of the OECD. The US would go from its current 15th to 20th and Australia would fall from 16th to 27th! How? Why?

The answer is in the measurement base. Broadband is sold to households and businesses, not to people individually. The statistics that everyone likes to compare everyone else with are "per capita" based. The differences come about because of differences in gdp, household sizes and the way businesses are structured.

Taking the last point first, New Zealand has a very large number of very small businesses, most of which will be based at home, with the business and the home sharing the same broadband connection.

Household size has a major impact as well. A country with more people per household will have a lower ranking than a country with fewer people per household, simply because of the average number of people living in each house.

This is partially explained by gdp per capita numbers: richer countries tend to have fewer people per household than poorer countries.

Boyle and Howell note that 62 per cent of the difference in the current broadband penetration rates between OECD members can be explained by differences in gdp per capita. That's a significant explanatory factor for just one measure.

There is plenty of room to increase broadband penetration, not only in New Zealand but also in many other countries, with policy initiatives that are useful in helping to drive penetration higher.

However, focusing on our OECD ranking as a measure of our "success" will not get us anywhere and may lead to any number of useless and counterproductive actions.

As I noted at the outset, it's election year; be very careful of politicians and other self-interest groups brandishing statistics.

They may not even realise that in calling for "improvements" they may be asking the impossible – and the implications could be "damning" for us all.

- © Fairfax NZ News