Lies, damn lies and polls

According to a recent poll, two out of three people believe there are too many surveys published in the media.

Conducted by me last night in my apartment, the research found that more than 66 percent of those asked thought newspapers, in particular, were simply too full of the findings of polls, studies, surveys and non-academic research.

One of those harangued into answering (my girlfriend) said she "didn't mind what I marked her down as thinking", as she was busy watching TV.

I, of course, said there were too many such surveys (Why else would I be writing this?), while the third respondent simply hissed - him being a red-eared slider turtle, with apparently no strong feelings on the subject.

The media have long used research for the basis of reports, and there is little wrong with that, as long as the findings put forward are of value. Polls are fun, and when related to lighter stories provide an easy way for a reporter to tell a tale.

Who doesn't want to know that 83 percent of Latvians prefer tobogganing to tightrope walking as a weekend leisure activity?

It's all good stuff, because it doesn't mean anything, it doesn't matter, it is insignificant.

And therein lies the rub.

When unaccredited, unauthorized, untested survey results are used for the basis of news reports, how can people tell if what they are reading actually means something?

As a psychology student many years ago, I spent countless hours conducting research. But before I was allowed to submit my findings, I first had to test their significance.

In scientific terms, research results mean little unless they can be shown to be statistically significant. In mathematics, the chi-square test, for example, is used to determine whether test results might simply have been achieved by chance, while in academic and commercial research, population sizes and demographics can have a huge impact on the end results.

Of course, it would be ridiculous to expect journalists to carry out scientific tests on all the data they cite, but they should at least be aware of the fact that in many cases, they are referring to little more than opinion.

Online polls are a prime example of how the views of at best unaccredited, at worst uniformed, individuals and groups are all too often presented as valid, or even as fact.

Before the advent of the Web and online forums, the opinions of the woman in the local cake shop or the man in the pub would hardly have been regarded as significant in research terms. Now, however, people just like them are known as "netizens" or "members of online communities" and it seems the media can't get enough of what they have to say.

It is not that their views are unimportant, but the way in which they are compiled and presented most certainly is.

For instance, in a follow-up survey conducted in my apartment just a few minutes ago, I discovered that now, just 50 percent of those asked thought there were too many surveys in the media. Unfortunately, my girlfriend is at work, and Charlie the turtle's response was unintelligible, as he had a mouthful of cabbage.

Is that significant? Probably not.

(China Daily 04/02/2008 page20)