WalesHome.org

YouGov has come to Wales - but will online polling add value to our politics?

OPINION RESEARCH has come a long way since 1916, when the Literary Digest pioneered national public opinion polling in the United States. Back then, the focus was very much on quantity, rather than quality of the sample itself, with the Digest simply sending out millions of self-completion surveys and adding up the responses. Though the Literary Digest did well in correctly predicting five consecutive presidential elections, it all came crashing down in 1936 when the clear sample bias towards its readership became apparent.

The alternative approach, spearheaded by ‘new kid on the block’ George Gallup, focussed on quality over quantity and looked to target a specific demographic selection of respondents. This approach beat the Literary Digest in predicting the Roosevelt election of 1936, and went against conventional wisdom in the UK by predicting Clement Atlee’s success at the general election in 1945.

Since then, polling companies and academics have tracked public opinion in the UK for every general election. When YouGov burst onto the political polling scene in 2001, it came at a time when the world was changing to the extent that traditional opinion research methods were being re-evaluated. At the 1992 general election, the polling companies infamously predicted that the Tories would lose their majority, and both 1997 and 2001 saw less-than-perfect polling performances. The general election result of 2005 came as a great relief to polling firms all over Britain, which saw them once again victorious in their predictions. But what were these problems faced during the 1990s, and have things now been solved?

If we consider ‘conventional’ sampling wisdom, we can start to see how things might have gone wrong in the 90s. In an ideal world, researchers would contact a purely random sample of names and addresses from the electoral register, and one hundred per cent of those contacted would agree to take part. However, we all know that a 100% response rate on polling of this scale is utterly impossible to accomplish, meaning that there are clear implications on the quality of the sample achieved.

We also know that politics changes quickly, which is why many opinion polls are these days conducted over-the-phone, using randomly generated landline telephone numbers. Again, these depend on a high response rate in order to ensure that any kind of sample bias is kept to a minimum. Are those at home when they are called attitudinally different to those who are not? And moreover, are those who agree to take part in opinion research attitudinally different to those who do not?

The rise of ‘yuppie’ culture and the mobile phone in the 1990s saw the beginning of people becoming increasingly mobile, increasingly busy and, therefore, increasingly non-contactable to landline telephone callers. Today, the younger population in particular is becoming a purely mobile generation, with landline telephone usage dwindling. Those who do have landlines are increasingly screening their calls in order to avoid that ubiquitous sales call. All this suggests that accurate opinion research needs – for its own sake – to embrace modernity.

This is why YouGov designs its samples and why it conducts its polls online. Relying on randomness is all well and good when you have a 100% response rate, but avoiding sample bias means not missing a certain ‘type’ of respondent, and when it becomes apparent that certain ‘types’ of respondent are being missed, it’s time to rethink the methods.

YouGov has a panel of more than 280,000 British adults (nearly 20,000 in Wales alone) from which we select samples designed to be both demographically representative (ensuring our samples are correctly balanced in terms of age, gender, region and socio-economic status) and attitudinally representative (controlling for newspaper readership and political party identification).

The benefit of conducting polls online means that respondents can take surveys in their own time, and they can be honest and frank in their responses, without feeling the need to ‘socially-satisfy’ an interviewer by admitting to something for which they worry they might be judged (such as voting Tory or – perhaps increasingly – voting Labour!).

The debate surrounding YouGov is largely limited to the debate surrounding internet penetration: when only 70% of the population in Wales has access to the internet, how can we be sure we’re not missing a certain ‘type’ of respondent, too? The answer is, again, in our sampling. As YouGov doesn’t rely on randomness to create its samples, we can be sure that we’re contacting the right number of people in the right demographic groups. The 30% of the population not online is predominantly older, predominantly of a lower socio-economic status and predominantly hidden from view. That does not mean that they are peculiar; people are, luckily for us opinion researchers, not all that unpredictable. In every group defined by demography, political preference and newspaper readership, we have specific people on our panel with whom we can construct representative samples: we can reach into every corner of Welsh society. And when it comes to political research, the views of a 79-year-old tabloid-reading Labour-identifying woman from Builth Wells who is online are unlikely to be much different from a 79-year-old tabloid-reading Labour-identifying woman from Builth Wells who isn’t.

The elephant in the room here, some would argue, is Welsh language proficiency, which deserves a mention, given the amount of correspondence I have received in recent weeks from concerned members of the Welsh public (my own mother included). The difficulty in defining a ‘Welsh speaker’ in quantitative terms, coupled with the lack of any useful statistics on Welsh language competency means that it is an unsuitable variable to accurately weight data by. I could claim to speak Welsh because of what I’ve seen written on road signs and having learned how to order ‘un peint arall, os gwelwch yn dda’. The 2001 Census didn’t ask about level of fluency; simply ‘can you understand, speak, read or write Welsh?’. Based on my previous statement, I could plausibly claim to do all four. I’m also likely, when asked by a researcher, to somewhat inflate my ability because I know that some form of ability is socially expected.

Our political party ID target percentages (derived from a number of sources, including respondents’ past vote, 2005 election results and respondents’ own stated party affiliation) encompasses well the attitudinal difference in Welsh respondents. While age, gender and even socio-economic status no longer give us a valuable insight into a respondent’s likely political preference, party ID and newspaper readership clearly can. Moreover, a weight for Welsh language competency would assume on a total level that those who speak Welsh are significantly different from those who do not, which is increasingly not the case: as we know, English-only speakers in Wales do vote Plaid and do support further devolution (just as Welsh speakers, on occasion, do not). While attitudinally there may be some differences between those who speak Welsh as a first language and those who don’t, to weight all our polling data by an unreliable, unknown figure, would lead to a misrepresentation of findings.

So can any of this explain why polling in Wales has been relatively thin on the ground? One of the main issues with opinion research in Wales historically has been similar to issues seen by those polling Britain more generally over the years: an inclination to overstate Labour Party support. For a long time, this was easy to conceal behind claims that Wales was a broadly ‘red’ country anyway, with most people erring on the side of left-wing-ism over conservatism. However, with Plaid Cymru having gained ground over the years and the Tories appearing recently to ‘mop up’ the residue of Labour disillusionment, methodological issues within Welsh opinion polling have been flung open to renewed scrutiny.

That said, the main reason for a lack of polling in Wales appears to have been as a result of a lack of clients commissioning the research. Few have commissioned polls in Wales, and those who have have had to commission often very expensive bespoke polls. Without a decent political Welsh Omnibus it has been difficult for people to commission just a small piece of political research, which is where YouGov can now step in. Confident, as we are, that our methods do not fall victim to the same public sector worker overstatement of years gone past, we’d like to think we can contribute towards a useful new kind of political discourse in Wales. In fact, back during the European election campaign our internal research was throwing up one ‘odd anomaly’ (which drove us repeatedly back to the drawing board) which suggested that the Tories were going to do better in Wales than Labour.

There is, of course, an ongoing argument about the extent to which political polling measures public opinion, and the extent to which it shapes it. The small amount of political polling we’ve done in Wales since our launch in October has been picked up by all Welsh media outlets, reported all over the country and discussed in the blogosphere extensively.

My view is that we cannot live in a representative democracy without those we’ve elected to represent us knowing what we think. Knowledge of public opinion is vital to keeping our elected politicians in check, both as a form of basic accountability and as a means through which our politicians can test their proposed policy objectives and, if needs be, shape them to better reflect our views.

As long as the methods are sound, and no one ‘type’ of voter is being over-represented, then political polling can be a vital part of the democratic process. This is why YouGov has, and always will, welcome debate and advice on all of its methodology. Transparency is something YouGov subscribes to wholeheartedly; it is integral to informative opinion polling and to the democratic process itself. And we’re proud to play a part in both.