Trying to make sense of Ontario election polls? Ignore them, say experts

Veteran pollsters have a word of advice for Ontario voters getting seasick from riding the shifting tides of the province’s election survey results: Ignore them, they’re probably wrong anyway.

At least 15 polls have been released since Premier Kathleen Wynne dissolved the legislature at the beginning of the month, and their varying results offer no consensus.

One day a pollster reports Tim Hudak’s Conservatives could form a majority government, if the numbers hold through to election day on June 12. The next day, a different pollster is out with numbers suggesting the Liberals are on course to be re-elected.

Seasoned pundits observing the campaign from the sidelines say the numbers flooding the headlines are inherently flawed and of limited value to the discerning voter.

Polls commissioned by mainstream media outlets share few traits in common, they said, adding firms use different methodologies, questions and sample sizes to obtain their results.

The one quality the polls do share, however, is a focus on surface questions that don’t tell the full story of what drives voters to the ballot box and don’t offer an accurate picture of what will happen once they get there, the pundits said.

Indeed, unreliable polls stole headlines away from election victors in Quebec and British Columbia during their most recent campaigns, as the final vote counts differed sharply from survey projections.

Veteran pollster Allan Gregg, who currently serves as Principal at Earnscliffe Strategy Group, said the general questions around voter intentions don’t mean much, since most people make their final decisions in the waning days of the campaign.

Polling firms working for the media also don’t have the financial resources to develop sound survey practices, he said, adding the political parties are the ones with the funds to invest in developing solid numbers and tracking more meaningful trends.

“It’s an entirely different beast, what the political parties are doing versus what the media is doing,” Gregg said in a telephone interview.

Daniel Cohn, professor of Public Policy and Administration at York University, said the problem runs even deeper than lack of finances.

Accurate research takes both money and time to amass, he said, adding the demands of today’s political climate make quick, cheap and inaccurate data collection the only option.

“To do a decent public opinion poll that’s going to be representative of public opinion because it’s going to be as close to random as you can get, that’s going to take you a couple of weeks,” he said. “And in a four-week election campaign and a 24-7 media cycle, that’s just not acceptable.”

The most common polls on the Ontario election have collected their data through online panels or automated telephone surveys, and experts argue both methods have fundamental flaws.

Online polls survey a pool of participants who have volunteered to offer their opinions. As a result, pollsters say the findings are not random and are therefore not necessarily representative of the whole population.

This view is shared by the Marketing Research and Intelligence Association, the polling industry’s professional body, which says online surveys cannot be assigned a margin of error due to their lack of random sampling.

Experts say randomness is also an issue with the cheapest form of telephone polls, known as Interactive Voice Response (IVR) or automated polls. Telephone polls by their nature tend to favour older voters who are more likely to both own a landline and pick it up when it rings, they say,adding some pollsters have begun including cellphone users in their samples.

Cohn said technology has compromised this method even further by shrinking the size of the respondent pool. People are increasingly ditching their landlines in favour of unlisted mobile phones and use common features like call display to screen out pollsters and their questions, he said.

Some research companies have tried to address this by revamping their IVR systems to keep up with the times.

Frank Graves, president of Ekos Research, says his firm has included cellphones in the polls conducted for the Ontario election.

His team also favours an approach of calling participants more than once and drafting multiple versions of the same questions to limit bias.

Both exercises cost time and money, he concedes, adding the gap between survey results and election-day figures usually has more to do with poor voter turnout than faulty research.

“That doesn’t mean it’s just ‘a pox on all your houses.’ No. You can do good, sound polling,” he said. “It would be great to devote more resources to do it better, but we don’t put out any poll that we don’t think is right.”

Some experts also say that telephone polling has its place when conducted the old-fashioned way.

Dimitri Pantazopoulos relied on live phone interviews while collecting polling data for B.C. Premier Christy Clark during the province’s last election.

While nearly all public surveys forecasted a win for the NDP, Pantazopoulos masterminded the only poll to accurately predict that Clark’s Liberals would carry the day.

Pantazopoulos said his focus wasn’t on soliciting responses from the largest swath of the population, adding sample sizes become less relevant after the first few hundred interviews.

He said his approach of running a smaller rolling poll throughout the campaign, rather than concentrating on specific field survey dates, allowed him to address what he feels is the key shortcoming of most mainstream surveys.

“They’re focused on the ballot number. Nobody’s sitting there and asking people what’s motivating people to vote, what’s going to change the undecided, what will move weak voters over to the other side,” Pantazopoulos said of media polls.

Pollsters then compound their problems by trying to explain their findings using factors they haven’t analyzed.

“They say, ‘oh, these guys have gone down. It must be because of the 100,000 public service cuts,'” Pantazopoulos said in reference to one of Hudak’s most controversial campaign pledges. “Well, maybe, but it may be because people don’t like the leader’s hair cut or something. We don’t know. Why go through all the effort of being scientific and then guess as to what the reason is?”

Gregg’s advice for voters musing on ballot-box outcomes?

“Wait till election day. You’ll find out sooner or later.”

Top Stories

Top Stories

Most Watched Today