Statistics Canada says more than two-thirds of Canadians willingly filled out long census questionnaires, despite fears few would bother after the governing Conservatives turned it into a voluntary exercise.
But even as figures released Friday showed 69.3 per cent of households replied to the controversial replacement for the long-form census, statisticians raised concerns about the quality of the data.
“You simply can’t get reliable data from a voluntary survey,” said John Brewster, who teaches statistics at the University of Manitoba and is president of the Statistical Society of Canada.
“I mean, this is something we teach in every course in statistics, for example. The data will almost certainly be biased. And we don’t really know at this time the magnitude or direction of that bias.”
Statisticians and users of census data have warned that a voluntary survey would skew the results because certain segments of the population — the rich, the poor and aboriginals, for example — would be less inclined to fill it out.
“I was expecting lower,” said Don McLeish, a former president of the society who teaches statistics at the University of Waterloo.
“Of course, until we know a little bit more about the missing 31 per cent, we won’t really know very much what biases result.”
Still, the results of the so-called National Household Survey beat Statistics Canada’s expectations. The agency says it assumed only half the households that received the form would fill it out.
“We’re very pleased with these results,” said Marc Hamel, director general of the census management office.
“They’re pretty much in the order of what we obtain on other social voluntary surveys here at Statistics Canada. So it’s a very important first step, but it’s a good one from a collection point of view.”
The Northwest Territories had the highest response rate at 83.9 per cent, while the Yukon had the lowest with 61.1 per cent.
Statistics Canada’s goal was to collect complete forms. However, the agency accepted incomplete forms if necessary. It is not yet known how many people answered a single question, some questions or all questions.
“Generally speaking, though, Canadians, once they commit to complete one of our surveys, will complete the entire form,” Hamel said.
“This is what we generally observed on all of our surveys.”
The federal government scrapped the mandatory long-form census last year and replaced it with a similar, but voluntary, document called the National Household Survey.
Cabinet ministers argued that it would be a suitable replacement for the long census, but the claim prompted the unprecedented resignation of Munir Sheikh, Statistics Canada’s chief.
Sheikh and many others have argued that a voluntary survey can never replace a mandatory survey. That is mainly because different slices of society will tend to approach a voluntary survey in different ways, leading to a distortion in responses.
Expert after expert has warned that the most vulnerable people and small communities in Canada will not be properly reflected in the responses.
Recently, Sheikh wrote in an essay that “it is a statistical fact that a voluntary survey cannot be a substitute for a mandatory census because of uneven response rates from different population groups and different size geographic areas. Increasing the sample size cannot offset this problem. Hence many data users, including the federal government, will lose the data quality that they need.”
The long-form census was the backbone of many of the agency’s other surveys, as well as policy decisions made at every level of government.
Adding to the concern was the publicity surrounding the cancellation of the long form. Many analysts worried that the Harper government’s denunciation of the census would discourage respondents from filling out the new voluntary form.
“The challenges were huge because of an easy confusion between the National Household Survey and the short-form census,” McLeish said.
“The first real success on their part is to maintain the rate on the short form. Because I think a lot of people would have understood from the verbiage surrounding this debate that all censuses were voluntary.”
Statistics Canada says it received the mandatory short form from 98.1 per cent of Canadian households, up a full percentage point from the last census in 2006.