Statistics Canada (mandatory long-form census)

What Happened

Statistics Canada’s mandatory long-form census was abolished in 2010 by the Conservative government despite protests from all sectors of society and testimony from the country’s two most senior statisticians, who claimed that the voluntary census would result in “useless” data. With the release of the data from the 2011 census, critics have confirmed that the low response rate and new methodology renders the National Household Survey incompatible with previous mandatory long-form censuses. The National Household Survey also compromises other StatsCan surveys which use the general census results as an anchor.  


Background

Canada’s Constitution sets out a legal requirement for a census and places this responsibility in federal jurisdiction. Statistical information provided through a mandatory census is a low-cost source of reliable and robust information about how our society works, offering the best information for evidence-based policy making. The data collected through the census helps a wide spectrum of governmental and non-governmental bodies reliably pinpoint trends and areas of concern, allowing for policy decisions to be made based on accurate information or evidence.

The mandatory long-form census was implemented in 1971. Since that time, the census has been comprised of two census forms: a short form and a long form. The short-census includes 8 questions and probes basic household composition information. The long-form census includes an additional 53 questions, probing respondents on a variety of demographic, social, and economic subjects, including things like citizenship and immigration status, ethnic origin, religion, place of birth of parents, education, income and housing, child care and support payments, labour market activities, and unpaid/household work. This data is used to plan public programs and projects such as equalization payments, Employment Insurance benefits, the Old Age Security program, and the Canada Pension Plan. The data also has an impact on public transit and transportation infrastructure, health-care infrastructure, social services, and education.

The short form is sent to 100% of Canadians and is mandatory. Until 2010, the long-form was mandatory, and was sent to 1 in 5 Canadians, with the data extrapolated to the rest of the population. While it was mandatory, the response rate for the long-form census was approximately 94%, producing data from a non-biased sample of the population and serving as one of the most important planning tools in Canada. Because this data is considered representative, data from the mandatory long-form census has been used as an “anchor”, reducing the risk of bias in other StatsCan surveys.

Because of its breadth and high-response rate, the mandatory long-form census has been one of the most reliable data sources in Canada. Reliable statistical information about all parts of society also supports government decisions to fight poverty and reduce the marginalization of disadvantaged groups. Measuring equality requires good, long-term and repeated data in order to determine if we are making progress. Without it, we simply don’t know.

Long Form Census Eliminated

On June 28, 2010, the Harper government replaced the mandatory long-form census with a voluntary National Household Survey (NHS).

The government's justification was that it wanted to ‘protect privacy’. However it was difficult to reconcile this argument with the fact that all the data is depersonalized for statistical purposes, meaning that it cannot be traced to any individual. The former federal Privacy Commissioner, Jennifer Stoddart, had called StatsCan’s protection of privacy exemplary, noting that for the 20 years prior to the 2006 census, the office had received just 50 complaints, only some of which were about the mandatory long-form census. The government also described the mandatory long-form census as coercive. No one has ever been jailed for refusing to answer census questions, and the government still chose to keep the 2011 Census of Agriculture and the short-form census mandatory.

In addition to eliminating the mandatory nature of the census, changes were introduced to the types of questions asked and some questions, for example the unpaid work question, were removed. Because the mandatory long-form census was cancelled with very little notice, StatsCan was unable to properly test run the new questions and assess how people would respond to the new methods.

In addition to these changes, accessing data is now more expensive. Where detailed neighbourhood level information was previously free, StatsCan is now charging for this data.

Due to budgetary cuts, other StatsCan surveys have been cut or compromised, including the Survey of Labour and Income Dynamics, the National Population Health Survey, the National Longitudinal Survey of Children and Youth, the Workplace and Employee Survey, the Survey of Household Spending, the Survey of Intellectual Property and Commercialization in the Higher Education Sector, the Survey of Federal Intellectual Property Management, the Annual Survey of Service Industries and the Survey of Financial Security.

Opposition to the 2010 Changes to the Census

The elimination of the mandatory long-form census contradicted advice from experts and professionals, including statisticians, economists, business people, doctors, lawyers, police officers, faith groups, anti-poverty groups, scholarly societies and advocates for linguistic minorities. At least 370 organizations from a wide cross-section of Canadian society have expressed their displeasure with the government’s decision.

Economists and statisticians defended the value and integrity of the long-form census, and warned against the biases that would occur in a voluntary survey. In 2010, the head of StatsCan, Dr. Munir Sheikh, resigned in protest. He alleged that the government mischaracterized his opposition to the changes to the census. (See our page on Dr Munir Sheikh here). In 2012, StatsCan's high-profile chief economic analyst Phil Cross also resigned , saying that internal debate at StatsCan was being suppressed in relation to questions about the mandatory long-form census.

The Canadian Medical Association was alarmed by the abolition of the mandatory long-form census, saying the decision would negatively affect the collection and use of health information.

The Canadian Federation of Independent Business, the Conference Board of Canada and the Canadian Chamber of Commerce all urged the government to reconsider its decision. The business sector had long opposed such a decision, and actually quashed a previous attempt to get rid of the long-form census by the Progressive Conservative government in 1986.

Faith-based organizations, non-profit groups, and Aboriginal and Francophone groups all spoke out in support of the mandatory long-form census, and against the effects that its elimination would have on religious, linguistic and cultural minorities, as well as economically disadvantaged Canadians.

Scholarly societies such as the Canadian Library Association, Canadian Anthropology Society, Canadian Sociological Association and the Canadian Historical Association also cautioned that the loss of “comparable, longitudinal, long-form data” would undermine the usefulness of the data as well as the aggregate statistics drawn from it. The data from the NHS would have limited utility as a historical data source and the loss of data would make it more difficult to identify and track inequality.

Women’s groups also protested the removal of the question on unpaid work, stating that the loss of this time-use data in the general census will impact the reliability and utility of the General Social Survey. The loss of this data is important given that care work is predominately performed by women and that an aging population will increase the demand for more work of this kind. In addition, women’s groups expressed concern that replacement of the mandatory long-form census with the optional NHS could lead to the undercounting of vulnerable women and girls, insufficient data to conduct gender-based analysis of programs and policies, and insufficient data to evaluate the impact of programs and policies on the status of women.

Both the Canadian and Quebec Bar Associations opposed the abolition of the mandatory long-form census, in part because the loss of robust statistical data makes it more difficult to develop evidence-based arguments in equality law cases under the Canadian Charter of Rights and Freedoms and hampers Canada's ability to evaluate whether or not it is meeting its international human rights law commitments.

The 2011 Results of the New Census

In 2006, the mandatory long-form census was sent to approximately one in five Canadian households with a response rate of 93.5%. The sampling rate for the NHS was set at one in three households, increasing the cost of the census by $22 million. The larger sampling size does not solve the issue of non-response bias. The response rate for the 2011 NHS was 68.6%, with much lower responses in many mid-sized cities, smaller communities and rural areas.

A number of communities had response rates lower than 25%, and a few communities had a response rate of zero. As a result, StatsCan sharply lowered the point at which results are suppressed at the neighborhood level due to a high non-response rate. In 2006, the point of suppression was 25%, but for the NHS, it was 50%. In spite of these changes to the suppression rate, StatsCan was nonetheless unable to report data for approximately 25% of the census subdivisions or municipalities. However, if the NHS were held to the standards of the 2006 census, the responses from 67% of Canadian neighborhoods would not be considered reliable, and therefore excluded. This suggests that the data may not be usable at the community level for planning purposes.

StatsCan had cautioned that the most significant source of non-sampling error for the NHS would be the non-response bias, which would have adverse effects on the quality of the data. As the response rate declines, the risk of non-response bias increases; the smaller the group that is surveyed, the less reliable the information. Crucially, the characteristics of people who tend not to reply to surveys are different from the characteristics of those who do respond. Amongst others, groups that tend not to respond to voluntary surveys include First Nations people, new immigrants, and recipients of needs-based payments from any level of the government. This means that the results are not representative of the population, overestimating and underestimating population counts of a number of groups. Media analysis indicates that the NHS underrepresents people on both income extremes, as well as those individuals who are most likely to need the very government services that rely on data generated from the census.

This low response rate confirms that StatsCan does not have the information necessary to conduct accurate critical assessments of Canada's economic and social needs. In addition to the low response rate, the new methodology renders the NHS incompatible with previous mandatory long-form censuses for comparison purposes, and the biases in the NHS will compromise other StatsCan surveys which use the general census information as a baseline.

Analysis of data from the NHS was released by StatsCan on five separate topics, in three waves. On May 8 2013, the first wave of analysis was released on Aboriginal Peoples, as well as on Immigration and Ethnocultural Diversity. Analysis of Education and Labour Force was released on June 26 2013, and the final wave on Income and Housing was released, after some delays, on September 11 2013.

Release of the analysis has been accompanied by much criticism. The first wave of results was described as lacking depth and breadth, leaving serious gaps in a critical area of public policy, and with StatsCan noting itself in a footnote of the report on Immigration and Ethnocultural Diversity in Canada, that some data was not in line with the administrative data from the Department of Citizenship and Immigration Canada. In its September 11 release, StatsCan indicated that data users should “use caution” when comparing NHS income data to older data because of the high non-response rate, as well as changes in methodology. In an emailed statement in July 2013, StatsCan stated that in areas with smaller populations and for some population groups, “the response rate may be insufficient to provide a valid statistical picture.”

Reactions to the 2011 Census Data

Critics have expressed concern that the NHS represents movement away from evidence-based policy making and that the biases of the NHS could allow governments to justify a reallocation of money away from programs for members of under-represented groups. Critics have also expressed concern that the data could be more easily manipulated by people with agendas.

Dr Munir Sheikh, the former chief statistician of StatsCan has described the NHS as a waste of money and has stated that the data generated is largely useless. Opposition leaders have promised to bring back the long-form census if their parties form government.

In July 2013, the City of Toronto decided that it would not use the NHS to make any historical comparisons with the 2006 census. Other municipalities have expressed similar concerns, with the City of Vancouver still assessing how to proceed with the data. The President of the Federation of Canadian Municipalities, the main lobby group for Canadian municipalities, is concerned that gaps in the NHS data will compromise infrastructure planning, as well as planning around affordable housing, transportation and bus routes, and programs for new Canadians.

Concerns have also been expressed at the provincial level. In July 2013, a spokesperson of the Ontario Finance Ministry remarked that they will be using “additional prudence in using data from the National Household Survey because of issues related to quality and comparability.” In October 2013, the Quebec government critiqued the elimination of the mandatory long-form census, saying that the NHS compromises factual and accurate knowledge of reality.

Researchers and statistics experts are concerned that NHS data on income produces “flawed data with harmful implications for public policy”. The president of the Canadian Economics Association has stated he will “stay away” from the NHS data when it comes to assessing trends on income inequality. Other researchers have found that the NHS income data does not align with annual tax-filer data.

A range of health policy experts, urban planners, and various consultants have stated that the data quality is worse than they anticipated, masking poverty and income inequality, and preventing analysts from accurately tracking historical trends. The Director of the Manitoba Centre for Health Policy has called the data flawed, noting that it underrepresents the number of poor children in Manitoba and could lead to “misaligned funding” for a range of programs, including early childhood education programs and immunization campaigns. Her team will instead rely on data from the 2006 census. Those working in public health may have to collect data themselves, ultimately redirecting funding from programmatic activity to information collection.

Despite external critiques and questions raised by StatsCan staff, Wayne Smith, Statscan’s chief statistician, insists that the NHS is a “success” and has suggested that it is a “disservice to Canadians” to criticize the quality of the NHS’ data. Statscan’s former chief statistician, Dr Munir Sheikh, maintains that more attention needs to be paid to the issue of StatsCan’s independence from government interference.

There are indications that the government may be open to changes in future census forms. In contrast to its approach to the 2011 survey, the federal government has asked the National Statistics Council and other groups for advice on preparing the 2016 census.  

Relevant Dates:

  • 1978: StatsCan formally adopts a no-layoff policy.
  • 1986: The Progressive Conservative government tries to abolish the long-form census, but this attempt fails.
  • 28 June 2010: The Harper government replaces the mandatory long-form census with a voluntary National Household Survey. Many experts and professionals in the public and private sectors condemned the change.
  • 21 July 2010: Dr. Munir Sheikh, the head of Statistics Canada, resigns in protest.
  • 2011: First year in which Canadians filled out the voluntary National Household Survey.
  • January 2012: Faced with budget cuts, StatsCan abandons its no-layoff policy.
  • March 2012: StatsCan’s budget is slashed by nearly 12% in the federal budget.
  • June 2012: In order to meet savings targets in the 2012 federal budget, StatsCan reduces 34 programs, including the Survey of Income and Labour Dynamics. StatsCan must cut $33.9 million by 2014-2015.
  • 1 February 2012: Phil Cross, Statistics Canada chief economic analyst resigns, claiming that internal debate in Statistics Canada was being suppressed on the topic of the long-form census.
  • 8 May 2013: The first wave of NHS data is released. It focuses on Aboriginal Peoples and Immigration and Ethnocultural Diversity.
  • 26 June 2013: The second wave of NHS data is released. The reports on Education and the Labour Force.
  • 11 September 2013: After some delays, StatsCan releases the final round of NHS data, focusing on Income and Housing. StatsCan acknowledges the change in data collection methods means that comparisons cannot be made between low income data in the NHS and data from previous census-based estimates.

Role or Position

Statistics Canada (StatsCan) produces statistics that help Canadians better understand their country - its population, resources, economy, society and culture. Under the Statistics Act, StatsCan must “collect, compile, analyse, abstract and publish statistical information relating to the commercial, industrial, financial, social, economic and general activities and conditions of the people of Canada."

The agency provides statistical information and analysis to assist in the development and evaluation of public policies and programs. The data StatsCan gathers is indeed to improve public and private decision-making such that all Canadians can benefit. The agency also promotes sound statistical practices. Over the decades, StatsCan has gained a reputation as one of the most reliable sources of statistical information in the world, thanks in part to its mandatory long-form census.

Implications and Consequences

  • Free Speech and Transparency:  Internal questions from StatsCan staff about the quality and distribution of the responses from the voluntary survey are limited by StatsCan management’s insistence on presenting the voluntary survey as a success. An independent StatsCan is necessary for sound policy decision-making, informed public choice and good government.
  • Free Speech & Transparency: The elimination of the mandatory long form census has led to the loss of policy-relevant, objective, reliable and robust data about the state of Canadian society. This compromises the ability of Canadians to engage in informed democratic debate. Good data collection is also necessary for good governance, policy formulation and the effective disbursal of public funds. Its absence will negatively impact the ability of governments, businesses, police forces, and others to do their job. The loss of quality census data is part of a broader trend of erosion of evidence-based public policy, which includes the stifling of scientific research and access to information.
  • Equality: The erosion of the quality of StatsCan data may bolster the growth of private data collection. Many organizations may no longer rely on the neighborhood or community-level StatsCan data due to its diminished quality and increased cost. Organizations who can afford it, may be forced to redirect portions of their programmatic funding in order to pay for private data collection. However, private data collection cannot replace public collection. The StatsCan data collection is mandated to improve public and private decision-making for the benefit of all Canadians. The same framework of accountability does not bind private data collection firms. Moreover, privately collected data will only be accessible to groups and individuals who can afford to pay for it.
  • Equality: Due to a lack of reliable data, Canada may become unable to meet its international obligations to provide evidence of its progress in realizing of economic and social rights. For example, Canada is a signatory to the United Nations’ Convention on the Elimination of All Forms of Discrimination Against Women and the 1995 Beijing Platform for Action. While these agreements obligate Canada to collect data on women and men’s unpaid work and participation in the informal sector, such questions were eliminated from the new census.
  • Equality: In concert with the loss of the universal long-form census, budget cuts now prevent StatsCan from collecting and analyzing some of the most informative longitudinal information on the labour force, workplace, health and child well being. This lack of data will have a disproportionately negative impact on systemically marginalized groups who rely on government supports and programs.
  • Equality: The non-response bias in the new census is due in part to the fact that marginalized groups are least likely to volunteer to complete a survey. This means that their circumstances will be underrepresented in survey results. As a consequence, Canadians will loose the ability to track progress on the equality concerns that are most relevant to the lives of marginalized peoples, such as access to quality education and housing.
  • Rule of Law: The inexistence or unavailability of reliable public data on the circumstances of the most marginalized members of Canadian society will hamper the ability of concerned Canadians to marshal the evidence necessary to use legal avenues to challenge government policies that harm marginalized groups.

 

Date published: May 2011

Updated: February 2012

Updated: 8 August 2014

 

Sources