Risks to the 2011 Census - A record of the seminar, by the Statistics Commission

How will we know whether the Census has been a success? What will make a good Census in 2011?

Tests for success

Views on what would constitute success were quite varied, but all participants agreed on the importance of reaching a consensus view. The issues highlighted ranged from completeness of enumeration and coverage (ie the range of questions), through to timeliness and value for money.

The need for high levels of accuracy varied between uses and users. Where the Census was being used to monitor targets, or to bid for resources, accuracy was crucial. It was noted that Census statistics drive policy decisions in many areas (social care, health, education) but that the value of Census statistics in such contexts is higher in the few years immediately after the Census and then reduces progressively.  

The role of local authorities

It was suggested that local authorities ought to be more involved in the Census planning, since they have organisational capacity, information on household addresses, an interest in the range of questions, and can engage local residents in its value.  Residents often want to engage more with local authorities and local authorities have an interest in promoting debate on how services can be improved, and in providing the public with better information to help them understand local authorities’ financial constraints. However, the suggestion that local authorities should be the prime census data collectors was challenged, although some thought that they could contribute to the enumeration process.

It was noted that realistic expectations have to be encouraged and that Census offices should focus on the single specific thing only the census can do - which is to provide consistent local area data. There was a good case for still greater focus on the hard-to-count areas than in the past.

Communicating the value of the Census

A good Census was held to need broad community support, but Census offices should seek to avoid people having too high expectations of the Census, and explain to citizens the extent to which the Census has a role in confirming what may already be known by government (eg basic household information) and what information is exploratory (eg socio-economic questions). 

It was widely hoped that the creation of the new Statistics Board might foster greater public confidence in official statistics and in the purposes behind the Census. However, fears over confidentiality and increasing public apathy towards official forms etc meant that Census offices will need to find a balance between enforcement and appealing to goodwill or community spirit.

Societal changes

Participants highlighted some of the societal and political changes that will make 2011 more difficult than previous censuses, including:

• The increasing complexity of households
• The fluidity in population
• Internal and external migration
• Increasing diversity
• Difficulty in finding people at home
• Apathy (as opposed to reluctance); people receive too much junk mail and regard the Census in the same light
• A lack of incentive to complete the Census form
• A changing political climate (dependent on election outcomes, new leaders, civil disobedience, identity cards, unfortunate media coverage etc)
• A common fear around joined-up data: people may perceive the Census as a state intrusion into their lives

Population concepts

Participants agreed that a good census needed a clear process of evaluation to determine how it performed. This should involve effective checking against other information sources to measure consistency of the results and the schedule should allow time for this evaluation. The case for the value of Demographic Analysis techniques to assessing the reliability and consistency of the Census results was made. However, there was less agreement on the specific data sources that should be used for the evaluation, and the meeting noted that the migration element is the hardest to count. 

It was noted that the quality of responses to questions in 2001 was not always good – only the questions that enumerators were required to check were well completed, and so more checks should to be done at the fieldwork stage.

Is there sufficiently wide recognition across government of the need for ownership of risks to be widely shared ahead of 2011?

Owning the risks

Ownership of risks was generally perceived to rest with Census offices but it was acknowledged that those offices must work in partnership with other organisations, including local authorities (address register, design of questionnaires etc), central government departments (as users of the Census), with international colleagues (for consultation and benchmarking) and with specialist users via the consultations. It was suggested that the commercial world might be able to provide valuable information for the address register. Census offices now have a closer relationship with local authorities than was the position in 2001, but were warned against treating all local authorities as equally engaged, since some put more resources into Census work than others and the quality of the information they can provide is extremely variable. Despite this, local information from local authorities has significant potential to help to ‘triangulate’ Census estimates and can inform analysis of risks and local circumstances; it was felt that all local authorities should be consulted and involved as much as possible.

Political engagement

Some participants were concerned that the future Statistics Board may have less influence with ministers than the current ONS, while others thought it an advantage that it will deal directly with Parliaments and offer the opportunity to reduce risks elsewhere.

It was noted that for the 2001 Census, ONS had active, well-managed engagement with politicians (elections, foot and mouth epidemic, riots – relations with Home Office and others was vital), demonstrating that the support of politicians was both important and achievable.

Consultation

On consultation, it was felt that there were some problematic areas to be resolved, including mitigating the risks around address lists; and consulting the private sector. It was suggested that information about the costs for each Census question could be published and this would help in consultation and engagement because users would be able to understand the trade-offs between different options. In addition, setting up Census Expert Groups now would facilitate consultation.

Have the Census offices taken effective approaches to deciding the questions in the Census against an ever-changing environment?

Census Questions

Comments related to the number of questions (the fewer the better from the point of view of good population estimates), whether new questions should be introduced to keep the Census ‘alive’ and innovative, and the inevitable tension over continuity and the need to strike a balance between innovation and continuity. A suggested approach was to have core questions which were constant between Censuses while the second part of the form might have new or changed questions. It was pointed out that statistical surveys can be split or joined to model against the Census at the moment. The earlier suggestion of a menu of question options was re-iterated. It was suggested that if we knew more about how people used the data then this might allow for more tailoring in the content of Census forms.

The wording of some Census questions varied between administrations but core questions were identical. For example, the wording and options of the ethnicity question vary to accommodate different perceptions of the word “black” (in Scotland there is public resistance to the term while in England it seems to be the preferred one). It was agreed that we need to ensure the results are comparable, and allow for the fact that questions are understood differently in different parts of the country. The agreement between the Census offices aims to standardise the output from varying inputs.

Local area focus

The Census offices face different challenges: the smaller scale in Scotland, for example, resulted in less difficulty in co-ordination or enumeration. It was suggested that ONS might tend to be a little isolated from large pools of users, and have limited understanding of local needs, since the focus within central government is mainly on large-scale policy-making.  Since some areas of the country are subject to more rapid population change than others, it was felt that a balance of emphasis on national and local data requirements was required.

Have the Census Offices carried out horizon scanning and engaged political and special interest groups in order to pre-empt disruption?

Advertising the census

In 2001, a creative approach was successful in persuading people of the value of the Census, by for example, including an enumerator in “East Enders”, or collaborating with The Sun newspaper’s ‘Sunsus’ prior to enumeration day. It was felt that in the current climate of “migration paranoia”, newspapers might be willing to promote the need to have a census, and people might recognise the need to measure migration movements. Conversely, there were public concerns about a surveillance society as well as people’s fears that their answers will be linked to administrative processes such as tax collection.

Horizon-scanning

Reference was made to some of the societal and political changes described earlier, and an additional suggestion was made for more emphasis on the address databases by pre-enumerating dwelling spaces rather than addresses. This would need close working with local authorities to eliminate uninhabited, or “ghost”, properties from the address lists.

ONS relocation

The relocation of ONS staff was considered to be a risk to the statistical system as a whole and there was concern that the focus on the Census could be affected by it. 

Secretariat, Statistics Commission

February 2007