Official Statistics: Perceptions and Trust, incorporating: Trust in official statistics: MORI report on behalf of the Statistics Commission

Chairman's foreword

It is self-evident that the public needs to be able to trust official statistics as being sufficiently accurate and not perverted for political ends. Without this trust, neither the public nor its elected representatives can hold government to account. Democracy is dependent on it.

The Statistics Commission believes that it is time to start to benchmark the current level of trust, with a view to reassessing it over the coming years. We understand that assessing and interpreting the level of trust is problematic. It seems likely to be affected by factors other than the statistics themselves or how they are created and published. The well-known decline in deference towards the organs of the state over the last half century may well have induced greater scepticism. The media, and in particular the print media, are sometimes scathing about what they see as failures of measurement or quality assurance and this must colour public trust to some extent.

This study complements a survey carried out by the Office for National Statistics (ONS). It seeks to determine what a group of key opinion-formers actually believe is the situation in regard to public trust in official statistics. In so far as there is perceived to be a problem, there is a problem. This research is primarily about understanding the nature of the problem and, as far as we can, its roots. The findings of the interviews undertaken on our behalf by MORI buttress the need for still greater effort in planning the collection of statistics across all of government and communication of the results, to give commentators, intermediaries and the public alike the confidence that UK official statistics are fit for purpose and can be trusted.

I would like to thank all those who contributed to this report. It was produced under the guidance of a Statistics Commission Project Board chaired by myself and I am grateful for the support of colleagues on the board. I should also like to thank Andrew Johnson, Suzanne Hall and Nick Pettigrew of MORI who undertook the interviews with opinion-formers and reflected their views in such a coherent fashion in their report. I am particularly grateful to all the interviewees who spared so much of their time and gave such thoughtful consideration to the issues.

Chairman

1. Introduction

The research exercise

This report considers the findings of interviews with 36 leading individuals from academia, the media, Parliament and government, public services, the business community and the voluntary sector. The interviews were undertaken on our behalf by MORI, whose own report is appended.

This research forms part of a wider programme designed to obtain a better understanding of issues of trust in official statistics (see below). The importance of these interviews is that they present for the first time a good sample of the views of 'opinion-formers' - those people who, through their various roles and public comments, influence a much wider body of opinion. We sought these views as the best way to secure a rounded picture of the issues that affect trust. We hoped also to identify measures that our interviewees believe would give them more confidence in the statistical outputs of government.

A full list of those interviewed is given in the MORI report. This group of people is not a representative sample, in the statistical sense, and the results should not therefore be interpreted as reflecting accurately those of any larger group. Nonetheless, the interviewees include many prominent individuals whose collected views are important in their own right.

The Commission hopes the wider programme of research will serve as a benchmark against which we can assess future levels of trust.

Main messages

The main messages that the Commission takes from the interviews are:

• On the whole, the interviewees believed that the quality of UK official statistics is up with the best in the world. But there is a recognition that scope still exists for improvement.
• There was a widely-held view that there needed to be greater distance between the producers of statistics and government, possibly with an independent regulatory body which would monitor the use of official statistics.
• Growing emphasis on performance indicators and targets has meant that official statistics are perceived as sometimes being pushed too far, beyond what they are capable of measuring. This is an issue that the Commission will be following up in the coming months.
• There is a need for a greater range of comparable statistics for the four countries of the UK. This has frequently been flagged up in the Commission's discussions with users and we are addressing the scale of the requirement through other work.
• Government statisticians could improve their communication with users, for example being clearer about the reasons for revisions and providing a fuller explanation of the figures. The media could also play a more constructive part in interpreting data.

Context for the research

From the perspective of the Statistics Commission, the primary context for this work is our report on the need for statistical legislation published in May 2004: Legislation to Build Trust in Statistics. That report drew attention to the importance of trust, arguing that public confidence in official statistics is essential for the effective working of the state and the effective delivery of public services. We argued:

"If decision-makers believe the evidence base is inadequate, they will tend to ignore it. …. Failure to make full use of available information must inevitably weaken decision-making. And whilst no precise economic cost can be placed on failure to have or use relevant information, the costs, when aggregated across the whole economy, are likely to be very high. More broadly, if the general public do not trust the statistics that underpin and explain policy initiatives, and that measure the success of those policies, then government and public bodies themselves will not be trusted."

Whilst the importance of public trust in official figures is widely accepted, there has been lively debate about the extent and nature of any problems in this regard. Some have argued that the issue of trust in statistics is intimately related to problems of trust in public life more generally and cannot, or should not, be addressed separately; others believe that it is an endemic problem afflicting official statistics in many countries and must therefore simply be tolerated.

The Statistics Commission believes that the negative effects of lack of trust are too serious to be brushed to one side by such arguments. Whatever can be done to enable the expert user and general public to have greater faith in official figures should be done. On this basis, it is all the more critical to understand the nature of the problem and to identify what the wider community would regard as beneficial measures to ameliorate it.

The public confidence project

In the course of 2004, considerations such as these led to the instigation of an ongoing programme of research work involving both the Office for National Statistics and the Statistics Commission. This report forms part of that programme. ONS is concentrating initially on assessing the views of the general public, partially funded by the Commission, whilst we are addressing the perceptions of those individuals who help the public develop these views.

The strands being managed by the ONS are:

• a quantified public survey of trust in official statistics as part of the ONS Omnibus Survey
• focus groups with the general public
• an assessment of comparable research in other countries.

An initial report on the Omnibus Survey findings was published in October 2004 (see Omnibus Survey: Initial findings on public confidence in official statistics. ONS, October 2004. www.statistics.gov.uk/about/data/public_confidence.asp). Among the key findings were:

"the majority of respondents believed official figures were changed to support a particular argument (68%), that there was political interference in their production (58%) and that mistakes were suppressed (69%)."

Also, the majority of respondents (59%) "did not agree that the Government uses official figures honestly when talking about its policies".

The Omnibus Survey results above answer one question and raise some others. They answer the question of whether public trust is low: it is. But they prompt questions about the underlying form of the problem and whether proposals already made by the Statistics Commission, and others, might reasonably be expected to have a beneficial impact.

It is evident in this study, as in much of our work over several years, that external commentators are not always fully aware of the realities of statistical work in government. We identify here some points that warrant particular emphasis.

Statistics are not 'facts'

The aim of statistical work is generally to produce usable information from incomplete data. Sampling, imputation and estimation are the essence of statistics, not totting up scores as in a cricket match. So official statistics are mostly the best estimates that can be made at the time they are produced, no more or less. To take a special case, National Accounts are not like business accounts. They are not 'added up from the bottom'. They are generated by bringing together, using specialist expertise, various imperfect estimates. The later availability of better information ensures these estimates can often be improved; it is only right that figures are updated as more becomes known. Thus revisions to statistics are a natural part of the process of updating the estimates. Inevitably, estimation processes are more error-prone than simply 'adding up' measurements. What is crucial however - as the interviewees stress - is how matters are handled when things do go wrong, for this has a big influence on trust.

Statistics are often not as timely as users want

With a very few exceptions, the relatively complex process of collecting data, generating estimates from incomplete information and quality assuring the process and results takes significant amounts of time - usually longer than users would wish. This is a fact of life. Whilst some processes can be speeded up (eg through use of better technology), the gestation period for production of official statistics is not usually a sign of professional failing or malevolent influence.

The production of statistics requires the involvement of the main government departments

The substantial and growing importance of information sources that are linked directly to the management of public services, and therefore in the control of the main government departments, means that the totality of the statistical work of government cannot be completely separated off into a distinct organisation. Examples of these sources include: NHS data in the case of the Department of Health; school education data in the case of DfES; police and immigration data in the case of the Home Office etc. The production of statistics involves all the main organs of government and will continue to do so. It needs to be seen and managed in that context. ONS has only modest involvement in the production of many government statistics.

Official statistics are not collected simply to paint a background picture for policy-makers and the public

The reality is that official statistics influence large numbers of important 'business' decisions day-to-day, across government, the public services and in the private and not-for-profit sectors. Whilst informing the public about the state of the country is a vital role and key to the democratic process, it is only part of the rationale for having an effective and trustworthy statistical system. The value of official statistics in relation to practical decision-making across the entire face of the UK is a powerful argument for investment in quality and trust.

Statistics are always likely to be used in a political way by politicians

The value of the MORI report lies in what it tells us about how knowledgeable and influential people perceive official statistics, rather than what it tells us about how good the statistical system actually is. Though these matters are related (for example the production of demonstrably error-prone statistics would undermine public trust), the linkage is not a simple one. The results of the interviews illuminate the multiple factors which seem to be involved.

The Commission's views on the main messages

The following extracts from the MORI report highlight and amplify the main messages summarised in the introduction (Section 1). The quotes from MORI are followed by comments from the Statistics Commission.

On quality

"On the whole, opinion leaders believe that the quality of official statistics in the UK is up with the best in the world. Much of the criticism they attract is seen as unwarranted - a result of the blame culture that exists in today's political climate and the lack of trust in government generally."

This is consistent with the Statistics Commission's view that the statistical work of government is generally done to a high standard, albeit constrained by past under-investment in information systems and technology. But the Commission also has real concerns about whether official statistics are keeping up with the changing needs of users from different sectors, about the frankness and fullness of the commentary that accompanies the figures and about communication with users of statistics and the public. These points are also reflected in some of the other comments made by interviewees.

On legislation and related matters

"…there is a strong feeling that action needs to be taken to increase trust in, and the credibility of, official statistics. For many, the key to achieving this lies with securing independence for the statistical service. … It is widely believed that by increasing the distance between Government and the statistical service, there will be less of an inclination to view its output with suspicion."

"…one of the criticisms of the statistical service is that some perceive that it does not collect the most relevant data. It is thought that by severing its ties to the Treasury, the statistical service would have greater freedom to collect information on a wider, or different, range of issues - the scope of which could be determined by close consultation with key constituents."

"Some feel that for the situation to improve, there is the need for a regulatory body which can not only monitor the use of statistics but can also act as a forum to which people can complain should they believe that figures are being used inappropriately. … Again, many feel that such a body should be independent from Government to ensure that it is perceived as a trusted organisation."

The Statistics Commission notes the view that there should be 'greater distance' between the civil servants responsible for the production of statistics and Ministers. And we note the variety of views on more formal regulatory arrangements. It is important to recognise however in this context that civil servants will always be accountable to their Ministers and that - under the system operated in British government - this arrangement should not be undermined. We do, however, see greater scope for independent regulatory arrangements that would support the officials responsible for statistical work. The recommendations we made in our report Legislation to Build Trust in Statistics suggest one way in which this could be achieved.

On planning for the collection, analysis and dissemination of official statistics

"The present Government, more than any other, is seen to have emphasised performance indicators, targets and evidence-based policies..."

"Some are of the view that for official statistics to be as useful as possible, it is necessary to foster closer co-operation and joint working with international agencies …"

"On a smaller scale, there is a need to ensure that, wherever possible, 'national' statistics means just that. Devolution has clearly impacted on the collection of data with responsibility for certain series being abrogated to individual legislative bodies. Consequently, there are instances when what are referred to as UK data, actually only cover England."

"Furthermore, in a climate of targets, performance indicators, and evidence-based policy, more statistics are used than ever before as drivers of policy and change. Official statistics have assumed a role of vital importance, hence the increasing scrutiny placed on them."

The Statistics Commission believes that the use of official statistics to underpin targets that the Government sets is both inevitable and appropriate in principle. However consideration must be given to whether 'fit for purpose' statistics can be provided in the appropriate timescale and to the possible negative consequences of 'politicisation' of the statistical messages. The latter is something that needs to be managed in part by ensuring that the production of statistics is seen to be free from undue political influence and in part by avoiding stretching the credibility of the statistical data too far. We will be developing our ideas on whether statistics are being pushed too far beyond their intended purpose as part of our work on quality issues, targets and performance indicators. We will also be examining the case for and against greater harmonisation of statistical outputs from the four UK administrations, and similarly in relation to other European countries. There are benefits both from consistent statistics and from statistics that are tailored to local circumstances. Finding the right balance must start from understanding the needs of all users of statistics.

On communication of statistics and their messages

"Improved communications from the statistical service are seen to be key if trust is to increase. The lack of explanation with regard to why data has to be revised, creates doubt and suspicion in the minds of some. … Overall, the need for improved and increasingly open communications on the part of the statistical service is seen to be vital if it is to rebuild trust."

"Some also feel that the media should take a far more proactive role when it comes to understanding and interpreting data. … It is important that those within the media who do operate from a trusted platform are able to interpret statistical information correctly so as to ensure that what they communicate is not only believed but is also accurate."

"The GSS should speak more about its successes to raise confidence. More positive communications would negate some of the damaging effects of blame in the media when there is an error."

"The quality marker 'National Statistics' is already applied to certain UK statistics and for these to be badged as such then they must meet certain criteria … However, awareness of this among respondents is very low."

The Statistics Commission strongly supports the thrust of these comments. Better communication of the messages contained in official statistics - as we have pointed out on many previous occasions - is likely to be one of the best ways to engender greater public trust. By 'better' we mean widespread and routine dissemination of information which is written in a way understandable to the lay public, with key messages highlighted, the limits to reliability of the evidence made clear, plus some interpretation of the results - and all this done in a timely fashion.

Conclusion

Important as they are, we do not seek to base our conclusions on these interviews alone. But, relating the evidence of the interviews to the surveys of public trust carried out by ONS and to our own research and consultation, we believe that this body of work strongly supports our call for action to address the concerns identified. We can summarise these conclusions as follows:

• Opinion-formers expect ever more of official statistics but should not let their expectations run too far ahead of what is realistically attainable.
  
  
• The Office for National Statistics seems in most respects to be up with the best of the statistics offices in the world on a technical basis.
  
  
• ONS must show an appropriate degree of independence from Ministers and be allowed to do so. That does not however imply it has an unchallengeable status; checks and balances are essential and public debate is a healthy component of the system.
  
  
• Trust depends in no small part on ONS being seen to manage the planning and collection of official statistics to meet user needs. It must also be seen to manage effectively the relationships between timeliness, reliability and predictability. This implies good communication of planned publication dates, the status of estimates, the likely timing of revisions and reasons for them. It also implies acknowledging problems quickly and acting to put things right.
  
  
• Trust can be damaged by any failure to engage openly in public debate and by under-stating successes. Whilst celebrating success should ideally be done by 'outsiders', government statisticians need to blow their own trumpet too.
  
  
• Trust requires being seen to be open about the challenges of measuring a fast-changing world.
  
  
• Trust in government statisticians depends not just on their technical competence but also on their understanding of the likely effects of their actions on the outside world. They will be judged partly on their ability to manage stakeholder relationships and the quality of their communications with them.
  
  
• Trust is vulnerable to confusion over the provenance of government data. The expert auditing of official statistics, eg by parties elsewhere within government other than by those who produce them, seems a policy which would help to foster trust.
  
  
• Trust in government statistics is often seen as equivalent to trust in ONS and its products. This is unhelpful: the statistical products of other departments are just as important and subject to controversy. Fostering trust requires both public understanding of which department of government is responsible for which statistical services and better mechanisms for cross-department control and management.
  
  
• The legislative proposals which the Statistics Commission has already put forward sit comfortably with the main points from the interviews.

Trust in Official Statistics:
Report to Statistics Commission By MORI

Part 2

Trust in Official Statistics
Research Study Conducted for the Statistics Commission


SUZANNE HALL
ANDREW JOHNSON
NICK PETTIGREW


MORI House
79-81 Borough Road
London
SE1 1FY
Tel: 020 7347 3000
Fax: 020-7347 3803
February 2005

Acknowledgements

MORI would like to thank the following for giving up their time so readily and offering their expertise:

We would also like to thank Richard Alldritt and Barbara Buckley Owen from the Statistics Commission, along with the Commissioners, for their support and advice throughout.

Thanks also to Mrs Armelle Thomas of MORI for arranging all the interviews so effectively.

Trust in Official Statistics:
Report to Statistics Commission By MORI

Part 2

Executive Summary

Encouragingly, there is broad consensus among all opinion formers as to the drivers of trust in official statistics and, furthermore, the measures that could be implemented to improve the situation. The issues outlined in the executive summary are those on which there was general agreement.

Official statistics: An overview

• The whole issue of trust in official statistics needs to be placed in context. On the whole, opinion leaders believe that the quality of official statistics in the UK is among best in the world. Much of the criticism they attract is seen as unwarranted - a result of the blame culture that exists in today's political climate and the lack of trust in government generally.
  
  
• Many feel that there needs to be an increased recognition that, sometimes, mistakes will happen; and that it is how these are dealt with which should be the focus, rather than the actual errors themselves.
  
  
• While stakeholders have clearly defined user needs and well-formed opinions on the state of the statistical service, not all differentiate between the different agencies that make up the Government Statistical Service. It is therefore important to bear in mind that some references to the Office for National Statistics may not actually be referring to this specific agency, but to statistical services more generally. Furthermore, there is no universal definition to as to what constitutes an 'official' statistic.

Drivers of trust

• Politicisation: The emphasis of performance indicators, targets and evidence based policies has increased the amount of data available and has placed statistics - which stakeholders point out are often not facts but rather a best estimate - in the position where they are required to do more than they are capable of.
  
  
• Coverage: There is a sense among some of the key constituents that the data collected are not as useful as they could be, for example, key developments such as the shadow economy are not afforded as much importance as some feel they should.
  
  
• Time lags: While the complexities of data collection and assimilation is understood, delays can mean the information is out-of-date and - just as seriously - raise the suspicion of political interference.
  
  
• Presentation: Some feel that Ministerial prior access can lead to 'polished' and selective presentation of data, or at least the appearance of it. Set against a political backdrop of potential or perceived 'spin', this undermines the credibility of official statistics.
  
  
• Revisions: While some understand the reasoning behind revisions made to data, many do not feel that this is communicated clearly enough. This lack of explanation creates confusion and suspicion.
  
  
• Errors in production: While the statistical service is not expected to be infallible, there are questions as to whether it handles criticism appropriately. Some users feel they are too slow to respond to complaints and, furthermore, still too defensive which serves to corrode trust.

Stakeholder suggestions for improving trust

• Independence: For many, the key to improving trust in official statistics lies in securing independence for the Government Statistical Service - an organisational structure similar to that of the National Audit Office with direct reporting to Parliament rather than the Treasury is seen to be the ideal.
  
  
• It is felt that this would allow the statistical service greater autonomy to collect relevant and useful statistics and that it would be seen as being free from pressure from Ministers.
  
  
• Some feel that the distance that already exists between the Government and its statistical services merely needs to be emphasised for there to be greater trust. For example, some suggest changing the website address from '.gov.uk' to '.org.uk.'
  
  
• Regulation: The establishment of a regulatory body to control the quality of official statistics, issue guidelines on how they should be used, and to also act as a body which users and the public can raise queries and complaints with is seen as a step forward. Again, most urge direct reporting to Parliament.
  
  
• A 'gold standard': Kite-marking official statistics is seen as one means of helping to differentiate between what is reputable and what is not. It is also thought that this would create something of a 'gold-standard' which would drive up the quality of other statistics. Ideally, some feel that kite-marks should also indicate degrees of provisionality on the statistics to ensure that users are clear as to whether the data are likely to be revised in the future.
  
  
• Reduce prior access: There are calls for Ministers' rights to prior access to data to be abolished or, at the least, reduced to around half an hour, as in some European countries to help emphasise the transparency of official statistics. Some also feel this could improve the quality of democratic debate, as political parties and other interested bodies would be able to analyse the data at the same time.
  
  
• Increase transparency: Some state the need for improved communications on the part of the statistical service. It is felt that it should clearly label estimated data and indicate a likely timescale as to when revised figures will be released. If this timetable slips, then an explanation should be provided.
  
  
• Linked to this is the issue of expected standards of behaviour when errors are made. On contentious issues, such as the Westminster Council census data, many are of the opinion that the statistical service should take further steps to be less defensive, more open in its response to criticism and proactive in answering queries.
  
  
• Increase explanation: A common view is that it is not the data per se that are typically questionable, but rather the interpretation. Hence, some feel that the statistical service should take a more proactive role in providing clear explanations about the statistics. Simplicity is seen to be key here, and many feel that graphical depictions of official figures would help achieve this and that, furthermore, there would be less scope for the media to twist the data into attention-grabbing but misleading headlines.
  
  
• Increase comparability: There is a call for closer co-operation with statistical agencies worldwide - not because these are seen as models to which we should aspire, but because it is perceived that greater commonality between the data series across different countries would increase the impact of the figures as key constituents could place the figures in context. Some also state the need for greater comparative data for the four home UK nations to be produced.
  
  
• Enhance public education: The need for improved public numeracy, while a long-term measure, is felt to be crucial if the public is to fully comprehend the importance and integrity of official statistics. Linked to this is the need for improved critical evaluation, perhaps to be taught as part of the citizenship agenda on the national curriculum.
  
  
• In order to engage the public with official statistics some suggest producing a television show presented by a charismatic and respected broadcaster, designed to give an overview of the week's statistical releases and how this impacts on society.

Credibility and trust in official statistics

It is important to make clear from the outset that the majority of opinion formers interviewed for this project do believe that the outputs from the Government Statistical Service are reliable and are produced without undue political influence. However, many external factors are seen to impact on the credibility of the outputs of the statistical service.

Methodology and data collection

The overriding sense is that the methodological approaches employed by the Government Statistical Service are robust and appropriate for the kind of uses that the data are put to. Many users of official statistics do not feel the need to check the sampling frame or other technical details prior to using the data - the reputation of the statistical service, and in particular the ONS, is such that they believe that this is simply not necessary.

"I work on the assumption that if the survey's done by the ONS and the such like, I don't need to look at the technical annexe to reassure myself about the sampling and the methodology"

Some mentioned methodological concerns with regard to the 2001 Census, but many are willing to adopt a pragmatic view; in spite of the controversy caused in the London Borough of Westminster for example, they take encouragement from seeing the Government Statistical Service willing to take risks, adapt and innovate.

"ONS took a lot of unfair stick for the Census. What they tried to do was use a new methodology and, to use an old phrase, they probably ended up being too clever by half…but that, to me, indicated the intellectual competence of ONS, that it is actually willing to learn - that's a very good thing"

Of course, there are those who view all data, regardless of source, with a degree of suspicion and the issues outlined above have only served to confirm this view.

"I'm sceptical of all data and I think that's the appropriate standpoint"

These comments tend to be made by the kind of audiences who are rooted in issues of validity, sampling and critical questioning - for example, academics. This is not surprising and should not be cause for too much concern; it is a reminder of their independent scrutiny.
In spite of the fact that the majority of respondents are confident that the approach taken by the Government Statistical Service is correct and proper, some are critical of the types of data that are collected or, indeed, omitted.

A case in point is the increasing focus on public sector productivity. Some are sceptical as to why this is collected in the first place, believing that public sector output is inherently unquantifiable and that the figures will be rendered useless either through developments in policy, or through greater levels of public education. For example, effective safety information from the fire service may reduce the number of individual cases dealt with, thus resulting in seemingly diminished outputs.

"People demand that there should be a measure of productivity in the public sector, what does that mean? If you look at education, the measure presumably has to be output per teacher per pupil for information provided. Now if you halve class sizes…so you have better quality communication between teachers and pupils…you're halving productivity. It's just nonsense"

"How do you measure the output of a hospital or a fire service…how do we measure preventative activity?"

Linked to this, some bemoan the absence of data that they feel should be collected but currently are not. An example here relates to education. The issue is complicated by the fact that Higher Education statistics are collected by a separate agency and some are surprised, and indeed confused, that these data do not fall under the auspices of the Government Statistical Service.

"The sector is fiercely resistant to being bean-counted. They never collect any useful information - even basic information such as how many people have 1sts, 2:1s, 2:2s"

While there is recognition that this situation has changed in recent years, the data available on degree results only go back for a decade. Prior to this, users were reliant on academics who did private studies into this issue.

Others note that as society and the economy change, the kind of data that we need to function efficiently and intelligently also change. For example, the decline in manufacturing industry has been offset by strong growth within the service sector. However, this shift is not reflected in the data collected.

"As the nature of the economy has changed, then the type of statistics that we want is changing. It's much more difficult to measure service sector data but I also feel the ONS doesn't measure it with the same degree of breadth as it does the traditional manufacturing measures"

Some also point to the shadow economy and make the point that this is an important area for examination (as it impacts hugely on issues such as GDP), yet there is currently little recognition that this is something that warrants priority attention.

"There is a general issue, even in the developed world, now about the reliability of data, particularly economic data, because of the growth of the shadow economy which is much larger than government policy is willing to admit in most OECD countries"

As such, many do feel that if the reputation of the Government Statistical Service is to improve, then it must do more to be proactive in the sphere of data collection and rise to the challenge of monitoring an increasingly complex world.

Time lags

Timeliness of statistical releases can impact on the credibility of the data in a number of ways. The lag between fieldwork and release, especially with regard to departmental data, is sometimes so long that by the time the figures are made public, the situation has changed to such an extent that they are felt to be no longer relevant. While this is not only frustrating in that the data are meaningless, it is also seen to demotivate those close to the issue at stake.

"The problem about information that's produced by every Government department is that it's two years out of date, so when you've just given your local hospital an award for being the cleanest place in East London and then two months later, the statistics come out which show it's the dirtiest place in East London because, actually, that was two years ago…that can be very demoralising for local managers"

Others are concerned that delays in release dates raise the suspicion of undue political pressure being applied to the statistical service and that figures are only issued when it is convenient for the Government to do so.

"Timing is crucial, so if you see the annual poverty figures delayed until after a major policy announcement or elections, that undermines it"

Many feel that, currently, there is a lack of effective communication on the part of the Government Statistical Service about when datasets are to be released. This impacts on its reputation and reflects the image that it is an agent of Government and at its bidding. More clarity and openness about the publication of figures is seen as necessary in order to combat this.

"In an ideal system, ONS will announce in advance when they're going to release certain time series and they will never be deflected from meeting those dates which requires both efficiency and integrity"

Many do understand why this situation has arisen and recognise that caution about what to release and when, is simply a natural response to the often unwarranted criticism of their outputs.

"There's a danger within ONS of becoming too defensive. Where I've seen this is in the whole disclosure business and the access of people outside ONS to potentially sensitive information…it's a culture of playing safe"

While many do recognise that it will take time to produce complicated and comprehensive datasets, some feel that better and clearer information should be provided at each stage of production to ensure that key constituents are kept on board and do not suspect that anything untoward is occurring.

Revisions

Many feel their trust in official statistics is diminished by revisions that are made to data series after their release. Some, especially low-level users of official statistics, do not appear to understand that revisions having to be made at a later date are the price paid for speedy release. It is their perception that the amendments made are an admission of error; not only does this reduce their confidence in that particular release, it casts a shadow over other outputs as well.

Many recognise the pressures that the Government Statistical Service is under to put figures into the public domain quickly, and understand that there will always be a battle between timeliness and reliability in that data that are issued quickly may be subject to fewer quality checks.

"There is a big trade-off between timeliness and accuracy and I think that's the struggle and I sympathise with the ONS that they're always fighting against this"

Some feel that this is simply a reflection of the trend towards short-termism on the political agenda. They recognise the negative impact that revisions can cause, particularly when they are picked up by the media and blown out of proportion. However, they still feel that the benefits gained from access to timely data offset the disadvantages that this may cause.

"There has to be a trade-off in users' minds as to whether there is value in having the earlier estimates. My view on things like GDP is that there is sufficient value in having a preliminary estimate of that, but occasional revisions to it are the price you pay for that. But it's clearly the sort of thing that the press occasionally get quite excited about"

However, even those who recognise the worth in the first issue of data series suggest that too much emphasis is placed on these kind of figures, which are expected to do more than they are capable of. It is this level of expectation that is placed on potentially unreliable data that can cause problems.

"People who haven't got close to statistics expect a degree of precision which is impossible"

Furthermore, while many do understand the need for revisions, they feel that the Government Statistical Service is often reticent to come forward with detailed explanations as to why these changes are necessary.

"You look at the data and there's been a revision but there's very seldom an explanation as to why"

Opinion leaders point out that this lack of transparency serves to undermine confidence, and that official statistics would be seen to be far more credible if the statistical service were to make clear, at every step, its intentions and thinking.

Errors in production

Closely related is the issue of errors which result in inaccurate data being released. Recent high profile incidents, such as the Westminster Council population figures in the 2001 Census, have brought this matter to the forefront.

"One of the biggest issues that the ONS has is, that over several well publicised cases, there have been mistakes…it is very important that there is a confidence level"

While no one is saying that the Government Statistical Service can be infallible, some do feel that these errors diminish confidence in their ability to produce accurate and meaningful data. What is more, it is not simply the fact that mistakes are made - some see this as inevitable, as it would be with any organisation or business - but rather the response to the errors, that is the problem here.

Many cite the refuted data from the 2001 Census as a case in point. Some feel that ONS was just too slow to react to outside criticism for fear of losing face.

"They have in the errors been amazingly slow at correcting them…because they didn't want to admit that they'd made a mistake"

Furthermore, some perceive that the statistical service believe themselves to be immune to criticism and are too quick to defend their approach, without presenting the necessary evidence to justify their stance.

"The original response was 'there isn't a problem, we've looked at that and we know about it'. It wasn't 'we'll have to have a good look at that and see if we can understand', which would give more trust and confidence"

It is clearly felt by some that the statistical service would benefit from showing more humility when errors are made. This increased level of openness in discussion would go some way to rectifying the scepticism that can surround their outputs.

However, it is not just errors by the Government Statistical Service that can impact on trust but mistakes made with statistics generally. A few make the point that there are now so many statistics in the public domain that when high-profile mistakes are made in data published by the private sector, then this can impinge on the credibility of all statistics.

Coverage

For some, the issue of coverage helps to explain credibility doubts about official statistics. They point out that particularly within the field of education, more often than not UK statistics simply refer to England rather than covering all four home nations. They recognise that this may simply be a result of devolution, but the lack of comparative data can cause problems and reduce the worth and relevance of the figures at hand. In some instances they do understand that there is actually no comparative data to hand: for instance, the different examination system in Scotland to England. However, where there are comparable situations some feel that the users of data would be best served by having access to national statistics rather than ones simply relating to England.

"What does the 'national' in ONS mean?"

Linked to this are issues surrounding the relocation of ONS to Newport. Some do feel that this could, if managed properly, be a beneficial shift for the statistical service in that it will put physical distance between the producers of data and Westminster, thus helping to foster an image of independence and freedom from ministerial influence.

"Slowly they will shed their Civil Service roots"

However, others are concerned about what this move will mean for the future quality of the statistical service. Particularly with regard to recruitment, some express doubts whether the service will be able to take its best statisticians with it. ONS may face problems encouraging people with a similar level of experience and expertise to work outside the capital.

"I think there will be a danger that they won't get people wanting to come in at senior level if they've got to live in Newport"

"They're increasing risk…you're going to recruit a whole load of new people in Newport and lose your London people. It's bound to be a very wobbly period"

Although this is only something to be aware of in the future, the fact that it is raised as a point of concern will mean that, if the risks are not identified and managed appropriately, this too could become another factor which hampers the credibility of official statistics.

Perceived Government influence

There is little consensus on whether the statistical service is subject to undue influence from the Government. Many, in particular those with a connection to Parliament, are confident that the statistical service operates independently and with integrity.

"I've never seen any indication that the ONS have been pressured by Government to produce a rogue result or not. I think in terms of ethical integrity, the ONS is undoubted"

However, for those outside the 'Westminster sphere of influence', there is far more scepticism as to whether this is actually the case. While most still regard the ONS as an authoritative, trusted source of information, there are doubts over the validity of figures produced by individual Government departments. This is largely due to the fact that the statisticians within departments - many of whom are perceived to be highly skilled and experienced technically - are, ultimately, answerable to Ministerial authority, and may face a conflict between meeting the needs of their masters on the one hand, and statistical truth on the other.

"Effectively, the statisticians within Government departments are subject to their political masters, they have to answer to that department and, I think, that creates all kinds of problems of conflicts of interest…they will often simply be reflecting political priorities"

A minority also express doubts about the ONS's freedom from political influence. Some feel it acts at the bidding of the Government and, consequently, is not immune to charges that its data may be suspect at times.

"The real problem ONS has is the perception that it is just an arm of Government and, thus, it is a propaganda arm of Government"

"The ONS unfortunately bows to pressure from Ministers so it actually becomes a useless, in the true sense of the word, organisation because once people start manipulating or changing their figures, then they are valueless"

Most of this is simply a suspicion, a feeling that since ONS is ultimately answerable to the Treasury, then it must be placed under pressure from time to time to selectively release data in a way that optimises the Government's position.

Indeed, a small number go so far as to say that they do not believe that the statistical service is immune from making changes to the data when it suits them.

"Let's face it, they negotiate on the information before it comes out"

Politicisation

Due to the political climate which favours targets, performance indicators and 'evidence-based policy', there is increased pressure on statistics to fulfil more roles, but also to be a favourable reflection that Government strategies are working. Due to the increased politicisation of statistics, some are concerned that this can lead to distortion and alteration.
"Some of the statistics that come out as targets really act as major incentives to crookedness"

This has particularly impacted on the statistics released by individual Government departments, which are seen to be particularly susceptible to this kind of pressure. Some feel that the need to be seen to be meeting targets has led to certain departments covering up their outputs, through delays in release for example, and this has resulted in diminished credibility.

"One of the things that concerns me about statistics produced by central Government departments is sometimes you get the feeling that these things are kicked into the long grass...so that nobody notices them. It just doesn't feel to me that it's as open and transparent as it could be"

It is this perception, that official statistics are a tool of Government to be used to justify policy developments and demonstrate progress, that can seriously undermine their credibility and trustworthiness.

Presentation

In spite of concerns over the politicisation of data, most do feel that official statistics are produced free from political influence and pressure. However, the problem for many is not with the data itself but rather what it is done with it on release.

"The trust in the data is not the problem, the problem is in the users of the data"

"It's not lies, damned lies and statistics, it's lies, damned lies and people who use statistics"

Prior Ministerial access is seen by many, most commonly those outside Parliament, as being a perceived invitation for official statistics to be manipulated so that they tell the 'correct' story.

"Personally I wouldn't give Ministers advance notice of any of the data…I'm not sure that the arguments for them to have it, in terms of making a response, overpower the argument that you just basically want to keep this as independent as possible"

"There is a problem that once statistics are handed over to Government departments, they are moulded and massaged to tell a particular story"

On the other hand, those within Westminster make the case that prior access is a necessary privilege for those in Government to ensure that Ministers have an appropriate response pre-prepared.

"It is a Minister's duty to be able to respond at the same time, so it becomes a single piece of news and therefore they need to have enough notice of it in order to prepare a response"

However, there are some who state that not only does prior access serve to undermine the credibility of official statistics due to accusations of spin and moulding but, furthermore, early sight of data also reduces the quality of public debate.

"Because the Government knows stuff quicker than other people, it's calling the shots in terms of the statistical debate"

While many recognise that prior access has been an incumbent's advantage for many years, the political climate in which statistics are used is thought to have changed to such an extent that this is a problem, where once it was not.

"It's a concern that goes back further than just the New Labour period. It goes right back to the Thatcher Government I think. What I think is new now is that it's placed in the context of a political culture dominated by spin"

What is more, some feel that there is a self-perpetuating cycle of decline in that once it is felt that some figures have been misinterpreted, then there will be a tendency to believe that all may have been tampered with.

"If people lose credibility in statistics because some are spun, they will start to believe that all are spun"

Definitions

It is not just prior access that leads to problems with presentation and, ultimately, diminished credibility of official statistics. There is also the issue of definitions. Many make the point that during the Thatcher administration, the definition for unemployment was changed 17 times - 16 of which lowered the rate. It is felt that this set something of a precedent for Governments to move the goalposts as and when required in order to ensure that the data they receive suit their needs. As one sceptic put it:

"When the Government misses a target, they just change the definition"

This creates something of a state of flux; users are unsure as to what is being measured, and are confused as to whether the data they receive are comparable with what went before.

"This Government particularly has moved the goalposts for the information they collect so often the criteria is different so people see statistics and think, well that's not what I saw last week"

"There's nothing worse than finding the base has been changed, so you just lose all track of comparability"

Media distortion

However, it is not just the politicians who are seen to be culpable for misrepresenting data. Inaccurate portrayal in the media is held, by some, as being responsible for a decline in trust. While many recognise that statistics are a convenient means of providing powerful headlines, this can serve to skew public understanding by taking a single figure out of context.
"They exaggerate in order to embarrass Governments, they look for headlines rather than reporting things straight. The main bias of the media is in not reporting the ordinary…if nothing has changed with inflation there will be no news on it and actually that's important"

Furthermore, there is a belief that many journalists, particularly those in print-based media, have a political bias and hence will pick and choose official statistics to support their viewpoint.
Additionally, some accuse the media of being too quick to criticise the statistical service in their reporting which only serves to erode public confidence and trust further.

"Statistics has a bad press…the media seem to be very happy to jump on statistics and statisticians"

In spite of this awareness that the media can sometimes misrepresent statistics in their reporting and are perceived by some to be prone to unjustifiably criticising the statistical service, there is a sense that there can be little comeback on this; in effect, the damage cannot be undone. A few mention that the Press Complaints Commission is ineffective in publishing apologies and rectifying mistakes with regard to the misuse of official statistics.

Stakeholder suggestions for improving public trust

Clearly, there are a number of factors which can impact on trust in official statistics. Encouragingly, however, many stakeholders make the point that there needs to be an increased recognition of the political climate in which statistics are now used, and how this impacts on how they are perceived. Furthermore, a number of measures are suggested by the stakeholders themselves as being effective ways to move forward in the areas that are highlighted as being a cause for concern.

Put the issue in context

It is important to recognise that the issue of trust in official statistics must be placed within the wider context of declining trust generally. This Government, like others, has been beset by problems with trust, with high-profile inquiries into its conduct highlighting the issue further. It is not surprising that official statistics have become bound up in this downward spiral and are viewed sceptically.

"It's part of a wider issue when you know that Government Ministers and Prime Ministers are found out to lie…then you lose trust in politicians and naturally, we'd lose trust in the way they use statistics, even if these are legitimate"

However, in spite of this, many of our respondents believe that the Government Statistical Service produces some of the best data in the world. In terms of methodological approach, rigour and accuracy, many feel that the UK is unrivalled. Some believe there is a need for the statistical service to make more of its successes, and that it should speak out about them in order to raise confidence.

"More voice would be a very good thing…it should be saying 'we have one of the most reliable sets of public numbers in the world, we know a tremendous amount about our collective lives courtesy of data collectors who are utterly trustworthy. There are occasional slips but if you take them in the fullness of the operation they are incredibly tiny and you, the public, should back off and start investing some credibility in numbers which blood and sweat have been invested to collect'. There's a rhetoric of positivity which we haven't had"

Additionally, some feel that the tendency of the media to exaggerate mistakes made by the statistical service is largely symptomatic of the blame culture that exists today. So much importance is now attached to statistics that when there are errors, it has become necessary to find individuals who can take the blame for this.

"We have gradually developed more and more of a blame culture whereby if there are mistakes, somebody is to blame"

Given the nature of today's society, it is therefore not surprising that some do not perceive official statistics to be particularly trustworthy, especially in view of the three-way breakdown of trust between the public, politicians and media. However, more positive communications from the statistical service may help to negate some of the damaging effects of this.

Use other evidence

There is also a sense among many of the key constituents that statistics are now expected to do more than they are actually capable of. This is partly due to an assumption that statistics are facts when actually they are simply the best estimate at the time. Furthermore, in a climate of targets, performance indicators, and evidence-based policy, more statistics are used than ever before as drivers of policy and change. Thus official statistics have assumed a role of vital importance, hence the increasing scrutiny placed on them.

Some feel that it is important to shift away from this over-reliance on statistics and for both policy-makers and the public to recognise that data cannot tell us everything that we need to know in order to make rational decisions. As such, some call for increased use of alternative forms of evidence as a means of supplementing statistics to give a more rounded picture of the world around us.

"You might get 53% of people who don't save enough for retirement but then you've got to say why are they not, and who is it? I think it's actually unrealistic in social policy areas to just look at the figures alone"

"Statistics, by themselves, are very unlikely to give you a full picture. Even the best statistics only give you part of the picture and other forms of research are always important"

Furthermore, there is a recognition that statistics are simply unable to measure all the various facets that, at times, need to be recorded. The point is made that while factors such as growth are relatively simple to measure quantitatively, conceptually at least, it can also be important to measure subjective issues such as happiness, or life satisfaction, which are much harder to break down into simple numbers. By relying less on official statistics to provide us with evidence and, instead, using additional forms of data, it is hoped that the statistics that are used will be afforded a greater degree of credibility.

Independence

In spite of this understanding of the context in which official statistics are used, there is a strong feeling that action needs to be taken to increase trust in, and the credibility of, official statistics. For many, the key to achieving this lies with securing independence for the statistical service. It is felt that, of late, the production of statistics has become politicised and when set against a backdrop of distrust in the Government generally, then a comprehensive restructuring of the statistical service is necessary. It is widely believed that by increasing the distance between Government and the statistical service, there will be less of an inclination to view its output with suspicion. The ideal is seen to be an organisational structure similar to that of the National Audit Office, whereby the statistical service would be answerable to Parliament rather than to the Treasury.

"You'd have an independently appointed board, to which the Chief Executive would be answerable, and then ultimately answerable to Parliament"

It is felt that such a move would not only free the statistical service from the charge that it is susceptible to Governmental pressure, but would also afford it greater autonomy in terms of the kind of work it conducts. As mentioned previously, one of the criticisms of the statistical service is that some perceive that it does not collect the most relevant data. It is thought that by severing its ties to the Treasury, the statistical service would have greater freedom to collect information on a wider, or different, range of issues - the scope of which could be determined by close consultation with key constituents. Such changes would encourage the image of a responsive statistical service, one which is free from actual or perceived Ministerial interference.

"We have to free it, and if you do that, although the numbers won't be different, the timing of their publication and the manner of their publication and the interpretation would be different"

Indeed, a few feel that until independence is realised, then the situation will stay as it is. They believe that in spite of recent positive measures, such as the implementation of the Code of Practice, the issue of Ministerial influence on official statistics is now so ingrained that it will take a complete reorganisation of the way that official statistics are produced, to act as a catalyst for change.

"We've now gone as far as we can go without being fully independent…there's nothing more we can do apart from having an independent national statistics body"

Other measures are suggested as a means of emphasising the distance that already exists between the Government and the statistical service. For example, some feel that by changing the website address suffix from .gov.uk to .org.uk then there would be less of a tendency to see official statistics as a product of Government designed to meet their needs.

"I'm not sure it's a good idea that the ONS website's address ends gov.uk. I'd feel slightly more comfortable if it ended org.uk"

Others are of the view that it is vital for the next National Statistician to clearly state his/her independence on appointment. For some, this should take the form of a high profile disagreement with Ministers; others do not feel the need for confrontation, and that simply stating their freedom from interference is enough.

However, this is not to say that all key constituents are in favour of the statistical service breaking their ties with the Treasury. Some - a minority - feel that the statistical service's primary obligations are to the state and then to the public, and feel that it is only right that ONS should be answerable to the Government.

"ONS is not, and I don't think should be, an independent body; it exists to provide the state with a body of information. To me, its primary obligations ought to be towards the state"

A 'gold standard'

Kite-marking official statistics is seen to be an effective means of differentiating between the output of the statistical service and that of other organisations. This is seen to be especially necessary now at a time where more statistics than ever are released and it is becoming harder to distinguish between what is reputable and what is not.

A few believe that by marking out official statistics in this way, it will also be possible to create a standard to which other organisations will aspire, thus driving up the quality of statistics generally.

"Government statistics would be seen as a golden standard of correctness to which we could all aspire and that might elevate Government statistics above some of the chit chat and frippery that goes on"

The quality marker 'National Statistics' is already applied to certain UK statistics and for these to be badged as such then they must meet certain criteria; be fit for their purpose, methodologically sound, politically independent and transparently produced in accordance with the Framework for National Statistics and also comply with the National Statistics Code of Practice. However, awareness of this among respondents is very low. Consequently, there is a clear need for this to be communicated more effectively so as to elevate the position of official statistics above other data that are released into the public domain.

Furthermore, some feel that kite-marks can be seen as a method of managing key constituents' expectations. In order to combat the uncertainty created by data revisions, some advocate using kite-marks as a symbol of whether the figures are final or likely to be revised in the future.

"A grading system in terms of three stars (for figures) it doesn't expect to revise, except under exceptional circumstances. Two stars for statistics that may get revised, but modestly, and one star or zero for stuff that is almost a finger in the air"

However, a few do warn against kite-marks acting as indicators of quality. While some do see the need to clearly distinguish between official and non-official outputs, they feel that any such identifiers must be carefully explained to ensure that they are not confused as being quality marks. This is due to the fact that, for some, there is an expectation that all official statistics should be of the highest possible standard, hence any badge to say that this is the case would be merely stating the obvious. However, there is also a concern that such a mark could lead to a simple acceptance of the data without any of the critical questioning that should accompany it.

"I'm very much against kite marks for statistical series which says this is high quality data. Firstly, all Government data ought to be high quality…and secondly, once you say it is good you let yourself in for blind faith"

Reduce prior access

For many, it is not the data that are at fault but rather the presentation that leads to a lack of trust. Many question the right of Ministers to access official statistics prior to their release and believe that this should be abolished or, at the very least, reduced.
Some draw on examples from overseas. They note the time difference in prior access and feel that Britain compares unfavourably in this regard.

"In most countries, the prior access is about half an hour - that's a huge difference and it does mean all sorts of mischief gets done"

There is a recognition by some that for key data series there will be a need for Ministers to have at least some advance warning to ensure that the response is appropriate. However, many feel there is a need to make this the exception rather than the rule.

Enhance comparability

Some are of the view that for official statistics to be as useful as possible, it is necessary to foster closer co-operation and joint working with international agencies. It must be stressed here that this is not due to the fact that data collection and interpretation are perceived to be of greater quality overseas than in the UK. Rather, it is to ensure that the data collected in the UK will have a comparative base. Many key constituents make the point that, while official statistics are key to judging performance and monitoring change, this information is meaningless unless it can be compared with similar situations elsewhere.

"It's very difficult to get meaningful, cross-border statistics about expected growth. And what we did discover is that you couldn't compare UK statistics with German statistics etc"

"International comparison is an extremely useful tool and all our statistics have got to be sufficiently robust and comparable to do that"

On a smaller scale, there is a need to ensure that, wherever possible, 'national' statistics means just that. Devolution has clearly impacted on the collection of data with responsibility for certain series being abrogated to individual legislative bodies. Consequently, there are instances when what are referred to as UK data, actually only cover England. To ensure that there are comparisons against which performance can be tracked and measured, there is a need not only to foster closer co-operation with statistical services overseas, but also within the UK as well.

Indeed, a few highlight the political trend towards localism, and state that if this is to become more of a driving force in British politics, then it will be necessary to have statistics to support this.

"With the increasing interest in our society of localism and comparison of where you live locally, or regionally, with other nations, I think official statistics should be about enabling local people to understand where they fit in"

Increase transparency

Improved communications from the statistical service are seen to be key if trust is to increase. As mentioned previously, the lack of explanation with regard to why data has to be revised, creates doubt and suspicion in the minds of some. Many feel the need for clearer explanation that first issue of figures are, in fact, just initial estimates and that there may be due cause at a later date for them to be amended as more information is gathered to ensure that the data are as accurate as possible.

"They have to present every figure as being 100% accurate…and, I think, in some ways it would be better served by having degrees of provisionality on the figures"

"There has to be communication from the Government and the ONS explaining that there are some things which are literally estimates - not facts, estimates"

A few feel that recent legislation, such as the Freedom of Information Act, will encourage this openness and will increase transparency.
"The more we can use the Freedom of Information Act to open up Government to make information available, the better"

However, there are those who remain unconvinced as to the powers of this piece of legislation.

"I think we will find that they have ways of letting you see what they want you to see or what you have to see if you know how to find it"

Overall, the need for improved and increasingly open communications on the part of the statistical service is seen to be vital if it is to rebuild trust.

Increase explanation

As stated previously, it is often the case that the statistics themselves are seen to be reliable and that the problems lie in the interpretation of them. Many respondents mention ways in which this can be overcome.

For some, graphical depictions of trends and data are an effective means of communicating information clearly. Due to the simplicity of visual presentation, it is felt that this could reduce misrepresentation of the data too.

"It's much more difficult to translate numbers than it is to translate words. The more we can use visual presentation to demonstrate trends and show comparisons in a user-friendly way, the better"

However, aside from the manner in which the data is presented, some feel that the statistical service should take a more proactive role in providing an explanation of the figures. At the very least, some feel the statisticians should highlight aspects of the series that are particularly interesting.

"Alerting us to things that have cropped up in the data is helpful"

As it stands, the situation exists that the statistical service is not in a position to freely comment publicly on its output which results in an interpretation being provided by those who do not necessarily have as good an understanding of the context as is required.

"There are huge amounts of social data thrown out raw and they mean nothing because they need interpretation…and the ONS usually feels it's not its job to provide that because it either requires a political judgement to be made or a…sociologically informed judgement which they traditionally haven't been terribly well equipped with"

Others suggest that there could be a role for an independent interpreter to fulfil this function. It is thought that this would lend credibility and trust to the interpretation and would ensure that the statistics are not misrepresented.

"Is there any role here for a trusted independent commentator on social trends? The interesting model here is the Netherlands; they have a body of about 90 economists and policy experts which analyses statistics and produces overview analysis"

"Ideally you'd have some kind of body, some people have talked about analogies to the Royal Society which, for example, was able to mobilise consensus and say the numbers mean X, Y and Z"

There are also calls for the utilisation of technology to a greater extent than is currently the case. The internet is seen as a valuable resource on which additional details can be posted in order to contextualise the data, so that they are meaningful and relevant.

"With the technology that's now available with the internet, you can do a lot more in terms of providing background information"

Some also feel that the media should take a far more proactive role when it comes to understanding and interpreting data. Some believe that many journalists do not have the technical wherewithal to understand and, therefore, comment on statistical releases.

"What the media are doing is failing to perform any check themselves. They're failing to contribute to things getting better…it's garbage in, garbage out. If the media wanted to go to a useful place, it would be to become more of a bastion of truth. They're bastions of truth in whether David Beckham is or is not having an affair but they're not bastions of truth on whether the outputs of the UK were cut by 3.5% or 5.3%"

On this, a few suggest that more rigorous training should be provided to remedy this.

"Given that I think the NUJ might control some of the training that they have, is there a role for the NUJ trying to have at least some basic understanding of not how to produce statistics, but how to interpret them?"

Ensuring that the media are well versed in statistical knowledge is key. Many within the media mention how they, as spokesmen, are a trusted source of information and that sections of the general public, who may not trust Government, are more inclined to trust their messages when they hear them via this intermediary.

"There is no question in my view that the public will often distrust the statistic. But equally I think the public actually are really distrusting the politician and if I used the same statistic they would probably believe it from me"

Given this, it is important that those within the media who do operate from a trusted platform are able to interpret statistical information correctly so as to ensure that what they communicate is not only believed but is also accurate.

Regulation

Some feel that for the situation to improve, there is the need for a regulatory body which can not only monitor the use of statistics but can also act as a forum to which people can complain should they believe that figures are being used inappropriately.

"Some regulatory body with functions where people could appeal to it and it would actually offer warnings about the use or misuse of statistical information"

Again, many feel that such a body should be independent from Government to ensure that it is perceived as a trusted organisation.
However, others feel that this is in fact a role that the Statistics Commission could grow into in time. Again, independence and direct reporting to Parliament is urged; however, it is believed that the Commission has the necessary expertise and influence to be able to make a real impact here.

"Being a constructive friend; that is a role that the Statistics Commission could rightfully, usefully play, without actually being seen as toadying up to Government"

There are also those, however, who do not see the need for regulation. Instead, they feel that the statistical service should, in a sense, act as its own watchdog and when they feel that their data are being misused, should be far more forthcoming in speaking up on this.

Enhance public education

There is a view that part of the problem lies with the low statistical literacy that exists within the UK. Due to the lack of public comprehension and an inability to interpret the figures correctly, it is felt that some of the negativity surrounding official statistics is a result of low level knowledge of this area.
"Statistical literacy in this country is relatively low and it's a real challenge for the nation at all levels to improve"

"It's a wider, cultural issue about maths being difficult and numbers being abstract and hard to understand which, I think, is hogwash. I think we have a very miserable attitude towards the quantified in this country - we're very frightened of it"

There is a sense among many of the key constituents that until significant changes within the education system are made, then this will continue to be a problem. It is recognised that this is a long-term measure and that there will be no immediate effects; but for the public to understand the data that they are presented with, such steps are seen to be vital.

However, it is not only on statistical literacy that the public lacks understanding. Some respondents also believe that more efforts need to be made to equip people with the tools they need to critically examine the data that they are presented with. This will not only require a greater level of understanding of statistics per se, but will also involve thorough questioning as to the source and methodology.

"If you can teach people to be critical, that would be great. If you can teach them that you don't necessarily believe the numbers unless you know where they have come from and what has been done with them"

Some advocate that such skills should be taught as part of the citizenship agenda and, by doing this, there would be greater levels of engagement with the political process.

"I know that citizenship in education is on the curriculum and one aspect of being a good citizen is being able to tell the good argument from a bad one"

It is suggested that, as a result of this lack of understanding, there is a tendency for people to turn to their own personal experience as a means of judging the country, rather than looking at the bigger picture.

"The purpose of statistics, in a sense, is to get people away from their own experience and towards a much broader view and people find that extraordinarily hard; how can you tell me that crime's going down when we've had two burglaries in this street last week?"

Our interviewees believe it is vital to present statistics to the public in a way that is not intimidating. For example, a few suggest that the statistical service could produce a booklet highlighting key trends on topics that may be of interest to the general public, such as the economy and the demographics of the population. This could be made available in public places in order to increase the level of understanding and engagement.

"There should be a booklet made available on a brief statistical picture of the UK…maybe not delivered to every home but available at public libraries"

Others advocate the production of a television programme which would explain the week's most significant statistical releases and would provide a simple interpretation of them.

"If you had a Sue Lawley or Des Lynam type figure, or someone who would present stories in statistics in an accessible way as part of TV. More headline statistical measures ought to be, as far as possible, presented in that way and there ought to be some attempt at getting them into more of these programmes"

"You could imagine a kind of half hour weekly slot…someone who would look at the week's statistics, go through them and bring on people who knew what they were talking about. I think it would be a wonderful educational opportunity"

Appendices

Background and Methodology

Background and objectives

MORI was commissioned to undertake a study on behalf of the Statistics Commission in order to establish stakeholders' opinions regarding levels of trust in official statistics. The objectives of this study are:

• to gauge perceptions of the credibility and reliability of official statistics
  
  
• to explore the ways in which official statistics are used and presented by government
  
  
• to determine how perceptions of the independence and credibility of these statistics can be adversely affected
  
  
• to understand why problems occur and whether the situation has deteriorated in recent years and
  
  
• to explore what measures and procedures can be put in place to ensure there is effective scrutiny with regard to the generation, presentation and use of statistics.

Methodology

Interestingly, even among key constituents, there is no universal consensus as to what constitutes an official statistic and, in some cases, who is responsible for producing them. Some differentiate between the outputs of ONS and individual Government departments, with others choosing to bracket them together.

There is an archipelago of official members and if the biggest island is ONS, it is by no means the only one"

However, the majority tend to classify official statistics as emanating from ONS or individual Government departments.

"Official statistics are, to me, anything that used to be produced by Government departments that are now under the umbrella of the ONS"

"They are something put out by the Office of National Statistics or some other similar body - basically put out by a Governmental agency"

There are those who widen this definition and believe that a number of different organisations can produce 'official' statistics, such as think tanks, independent research organisations/trusts and charities.

"It's a statistic that an authoritative body puts out as fact"

Further verbatim quotes

We have included in the appendix some further verbatim comments. Please note that these comments need to be taken in the context in which they were made. They do not necessarily reflect the views of other individual respondents.

Methodology and data collection

"My basic view is that almost always the numbers are correct, they just may not be the numbers we really want"

Revisions

"Revisions to statistics certainly diminish my confidence…it's like the thirteenth stroke of a clock; it's wrong in itself and casts doubt on everything that went before"

"What does concern me is when…they are treated like complete gospel"

"Somebody changed a figure…and then put the numbers out a week later and there was no explanation at all, and I thought that just shows a cultural problem in that they should've wanted to"

Errors in production

"The last Census where we had the alleged missing millions…I think ONS were quite slow in actually taking that issue up"

"When they get criticised they immediately go into their defensive mode and say, 'well of course this is right'"

Perceived Government Influence

"It's actually incredibly important for Government to keep out, for Ministers to keep out, and they don't at the moment"

"Statistical purity is lost in the bigger political picture"

"There's definitely some relationship between the politicians and the ONS and you can't tell how much that affects them"

Politicisation

"There is a view that statistics produced by Government departments tend to be more politicised"

"I've had numerous examples of consultants, doctors and Health Authority managers coming to me and saying, well, we can't afford to do what we want in the best interests of the patient because we're trying to meet this particular national statistic"

Presentation

"Too many people are given sight of them too far in advance so that they can be spin doctored. That can undermine credibility"

"What is their right to have it before us? It gives them potential time to work out how to present it"

"Britain has one of the worst records of giving Ministers prior access to statistical data. And once you do that you risk having the data spun and discounted and not properly debated because the Opposition has as much right as the Government to actually look at official statistics and comment on them and understand them, but the Government has much more prior access"

"I think there is inevitably a lot of massaging…I think this is a post-1997 phenomenon"

Media distortion

"I think a lot of the newspapers are as political and as selective as the politicians"

"It has to be said that the Press Complaints Commission is not distinguished for actually achieving timely remedies to mis-statements about statistics"

Independence

"It would give the impression of more distance"

Enhance comparability

"I think it's increasingly the case that all official statistics will need to be disaggregated by geography"

Increase transparency

"Just be honest and straight and then we know how to interpret it"

Increase explanation

"There's a press conference today on new figures for NHS productivity - how many journalists will attend who are what you might call really up to speed; six, five, four? They produced a paper earlier this summer on public sector productivity - how many newspapers thought it was worth publishing? Answer, one"

Regulation

"If you had a regulator you could actually attempt to force organisations to justify what they're doing in proper scientific terms"

"It would be quite nice if they saw themselves as referees occasionally"

"I like the idea of an independent Statistics Commission, I think that is useful"

"The role of the Statistics Commission and the other scrutinising bodies is crucial and their role needs to be encouraged so that we have independent scrutiny"

Enhance public education

"What is going wrong with the teachings of mathematics that children come out of school after 10 years unable to count and unable to do simple arithmetic and even unable to understand the most simple concepts like percentages"

"It's to do with a bloody minded attitude which seems to have set in on the part of a large number of members of the general public which says, I live my life, I often live it quite well, I'm not going to have any trouble with these people up here"

 

Discussion Areas

The agenda of the discussion was very much shaped by the opinion formers themselves. Outlined below are the key areas that were covered in the interviews.

• Definitions of 'official statistics'
  
  
• The role and purposes of official statistics
  
  
• Factors that may undermine the credibility of official statistics
  
  
• How the issue of trust in official statistics may have changed over time
  
  
• Examples of good practice, in both the UK and overseas, that have been implemented or that the statistical service could draw on
  
  
• Methods of presenting official statistics to the public so the data is both understandable and credible
  
  
• How the media may affect trust in official statistics
  
  
• Levels of awareness of measures in place designed to protect the integrity of official statistics and
  
  
• Suggestions for other measures that could be implemented in the future to help improve trust in official statistics.