Previously I have argued that statistical literacy is relevant to film studies because much research on the cinema presents quantitative information in numerical, graphical, and tabular forms, and it is therefore necessary to be statistically literate in order to understand research on film industries, film style, film audiences, and film perception (see here).
‘Evidence-based policymaking’ has become one of the key phrases of the past 15 years, and refers to ‘a policy process that helps planners make better-informed decisions by putting the best available evidence at the centre of the policy process’ (Segone & Pron 2008). Statistics have been described statistics as the ‘eyes’ of policymakers (AbouZahr, Ajei, & Kanchanachitra 2007), while Scott (2005: 40) writes that ‘good policy requires good statistics at different stages of the policymaking process, and that investment in better statistics can generate higher social returns.’ Most people involved in a decision-making process will be using data collected, analysed, and interpreted not by themselves but by professional statisticians, sociologists, market researchers, economists, and so on. It is important to recognise that while we need to be able to understand the information presented to us as part of the making of policy we do not necessarily need to be involved in the research process itself. You can criticise research even if you are not a researcher, and you can criticise statistics in research even if you are not a statistician. It is necessary, therefore, to bear in mind the difference between ‘statistical competence’ and ‘statistical literacy’ I noted in my earlier post.
A distinction can be made between people who are users of statistics and those who are provider of statistics. Whilst it may be unrealistic for professional decision-makers and practitioners to be competent doers of statistics, it is both reasonable and necessary for such people to be able to understand and use statistics in their professional practice. Integrating statistics into practice is a central feature of professions. An increasingly necessary skill for professional policy-makers and practitioners is to know about the different kinds of statistics which are available; how to gain access to them; and, how to critically appraise them. Without such knowledge and understanding it is difficult to see how a strong demand for statistics can be established and, hence, how to enhance its practical application (Segone & Pron 2008).
Participating in a policy making process therefore requires – as a minimum – the ability evaluate research and to understand quantitative information presented in a variety of forms. The Australian Bureau of Statistics put this very clearly:
The availability of statistical information does not automatically lead to good decision-making. In order to use statistics to make well-informed decisions, it is necessary to be equipped with the skills and knowledge to be able to access, understand, analyse and communicate statistical information. These skills provide the basis for understanding the complex social, economic and environmental dimensions of an issue and transforming data into usable information and evidence based policy decisions.
If you do not understand the information provided to you, the methodologies used, and the pitfalls of both how can you make a sensible decision about which policies have been effective in the past and how can you decide which will provide the best policy for success in the future? Or, as Florence Nightingale wrote, ‘Of what use are statistics if we do not know what to make of them?’
These issues are directly relevant to film studies and its relation to policymaking for film and film education in the UK. The DCMS policy review published in 2012 recognised ‘the need for a strong evidence base for film policy’ and recommended the establishment of a ‘research and knowledge function’ for the BFI in order to
a) collaborate with industry and stakeholders to generate robust information and data on which to base policy interventions, b) assist in the design of BFI policy and funding interventions from the outset to produce learning that can inform future policy, c) actively disseminate results and learning from funding interventions, and d) over time build and maintain a valuable and accessible knowledge base for the benefit of the public, the BFI, Government, industry, academia and all other stakeholders in film.
Evidence-based policymaking has clearly arrived at the BFI, and statistics will inevitably be a part of this process. The BFI’s research outputs already have a substantial statistical component. Obviously, the statistical yearbook is the standout case here, but the Opening Our Eyes report (see here) and the recent policy review both use information presented in numerical, tabular, and graphical forms. These are intended to be used as part of the evidence base for subsequent policy making regarding film education and training (as articulated in the New Horizons document, see here), film distribution, and film production.
Other agencies also produce data-heavy reports. For example, Skillset notes that ‘research provides the evidence, authority and justification for all we do’ and includes large amounts of statistical information in its surveys. There is also much research available from the EU that is loaded with statistics. To these we can add trade publications (Screen International, Variety, etc) and academic research on the cultural economics of film (such as those papers collected together for last week’s post here). Again, this is information that is supposed to provide a basis for decision-making about UK film policy, and all of it containing quantitative information to be used as the desired evidence-base.
The ability to participate in debates is predicated on an assumption that those involved in this process are sufficiently statistically literate to be able to work with the available data and analyses thereof. However, statistical literacy is not a part of the film studies curriculum in the UK at any level. Consequently, film scholars who do not possess the required level of statistical literacy will not be able to fully engage with any evidence-based policy process. Furthermore, film studies courses are not producing graduates with the required skills to participate in debates on film policy in the UK and so this situation will not change. This cuts both ways:
- If you’re not statistically literate, how are you going to know which questions to ask of the information presented to you?
- If you’re not statistically literate, how are you going to communicate your ideas to those with ultimate responsibility for decision-making?
Since the BFI was re-constituted following the abolition the UK Film Council, film studies has to work harder to make its voice heard in the same quarters as industry bodies that have much more experience of lobbying government agencies and are much more effective at it. There is a risk that film studies will be overlooked: for example, in New Horizons ‘education’ tends to be equated with ‘training’ and academic film studies is largely absent, while the panel for the DCMS policy review did not include a single academic working on film in any field let alone film studies. Without taking statistical literacy seriously film studies will find it more difficult to make its voice heard, and risks being reduced to a passive observer of the policymaking process unable to engage in key aspects of the debate because of a lack of relevant skills in understanding the complex and varied dimensions of an issue.
The other side of this coin is that if the BFI is going to produce numerous reports containing large amounts of quantitative information and expects (deep breath) ‘stakeholders’ to participate in an evidence-based policymaking process then it needs to ensure those involved are sufficiently literate to work with statistics. Are film producers statistically literate? Is the Minister for Culture, Communications, and Cultural Industries statistically literate? Is Amanda Nevill statistically literate? The BFI has to take a lead in promoting statistical literacy in order to render consultation processes meaningful, and other film and education bodies have to follow.
The alternative is to have an evidence-based policymaking process in which no-one is able to communicate, understand, and/or challenge the evidence effectively.
Scott C 2005 Measuring up to the measurement problem: the role of statistics in evidence-based policymaking, in New Challenges for the CBMS: Seeking Opportunities for a More Responsive Role. Proceedings of the 2005 CBMS Network Meeting, Colombo, Sri Lanka, 13-17 June 2005: 35-93.
Segone M and Pron N 2008 The role of statistics in evidence-based policymaking, UNECE Work Session on Statistical Dissemination and Communication, Geneva, 13-15 May 2008.