Decision science and environmental policy

Achieving better outcomes from environmental policy is more than just good science

Decision scientist and policy person out in the field. Researchers want their research to inform policy and management but it often doesn’t happen. For greater interaction, scientists need to acknowledge the many cultural differences between the the world of research and policy formation.

Decision scientist and policy person out in the field. Researchers want their research to inform policy and management but it often doesn’t happen. For greater interaction, scientists need to acknowledge the many cultural differences between the the world of research and policy formation.


Key messages:
  • Decision science is only one of many factors that may be relevant in the environmental-policy process.
  • Decision science has enormous potential to make policies more effective at delivering outcomes that are highly valued by the community.
  • There are a number of challenges for decision science if it is to meet its potential.
  • There are many examples where decision science has met these challenges and made important contributions to environmental policy.
  • Key success factors for decision scientists to contribute to environmental policy include good communication, pragmatism, patience, persistence, a capacity to develop meaningful relationships with policy makers, and choosing the right decision-science projects in the first place – ones that are likely to make a difference.

The creation and design of an environmental policy involves a myriad of decisions. Here are just a few of them:

What’s the need?

Should there be a policy at all? Which specific environmental issues should be covered by the policy? What goals or targets should be set for the policy? Over what spatial scale should the policy operate? Over what time frame should it operate?

Where’s the emphasis?

Should effort and funding be targeted to key priority issues or locations, or spread relatively evenly across issues or locations? Should the policy involve public education? If so, who needs educating, what about, and how? Should the policy involve regulation of the actions of individuals or businesses? If so, how large should the fines for non-compliance be? How much should be invested in inspection and testing of compliance?

Where’s the focus?

Should the policy involve payments to reward positive environmental behaviour? If so, how large and how frequent should the payments be? Should people be rewarded for approved actions, or only if those actions result in positive environmental changes? Is research or a pilot study needed prior to full implementation? What approach to monitoring the results of the policy should be used? And, of course, each question (of which the above is just a logical beginning) spawns its own set of sub-questions, which in turn generate even more, and so on.

The relevance of decision science

While easy to pose, most of these questions are not straightforward to answer. The alternative options to each decision may have multiple consequences, some of which are hard to predict. In most cases, there is high uncertainty about the environmental consequences of the various policy options. Relevant considerations include social, administrative, political and financial issues, not just environmental consequences. These different issues need to be drawn together and weighed up, but the best way to do that may not be apparent.

Decision science is definitely relevant to this. It provides a set of concepts, models and tools that have been developed to help address complex decisions under uncertainty. It aims to formulate each decision problem into a coherent and logical structure, so that it is easier to think about the options. In so doing, the consequences of these options become more transparent and easier to compare.

This helps decision makers compare and rank policy options in ways that are logically sound. While this seems like a sensible and rational thing to do, it’s something that environmental managers and policy makers often fail to practice. For example, we know a lot about what are good approaches to prioritisation and yet this knowledge is frequently not applied (eg, consider how prioritisation metrics are often poorly constructed, see Decision Point #82). Or, as another example, a lot of decision science has gone into how biodiversity offsets should be estimated (see Decision Point #69, p10,11) but these lessons are sometimes not well accounted for in policy. If done well, decision science should also help decision makers avoid some of the biases that plague most decision making (See Decision Point #93). On the face of it, decision science should be highly useful to environmental policy makers.

Nevertheless, for a range of reasons (some of which are discussed here), most environmental policy decisions are made without help from decision science. There are many excellent examples where it has played a helpful role, or even a pivotal role, but they are the exceptions. The more common situation is that prioritisations are done poorly or not at all, that decisions are made without thorough and systematic analysis of the options, and that ongoing monitoring of the outcomes of our environmental decisions is done inadequately.

In this article I will reflect on the nexus between environmental policy and decision science. What aspects of the policy process make it difficult for decision science to play a bigger role? I’ll briefly present some examples of environmental policy development with varying levels of use of tools developed by decision scientists. And, for those decision scientists who aspire to play a bigger role in environmental policy, I’ll discuss strategies they may want to consider.

Challenges for decision scientists

Go East young man! David Pannell (pictured above) is a resource economist who has put considerable effort into influencing environmental policy. Based in Perth, many is the time he has travelled east to Canberra to lobby for cost effective and robust policy prescriptions.

Go East young man! David Pannell (pictured above) is a resource economist who has put considerable effort into influencing environmental policy. Based in Perth, many is the time he has travelled east to Canberra to lobby for cost effective and robust policy prescriptions.

In my time working in the field of environmental decision science, working with researchers from a variety of disciplines, I have identified a range of challenges that have increased the difficulty of decision science influencing policy outcomes. These include:

Non-scientific considerations matter: Sometimes scientific information may be known to policy makers, but the decision reached may still appear to be inconsistent with the science. This may or may not be a concern. It may simply reflect that policy makers and managers must consider additional factors, such as legal mandates, societal desires, economic benefits and costs, rights, distributional equity and procedural fairness.

Hidden agendas: There may be political or bureaucratic objectives unrelated to the public interest, so that research that seeks to advance the public interest is not given the priority it deserves.

Policy fashions and crises: Policy attention tends to be directed to certain issues with high currency, often issues where there is a perceived crisis, and this may leave little scope for research (or any other input) to influence policy in an area that is not currently high on the agenda. On the other hand, in certain circumstances, it may be easier to foster policy change in an area that is not in the public spotlight, at least from the perspective of avoiding controversy. This depends on convincing policy makers that a change is justified, which is likely to be difficult for non-topical issues.

Difficulty getting access: Policy makers often prefer local trusted sources for information. This is understandable, given the flood of information that policy makers can face on some issues, but it does not ensure that the most appropriate information is used.

Distrust: There may be suspicion about motivations of researchers, so that they are treated as just another interest or lobby group. Even without overt advocacy, some environmental scientists tend to intertwine facts and values, and this affects the perceived independence of their scientific advice.

Lack of expertise: In some policy agencies there is rapid turnover or movement of staff leading to a lack of expertise in responsible staff and a lack of knowledge of relevant research and researchers. A culture may develop in government agencies that detailed subject knowledge is not necessary. Another set of challenges arises from differences between the circumstances, cultures and reward systems of researchers and policy makers.

Different outcomes sought: Researchers place a high value on knowledge and innovation while policy officers seek to advance the public interest, to capture resources, and to please their political masters, who are primarily concerned about being reelected.

Source of recognition: The sources of recognition for practitioners are different: from an administrative or political master in the case of policy, and from peers in the case of research.

Achievements rewarded: Policy officers tend to be promoted based on their ability to successfully implement desired policy programs, while researchers are promoted according to their productivity of research outputs, especially of those judged to be high in scientific quality.

Controversy vs compromise: Policy officers aim to resolve management or political problems with minimal controversy, making pragmatic compromises wherever necessary, whereas a healthy scientific discipline thrives on open debate, and will not compromise the truth.

Communication: Researchers and policy makers speak different languages, with different acronyms and jargon and different hidden assumptions. Scientific communication can be hard to understand even between different scientific disciplines. Policy officers deal mainly with very brief, simply written and highly interpreted/synthesised material conveying only essentials with a focus on practical implications and recommendations. On the other hand, even brief scientific writing is considerably more detailed and qualified and much research deals with practical implications as an afterthought, if at all.

Time frame: Once the decision has been made to develop or change a policy, the time frame for the process is usually short. This can place participants under great pressure, with little time for careful consideration or collection of information. Research is generally slow and unresponsive to urgent policy needs, although it can be responsive in the longer term.

Supply vs demand: Policy usually addresses a problem identified by someone else (demand-driven), while the directions of research are often selected by researchers (supply-driven), particularly in a university environment. In a government research agency, research directions may be a mix of supply- and demand-driven.

Complexity vs simplicity: Policy officers prefer simple, straightforward advice with few, if any, caveats, whereas researchers tend to enjoy unravelling the full complexity of an issue, with all caveats highlighted.

Specialisation vs breadth: University training and the academic reward system encourages narrow specialisation, whereas policy officers need to consider a broad range of factors.

People focus: Policy involves intensive interaction among diverse groups of people, requiring highly developed social skills, while for some scientists working intensively with people is not comfortable. Of course, good interpersonal skills are valuable in both realms, but I believe that there is a difference in the degree of their importance. Given these many challenges, it might seem remarkable that research ever plays a meaningful role in the development of policy. Often it doesn’t, but sometimes it does. Consider the following case studies.

Case studies of impact

Fiona Gibson (working with a range of colleagues, including me) recently analysed the factors that facilitated or inhibited the use of rigorous decision-support tools in Australian environmental policy and management (Gibson et al, 2017). We selected seven case studies of policy development and conducted personal interviews with managers and policy makers who had been directly involved.

The case studies we reviewed involved:

  • the Southern and Eastern Scalefish and Shark Fishery policy (perceived by interviewees to be an example of where decision support tools were much used = high);
  • the Representative Areas Program (high);
  • South West Marine Reserve Network (low);
  • National Reserve System (none); and
  • three examples of Threatened Species Protection in different jurisdictions, Australian national (none), New Zealand (moderate) and New South Wales (moderate).

The selection of case studies was not intended to be representative of all possible conservation policies. However, they were a diverse selection and provided insights that are likely to be transferable to other case studies and policies.

Drawing from the existing literature, the researchers identified a range of factors that are likely to promote or prevent the uptake of decision support tools in environmental and conservation programs. I’d suggest that these can be interpreted as factors influencing the uptake of decision science more generally, not just decision-support tools.

  • Presence of a champion for the tool within the agency
  • Presence of an advocate for the tool outside of the agency
  • Existence of a relationship between agency staff and tool experts
  • Presence of large numbers of stakeholder groups affected by the policy outcome
  • Ability of the tool to deal with missing information
  • Whether the tool can be applied quickly
  • Whether the policy process allows adequate time for tool use
  • Whether the tool capabilities align with policy objectives and policy operation

These factors were used to develop the questions used in the policy-maker interviews.

Those factors rated as most important by the interviewed policy officers and managers were the alignment of a decision support tool with policy objectives and operation, and its ability to be useful even when there is missing data.

The issue of alignment with objectives arose because in some cases a tool was perceived to be focused on environmental outcomes and not able to adequately represent non-environmental policy objectives.

Lack of complete data is almost always a challenge when considering policy options, and the tools that were viewed favourably and utilised more extensively tended to be viewed as being amenable to strategies such as expert elicitation to fill data gaps.

The least important factors, at least for these interviewees, were the presence of a champion of the decision-support tool within the management agency, and the time required to apply the tool. The finding regarding an internal champion conflicts directly with some previous studies where the presence of a champion was critical to effective use of the tool (eg, Decision Point #76).

Interestingly, the interviews revealed additional important factors that had not previously been identified in the literature. These included: the existence of multiple (potentially unstated) policy objectives, and the autonomy of the agency. Where there were two conflicting objectives (eg, conservation and sustainable economic use of the resource), some interviewees lacked confidence that the tool could handle this, or would give the competing objectives weights consistent with those of the policy makers. It was felt by some interviewees that relatively autonomous government agencies are less prone to intervention by a government minister concerned with the politics of an issue. This lower risk of interference made it easier for transparent and systematic decision processes to operate. The particular autonomous bodies in these case studies also had a greater emphasis on day-to-day engagement with stakeholders, such that the potential benefits of a decision support tool in enhancing engagement may have been more apparent.

The findings of Gibson et al (2017) reflect the perceptions of the interviewees. Notably, it appeared that some of the most negative perceptions about particular tools may have arisen from misunderstandings about the tools, or a lack of capability to use the tool appropriately. There were examples where a tool that was rated negatively by one agency for a particular criterion (eg, ability to cope with missing data), was rated positively by another agency for the same criterion, and the other agency had actually used it successfully. This highlights the importance of clear communication channels and the provision of training and support to allow the decision tools to deliver their potential.

So, if you are a decision scientist, how might you make a bigger impact when it comes to the development of policy? In the box on ‘decision scientists in the policy arena’ I outline a set of strategies you might consider. It is is not a prescriptive or complete list but these 13 strategies are important considerations for any researcher wanting to influence policy.

If you are a decision scientist who aspires to influence policy, how would you rate if you turned these strategies into a checklist?

More info: David Pannell david.pannell@uwa.edu.au

Reference

Gibson FL, AA Rogers, ADM Smith, A Roberts, H Possingham, M McCarthy & DJ Pannell (2017). Factors influencing the use of decision support tools in the development and design of conservation policy. Environmental Science & Policy 70: 1-8.

Gibbons P, C Zammit, K Youngentob, HP Possingham, DB Lindenmayer, S Bekessy, M Burgman, M Colyvan, M Considine, A Felton, RJ Hobbs, K Hurley, C McAlpine, MA McCarthy, J Moore, D Robinson, D Salt & B Wintle (2008). Some practical suggestions for improving engagement between researchers and policy-makers in natural resource management.  Ecological Management & Restoration 9: 182-186.  (See Decision Point #25, p8)

Pannell DJ & AM Roberts (2009). Conducting and delivering integrated research to influence land-use policy: salinity policy in Australia. Environmental Science and Policy 12: 1088-1099.


Decision scientists in the policy arena

The following strategies, adapted from Pannell and Roberts (2009) and Gibbons et al (2008), are provided as food for thought for decision scientists. They are not in order of priority and they shouldn’t be considered as a prescriptive recipe. Rather, think of them more as ingredients. It depends on how you combine them as to what type of cake you create.

  1. Develop relationships with policy makers: Attempt to establish a high level of mutual understanding and trust. Information needs to flow in both directions. Only that way can you understand their perspectives and needs, and they understand your contribution.
  2. Research is not enough: Appreciate that good science is needed but is not sufficient for decision makers. In considering policy options, policy makers will probably be more concerned with social, economic, political or administrative aspects than with technical aspects of science.
  3. Practice excellent communication: In communications, recognise the lack of time that policy makers have. Be very brief, focus on clear messages, use simple language, free of jargon, using a mixture of approaches. Written material is useful but is not sufficient. Even more important is effective verbal communication.
  4. Simplicity is essential: As far as possible, the solutions one offers need to be simple, transparent and understandable. Policy makers are likely to be suspicious of solutions that rely on complex and opaque computer models (although the success of climate modellers in influencing the debate about climate change shows that models can be used if their message is sufficiently clear and simple).
  5. Work with intended users: This will help to ensure that the solution being proposed is in fact practical and sufficiently simple. It will help to make sure that their issue of concern is addressed in a way that is relevant to them. When attempting to convince policy makers, it helps to be able to demonstrate that the solutions being proposed are already in use in the  real world.
  6. Distinguish between knowledge and values: Be clear that the values that policy attempts to enhance are based on the desires of the community, not those of researchers. It is acceptable for research to deal with values (eg, studies of the non-market values of environmental outcomes) but one must be clear that policy makers will have their own views about the values. Traditionally, science deals primarily with knowledge rather than values, but of course scientists are influenced by their own values.
  7. Be pragmatic: One has to accept compromise, and it may be necessary to make conscious decisions about where you can and cannot afford to give ground.
  8. Be patient and persistent: Your work may not be influential at first, but its acceptance could grow over time if you are persistent. Establish networks and build support for your ideas over time. Repetition is essential, even to people who are already on your side.
  9. Be resilient: Numerous problems, frustration and setbacks will arise. People with vested interests in the status quo will actively resist proposals for change. These people may be insiders to the policy organisation and so have better access to decision makers than outside researchers do.
  10. Timeliness is important: Be prepared to respond quickly to requests for information. Policy makers cannot wait for additional research.
  11. Find a champion: Earlier I’ve noted mixed evidence on this, but at least in some cases it is likely to be worth cultivating a champion for your work within the policy organisation.
  12. Avoid any appearance of vested interest: Do not present findings and seek funds at the same time.
  13. Choose your research/analysis topics well: It doesn’t matter how good your communication is or how strong your policy networks are if the topics you are researching or analysing are not important to policy makers, or are not providing information that can help them. This seems obvious, but is sometimes not considered sufficiently when decision scientists are planning their own work.

Leave a Reply