Good environmental decisions and strong metrics

Many environmental projects on private land involve planting native trees. It’s easy to estimate how many trees will be planted but a lot harder to compare the expected outcome of one tree-planting project with another. Answering the nine questions posed here for each project is a good first step towards undertaking a robust ranking.

Many environmental projects on private land involve planting native trees. It’s easy to estimate how many trees will be planted but a lot harder to compare the expected outcome of one tree-planting project with another.
Answering the nine questions posed here for each project is a good first step towards undertaking a robust ranking.

Good environmental decision making is information-intensive. Environmental managers invest a lot in monitoring and research to collect information, but often take a rough-and-ready approach to combining that information into a form that is useful for decision making. Does this matter? Does it make a difference to environmental outcomes to use a theoretically sound decision metric, compared with a weak decision metric? That was the question David Pannell and Fiona Gibson set out to answer by comparing environmental outcomes generated by these two approaches.

What they found, in short, was that it does matter which decision metric you use. Indeed, it can make an enormous difference. As a consequence, many decision metrics used by environmental managers result in us missing out on very large environmental benefits.

A common approach used by environmental managers to score the projects they have to choose between is to define a set of variables believed to correlate with projects’ benefits and costs, and combine them into a formula or metric so that projects can be compared. Numerical values or scores are assigned to each potential project and these scores are used to rank the projects.

There are many different ways the various benefits and costs of a project could be combined and there are thousands of different decision metrics in practice around the world. Unfortunately, many (if not most) of these decision metrics have problems in the way they determine the value of the project. Commonly used decision metrics have a range of weaknesses, including adding variables that should be multiplied, omitting important variables related to environmental benefits, omitting project costs, or subtracting costs rather than dividing by them.

The researchers estimated the environmental losses resulting from each of these weaknesses. They found that poor metrics resulted in environmental losses of up to 80% – not much better than completely random uninformed project selection. The most costly errors omitted information about environmental values, project costs or the effectiveness of management actions. Using a weighted-additive decision metric for variables that should be multiplied is another costly error commonly made in real-world decision metrics. They found that omitting information about project costs or the effectiveness of management actions, or using a weighted-additive decision metric (that should be multiplied) can reduce potential environmental benefits by 30 to 50%.

They also demonstrated that the quality of the decision metric makes a much bigger difference to environmental outcomes than the quality of the information used within it.

So, your choice of metric matters. Simply choosing a logical metric can improve environmental outcomes more than even obtaining substantial increases in environmental budgets. Of course, getting a bigger slice of the budget will help, but it is critical to ensure that any money is spent wisely by using a good metric.

See Decision Point #82 for the complete story


Reference

Pannell DJ (2013). Ranking environmental projects. Working paper 1312, School of Agricultural and Resource Economics, UWA. Crawley, WA. http://ageconsearch.umn.edu/handle/156482

Pannell DJ & FL Gibson (2014). Testing metrics to prioritise environmental projects. Working Paper 1401, School of Agricultural and Resource Economics, UWA. Crawley, WA. http://ageconsearch.umn.edu/handle/163211

Leave a Reply