Building models for better decisions

The role of ecological understanding, models & model makers in decision making

By Peter Vesk and Libby Rumpff (University of Melbourne)

Models are basic to good decision making. System models are representations of the dynamics of an ecological system, a conceptual map of how the system works. They enable us to specify our thinking on how the system responds to management. Without them in our decision frame it’s unlikely our choices will be well founded. What’s more, and just as important, without a system model the potential to learn is limited.

Of course, a good decision can be made without a formal model. However, it should be recognized that however a decision is made there is always at least a ‘mental model’ guiding the decision maker(s). A mental model is simply someone’s thinking about how something works in the real world. Sometimes it’s referred to as intuitive decision making; in other words, decisions are based on someone’s intuition.

Relying on a mental model can be problematic in that it is not documented, so the underpinning logic can’t be interrogated or transferred, and its compatibility with observed data cannot be tested. In addition, there is a greater risk that the model is myopic, and does not consider the suite of objectives or actions relevant to the context of the decision.

Formal models are documented for all to see. They can be qualitative or quantitative, and both have their uses. A qualitative model helps to clarify thinking, and test logic, coherence and communicability. A qualitative model may provide a starting point to sketch out the dynamics of the system as a way to brainstorm and check that the relevant values, threats and drivers, and associated interventions are captured in the model. Sometimes it becomes evident that knowledge is insufficient. The qualitative model provides the initial basis for the understanding of structural uncertainty and key knowledge needs. Qualitative models can be subsequently developed into quantitative models with the accumulation of (expert or empirical) knowledge and data.

Where a qualitative model falls down is in the capacity to be tested or updated with data; how does one accept, reject or modify a model in the light of observations when no quantitative prediction is made.

Models for Structured Decision Making

Increasingly, policy makers and managers are encouraged to adopt structured approaches to addressing environmental problems. An explicit understanding of how ecological systems function (via system or process models) is essential for this for many reasons. Science and system models help to articulate the responses of the ecological objectives of interest, they assist in predicting the possible outcomes of management, and they aid in understanding the uncertainty and knowledge gaps that impede management decisions.

In Structured Decision Making (SDM), modelling has the role of making the predictions that link alternative actions to the desired outcomes (see Decision Point #104), and to identify whether there is critical uncertainty about the system that prevents making obvious decisions. In filling these needs, system models provide the backbone of any analysis of trade-offs and provides a plan for learning through the process of adaptive management (see Decision Point #102).

Armed with clarity on the decision maker’s objectives and the alternatives available, there should be no barrier to developing a fit-for-purpose model (see Decision Point #74). However, often this doesn’t happen. Why is that? In our experience it has a lot to do with the social dynamics involved in building ‘decision support tools’ to support the decision-making process. In what follows we will examine this and discuss some of the triggers underpinning the development of system models.


Key messages:

1. System models can provide explicit representations of how an ecosystem functions. In the context of making decisions, this allows a manager to explore the impacts of management actions on the key values (objectives) at hand.

2. Models are sometimes developed in response to a demand for assistance from managers. Models are also developed by researchers to supply a solution to an environmental problem. Supply- and demand-driven models have different development pathways and these differences help explain some of the successes and frustrations in modelling for environmental problems.

3. Clarity about the context of the problem is critical; elements include values (objectives), performance measures, spatial and temporal scales, and alternative actions. a. In the case of a demand for a system model, it is crucial that decision makers and modellers define these elements of the problem context because they will shape the development and specification of the model. b. From a supply side, judging whether a given model is fit for purpose requires evaluating whether a model aligns with each of these elements of the problem context.

4. All models are ‘wrong’ but some can be useful if they help us improve management and assist us in learning. Understanding the purpose of the model is crucial; at times qualitative models can help creative thinking about how the system works and exploration of key uncertainties; at other times models are required to make quantitative predictions about critical performance measures under alternative management actions and scenarios. Understanding the context is important to managing frustration and providing effective decision support.

5. Models for decision support need to account for uncertainty to allow a decision maker to express their attitude to risk.

6. To be useful in decision support, models need to be understood and accepted by decision makers. This requires good communication about assumptions of structure and function of the models. Development of new models may benefit considerably from decision makers participating in model development.


The demand and supply of system models

In order to gain some understanding of the process of modelling to support decision-making, it’s useful to examine some of the triggers for their development. One obvious trigger is when decision makers ask for a model to be developed to help them choose between available options. For this discussion we call this scenario demand-driven.

Supply-driven modelling for environmental problems also exists. Commonly, models are available to describe ecological systems arising from years of ecological research. And when calls to make decisions for an environmental problem arise, the nearest, or most elaborate, or best-fitted models are proposed as suitable for the problem at hand.

DPoint #106 (Oct 2018) (high res pdf for printing)_Page_15_Image_0001 Alternatively, researchers may become aware of an environmental problem and identify a knowledge gap or technological opportunity for extending model specification. Arising from discussions around policy for environmental problems, a scientist may respond to what they see as failings of existing knowledge or models concerning the problem at hand by developing models to incorporate processes that they identify as being omitted or ignored.

The funding and rewards for academic researchers tend to emphasize the supply-driven mode of knowledge acquisition. Problems can arise with this when applied to environmental decision making. Equally, the development of models for the SDM process in the demand-driven situation brings its own challenges to be resolved for a model to be fit for purpose (see Table 1).

Complex environmental challenges usually take considerable time to resolve. It’s not uncommon for these challenges to result in the development of many models answering different needs as different groups and decision makers become involved. Some of the models developed will likely be demand-driven, meeting the requirements of different groups and different decision makers. Some models will be supply-driven, developed by researchers attempting to understand the nature of the challenge. Such was the case when it was proposed that management should thin degraded woodlands in Victoria to restore some of their natural values. We were involved in the development of some of the models that were used to assist with decision making surrounding this challenge.

The following case study on thinning for conservation demonstrates many of the issues we have discussed here regarding demand- and supply-driven models, and how different managers may have different decision scope and values at hand.

Navigating the development of a fit-for-purpose model: Modelling thinning-for-conservation outcomes

Box-ironbark forests and woodlands in Northern Victoria have been degraded through a history of timber exploitation, gold mining and pastoralism. Many of these forests have a structure characterized by dense stands of small, even-aged stems, few habitat resources for dependent fauna and sparse, low-diversity plant understories. How do you restore some of the natural values to these degraded forests?

Drawing on a history of thinning from production forestry, it was suggested that cutting down some trees in crowded stands might help restore a more natural forest structure. We became involved in thinning for conservation when Parks Victoria was implementing an experimental thinning trial. Parks Victoria wanted to model the consequences of thinning actions on public land for conservation objectives. Our part in this journey started as a small program of research that has continued now for more than a decade.

At the heart of the matter was the question: would thinning these forests result in improvements in the population status of native plants and animals (compared to continuing timber harvest or doing nothing)? Later, we worked with another agency of the Victorian government on a similar question, but arising from private landholders seeking funding through land stewardship schemes (Bush Tender) to conduct thinning activities on private land, notionally for conservation outcomes.

We developed and used several models over this time, including tree diameter growth models, non-spatial and static logic trees of wildlife population viability, expert-parameterized stand dynamics simulators, statistical models of stand density and vegetation condition, and decision trees. Each of these models had slightly different performance variables, at different scales, with different treatment of space and time.

  • The logic trees of population viability concerned single animal species and the probability that a R>1 (positive population growth rate) could be supported on such a stand, given the species habitat requirements for foraging and breeding. These models were demand-driven; they were developed in response to a request from Parks Victoria.
  • The stand dynamics models made predictions through time of proportions of simulated stands with desirable stand structures, principally high vs low density of large stems. These models were supply-driven, being developed to address shortcomings of the former, static models.
  • The statistical models characterized relationships between vegetation condition or quality—the abundance and richness of native and exotic understorey plants—and stand density, and with effects from thinning actions. These were again demand-driven, arising in response to a request for information about private land thinning activities.

Screen Shot 2018-10-05 at 3.12.59 pm

Those various models made different predictions about the likely benefits of thinning actions. The logic trees of population viability and the stand dynamics models both revealed such uncertainty, that no clearly identifiable winning action could be identified. By careful accounting of uncertainty, these illustrated that the decision needed to account for a decision maker’s attitude to risk. If a decision maker was risk-averse, worried about getting fewer large trees 100 years hence, they would be best advised to not conduct any cutting. But if they were aspirational, favouring the best outcome with little regard to how likely it was, a manager would favour ecological thinning.

The statistical models of vegetation condition indicated that thinning could improve some aspects of the native plant diversity, but could also benefit weedy grasses—not a good outcome. They also specified the dependence of the outcome of the land use of the site and implications for seedbanks and soil condition.

How did those models perform or what purpose did they serve? The models served to highlight aspects of the problem: performance variables (stand structure, viable wildlife populations, aspects of vegetation condition) relevant to values; causal structure of the forest ecosystem and management interventions. The stand dynamics models highlighted the long lags in forest responses to intervention and large contribution of disagreement among experts about the fundamental processes driving stand dynamics in these forest and uncertainty about the rates at which some of processes occurred (eg growth rates, probability of cut stems resprouting). By accounting uncertainty and highlighting aspects most critical to resolve, these aided maximizing the chance of desirable outcomes for wildlife and plants (see Decision Point #105).

The statistical models were able to be incorporated into a decision tree to guide how the agency may choose to respond to landholder bids for funds from conservation incentive schemes. It enabled a reflection of the decision maker’s longer-term desire to learn about the outcome of interventions so as to respond to funding bids more effectively in the future.

If you’d like to read more about the challenge of thinning for conservation, see Czembor et al, 2009 and Jones et al, 2015.

On reflection, were the models effective in resolving the problem? It is possible to draw the conclusion that the models are not being used by the responsible agencies to justify a widespread decision on thinning activities, and so they have failed in being useful to the resolution of the environmental problem of high stem density in degraded forests. That is a harsh and rigid evaluation. As laid out early in this article, models play different roles in different contexts.

We were engaged to conduct research and modelling by agency staff at some remove from actual decision makers (ministers and elected representatives or deputy secretary of government agencies). A decision was probably not on the table, rather the research was commissioned to clarify the problem, prompting consideration of the scope of the problems, scales for consideration, necessary values and performance variables.

Did we address each of the potential issues in decision-support tools (Table 1)? No, but by framing and solving ‘toy’ versions or facsimiles of the problem of high stem density and the potential role of thinning, the decision problem becomes better understood. In hindsight there were situations where perhaps more effort to work with our partners to identify the real decision to be made, and the real decision maker, might have helped us address the right objectives, measures, scales etc. Perhaps that is because we were learning as we went, and trying to be play multiple roles: decision analysts and scientists and modellers.

A checklist for an effective model?

How can you be certain you’ve done a good job of developing a model? Basically, if it helps in better understanding how an ecosystem functions and assists in learning then you’re on the right track. Here’s a short checklist of what needs to be considered for an effective model. Of course, there’s a lot more that goes into an effective model but if you can’t check off these five basic points then the long-term value of your effort is far from assured.

  1. Identify the fundamental objective(s) of the decision maker. What do they really care about having more or less of? Which of these are the focus of your model?
  2. Identify some attribute(s) valued by the decision maker that would serve as a performance variable(s).
  3. Ensure your model makes predictions about your performance variable with uncertainty, and include management actions that would enable you to compare the outcome of scenarios and actions on a common, measurable scale. This allows for attitudes to risk to be accommodated
  4. Ensure the spatial and temporal scales are relevant to those of the decision maker, both in terms of specification of performance measures, and the timeframes over which performance is examined; eg, if a manager is focused on national status of a threatened species, the performance measure may examine the potential national distribution of the species, over years to decades to centuries.
  5. Have your model understood and accepted by the relevant decision maker and have it reviewed by a peer. Modelling is difficult and it is easy to make mistakes or to have missed inconsistencies in one’s model.

More info: Peter Vesk pvesk@unimelb.edu.au

References: 

Czembor CA & PA Vesk (2009). Incorporating between-expert uncertainty into state-and-transition simulation models for forest restoration. Forest Ecology and Management 259: 165-175.

Jones CS, DH Duncan, L Rumpff, FM Thomas, WK Morris & PA Vesk (2015). Empirically validating a dense woody regrowth ‘problem’ and thinning ‘solution’ for understory vegetation. Forest Ecology and Management 340: 153-162.

Leave a Reply