The role of ecological understanding, models & model makers in decision making
System models can provide explicit representations of how an ecosystem functions. In the context of making decisions, this allows a manager to explore the impacts of management actions on the key values (objectives) at hand.
Models are sometimes developed in response to a demand for assistance from managers. Models are also developed by researchers to supply a solution to an environmental problem. Supply- and demand-driven models have different development pathways and these differences help explain some of the successes and frustrations in modelling for environmental problems.
Clarity about the context of the problem is critical; elements include values (objectives), performance measures, spatial and temporal scales, and alternative actions. a. In the case of a demand for a system model, it is crucial that decision makers and modellers define these elements of the problem context because they will shape the development and specification of the model. b. From a supply side, judging whether a given model is fit for purpose requires evaluating whether a model aligns with each of these elements of the problem context.
All models are ‘wrong’ but some can be useful if they help us improve management and assist us in learning. Understanding the purpose of the model is crucial; at times qualitative models can help creative thinking about how the system works and exploration of key uncertainties; at other times models are required to make quantitative predictions about critical performance measures under alternative management actions and scenarios. Understanding the context is important to managing frustration and providing effective decision support.
Models for decision support need to account for uncertainty to allow a decision maker to express their attitude to risk.
To be useful in decision support, models need to be understood and accepted by decision makers. This requires good communication about assumptions of structure and function of the models. Development of new models may benefit considerably from decision makers participating in model development.
Models are basic to good decision making. System models are representations of the dynamics of an ecological system, a conceptual map of how the system works. They enable us to specify our thinking on how the system responds to management. Without them in our decision frame it’s unlikely our choices will be well founded. What’s more, and just as important, without a system model the potential to learn is limited.
Of course, a good decision can be made without a formal model. However, it should be recognized that however a decision is made there is always at least a ‘mental model’ guiding the decision maker(s). A mental model is simply someone’s thinking about how something works in the real world. Sometimes it’s referred to as intuitive decision making; in other words, decisions are based on someone’s intuition.
Relying on a mental model can be problematic in that it is not documented, so the underpinning logic can’t be interrogated or transferred, and its compatibility with observed data cannot be tested. In addition, there is a greater risk that the model is myopic, and does not consider the suite of objectives or actions relevant to the context of the decision.
Formal models are documented for all to see. They can be qualitative or quantitative, and both have their uses. A qualitative model helps to clarify thinking, and test logic, coherence and communicability. A qualitative model may provide a starting point to sketch out the dynamics of the system as a way to brainstorm and check that the relevant values, threats and drivers, and associated interventions are captured in the model. Sometimes it becomes evident that knowledge is insufficient. The qualitative model provides the initial basis for understanding of structural uncertainty and key knowledge needs. Qualitative models can be subsequently developed into quantitative models with the accumulation of (expert or empirical) knowledge and data. Where a qualitative model falls down is in the capacity to be tested or updated with data; how does one accept, reject or modify a model in the light of observations when no quantitative prediction is made?
Models for Structured Decision Making
Increasingly, policy makers and managers are encouraged to adopt structured approaches to addressing environmental problems. An explicit understanding of how ecological systems function (via system or process models) is essential for this for many reasons. Science and system models help to articulate the responses of the ecological objectives of interest, they assist in predicting the possible outcomes of management, and they aid in understanding the uncertainty and knowledge gaps that impede management decisions.
In Structured Decision Making (SDM), modelling has the role of making the predictions that link alternative actions to the desired outcomes (see Decision Point #104), and to identify whether there is critical uncertainty about the system that prevents making obvious decisions. In filling these needs, system models provide the backbone of any analysis of tradeoffs and provides a plan for learning through the process of adaptive management (see Decision Point #102).
Armed with clarity on the decision maker’s objectives and the alternatives available, there should be no barrier to developing a fit-for-purpose model (see Decision Point #74). However, often this doesn’t happen. Why is that? In our experience it has a lot to do with the social dynamics involved in building ‘decision support tools’ to support the decision-making process. In what follows we will examine this and discuss some of the triggers underpinning the development of system models.
The demand and supply of system models
In order to gain some understanding of the process of modelling to support decision-making, it’s useful to examine some of the triggers for their development. One obvious trigger is when decision makers ask for a model to be developed to help them choose between available options. For this discussion we call this scenario demand-driven.
Supply-driven modelling for environmental problems also exists. Commonly, models are available to describe ecological systems arising from years of ecological research. And when calls to make decisions for an environmental problem arise, the nearest, or most elaborate, or best-fitted models are proposed as suitable for the problem at hand.
Alternatively, researchers may become aware of an environmental problem and identify a knowledge gap or technological opportunity for extending model specification. Arising from discussions around policy for environmental problems, a scientist may respond to what they see as failings of existing knowledge or models concerning the problem at hand by developing models to incorporate processes that they identify as being omitted or ignored.
The funding and rewards for academic researchers tend to emphasize the supply-driven mode of knowledge acquisition. Problems can arise with this when applied to environmental decision making. Equally, the development of models for the SDM process in the demand-driven situation brings its own challenges to be resolved for a model to be fit for purpose (see Table 1).
Complex environmental challenges usually take considerable time to resolve. It’s not uncommon for these challenges to result in the development of many models answering different needs as different groups and decision makers become involved. Some of the models developed will likely be demand-driven, meeting the requirements of different groups and different decision makers. Some models will be supply-driven, developed by researchers attempting to understand the nature of the challenge. Such was the case when it was proposed that management should thin degraded woodlands in Victoria to restore some of their natural values. We were involved in the development of some of the models that were used to assist with decision making surrounding this challenge.
The following case study on thinning for conservation demonstrates many of the issues we have discussed here regarding demand- and supply-driven models, and how different managers may have different decision scope and values at hand.
Navigating the development of a fit-for-purpose model: Modelling thinning-for-conservation outcomes
Box-ironbark forests and woodlands in Northern Victoria have been degraded through a history of timber exploitation, gold mining and pastoralism. Many of these forests have a structure characterized by dense stands of small, even-aged stems, few habitat resources for dependent fauna and sparse, low-diversity plant understories. How do you restore some of the natural values to these degraded forests?
Drawing on a history of thinning from production forestry, it was suggested that cutting down some trees in crowded stands might help restore a more natural forest structure. We became involved in thinning for conservation when Parks Victoria was implementing an experimental thinning trial. Parks Victoria wanted to model the consequences of thinning actions on public land for conservation objectives. Our part in this journey started as a small program of research that has continued now for more than a decade.
At the heart of the matter was the question: would thinning these forests result in improvements in the population status of native plants and animals (compared to continuing timber harvest or doing nothing)? Later, we worked with another agency of the Victorian government on a similar question, but arising from private landholders seeking funding through land stewardship schemes (Bush Tender) to conduct thinning activities on private land, notionally for conservation outcomes.
We developed and used several models over this time, including tree diameter growth models, non-spatial and static logic trees of wildlife population viability, expert-parameterized stand dynamics simulators, statistical models of stand density and vegetation condition, and decision trees. Each of these models had slightly different performance variables, at different scales, with different treatment of space and time.
- The logic trees of population viability concerned single animal species and the probability that a R>1 (positive population growth rate) could be supported on such a stand, given the species habitat requirements for foraging and breeding. These models were demand-driven; they were developed in response to a request from Parks Victoria. [Terry Walshe developed these models with faunal experts. The log trees utilised combinations of and /or statements to define the necessary and substitutable resources for population growth. These were coupled with expert-specified probabilistic statements of the supply of those habitat attributes in different forest states. The result being probabilistic statements about the likelihood of viable populations of the yellow footed antechinus, diamond firetail, brown treecreeper and sugar glider.]
- The stand dynamics models made predictions through time of proportions of simulated stands with desirable stand structures, principally high vs low density of large stems. These models were supply-driven, being developed to address shortcomings of the former, static models. [These models were developed by Chrissy Czembor and utilized state-and-transition models. Czembor worked with individual experts to define the state and transition agents that could affect the change of the state of the forest from one state to another, at each timestep. These transition agents included: growth, coppice growth, fire, windthrow and management actions including thinning operations. Uncertainty in the models was rigorously accounted via Monte-Carlo sampling routines in the software, eliciting ranges for parameters, and through different experts specifying different parameter values.]
- The statistical models (see the box on ‘to thin or not to thin’) characterized relationships between vegetation condition or quality—the abundance and richness of native and exotic understorey plants—and stand density, and with effects from thinning actions. These were again demand-driven, arising in response to a request for information about private land thinning activities.
Those various models made different predictions about the likely benefits of thinning actions. The logic trees of population viability and the stand dynamics models both revealed such uncertainty, that no clearly identifiable winning action could be identified. By careful accounting of uncertainty, these illustrated that the decision needed to account for a decision maker’s attitude to risk. If a decision maker was risk-averse, worried about getting fewer large trees 100 years hence, they would be best advised to not conduct any cutting. But if they were aspirational, favouring the best outcome with little regard to how likely it was, a manager would favour ecological thinning.
The statistical models of vegetation condition indicated that thinning could improve some aspects of the native plant diversity, but could also benefit weedy grasses—not a good outcome. They also specified the dependence of the outcome of the land use of the site and implications for seedbanks and soil condition.
How did those models perform or what purpose did they serve? The models served to highlight aspects of the problem: performance variables (stand structure, viable wildlife populations, aspects of vegetation condition) relevant to values; causal structure of the forest ecosystem and management interventions. The stand dynamics models highlighted the long lags in forest responses to intervention and large contribution of disagreement among experts about the fundamental processes driving stand dynamics in these forest and uncertainty about the rates at which some of processes occurred (eg growth rates, probability of cut stems resprouting). By accounting uncertainty and highlighting aspects most critical to resolve, these aided maximizing the chance of desirable outcomes for wildlife and plants (see Decision Point #105).
The statistical models were able to be incorporated into a decision tree to guide how the agency may choose to respond to landholder bids for funds from conservation incentive schemes. It enabled a reflection of the decision maker’s longer-term desire to learn about the outcome of interventions so as to respond to funding bids more effectively in the future.
If you’d like to read more about the challenge of thinning for conservation, see Czembor et al, 2009 and Jones et al, 2015 (and see the box on ‘to thin or not to thin’).
On reflection, were the models effective in resolving the problem? It is possible to draw the conclusion that the models are not being used by the responsible agencies to justify a widespread decision on thinning activities, and so they have failed in being useful to the resolution of the environmental problem of high stem density in degraded forests. That is a harsh and rigid evaluation. As laid out early in this article, models play different roles in different contexts.
We were engaged to conduct research and modelling by agency staff at some remove from actual decision makers (ministers and elected representatives or deputy secretary of government agencies). A decision was probably not on the table, rather the research was commissioned to clarify the problem, prompting consideration of the scope of the problems, scales for consideration, necessary values and performance variables.
Did we address each of the potential issues in decision-support tools (Table 1)? No, but by framing and solving ‘toy’ versions or facsimiles of the problem of high stem density and the potential role of thinning, the decision problem becomes better understood. In hindsight there were situations where perhaps more effort to work with our partners to identify the real decision to be made, and the real decision maker, might have helped us address the right objectives, measures, scales etc. Perhaps that is because we were learning as we went, and trying to be play multiple roles: decision analysts and scientists and modellers.
More info: Peter Vesk email@example.com
Czembor CA & PA Vesk (2009). Incorporating between-expert uncertainty into state-and-transition simulation models for forest restoration. Forest Ecology and Management 259: 165-175.
Jones CS, DH Duncan, L Rumpff, FM Thomas, WK Morris & PA Vesk (2015). Empirically validating a dense woody regrowth ‘problem’ and thinning ‘solution’ for understory vegetation. Forest Ecology and Management 340: 153-162. This paper was discussed in Decision Point #86
Hauser CE & MA McCarthy (2009). Streamlining ‘search and destroy’: cost-effective surveillance for pest management. Ecology Letters 12: 683-692. You can read a discussion on this work in Decision Point #31, p4-5.
Williams NSG, AK Hahs & JW Morgan (2008). A dispersal-constrained habitat suitability model for predicting invasion of alpine vegetation. Ecological Applications 18: 347—359.
Rumpff L, DH Duncan, PA Vesk, DA Keith & B Wintle (2010). Stateand-transition modelling for Adaptive Management of native woodlands. Biological Conservation 144: 1224–1236. This story was discussed in Decision Point #47, pages 4-6.
Guillera-Arroita G, JJ Lahoz-Monfort, J Elith, A Gordon, H Kujala, PE Lentini, MA McCarthy, R Tingley & BA Wintle (2015). Is my species distribution model fit for purpose? Matching data and models to applications. Global Ecology and Biogeography 24: 276-292. This paper was discussed in Decision Point #97
A model needs balance
Models often have limited uptake out in the real world because of a lack of technical expertise within an agency, cynicism about the value of models in decision making, and an inappropriate balance between complexity and simplicity of a model. Finding a good balance between simplicity and accuracy in models of ecosystem behaviour is a major challenge. A model which represents a particular system well can result in poor generality, and a model that is too simple can result in a poor representation of fundamental ecosystem dynamics. Care should be taken to restrict the number of variables and states to those which are thought to be most important and relevant to the objective at hand. Uncertainty about system processes and responses to management can make this a difficult step, but building and testing multiple models, in addition to sensitivity analysis, can help obtain the right level of model simplicity.
Models enable structured learning
Despite significant investment and decades of effort to reverse widespread declines in species and habitat in regional Australia, most indicators are suggesting the situation is only growing worse. What’s more, resource management agencies have found it difficult to demonstrate how the investment of public funds has contributed to positive ecological change.
Of course, demonstrating the return on NRM investment is always a major challenge. Management interventions are made over a range of scales in time and space, and the behaviour of the ecological systems we’re dealing with is highly complex and variable. The sin is not that we haven’t tried to manage for positive change; rather it’s that we haven’t effectively learnt from our efforts. Ecologists and practitioners have generally failed to undertake structured learning from our successes and failures. Indeed, much data and priceless experience has essentially disappeared. With growing ecological pressures and increasingly limited resources, surely that’s unacceptable?
So, what do we need to do to effectively learn? What’s required is a management framework with clear objectives, a capacity to assess progress toward those objectives, and the ability to evaluate the extent to which particular management actions contribute to that progress. Adaptive management provides just such a framework and at its centre is a process model of how we think the system works.
A checklist for an effective model?
How can you be certain you’ve done a good job of developing a model? Basically, if it helps in better understanding how an ecosystem functions and assists in learning then you’re on the right track. Here’s a short checklist of what needs to be considered for an effective model. Of course, there’s a lot more that goes into an effective model but if you can’t check off these five basic points then the long term value of your effort is far from assured.
- Identify the fundamental objective(s) of the decision maker. What do they really care about having more or less of? Which of these are the focus of your model?
- Identify some attribute(s) valued by the decision maker that would serve as a direct and sensitive performance variable(s).
- Ensure your model makes predictions about your performance variable with uncertainty, and include management actions that would enable you to compare the outcome of scenarios and actions on a common, measurable scale. Incorporating uncertainty allows for attitudes to risk to be accommodated.
- Ensure the spatial and temporal scales are relevant to those of the decision maker, both in terms of specification of performance measures, and the timeframes over which performance is examined; eg, if a manager is focused on national status of a threatened species, the performance measure may examine the potential national distribution of the species, over years to decades to centuries.
- Ideally, develop your model with the relevant experts and stakeholders (ie, a participatory approach). Have your model understood and accepted by the relevant decision maker and have it reviewed by a peer. Modelling is difficult and it is easy to make mistakes or to have missed inconsistencies in one’s model.
The process in making adaptive management meaningful
Using process models to guide investment in the management of native vegetation
What about models for vegetation change at the scale of the landscape? NRM agencies often have objectives of increasing the extent of native vegetation at a landscape scale. One Catchment Management Authority in Victoria had an objective to increase the extent and condition of native vegetation across the catchment (a region of around 5,000 km2). What is an appropriate system model for this? (Rumpff et al, 2010).
Vegetation condition may be assessed with multiple measures of abundance and diversity of structural components of woodland vegetation (eg, numbers of large trees, shrub cover, native species understorey, and the like) though not many people would naturally think in terms of those multiple attributes at once. However, state and transition models* (STMs) have been an important tool for managers to represent the possible changes that might occur within a landscape, and think about possible transitions between the states in which it can exist (Figure 1 shows a process diagram of how the STM is developed. Figure 2 shows the resulting conceptual state-and-transition model for non-riparian woodlands). We reasoned that this STM framework was useful for communicating (or reporting) the state of the system. But underlying this we utilized the multiple attributes within a Bayesian network.
A great strength of this approach was that the models could be initially specified from the scientific literature and expert information. The models could then be updated with observational data. Bayesian updating is a great match to an adaptive management program: the initial parameterization of the model represents existing hypotheses about how the system works, with uncertainty represented and probability distributions. As data from experiments and monitoring are accumulated, the probability distributions are changed and thereby the model changes. This can allow evaluations of outcomes under different cases or situations or actions.
The model was non-spatial, it included multiple attributes that comprised elements of a complex multiple objective. It included management actions that might be employed by landholders and managers.
After just one monitoring/learning cycle, seven years after the first investments, we found that updated models predict markedly different transition probabilities compared with initial models based on expert opinion. This has strong implications for the cost-efficiency of restoration strategies. The STM approach provides a sound theoretical basis for restoration decisions, while the Bayesian network implementation provides a workable framework for using the STM adaptively.
*A modelling approach which is becoming increasingly common in native vegetation management are State-and-transition models (STMs). These models describe different states of vegetation condition that may occur, and the possible transitions that may occur between these states. They are popular because there is recognition that there are different states of vegetation condition in the landscape with different land-use histories, multiple non– linear pathways of change between states, threshold behaviour, and possible barriers to restoration. Their adoption signalled a move away from the traditional views of Clementsian succession, whereby an initial management intervention (such as fencing or tree planting) will theoretically result in a single linear successional trajectory towards a ‘climax’ or reference vegetation community.
Hunting down hawkweed
A model for alpine success
Effectively detecting an invasive species during its earliest phase of introduction can save a lot of future damage and expense. Yet this is also when the species is most difficult to detect, lurking at a low density in a large landscape. So, how much should we be willing to spend searching for these ‘needles in haystacks’? Decision researchers Cindy Hauser and Michael McCarthy developed a pest surveillance model that shows how we can best design searches that keep costs down (Hauser and McCarthy 2009).
Some of the factors their model needed to deal with include:
Probability of occurrence: Some locations will be more likely to harbour the pest than others. Habitat suitability models can map where the pest is most likely to find a happy home, and can incorporate dispersal to predict where the pest is most likely to be at a particular time.
Ease of detection: Some habitats are easier to search than others. We need to understand how a habitat’s terrain and the search effort employed affect our chances of successfully detecting the pest.
Benefits of detection: What will be gained if we find and treat the pest now rather than later? Economic, biodiversity and other values may indicate that some areas are more important for protection from the pest than others.
They tested their model on how it might help eradicate orange hawkweed (Hieracium aurantiacum) an invasive daisy which poses a major threat to Australian grasslands and temperate areas.
Researchers have previously predicted the occurrence of orange hawkweed on the Bogong High Plains in Victoria using measures of vegetation, wetness, disturbance and wind dispersal (Map 1). These predictions are vital for prioritising surveillance efforts.
To use the model for surveillance planning approach, we also need information about detection. Researcher Nicholas Williams split the landscape into ‘easy to search’ and ‘difficult to search’ vegetation categories and estimated detection rates based on his searching experience (Map 2; Williams, Hahs and Morgan 2008).
Hauser and McCarthy then conducted ‘detection experiments’ with hidden hawkweeds and volunteer searchers, allowing for more rigorous modelling. Their method indicates that effort should be concentrated in a small proportion of sites with a high probability of orange hawkweed occurrence, though it also depends on the vegetation type (Map 3).
It is worthwhile visiting grassy sites, at least briefly, even when the probability of presence is quite small because they are easy to search. However, difficult-to-search shrubby sites tend to be surveyed either thoroughly or not at all. Their inclusion in the surveillance plan requires a high probability of pest presence and large detection benefits.
To thin or not to thin
Understanding dense eucalypt stands and the pros and cons of thinning
In Victoria, there is an increasing call for management of dense eucalypt stands on both private and public land. The most commonly cited management option is thinning – cutting down a proportion of stems and applying herbicide to prevent regrowth. The theory is that the release from competition should make the remaining stems grow faster, larger, and broader, as well allowing the recovery of understorey vegetation.
The questions for managers then are: How bad is a dense stand for biodiversity and what is the benefit of thinning? But perhaps more importantly, we need to ask whether thickets pose a problem that warrants major investment from government. At what scale would thickets need to be a dominant form to cause concern for those species, and communities, and at what scale is the treatment cost effective?
Chris Jones and colleagues sought answers to these questions using data from two separate field projects conducted in box-ironbark woodlands and forests in central Victoria, where they evaluated the vegetation structure of dense regrowth stands of eucalypts, and the effect of thinning management (Jones et al, 2015). In order to determine what density of stems and cover of understorey ‘should’ be expected in natural systems, they evaluated their results in relation to published benchmarks of stem density and understorey vegetation cover.
They found that stands with stem density greater than benchmark levels suppress native understorey vegetation cover below its benchmark levels. Thinning stems can restore native understorey vegetation (richness and cover) in the short term, providing the soil seedbank has not been removed and there is no excessive grazing. This is the desired outcome from thinning, but the catch is that BOTH native and exotic species can recover following thinning.
Is my model fit for purpose?
Matching data and species distribution models to applications
Species distribution models (SDMs) are becoming a fundamental tool in environmental decision making. For instance, SDMs are used to identify areas suitable for reintroduction of threatened species, sites at risk of biological invasions or to direct the search for new populations of species.
There are many considerations involved in building useful correlative SDMs. For an SDM to have good predictive ability we need to identify critical environmental predictors. For example, do average temperature, average rainfall and soil pH accurately capture why this plant species happens here and not there?
Defining a suitable extent for the model is also fundamental. Am I interested in describing the habitat preferences for this mammal species at a continental scale, or do I want to understand its preferences at a local scale? There is a lot written about these and other aspects of building SDMs (and Brendan Wintle has developed an excellent checklist of the basics in Decision Point #67.
An important consideration is that the type of data available for a species affects the interpretation and reliability of SDM outputs? This is a critical aspect in the practice of species distribution modelling yet it’s often overlooked. Users often underestimate the strong links between data type, model output and suitability for end-use. Species distribution models can lead to suboptimal conservation outcomes and misguided theory if the underlying data are not suited to the intended application.