Mathematical models of infectious disease outbreaks are too often developed and presented with insufficient attention to the uncertainties that underlie them, say British researchers who carefully examined how uncertainties were addressed in a number of outbreak models.
The authors, from the University of Liverpool and Lancaster University, concluded that "greater consideration of the limitations and uncertainties in infectious disease modelling would improve its usefulness and value," says a press release yesterday from the Liverpool school. Their report was published last week in PLoS One.
Disease models are often used, as the press release notes, to try to predict how outbreaks will progress and to influence response policies. For example, modeling has been used for diseases such as SARS (severe acute respiratory syndrome), the 2009 H1N1 influenza pandemic, and foot-and-mouth disease (FMD), and it is being used in addressing bovine tuberculosis.
The researchers interviewed 24 scientists, including modelers, infectious disease biologists, public health and veterinary experts, as well as policy makers and other users of model results. They also studied policy documents, scientific reports, textbooks, and reports of inquiries into major animal disease outbreaks.
Although it was unintentional, the researchers say their findings were largely shaped by models of FMD, especially those related to the devastating UK outbreak of 2001, in which modeling "had a central and controversial role." FMD is a highly contagious disease of cattle.
Uncertainty is well recognized in modeling, as evidenced by the familiar mantra "All models are wrong, but some models are useful," the authors observe. The saying is found in disease-modeling textbooks and in statements by policy makers.
Emphasizing the central role of uncertainty, they comment, "The proliferation of potential sources of uncertainty that arise" as models are developed and refined "makes enumeration of all of these uncertainties an impossibility, let alone full consideration of all their implications for the veracity of the model conclusions."
"It is accepted that models will never be able to predict 100% the size, shape or form of an outbreak and it is recognised that a level of uncertainty always exists in modeling," lead author Robert M. Christley, BVSc, PhD, of the University of Liverpool commented in the press release.
"However, modellers often fear detailed discussion of this uncertainty will undermine the model in the eyes of policy makers. This study found that the uncertainties and limitations of a model are sometimes hidden and sometimes revealed, and that which occurs is context dependent.
"Whilst it isn't possible to calculate the level of uncertainty, a better understanding and communication of the model's limitations is needed and could lead to better policy."
One general finding was that modelers, when communicating their findings to users, described a tension between "being open regarding issues of uncertainty and the need to protect the authority of modeling by minimizing the impact of that uncertainty," according to the report.
Another conclusion was that model-users felt ill equipped to judge a model on its technical merits and relied on the modelers themselves to interpret model uncertainties, the report says. This puts modelers in a strong negotiating position, which users typically counter, if they feel the need, by referring to external factors or sensitivities.
In summing up, the authors say that the variety of ways in which uncertainty is communicated and presented in disease models renders it a "plastic quality" that is interpreted and addressed in different ways. This "flexibility" is among the factors that determine the usefulness of models.
The researchers conclude that their aim is not to condemn the use of disease modeling in decision-making. Rather, they agree with certain other analysts that "domains in which ambiguity, uncertainty and ignorance are key features, such as is the case for dynamical modelling of emerging novel epidemics, are better addressed by plural, conditional and nuanced advice, of which infectious disease dynamical modelling may form a (modestly) useful part."
Christley RM, Mort M, Wynne B, et al. 'Wrong, but useful': negotiating uncertainty in infectious disease modeling. PLoS One 2013; online publication Oct 16 [Full text]
See also:
Oct 21 University of Liverpool press release
