Quantitative models are an important part of environmental and economic research and policymaking. For instance, IIASA models such as GLOBIOM and GAINS have long assisted the European Commission in impact assessment and policy analysis2; and the energy policies in the US have long been guided by a national energy systems model (NEMS)3.
Despite such successful modelling applications, model criticisms often make the headlines. Either in scientific literature or in popular media, some critiques highlight that models are used as if they are precise predictors and that they don’t deal with uncertainties adequately4,5,6, whereas others accuse models of not accurately replicating reality7. Still more criticize models for extrapolating historical data as if it is a good estimate of the future8, and for their limited scopes that omit relevant and important processes9,10.
Validation is the modeling step employed to deal with such criticism and to ensure that a model is credible. However, validation means different things in different modelling fields, to different practitioners and to different decision makers. Some consider validity as an accurate representation of reality, based either on the processes included in the model scope or on the match between the model output and empirical data. According to others, an accurate representation is impossible; therefore, a model’s validity depends on how useful it is to understand the complexity and to test different assumptions.
Given this variety of views, we conducted a text-mining analysis on a large body of academic literature to understand the prevalent views and approaches in the model validation practice. We then complemented this analysis with an online survey among modeling practitioners. The purpose of the survey was to investigate the practitioners’ perspectives, and how it depends on background factors.
According to our results, published recently in Eker et al. (2018)1, data and prediction are the most prevalent themes in the model validation literature in all main areas of sustainability science such as energy, hydrology and ecosystems. As Figure 1 below shows, the largest fraction of practitioners (41%) think that a match between the past data and model output is a strong indicator of a model’s predictive power (Question 3). Around one third of the respondents disagree that a model is valid if it replicates the past since multiple models can achieve this, while another one third agree (Question 4). A large majority (69%) disagrees with Question 5, that models cannot provide accurate projects, implying that they support using models for prediction purposes. Overall, there is no strong consensus among the practitioners about the role of historical data in model validation. Still, objections to relying on data-oriented validation have not been widely reflected in practice.
Figure 1: Survey responses to the key issues in model validation. Source: Eker et al. (2018)
According to most practitioners who participated in the survey, decision-makers find a model credible if it replicates the historical data (Question 6), and if the assumptions and uncertainties are communicated clearly (Question 8). Therefore, practitioners think that decision makers demand that models match historical data. They also acknowledge the calls for a clear communication of uncertainties and assumptions, which is increasingly considered as best-practice in modeling.
One intriguing finding is that the acknowledgement of uncertainties and assumptions depends on experience level. The practitioners with a very low experience level (0-2 years) or with very long experience (more than 10 years) tend to agree more with the importance of clarifying uncertainties and assumptions. Could it be because a longer engagement in modeling and a longer interaction with decision makers help to acknowledge the necessity of communicating uncertainties and assumptions? Would inexperienced modelers favor uncertainty communication due to their fresh training on the best-practice and their understanding of the methods to deal with uncertainty? Would the employment conditions of modelers play a role in this finding?
As a modeler by myself, I am surprised by the variety of views on validation and their differences from my prior view. With such findings and questions raised, I think this paper can provide model developers and users with reflections on and insights into their practice. It can also facilitate communication in the interface between modelling and decision-making, so that the two parties can elaborate on what makes their models valid and how it can contribute to decision-making.
Model validation is a heated topic that would inevitably stay discordant. Still, one consensus to reach is that a model is a representation of reality, not the reality itself, just like the disclaimer of René Magritte that his perfectly curved and brightly polished pipe is not a pipe.
Eker S, Rovenskaya E, Obersteiner M, Langan S. Practice and perspectives in the validation of resource management models. Nature Communications 2018, 9(1): 5359. DOI: 10.1038/s41467-018-07811-9 [pure.iiasa.ac.at/id/eprint/15646/]
Nuccitelli D. Climate scientists just debunked deniers’ favorite argument. The Guardian. 2017. https://www.theguardian.com/environment/climate-consensus-97-per-cent/2017/jun/28/climate-scientists-just-debunked-deniers-favorite-argument
By Sandra Ortellado, IIASA 2018 Science Communication Fellow
If fashion is the science of appearances, what can beauty and aesthetics tell us about the way we perceive the world, and how it influences us in turn?
From cognitive science research, we know that aesthetics not only influence superficial appearances, but also the deeper ways we think and experience. So, too, do all kinds of creative thinking create change in the same way: as our perceptions of the world around us changes, the world we create changes with them.
From the merchandizing shelves of H&M and Vero Moda to doctoral research at the Faculty of Information Technology at the University of Jyväskylä, Finland, 2018 YSSP participant Laura Mononen has seen product delivery from all angles. Whether dealing with commercialized goods or intellectual knowledge, Mononen knows that creativity is all about a change in thinking, and changing thinking is all about product delivery.
“During my career in the fashion and clothing industry, I saw the different levels of production when we sent designs to factories, received clothing back, and then persuaded customers to buy them. It was all happening very effectively,” says Mononen.
But Mononen saw potential for product delivery beyond selling people things they don’t need. She wanted to transfer the efficiency of the fashion world in creating changes in thinking to the efforts to build a sustainable world.
“Entrepreneurs make change with products and companies, fashion change trends and sell them. I’m really interested in applying this kind of change to science policy and communication,” says Mononen. “We treat these fields as though they are completely different, but the thing that is common is humans and their thinking and behaving.”
Often, change must happen in our thinking first before we can act. That’s why Mononen is getting her doctorate in cognitive science. Her YSSP project involved heavy analysis of systems theories of creativity to find patterns in the way we think about creativity, which has been constantly changing over time.
In the past, creativity was seen as an ability that was characteristic of only certain very gifted individuals. The research focused on traits and psychological factors. Today, the thinking on creativity has shifted towards a more holistic view, incorporating interactions and relationships between larger systems. Instead of being viewed as a lightning bolt of inspiration, creativity is now seen as more of a gradual process.
New understandings of creativity also call on us to embrace paradoxes and chaos, see ourselves as part of nature rather than separate from it, experience the world through aesthetics, pay careful attention to our perception and how we communicate it, and transmit culture to the next generation.
Perhaps most importantly, Mononen found in her research that the understanding of creativity has changed to be seen as part of a process of self-creation as well as co-creation.
“The way we see creativity also influences ourselves. For example if I ask someone if they are creative, it’s the way they see themselves that influences how creative they are,” says Mononen. “I have found that it’s more crucial to us than I thought, creativity is everywhere and it’s everyday and we are sharing our creativity with others who are using that to do something themselves and so on.”
This means on the one hand that we use our creativity to decide who we are and how we see the world around us for ourselves. But it also means that the outcomes and benefits of creativity are now intended for society as a whole rather than purely for individuals, as it was in the past. It may sound like another paradox, but being able to embrace ambiguity and complexity and take charge of our role in a larger system is important for creating a sustainable future.
“From the IIASA perspective this finding brings hope because the more people see themselves as part of systems of creating things, the more we can encourage sustainable thinking, since nature is a part of the resources we use to create,” says Mononen.
Mononen says a systems understanding of creativity is especially important for people in leadership positions. If a large institution needs new and innovative solutions and technology, but doesn’t have the thinking that values and promotes creativity, then the cooperative, open-minded process of building is stifled.
Working in both the fashion industry and academic research, Mononen has encountered narrow-minded attitudes towards art and science firsthand.
“Communicating your research is very difficult coming from my background, because you don’t know how the other person is interpreting what you say,” says Mononen. “People have different ideas of what fashion and aesthetics are, how important they are and what they do. Additionally, scientific concepts are used differently in different fields.”
“We are often thinking that once we get information out there, then people will understand, but there are much more complex things going on to make change and create influence in settings that combine several different fields.” says Mononen.
For Mononen, the biggest lesson is that creativity can enhance the efforts of science towards a sustainable world simply by encouraging us to be aware of our own thinking, how it differs from that of others, and how it affects all of us.
“When you become more aware of your ways of thinking, you become more effective at communicating,” says Mononen. “It’s not always that way and it’s very challenging, but that’s what the research on creativity from a systems perspective is saying.”
By Marcus Thomson, researcher, IIASA Ecosystems Services and Management Program
While living in Cairo in 2010, I witnessed first-hand the human toll of political and environmental disasters that washed over Africa at the end of the last century. Unprecedented numbers of migrants were pressing into North Africa, many pushed out of their homelands by conflict and state-failure, pulled towards safer, richer, less fragile places like Europe. Throughout Sub-Saharan Africa, climate change was driving up competition for scarce land and water, and raising pressure on farmers to maintain the quantity and quality of their crops.
It is a similar story throughout the developing world, where many farmers do without the use of expensive chemical fertilizer and pesticides, complex irrigation, or boutique seed varieties. They rely instead on traditional land management practices that developed over long periods with consistent, predictable conditions. It is difficult to predict how dryland farmers will respond to climate change; so it is challenging to plan for various social, economic, and political problems expected to develop under, or be exacerbated by, climate change. Will it spur innovation or, as has been argued for the Syrian civil war, set up conflict? A major stumbling block is that the dynamics of human social behavior are so difficult to model.
Instead of attempting to predict farmers’ responses to climate change by modelling human behavior, we can look to the responses to environmental changes of farmers from the past as analogues for many subsistence farmers of the future. Methods to fill in historical gaps, and reconstruct the prehistoric record, are valuable because they expand the set of observed cases of societal-scale responses to environmental change. For instance, some 2000 years ago, an expansive maize-growing cultural complex, the Ancestral Puebloans (APs), was well established in the arid American Southwest. By AD 1000, members of this AP complex produced unique and innovative material culture including the famed “Great Houses”, the largest built structures in the United States until the 19th century. However, between AD 1150 and 1350, there was a profound demographic transformation throughout the Southwest linked to climate change. We now know that many APs migrated elsewhere. As a PhD student at the University of California, Los Angeles, I wondered whether a shift to cooler, more variable conditions of the “Little Ice Age” (LIA, roughly AD 1300 to 1850) was linked to the production of their staple crop, maize.
I came to IIASA as a YSSP in 2016 to collaborate with crop modelers on this question, and our work has just been published in the journal Quaternary International. I brought with me high-resolution data from a state-of-the-art climate model to drive the crop simulations, and AP site information collected by archaeologists. Because AP maize was quite different from modern corn, I worked with IIASA soil scientist Juraj Balkovič to modify the crop simulator with parameters derived from heirloom varieties still grown by indigenous peoples in the Southwest. I and IIASA economic geographer Tamás Krisztin developed a statistical technique to analyze the dynamical relationship between AP site occupation and simulated yield outcomes.
We found that for the most climate-stressed high-elevation sites, abandonments were most associated with increased year-to-year yield variability; and for the least stressed low-elevation and well-watered sites, abandonment was more likely due to endogenous stressors, such as soil degradation and population pressure. Crucially, we found that across all regions, populations peaked during periods of the most stable year-to-year crop yields, even though these were also relatively warm and dry periods. In short, we found that AP maize farmers adapted well to gradually rising temperatures and drought, during the MCA, but failed to adapt to increased climate variability after ~AD 1150, during the LIA. Because increased variability is one of the near certainties for dryland farming zones under global warming, the AP experience offers a cautionary example of the limits of low-technology adaptation to climate change, a business-as-usual direction for many sub-Saharan dryland farmers.
This is a lesson from the past that policymakers might take note of.
 Kelley, C. P., Mohtadi, S., Cane, M. A., Seager, R., & Kushnir, Y. (2015). Climate change in the Fertile Crescent and implications of the recent Syrian drought. Proceedings of the National Academy of Sciences, 201421533.
 Thomson, M. J., Balkovič, J., Krisztin, T., MacDonald, G. M. (2018). Simulated crop yield for Zea mays for Fremont Ancestral Puebloan sites in Utah between 850-1499 CE based on temperature dailies from a statistically downscaled climate model. Quaternary International. https://doi.org/10.1016/j.quaint.2018.09.031
Brian, now 71, is one of the most influential early thinkers of the SFI, a place that without exaggeration could be called the cradle of complexity science.
Brian became famous with his theory of increasing returns. An idea that has been developed in Vienna, by the way, where Brian was part of a theoretical group at the IIASA in the early days of his career: from 1978 to 1982.
“I was very lucky,” he recalls. “I was allowed to work on what I wanted, so I worked on increasing returns.”
The paper he wrote at that time introduced the concept of positive feedbacks into economy.
The concept of “increasing returns”
Increasing returns are the tendency for that which is ahead to get further ahead, for that which loses advantage to lose further advantage. They are mechanisms of positive feedback that operate—within markets, businesses, and industries—to reinforce that which gains success or aggravate that which suffers loss. Increasing returns generate not equilibrium but instability: If a product or a company or a technology—one of many competing in a market—gets ahead by chance or clever strategy, increasing returns can magnify this advantage, and the product or company or technology can go on to lock in the market.”
(W Brian Arthur, Harvard Business Review 1996)
This was a slap in the face of orthodox theories which saw–and some still see–economy in a state of equilibrium. “Kind of like a spiders web,” Brian explains me in our short conversation last Friday, “each part of the economy holding the others in an equalization of forces.”
The answer to heresy in science is that it does not get published. Brian’s article was turned down for six years. Today it counts more than 10.000 citations.
At the latest it was the development and triumphant advance of Silicon Valley’s tech firms that proved the concept true. “In fact, that’s now the way how Silicon Valley runs,” Brian says.
The youngest man on a Stanford chair
William Brian Arthur is Irish. He was born and raised in Belfast and first studied in England. But soon he moved to the US. After the PhD and his five years in Vienna he returned to California where he became the youngest chair holder in Stanford with 37 years.
Five years later he changed again – to Santa Fe, to an institute that had been set up around 1983 but had been quite quiet so far.
Q: From one of the most prestigious universities in the world to an unknown little place in the desert. Why did you do that?
A: In 1987 Kenneth Arrow, an economics Nobel Prize winner and mentor of mine, said to me at Stanford: We’re holding a small conference in September in a place in the Rockies, in Santa Fe, would you go?
When a Nobel Prize winner asks you such a question, you say yes of course. So I went to Santa Fe.
We were about ten scientists and ten economists at that conference, all chosen by Nobel Prize winners. We talked about the economy as an evolving complex system.
Veni, vidi, vici
Brian came – and stayed: The unorthodox ideas discussed at the meeting and the “wild” and free atmosphere of thinking at “the Institute”, as he calls the Santa Fe Institute (SFI), thrilled him right away.
In 1988 Brian dared to leave Stanford and started to set up the first research program at Santa Fe. Subject was the economy treated as a complex system.
Q: What was so special about SF?
A: The idea of complexity was quite new at that time. But people began to see certain patterns in all sorts of fields, whether it was chemistry or the economy or parts of physics, that interacting elements would together create these patterns…To investigate this in universities with their particular disciplines, with their fixed theories, fixed orthodoxies–where it is all fixed how to do things–turned out to be difficult.
Take the economy for example. Until then people thought it was in an equilibrium. And there we came and proved, no, economics is no equilibrium! The Stanford department would immediately say: You can’t do that! Don’t do that! Or they would consider you to be very eccentric…
So a bunch of senior fellows at Los Alamos in the 1980s thought it would be a good idea if there was an independent institute to research these common questions that came to be called complexity.
At Santa Fe you could talk about any science and any basic assumptions you wanted without anybody saying you couldn’t or shouldn’t do that.
Our group as the first there set a lot of this wild style of research. There were lots of discussions, lots of open questions, without particular disciplines… In the beginning there were no students, there was no teaching. It was all very free.
This wild style became more or less the pattern that has been followed ever since. I think the Hub is following this model too.
The magic formula for excellence
Q: Was this just a lucky concurrence: the right people and atmosphere at the right time? Or is there a pattern behind it that possibly could be repeated?
A: I am sure: If you want to do interdisciplinary science – which complexity is: It is a different way of looking at things! – you need an atmosphere where people aren’t reinforced into all the assumptions of the different disciplines.
This freedom is crucial to excellent science altogether. It worked out not only for Santa Fe. Take the Rand Corporation for instance, that invented a lot of things including the architecture of the internet, or the Bell Labs in the Fifties that invented the transistor. The Cavendish Lab in Cambridge is another one, with the DNA or nuclear astronomy…
The magic formula seems to be this:
First get some first rate people. It must be absolutely top-notch people, maybe ten or twenty of them.
Make sure they interact a lot.
Allow them to do what they want – be confident that they will do something important.
And then when you protect them and see that they are well funded, you are off and running.
Probably in seven cases out of ten that will not produce much. But quite a few times you will get something spectacular – game changing things like quantum theory or the internet.
Don’t choose programs, choose people
Q: This does not seem to be the way officials are funding science…
A: Yes, in many places you have officials telling people what they need to research. Or where people insist on performance and indices… especially in Europe, I have the impression, you have a tradition of funding science by insisting on all these things like indices and performance and publications or citation numbers. But that’s not a very good formula.
Excellence is not measurable by performance indicators. In fact that’s the opposite of doing science.
I notice at places where everybody emphasize all this they are not on the forefront. Maybe it works for standard science; and to get out the really bad science. But it doesn’t work if you want to push boundaries.
Many officials don’t understand that.
In Singapore the authorities once asked me: How did you decide on the research projects in Santa Fe? I said, I didn’t decide on the research projects. They repeated their question. I said again, I did not decide on the research projects. I only decided on people. I got absolutely first rate people, we discussed vaguely the direction we wanted things to be in, and they decided on their research projects.
That answer did not compute with them. They are the civil service, they are extraordinarily bright, they’ve got a lot of money. So they think they should decide what needs to be researched.
I should have told them – I regret I didn’t: This is fine if you want to find solutions for certain things, like getting the traffic running or fixing the health care system. Surely with taxpayer’s money you have to figure such things out. But you will never get great science with that. All you get is mediocrity.
Of course now they asked, how do we decide which people should be funded? And I said: “You don’t! Just allow top people to bring in top people. Give them funding and the task of being daring.”
Any other way of managing top science doesn’t seem to work.
I think the Hub could be such a place – all the ingredients are here. Just make sure to attract some more absolutely first rate people. If they are well funded the Hub will put itself on the map very quickly.
Interview with Naoko Ishii, CEO and Chairperson of the Global Environment Facility (GEF), an independent organization that provides grants for projects working towards sustainability. IIASA, the GEF, and the United Nations Industrial Development Organization (UNIDO) have recently partnered on a new project to explore integrated solutions for water, energy, and land.
Q What is sustainable development and why is it important? A As Brundtland put it, sustainable development meets the needs of the present without compromising the ability of future generations to meet their own needs.
If we do not achieve sustainable development, we will fail to provide even the barest essentials of life—food, water, and shelter—for the growing population. The extra two billion people that will inhabit the world in 2050 can only be accommodated if we are serious about sustainable development.
On a personal level I care about sustainable development because I care about the future, I care about young people, and I care about humanity. Achieving sustainable development is, in my opinion, the single most important issue we face today. Without it, all life on Earth is in jeopardy.
The Global Environment Facility (GEF) was created on the eve of the 1992 Earth Summit in Rio to assist in the protection of the global environment and promote sustainable development. The benefits of such an endeavor have only become clearer over time. It is no coincidence that in 2015 all nations of the world will adopt a set of sustainable development goals which place a strong emphasis on the “global commons,” and that in parallel we have a new global agreement on climate change within reach.
How do you see the world in 2050? What are your most optimistic and pessimistic visions? I am an optimistic person so I will say that, by 2050, every government, every business, and every individual will take the environment into consideration in all their actions. By 2050, we will all be caring for the Earth, taking responsibility for the use of our planet’s resources, and building economies which will leave no one without dignity or necessary subsistence. We will live within safe planetary boundaries. Pessimism is not an option for me.
How can science help the world achieve sustainable development? Science plays a critical role. We need it to monitor the state of our resources, the impacts of our activities, and the progress being made. Science can also help identify solutions. It can help encourage businesses to make smart decisions, for example, about saving money though energy efficiency, risk mitigation, and new revenue opportunities driven by innovation and new business models.
Sustainable development is a truly cross-cutting endeavor: it spans many sectors, from agriculture to economics, and transcends national boundaries. Science can play an important role by producing research that is integrated, cross-sectoral and international. In this way, synergies, co-benefits, and trade-offs can be explored in order to identify the smartest paths to achieving multiple sustainable development goals at the same time
How do you see the role of Global Environment Facility in implementing the Sustainable Development Goals? The GEF is uniquely placed to support the global commons—the planet’s finite environmental resources that provide the stable conditions required for a sustainable, prosperous future for all. Our new strategy—GEF2020—lays out an ambitious vision for the GEF, aimed at addressing the underlying drivers of environmental degradation and delivering integrated, holistic, solutions. We are building on more than 20 years of experience providing support to over 165 countries. By working with national governments, local communities, the private sector, civil society organizations and indigenous peoples, we help find and implement integrated solutions to global challenges.
What are the advantages of a cross-sectoral and cross-border approach to identifying paths to sustainable development? Many environmental challenges and threats to sustainable development do not respect borders. Moreover, they are often interdependent, or share common drivers. For example, biodiversity loss and climate change is partly driven by unsustainable forest management, which is in turn connected to production of globally traded commodities like palm oil or soy. Problems like this require an integrated, cross-cutting approach.
Given the importance of cross-sectoral interventions, at the GEF we will be implementing a program of integrated approach pilot projects. We believe that these will help countries and the global community in tackling underlying drivers of environmental degradation. I am also very excited about a research program we have recently launched in partnership with IIASA and the United Nations Industrial Development Organization, focusing on development and implementation of integrated solutions to tackle the water-food-energy nexus.
Note: This article gives the views of the interviewee, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.