Jan 24, 2019 | Data and Methods, Systems Analysis, Women in Science
By Sibel Eker, IIASA postdoctoral research scholar
Ceci n’est pas une pipe – This is not a pipe © Jaka Vukotič | Dreamstime.com
Quantitative models are an important part of environmental and economic research and policymaking. For instance, IIASA models such as GLOBIOM and GAINS have long assisted the European Commission in impact assessment and policy analysis2; and the energy policies in the US have long been guided by a national energy systems model (NEMS)3.
Despite such successful modelling applications, model criticisms often make the headlines. Either in scientific literature or in popular media, some critiques highlight that models are used as if they are precise predictors and that they don’t deal with uncertainties adequately4,5,6, whereas others accuse models of not accurately replicating reality7. Still more criticize models for extrapolating historical data as if it is a good estimate of the future8, and for their limited scopes that omit relevant and important processes9,10.
Validation is the modeling step employed to deal with such criticism and to ensure that a model is credible. However, validation means different things in different modelling fields, to different practitioners and to different decision makers. Some consider validity as an accurate representation of reality, based either on the processes included in the model scope or on the match between the model output and empirical data. According to others, an accurate representation is impossible; therefore, a model’s validity depends on how useful it is to understand the complexity and to test different assumptions.
Given this variety of views, we conducted a text-mining analysis on a large body of academic literature to understand the prevalent views and approaches in the model validation practice. We then complemented this analysis with an online survey among modeling practitioners. The purpose of the survey was to investigate the practitioners’ perspectives, and how it depends on background factors.
According to our results, published recently in Eker et al. (2018)1, data and prediction are the most prevalent themes in the model validation literature in all main areas of sustainability science such as energy, hydrology and ecosystems. As Figure 1 below shows, the largest fraction of practitioners (41%) think that a match between the past data and model output is a strong indicator of a model’s predictive power (Question 3). Around one third of the respondents disagree that a model is valid if it replicates the past since multiple models can achieve this, while another one third agree (Question 4). A large majority (69%) disagrees with Question 5, that models cannot provide accurate projects, implying that they support using models for prediction purposes. Overall, there is no strong consensus among the practitioners about the role of historical data in model validation. Still, objections to relying on data-oriented validation have not been widely reflected in practice.
Figure 1: Survey responses to the key issues in model validation. Source: Eker et al. (2018)
According to most practitioners who participated in the survey, decision-makers find a model credible if it replicates the historical data (Question 6), and if the assumptions and uncertainties are communicated clearly (Question 8). Therefore, practitioners think that decision makers demand that models match historical data. They also acknowledge the calls for a clear communication of uncertainties and assumptions, which is increasingly considered as best-practice in modeling.
One intriguing finding is that the acknowledgement of uncertainties and assumptions depends on experience level. The practitioners with a very low experience level (0-2 years) or with very long experience (more than 10 years) tend to agree more with the importance of clarifying uncertainties and assumptions. Could it be because a longer engagement in modeling and a longer interaction with decision makers help to acknowledge the necessity of communicating uncertainties and assumptions? Would inexperienced modelers favor uncertainty communication due to their fresh training on the best-practice and their understanding of the methods to deal with uncertainty? Would the employment conditions of modelers play a role in this finding?
As a modeler by myself, I am surprised by the variety of views on validation and their differences from my prior view. With such findings and questions raised, I think this paper can provide model developers and users with reflections on and insights into their practice. It can also facilitate communication in the interface between modelling and decision-making, so that the two parties can elaborate on what makes their models valid and how it can contribute to decision-making.
Model validation is a heated topic that would inevitably stay discordant. Still, one consensus to reach is that a model is a representation of reality, not the reality itself, just like the disclaimer of René Magritte that his perfectly curved and brightly polished pipe is not a pipe.
References
- Eker S, Rovenskaya E, Obersteiner M, Langan S. Practice and perspectives in the validation of resource management models. Nature Communications 2018, 9(1): 5359. DOI: 10.1038/s41467-018-07811-9 [pure.iiasa.ac.at/id/eprint/15646/]
- EC. Modelling tools for EU analysis. 2019 [cited 16-01-2019]Available from: https://ec.europa.eu/clima/policies/strategies/analysis/models_en
- EIA. ANNUAL ENERGY OUTLOOK 2018: US Energy Information Administration; 2018. https://www.eia.gov/outlooks/aeo/info_nems_archive.php
- The Economist. In Plato’s cave. The Economist 2009 [cited]Available from: http://www.economist.com/node/12957753#print
- The Economist. Number-crunchers crunched: The uses and abuses of mathematical models. The Economist. 2010. http://www.economist.com/node/15474075
- Stirling A. Keep it complex. Nature 2010, 468(7327): 1029-1031. https://doi.org/10.1038/4681029a
- Nuccitelli D. Climate scientists just debunked deniers’ favorite argument. The Guardian. 2017. https://www.theguardian.com/environment/climate-consensus-97-per-cent/2017/jun/28/climate-scientists-just-debunked-deniers-favorite-argument
- Anscombe N. Models guiding climate policy are ‘dangerously optimistic’. The Guardian 2011 [cited]Available from: https://www.theguardian.com/environment/2011/feb/24/models-climate-policy-optimistic
- Jogalekar A. Climate change models fail to accurately simulate droughts. Scientific American 2013 [cited]Available from: https://blogs.scientificamerican.com/the-curious-wavefunction/climate-change-models-fail-to-accurately-simulate-droughts/
- Kruger T, Geden O, Rayner S. Abandon hype in climate models. The Guardian. 2016. https://www.theguardian.com/science/political-science/2016/apr/26/abandon-hype-in-climate-models
Nov 14, 2018 | Ecosystems, Environment, Systems Analysis
By Alma Mendoza, Colosio Fellow with the IIASA Ecosystems Services and Management Program
Changes in land use cover can have a crucial impact on the environment in terms of biodiversity and the benefits that ecosystems provide to people. Assessing, quantifying, and identifying where these changes are the most drastic is especially important in countries that have high biodiversity along with high rates of natural vegetation loss. Socioeconomic pressures often drive land use change and the impacts are expected to increase due to population growth and climate change.
To better understand the possible impacts of land use change in Mexico over the short, medium, and long term, my colleagues and I used the Shared Socioeconomic Pathways–a set of pathways that span a wide range of feasible future developments in areas such as agriculture, population, and the economy–together with a set of climatic scenarios known as the Representative Concentration Pathways. We focused on Mexico, because the country is large enough to encompass different ecosystems, socioeconomic characteristics, and climates. In addition, Mexico is characterized by high deforestation rates, huge biodiversity, and a large number of communities with contrasting land management practices. Incorporating all these features, allowed us to take the complexity of socioecological systems into account.
We designed a model to test how socioeconomic and biophysical drivers, like slope or altitude, may unfold under different scenarios and affect land use. Our model includes 13 categories of which eight represent the most important ecosystems in Mexico (temperate forests, cloud forests, mangroves, scrublands, tropical evergreen and -dry forests, natural grasslands, and other vegetation such as desert ecosystems or natural palms), four represent anthropogenic uses (pasture, rainfed and irrigated agriculture, and human settlements), and one constitutes barren lands. We set two plausible scenarios: “Business as usual” and an optimistic scenario called the “green scenario”. We projected the “business as usual” scenario using medium rates of vegetation loss based on historical trends and combined it with a medium population and economic growth with medium increases in climatic conditions. For the “green scenario”, we projected the lowest rates of native vegetation loss and the highest rates of native vegetation recovery with a low population and medium economic growth in a future with low climatic changes.
Skyline of Mexico City © Shane Adams | Dreamstime.com
Our results show that natural vegetation will undergo significant reductions in Mexico and that different types of vegetation will be affected differently. Tropical dry and evergreen forests, followed by ‘other’ vegetation and cloud forests are the most vulnerable ecosystems in the country. For example, according to the “business as usual” scenario, tropical dry forests might decrease in extent by 47% by the end of the century. This is extremely important considering that the most recent rates, for the period 2007 to 2011, were even higher than the medium rates we used in this scenario. In contrast, the “green scenario” allowed us to see that, with feasible changes of rate, this ecosystem could increase their distribution. However, even 80 years of regeneration would not be enough to reach the extent these forests had in 1985, when they accounted for around 12% of land cover in Mexico. Moreover, the expansion of anthropogenic land cover (such as agriculture, pastures, and human settlements) might reach 37% of land cover in the country by 2050 and 43% by 2100 under the same scenario. In terms of CO2 emissions due to land use cover change we found that Mexico was responsible for 1-2% of global emissions that are the result of land use cover change, but by 2100 it could account for as much as 5%.
Our findings show that conservation policies have not been effective enough to avoid land use cover change, especially in tropical evergreen forests and drier ecosystems such as tropical dry forests, natural grasslands, and other vegetation. Cloud forests have also been badly affected. As a biologically and culturally rich country, Mexico is responsible for maintaining its diversity by implementing a sustainable and intelligent management of its territory.
Our study identified hotspots of land use change that can help to prioritize areas for improving environmental performance. Our project is currently linking the hotspots of change with the most threatened and endemic species of Mexican terrestrial vertebrates (mammals, amphibians, reptiles, and birds) to provide useful results that can help prioritize ecosystems, species, or municipalities in Mexico.
Reference:
Mendoza Ponce A, Corona-Núñez R, Kraxner F, Leduc S, & Patrizio P (2018). Identifying effects of land use cover changes and climate change on terrestrial ecosystems and carbon stocks in Mexico. Global Environmental Change 53: 12-23. [pure.iiasa.ac.at/15462]
Note: This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.
Oct 16, 2018 | Alumni, Climate Change, Environment, Science and Policy, Women in Science, Young Scientists
Laura Mononen experiencing a creative ”world flow” in the art installation ‘Passage’ by Matej Kren in Bratislava | © Kati Niiles
By Sandra Ortellado, IIASA 2018 Science Communication Fellow
If fashion is the science of appearances, what can beauty and aesthetics tell us about the way we perceive the world, and how it influences us in turn?
From cognitive science research, we know that aesthetics not only influence superficial appearances, but also the deeper ways we think and experience. So, too, do all kinds of creative thinking create change in the same way: as our perceptions of the world around us changes, the world we create changes with them.
From the merchandizing shelves of H&M and Vero Moda to doctoral research at the Faculty of Information Technology at the University of Jyväskylä, Finland, 2018 YSSP participant Laura Mononen has seen product delivery from all angles. Whether dealing with commercialized goods or intellectual knowledge, Mononen knows that creativity is all about a change in thinking, and changing thinking is all about product delivery.
“During my career in the fashion and clothing industry, I saw the different levels of production when we sent designs to factories, received clothing back, and then persuaded customers to buy them. It was all happening very effectively,” says Mononen.
But Mononen saw potential for product delivery beyond selling people things they don’t need. She wanted to transfer the efficiency of the fashion world in creating changes in thinking to the efforts to build a sustainable world.
“Entrepreneurs make change with products and companies, fashion change trends and sell them. I’m really interested in applying this kind of change to science policy and communication,” says Mononen. “We treat these fields as though they are completely different, but the thing that is common is humans and their thinking and behaving.”
Often, change must happen in our thinking first before we can act. That’s why Mononen is getting her doctorate in cognitive science. Her YSSP project involved heavy analysis of systems theories of creativity to find patterns in the way we think about creativity, which has been constantly changing over time.
In the past, creativity was seen as an ability that was characteristic of only certain very gifted individuals. The research focused on traits and psychological factors. Today, the thinking on creativity has shifted towards a more holistic view, incorporating interactions and relationships between larger systems. Instead of being viewed as a lightning bolt of inspiration, creativity is now seen as more of a gradual process.
New understandings of creativity also call on us to embrace paradoxes and chaos, see ourselves as part of nature rather than separate from it, experience the world through aesthetics, pay careful attention to our perception and how we communicate it, and transmit culture to the next generation.
Perhaps most importantly, Mononen found in her research that the understanding of creativity has changed to be seen as part of a process of self-creation as well as co-creation.
“The way we see creativity also influences ourselves. For example if I ask someone if they are creative, it’s the way they see themselves that influences how creative they are,” says Mononen. “I have found that it’s more crucial to us than I thought, creativity is everywhere and it’s everyday and we are sharing our creativity with others who are using that to do something themselves and so on.”
This means on the one hand that we use our creativity to decide who we are and how we see the world around us for ourselves. But it also means that the outcomes and benefits of creativity are now intended for society as a whole rather than purely for individuals, as it was in the past. It may sound like another paradox, but being able to embrace ambiguity and complexity and take charge of our role in a larger system is important for creating a sustainable future.
“From the IIASA perspective this finding brings hope because the more people see themselves as part of systems of creating things, the more we can encourage sustainable thinking, since nature is a part of the resources we use to create,” says Mononen.
Mononen says a systems understanding of creativity is especially important for people in leadership positions. If a large institution needs new and innovative solutions and technology, but doesn’t have the thinking that values and promotes creativity, then the cooperative, open-minded process of building is stifled.
Working in both the fashion industry and academic research, Mononen has encountered narrow-minded attitudes towards art and science firsthand.
“Communicating your research is very difficult coming from my background, because you don’t know how the other person is interpreting what you say,” says Mononen. “People have different ideas of what fashion and aesthetics are, how important they are and what they do. Additionally, scientific concepts are used differently in different fields.”
“We are often thinking that once we get information out there, then people will understand, but there are much more complex things going on to make change and create influence in settings that combine several different fields.” says Mononen.
For Mononen, the biggest lesson is that creativity can enhance the efforts of science towards a sustainable world simply by encouraging us to be aware of our own thinking, how it differs from that of others, and how it affects all of us.
“When you become more aware of your ways of thinking, you become more effective at communicating,” says Mononen. “It’s not always that way and it’s very challenging, but that’s what the research on creativity from a systems perspective is saying.”
Jul 9, 2018 | Alumni, Communication, IIASA Network, South Africa, Systems Analysis, Young Scientists
By Sandra Ortellado, IIASA Science Communication Fellow 2018
In 2007, Sepo Hachigonta was a first-year PhD student studying crop and climate modeling and member of the YSSP cohort. Today, he is the director in the strategic partnership directorate at the National Research Foundation (NRF) in South Africa and one of the editors of the recently launched book Systems Analysis for Complex Global Challenges, which summarizes systems analysis research and its policy implications for issues in South Africa.
From left: Gansen Pillay, Deputy Chief Executive Officer: Research and Innovation Support and Advancement, NRF, Sepo Hachigonta, Editor, Priscilla Mensah, Editor, David Katerere, Editor, Andreas Roodt Editor
But the YSSP program is what first planted the seed for systems analysis thinking, he says, with lots of potential for growth.
Through his YSSP experience, Hachigonta saw that his research could impact the policy system within his home country of South Africa and the nearby region, and he forged lasting bonds with his peers. Together, they were able to think broadly about both academic and cultural issues, giving them the tools to challenge uncertainty and lead systems analysis research across the globe.
Afterwards, Hachigonta spent four years as part of a team leading the NRF, the South African IIASA national member organization (NMO), as well as the Southern African Young Scientists Summer Program (SA-YSSP), which later matured into the South African Systems Analysis Centre. The impressive accomplishments that resulted from these programs deserved to be recognized and highlighted, says Hachigonta, so he and his colleagues collected several years’ worth of research and learning into the book, a collaboration between both IIASA and South African experts.
“After we looked back at the investment we put in the YSSP, we had lots of programs that were happening in South Africa, and lots of publications and collaboration that we wanted to reignite,” said Hachigonta. “We want to look at the issues that we tackled with system analysis as well as the impact of our collaborations with IIASA.”
Now, many years into the relationship between IIASA and South Africa, that partnership has grown.
Between 2012 and 2015, the number of joint programs and collaborations between IIASA and South Africa increased substantially, and the SA-YSSP taught systems analysis skills to over 80 doctoral students from 30 countries, including 35 young scholars from South Africa.
In fact, several of the co-authors are former SA-YSSP alumni and supervisors turned experts in their fields.
“We wanted to use the book as a barometer to show that thanks to NMO public entity funding, students have matured and developed into experts and are able to use what they learned towards the betterment of the people,” says Hachigonta. The book is localized towards issues in South Africa, so it will bring home ideas about how to apply systems analysis thinking to problems like HIV and economic inequality, he adds.
“It’s not just a modeling component in the book, it still speaks to issues that are faced by society.”
Complex social dilemmas like these require clear and thoughtful communication for broader audiences, so the abstracts of the book are organized in sections to discuss how each chapter aligns systems analysis with policymaking and social improvement. That way, the reader can look at the abstract to make sense of the chapter without going into the modeling details.
“Systems analysis is like a black box, we do it every day but don’t learn what exactly it is. But in different countries and different sectors, people are always using systems analysis methodologies,” said Hachigonta, “so we’re hoping this book will enlighten the research community as well as other stakeholders on what systems analysis is and how it can be used to understand some of the challenges that we have.”
“Enlightenment” is a poetic way to frame their goal: recalling the age of human reason that popularized science and paved the way for political revolutions, Hachigonta knows the value of passing down years of intellectual heritage from one cohort of researchers to the next.
“You are watching this seed that was planted grow over time, which keeps you motivated,” says Hachigonta.
“Looking back, I am where I am now because of my involvement with IIASA 11 years ago, which has been shaping my life and the leadership role I’ve been playing within South Africa ever since.”
Jan 22, 2018 | Energy & Climate, Poverty & Equity
By Narasimha Rao, Project Leader of the Decent Living Energy (DLE) Project, IIASA Energy Program
Is there a conflict between reducing global income inequality and combating climate change? This seems like an odd question, given that these challenges have a lot in common. Raising the living standard of the poor for example, makes them resilient to climate impacts; less inequality can mean more political mobilization to establish climate policies; and changes in social norms away from material accumulation can reduce inequality and emissions. Academics have however been curious about the following phenomenon: In many countries, a dollar spent at higher income levels is less energy intensive than at lower income levels (known as “income elasticity of energy”). That is, rich people – although they consume much more in total – spend additional income on services or can afford energy-efficient goods, while the new middle class buy energy-intensive goods, like appliances and cars.
Many imagine China as a template for this type of fast growth. If globally significant, this effect would imply that growth that is more equitable would also be more emissions-intensive, and that we would have to pay particular attention to ensuring that climate policies reach the rising middle class in developing countries. While several studies have examined this phenomenon in specific countries, no one has examined its global significance. We set out to do that.
Energy intensity (MJ per $) lower in a high-growth, low inequality world (green line, Gini=0.29) compared to a low-growth, high inequality world (blue line, Gini=0.45). Gini reflects between-country inequality only.
Our analysis suggests that the energy-increasing effect of lowering inequality is more of a distraction than a concern. We compared scenarios of equitable and inequitable income growth, both within and between countries, assuming the most extreme manifestation of the income elasticity. Within any country, given the slow pace at which inequality typically evolves even with the most extreme known income elasticity and reduction in country inequality, greenhouse gas emissions would increase by less than 8% over a couple of decades. However, when one considers a more equitable distribution of growth between countries, global emissions growth may decrease when compared to growth that occurs in industrialized countries. This is because poorer countries have more potential for technological advancements that reduce the energy intensity of growth than richer countries do. That is, more income growth in poorer countries provides more opportunity for efficiency improvements that influence the emissions of very large populations. Furthermore, China is a poor model for poor countries at large, many of which have relatively low energy intensities, even today.
Climate stabilization at the level aspired to by the Paris Climate Agreement requires that we (i.e. the world) decarbonize to zero annual emissions around 2050, which means that even developing countries have to make aggressive strides towards integrating climate goals into development. Nevertheless, there is no sufficient basis for considering that equitable growth, and by implication the poor’s energy intensity, is part of the problem. To the contrary, the potential for co-benefits from equitable growth for climate change are enormous, but unfortunately under-explored, particularly in quantitative studies. Research should focus on quantifying the role of changing social norms – less consumerism, political mobilization, and other social changes that are typically associated with lower inequality – on reducing greenhouse gases.
Reference:
Rao, ND, Min J. Less global inequality can improve climate outcomes. Wiley Interdisciplinary Reviews: Climate Change. 2018;e513. https://doi.org/10.1002/wcc.513
Note: This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.
You must be logged in to post a comment.