Open science has to go beyond open source

By Daniel Huppmann, research scholar in the IIASA Energy Program

Daniel Huppmann sheds light on how open-source scientific software and FAIR data can bring us one step closer to a community of open science.

© VectorMine | Dreamstime.com

Over the past decade, the open-source movement (e.g., the Free Software Foundation (FSF) and the Open Source Initiative (OSI)) has had a tremendous impact on the modeling of energy systems and climate change mitigation policies. It is now widely expected – in particular by and of early-career researchers – that data, software code, and tools supporting scientific analysis are published for transparency and reproducibility. Many journals actually require that authors make the underlying data available in line with the FAIR principles – this acronym stands for findable, accessible, interoperable, and reusable. The principles postulate best-practice guidance for scientific data stewardship. Initiatives such as Plan S, requiring all manuscripts from projects funded by the signatories to be released as open-access publications, lend further support to the push for open science.

Alas, the energy and climate modeling community has so far failed to realize and implement the full potential of the broader movement towards collaborative work and best practice of scientific software development. To live up to the expectation of truly open science, the research community needs to move beyond “only” open-source.

Until now, the main focus of the call for open and transparent research has been on releasing the final status of scientific work under an open-source license – giving others the right to inspect, reuse, modify, and share the original work. In practice, this often means simply uploading the data and source code for generating results or analysis to a service like Zenodo. This is obviously an improvement compared to the previously common “available upon reasonable request” approach. Unfortunately, the data and source code are still all too often poorly documented and do not follow best practice of scientific software development or data curation. While the research is therefore formally “open”, it is often not easily intelligible or reusable with reasonable effort by other researchers.

What do I mean by “best practice”? Imagine I implement a particular feature in a model or write a script to answer a specific research question. I then add a second feature – which inadvertently changes the behavior of the first feature. You might think that this could be easily identified and corrected. Unfortunately, given the complexity and size to which scientific software projects tend to quickly evolve, one often fails to spot the altered behavior immediately.

One solution to this risk is “continuous integration” and automated testing. This is a practice common in software development: for each new feature, we write specific tests in an as-simple-as-possible example at the same time as implementing the function or feature itself. These tests are then executed every time that a new feature is added to the model, toolbox, or software package, ensuring that existing features continue to work as expected when adding a new functionality.

Other practices that modelers and all researchers using numerical methods should follow include using version control and writing documentation throughout the development of scientific software rather than leaving this until the end. In addition, not just the manuscript and results of scientific work should be scrutinized (aka “peer review”), but such appraisal should also apply to the scientific software code written to process data and analyze model results. In addition, like the mentoring of early-career researchers, such a review should not just come at the end of a project but should be a continuous process throughout the development of the manuscript and the related analysis scripts.

In the course that I teach at TU Wien, as well as in my work on the MESSAGEix model, the Intergovernmental Panel on Climate Change Special Report on Global Warming of 1.5°C scenario ensemble, and other projects at the IIASA Energy Program, I try to explain to students and junior researchers that following such best-practice steps is in their own best interest. This is true even when it is just a master’s thesis or some coursework assignment. However, I always struggle to find the best way to convince them that following best practice is not just a noble ideal in itself, but actually helps in doing research more effectively. Only when one has experienced the panic and stress caused by a model not solving or a script not running shortly before a submission deadline can a researcher fully appreciate the benefits of well-structured code, explicit dependencies, continuous integration, tests, and good documentation.

A common trope says that your worst collaborator is yourself from six months ago, because you didn’t write enough explanatory comments in your code and you don’t respond to emails. So even though it sounds paradoxical at first, spending a bit more time following best practice of scientific software development can actually give you more time for interesting research. Moreover, when you then release your code and data under an open-source license, it is more likely that other researchers can efficiently build on your work – bringing us one step closer to a community of open science!

Note: This article gives the views of the authors, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

5 years of Vietnam membership at IIASA

Tran Thi Vo-Quyen, IIASA guest research scholar from the Vietnam Academy of Science and Technology (VAST), talks to Professor Dr. Ninh Khac Ban, Director General of the International Cooperation Department at VAST and IIASA council member for Vietnam, about achievements and challenges that Vietnam has faced in the last 5 years, and how IIASA research will help Vietnam and VAST in the future.

Professor Dr. Ninh Khac Ban, Director General of the International Cooperation Department at VAST and IIASA council member for Vietnam

What have been the highlights of Vietnam-IIASA membership until now?

In 2017, IIASA and VAST researchers started working on a joint project to support air pollution management in the Hanoi region which ultimately led to the successful development of the IIASA Greenhouse Gas – Air  Pollution Interactions and Synergies (GAINS) model for the Hanoi region. The success of the project will contribute to a system for forecasting the changing trend of air pollution and will help local policy makers develop cost effective policy and management plans for improving air quality, in particular, in Hanoi and more widely in Vietnam.

IIASA capacity building programs have also been successful for Vietnam, with a participant of the 2017 Young Scientists Summer Program (YSSP) becoming a key coordinator of the GAINS project. VAST has also benefited from two members of its International Cooperation Department visiting the IIASA External Relations Department for a period of 3 months in 2018 and 2019, to learn about how IIASA deals with its National Member Organizations (NMOs) and to assist IIASA in developing its activities with Vietnam.

What do you think will be the key scientific challenges to face Vietnam in the next few years? And how do you envision IIASA helping Vietnam to tackle these? 

In the global context Vietnam is facing many challenges relating to climate change, energy issues and environmental pollution, which will continue in the coming years. IIASA can help key members of Vietnam’s scientific community to build specific scenarios, access in-depth knowledge and obtain global data that will help them advise Vietnamese government officials on how best they can overcome the negative impact of these issues.

As Director General of the International Cooperation Department, can you explain your role in VAST and as representative to IIASA in a little more detail?

In leading the International Cooperation Department at VAST, I coordinate all collaborative science and technology activities between VAST and more than 50 international partner institutions that collaborate with VAST.

As the IIASA council representative for Vietnam, I participate in the biannual meeting for the IIASA council, I was also a member of the recent task force developed to implement the recommendations of a recent independent review of the institute. I was involved in consulting on the future strategies, organizational structure, NMO value proposition and need to improve the management system of IIASA.

In Vietnam, I advised on the establishment of a Vietnam network for joining IIASA and I implement IIASA-Vietnam activities, coordinating with other IIASA NMOs to ensure Vietnam is well represented in their countries.

You mentioned the development of the Vietnam-IIASA GAINS Model. Can you explain why this was so important to Vietnam and how it is helping to improve air quality and shape Vietnamese policy around air pollution? 

Air pollution levels in Vietnam in the last years has had an adverse effect on public health and has caused significant environmental degradation, including greenhouse gas (GHG) emissions, undermining the potential for sustainable socioeconomic development of the country and impacting the poor. It was important for Vietnam to use IIASA researchers’ expertise and models to help them improve the current situation, and to help Vietnam in developing the scientific infrastructure for a long-lasting science-policy interface for air quality management.

The project is helping Vietnamese researchers in a number of ways, including helping us to develop a multi-disciplinary research community in Vietnam on integrated air quality management, and in providing local decision makers with the capacity to develop cost-effective management plans for the Hanoi metropolitan area and surrounding regions and, in the longer-term, the whole of Vietnam.

About VAST and Ninh Khac Ban

VAST was established in 1975 by the Vietnamese government to carry out basic research in natural sciences and to provide objective grounds for science and technology management, for shaping policies, strategies and plans for socio-economic development in Vietnam. Ninh Khac Ban obtained his PhD in Biology from VAST’s Institute of Ecology and Biological Resources in 2001. He has managed several large research projects as a principal advisor, including several multinational joint research projects. His successful academic career has led to the publication of more than 34 international articles with a high ranking, and more than 60 national articles, and 2 registered patents. He has supervised 5 master’s and 9 PhD level students successfully to graduation and has contributed to pedagogical texts for postgraduate training in his field of expertise. 

Notes:
More information on IIASA and Vietnam collaborations. This article gives the views of the authors, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

When science meets art: The birth of a new mindset

by Merlijn Twaalfhoven

Composer Merlijn Twaalfhoven co-created the IIASA session: Courage – A dance science performance debate about sustainable futures by Gloria Benedikt who leads the Art and Science Project at IIASA, at the EuroScience Open Forum in Toulouse. He tells us more about how this merging of worlds can create new bridges between art, science, and societal issues.

An urgent need for better questions

There I was, an artist invited to the largest conference on science and innovation in Europe – the EuroScience Open Forum, commonly referred to as ESOF. Amidst showcases and lectures about robotics, space exploration, and gene technology, I could share my ideas about the connection between art and science.

Merlijn Twaalfhoven,  © Guillaume Gimenez

This topic can easily become filled with abstractions or generalities. It is true that science and art are both about creativity and research, but is there a concrete way for them to find each other? Is there urgency to bridge these two worlds? The answer to both these questions is a resounding yes.

Different eyes

Art and science are like children that always loved to play together. The worlds of both are filled with curiosity, take steps into the unknown, and embrace uncertainty and adventure. Both are not satisfied with the answers that are provided, and keep asking questions – obsessed with what is below the surface, beyond the horizon, and beneath the solid ground of our convictions.

These two children have much in common, yet they have different eyes, they see things differently. While observing the same world, they pose different questions.

One would ask, ‘What is this?’, ‘How did it become this way?’ and ‘Why is this the way it is?’ While the other asks, ‘What might this also be?’, ‘How can we see this differently?’ and ‘What will this be if…?’

Art and science have always learned and grown together, but over the last few centuries, they have started to grow more and more apart. We might say they have grown up. They each developed their curiosity and creativity into diligent practices and professional attitudes, and started to work within strict disciplinary boundaries. The playfulness and free experimenting sadly transformed into competition and a fixation on what they call “excellence”.

Who is asking the questions?

Today, while global warming, inequality, digitalization, and migration bring instability and change to our societies and global mindsets, there is an urgent need for answers, reliability, and guidance.

Leaders of all kinds, including artists that might influence the way we feel and scientists that might direct the way we reason, are pressed to give such answers. We all add our answers to platforms and channels of news, opinion, soundbites, and statements. It is a cacophony. A flood without structure.

Nevertheless, do we actually know who is asking the questions? Is it the entertainers, talk show hosts, and commercial news media? Will we let the most dominant voices define the big questions of our time? Or can we bring better questions built on the observations, insights, and intuition of artists and scientists to the surface?

Show, don’t talk

We know what the big challenges in society are, and the urgent steps needed to prevent catastrophic global warming, societal polarization, and mass migration are well defined. But how do we, as curious, experienced, and highly trained artists, scientists, researchers, and explorers join forces to make the urgent and engaging questions become not just loud and clear, but part of everybody’s daily reality? Can we not just write or talk about warnings and concerns, but rather create and build questions that ask for a vision that is attractive and beautiful – a way forward that we can experience directly and that makes all of us want to make the world great again?

Beauty

A first step is to find beauty in the world around us. Instead of asking, ‘Why are these people poor?’ we can ask, ‘What makes you proud?’ Instead of asking, ‘How do we stop people from buying cars?’ we can ask, ‘How can I relieve you from the burden of owning (cleaning, insuring, repairing) a car?’ Instead of asking, ‘Why are people unhealthy?’ we can ask, ‘When do you feel free of stress?’

Imagination

A second step is to design ways to express this beauty and to imagine new possibilities. How can we show others the pride you feel? What is the value you get when you do not own a car? What does a stress-free moment sound like?

Play

A third step is to connect people to a changing and evolving reality in playful and challenging ways. What makes your pride contagious to others? How do we practice freedom from possessions? How do we build moments of stress-free time into a rhythm?

These are only a few examples of how art can expose beauty in urgent research topics, create forms of expression to give new ideas presence, and build living and changing structures where a new scientific/artistic mindset may flourish.

What will be the laboratories in which we develop and explore new questions? When will these grownups – art and science – decide to approach each other again, remember their playful childhood, and fall in love all over again? It might be a fruitful affair and give birth to a new mindset that can help us face today’s challenges.

ESOF 2018

Thank you: In writing this text, I received some valuable contributions from Gloria Benedikt and Anne Glover.

Note: This article gives the views of the authors, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

How to achieve scientific excellence

W. Brian Arthur from the Santa Fe Institute (SFI), and a former IIASA researcher, talks about increasing returns and the magic formula to get really great science.

Recently, Brian stopped in at the Complexity Science Hub Vienna, of which IIASA is a member institution, and spoke to Verena Ahne about his work.

Brian Arthur (© Complexity Science Hub)

Brian Arthur (© Complexity Science Hub)

Brian, now 71, is one of the most influential early thinkers of the SFI, a place that without exaggeration could be called the cradle of complexity science.

Brian became famous with his theory of increasing returns. An idea that has been developed in Vienna, by the way, where Brian was part of a theoretical group at the IIASA in the early days of his career: from 1978 to 1982.

“I was very lucky,” he recalls. “I was allowed to work on what I wanted, so I worked on increasing returns.”

The paper he wrote at that time introduced the concept of positive feedbacks into economy.

The concept of “increasing returns”

Increasing returns are the tendency for that which is ahead to get further ahead, for that which loses advantage to lose further advantage. They are mechanisms of positive feedback that operate—within markets, businesses, and industries—to reinforce that which gains success or aggravate that which suffers loss. Increasing returns generate not equilibrium but instability: If a product or a company or a technology—one of many competing in a market—gets ahead by chance or clever strategy, increasing returns can magnify this advantage, and the product or company or technology can go on to lock in the market.”

(W Brian Arthur, Harvard Business Review 1996)

This was a slap in the face of orthodox theories which saw–and some still see–economy in a state of equilibrium. “Kind of like a spiders web,” Brian explains me in our short conversation last Friday, “each part of the economy holding the others in an equalization of forces.”

The answer to heresy in science is that it does not get published. Brian’s article was turned down for six years. Today it counts more than 10.000 citations.

At the latest it was the development and triumphant advance of Silicon Valley’s tech firms that proved the concept true. “In fact, that’s now the way how Silicon Valley runs,” Brian says.

The youngest man on a Stanford chair

William Brian Arthur is Irish. He was born and raised in Belfast and first studied in England. But soon he moved to the US. After the PhD and his five years in Vienna he returned to California where he became the youngest chair holder in Stanford with 37 years.

Five years later he changed again – to Santa Fe, to an institute that had been set up around 1983 but had been quite quiet so far.

Q: From one of the most prestigious universities in the world to an unknown little place in the desert. Why did you do that? 

A: In 1987 Kenneth Arrow, an economics Nobel Prize winner and mentor of mine, said to me at Stanford: We’re holding a small conference in September in a place in the Rockies, in Santa Fe, would you go?

When a Nobel Prize winner asks you such a question, you say yes of course. So I went to Santa Fe.

We were about ten scientists and ten economists at that conference, all chosen by Nobel Prize winners. We talked about the economy as an evolving complex system.

Veni, vidi, vici

Brian came – and stayed: The unorthodox ideas discussed at the meeting and the “wild” and free atmosphere of thinking at “the Institute”, as he calls the Santa Fe Institute (SFI), thrilled him right away.

In 1988 Brian dared to leave Stanford and started to set up the first research program at Santa Fe. Subject was the economy treated as a complex system.

Q: What was so special about SF?

A: The idea of complexity was quite new at that time. But people began to see certain patterns in all sorts of fields, whether it was chemistry or the economy or parts of physics, that interacting elements would together create these patterns…To investigate this in universities with their particular disciplines, with their fixed theories, fixed orthodoxies–where it is all fixed how to do things–turned out to be difficult.

Take the economy for example. Until then people thought it was in an equilibrium. And there we came and proved, no, economics is no equilibrium! The Stanford department would immediately say: You can’t do that! Don’t do that! Or they would consider you to be very eccentric…

So a bunch of senior fellows at Los Alamos in the 1980s thought it would be a good idea if there was an independent institute to research these common questions that came to be called complexity.

At Santa Fe you could talk about any science and any basic assumptions you wanted without anybody saying you couldn’t or shouldn’t do that.

Our group as the first there set a lot of this wild style of research. There were lots of discussions, lots of open questions, without particular disciplines… In the beginning there were no students, there was no teaching. It was all very free.

This wild style became more or less the pattern that has been followed ever since. I think the Hub is following this model too.

The magic formula for excellence

Q: Was this just a lucky concurrence: the right people and atmosphere at the right time? Or is there a pattern behind it that possibly could be repeated?

A: I am sure: If you want to do interdisciplinary science – which complexity is: It is a different way of looking at things! – you need an atmosphere where people aren’t reinforced into all the assumptions of the different disciplines.

This freedom is crucial to excellent science altogether. It worked out not only for Santa Fe. Take the Rand Corporation for instance, that invented a lot of things including the architecture of the internet, or the Bell Labs in the Fifties that invented the transistor. The Cavendish Lab in Cambridge is another one, with the DNA or nuclear astronomy…

The magic formula seems to be this:

  • First get some first rate people. It must be absolutely top-notch people, maybe ten or twenty of them.
  • Make sure they interact a lot.
  • Allow them to do what they want – be confident that they will do something important.
  • And then when you protect them and see that they are well funded, you are off and running.

Probably in seven cases out of ten that will not produce much. But quite a few times you will get something spectacular – game changing things like quantum theory or the internet.

Don’t choose programs, choose people

Q: This does not seem to be the way officials are funding science…

A: Yes, in many places you have officials telling people what they need to research. Or where people insist on performance and indices… especially in Europe, I have the impression, you have a tradition of funding science by insisting on all these things like indices and performance and publications or citation numbers. But that’s not a very good formula.

Excellence is not measurable by performance indicators. In fact that’s the opposite of doing science.

I notice at places where everybody emphasize all this they are not on the forefront. Maybe it works for standard science; and to get out the really bad science. But it doesn’t work if you want to push boundaries.

Many officials don’t understand that.

In Singapore the authorities once asked me: How did you decide on the research projects in Santa Fe? I said, I didn’t decide on the research projects. They repeated their question. I said again, I did not decide on the research projects. I only decided on people. I got absolutely first rate people, we discussed vaguely the direction we wanted things to be in, and they decided on their research projects.

That answer did not compute with them. They are the civil service, they are extraordinarily bright, they’ve got a lot of money. So they think they should decide what needs to be researched.

I should have told them – I regret I didn’t: This is fine if you want to find solutions for certain things, like getting the traffic running or fixing the health care system. Surely with taxpayer’s money you have to figure such things out. But you will never get great science with that. All you get is mediocrity.

Of course now they asked, how do we decide which people should be funded? And I said: “You don’t! Just allow top people to bring in top people. Give them funding and the task of being daring.”

Any other way of managing top science doesn’t seem to work.

I think the Hub could be such a place – all the ingredients are here. Just make sure to attract some more absolutely first rate people. If they are well funded the Hub will put itself on the map very quickly.

This interview was originally published on https://www.csh.ac.at/brian-arthurs-magic-formula-for-excellence/

Parched: The war for water in Mexico City

By Parul Tewari, IIASA Science Communication Fellow 2017

Mexico City has been experiencing a major water crisis in the last few decades and it is only getting worse. To keep the water flowing, the city imports large amounts of water from as far as 150 kilometers.

Not only is this energy-intensive and expensive, it creates conflict with the indigenous communities in the donor basins. Over the last decade, a growing number of these communities have been protesting to reclaim their rights to water resources.

The ancient city of Tenochtitlan as depicted in a mural by Diego Rivera
(cc) Wikimedia Commons

As part of the 2017 Young Scientists Summer Program at IIASA, Francine van den Brandeler studied the struggle that Mexico City is facing as it tries to provide water to its growing population and expanding economy. Local aquifers have been over-exploited, so water needs to be imported from distant sources, with high economic, social, and environmental impacts. Van den Brandeler’s study assesses the effectiveness of water use rights in promoting sustainable water use and reducing groundwater exploitation in the city.

“A few centuries back, Tenochtitlan, the place where Mexico City stands today, was known as the lake city,” says Van den Brandeler. The Aztecs had developed a sophisticated system of dikes and canals to manage water and mitigate floods. However, that changed quickly with the arrival of the Spaniards, who transformed the natural hydrology of the valley. As the population continued to grow over the next centuries, providing drinking water became an increasing challenge, along with controlling floods. As the lake dried up, people pumped water from the ground and built increasingly large infrastructure to bring water from other areas.

Communities from lower-income groups, living in informal settlements on the outskirts of the metropolitan region are more vulnerable to this scarcity. Many live on just few liters of water every day, and do not have access to the main water supply network, instead relying on water trucks which charge several times the price of water from the public utility.

“In wealthier areas people consume much more than the average European does every day. It is a question of power and politics,” says van den Brandeler. “The voices of marginalized communities go unheard.”

Many people rely on delivery service for drinking water.
© Angela Ostafichuk | Shutterstock

The more one learns about the situation, the more complicated it becomes. The import of water started in the 1940’s. But with a massive increase in population in the last couple of decades, the deficits have become much worse.

The government’s approach has been to find more water rather than rehabilitating or reusing local surface and groundwater sources, or increasing water use efficiency, says van den Brandeler. Therefore wells are being drilled deeper and deeper—as much as 2000 meters into the ground—as the water runs out.

Some people have started initiatives to harvest rainwater, but it is not considered a viable solution by those in charge. “A lot of it has to do with their worldview and general paradigm. The people working at the National Water Commission and the Water Utility of Mexico City have been trained as engineers to make large dams and put pipes in the ground. They don’t believe in small-scale solutions. In their opinion when millions of people are concerned, such solutions cannot work,” says van den Brandeler.
Although the city gets plenty of rain during the rainy season, it goes directly into the drainage system which is linked to the sewage system. This contaminates the water, making it unusable. At the same time, almost 40% of the water in Mexico City’s piped networks is lost due to leakages.

Policy procedures and institutional functioning also remain top-down and opaque, van den Brandeler has found. One of the policy tools for curbing excess water use are water permits for bulk use, for agriculture, industry, or public utilities supplying water. Introduced in the 1940s, lack of proper enforcement has created misuse and conflicts.
For example, while farmers also require a permit that specifies the volume of water they may use each year, they do not pay for their water usage. However, it is difficult to monitor if farmers are extracting water according to the conditions in the permit. Since they do not pay a usage fee, there is also less incentive for the National Water Commission to monitor them. As a result, a huge black market has cropped up in the city where property owners and commercial developers pay exorbitant prices to buy water permits from those who have a license. Since the government allows the exchange of permits between two willing parties, they make it appear above-board. However, it has contributed to the inequalities in water distribution in the city.

With the water crisis worsening every year, Mexico City needs to find a solution before it runs out of water completely. Van den Brandeler is hopeful for a better future as she studies the contributing factors to the problem. She hopes that the water use permits are better enforced and users are given stronger incentives to respect their allocated water quotas. Further, if greater efforts are made within the metropolis to repair decaying infrastructure and scale up alternatives such as rainwater harvesting and wastewater reuse, the city won’t have to look at expensive solutions if adopted in a decentralized manner.

About the Researcher

Francine van den Brandeler is a third year PhD student at the University of Amsterdam in Netherlands. Her research is on the spatial mismatches between integrated river basin management and metropolitan water governance – the incompatibility of institutions and biophysical systems-, which can lead to fragmented water policy outcomes. Fragmented decision-making cannot adequately address the issues of sustainability and social inclusion faced by megacities in the Global South. She aims to assess the effectiveness of policy instruments to overcome this mismatch and suggest recommendations for policy (re)design. At IIASA she was part of the Water Program and worked under the supervision of Sylvia Tramberend and Water Program Director Simon Langan.

This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.