Intelligent cooperation

By Valeria Javalera Rincón, IIASA CONACYT Postdoctoral Fellow in the Ecosystems Services and Management and Advanced Systems Analysis programs.

What is more important: water, energy, or food?

If you work in the water, energy or agriculture sector we can guess what your answer might be! But if you are a policy or decision maker trying to balance all three, then you know that it is getting more and more difficult to meet the growing demand for water, energy, and food with the natural resources available. The need for this balance was confirmed by the 17 Sustainable Development Goals, agreed by 193 countries, and the Paris climate agreement. But how to achieve it? Intelligent cooperation is the key.

The thing is that water, energy, and food are all related in such a way that are reliant on each other for production or distribution. This is the so-called Water-Energy-Food nexus. In many cases, you need water to produce energy, you need energy to pump water, and you need water and energy to produce, distribute, and conserve food.

Many scientists have tried to relate or to link models for water, agriculture, land, and energy to study these synergic relationships. In general, so far, there are two ways that this has been solved: One is integrating models with “hard linkages” like this:

In the picture there are six models (let’s say water, land use, hydro energy, gas, coal, food production models) that are then integrated into just one. The resulting integrated model then preserves the relationships but is complex, and in order to make it work with our current computer power you often have to sacrifice details.

Another way is to link them is using so-called “soft linkages” where the output of one model is the input of the next one, like this:

In the picture, each person is a model and the input is the amount of water left. These models all refer to a common resource (the water) and are connected using “soft linkages.” These linkages are based on sequential interaction, so there is no feedback, and no real synergy.

The intelligent linker agent

But what if we could have the relations and synergies between the models? It would mean much more accurate findings and helpful policy advice. Well, now we can. The secret is to link through an intelligent linker agent.

I developed a methodology in which an intelligent linker agent is used as a “negotiator” between models that can communicate with each other. This negotiator applies a machine-learning algorithm that gives it the capability to learn from the interactions with the models. Through these interactions, the intelligent linker can advise on globally optimal actions.

The knowledge of the intelligent linker is based on past experience and also on hypothetical future actions that are evaluated in a training process.  This methodology has been used to link drinking water networks, such as Barcelona’s drinking water network.

When I came to IIASA, I was asked to apply this approach to optimize trading between cities in the Shanxi region of China. I used a set of previously development models which aimed to distribute water and land available for each city in order to produce food (eight types of crops) and coal for energy. The intelligent linker agent optimizes trading between cities in order to satisfy demand at the lowest cost for each city.

The purpose of this exercise was to compare the solutions with those from “hard linkages” – like those in the first picture. We found that the intelligent linker is flexible enough to find the optimal solution to questions such as: How much of each of these products should each city export/import to satisfy global demand at a global lower economic and ecological cost? What actions are optimal when the total production is insufficient to meet the total demand? Under what conditions is it preferable to stop imports/exports when production is insufficient to supply the demand of each city?

The answers to these questions can be calculated by the interaction with the models of each city just by the interfacing with the intelligent linker agent, this means that no major changes in the models of each city were needed. We also found that, under the same conditions, the solutions using the intelligent linker agent were in agreement with those found when hard linking was used.

My next challenge is to build a prototype of a “distributed computer platform,” which will allow us to link models on different computers in different parts of the world—so that we in Austria could link to a model built by colleagues in Brazil, for example.  I also want to link models of different sectors and regions of the globe, in order to prove that intelligent cooperation is the key to improving global welfare.

References

Xu X, Gao J, Cao G-Y, Ermoliev Y, Ermolieva T, Kryazhimskiy AV, & Rovenskaya E (2015). Modeling water-energy-food nexus for planning energy and agriculture developments: case study of coal mining industry in Shanxi province, China. IIASA Interim Report. IIASA, Laxenburg, Austria: IR-15-020

Javalera V, Morcego B, & Puig V, Negotiation and Learning in distributed MPC of Large Scale Systems, Proceedings of the 2010 American Control Conference, Baltimore, MD, 2010, pp. 3168-3173. doi: 10.1109/ACC.2010.5530986

Valeria J, Morcego B, & Puig V, Distributed MPC for Large Scale Systems using Agent-based Reinforcement Learning, In IFAC Proceedings Volumes, Volume 43, Issue 8, 2010, Pages 597-602, ISSN 1474-6670, ISBN 9783902661913, https://doi.org/10.3182/20100712-3-FR-2020.00097.

Morcego B, Javalera V, Puig V, & Vito R (2014). Distributed MPC Using Reinforcement Learning Based Negotiation: Application to Large Scale Systems. In: Maestre J., Negenborn R. (eds) Distributed Model Predictive Control Made Easy. Intelligent Systems, Control and automation: Science and Engineering, vol 69. Springer, Dordrecht

Javalera Rincón V, Distributed large scale systems: a multi-agent RL-MPC architecture, Universitat Politècnica de Catalunya. Institut d’Organització i Control de Sistemes Industrials,Doctoral thesis. 2016. http://upcommons.upc.edu/handle/2117/96332

Note: This article gives the views of the author and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.