Communicating scientific uncertainty

Michaela Maier is a professor of applied communication psychology at the Institute of Communication and Media Psychology, University of Koblenz-Landau, Germany. Her work spans many forms of communication from crisis to election coverage to participatory decision making. She talks to IIASA Science Writer and Editor Daisy Brickhill.  

Why is it important to communicate scientific uncertainty?

For us as researchers scientific uncertainty is something very natural that we consider all the time. We assess things like quality of samples used or numbers of replications, we justify and evaluate our research by taking into account a wider body of literature that may or may not show similar results. However, the degree of scientific uncertainty is not something that the public is so aware of. It is quite likely that it has not have been part of their education to learn the methods of evaluating the degree of (un)certainty of scientific evidence.

© Michaela Maier

Despite that, the public will almost definitely experience situations where they will have to evaluate uncertainty. As patients they might have to give their consent to a new medical treatment; as consumers they might need to decide whether to buy products that include, say, nanotechnology or genetically modified plants. Also as citizens—scientific evidence is relevant to many political decisions and as a voter you have to decide about these policies on election day. There are many situations where laypeople have to make judgements based on scientific findings—and we should communicate about the (un)certainty of this evidence in terms that people can understand.

Communicating uncertainty and how it works in one field or for one result can also give people the tools to understand and make judgments about other cases. Uncertainty can cover many things – is the sample large enough to draw any firm conclusions? Is it really representative of the whole population you are interested in?

When communicating complex topics, scientists and journalists can be nervous that talking about uncertainty will undermine the public’s interest or trust in the research – you’ve done some research on this?

 Yes, we used communication of uncertainty around the safety of nanotechnology as a case study. Before the experiment we asked the participants a series of questions on how interested they were in science in general, in getting engaged in citizen science projects for example, or how likely they were to go to science museums. We also asked about their trust in scientists using measures of their perceptions of scientists’ competence, willingness to protect the public from technological risks, and honesty. We then sent them media reports over the course of six weeks, and after that we gave them the surveys again.

We found that communicating uncertainty didn’t undermine interest in science. In fact, what we found was that for a certain group of people the interest in science increased when uncertainty was discussed in the media reports they read. These people have what is called a low ‘need for cognitive closure.’ This means they are more open-minded and have a willingness to consider new or inconsistent information.

And did it undermine trust?

Communicating uncertainty didn’t seem to make any difference to the level of trust in scientists in people with either high or low need for cognitive closure. Ultimately our work showed that you won’t harm interest in science or trust in scientists by communicating uncertainty. It might not make much difference to some people but for others they will become even more engaged. It’s a very positive message.

Outside of an experimental environment are there any ways of engaging people with these complex issues of uncertainty?   

It’s certainly a challenge for us to find the right formats. Narrative structures are an important format to pursue I think. There is a lot of evidence that narrative structures—storytelling—help people deal with complex information and that they really learn from it. Narrative structures focus more on the characters, they have a storyline, they might give the audience a chance to identify with the researcher, for instance.

This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

Diversity in dialogue

By Daisy Brickhill, IIASA science writer and editor.

“If you are the first to break a glass ceiling, you’re the one who gets scratched.” Tarja Halonen is a first for many things. She was the first female trade unions lawyer in Finland, the first female foreign minister for the country, and its first female president. She was also the first ex-president to spend her sabbatical at IIASA, meeting staff and discussing the institute’s work.

With a career rooted in social justice and a firm commitment to human rights, Halonen was eager to engage all at IIASA in discussions on equality and diversity, both within the institute and in wider society.

Damaging gender stereotyping was a theme which was touched on often in the Women in Science session, a meeting of female scientists from all disciplines and of all ages from around the institute. Halonen spoke of her own experiences of the monotonous gender pigeon-holing she experienced in politics “If you are married as a female politician you are either there because you have a strong man behind you or you are neglecting your family. And your appearance is always fair game. I remember the media saying that my handbag was too big. I was betraying the people of Finland with my big handbag!”

Women in science at IIASA © Michaela Rossini

Women in science session with President Halonen at IIASA. © Michaela Rossini | IIASA

A key stereotype that can strongly affect both women and men is that childcare is “women’s work,” and as such unimportant. Parents of both genders should be supported to be active members of their own family. Men are often excluded from this—there is no statutory paternity leave in Austria, for instance—and women, expected to take on the majority of caring, are in turn excluded from their careers at a key point, especially if childcare services are poor or prohibitively expensive.

So how do we change things? All participants were eager to discuss ways of improving the situation. We can start by working from the bottom up, Halonen urged, doing everything we can in our immediate environments to improve things. Discuss diversity and equality issues as often as you can with as many people as you can. Be aware of your own unconscious biases, perhaps attend training to help challenge yourself, not just about gender but all types of diversity. Build pressure by enlisting the help of those outside your workplace too—visitors, funders, boards, and committees.

At this point in the Equality and Diversity session the door opened to admit IIASA Director General and CEO Professor Pavel Kabat, and Halonen turned to him with a twinkle in her eye: “Ah Pavel, we are so pleased to see you! You have come at exactly the right time, we were just discussing how we can change you and the whole institution.”

 © Katherine Leitzell

President Halonen and Prof. Dr Kabat © Katherine Leitzell | IIASA.

Engaging with the discussion, Kabat said “We are looking into improving things at IIASA, especially through our new human capital management.”

Importantly, the case is not hard to make: diversity and equality is good for everyone. “Even if we limit our argument to money alone we can still see clear benefits,” says Halonen. “Giving women more equality in Finland actually caused a rise in the country’s GDP. Turns out it wasn’t a smart financial decision to exclude half the population.” What a surprise.

Top-down measures also have their part to play. Quotas for gender balance—on panels or in committees for instance—can be controversial but they needn’t be, says Halonen briskly. “Set the limits at 40:60 in either direction, and remember, quotas are not the end point, they are a step towards something.” Seeing women in positions of power is important not just as role models for other women, it is important for men, young men and boys in particular, who will grow up happy to work for a female boss, and benefit from collaborations with female colleagues based on mutual respect.

Compulsory paternity leave is another controversial top-down measure that may help change attitudes. One participant in the Equality and Diversity session said that he would welcome this, and not only on a personal level. “I can imagine it would benefit not just the individual fathers and children but society as a whole, IIASA should do studies on this.”

There is hope in the air. As part of the Sustainable Development Goals all 193 member states of the UN have agreed that we need to achieve gender equality. Despite this, change won’t come as a single revolution, Halonen warned, it will come bit by bit as attitudes and rules slowly change. IIASA, as a thought-leader on sustainability transformations, can be a pioneer. After all, having equality and diversity at IIASA, an institute that aims to tackle problems across the globe, bridging cultures, peoples and genders, can only improve our science.

Further info

Dialogue session with Fischer, Halonen, and young scientists: World Leaders and future thought leaders

Lecture by Her Excellency Tarja Halonen

Note: This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

Altmetrics—measuring impact on an article level

By Luke Kirwan, open access manager, IIASA library.

Who reads your paper after it’s published? Altmetrics (Alternative Metrics) are a recent innovation designed to supplement traditional metrics, such as impact factors, which provide an overview of a journal’s standing within academia. However, these traditional measures are unable to take into account the dramatic changes in publishing and dissemination that the internet has brought.

Altmetrics complement traditional methods by accounting for the broader reach and impact of research online. They achieve this by tracking the digital identifier of research output, through DOIs, PubMed ids, arXiv ids, across a broad range of social media, mainstream media, blogs, policy documents, and reference management tools. This provides an immediate, broader, and more detailed overview of the impact of your publication. Each mention is given a weighted score, so for example a mention in a scientific publication is weighted far higher than a tweet. There are several well-documented studies of the benefits of article level metrics over impact factors.

When we designed the new IIASA Publication Repository (PURE), we incorporated altmetrics, because in today’s world, traditional metrics like impact factors no longer tell the whole story. The concept behind altmetrics is to broaden how impact is measured, beyond simply counting citations. I think of altmetrics and impact factors as two complementary tools with the same goal, but different ways of getting there.

pic1

You can see the citation and usage data for an article in the Altmetric “donut” at the bottom of a record. By clicking on the link to Altmetric.com you can then access more detailed information about that publication’s reach. One of the core concepts behind altmetrics is that people should be able to access every mention that has gone into the weighted score. So by viewing the details for a record on Altmetric.com you can also find out who was tweeting about your paper, what newspaper articles it was mentioned in, who was blogging about it, and any other coverage. This means that you can immediately see the type of attention being generated by a piece of research, where this attention is coming from, what influence it has, and the geographic spread. Altmetrics relies on real-time data so results can be quantified rapidly. The goal is to quickly provide an overview of the diversity of areas where your research is being talked about. It is important to note that altmetrics is only assessing mentions of a research paper—it makes no judgement on quality. So this feedback needs to be read in the context of the quality of the paper and the research. This is partially the reason why Altmetric are so open about what information they use to generate their results.

So what is a good altmetric score? This is not a particularly straight forward question as an altmetric score measures attention, which can be positive or negative. To help you understand your altmetric score select the “Score in Context” option. This will breakdown how the score compares with similar articles, all tracked research, and the output from that journal.

pic2

There are a number of ways you can help boost the attention your research is receiving. The first is to ensure that your research is made available in a repository as soon as possible. The sooner your research is accessible, the sooner it can be used and referenced. The better the metadata for a record the more visible is will be online. Older entries are also accounted for, though of course publications over ten years old do not benefit as much from social media. We are currently working through older entries in PURE to update their metadata. Tweeting and sharing your research will also help, especially if you use the social media analytics to identify who has been using your work in the past and develop connections with them.

A second way to improve your score is to develop your presence on informal social networks, like Twitter, blogs, and more structured academic networks like Google Scholar and Academia.edu. By linking them together you will boost your online presence and help ensure that your research reaches as many people as possible. Altmetrics relies on digital connections between your output, yourself, and your peer network so strengthening these linkages will help improve your visibility.

Altmetrics introduction video https://www.youtube.com/watch?v=6R6WdoQxvUE

OpenAIRE guidelines for researchers https://www.openaire.eu/intro-researchers

Note: This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

Science communication in the age of cat videos

By IIASA Science Writer and Editor Daisy Brickhill.

“The thing about communicating science today is…people can always watch cat videos instead. And let’s face it, some of those clips are really funny.”

Marshall Shepherd, former president of the American Meteorological Society, smiles at the audience of this science communication seminar, aware of the frustrated sighs going on in the room, and in some cases the blank incredulity—people wouldn’t watch cat videos when they could be paying attention to my science, surely?

We are at the AAAS annual meeting, a vast conference with around 10,000 attendees from all walks of life, from toddlers to retired professors. The science presented here is truly diverse, and covers everything from radically successful new cancer treatments, to advances in artificial intelligence, to the IIASA session on how we can hope to achieve all 17 UN Sustainable Development Goals.

Shepherd is speaking of his long career engaging with the public about his work on weather systems and climate change. “Get out of your ivory tower,” he urges all researchers. There are important issues at stake, and what if no one speaks for the scientific evidence?

However, communicating science effectively is not easy. Understanding something does not mean you are automatically good at explaining it. All through academic training researchers learn how to speak to people in their own field, who talk just like them. That’s important, they might be your next reviewer, after all. But it is only one, narrow form; engaging the public requires a high level of understanding, not just of the topic, but of the audience and communication itself.

“We have left behind the old idea of science communication where brains are empty vessels waiting to be filled,” says the next speaker, Barbara Klein Pope, executive director for communications for the National Academies. “They are a swamp, and we need to explore that swamp to communicate properly.” She describes research which tested the effects of different types of communication on people’s perceptions of social science, in terms of whether they felt it was worth funding, for instance (oh yes, I sense the academic ears pricking up now).

swamp

The mind is not an empty vessel waiting to be filled, it’s a swamp to be navigated.

The findings of this work led to a framework of three clear messages. First, use exemplars—a good example can do wonders—yes, your research might be relevant across reams of different cases but general, expansive terms are often vague and a simple example can bring clarity.

Second, the all-important yet surprisingly often neglected question, “Why do we care?” Bear in mind also that it’s not why you care, you’ve made a career out of this science, we know why you care, but why should your audience care.

Finally, use metaphors. Science is often very complex, and pretty much anyone outside your field will need something they can relate to—a familiar concept that they can use to begin to explore the new territory. In case you need more convincing, the use of metaphors was shown to have a significant effect on whether the public felt the work was worth funding.

At the end of that session I was struck by the parallels between this session and another I attended on science-policy interactions with speakers Vladimír Sucha, Daniel Sarewitz, and Peter Gluckman, all working at the forefront of science-policy.

Trust, built on good communication, is vital, the speakers all agreed. Interesting conclusions should not be buried at the end of a report, they should be at the start, just as they would be for the public, and any article or briefing should be kept as short and relevant as possible. Examples and metaphors play a role here too, and a good story with persuasive anecdotes can have much more impact than a dry report.

What not to do, Sucha reminds us, is send an email saying “Here are the links to 200 peer-reviewed papers on this, you’ll find it all there.” After all, policymakers can access cat videos just as easily as the rest of us.

Note: This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.

Turning Vienna into a city of science

Known to the world as a metropolis of music the science in Vienna does not receive the recognition and international visibility its excellence deserves. To change this would require not so much more money but a new mindset, agree two prominent players in scientific research in Vienna: Director General and CEO of the International Institute for Applied Systems Analysis (IIASA) Professor Dr. Pavel Kabat and President of the Institute of Science and Technology Austria (IST Austria) Dr. Thomas Henzinger.

How does Vienna and its scientific research community benefit from the presence of the two institutions and vice versa?

Henzinger:  Vienna is a hub for scientific research in Europe. There are a number of universities and institutions in Vienna and they all have an important part to play in the research ecosystem. In the end this profits everybody because as the critical mass of research grows the easier it is to hire people. It’s like gravity — big centers attract more of the best researchers from around the world. The Science Ball is a — uniquely Viennese — sign of this. We are now firmly “on the map”, and in Vienna you show that by hosting a ball!

Kabat:  I agree. IIASA has a number of fruitful connections with Viennese institutions. For example, IIASA and OäW have worked together to organize a series of public lectures and debates with prominent scientists for the Viennese academic and political community.  Our scientific collaborations with researchers in Vienna and Austria as a whole are also very strong, and have resulted in the publication of over 1050 scientific papers since 2008.

The Science Ball, bringing together Vienna’s diverse scientific community.

Vienna is known as the “City of Music” because of its musical legacy, but why is science not also an important part of the city’s image?

Kabat:  This is something close to my heart. IIASA is doing top-level science on transitions towards sustainability; the world is now at a cross-roads and we need to be taking steps in sectors from energy and water all the way to financial systems. Communicating this can be very difficult, so we are using new and unusual collaborations that are made possible by this fantastic Viennese environment. We are working with music, ballet, and the opera. We have partnered up with the Vienna Philharmonic Orchestra, for example, and with dancers from the State Opera to communicate these complex concepts. Science and the arts both have a vital part to play in Vienna’s past and future. I dream of a scientific tour through Vienna featuring collaborations between theatres, museums, and scientific institutions.

Henzinger:  There is a lot of history between the golden age of science in Vienna and today, and I think there is a large amount of effort and also a lot of progress in reviving Vienna as a city for science. Science by its very nature is one of the most borderless activities of humanity there is and it can only thrive in a completely open environment. It is no surprise that the glory days of science in Vienna were when it was the hub of a multi-national empire. I think we can only get back to that by becoming much more open-minded and much more international as a country.

The city of Vienna is not legally responsible for science funding, but it is a central research hub and the biggest university city in central Europe. What can the city do to improve its image as a center of scientific excellence?

Kabat:  I think a change is needed in the portrayal of Vienna as a whole. There is promotion of music, dance, and the arts. All these are great, but institutions like IST Austria and IIASA should also be used to show that Vienna really is one of the major science hubs of Europe and the world. Emphasizing this would require very little investment but would benefit both Vienna and science in the city. All the components are here, what it needs is a coordinated effort and a vision.

Henzinger:  Vienna has an enormous advantage in that is known as a fantastic place to live. The city needs to actively attract not only world-class researchers but all kinds of science-related businesses and organizations. Vienna as a whole must make concerted effort to advertise itself as an attractive location for students, companies, and professionals from all over the world.

Students do not know that if they come to study at Vienna University, for example, they may also be able to benefit from collaborations with scientists working IIASA and IST Austria, who may be able to advise or even co-supervise them. This dynamic and varied environment is a key part of what Vienna can offer, not only the individual institutions. The ball is the perfect step in that direction. It is very clearly an effort that transcends any particular institution.

Kabat:  We should continue this talk, not just with the two of us but with all leaders of Viennese scientific institutions, and the mayor, to have a free and frank discussion. Science brings a huge amount to the city of Vienna and it should be recognized. The ball, as you say, is an excellent occasion to bring together Vienna’s vibrant scientific community and celebrate it!

Science for policy’s sake

By Daisy Brickhill, IIASA Science Writer and Editor

“In some senses, the science-policy process can be likened to a sausage being made,” said Dr E. William Colglazier in his lecture at IIASA this week. We could take this in different ways: that it is messy, perhaps, or that things get churned around or made to fit. But the most important parallel must surely be that if it is done right it can bring huge benefits. In this case not for your taste buds, but for humanity.

sausage2

It’s a messy business, but it’s surely worth it if we get it right: comparing the science-into-policy process with the art of sausage making.

If anyone knows what the science-policy process is like, it is Colglazier. Soon after completing his PhD in theoretical physics in 1971 he became a fellow of the American Association for Advancement of Science, providing advice for policymakers. He has been at the forefront of the science-policy interface ever since, and is perhaps best known for his role as the Science and Technology Adviser to the US Secretary of State from 2011 to 2014.

During his lecture Colglazier explored how scientists can best advise policymakers. “There’s an old joke: if a policymaker asks a scientist what time it is, the scientist will tell him how to build a watch.” To avoid this, Colglazier says, scientists need a fundamental understanding of both the needs and time frames of policy making.

The best way to achieve this is to engage. A scientific advisor is not a distant voice, hovering between knowledge and policy. Scientists must not be afraid to get involved; it is only through lots of interaction with policymakers that they will begin to understand what is needed. “Scientific advising,” says Colglazier, “is a contact sport.”

For their part, policymakers do not always understand the scientific process. To prevent this causing misunderstandings, scientists must be clear about the uncertainties in the science, and what it can and cannot say. They must explain exactly how the evidence leads to the recommendations they have given.

Colglazier also emphasized the importance of communication, something I silently cheered for, as a science writer often hoping to reach policymakers. “Telling a good story with persuasive anecdotes is often more influential than a dry, hundred-page report,” he said, and I couldn’t agree more. I have seen articles about accurate, rigorous, and important science drift by unnoticed where others, based on a more trivial studies, spark debate and engagement. The difference is often that those in the former category are three pages longer, full of impenetrable jargon, and bury their juicy conclusions at the bottom.

_MG_9269

Dr E. William Colglazier giving his lecture at IIASA this week as part of the first joint JRC-IIASA summer school on evidence and policy.

Sometimes, scientists are asked to advise on issues that go beyond science, straying into value judgements. For example, when assessing an environmental risk a scientist can give the numbers and the uncertainties and information on the consequences. But they cannot provide a definitive answer to the question that the policymaker is really asking: how safe is safe enough?

Does that mean scientists should steer clear of this territory entirely? No, says Colglazier. “Feel free to give advice when you are asked, but be honest about what the science can say.” The important thing is to remember that scientists have no special expertise when dealing with value judgements.

Ultimately, the science-into-policy process is a messy one. Scientists find it difficult to grind up the prime fillet steak of their data into the mincemeat needed for policy making. But the importance of this step should not be underestimated. Science and policy must work together if we are to achieve a sustainable future for humanity.

 

Note: This article gives the views of the author, and not the position of the Nexus blog, nor of the International Institute for Applied Systems Analysis.