If you can’t get enough of prediction by forecasting weather or climate–or basketball tournament brackets, elections, and the Oscars—here’s the game for you: Try the web-based opinion poll of climate and earth scientists—the Vision Prize.
A nonpartisan research project using the Web for incentivized polling, the Vision Prize is testing a new way to get scientists to speak candidly to the public about climate change, without media filtering.
“We all recognize the need to do science communication, but we still seem to struggle to do this well,” says Jonathan Foley, Director of the University of Minnesota Institute on the Environment. “As a new approach to this problem, Vision Prize deserves our attention.”
Foley’s institute is collaborating with Carnegie Mellon University to try out this poll-based experimental method. The advisors include behavioral psychologists, economists, and public policy scholars, so participating scientists may feel a bit like the tables have been turned, with humans becoming the lab rats on the treadmills.
Don’t let that stop you, however. Participants are having fun at it, getting serious results, learning about their colleagues, and yes—there’s that “incentivized” part—they’re winning prizes, too. Participants not only make gut-check projections about future climate but also predict what their colleagues think—a way for the rest of us to gauge their confidence and consensus.
Surprisingly, so far the results show that experts systematically underestimate the consensus amongst their peers. For example:
(“Expected Answers” are predictions of what the most likely answers were; “Actual Answers” are predictions of climate made by those surveyed.) The participants who best predict their colleagues’ opinions win gift cards towards the charity of their choice. All in all, scientists should feel right at home doing public service to win a chance to do even more public service.
Poll results are annotated by a measure of the stature of the participants within the climate science community. The web site provides h-index scores that factor both how many papers participants have published and how often those papers are cited. In a press release today, Vision Prize noted that the polls have been taken by some impressive participants.
“We’re very encouraged by the high quality of our 275+ expert participants,” says Peter Kriss, the director of research. Vision Prize provides mean h-index scores to give readers of the poll an approximate metric for assessing the relative expertise of the participants who selected a given answer. “We were very impressed to find mean h = 36 among our top 50 experts,” says Kriss. As a point of reference, h ≈ 12 might be a typical value for advancement to tenure at major research universities; membership in the U.S. National Academy of Sciences may typically be associated with h ≈ 45 and higher (Hirsch, PNAS, 2005).
The mean for all participants so far is h=13, but that may change soon.Vision Prize is now actively seeking additional participants from the scientific community, including doctoral students. “The larger the number of climate and earth scientists participating, the more useful the results,” says Vision Prize Managing Director Mark Kriss.
Vision Prize makes its results open to the public, so it’s certainly imaginable that not only are policy makers and pundits paying attention but also some prediction obsessed spectators who themselves are predicting how the polls turn out. The web site is currently polling about a round of topics that began in September 2012 and will wrap up at midnight, April 30th.
If you qualify, by all means get into the game as a participant as soon as possible. Answering the survey questions takes about five minutes, after quick registration. Mark Kriss, Vision Prize managing director, says, “Early voting helps boost participation rates so your post would be timely now. The larger the number of participating scientists, the more useful the results.”
By the way, even if you don’t end up participating, the Vision Prize is also open to suggestions about questions to ask the experts.