The Climate Collaboratorium: Project Overview

Full text

The Climate Collaboratorium:
Project Overview
Thomas W. Malone, Robert Laubacher, Josh Introne, Mark Klein,
Hal Abelson, John Sterman, and Gary Olson*
MIT Center for Collective Intelligence
MIT Center for Collective Intelligence Working Paper No. 2009-03
September 2009
MIT Center for Collective Intelligence
Massachusetts Institute of Technology
http://cci.mit.edu
* University of California Irvine

2
SUMMARY
The goal of the Climate Collaboratorium project is to harness the collective intelligence of
thousands of people around the world to address one of the most important problems confronting
humanity today: global climate change. Inspired by systems like Wikipedia and Linux, the project will
develop a global, on-line forum in which people can create, analyze, and ultimately select detailed plans
for what we humans can do about global climate change.
At the core of the system will be an evolving collection of user-created plans based on
computational models of the actions humans can take and the predicted impacts of those actions. For
instance, the actions in a plan might include reducing greenhouse gas emissions by various amounts or
implementing policies such as a carbon tax. The impacts might include concentrations of carbon dioxide
in the atmosphere, temperature changes, and economic costs. Users can also debate plans by entering
arguments for or against the various plans or by entering other comments about the plans’ plausibility and
desirability. And users can rate the credibility of various arguments and vote on the plans and positions
they find most feasible, credible, and desirable.
Just as important as developing the software capabilities, the project will also develop the
community and organizational processes to use this software, and empirically evaluate the actual use of
the system.
This paper provides an overview of the project’s plans and current status.
Anticipated contributions
At a minimum, this work will provide lessons about how to design large-scale collective decision-
making tools that integrate computer modeling, on-line debates, and group voting and rating. We believe
such tools will be useful for a wide range of societal and managerial problems.
More specifically, this on-line forum will also help educate the general public about the real issues
involved in global climate change. And most importantly, by constructively engaging a broad range of
scientists, policy makers, business people, and ordinary citizens, this forum may help develop plans and
policies that are actually better than any that would have otherwise been developed.
If the project is successful in its grandest ambitions, it will lead to the creation in a few years of a
societal institution that is comparable to (though perhaps somewhat smaller than) Wikipedia. It will be an
on-line community used routinely by scientists, politicians, journalists, and anyone with a professional
interest in climate change. It will also be a standard resource for educators, students, and any citizens who
want to learn about climate change. And it will be a place where any citizens who care can either express
their opinions about climate change directly, or delegate their "proxy" to others who will vote on their
behalf.
In short, it could become a combination of a kind of simulation game for climate change, a
Wikipedia for controversial topics, and an electronic democracy on steroids.

3
1. Introduction
Many people believe that global climate change is one of the most important problems currently
facing humanity. This problem is also unusual in the degree to which it is truly universal: it affects every
one of us and is affected by all of our actions. If ever there were a challenge that called for harnessing the
best collective intelligence our species can muster, this may be it.
Left to their own devices, scientists, journalists, politicians, businesses, and consumers will
certainly do something about this problem. But the inefficiencies, delays, and distortions of traditional
mass media, political decision-making, markets, and scientific publication mean that the results will
almost certainly not be as good as we might hope.
Fortunately, however, in just the last decade or so, a new way of solving global problems has
become possible. Examples like Wikipedia and Linux illustrate how it's now possible to combine the
work of thousands of people in ways that would have been impossible only a few years ago.
We believe this new approach can be used to help solve the problem of global climate change, and
the goal of this project is to create the foundations for doing so. We seek to develop a global, on-line
community—called the Climate Collaboratorium—in which thousands of people around the world will
create, analyze, and ultimately select detailed plans for what we humans can do about global climate
change.
At a minimum, this on-line forum can help educate the general public about climate change issues.
But we believe there is also a real chance it can facilitate a more productive global conversation than
would otherwise have occurred among scientists from different disciplines, policy makers, business
people, and ordinary citizens of countries around the world. In particular, we hope that with thousands of
people constructively involved in the conversation, the plans and policies that emerge from this process
will be better than anything we would have developed otherwise.
Understanding how to structure software tools and human communities to have productive
conversations like this involves core research questions in collective intelligence. Climate change is an
example of a broader class of important societal and managerial decisions involve that are sometimes
called “wicked problems.” In wicked problems, people disagree about how to formulate the problems
and about which answers are best (see, e.g., Churchman, 1967; Rittel & Webber, 1973; Conklin, 2005).
While computational models of costs and other factors are often important in solving these problems, the
models alone are not enough. A network of people must also be involved to refine the models, analyze
model results and other information, and ultimately make decisions.
When the people involved in making these decisions can only use communication tools like face-
to-face meetings, telephones, and paper-based communications, it is very difficult to have more than a
few people deeply engaged in the analysis and decision-making. But a new generation of communication
technologies based on the Internet are now making it possible to develop collective intelligence systems
that deeply engage far larger groups of people in intelligently analyzing and making complex decisions.
The goal of this project is to develop such a system in the domain of global climate change, but we
hope that the lessons we learn in doing so will also be relevant for many other important wicked problems
such as healthcare, education, and business strategy.

4
2. Our approach
At the core of the system we hope to create is an evolving collection of user-created plans for what
we humans can do about global climate change. For example, in the early prototype system we have
already developed, the plans address the question of what agreement should emerge from the UN climate
meeting in Copenhagen in December 2009 (see Figure 1). Plans are based on computational models that
include both actions humans can take and the predicted impacts of those actions (see Figure 2). For
instance, the actions in a plan might include reducing greenhouse gas emissions by various amounts,
adopting particular technologies, or implementing policies such as a carbon tax. The impacts of a plan are
predicted by the computational models and might include factors such as concentrations of carbon dioxide
in the atmosphere, costs of mitigation, temperature changes, sea level changes, or other impacts on water
supplies and agriculture.
Figure 1: Users of the Collaboratorium can inspect and vote for competing climate change plans.

5
Figure 2: Users can create new climate change plans by specifying plan parameter values, and get
immediate feedback on the impact of those plans.
Users can also debate plans by entering arguments for or against the various plans or by entering
other comments about the plans’ feasibility and desirability (see Figure 3). By emphasizing plans that
include quantitative models of impacts, the system encourages people to focus their discussions on issues
that have substantial impacts and that are consistent with the actual constraints of physical and economic
reality. At the same time, by including qualitative debates about plans (and other issues such as model
assumptions), the system allows for much richer kinds of interaction than can be enabled by simply
sharing models alone.
Finally, users can rate the credibility of various arguments and vote on the plans and positions they
find most feasible, credible, and desirable (see Figures 1 and 3). We expect that for some issues (such as,
for example, appropriate parameter values for the reflectivity of polar ice caps), only the votes and ratings
of experts will be relevant. But for other issues (such as the overall desirability of a plan), the votes of all
users of the system are probably important. By allowing this kind of fine-grained voting and rating, the
system helps the whole community of users focus on the plans that seem most promising based on both
the best available evidence from a wide range of experts and the value judgments of a wide range of
citizens.

6
Figure 3: Users can debate the merits of different plans by entering arguments for or against the plans.
They can also enter arguments that agree or disagree with other arguments, and they can rate the
credibility of different arguments.
To make this approach even more concrete, please see the Appendix for imaginary stories (or “use
cases”) about how different kinds of people—a scientific expert, a policy advocate, a high school student,
an open source software developer, and a congressional staffer--could use the Climate Collaboratorium in
different ways.
These stories illustrate how it is possible to synergistically combine the strengths of human and
automated intelligence to create a social-computational system where people define and evaluate the
alternatives to be considered, and computers do the rapid inferencing needed to assess the consequences
of each candidate alternative. Furthermore, the computational system automatically integrates inputs
from many different people. For instance, physicists who study the upper atmosphere, chemists who
study the ocean, and economists who study taxation can each input parameters relative to their own
expertise and the system can rapidly compute the combined implications of all these assumptions.
Finally, the computational system allows people to easily to share alternatives, debates, ratings, and votes

7
so the community can efficiently consider opinions and information from far more people than would be
possible with previous communication methods.
2. Research challenges
To implement this approach, several kinds of software capabilities are needed, including (1) easy
access to computer modeling (to represent actions and predict their impacts), (2) on-line debates (to
debate the relative plausibility and desirability of various plans and positions) and (3) collective decision-
making (to vote for and rate various plans and positions). Also, just as important as developing the
software capabilities will be (4) developing the community and organizational processes to use this
software, and (5) evaluating these technological and organizational innovations in the context of real-
world climate change deliberations. In the following sections, we discuss each of these research
challenges in turn.
2.1. Computer modeling
We use computer models to predict the quantitative impacts of the actions in different plans. In the
field of climate change, there are already a number of such models (e.g., Nordhaus 1994; Dowlatabadi
1995; Weyant 1999; Fiddaman 2002; Sterman 2002; Prinn et al. 1999; Sokolov et al. 2009). These
integrated simulations include economic and social factors such as population levels, transportation usage,
and energy intensity, as well as physical factors such as carbon emissions, ocean chemistry, and the
effects of atmospheric carbon on temperatures at the earth’s surface.
In most cases, the simulation models that have been developed so far have been written and used by
small groups of people who decide what assumptions to make, what possibilities to examine, and how to
interpret the results. Few people other than the model creators even understand the assumptions the
models make, much less their implications. And even in cases where the models themselves have been
widely disseminated, it has proven difficult to learn from or integrate the results of the diverse
experiments different people have done with the same models.
While these modeling efforts have been of great value, another complementary approach to modeling
is now becoming possible. This new, Internet-enabled approach to modeling has the potential to
dramatically increase the number of people who can be constructively engaged with the models. We call
this new approach radically open computer modeling. Two key elements are needed for radically open
computer modeling: The models must be accessible to and modifiable by many people.
We plan to use a phased approach to incorporating such radically open models into our system. In
the first phase, which we have already achieved in our prototype system, users can create and store
multiple plans based on different inputs for the same model. The model we use for this purpose in our
prototype system is the C-LEARN model, a Web-accessible version of the C-ROADS model (Sawin et al.
2009). This model allows users to specify several simple parameters for emission reductions and
sequestration in different regions of the world, and it then predicts carbon concentration and average
global temperature rise.
In the second phase, users will be able to choose among a growing set of alternative models to
specify their plans. For example, in some cases, we will link our system electronically to other models
available over the Internet so users of our system can easily specify and store plans that use results from
those other models. We also expect to include a capability that makes it easy for users to specify new
models using spreadsheets. One particularly appealing way of using this capability will be to create
simplified spreadsheet models that are essentially “response surfaces” based on previous results from
large-scale models like those listed above. In this way, users of our system can get fast, approximate
predictions by interpolating among the results of previous runs of much more complex models.
In the third phase, users will be able to combine models in limited ways, using the outputs of one
model as the inputs to another one. For instance, one model might use assumptions about economic
policies and technology development to predict emissions from different geographical regions. Then the
results of this model might be used as input to another model (like C-LEARN) to predict the effects of
these emissions on carbon concentration and temperature change. In the coming year, we expect to have

8
limited capabilities for this kind of model recombination and to develop a more general framework that
will allow more powerful kinds of model combination in future years.
For example, at a minimum, it will be important to have facilities for making the different software
modules “plug compatible.” More subtly, it will also be desirable to provide functionality that helps users
detect when two modules make different substantive modeling assumptions and so cannot be sensibly
combined, even if they are technically “plug compatible.”
Other research challenges that may be addressed in this area include:
(a) How can we display the results of complex climate models in a way that is accessible to as
broad a community as possible, including non-scientists (Danis et al, 2008)?
(b) How can we appropriately represent the uncertainty in model outputs, visualize multi-
dimensional data, and do sensitivity analyses over model inputs?
(c) How can "radically open modeling" work as the number, types, and locations of models
increase? For instance, how can acceptable response times be maintained if the models are
hosted on different machines?
Even though this area of radically open computer modeling provides opportunities to engage a
number of challenging research questions like these, it is important to realize that the overall success of
the project does not depend on answering all these difficult questions. We believe that the overall project
can be successful even with only the early computational capabilities planned for the second phase (a
limited number of separate, simple models). Even if no other computational tools are available to help,
the social network of people can deal with all the other complexities about model compatibility,
sensitivity analysis, and so forth.
2.2. On-line debates
For wicked problems like climate change, it is unlikely that any single model or combination of
models will reflect all the issues and points of view that need to be considered in making a decision.
Instead, it is important to be able to capture and share the different perspectives and arguments relative to
different aspects of the decision.
For example, in some cases, it is important to represent debates about how the world works as
reflected in specific model assumptions for factors like the rate at which the ocean absorbs carbon from
the atmosphere or the likely rate of energy technology innovation in future decades. In other cases, it is
important to represent debates about the human values to be used in making decisions. For instance,
different people have different views about how much developing countries should be expected to reduce
their emissions relative to developed countries that have been emitting large amounts of carbon for
decades.
There are already a number of on-line tools for gathering and sharing knowledge and opinions
from large numbers of users about topics like these. For example, web forums and blogs are good at
eliciting a broad range of opinion and perspectives, but they are often unruly and difficult to comprehend.
Wikis, by contrast, are often easy to comprehend, but they frequently flatten out the range of perspectives
that can be expressed. And “argument maps” have some of the benefits of both forums and wikis: They
allow large numbers of users to express different perspectives, but they lay out, in a clear and succinct
way, the major positions and arguments on each of the key issues (e.g., Conklin and Begeman 1988;
Klein and Iandoli 2008).
In the Climate Collaboratorium, we plan to experiment with a variety of these approaches,
including threaded discussions, wikis, and argument maps, with special emphasis on incorporating
innovations in large-scale argumentation (see, e.g., Klein and Iandoli 2008).
Possible research questions to address in this area include:
(a) How can moderators (equivalent to the Wikipedia stewards) best contribute to keeping the
debates useful and constructive? For instance, should all new contributions be immediately
visible to the public and then become invisible only if a moderator later removes them? Or
should all new contributions have to be approved by a moderator before becoming visible to the
public? Or should both of these methods be used in different parts of the system?

9
(b) How can belief (or confidence ratings) be automatically propagated through argument maps in
ways that people find useful and intuitive? (e.g. Introne, 2009; Lowrance et al., 2008)?
(c) What tools can effectively visualize large argument maps in ways that are understandable and
engaging? (Conklin and Begeman, 1988; Conklin, 2005; Klein and Iandoli, 2008)?
As with the research questions in computer modeling, it is important to realize that the overall
system can be successful even without extensive or innovative solutions to these research questions. We
believe that even simple threaded discussions or wikis would provide enough value to be useful, but we
believe there are also promising opportunities to provide much more valuable functionality.
2.3. Collective decision-making
A system that includes lots of radically open models and on-line debates could help people explore
issues in depth, but it could not facilitate actual decisions without one more key element: collective
decision-making tools. Therefore, the third key element of the Collaboratorium involves users voting to
express their opinions and preferences. At the highest level, users will vote on the plans that are proposed.
But they will also be able to vote on sub-elements of plans and weigh in on which goals they find most
desirable, which models and model assumptions they find most plausible, and which arguments they find
most compelling.
The current version of our system already includes simple voting and rating functionality, and we
believe that the system could be successful with only these simple capabilities. However, we expect to
extend the collective decision-making functionality in various ways over the course of the project. For
instance, possible issues to be explored include:
(a) What practical methods will work to authenticate voters so that (i) it is difficult for people to
vote multiple times, and (ii) votes can be credibly tallied separately for different kinds of people
(e.g., scientists on scientific questions and citizens on questions involving national interests)?
(b) How can we allow people to give proxies for their votes to people they trust in a fine-grained
(issue- and time-specific) way, so that specialized knowledge can be applied as needed and
ordinary people don’t have to spend all their time voting?
(c) How can voting systems be designed to avoid intentional manipulation of outcomes by small
numbers of highly motivated partisans?
(d) How can voting systems be designed to avoid undue influence of the results by the opinions of
early voters (Salganik et al., 2006)?
(e) What kinds of voting (including majority voting, plurality voting, and various kinds of
preference ordering) are most appropriate for different kinds of issues (Levin & Nalebuff,
1995)?
2.4. Organizational design and community development
Perhaps the largest challenges with launching the Collaboratorium are not technical, but rather,
organizational and social. We will need to attract a community of users that is engaged around the climate
issue, incentivize them to participate as volunteers, and establish appropriate rules and structures for their
interactions.
For example, one possible way of structuring the community might distinguish the following roles
for participants:
(1) Readers – Users who view the site but do not contribute anything
(2) Voters – Users who vote on models and debates, but do not make any other contributions
(3) Contributors – Users who vote, participate in debates, suggest plans, and add or refine models
(4) Moderators – Graduate students, young professionals, and others who help review and maintain
the quality of the models, plans, and debates. (Moderators have special system privileges.)
(5) Scientific Council – Scientific and other experts who advise Moderators,
(6) Scientific Advisory Board – well-known experts who provide overall guidance for the project
and help recruit members of the Scientific Council.

10
To recruit users, we are recruiting a cadre of early Contributors and Moderators, and in parallel
with this, we are also forming a Scientific Advisory Board of distinguished experts in various scientific
disciplines. We then anticipate reaching out to existing groups with an interest in climate change—
including scientific, educational, and political organizations, as well as on-line communities that
congregate around popular blogs and Web forums on relevant topics. Because climate change is a global
issue, a particular challenge for us will be to attract a critical mass of users from outside the U.S.
As an example of the kinds of organizational processes needed, one possibility we are considering
is to have weekly “contests” during which users refine plans for, say, Monday through Friday, and then
vote on the plans over the weekend. We expect that this weekly rhythm might help maintain excitement
and commitment over time.
More generally, we expect to explore issues like the following:
(a) What kinds of social, psychological, economic, and other incentives will effectively motivate
the different kinds of participants needed (scientific experts, policy makers, climate activists,
businesspeople, software developers, ordinary citizens) (Preece & Shneiderman, 2009)?
(b) What kinds of community norms and rules will lead to effective interactions (Olson,
Zimmerman, & Bos, 2008)?
(c) Can playful motivations (like contests) be used to attract more people? Will the inclusion of
media such as user-created video and fan fiction help make the site more engaging?
2.5. Empirical evaluation
The primary initial focus of this project is to create the software capabilities and user community
described above, because without this, the phenomenon we want to study will not even exist. But a
secondary focus of our initial work is to do some preliminary empirical evaluation of the system we
create. For example, we may do preliminary analyses of the following types:
(a) Activity log analysis. The system will automatically log user actions (such as changes to plans,
models, debates, and votes), and we intend to analyze the patterns in these kinds of user
behavior. For instance, similar to studies of Wikipedia by Viegas and colleagues (Viegas,
Wattenberg & McKeon, 2007; Viegas, 2007, Viegas, Wattenberg, Kriss, and Ham, 2007), we
will investigate questions such as: What kinds of actions precede significant numbers of vote
changes? To what extent does the user community act like stable, non-overlapping factions,
and to what extent is there more fluidity in opinions and collaboration?
(b) Demographic analysis. We expect to give users the option of providing various kinds of
demographic data on the website (such as occupation, age, country of residence, and political
affiliation). For users who voluntarily provide this information, we will then be able to analyze
questions like: How are the models used and by whom? Who makes what kinds of
contributions to discussions? What does voting behavior look like for different kinds of users?
(c) Surveys. Users who register on the website will provide their email addresses, and we expect to
conduct occasional email surveys using these addresses. Questions might include how users
found out about the Collaboratorium, what effect it’s having on their views of the climate
debate, and what activities regarding the climate debate the Collaboratorium has triggered.
Along the same lines, we may also engage some non-users to get background information about
the general state of awareness regarding climate issues.
(d) Interviews. We also expect to conduct interviews with selected users to collect more in-depth
information than can be obtained in a survey. The samples of users to be contacted about
participating in such interviews will be constructed based on demographic and other
information about user activities on the site. It would be interesting for example, to interview
users who are in policy-making positions about what effect, if any, the Collaboratorium has had
on their activities.
(e) Usability testing. Finally, since this will be a relatively complex site and could present users
with a numbers of challenges, we expect to do various kinds of systematic usability testing
(such as “think-aloud protocols”) to help make the site more understandable and usable.

11
3. Project plan and risks
3.1. Software and community development plan
The system we propose to build is an example of a “crowdsourced” system (see, e.g., Howe, 2005;
Malone, Laubacher, & Dellarocas, 2009; Kazman & Chen, 2009). As such, some traditional software
development techniques are not relevant. For example, as Kazman and Chen (2009) say, “One cannot
conceive of a crowdsourced system’s functionality in terms of ‘releases’ any more than a city has a
release.”
In the spirit of such crowdsourced systems, we believe that a critical factor in the success of this
project will be the degree to which we can use an iterative development methodology to co-evolve the
software capabilities along with the community. In other words, we believe that the choices of when and
what functionality to add should be heavily influenced by the users of the system.
Thus, rather than specifying our project plan in terms of specific system capabilities to be
developed at specific times, we believe it is more useful to suggest a broad set of directions for feature
evolution (as we have done above), and then specify our project plan in terms of targets for the numbers
of users the system will attract and the kinds of influence it will have.
For instance, if the project is successful, it should involve at least hundreds of people in the first
year, at least a thousand people in the second year, and eventually, perhaps, tens of thousands more. We
also believe it is important that the users include a critical mass of legitimate experts in climate-related
fields and substantial representation from non-US participants. Finally, a measure of success of the
system will be the degree to which it influences important policy decisions related to climate change. For
example, one kind of evidence of success on this dimension would be news articles or other quotations
from influential policy makers showing that the Climate Collaboratorium had an important influence on
international treaty negotiations or legislative actions by specific countries.
3.2. Risks
The practical goals described above for the Climate Collaboratorium are very ambitious for a
university-based research project. We realize that there are numerous potential barriers to the achievement
of all these goals, including many kinds of technical, organizational, financial, and political risks. But we
believe that the problem of global climate change is important enough to warrant proceeding, even though
the ultimate practical success of the project is by no means guaranteed.
It is also important to realize that the project is designed in such a way that even if it does not
succeed in its grandest practical ambitions, many important scientific and educational benefits can come
from quite limited progress along the way. For instance, if we are unable to attract sufficient users to the
site for it to be influential, our empirical evaluations should help identify the motivational factors that did
and did not work. And we hope that lessons like this can help us or others design even better systems in
the future for facilitating large-scale model-centric collective decision-making in climate change or other
areas.
4. Anticipated results of the research
4.1. Intellectual contributions
The primary intellectual contributions of this work will be lessons about how to design large-scale
model-centric collective decision-making tools and the communities to use them. In particular, we hope
to understand much more deeply how to combine three capabilities that have each been used separately,
but never to our knowledge, been combined: computer modeling, on-line debates, and group voting and
rating.
Existing collective intelligence technologies such as email, blogs, instant messaging, wikis, web
forums, voting, and even argumentation systems, have enabled unprecedented opportunities for
information sharing, but the users of such systems are still, in a sense, “just talking.” They might, for

12
instance, spend huge amounts of time debating issues that are actually trivial in the overall scheme of
things. And they have no systematic way of understanding how positions on different issues interact with
each other. For example, will the potential benefits of energy-saving light bulbs in US homes be
completely overwhelmed by coal-powered electric plants in China? And if people telecommute from
home more often, will the reduced carbon emissions from less commuting be outweighed by increased
emissions from more home heating? What makes our approach unique is that it explores how the
collaborative decision-making can be mediated via the use of models that allow the community to focus
on exploring the solution alternatives that have a real chance of being viable and attractive. While the
proposed work will be applied initially in the climate change realm, it should develop ideas that will be
applicable to a very wide range of collective problem-solving tasks.
4.2. Broader impacts
Public engagement with science. In addition to merely educating the public about scientific research,
this project has the potential to actively engage the public in the process of applying scientific research to
social decision-making. This public engagement with science has the potential to provide numerous
benefits, such as improving public attitudes toward science and increasing the quality of scientifically
informed social decision-making (Leshner, 2003).
Benefits to society. Perhaps the most important impacts of this work will be its benefits to society as
a whole. At a minimum, even if the project fails to achieve its grandest practical ambitions, it will
provide useful lessons for designing collaborative tools to help solve a wide variety of wicked problems
in society and in businesses.
And if the project is successful in its grandest ambitions, it will lead to the creation in a few years of
a societal institution that is comparable to (though perhaps somewhat smaller than) Wikipedia. It will be
an on-line community used routinely by scientists, politicians, journalists, and anyone with a professional
interest in climate change. It will also be a standard resource for educators, students, and any citizens who
want to learn about climate change. And it will be a place where any citizens who care can either express
their opinions about climate change directly, or delegate their "proxy" to others who will vote on their
behalf. In short, it could become a combination of a kind of simulation game for climate change, a
Wikipedia for controversial topics, and an electronic democracy on steroids.
For further information
Additional information about this project is available on the project web page:
http://cci.mit.edu/research/climate.html.

13
APPENDIX
Imaginary stories about different kinds of Climate Collaboratorium users
1
Imagine a possible future a few years from now, when the Climate Collaboratorium, is being used
by thousands of people around the world on all sides of the issues--experts, policy analysts, legislators,
and concerned citizens--to collectively develop and debate scenarios for what to do about global climate
change. And consider the following stories about how different kinds of people might participate in this
community:
A scientific expert
As a graduate student in MIT Prof. Susan Lim’s lab, one of Steve McKinnon’s responsibilities is to
report the lab’s new results on the Climate Collaboratorium website. Today is an especially good day for
the lab. Nature magazine has just published an article from Lim’s lab estimating that the absorption rate
of carbon dioxide in the Southern Ocean is a surprisingly high 0.45 gigatons of carbon per year.
Steve goes to the section of the Collaboratorium website that summarizes the key arguments about
many issues related to global climate change. He finds the issue called “What is the carbon dioxide
absorption rate in the Southern Ocean?” and adds a new position stating the new result. Then, to support
the new position, he adds an argument that briefly summarizes the Nature article and includes a link to its
on-line version. This new result and its backing are now easily available to anyone in the world who
cares about this issue.
The positions that people had entered previously for the issue almost all had lower values than the
new position. Most of the positions had been rated by a group of expert ocean scientists, including Prof.
Lim, and the most highly rated previous position (with endorsements from 39% of the experts voting) was
an absorption rate of 0.33 gigatons of carbon per year. Because of the reputation of Lim’s lab and the
precision of their new measurement technique, Steve is confident that many of these experts will change
their ratings over the next few weeks, and the new position he has just entered will become the most
highly rated one.
Steve could stop there for today, but he is curious how much effect this new result will have on the
overall climate situation. To test this, he goes to another part of the Collaboratorium that shows complete
scenarios for what will happen under different policy choices and other assumptions. He picks two
widely followed scenarios: (1) the “Business as usual” scenario maintained by the MIT Program on the
Science and Policy of Global Change to reflect what will likely happen over the next 100 years with no
significant changes in current policies and lifestyles, and (2) the “Sierra Club” scenario, maintained by the
Sierra Club to reflect a set of assumptions about aggressive policy and lifestyle changes by people and
governments around the world.
In both scenarios, Steve goes to the issue for which he just added a new position, and indicates that
he wants to adopt the new position. This automatically creates two new scenarios (shown as “children” of
the scenarios with which he started). Then Steve clicks the “Simulate now” button and watches the new
results appear. Even though Steve had hoped for a more dramatic change, the new simulations both show
decreases relative to the starting scenarios of about .03 degrees Celsius in the global average temperature
in 2100, with corresponding improvements in global economic output, adverse weather events (like
hurricanes), and estimated quality of life.
Of course, the “Simulate now” option uses many rough approximations in its calculations. Full-
scale simulations, using much more detailed calculations, can take days to run even on today’s computers.
But Steve thinks these new results are promising enough that he clicks the button to “Request full
simulation” for both new scenarios. He is pretty confident that at least one of the leading groups with
full-scale simulations will be intrigued enough to run his new scenarios.
1
This Appendix is adapted from material in Malone and Klein (2007). The individuals and events described in
these stories are fictional, but the organizations named are real.

14
An advocate
Sarah Schmitt is the Director of Policy Analysis for the United States Climate Action Partnership
(USCAP), an alliance of businesses and environmental groups including Boston Scientific, BP, General
Electric, General Motors, and Shell.
Starting about 18 months ago, Sarah and her staff encouraged the creation of a set of detailed
scenarios in the Climate Collaboratorium showing the consequences of USCAP’s proposed program for
capping emissions and then letting organizations buy and sell emission permits among themselves (“cap
and trade”). Most of the people working on these scenarios were not actually employees of USCAP.
Some were employees of USCAP member organizations paid to do this as part of their job, but many
were volunteers--students, retired scientists, environmental hobbyists, and Wikipedia contributors—
attracted to the USCAP scenarios because they seemed particularly promising.
The scenarios included detailed scientific assumptions about population growth, emission rates for
different kinds of vehicles, upper atmosphere chemistry, and so forth. But in most cases, the people
developing the USCAP scenarios were not computer programmers or climate change scientists
themselves. Instead, they were able to build their scenarios by just selecting combinations of positions
which had already been entered by specialists in different disciplines.
Importantly, the simulated results of the USCAP cap-and-trade program were good under all the
plausible combinations of different scientific assumptions. In fact, after USCAP endorsed these
scenarios, over 80,000 other Collaboratorium users also endorsed them as their preferred alternatives.
Despite many conversations over the years, the US Chamber of Commerce is still not a member of
USCAP. In fact, the Chamber of Commerce has tried to represent the interests of its many small-business
members by developing a competing set of scenarios in the Collaboratorium. Today Sarah’s staff has just
shown her a new set of scenarios that they think incorporate the key elements of both the Chamber of
Commerce and USCAP scenarios.
After seeing the new scenarios, Sarah decides to recommend to her boss that USCAP propose these
new scenarios to the Chamber of Commerce. Of course, there will be more negotiations about this. But
Sarah is convinced that some form of these new scenarios will be enough to bring the Chamber of
Commerce on board.
And when that happens, the Chamber of Commerce will bring with them the endorsements of over
50,000 of their members who have given their Collaboratorium proxies to the Chamber of Commerce.
With these new endorsements, the USCAP scenarios will become—at least for a while—the most highly
ranked scenarios in the entire Collaboratorium!
A high school student.
Dinesh Rao is a high school student in Mumbai, India. He first heard of the Climate
Collaboratorium in his honors science class last year. One of the class’s homework assignments was to
use a simple entry-level simulation in the Collaboratorium and find a combination of parameters that
would reduce atmospheric concentration of carbon dioxide to less than 350 parts per million by the year
2100.
Dinesh’s teacher had also asked them to study the most highly rated arguments on both the “yes”
and “no” positions for the issue “Should we rely on unknown future technological advances to solve
climate problems that would be expensive to address today?” Unlike many of the positions in the
Collaboratorium, this one is a philosophical issue, not directly tied to specific parameters or other
simulation elements in particular scenarios. But this issue has attracted a lot of attention, and the
arguments on both sides of the issue are very well developed.
Dinesh found the climate change work so interesting that this year he joined a student team in the
FIRST Climate Change competition. (FIRST is a US-based non-profit organization founded to inspire
young people's interest and participation in science and technology. For years, it has organized
competitions where teams of high school students build robots, and it has recently started a new
competition based on the Collaboratorium.)

15
The goal of this competition is to come up with climate change scenarios that are both plausible and
desirable. To be plausible, a scenario has to use positions that have probabilities of at least 50% as rated
by FIRST-approved experts in the relevant subjects. To be desirable, a scenario should have as little
increase in global average temperature as possible, while also having as high a value as possible for
global economic output and estimated quality of life.
Dinesh’s team is working on a detailed scenario that includes massive use of telework to replace
most daily commuting and business travel. Dinesh and a couple of his teammates are excellent computer
programmers, so they are writing some new software modules for the simulation itself. Another couple
of his teammates are artists developing a video enactment of what life would be like for a Mumbai call
center worker under this scenario. Dinesh’s team has already won several awards in the Mumbai
competition, and he thinks they have a good chance of winning more in the Indian national competition.
Dinesh also knows that last year, many elements from the scenario developed by a winning high
school team in California were later adopted by the Conservative Party in the U.K., and he dreams that his
team’s work may someday be as influential.
An open source software developer
Matt Shields works in Santa Clara, California, designing integrated circuits for Cisco. He loved
programming in college, but he doesn’t get much chance to write software at work now. So for several
years, he has spent some of his nights and weekends contributing to the Linux open source software
project.
Matt is also passionate about environmental issues. He drives a hybrid car, recycles everything he
can, and has been thinking recently about trying to get involved with some kind of environmental
activism. When he heard about the Climate Collaboratorium, he knew that was for him! Here was a
place where he could use his programming skills, indulge his passion for software, and try to make the
world better, all at the same time.
The current version of the most widely used Collaboratorium simulator divides world economic
activity into five major regions. For the last six months, Matt has been working on new software modules
that will allow people to subdivide these regions into much smaller units (such as countries, states, and
cities). Matt had a clever idea about how to do this, and he’s now almost through debugging the new
software he wrote. With a little luck, he hopes to get his new modules accepted by the Collaboratorium
software committee and included in an official software release sometime next month. And he hopes that
soon after that, people all over the world will be using his software to develop more detailed models of
their own cities and regions.
One of Matt’s friends recently got a job offer from Google, based in part on the volunteer
programming work he had done on the Collaboratorium. Matt is happy with his current job, so he’s not
looking for that. But he sometimes daydreams about a day--many years from now--when he can tell
stories to his grandchildren about how, when he was a young man in his twenties, he helped saved the
world from a global climate catastrophe.
A congressional staffer
Mary Dominguez is a Senior Legislative Assistant for U.S. Sen. Karen Williams (Democrat,
Colorado). Mary is responsible for all environmental legislation in Sen. Williams’ office, and starting a
couple of years ago, she began monitoring changes in the Climate Collaboratorium fairly frequently. She
has noticed that shifts in Collaboratorium endorsements are often good predictors of how public opinion
polls will change several months later.
Over the last few weeks, she has noticed a significant increase in endorsements for tighter limits on
a cap-and-trade program in the US, so Mary decides to talk to Sen. Williams later today about how this
new information should influence the environmental bill the Senator is currently drafting with a
Republican colleague.

16
REFERENCES
Churchman, C.W. (1967). Wicked Problems. Management Science. 14(4): p. B141-B142.
Conklin, Jeff (2005). Dialogue Mapping: Building Shared Understanding of Wicked Problems, New
York: Wiley.
Conklin, J., and M. L. Begeman. 1988. gIBIS: a hypertext tool for exploratory policy discussion. ACM
Transactions on Information Systems (TOIS) 6, no. 4: 303-331.
Danis, C. M., Viégas, F. B., Wattenberg, M. & Kriss, J. (2008) “Your Place or Mine? Visualization as a
Community Component” Computer-Human Interaction (CHI) 2008 Conference, Florence, Italy, April
5-10, 2008 (http://researchweb.watson.ibm.com/visual/papers/chi08.pdf).
Dowlatabadi, H. 1995. Integrated assessment models of climate change: An incomplete overview. Energy
Policy 23 (4/5): 289–296.
Fiddaman, T. 2002. Exploring policy options with a behavioral climate-economy model. System
Dynamics Review, 18 (2): 243-267.
Howe, J. (2008) Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business. New
York: Crown Business.
Introne, J. 2009. Supporting Group Decisions by Mediating Deliberation to Improve Information Pooling.
In . Proceedings of Group 2009, Sanibel, FL: ACM Press, May 10.
Kazman, R. and Chen, H. 2009. The metropolis model a new logic for development of crowdsourced
systems. Commun. ACM 52, 7 (Jul. 2009), 76-84. DOI= http://doi.acm.org/10.1145/1538788.1538808
Klein, Mark and Iandoli, Luca, Supporting Collaborative Deliberation Using a Large-Scale
Argumentation System: The MIT Collaboratorium, MIT Sloan Research Paper No. 4691-08, 2008,
http://ssrn.com/abstract=1099082.
Leshner, Alan I. (2003) Public Engagement with Science, Science 299 (5609), 977 (14 February 2003),
http://www.scienceonline.org/cgi/content/summary/299/5609/977.
Levin, J. and B. Nalebuff. 1995. An Introduction to Vote-Counting Schemes. Journal of Economic
Perspectives, 9 (3), 3-26.
Lowrance, J., I. Harrison, A. Rodriguez, E. Yeh, T. Boyce, J. Murdock, J. Thomere, and K. Murray. 2008.
Template-Based Structured Argumentation. In Knowledge Cartography: Software Tools and Mapping
Techniques, ed. A. Okada, S. J. Buckingham Shum, and T. Sherborne. Advanced Information and
Knowledge Processing. Springer.
Malone, T. W. and Klein, M. Harnessing collective intelligence to address global climate change (Invited
Lead Essay). Innovations: Technology | Governance | Globalization, Summer 2007, 2 (3), 15-26
http://www.mitpressjournals.org/doi/abs/10.1162/itgg.2007.2.3.15
Malone, T. W., Laubacher, R., & Dellarocas, C. Harnessing Crowds: Mapping the Genome of Collective
Intelligence. MIT Center for Collective Intelligence Working Paper, January 2009
(http://cci.mit.edu/publications/CCIwp2009-01.pdf).
Nordhaus W. D. 1994. Managing the Global Commons. MIT Press: Cambridge, MA.
Olson, Gary M., Ann Zimmerman & Nathan Bos (Eds.) Scientific Collaboration on the Internet, MIT
Press, 2008.
Preece, Jennifer and Shneiderman, Ben (2009) "The Reader-to-Leader Framework: Motivating
Technology-Mediated Social Participation," AIS Transactions on Human-Computer Interaction (1) 1,
pp. 13-32. Available at: http://aisel.aisnet.org/thci/vol1/iss1/5
Prinn, R.G., H.D. Jacoby, A.P. Sokolov, C. Wang, X. Xiao, Z. Yang, R.S. Eckaus, P.H. Stone,
A.D.Ellerman, J.M. Melillo, J. Fitzmaurice, D. Kicklighter, G. Holian and Y. Liu. Integrated global
system model (IGSM) for climate policy assessment: Feedbacks and sensitivity studies.
Climatic Change, 41 (3): 469-549, 1999
Rittel, Horst, and Melvin Webber (1973). "Dilemmas in a General Theory of Planning,", Policy Sciences,
Vol. 4, pp. 155-169, Amsterdam: Elsevier Scientific Publishing Company, Inc. (Reprinted in N. Cross
(ed.), Developments in Design Methodology, J. Wiley & Sons, Chichester, 1984, pp. 135-144.)

17
Sawin, Elizabeth, Andrew P. Jones, Tom Fiddaman, Lori S. Siegel, Diana Wright, Travis Franck,
Andreas Barkman, Tom Cummings, Felicitas von Peter, Jacqueline McGlade, Robert W. Corell, and
John Sterman. Current Emissions Reductions Proposals in the Lead up to the COP-15 are Likely to be
Insufficient to Stablize Atmospheric CO2 Levels: Using C-ROADS—A Simple Computer Simulation
of Climate Change—to Support Long-Term Climate Policy Development, Presented at the “Climate
Change – Global Risks, Challenges, and Decisions” Conference, University of Copenhagen,
Copenhagen, Denmark, 10 March 2009.
Sokolov A, Stone P, Forest C, Prinn R, Sarofim M, et al. (2009) Probabilistic forecast for 21st century
climate based on uncertainties in emissions (without policy) and climate parameters. Journal of
Climate: In Press
Sterman, J. D. 2002. All Models are Wrong: Reflections on Becoming a Systems Scientist. System
Dynamics Review 18 (4): 501-531.
Viegas, Fernanda B., Wattenberg, Martin and McKeon, Matthew M. (2007): The Hidden Order of
Wikipedia. In: Schuler, Douglas (ed.) OCSC 2007 - Online Communities and Social Computing -
Second International Conference July 22-27, 2007, Beijing, China. pp. 445-454.
Viegas, Fernanda B. (2007): The Visual Side of Wikipedia. In: HICSS 2007 - 40th Hawaii International
International Conference on Systems Science 3-6 January, 2007, Waikoloa, Big Island, HI, USA. p. 85.
Viegas, Fernanda B., Wattenberg, Martin, Kriss, Jesse and Ham, Frank van (2007): Talk Before You
Type: Coordination in Wikipedia. In: HICSS 2007 - 40th Hawaii International International
Conference on Systems Science 3-6 January, 2007, Waikoloa, Big Island, HI, USA. p. 78.
Weyant, J. P., ed.. 1999. The Costs of the Kyoto Protocol: a Multi-Model Evaluation. Energy Journal,
Special Issue.