Content uploaded by Erik Duhaime
Author content
All content in this area was uploaded by Erik Duhaime on Dec 05, 2017
Content may be subject to copyright.
Collective Intelligence 2015
Broad Participation in Collective Problem Solving Can
Influence Participants and Lead to Better Solutions:
Evidence from the MIT Climate CoLab
ERIK P. DUHAIME, Massachusetts Institute of Technology
GARY M. OLSON, University of California, Irvine
THOMAS W. MALONE, Massachusetts Institute of Technology
1. INTRODUCTION
The MIT Climate CoLab is crowdsourcing the problem of what humanity can do about global climate
change. It thus seeks to harness the collective intelligence of thousands of people around the world in
a form of very large-scale collective problem solving. Here, we offer the first comprehensive picture of
the characteristics and behavior of the Climate CoLab community. Through online surveys and
analysis of web activity, we find that the CoLab community is geographically diverse, highly educated,
and highly experienced with climate issues. We also find that people who are outside the usual
conversations about climate change are influenced by and contribute effectively to this collective
problem solving effort. More specifically, members who don’t have graduate education, don’t have
previous climate-related experience, and/or don’t live in the United States report significantly higher
levels of learning, belief change, and increase in climate-related activity as a result of their CoLab
participation. And, more importantly, we find that these members—in addition to women—are at
least as likely as others to submit high quality proposals.
The MIT Climate CoLab 1.1
The MIT Climate CoLab software platform (www.climatecolab.org) allows individuals and teams of
people from all over the world to develop proposals for what we should do about climate change
[Malone and Klein 2007, Introne, Laubacher and Malone 2011, Introne et al. 2011, Introne et al.
2013]. The proposals can include any ideas users have for technical, economic, political, or other
changes that should be made, as well as discussions of how the changes could be made, why they are
feasible, and why they are desirable. Each proposal has an associated discussion forum in which other
users can comment on the proposal, and all proposals have a wiki-like editing capability. In order to
keep the proposals grounded in the actual physical and economic reality we face, some proposals also
include computer simulation models to predict the potential impacts of the proposals on things like
temperature change, sea level rise, and various kinds of economic costs [Sewin et al. 2009, Sterman
2008, 2011, Sterman et al. 2012, 2013].
As of January 2015, more than 275,000 people from virtually every country in the world have visited
the CoLab, over 33,000 have registered as members, and over 950 have contributed to at least one
proposal. The primary driver of activity in the CoLab has been a series of four progressively more
elaborate annual contests. The most recent annual contest included 18 different sub-contests on
topics like how to reduce emissions in the transportation sector and how to change public attitudes
about climate change. Some of these contests were run in conjunction with organizations such as the
Union of Concerned Scientists (www.ucsusa.org) and Carbon War Room (www.carbonwarroom.com).
1
2 E.P. Duhaime, G.M. Olson and T.W. Malone
Collective Intelligence 2015
Each contest had advisors and judges – including experts from organizations like NASA, the World
Bank, MIT, and Stanford as well as one former US Secretary of State (Shultz), two former US
Congress members (Inglis and Sharp), and two former heads of state (Robinson and Bruntland).
After members submit proposals, judges select the most promising entries to be finalists and provide
feedback on how they can be improved. After the finalists revise their proposals, the judges select the
Judges’ Choice Awards and the community votes for the Popular Choice Awards. The Climate CoLab
offers a cash award to one Grand Prize winner and all the Popular and Judges’ Choice winners receive
opportunities to present their ideas to top experts and potential implementers, most recently at the
Crowds & Climate Conference at MIT in November 2014.
2. METHODS
We designed a brief 16-item survey to explore how the CoLab has affected both the beliefs and
behavior of registered members. In order to benchmark our sample, we started with two questions
that had been asked in a recent well-known study of the American public’s beliefs about climate
change [Leiserowitz et al. 2014]. Next we asked nine questions about the respondent’s attitudes and
behaviors before and after joining the CoLab and, finally, five general demographic questions. A total
of 12,713 e-mail invitations were sent to CoLab members who were registered as of July 2014. This is
substantially less than the 33,000 current members because the community has grown rapidly in
recent months. A total of 1,082 members responded to the survey, and we analyzed the data for all but
the 60 respondents who answered three or fewer questions.
We also sent a brief additional survey to all finalists and winners of CoLab contests who did not
respond to the initial survey in order to oversample these members on key questions of interest. An
additional 17 members responded to this follow-up survey, bringing our total sample to 1,039
members, but our sample sizes for questions in the next sections vary from 945 to 1018 because not all
respondents answered every question. The behavior of all registered members on the Climate CoLab
site is logged, including whether a person’s proposal is selected as a finalist or a winner. Thus, using
e-mail addresses, we were able to link the survey responses to the activity profiles for 990 of the 1,039
survey respondents.
3. RESULTS
Characteristics of CoLab Members 3.1
Respondents were 63% male, median age of 30-39, highly educated (58% attend or completed graduate
or professional school), and from all over the world (48% from outside the United States). While only
64% of Americans believe that global warming is happening and 60% of those think that it is at least
partially caused by human activities [Leiserowitz et al. 2014], almost all of our respondents believe
these things (97.2% and 97.5%, respectively). Members were also highly engaged in activities related
to climate change prior to joining the CoLab. Almost half (48.4%) of respondents reported that they
were previously engaged in activities related to climate change on at least a weekly basis, 23.4%
reported at least monthly, 20.5% occasionally, and just 7.7% reported that they engaged in no
activities related to climate change prior to joining the site.
The Effects of Membership on Learning, Beliefs, and Activity on Climate Change Issues 3.2
Broadly speaking, respondents reported high levels of learning since joining the site (71% said they
learned “A lot” or “Some” about climate change), substantial changes in beliefs (40% said that they
now think it is “more important for humans to take actions to reduce the bad effects of climate
Broad Participation in Online Problem Solving Benefits Participants and Leads to Better Solutions 3
Collective Intelligence 2015
change”), and substantial increases in activity related to climate change (46% reported that the CoLab
has had a moderate or considerable influence).
More interestingly, these changes were all greater for people who did not have graduate school
experience (t(815) = 2.86, p < .01; t(792) = 4.69, p < .01; and t(810) = 2.93, p < .01, respectively), had
little or not previous involvement with climate issues (t(488) = 1.80, p < .10, t(446) = 5.33, p < .01,
t(455) = 1.99, p < .05, respectively), and who were from outside the United States (t(832) = 5.55, p < .01,
t(861) = 8.75, p < .01, t(868) = 9.60, p < .01, respectively).
Characteristics of CoLab Contest Entrants and Finalists 3.3
We model a CoLab member’s decision to exert effort and submit a proposal to a CoLab contest as a
function of the characteristics of that member. Specifically we estimate the equation:
Pr (Submit Proposal) = β0 + β1Experience + β2Education + β3Gender + β4American + ε (Eq. 1)
where Experience is a dummy variable indicating whether a member was engaged in activities
related to climate change on a weekly or monthly basis prior to joining the CoLab, Education
indicates whether a member has graduate school experience, and Gender and American indicate
whether members are male and from the United States, respectively. We find that the probability of
submitting a proposal increases with prior climate-related experience (β1 = .094, SE = .029, t(887) =
3.29, p < .01) being male (β3 = .094, SE = .027, t(887) = 3.54, p < .01), and being from outside the
United States (β4 = -.057, SE = .026, t(887) = -2.18, p < .05). Education had no significant effect.
We also use the same variables to model the probability that a member was selected as a finalist,
conditional on having submitted a proposal. There are far fewer finalists (n = 54) than proposal
authors and several of these authors did not respond to all key questions of interest in the survey, so
none of the variables in this model are found to have a significant effect on the probability that a
proposal is selected as a finalist. However, we note that each of the regression coefficients is negative.
This suggests that contest entrants without previous climate-related experience, without graduate
school experience, who are women, and who are from outside of the United States are no less likely
(and perhaps more likely) to become finalists. Specifically, the regression estimates that the
probability of an author being selected as a finalist increases by 5.4% when the author is female (95%
confidence interval: -9% to +20%), by 5.0% when the author has no graduate school experience (95%
confidence interval: -8% to +18%), by 2.8% if the author has no previous climate-related experience
(95% confidence interval: -13% to +19%), and by 4.7% if the author is not from the United States (95%
confidence interval: -8% to +18).
4. CONCLUSION
The Climate CoLab is clearly succeeding in its goal of crowdsourcing the problem of what to do about
climate change insofar as it has attracted a large, global community that is already highly educated
and active with climate issues. Additionally, by encouraging broad participation from individuals not
previously involved in climate issues, those without postgraduate education, and those from outside
the U.S., the CoLab has fostered high levels of learning and attitude change. Finally, and perhaps
most importantly, we find suggestive evidence that these users—as well as women—are at least as
likely as others to submit high quality proposals. In other words, people who might otherwise be less
likely to be included in conversations about climate issues are at least as likely to make useful
contributions in this online community.
4 E.P. Duhaime, G.M. Olson and T.W. Malone
Collective Intelligence 2015
REFERENCES
!"#$%"&'()*'(+,-.,/0&$'(1*'(2(3,4%"&'(5*(6*(789::;*(1<3=>((=(?$,@&A%$B(#%(C",.4&(<D&"(E&F&4%D@&"#(3�%G%4%HI&J(I"(K4I@,#&(K0,"H&(
=JJ&JJ@&"#(3%G&4I"H*(!"""(#$%&'()*+,,-.7L;'(MLNL:*((
!"#$%"&'()*'(+,-.,/0&$'(1*()*'(<4J%"'(O*(3*'(2(3,4%"&'(5*(6*(789::;*(/0*,1234(&*,1$5(67,5()8*,9:(2*,4$;*2<6(9*;,:$22(6$)(&3=*,>2(??3?8@(P,D&$(
D$&J&"#&G(,#(#0&(K%"Q&$&"/&(%"(K%44,.%$,#I%"(5&/0"%4%HI&J(,"G(RSJ#&@J(7KR5(89::;'(P0I4,G&4D0I,'(P=*(
!"#$%"&'()*'(+,-.,/0&$'(1*()*'(<4J%"'(O*'(2(3,4%"&'(5*(789:T;*(R%4FI"H(6I/B&G(R%/I,4(P$%.4&@J(AI#0(R%/I%N/%@D-#,#I%",4(RSJ#&@J*(A!<AB?9&23:0*,
!?&*2238*?C+,-D7:;'(UMNM8*(
+&IJ&$%AI#V'(=*'(3,I.,/0'(C*'(1%J&$N1&"%-Q'(K*'(?&I".&$H'(O*'(2(1%J&"#0,4'(R*(789:U;*(1234(&*,:0(?8*,3?,&0*,E4*)3:(?,43?;7,E>)32,-FGH@(W,4&(
X"IF&$JI#S(,"G(O&%$H&(3,J%"(X"IF&$JI#S*(Y&A(Z,F&"'(K5>(W,4&(P$%[&/#(%"(K4I@,#&(K0,"H&(K%@@-"I/,#I%"(
3,4%"&'(5*(6*'(2(\4&I"'(3*(7899];*(Z,$"&JJI"H(/%44&/#IF&(I"#&44IH&"/&(#%(,GG$&JJ(H4%.,4(/4I@,#&(/0,"H&*(!??$=(&3$?97,/*:0?$2$8I+,J$=*)?(?:*+,
J2$6(23C(&3$?+,-7T;'(:MN8L*((
Sawin, E. R., Jones, A. P., Fiddaman, T., Siegel, L. S., Franck, D. W. T., Barkman, A., Cummings, T., von Peter, F., McGlade, J., & Corell, R. W.
(2009). Current emissions reductions proposals in the lead-up to Cop-15 are likely to be insufficient to stabilize atmospheric Co2 levels: Using
C-Roads–a simple computer simulation of climate change–to support long-term climate policy development. In proceedings of the Climate
Change–Global Risks, Challenges, and Decisions Conference, Copnehagen, Denmark.
Sterman, J. D. (2008). Risk communication on climate: mental models and mass balance. Science, 322(5901), 532-533.
R#&$@,"'()*(E*(789::;*(K%@@-"I/,#I"H(/4I@,#&(/0,"H&($IJBJ(I"(,(JB&D#I/,4(A%$4G*(1234(&3:,10(?8*+,GF.7U;'(^::N^8L((
R#&$@,"'()*(E*'(?IGG,@,"'(5*'(?$,"/B'(5*'()%"&J'(=*'(3/K,-4&S'(R*'(1I/&'(P*'(R,AI"'(C*'(2(RI&H&4'(+*(789:8;*(K4I@,#&(I"#&$,/#IF&>(#0&(KN1<=ER(
/4I@,#&(D%4I/S(@%G&4*(#I9&*4,KI?(43:9,L*=3*'+,-.7T;'(8_M*((
Sterman, J. D., Fiddaman, T., Franck, T., Jones, A., McCauley, S., Rice, P., Sawin, E., & Siegel, L. (2013). Management flight simulators to
support climate negotiations. Environmental Modelling & Software, 44, 122-135.