Content uploaded by Dennis Friess
Author content
All content in this area was uploaded by Dennis Friess on Jun 03, 2017
Content may be subject to copyright.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
1
Analyzing Crowd Discussion
Towards a more complete model to measure and explain online deliberation
by Dennis Friess & Christiane Eilders
Institute of Social Sciences | University of Düsseldorf | Germany
Abstract
This article proposes an empirical model for the analysis of online deliberation. While most em-
pirical studies exclusively focus on measuring the degree of deliberativeness in a given text, this
paper adopts a more comprehensive perspective on online deliberation by considering the condi-
tions and outcomes of deliberation in addition to the measurement of deliberativeness. The em-
pirical model of online deliberation draws on existing theoretical and empirical work. It includes
assumptions on three levels: the conditions fostering deliberation (institutional input level), the
parameters of the communication process itself (communicative throughput level) and the ex-
pected benefits of deliberation (productive outcome level). Every level addressed in the model is
both rooted in deliberative theory and complemented by empirical findings. For every level a set
of empirical dimensions is developed accordingly. The model is intended to serve as a framework
for experimental research investigating relationships between input, throughput and outcome of
deliberation.
Keywords: Deliberative Theory, Online Deliberation, Measurement, Empirical Research.
Author Notes
Dennis Friess is Research Fellow at the Department of Communication at the University of
Düsseldorf (Germany). Contact: dennis.friess@hhu.de
Christiane Eilders is Professor for Communication and Media Studies at the Department of Com-
munication at the University of Düsseldorf (Germany). Contact: christiane.eilders@hhu.de
Contact
Dennis Friess
Universität Düsseldorf
Universitätsstr. 1
40225 Düsseldorf / Germany
dennis.friess@hhu.de | 0049 211 8114 772
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
2
Introduction
Democratic policy-making processes are met with skepticism in terms of the acceptance of poli-
cies. This is often reflected to as a crisis of democracy. In reaction to this crisis political elites and
institutions have started to provide new possibilities of participation for affected stakeholders.
Hence Zittel (2005) is correct when he states that “political elites usually do not acknowledge a
crisis without having a cure ready at hand. Consequently, policies that would provide new oppor-
tunities for political participation are up on the political agenda as an answer to this perceived
crisis of democracy.” (Zittel 2005: 2). Formerly closed decision systems are opened to establish
communicative connections by providing opportunities of public participation. It is expected that
legally binding norms could be vest with more legitimacy, acceptance and quality than in the pre-
vious, strictly representative mode. These expectations reflect ideas constantly formulated by
theorists of deliberative democracy during the last three decades (e.g. Dryzek 2000; Gutman &
Thompson 1996; Habermas 1992). Theorists argue that even under conditions of entrenched
conflict and uncertainty not only mutual understanding could be reached, but consensual rational
solutions could emerge from deliberation (Landwehr 2012: 355). Therefore, deliberation is a po-
litical mode for generating legal norms and it potentially provides the solution for the smoldering
legitimacy crisis (Habermas 1975; Habermas 1992).
As internet technologies have gained more attention during the last two decades, possibilities for
new large-scale public deliberation became increasingly popular in academia and in practice. The
internet has often been presented as the healer providing the infrastructure for the public sphere
which deliberative advocates have dreamed of (Graham & Witschge 2003: 173). Accordingly,
deliberative democracy is one of the most influential theoretical concepts in the ongoing debate
on the relationship of democracy and internet technology (Chadwick 2009). Empirical research
on online deliberation has experienced a sharp increase in recent years and a massive body of
theoretical and empirical literature is available now (e.g. Black et al. 2011; Davis 2010; Gerhards
& Schäfer 2010; Davies & Gangadharan 2009; Dahlberg 2001). Thus large numbers of sugges-
tions have been presented on how to assess online communication by using deliberative theory as
a yardstick (e.g. Stromer-Galley & Martinson 2009; Stromer-Galley 2007; Trénel 2004; Graham &
Witschge 2003). However, the field of online deliberation is still “under construction” (Coleman
& Moss 2012). Thus, we have to investigate issues of design, research and practice of online de-
liberation normatively (how things should be ideally), descriptively (how things are empirically)
and prescriptively (how we can change things for progress) (Davies & Gangadharan 2009: 7).
This article aims to contribute to these questions introducing an empirical model of deliberation.
Our model is based on most fundamental assumptions from deliberative theories. Starting from
these basic ideas of deliberation helps us to overcome the fuzziness of the concept which is a
major problem of empirical deliberation research (Mutz 2008). The objective of the paper is to
suggest how online crowd communication could be analyzed in a deliberative perspective. While
most empirical studies exclusively focus on measuring the degree of deliberativeness in a given
text (e.g. Black et al. 2011; Xiang, Yuen-Ying & Zhen-Mei 2008; Monnoyer-Smith 2006) this pa-
per adopts a more comprehensive perspective on online deliberation. We argue that scholars of
online deliberation have to discuss three aspects of deliberation. The institutional conditions of a
given communication space, the quality of the communication itself and the outcomes of deliber-
ation. These aspects are considered as levels in our empirical model: the conditions fostering de-
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
3
liberation (institutional input level), the parameters of the communication process itself (communicative
throughput level) and the expected benefits of deliberation (productive outcome level).
In order to develop this model step by step we will start with a theoretical definition of delibera-
tion based on five fundamental assumptions of the tradition. We then propose a very simple un-
derstanding of deliberation which can be transferred into an empirical model to be used for the
analysis of deliberation online. In the next step we introduce empirical dimensions and develop
the empirical model. We apply the model to a case study on an online discussion about new PhD
guidelines. Here we exclusively focus on the measurement of deliberativeness. Finally, we will
discuss the findings and the proposed model.
A common ground in deliberative theory?
In the last 30 years deliberative theories have become extremely popular prompting Dryzek
(2000) to state that a “strong deliberative turn” within political theory has taken place. Reviewing
the history of deliberative theory Chambers (2003) notes that deliberative democracy has matured
from a ‘theoretical statement’ into a ‘working theory’ (Chambers 2003: 307). Beside the massive
attention within political theory, additionally, the advent of the internet contributed to the boast
of deliberative theory. Authors like Pateman (1970), Barber (1984) and Habermas (1992) provid-
ed the theoretical framework for intellectual reflections how the internet could foster democracy.
Chadwick (2009) points out that the debate on the relationship of internet technologies and de-
mocracy has been highly inspired by ideas of deliberative and participatory democracy. The ideal
of the deliberative public sphere presented by Habermas “is probably the most influential con-
cept in the scholarly writing on e-democracy.” (Chadwick 2009: 14).
However, due to the widespread interest deliberative theories have received during the last years
the concept itself is rather fuzzy. Thus, there is no consensus on what deliberation exactly is or
should be (Delli Carpini, Cook & Jacobs 2004). Beyond the minimal agreement that democratic
processes should involve communication rather than aggregation and voting, there is hardly any
consensus on the details of the concept of deliberation (Bächtiger & Pedrini 2010: 10). Dahlberg
(2007) points out that definitions and criteria of deliberation used within the scientific field vary
as authors rely upon readings of a wide range of deliberative theories. Especially the growing
body of empirical literature on deliberation has stretched the concept, which means that many
authors define almost every type of communication as deliberation (Bächtiger et al. 2010). Ac-
cordingly, it remains unclear which conditions are necessary or sufficient for the concept of de-
liberation. Gonzalez-Bailon, Kaltenbrunner and Banchs (2010: 3) state that “without these condi-
tions, deliberation is a moving target: it is difficult to match with any particular instance of public
discussion and it can always be argued that some crucial element is missing that disqualifies the
entire empirical approach. The problem with this lack of conceptual clarity is not only that it goes
against the basic principle of scientific refutability, hampering the development of the theory, but
also that it blurs the boundaries between the definition of deliberation and its evaluation.” To
overcome the fuzziness of the concept we propose a definition, which is based on five shared
and recognized assumptions of deliberative theory.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
4
Firstly, deliberative theory starts with turning away from liberal or economic theories and voting-centric
views of democracy, towards a more talk-centric view of democracy. “Voting-centric views see
democracy as the arena in which fixed preferences and interests compete via fair mechanisms of
aggregation. In contrast, deliberative democracy focuses on the communicative processes of
opinion and will-formation that precede voting.” (Chambers 2003: 308). Accordingly, deliberative
theories are rooted in the tradition of the Frankfurt School established by theorists like Theodor
Adorno, Max Horkheimer, Walter Benjamin or Jürgen Habermas. Critics on the current repre-
sentative mode of democracy are a kind of standard part of deliberative theories, but theorists
vary how critical they are of representative institutions. However, most of them standing in
strong opposition to liberal or elitist theories of democracy like Schumpeter or Downs have pre-
sented them.
The second feature is the rationalizing potential of human communication which is seen as a key source
of legitimacy. Authors like Habermas (1992) emphasize that the only “legitimated order is one
that could be justified to all those living under its laws” (Chambers 2003: 308). Any act of power
has to be publicly articulated, explained and justified within the normative framework of the
“forceless force of the better argument” (Habermas 1975: 108). Thus the medium for legitimacy
is human communication.
The third feature regards the rules which have to be observed in the communication process. Even though,
scholars agree that deliberation is a demanding type of communication, which have to follow
certain rules, the exact rules are matter of academic disputes (Bächtiger & Pedrini 2010: 10). But
even if divergences persist among political theorists about the main features or standards of de-
liberation, they all agree that deliberation can be distinguished from other types of communica-
tion (Monnoyer-Smith 2006). However, there is some sort of consensus that deliberation is a
rational, interactive and respectful form of communication.
Fourthly, deliberative theories assume beneficial outcomes of deliberation. While all theories agree that
the process of deliberation is going to produce salutary outcomes, there is little consensus on the
specific results (e.g. stronger sense of political efficacy, public-spirited attitudes, willingness to
compromise, more informed citizens, increase of perceived legitimacy) (Mutz 2008: 523).
The premises of talk centrism, the legitimizing power of communication and the standards of the
communication as well as the beneficial outcomes of the deliberation process are important as-
sumptions shared among deliberative theorists. However, those criteria are not crucial for the
theories about deliberative democracy. Deliberative democracy puts the focus on the public sphere and the
question concerning who should be included in the process of deliberation. Gutmann and
Thompson (2004) pointed out that “what makes deliberative democracy democratic is an expan-
sive definition of who is included in the process of deliberation – an inclusive answer to the ques-
tions of who has the right (and the effective opportunity) to deliberate […].“ (Gutmann &
Thompson 2004: 9f.) While deliberation refers to a specific type of communication, which is char-
acterized by certain rules and distinguished from other types of communication, deliberative democ-
racy also includes assumptions on the institutional setting where deliberation should take place
and who should be included. Thus, the specific communication modus of deliberation is a neces-
sary but not a sufficient condition for deliberative democracy. Accordingly, theories on delibera-
tive democracy not only make demanding assumptions about the communication process itself,
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
5
they are also making demanding claims on the inclusiveness of the public sphere were delibera-
tion should take place.
To sum up, these five assumptions shared by most scholars of deliberation constitute a common
basis of deliberative theory. Accordingly, we define deliberative democracy as a talk-centric political
modus rooted in the critical tradition of the Frankfurt School where political discourse is per-
ceived as the source of legitimacy. In order to unfold its legitimizing power and producing demo-
cratically beneficial outcomes, communication has to follow certain rules which make delibera-
tion a demanding type of communication. Additionally, deliberative democracy requires a high
level of egalitarian inclusion within the public sphere.
A basic Model of Deliberation
The five features above provide the framework necessary to develop a very simple model of de-
liberation. While deliberative theories are very different in detail, all of them address a relation-
ship between certain conditions for communication, a specific type of communication and the outcomes
that are produced due to such a communicative process. Considering this, Wessler (2008) devel-
oped a normative model of deliberation for comparative empirical analysis of political media con-
tent. His model distinguishes three dimensions. The input dimension focuses on the equal oppor-
tunities for topics, ideas and arguments (Wessler 2008: 3). The throughput dimension approaches
the “questions of how public deliberation should be conducted” (Wessler 2008: 4). He points out
the three important criteria of justification, rebuttal and civility. Finally, the last dimension con-
siders the outcomes of deliberation (Wessler 2008: 5). Wessler draws on an analytical heuristic in-
troduced by Ferree, Gerhards and Rucht (2002a/200b) in their comparative study on the US and
German discourses on abortion. They discuss normative criteria of public sphere concepts from
different theoretical traditions by asking four questions: Who should speak? In what sort of pro-
cess? How should ideas be presented? And finally, what is the relationship between discourse and
decision-making outcomes? (Ferree, Gerhards & Rucht 2002b: 316). In order to conduct a system-
atic review of empirical deliberation research Bächtiger and Wyss (2013) introduced an empirical
model, which distinguish between conditions for deliberation, the process of deliberation and the
normatively desirable outcomes. Bächtiger and Wyss also sketch out empirical indicators for each
level, which makes this model workable for empirical research.
While other authors presented remarkable ideas for the empirical analyses of mediated delibera-
tion (e.g. Maia 2012; Wessler 2008; Ferree, Gerhards & Rucht 2002a) or the comparative analyses
of political systems (Bächtiger & Whyss 2013; Steiner et al. 2004), the field of online deliberation
lacks such sophisticated approaches which provide ground for empirical research. Even if the
body of literature on online deliberation has grown fast during the last two decades, the different
studies neglect the broader context of deliberation. In order to overcome this problem we sketch
a simple idea of deliberation by pointing out that all deliberative theories draw a relation between
certain conditions for deliberation, specific standards for the process of communication and the
desirable outcomes claimed by normative theory. Accordingly, our model includes three levels:
the conditions fostering deliberation (institutional input level), the standards of the communication
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
6
process itself (communicative throughput level) and the expected benefits of deliberation (productive
outcome level).
Fig. 1 Basic Model of Deliberation
We argue that this model provides a more complete framework for online deliberation research,
because it covers the whole process of deliberation. Every level addressed in the model is both
rooted in deliberative theory and complemented by empirical findings. Therefore, the next sec-
tion will briefly address every level.
Institutional input level
At the level of institutional inputs the focus is on the conditions for deliberation. The crucial ques-
tion is how online spaces should be designed and organized to foster deliberation. Informed
guesses can be drawn from the theoretical literature about the normative qualities of the public
sphere. The normative concept of the public sphere described by Habermas (1992) is used as a
yardstick for judging the institutional design of a certain communication space. Inclusiveness in
terms of equal access and participation opportunities or openness for all topics, issues and posi-
tions - or more precisely all disputed validity claims - are important factors. Another essential
aspect following from Habermas’ ideal speech situation is the absence of power. The only force
accepted is the force of the better argument (Habermas 1981). Further, it is a crucial question
whether fundamental preconditions for deliberation like conflict and need for decision (Gutmann &
Thompson 2004) are fulfilled.
A growing body of research has identified various social as well as technical features foster delib-
eration (e.g. Coleman & Moss 2012; Karlsson 2012; Towne & Herbsleb 2012; Zhang, Cao &
Tran 2012; Eveland & Hively 2009; Stromer-Galley & Martinson 2009; Himelboim 2009;
Himelboim 2008; Wright & Street 2007; Wise, Hamman & Thorson 2006; Janssen & Kies 2005).
A review of the empirical findings helps disclosing crucial design factors, which are likely to have
an effect on the deliberative quality of online discussions. Janssen and Kies (2005) stress that real-
time discussion spaces like chat rooms are more likely to attract small talk and jokes rather than
deliberation. Therefore online discussion space should be asynchronous to allow participants to
spend more time reflecting and justifying their contributions (Janssen & Kies 2005: 321).
Stromer-Galley and Martinson (2009) confirm that synchronized communication has a negative
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
7
influence on different dimension of deliberation. “Synchronous chat seems especially problematic
for creating quality interaction, because of its apparent lack of coherence. […]. Because the chat
environment happens in near-real-time yet does not easily facilitate turn-taking, ideal argument
cannot occur. Messages are short, leading to under-developed arguments; there is a high level of
personal attack; and the overall dialogue is incoherent as multiple people talk at roughly the same
time, intermixing two or more separate lines of conversation.” (Stromer-Galley & Martinson
2009: 197). While the technical structure of a communication space should concede time for re-
flection, this does not apply to the visibility of the produced content. Towne and Herbsleb (2012)
acknowledged that user content should appear immediately in order to motivate contributions
and lower perceived entry barriers (Towne & Herbsleb 2012: 102).
Moderation and anonymity are further design factors, which are controversially discussed in the
literature. However, empirical evidence is winning ground that identification of users fosters delibera-
tion in terms of civility, rationality and sincerity (Coleman & Moss 2012: 8). Janssen and Kies
(2005) conclude that “the identification of the participants is a fundamental element for explain-
ing the quality and the persistency of a political debate” (Janssen & Kies 2005: 321). Towne and
Herbsleb (2012) note that identification also has an impact on content quality. Whereas anonymi-
ty is able to push the quantity of participation, it simultaneously lowers the quality of the content
(Towne & Herbsleb 2012. 108).
Likewise, moderation is controversially discussed, but empirical evidence suggests that it can have
significant positive effects on the deliberative quality of online debates (Wright & Street 2007;
Jansen & Kies 2005; Edwards 2002). While online libertarians generally reject moderation as an
illegitimate form of censorship (Coleman & Moss 2012: 8) Janssen and Kies (2005) stress the
type of moderation. “The moderator can be a ‘censor’ – for example by removing opinions that
are at odds with the main ideology of the discussion space – or he can be ‘promoter of delibera-
tion’ by, for example, implementing a system of synthesis of debate, by giving more visibility to
minority opinions, by offering background information related to the topics etc.” (Janssen & Kies
2005: 321) Wright and Street (2007) conclude that moderation is crucial to enable respectful, ra-
tional, focused and fair online deliberation.
Another important design feature is the perceived power of communication spaces. Janssen and Kies
(2005) distinguish between strong and weak discussion spaces. A discussion space (e.g. an online
forum) is considered strong if people think that their contribution is able to influence political
outcomes. On the contrary, online public spaces are weak if participants do not believe that their
participation has any impact (Jansen & Kies 2005: 324). Drawing on empirical findings by Cole-
man, Hall and Howell (2002) and Jensen (2003) they conclude that strong discussion spaces tend
to be more deliberative than weak discussion spaces (Jansen & Kies 2005: 324).
If the aim of a communication space is to produce something substantial, the technical structure
should enable a division of labor. The division of large tasks into smaller units is one of the key les-
sons from online cooperation projects like Linux or Wikipedia and should be adapted for online
deliberation. Giving participants the opportunity to choose a task of their personal interest, moti-
vation or competence makes it likely to have qualitative spillover effects on the final outcomes
(Towne & Herbsleb 2012: 103). The debate on task definitions and distinctions can lead to a
more precise and informed picture of the whole project and its details. Noveck (2009) points out:
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
8
“The more specific the question, the better targeted and more relevant the response will be.”
(Noveck, 2009: 171)
The technical opportunity of horizontal interaction with other users is another factor affecting delib-
eration. For example, the online consultation by the Commission of the European Union only
allows inputs towards the commission. Other civil society groups cannot be contacted, which
means that this instrument could never be deliberative.
Finally, since deliberation rests upon the rational weighing of different arguments and infor-
mation, online spaces for deliberation should provide relevant information or encourage people to
post relevant and adequate information.1 Himelboim (2009) studying 20 political online forums
found that 95 percent of the most recognized users posted external information. This indicates a
relation between information supply and replies, which touches the deliberative dimension of
interactivity.
Communicative throughput level
The communicative throughput level refers to the most important normative claims of delibera-
tive theory regarding the question of how the communication process should look like, particularly
how the participants should react to each other’s ideas (Wessler 2008: 4). Rules addressed on this
level drawn from Habermas’ (1983) discourse ethics. We further integrated suggestions from
empirical research on the measurement of deliberation (e.g. Black et al. 2011; Stromer-Galley
2007; Steiner et al. 2004; Trénel 2004; Graham & Witschge 2003; Dahlberg 2001).
In order to identify relevant elements of deliberative communication we reviewed 16 different
empirical instruments to measure deliberativeness. Although the theoretical foundations, defini-
tions and terms vary between the 16 studies, four key dimensions of deliberation could be identi-
fied. Accordingly, deliberation is rational, interactive, equal and respectful. Drawing from the
widely recognized Discourse Quality Index from Steiner et al. (2004), which Habermas himself
described as “splendid comparative study [which] reaches just to the centre of the whole ap-
proach to deliberative politics“ (Habermas 2005: 389), we add the dimension of common good
reference and constructiveness. While the exact operationalization of these dimensions is beyond
the scope of this paper, we want to briefly explain the six deliberative dimensions.
The probably most crucial dimension of deliberation is rationality. The prevailing opinion in litera-
ture suggests that in deliberative communication positions are substantiated with arguments and
empirical evidence (Ryfe 2005). Gutmann & Thompson (2004) define reason-giving a one of the
key elements of deliberation. Habermas (1992) drawing on Cohen (1989) underlines the critical
exchange and challenge of rational arguments as the core of deliberation as well. Manin (1987)
states that a diversity of arguments is an essential condition for the rationality of the process. Ac-
cordingly, it is necessary that participants change their opinion in the light of arguments.
The important role of rationality affects the second key dimension of deliberation: interactivity.
The assumption of communicative rationality rests on the premise that participants interact with
each other. Thus deliberation is a social process of giving and taking (Dryzek 1990), which in-
1 Providing relevant information also has been one key element of Fishkins (2009/1995) deliberative Poll studies
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
9
cludes both listening and responding (Barber 1984). Arguments should not just be articulated, but
rather also listened to and replied. The interactive mode of deliberation implies an exchange of
arguments. Habermas (1983) additionally stresses the need for role taking and empathy, which
means that every participant has to be able to take the perspective of other participants.
The third important characteristic of deliberation is equality. This dimension touches on the con-
dition of inclusiveness (Trénel 2004), which is also relevant on the input level. Everybody who is
affected by a policy should have the same opportunity to participate in deliberation (Habermas
1992). In addition, Gutmann and Thompson (2004) stress the criteria of accessibility and inclu-
siveness. While deliberation should be open to anyone, there is no need for citizens to participate
(Trénel 2004). However, equality and inclusiveness in this sense is rather placed at the level of
institutional inputs. On the communicative throughput level we focus on the equal opportunities
to articulate arguments and reply to other participant claims. It is crucial that every claim is treat-
ed equally and has the same chance to be deliberated. The only power present is the “forceless
force of the better argument” (Habermas 1975: 108).
Another core dimension of deliberation is civility. First of all, this dimension reflects the need for
mutual recognition of the participants in the sense that everybody is recognized as an equal and
rational actor able to speak in his or her own manner. The mutual recognition is the fundamental
premise for reaching rational consensus by the balanced exchange of arguments including re-
spectful listening (Barber 1984). Trénel (2004) points out that being ready for being convinced by
others requires to show respect and empathy towards the other participants (Trénel 2004: 3) Ob-
viously, civility also includes the absence of disrespectful and discriminating speech acts.
In an ideal process of deliberation participants justify their positions by referring to the common
good (Bächtiger & Wyss 2013). Framing the arguments in the “perspective of the common good
enables participants from diverging interest groups to convince each other” (Trénel 2004: 18).
Thompson (2008) argues that public reasoning in front of a diverse audience makes it more likely
that speakers “appeal to more general principles” (Thompson, 2008: 510), which are in line with
the common good. Manin (1987) puts it similar, when he emphasises that deliberation “provides
an incentive to generalization“ (Manin 1987: 359).
Finally, constructiveness can be considerd a relevant dimension of deliberation (e.g. Monnoyer-Smith
& Wojcik 2012; Black et al. 2011; Steiner et al. 2004). This is hinted to rationality, which implies a
constructive atmosphere in which consensus is the final goal (Habermas 1992). Consequently, the
orientation towards a common ground and agreement is a fundamental part of deliberation.
Productive Outcome level
The third level of deliberation regards the outcomes promised by deliberative theories. Pateman
(1970) puts it correctly when she says that “results that accrue through the participatory process
provide an important justification” (Pateman 1970: 25) for the whole idea and claim for more
participation and deliberation. Therefore we have to investigate if theoretical promises are met
empirically. Investigating whether deliberative democracy is a falsifiable theory, Mutz (2008)
pointed out that promises made by normative theories constitute the shared ground for empirical
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
10
research and normative theory because it can be observed to which degree they were fulfilled
(Mutz 2008: 523).
However, desirable outcomes of deliberation are a matter of dispute (Bächtiger & Wyss 2013:
164). The literature suggests a differentiation between outcomes on the individual level and out-
comes related to the quality of decisions. At the individual level we can summarize outcomes that
are contributing to what some authors call the “deliberative citizen” (Coleman & Moss 2012).
Outcomes on the individual level include increase of tolerance (Gutman & Thompson 1996; Manin
1987), more political knowledge (Parry, Moyser & Day 1992), a stronger sense of political efficacy
(Pateman 1970), more public spirited attitudes and willingness to compromise (Chambers 1996;
Barber 1984) or to transform preferences (Fishkin 2009).
At the result-related outcome level deliberative theorists have spelled out various benefits. Some
argue that ideal processes of deliberation will generate consensual decisions with high epistemic quali-
ty, which enhance legitimacy of the final decision (Habermas 1992; Barber 1984).2 However,
Thompson points out that “there is no consensus among deliberative theorists themselves that
consensus should be the goal of deliberation” (Thompson 2008: 508). For example Bohman
(2007) is skeptical about consensus as the result of deliberation and suggests error avoidance as the
main goal to be achieved by deliberation. Mendelberg (2002) referring to Chambers (1996) states
that “Political decisions will become more considered and informed by relevant reasons and evi-
dence” (Mendelberg 2002: 153) which finally contributes to a higher quality and acceptance of
legally binding norms.
An empirical Model of Online Deliberation
After every level has been addressed we want to move on to create an empirical model for the
analysis of online discussions. Therefore, all the different dimensions discussed for each of the
three levels are transferred into a model which follows the basic model of deliberation above.
The institutional input level comprises the theoretical dimensions of inclusiveness, equality and
openness. Additionally, we include the design features of asynchronous communication, content
visibility, moderation, identity, information, division of labor, horizontal interaction and per-
ceived power. Empirically we have to investigate the markedness of those dimensions in the
structural architecture of a certain communication space in order to investigate its deliberative
potential. The institutional design is conceptualized to shape the throughput which will represent
the focus of our empirical attention in the upcoming section.
The communicative throughput level includes the dimensions of rationality, interactivty, civilty, equali-
ty, common good reference and constructiveness. Each of the dimensions can be translated into
empirical indicators and transferred into a coding scheme for content analyses of a certain online
content. This offers the opportunity to investigate the quality of a debate in a deliberative per-
spective and sheds light on the question: “Did they deliberate?” (Knobloch et al. 2013).
2 Mutz (2008) points out that regardless of the perception, some authors argue the inherent legitimization of such processes (Mutz 2008: 524).
Obviously, this position provides no starting point for empirical research.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
11
Moving forward in our model, the quality of the throughput has an affect at the productive outcome
level. Here we distinguished between individual outcomes like the increase of tolerance, efficacy,
knowledge, public spirit, acceptance and emergence of preference transformation. The empirical
analyses of deliberative outcomes at the individual level therefore have to focus on those indica-
tors. Result-related outcomes are consensus, epistemic quality and legitimacy. Here empirical
research has to investigate if decisions are perceived as more acceptable, legitimized or of a high-
er quality by the participants or experts. Consensus could be measured by surveys or the analysis
of votes if given.
Fig. 2 Basic Model Empirical Dimension included
The boxes at the bottom indicated which part of the model can be considered as independent
variable (IV) or dependent variable (DV) in the case of experimental or quasi-experimental re-
search design. For example we could investigate if there are dependencies between certain design
features at the institutional input level (IV) and specific dimensions of deliberativeness at the
communicative throughput level (DV). Likewise, we could check for such relations between de-
liberativeness at the communicative throughput level (IV) and certain outcomes (DV)
Case Study: deliberating PhD guidelines online
The model served as a basic framework for the analysis of deliberation in a case study. This case
study was realized within a broader interdisciplinary research project focusing on issues on cooper-
ative internet mediated norm setting. 3 The rules dealt with in the case study regard new PhD guidelines
in one faculty of the authors’ university. The discussion and adoption of guidelines for academic
degrees is one of the main tasks of the faculty council where the majority of votes are held by
professors. Due to the heterogeneity of disciplines in this faculty, the requirements for a PhD
were a matter of conflict. The faculty council decided to set up an online forum for the discus-
sion among stakeholders (professors, academic staff and PhD students as well as the original
3 Cooperative norm setting defines a collective process of drafting, discussing and deciding on norms that are binding for the respective group.
The participation of all group members does not preclude a differentiation of duties, expertise and rights among them (Normsetting 2014). Our
interest is in understanding how groups of individuals develop norms and how the internet can help to facilitate this norm setting process. Specifi-
cally, our question is: What are the requirements, options and consequences of realizing internet-mediated cooperative norm setting in distinct
social domains (Normsetting 2014). The work done by the research group is organized in eight research projects from various disciplines includ-
ing business administration, computer science, law, communication science, politics and sociology.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
12
members of the council). The debate was expected to increase both quality and acceptance of the
new guidelines. The process was organized in five phases from discussing fundamental principles
to passing the final draft in the council.4 At the end of this process, several hundred people had
participated in the redraft of the guidelines by posting suggestions, discussing the suggestions and
voting on the suggestions in the forum.
Data and Method
The main objectives for this case study were to design an online forum, which optimally supports
online deliberation and to investigate deliberativeness of the discourses in the forum. In particular
we focus on the question: How deliberative is the debate? Thus, empirically we exclusively reflect on
the communicative throughput level while the institutional input level is only considered for the
design of the forum.
Accordingly, the institutional setting of the platform complies with our proposed design criteria
above. The forum was accessible for all potentially affected faculty members (about 1.400 partici-
pants) which meets the theoretical criteria of inclusiveness and accessibility. Within the forum all
comments had the same chance to be discussed. However, as the names and titles of participants
were visible, status power could have affected communicative behavior in the debate. To ensure
the optimum support of the deliberative process the online forum’s design considered all factors
known to support the quality of online discourses. Thus, communication was organized asyn-
chronously while content was promptly displayed. A discreet moderation was provided and user
names were visible for each post. Users were free to choose any of the participation options (vot-
ing, commenting or introducing new proposals). They were also free to choose in which of the
different debates they wanted to participate. The forum provided a section of relevant infor-
mation. Hence, the starting page of the forum made clear that all the contributions would be
considered in the final draft the forum could be considered as a strong communication space. In
sum, conditions for the deliberative process could be regarded as almost optimal.
Fig. 3 Screenshot showing an overview of the fundamental principles which constitute a discussion space5
4 The final draft of the PhD guidelines was in the responsibility of the faculty council due to legal reasons. Beforehand, the council expressed a
strong commitment and follow the suggestions accruing through the participatory process. Finally, the guidelines were passed unanimously.
5 Names were made unrecognizable due to data privacy protection.
voting score
topic
number of comments
name and status of the propos-
er
information on the
process
relevant information
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
13
Fig. 4: Screenshot of the forum showing a discussion space (thread). 6
The degree of deliberativeness was assessed via a quantitative content analysis of a total of 435
posts of the first phase where fundamental principals of the guidelines where discussed. The unit
of analysis was one single post within one discussion space (thread). Every debate on one guide-
line principle was defined as a discussion space (thread). The debate contained 25 of such discussion
spaces. The coding scheme included six dimensions. The table below shows each dimension and
its indicators.
Dimension
Indicators
rationality
topic relevance
position statement
argumentation
demand for information
information presentation
interactivity
substantial interaction
critical interaction
supportive interaction
argument engagement
civility
recognition of the right to speak
respect
common good references
Common good reference
constructiveness
Constructive communication
(e.g. searching for common ground, debate summery,
solution proposals)
emotional talk
Negative emotions
Positive emotions
All discussions were coded by two coders who received coder training for two days. Intercoder
reliability was at H = 0.90 which can be considered as excellent.
6 Names were made unrecognizable due to data privacy protection.
topic
voting score
proposed principal (norm)
justification for proposal
comments/posts
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
14
Findings
Findings overall showed a high degree of deliberativeness. Almost all 435 analyzed comments
were clearly related to the topic of the forum (98 percent) and contained a clear position (96 per-
cent). Two thirds of all comments showed at least one valid argument. About 12 percent of the
comments specifically asked for further information, while more than 25 percent provided addi-
tional information. These findings suggest a high level of rationality. Especially the number of
arguments and the high level of coherence (talking on topic and clear position) support this con-
clusion. Less than 9 percent of the comments contained emotional elements. While the role of
emotions in the context of deliberation is controversial (e.g. Bickford, 2011; Graham, 2010), the
low level of emotional comments can be interpreted in favor of the debate’s rationality.
In terms of interactivity we found that 55 percent of the analyzed comments included references
to other comments. About 21 percent of all posts made a critical reference to another comment
and 26 percent supported other posts. About 25 percent of all comments were replying to a spe-
cific argument. While the first impression may indicate an average level of interactivity, the fact
that many of the posts were referring to the initial proposal for a new guideline draws another
picture. Thus a reply like “good idea, no doubt on that” was not coded as substantial reply. Only
replies which specifically addressed other participants’ posts were coded. Considering this, the
findings indicate a fairly high degree of interactivity.
Uncivil communication was not found at all. There was no withdrawal of the right to speak and
almost no disrespectful or discriminating content. On account of the fact that all the participants
were members of a university these findings are not surprising.
Findings regarding common good references indicate the difficulty of this measure. The majority
of the comments remain neutral on this dimension, which means that about 94 percent of the
analyzed comments made no specific statement towards potential benefits for a particular group
nor for the entire community (common good). Only 4 percent of comments considered particu-
lar interests, while 2 percent tried to have all stakeholders’ interests in mind.
More than 20 percent of the comments included constructive elements which mean that partici-
pants tried to find a common ground or proposed new solutions. The interpretation of this find-
ing is difficult. However, it could be argued that a higher level of constructiveness would rather
indicate a fuzzy and conflicted discussion. We also have to consider that constructive communi-
cation needs a critical amount of comments beforehand. Briefly, only if four people talking con-
troversially, one could speak up trying to restructure the debate.
In terms of equality, we found that no discussion space (thread) was dominated by a small num-
ber of users. No single participant generated more than 20 percent of all comments within a cer-
tain debate on one of the guideline principals. Looking at the entire debate of all 25 discussed
principals shows findings familiar from previous research on online discussion. Only a small
number of participants accounted for the majority of the content (e.g. Strandberg 2008; Albrecht
2006). About 12 percent of the most active users generated 42 percent of all 435 comments.
However, this phenomenon occurs only at the debate level and does not count at the level of
single discussion spaces (thread).
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
15
Discussion and Conclusion
In conclusion, the results indicate that the institutional design fostered a high degree of delibera-
tiveness, but the relation between different designs and the degree of deliberativeness has not yet
been investigated at this point and remains an open task for further research. However, the main
purpose of this paper is to introduce an empirical model which could help to investigate such
relationships. Additionally, it presents an instrument to assess deliberativeness in online discus-
sions. Findings from the case study illustrate that it works for online debates. The study is an
example of successful internet mediated cooperative norm setting. It shows that if deliberative
standards at the institutional input level are met, there is considerable deliberation at the level of
communicative throughput.7
The last 20 years have provided us with remarkable research and important findings on online
deliberation. We hope that our proposed model is able to fruitfully contribute to further research
in “the Blossoming Field of Online Deliberation” (Davies 2009). Therefore, we want to briefly
outline three aspects to conclude.
Firstly, while most empirical studies analyzing online deliberation focus exclusively on dimen-
sions like rationality, civility, equality or interactivity (communicative throughput level), they neglect the
institutional setting for and the outcomes of deliberation (e.g. Black et al. 2011; Xiang, Yuen-Ying
& Zhen-Mei 2008; Stromer-Galley 2007). By considering institutional conditions for and desira-
ble outcomes of deliberation our proposed model provides a sophisticated approach for further
research. Considering the institutional input level of deliberation can help further research in two
dimensions. Firstly, we can use those indicators as a “set of ideal requirements of public sphere
discourse” (Dahlberg 2001: 622) to identify the deliberative potential of a given communicative
space (e.g. a discussion forum). Doing this avoids excessive expectations regarding the communi-
cation processes, which cannot be fulfilled due to the institutional design. Secondly, we can use
these factors to design online communication spaces. However, while the design elements could
help to support deliberation, there is no guarantee that they do as the context factors and social
dynamics cannot be directly shaped by the initiators. Other factors like group size (Himelboim
2008), group heterogeneity (Karlsson 2012; Zhang, Cao & Tran 2012) or response rate (Wise,
Hamman & Thorson 2006) also affect deliberation but can only rarely be influenced by the de-
signers. Focusing on the empirically observable outcomes at the productive outcome level could help
us to get further information on the concrete benefits of the demanding process of deliberation.
At the same time these beneficial outcomes serve as a yardstick to challenge the arguments by
deliberative advocates (Mutz 2008: 524).
Secondly, our model is rooted in the most basic assumption of deliberative tradition. This pre-
vents it from being arbitrary. While some scholars argue for stepping back from the demanding
communication type of deliberation towards more casual or natural forms of political talk (e.g.
Warren 2007; Young 2002; Dryzek 2000), we argued on the ground of normative deliberative
theory. Advocates of what Bächtiger et al. (2010) called “typ II deliberation” almost defining eve-
ry type of communication as deliberation giving up those demanding standards of deliberation
7 While the outcomes of the process were neglected in this paper, we could also report positive evidence in line with deliberative theory claims in
this regard. The highly controversial issue of PhD guidelines was passed unanimously which hits the assumption of consensus (Habermas 1992).
Surveys that were sent to all participants suggest that the majority was satisfied with the guidelines’ quality, acceptance and legitimacy. Most partic-
ipants perceived the process as fair and voted for more online mediated cooperative norm setting for collective binding norms.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
16
and thus lack theoretical coherence. Therefore, if we want to study online deliberation, taking all
the desirable assumptions and promises, we have to stick to the theorized roots of the tradition.
However, this should not be an argument for conservative research refusing to progress, but we
have to be very careful using the term deliberation while at the same time ignoring fundamental
assumptions of its concept.
Finally, our proposed model sets the framework for experimental and quasi-experimental re-
search investigating relations between the different levels of deliberation. Presenting empirical
dimension for each level could guide further online deliberation research to investigate linkages
between those levels. This could help us to open the black box of deliberation (Mutz 2008) by
carefully examining relations between certain conditions (institutional input level), dimensions of
deliberation (communicative throughput level) and different dimension of outcomes (productive
outcome level) of deliberation. While there has been remarkable empirical research on delibera-
tion, those connections remain largely unknown. Therefore, experimental and quasi-experimental
design provides a fair chance for connecting the dots between different aspects and levels of de-
liberation.
References
Albrecht, S. 2006. Whose Voice is heard in online Deliberation? A study of participation and
representation in political debates on the Internet. Information, Communication & Society, 9(1),
62–82.
Barber, B. 1984. Strong democracy. Participatory Politics for a New Age
.
Berkeley: University of
California Press.
Bächtiger, A. & Wyss, D. 2013. Empirische Deliberationsforschung – eine systematische Über-
sicht. Zeitschrift für vergleichende Politikwissenschaft, (2013)7, 155-181.
Bächtiger, A. & Pedrini, S. 2010. Dissecting deliberative democracy. A review of theoretical con-
cepts and empirical findings. In I. Kenichi, L. Morales & M. Wolf (Ed.), The Role of Political
Discussion in Modern Democracies in a Comparative Perspective. London: Routledge.
Bächtiger, A., Niemeyer, S., Neblo, M., Steenbergen, M. R. & Steiner, J. 2010. Symposium: To-
ward More Realistic Models of Deliberative Democracy. Disentangling Diversity in Deliber-
ative De-mocracy: Competing Theories, Their Blind Spots and Complementarities.
The Journal of Political Philosophy, 18(2010), 32-63.
Bickford, S. 2011. Emotion Talk and Political Judgment. The Journal of Politics, 73(4), 1025–1037.
Black, L. W., Welser, H. T., Cosley, D. & DeGroot, J. M. 2011. Self-Governance through Group
Discussion in Wikipedia: Measuring Deliberation in Online Groups. Small Group Research,
42(5), 595-634.
Bohman, J. 2007. Political Communication and the epistemic value of diversity: deliberation and
legitimation in media societies. Communication Theory, 17(4), 348-355.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
17
Chadwick, A. 2009. Web 2.0: New Challenges for the Study of E-Democracy in an Era of Infor-
mational Exuberance. I/S: A Journal of Law and Policy for the Information Society, 5(1), 9-41.
Chambers, S. 2003. Deliberative democratic theory. Annual Review of Political Science, 6(2003), 307-
326.
Chambers, S. 1996. Reasonable Democracy. Ithaca, NY: Cornell University Press.
Cohen, J. 1996. Procedure and Substance in Deliberative Democracy. In S. Benhabib (Ed.).
Democracy and Difference. Contesting the Boundaries of the Political. Princeton, NJ: Princeton
University Press, 95-116.
Coleman S; Moss G. S 2012. Under Construction: the Field of Online Deliberation Research,
Journal of Information Technology & Politics. 9(1), 1-15.
Coleman, S., Hall, N. and Howell, M. 2002. Hearing Voices. The Experience of Online Public
Consultation and Discussion in UK Governance, UK: Hansard Society.
Dahlberg, L. 2001. The Internet and Democratic Discourse. Exploring the prospects of online
deliberative forums extending the public sphere. Information, Communication & Society, 4(2001),
615-633.
Davies, T. 2009. The Blossoming Field of Online Deliberation. In T. Davies & S. P.
Gangadharan (Ed.). Online Deliberation. Design, Research, and Practice. CSLI: Publications. 1-19
Davies, T. & Gangadharan, S. P. 2009. Online Deliberation. Design, Research, and Practice. CSLI:
Publications.
Davis, A. 2010. New media and fat democracy: the paradox of online participation. New media &
Society, 12(5), 745-761.
Dryzek, J. S. 2000. Deliberative democracy and beyond. New York: Oxford University Press.
Dryzek, J. S. 1990. Discursive democracy: politics, policy and political science. Cambridge: Cambridge
University Press.
Edwards, A. 2002. The Moderator as an Emerging Democratic Intermediary: The Role of the
Moderator in Internet Discussions about Public Issues, Information Polity 7(1): 3–20.
Eveland, W. J. & Hively, M. H. 2009. Political Discussion Frequency, Network Size and ‘Hetero-
geneity’ of Discussion as Predictors of Political Knowledge and Participation.
Journal of Communication, 59 (2009), 205–224.
Ferree, M. M., Gerhards, J. & Rucht, D. 2002a. Shaping Abortion Discourse. Democracy and the Public
Sphere in Germany and the United States. Cambridge: Cambridge University. Press
Ferree, M. M., Gerhards, J. & Rucht, D. 2002b. Four models of the public sphere in modern de-
mocracies. Theory and Society, 31(2002), 289-324.
Fishkin, J. S. 2009. When the People Speak. Deliberative Democracy and Public Consultation. Oxford:
Oxford University Press.
Fishkin, J. S. & Luskin, R. C. 2005. Experimenting with a Democratic Ideal: Deliberative Polling
and Public Opinion. Acta Politica 40(3), 284–299.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
18
Gerhards, J. & Schäfer, M. S. 2010. Is the internet a better public sphere? Comparing old and
new media in the USA and Germany. New Media & Society, 12(1), 143-160.
Graham, T. & Witschge, T. 2003. In search of online deliberation: Towards a new method for
examining the quality of online discussions. Communications, 28(2), 173-204.
Gutmann, A. & Thompson, D. 2004. Why Deliberative Democracy. Princeton: Princeton University
Press.
Gutmann, A. & Thompson, D. 1996. Democracy and Disagreement. Cambridge: Belknap Harvard.
Habermas, J. 2005. Concluding Comments on Empirical Approaches to Deliberative Politics.
Acta Politica, 40(2005), 384–392.
Habermas, J. 1992. Faktizität und Geltung. Frankfurt: Suhrkamp.
Habermas, J. 1983. Moralbewusstsein und kommunikatives Handeln. Frankfurt: Suhrkamp.
Habermas, J. 1981. Theorie des kommunikativen Handelns. Frankfurt: Suhrkamp.
Habermas, J. 1975. Legitimation Crisis. Boston: Beacon Press.
Himelboim, I. 2009. Discussion catalysts in online political discussions: Content importers and
conversation starters. Journal of Computer-Mediated Communication, 14(4), 771-789.
Himelboim, I. 2008. Reply distribution in online discussions: A comparative network analysis of
political and health newsgroups. Journal of Computer-Mediated Communication, 14(1), 156-177.
Janssen, D. & Kies, R. 2005. Online forums and deliberative democracy. Acta Politica, 40(2005),
317-335.
Jensen, J.L. 2003. Public spheres on the internet: anarchic or government-sponsored — a com-
parison, Scandinavian Political Studies 26(4): 349–374.
Karlsson, M. 2012. Understanding Divergent Patterns of Political Discussion in Online Forums –
Evidence from the European Citizens’ Consultation. Journal of Information Technology &
Politics, 9(1), 64-81.
Knobloch, K, Gastil, J., Reedy, J. & Walsh, K. C. 2013. Did They Deliberate? Applying an Evalu-
ative Model of Democratic Deliberation to the Oregon Citizens' Initiative Review.
Journal of Applied Communication Research, 41(2), 105-125
Landwehr, C. 2012. Demokratische Legitimation durch rationale Kommunikation. Theorien
deliberativer Demokratie. In O. W. Lembcke, C. Ritzi & G. S. Schaal (Ed.). Zeitgenössische
Demokratietheorie. Band 1: Normative Demokratietheorien. Wiesbaden: Springer VS, 355-386.
Maia, R. C. M. 2012. Deliberation, the Media and Political Talk. New York: Hampton Press.
Manin, B. 1987. On Legitimacy and Political Deliberation, Political Theory, 15(3), 338-368.
Mendelberg, T. 2002. The deliberative citizen: Theory and evidence. Research in Micropolitics, 6,
151-193.
Monnoyer-Smith, L. & Wojcik, S. 2012. Technology and the quality of public deliberation: a
com-parison between on and offline participation. International Journal of Electronic Gov-
ernance, 5(1), 24-49.
Dennis Friess & Christiane Eilders Conference Paper IPP2014 Oxford Internet Institute
19
Monnoyer-Smith, L. 2006. Citizen’s Deliberation on the Internet: An Exploratory Study. Interna-
tional Journal of Electronic Government Research, 2(3), 58-74.
Mutz, D. C. 2008. Is Deliberative Democracy a Falsifiable Theory? Annual Review Political Science.
11(2008), 521-38.
Noveck, B. S. 2009. Wiki government: How technology can make government better, democracy stronger and
citizens more powerful. Harrisonburg, VA: Brookings Institution Press.
Pateman, C. 1970. Participation and democratic theory. Cambridge: Cambridge University Press.
Ryfe, D. 2005. Does deliberative democracy work? Annual Review of Political Science, 8(2005), 49-71.
Steiner, J., Bächtiger, A., Spörndli, M. & Steenbergen, M. (2004). Deliberative Politics in Action: Ana-
lysing Parliamentary Discourse. Cambridge: Cambridge University Press
Strandberg, K. 2008. Public deliberation goes on-line. An analyses of citizens‘ political discussion
on the internet prior to the Finnish parliamentary elections in 2007. Javnost The Public, 15(1),
71-90.
Stromer-Galley, J. & Martinson, A. M. 2009. Coherence in political computer-mediated commu-
nication: analyzing topic relevance and drift in chat. Discourse and Communication, 3(2), 195-
216.
Stromer-Galley, J. 2007. Measuring Deliberation's Content: A Coding Scheme. Journal of Public
Deliberation, 3(1), Article 12.
Trénel, M. 2004. Measuring the quality of online deliberation. Coding scheme 2.2. Unpublished
paper, Social Science Research Center Berlin, Germany. Download: http://www.wz-
berlin.de/~trenel/tools/qod_2_0.pdf
Thompson, D. F. 2008. Deliberative Democratic Theory and Empirical Political Science. Annual
Review of Political Science, 11(2008), 497–520.
Towne, W. B. & Herbsleb, J. D. 2012. Design Consideration of Online Deliberations Systems.
Journal of Information Technology & Politics, 9(1), 97-115.
Warren, M. 2007. Institutionalizing deliberative democracy. In S. Rosenberg (Ed.). Deliberation,
Participation and Democracy: Can the People Govern?. London: Palgrave-Macmillan, 272–288.
Wessler, H. 2008. Investigating deliberativeness comparatively. Political Communication, 25(1), 1-22.
Wright, S. & Street, J. 2007. Democracy, deliberation and design: the case of online discussion
forums. New Media & Society, 9(5), 849-869.
Xiang, Z.; Yuen-Ying, C. y Zhen-Mei, P. 2008. Deliberativeness of Online Political Discussion: A
Content Analysis of the Guangzhou Daily Website. Journalism Studies, 9(5), 759-70.
Young, Iris M. 2002. Inclusion and Democracy. Oxford: Oxford University Press.
Zhang, W.; Cao, X.; Tran, M. N. 2012. The structural features and the deliberative quality of
online discussions. Telematics and Informatics. 30(1), 74-86.
Zittel, T. 2005. Participatory Engineering: Can Democratic Reform Increase Political Participation. Annual
Conference of the American Political Science Association, Washington D.C.
