Conference PaperPDF Available

Citizen science, gallaxies and tropes: Knowledge creation in impromptu crowd science movements



This paper analyses two impromptu crowd science movements, comparing the TVTropes wiki community with the Bechdel Wallace test movement. Impromptu crowd science contributors engage in methodical knowledge creation, in absence of academic scientists' involvement. The two examples illustrate the potential of these movements to create data, analyses and concepts relevant for social and humanistic disciplines, and to inspire scholarly research.
Citizen Science, Gallaxies and Tropes
Knowledge Creation in Impromptu Crowd Science Movements
Cosima Rughiniș
Faculty of Sociology and Social Work
University of Bucharest
Abstract— This paper analyses two impromptu crowd science
movements, comparing the TVTropes wiki community with the
Bechdel Wallace test movement. Impromptu crowd science
contributors engage in methodical knowledge creation, in
absence of academic scientists’ involvement. The two examples
illustrate the potential of these movements to create data,
analyses and concepts relevant for social and humanistic
disciplines, and to inspire scholarly research.
Keywords—impromptu crowd knowledge; computer supported
collaborative work;; Bechdel Wallace test
I. I
This paper aims to document a novel form of collaborative
knowledge creation, the impromptu crowd science of media
tropes, by studying the digitally coordinated work of the
community of contributors to the wiki, in
comparison with the Bechdel-Wallace test movement.
The concept of ‘impromptu crowd science’ has been
advanced in [1] to describe the distributed research work of
people interested in the Bechdel-Wallace test of gender
representation in films. We argue that we can identify a
second instance of impromptu crowd science in the wiki. In what follows we present the specific
features of this community of practice, and we discuss its
relevance in relation to current discussions of crowd science,
with a focus on social and humanistic disciplines.
The paper is structured as follows: the next section
discusses varieties of crowd science and outlines research
questions. The following section presents knowledge creation
activities in the Bechdel Wallace test and TVTropes
movements, from data production to data analysis, conceptual
work and contributor specialization. The final section
concludes the paper, discussing the specificity of the two
movements and their relevance for academic research.
With the advance of online collaborative platforms and
social media, the scientific community has sought for ways to
engage the digital crowds in contributing to large scale
scholarly challenges. Citizen science, or crowd science,
includes projects in which large numbers of lay contributors
(who, as a rule, do not know each other) take part in various
tasks, coordinating their work through digital interfaces.
Typical citizen science projects are initiated and organized by
teams of scientists, who supervise citizen contributors’ inputs
and are in charge with final stages of data analysis and
reporting. Still, some contributors from the ‘crowd’ may be
involved in all stages of the research project.
The two defining features of crowd science projects,
according to Franzoni and Sauermann [2], are openness in
project participation and openness in the disclosure of
intermediate inputs (such as methods, data, analyses). This
combination differentiates crowd science from innovation
contests or crowdsourcing in scientific projects, which relies
on open participation but keeps the scientific activity,
including methods, datasets, and preliminary findings, largely
The impromptu crowd science movements discussed in
this paper, centered on the Bechdel Wallace test and the wiki, share both features: anybody is welcome
to participate, and there is wide public availability of methodic
guidelines, datasets and various analysis results. The key
difference, in relation to the typical citizen science projects,
consists in their independence from scholarly institutions.
They are not organized by scientific teams, and their results
are not published in academic venues. They are taken forward
by contributors whose scientific expertise is unknown. They
have also been largely ignored as topics of scholarly research,
with few exceptions - such as Selisker’s discussion of the
Bechdel Wallace test [3] or Börzsei’s thesis on literary
criticism in new media [4]. Unlike Wikipedia, a collaborative
movement which aims to synthesize existing, published
knowledge in an encyclopaedic style, the two impromptu
crowd-science movements aim to create new knowledge,
largely independently from any professional scientific project.
Yet, there include both explicit1 and implicit dialogues with
ideas and concepts from academic fields. TVTropes pages
often can be read as a respecification of concepts from
scientific fields, for the purpose of understanding creative
works. As an example, the “Establishing Character Moment”2
can be considered a discursive respecification of the concept
of “first impressions” from social psychology [5],
accomplished with the community’s specific focus and
See for example the TVTropes “Books on Trope” page:
The TVTropes “Establishing Character Moment” page:
From a disciplinary perspective, both movements are
related to the humanities and social sciences and are focused
on a descriptive mission, identifying facts, types and
tendencies3. The Bechdel-Wallace movement pursues a
critical agenda, but strives for ideological
neutrality. Yet, their knowledge-making and falsification
infrastructure is substantially less developed than in
professional scientific programmes. Overall, the two
movements are significant contributors to knowledge through
their descriptive work, creating comprehensive representations
of mediated communication.
This paper aims to identify and compare the specific
contributions to knowledge of the two impromptu crowd
science movements. We will discuss the types of data,
analyses and findings of the two communities, and the role of
collaborative and collective contributions, focusing on the wiki. The background for our discussion on the
Bechdel Wallace test movement can be found in [1].
A. Variations in the division of labor
Franzoni and Sauermann [2] propose a classification of
crowd science projects according to the complexity of tasks
delegated to contributors, and their expected level of skills.
Most citizen science projects outsource well-structured
independent subtasks, which require only basic skills. Galaxy
Zoo4 is a typical instance, in which contributors classify
galaxies, displayed in online photographs, according to their
shapes. This task requires only a couple of mouse clicks per
classification. These projects represent the “distributing
coding” type of citizen science. Other crowd science projects
are similar in task structure but require higher levels of skills
for identifying the right specimens, amounting to “distributed
data collection” projects – such as the ability to identify birds
from a given species. If tasks also become more complex, we
find “distributed sub-problems” projects, such as Foldit5 – in
which contributors were required to solve complex visual
puzzles contributing to the study of protein folding. Even
more complicated and less pre-structured tasks lead to
“collective problem-solving” projects, such as in Polymath6,
where contributors with professional expertise share their
efforts to tackle difficult challenges.
Starting from this classification of contributors’
involvement, the paper aims to answer the following question:
What kinds of discovery work, learning and expertise are
encouraged in the two impromptu crowd science movements
under study?
B. Disciplinary variations
Kullenberg and Kasperowski [6] review Web of Science
(WoS) indexed academic publications referring to crowd
science to identify patterns in its distribution and evolution.
They conclude that the two largest threads of research relying
There are also some attempts at formulating explanations, particularly
relating gender representations in film to the gender composition of the
production team.
The Galaxy Zoo project:
The Foldit project:
The Polymath project:
on crowds to complete scientific work consist in: a) biology,
conservation and ecology, where contributors collect and
classify empirical data; b) geographic information research,
where contributors collect geographic information. Astronomy
and bioscience have emerged as the fields that took advantage
of the breakthrough of digitally based crowd science projects
in the 2000s (ibid). The authors remark the virtual absence of
projects in the social sciences and humanities, at least from
the population of projects generating scientific publications in
Still, digital humanities have also explored this novel form
of distributed scholarly work. Carletti et al. [7] review a
variety of humanities projects that rely on crowds,
distinguishing initiatives that ask contributors to optimize an
existing collection, or to create new resources. The first
category includes projects of: crowd-based curation through
tagging, selection or classification, revision through
transcription or correction, or location through mapping or
other forms of geographical matching. The public may also
contribute to creating novel items in collections, by
documenting private or public events, or by adding value to
locations through storytelling.
Starting from this disciplinary distinction, we observe that
both instances of impromptu crowd science that we discuss,
the Bechdel Wallace test movement and the
community, are relevant for social and humanistic disciplines.
They are related to media critique and also to a variety of
social fields that inform the study of creative works, such as
philosophy, sociology, psychology, gender studies etc. This
paper will thus try to address the following question:
What is the relevance of collaborative and collective
participation in these two knowledge creation movements,
given their disciplinary orientation towards social and
humanistic research?
Last but not least, in order to specify the added value of
these two impromptu crowd science movements for scholarly
research, we will discuss the issue of how academic
researchers’ work can derive value from these movements’
How can scholarly research enter into a direct dialogue
with impromptu crowd science findings?
The concept of impromptu crowd science [1] was
proposed to describe digital crowds who take part in
distributed knowledge-creation activities methodically, in the
absence of any scientific team to initiate or coordinate their
work. Such contributors rely on concepts, methods and
instruments to generate empirical data, to analyze it and to
report findings, usually in publications of broad interest such
as blogs, wikis, or digital magazines.
We argued that the Bechdel-Wallace test has become the
instrument of choice for a thread of digitally-mediated,
popular media critique. A brief summary: the Bechdel-
Wallace test, introduced in 1985 by graphic novelist Alison
Bechdel in her comic strip “The Rule”[8], asks whether a film
satisfies three conditions:
1) There are at least two women characters7;
2) That talk with one another…
3) … about something else than a man.
The test, remarkable through its simplicity and through its
unexpectedly high fail rate for Hollywood productions [9]–
[12], gained popularity in the 2000’s and, since then, it has
enabled thousands of people to assess films, record their
conclusions on a collaborative archiving platform [13] and,
occasionally, analyze available data [14] [15] and discuss the
current state and the evolution of gender representation in
films [16] [12]. People who use the Bechdel Wallace test
publish, discuss and archive their findings on a variety of
online sites and platforms. In counterdistinction, the second
example,, offers a more centralized instance of
an impromptu crowd science movement. is a wiki platform founded in 2004 by a
programmer working under the name of Fast Eddie8. It is
currently owned by three partners, and managed by a team of
five members. In December 2014 TVTropes successfully
launched a Kickstarter campaign to redesign the site, earning a
total of USD 105.186 from 3109 backers9. This is an indicator
of the popularity of the site and the commitment of its
contributors and readers.
A. Data production in the two impromptu crowd science
movements is dedicated to the identification and
classification of media tropes. The platform definition for
tropes is: “a storytelling device or convention, a shortcut for
describing situations the storyteller can reasonably assume the
audience will recognize. Tropes are the means by which a
story is told by anyone who has a story to tell. We collect
them, for the fun involved”10. To quote Madrugada, one of the
project administrators interviewed in 2011 by Meeks [17]:
“What we do here? Pattern-spotting. Pattern analysis.
Ingredient identification. By that last one, I’m talking about
treating fiction the way some folks treat a food they’ve never
had before; tasting it and then trying to figure out what all
went into making it”. The platform encourages a certain
detachment and analytical stance towards creative works,
offering participants both a set of inputs for their judgments
(specific tropes and a local theory of tropes), and a modulation
of their relationship with fiction [18].
The quest for tropes is by no means unique to this wiki.
Borzsei [19] identifies a functional similarity between
TVTropes use of the ‘trope’ concept and Umberto Eco’s
In current practice the first condition refers only to characters who have
TVTropes “About us” page:
The TV Tropes Revitalization Project:
TVTropes “Tropes” page;
definition of “intertextual archetypes” (p. 3-4). She traces the
scholarly roots of TVTropes to early 20th century, through the
literary movements of Russian Formalism and archetypal
literary criticism (p. 6). The wiki also includes a dedicated
section for “Books on tropes”, introducing it with the
observation “We here at TV Tropes are not the first to collect
tropes and try to put them in some semblance of order”11.
These acknowledged conceptual connections situate
TVTropes in a position of dialogue with a scholarly
community interested in media analysis and critique.
Unlike scholarly work, involving one or several highly
qualified individuals as the authors of analyses and
publications, TVTropes engage thousands of contributors in
the distributed, collaborative work of trope identification and
classification. How can we understand the complexity of
contributors’ tasks, in order to map TVTropes in the
continuum of crowd science projects outlined by Franzoni and
Sauermann [2]?
1) Data production on
The main task of “tropers”, the name under which
TVTropes contributors self-identify, is to pinpoint tropes,
describe them, and match them with creative works. Tropes
are patterns – but not all patterns are tropes. For a given
pattern to be a trope, it needs to convey meaning to its
audience, to function as a resource for authors in
communicating with their publics. This means that the basic
unit of knowledge created on this wiki, the trope, is usually
not an easily identifiable, self-evident item. Whether a certain
recurrent situation amounts to a trope, or not, is often a matter
of controversy. This is why TVTropes includes a complex set
of guidelines and an online debate area for tropers aiming to
pinpoint a specific trope and put it into words: the “Trope
Launch Pad”12.
The methodological guidelines include pages such as:
What is a trope?13 What is not a trope14? How can a trope be
used in a given work15? For example, a trope can be played
straight – but it can also be inverted, subverted, parodied,
averted, played for laughs, or played for drama – among
Contributors who have found a candidate for a trope are
encouraged to submit it to the Trope Launch Pad, and to
discuss in the community whether it is “tropable” or not. This
is also the place to find the trope “a snappy name16” and to
identify illustrative examples, in collaboration with other
TVTropes “Books on Trope” page:
The Trope Launch Pad:
TVTropes “Trope” page:
TVTropes “Not a trope” page:
TVTropes “Playing with a trope” page:
The Trope Launch Pad:
This construction of guidelines and collaborative settings
indicate that the first step in the production of empirical data
in the TVTropes community is a complex task. Moreover, it is
a task that relies both on collaborative, dialogical decision
making, and on cumulative contributions from individuals that
do not interact directly – which we have termed ‘collective
contributions’. Collaboration is required, among others, to
make common decisions on whether a pattern is worth to
become a trope, and to fine tune its title. Collective
contributions include examples on the trope page, which a
contributor usually adds without requiring debate or collective
If we examine data production as a process of trope
discovery, then guidelines for trope formulation are the
equivalent of disciplinary conditions of truth. As such, they
are critical for the evolution of this community of knowledge
making, since the explicit and implicit rules of trope
identification will determine what counts as the basic fact in
this line of inquiry17.
After a trope has been formulated, validated and has
received its own page on the wiki, contributors continue to
improve on this description by adding examples of works that
illustrate the trope, and by indexing the trope with old or new
indexes. Each trope includes, on its page, a series of examples
classified by media – ranging from several items to hundreds
of examples. There are also hundreds of indices that
contributors can use to classify tropes, many of them listed
under the Index Index18. Matching tropes with specific works
and with indexes is, as a rule, an individual contribution that
does not require collective approval, although others may
disagree and require argumentation and possibly conflict
arbitration, as in any wiki authorship system.
2) Data production in the Bechdel-Wallace test movement
Unlike the work of identifying and formulating tropes, the
task of assessing whether a film passes or fails the Bechdel
Wallace test is more straight-forward. The first two conditions
can be a matter of interpretation, but it happens quite rarely –
that is, whether there are two named women characters in a
film, who talk to one another. The third condition raises more
often divergent opinions, because whether a conversation
counts as being “about a man” or about something else is
sometimes not obvious. For example, the
contributors for Star Wars: The Force Awakens rated it as
passing all three conditions, noticing that “Maz Kanata and
Rey talk about Rey's Destiny.” Still, was this discussion not
about Luke and Vader, which are men? The outcome was
decided by the site admin on the basis of collaborative
argumentation on the movie page19. In similar cases, coding a
film as a pass or a fail for the third condition depends on
collaborative interpretation. In other instances, the first
contributors may omit one conversation that would qualilfy
the film as a pass; the diagnosis is updated when a later
contribution invokes a dialogue sequence that is accepted as
valid (see for example the discussion on the Batman v
I am grateful for this insight from an anonymous reviewer.
TV Tropes Index Index page:
Star Wars: The Force Awakens on
Superman film20). In such instances, the final verdict relies on
the collective, distributed effort of observing the film,
memorizing relevant sequences, and contributing them on the platform.
Contributors’ work Bechdel-Wallace
test movement
diagnosis for the
Identifying “tropable”
Formulating the trope
Specifying trope use
(“playing with”)
Collective data
(movie sequences)
for the pass/fail
Creating links:
- Matching tropes with
- Indexing tropes
Data analysis Individual or group
authorship, on
individually or
elaborated datasets
authorship of trope
indexes, relationships,
Conceptual work Individual or group
Wiki page
presentations on
authorship of meta-
discussion pages
Collaborative emotion
work: ethical
positioning, humor,
B. Data analysis in the two impromptu crowd science
As discussed in [1], people interested in the Bechdel-
Wallace test have taken the opportunity to analyze the rich
datasets generated by or by other forms of
coding. The approach has been mostly quantitative, involving
forms of data visualization, formulation of trends, and
correlating pass/fail information with data about films’
production teams [14] or financial success [16].
On the contrary, the approach in working with tropes on is dominantly qualitative. Tropers’ work
consists, to a large extent, in establishing links between tropes,
between tropes and creative works, and between tropes and
classification devices such as indices, meta-tropes and trope
patterns. This makes a very densely
interconnected site, at least in comparison with other notable
wikis such as Wikipedia [17]. Contributors’ work of analysis
of one or multiple tropes therefore include operations such as:
Tropes classifications through indices, either among
those already formulated, or by creating new indices;
Identifying relations between tropes – such as meta-
tropes, sub-tropes, sister tropes, or other related tropes;
Identification of patterns among tropes, such as
‘sorting algorithms’ or ‘sliding scales’21.
Batman v Superman on
TVTropes Sorting Algorithm of Tropes page:
Knowledge creation
Bechdel-Wallace test
Use of digital
Distributed: archival
platform, blog and
journal articles, wiki
Centralized: the wiki
(and similar wikis)
Main empirical data Pass/fail results for
specific films
Trope examples in
various works
Approach in dealing
with data
Mostly quantitative Mostly qualitative
Analysis Visualizations
Correlations with film
authorship, financial
Trope classifications
Trope patterns
Trope evolution
Trope reception
Conceptual work Discussion of the test
in relation to women
representation in media
Impasse in identifying
the value of the test
What are tropes?
What are the trope
patterns specific to
various topics (gender,
race, death, morality
Contributors’ gradual
There is no clear
learning trajectory or
‘Tropers’ are expected
to learn the troping
practice by
participation in the
There are also other forms of analysis, such as studying the
evolution in time of a certain trope (on the page “Discredited
trope”22), or the public reception of tropes, especially as they
travel to a different cultural context, across space or time (see
the page on “Unfortunate Implications”23).
The results of these analyses are published on the wiki, and
are tightly interlinked with relevant tropes and creative works.
There are also several external publications that rely on data
created by Such works include Harris’ Periodic
Table of Storytelling [20], that uses tropes’ title and
popularity; Meeks’ analysis of network patterns on the wiki
[17], analyzing links; Assogba, Ros and Vallandingham’s
Stereotropes [21], analyzing the ‘always female’ and ‘always
male’ tropes.
C. Conceptual work in the two impromptu crowd science
As discussed in [1], the conceptualization effort related to
the Bechdel Wallace test has been focused on clarifying the
meaning of the test. If it is an indicator of the quality of
women representation in films, what precisely does it
indicate? Contributors have offered a range of answers,
without finding a distinctive value for the test. We argue in [1]
that the dialogue between scholarly research and the Bechdel
test movement has managed to overcome this impasse.
Specifically, Selisker [3] argues that the test highlights the
quality of feminine characters’ networks, rather than the
quality of characters themselves.
TVTropes contributors are engaged in a broader
conceptualization work. As discussed before, we find on the
TVTropes Discredited Trope page:
TVTropes Unfortunate Implications page:
platform a growing conceptualization of tropes, including: a
discussion of what is a trope and what is not a trope; a
formulation of types of tropes and possible relationships
between tropes; a typology of tropes’ possible uses in a
creative work.
Since tropes cover a wide range of topics, we also find on
the platform emerging conceptualizations of field-specific
representations in media. Based on lists, classifications and
patterns of tropes, contributors interact with an evolving
conceptualization of gender and race representations in a
variety of media – as well as representations of morality,
beauty, or death, among others. All these discussions result
from collaborative authorship and cumulative, collective
This diverse collection of trope patterns, on different
topics, makes TVTropes a useful resource for scholars
interested in detailed studies of a specific creative work. As an
example, we can examine Ubisoft’s game Valiant Hearts: The
Great War [22]. Researchers have studied this game to
understand the role of video games in digital commemoration
[23] and in representing historical events [24]–[26]. The game
is discussed on on a dedicated page24, including
a list of in-game tropes, ordered alphabetically, identified by
platform contributors. Several tropes are relevant for
examining the game’s portrayal of soldiers – including those
dedicated to combatants from colonies (“Hero of another
story”), to representations of generals, sergeants, deserters,
and heroes, and also to the portrayal of the “Dog hero”. The
page also includes the “Grey and gray morality” trope25,
framing the game in a broader landscape of approaches to
morality in creative works, as discussed in pages such as the
“Shades of conflict” scale26 and the Neutrality Index27.
D. Contributor specialization
The Bechdel Wallace test movement relies on two main
tasks: production of data, and analysis. Production of data
does not require any special individual skills; the collective
accumulation of evidence on a given film, with occasional
collaborative decision making, leads to a diagnosis of pass /
fail. Analyses of datasets of pass/fail data require statistical
analysis skills, which are not specific to this topic. We can
conclude that this movement does not present a clear path of
skill evolution or topic-specific learning.
In counterdistinction, the TVTropes movement offers a
clear path of evolution for the would-be troper. Contributors
have much to learn, starting from the initial tasks of trope
identification and formulation, to more complicated tasks of
creating new indices, finding novel trope patterns and
relationships between tropes, or discussing the relationships
between tropes and the real life.
TVTropes Valiant Hearts page:
TVTropes Grey and Gray morality page:
TVTropes Shades of conflict page:
TVTropes Neutrality Index page:
Unlike other knowledge making and sharing platforms,
such as Stack Exchange or Wikipedia, TVTropes does not
offer contributors a reputation system. User profiles are
minimalistic, and there is no badge architecture to ‘talk’ about
users’ achievements and skills [27], [28], [29]. There is no
gamification to support user engagement through simple
gameplay [30]. Still, the quizzical and often self-referential
writing style creates a playful atmosphere that supports
enjoyment and challenges writers to live up to the
community’s style. This also opens a secondary path of user
specialization, in learning the community rhetoric.
This paper discusses the concept of impromptu crowd
science and illustrates it through a comparison between two
instances: the TVTropes wiki community and the Bechdel
Wallace test movement. Table I and Table II present a
synthetic comparison of crowd contributions and knowledge
creation in the two movements.
Both movements create new knowledge, in the field of
media critique, relevant for social and humanistic disciplines.
While the Bechdel Wallace movement relies on relatively
straightforward data production and dominantly quantitative
analyses, the reciprocal is true of Trope
specification is a relatively complex task, and contributors are
also engaged in broader activities of qualitative analysis and
conceptualization. Collaboration is essential for tropers’
activity, because tropes are identified through their capacity to
resonate with a public – thus requiring a shared validation.
The two knowledge creation movements work
independently of any scholarly research project or institution.
At the same time, there is considerable potential in linking
academic research with the work of impromptu crowd science
contributors. offers a comprehensive and
detailed conceptualization of tropes in various media, and
broad list of field-specific and work-specific tropes. These
results of tropers’ collaborative and cumulative contributions
offer scholars a valuable starting point for analyses of specific
creative works.
[1] C. Rughiniș, R. Rughiniș, and B. Humă, “Impromptu Crowd Science
and the Mystery of the Bechdel-Wallace Test Movement,” in
Proceedings of the 2016 CHI Conference Extended Abstracts on Human
Factors in Computing Systems - CHI EA ’16, 2016, pp. 487–500.
[2] C. Franzoni and H. Sauermann, “Crowd science: The organization of
scientific research in open collaborative projects,” Res. Policy, vol. 43,
no. 1, pp. 1–20, Feb. 2014.
[3] S. Selisker, “The Bechdel Test and the Social Form of Character
Networks,” New Lit. Hist., vol. 46, no. 3, pp. 505–523, 2015.
[4] L. K. Börzsei, “Literary Criticism in New Media,” Loránd Eötvös
University, 2012.
[5] B. Humă, “Enhancing the authenticity of assessments through grounding
in first impressions.,” Br. J. Soc. Psychol., Oct. 2014.
[6] C. Kullenberg and D. Kasperowski, “What Is Citizen Science? – A
Scientometric Meta-Analysis,” PLoS One, vol. 11, no. 1, pp. 1–16,
[7] L. Carletti, D. McAuley, D. Price, G. Giannachi, and S. Benford,
“Digital Humanities and Crowdsourcing: An Exploration,” 2013.
[8] A. Bechdel, “The Rule,” Dykes to Watch Out For, 2005. [Online].
[9] K. McKinney, “Almost half of 2015’s top movies failed the Bechdel
test,”, 2015. [Online]. Available:
[10] Contributors of, “Bechdel Test Movie List - Stats and
Graphs,” 2016. [Online]. Available:
[11] A. Sarkeesian, “The Oscars and The Bechdel Test,” Feminist Frequency,
2012. [Online]. Available:
[12] Silk Team, “Women in Film: A Data Analysis of 1500 Movies on
Bechdel Test Criteria,” 2015. [Online]. Available: http://women-in-
[13] Contributors of, “Bechdel Test Movie List,” 2015.
[Online]. Available:
[14] L. Friedman, M. Daniels, and I. Blinderman, “Men making movies
about men,” 2015. [Online]. Available:
[15] D. Mariani, “Visualizing the Bechdel Test,” 2013. [Online]. Available:
[16] W. Hickey, “The Dollar-And-Cents Case Against Hollywood’s
Exclusion of Women,” FiveThirtyEight, 2014. [Online]. Available:
[17] E. Meeks, “TVTropes Pt 1: The Weird Geometry of the Internet,” 2011.
[Online]. Available:
[18] R. Rughiniș, “Serious Games as Input versus Modulation: Different
Evaluations of Utility,” in 26th Conference on People and Computers
BCS-HCI 2012, 2012, pp. 175–184.
[19] L. K. Börzsei, “Literary Criticism in New Media,” Loránd Eötvös
University, 2012.
[20] J. Harris, “Periodic Table of Storytelling,” 2011. [Online]. Available:
[21] Y. Assogba, I. Ros, and J. Vallandigham, “Stereotropes.” [Online].
[22] Ubisoft, “Valiant Hearts. The Great War.” 2014.
[23] R. Rughiniș and Ștefania Matei, “Play to Remember: The Rhetoric of
Time in Memorial Video Games,” in Human-Computer Interaction:
Interaction Technologies Volume 9170 of the series Lecture Notes in
Computer Science, 2015, pp. 628–639.
[24] C. Kempshall, “Pixel Lions – the image of the soldier in First World
War computer games,” Hist. J. Film. Radio Telev., vol. 35, no. 4, pp.
656–672, 2015.
[25] Ş. Matei, “Games as a Genre of Historical Discourse. The Past on Fast
Forward,” in Proceedings of DiGRA 2015: Diversity of play: Games –
Cultures – Identities, 2015.
[26] C. Kempshall, “‘They Will Not Be Able To Make Us Play It Again
Another Day’ — The End in First World War Games,” in The First
World War in Computer Games, London: Palgrave Macmillan UK,
2015, pp. 82–95.
[27] R. Rughiniș, “Talkative Objects in Need of Interpretation. Re-Thinking
Digital Badges in Education,” in CHI ’13 Extended Abstracts on Human
Factors in Computing Systems, 2013, pp. 2099–2108.
[28] R. Rughinis and S. Matei, “Badge Architectures as Tools for Sense-
Making and Motivation in Engineering Education,” International Journal
of Engineering Pedagogy (iJEP), vol. 5, no. 4. pp. 55–63, 2015.
[29] J. Antin and E. Churchill, “Badges in Social Media: A Social
Psychological Perspective,” in Proceedings of the 2011 annual
conference on Human factors in computing systems CHI 11, 2011, pp.
[30] R. Rughiniș, “Gamification for Productive Interaction. Reading and
Working with the Gamification Debate in Education,” in The 8th Iberian
Conference on Information Systems and Technologies CISTI 2013,
2013, pp. 1–5.
... Another example of complex networks are those generated by crowd science movements [26], such as TV Tropes. TV Tropes is a wiki community created in 2004 that unites enthusiasts who have been collecting tropes collaboratively since 2014. ...
... It currently has over 28,000 pages of tropes, and 77,000 of cultural works, including films, TV, comics, video games, books and other media. In [26], this wiki community was studied, indicating that the produced network generates knowledge relevant to social and humanities disciplines, and that it can serve as a starting point for academic studies. However, they conclude that the definition of tropes is a relatively complex task, and that the contributors of this community approach it using a qualitative and conceptual approach. ...
Full-text available
Creating a story is a challenging task due to the the complex relations between the parts that make it up, which is why many new stories are built on those cohesive elements or patterns, called tropes that have been shown to work in the past. A trope is a recurring storytelling device or pattern, or sometimes a meta-element, used by the authors to express ideas that the audience can recognize or relate to, such as the Hero’s Journey . Discovering tropes and how they cluster in popular works and doing it at scale to generate new plots may benefit writers; in this paper, we analyze them and use a principled procedure to identify trope combinations, or communities, that could possible be successful. The degree of development of these different communities can help us identify areas that are under-developed and, thus, susceptible to such a type of development. To detect these communities, with their associated degree of development and interest, we propose a methodology based on scientometric and complex network analysis techniques. As a secondary objective, we will obtain a general perspective in the trope and films network: the tropesphere . We have used a dataset of 10,766 movies and 25,776 tropes associated with them, together with rating, genres and popularity. Our analysis has shown that not only there are different trope communities associated with specific genres, and that there are significant differences between the rating and popularity of these communities but also there are differences on the level of development between them: emerging/declining, specific, transversal or motor.
... Data mining has also been applied to HS. Indeed there have been two Data Mining Challenges (AAIA'17 3 and AAIA'18 4 ) using this game as a test-bed. However, the 2017 Challenge and the derived papers [18,19] were devoted to help the AI win the game, whereas the 2018 edition and related papers [20,21] had the aim to predict the win-rates for specific decks. ...
Digital Collectible Cards Games such as Hearthstone have become a very prolific test-bed for Artificial Intelligence algorithms. The main researches have focused on the implementation of autonomous agents (bots) able to effectively play the game. However, this environment is also very attractive for the use of Data Mining (DM) and Machine Learning (ML) techniques, for analysing and extracting useful knowledge from game data. The objective of this work is to apply existing Game Mining techniques in order to study more than 600,000 real decks (groups of cards) created by players with many different skill levels. Data visualisation and analysis tools have been applied, namely, Graph representations and Clustering techniques. Then, an expert player has conducted a deep analysis of the results yielded by these methods, aiming to identify the use of standard - and well-known - archetypes defined by the play methods will also make it possible for the expert to discover hidden relationships between cards that could lead to finding better combinations of them, enhancing players’ decks or, otherwise, identify unbalanced cards that could lead to a disappointing game experience. Moreover, although this work is mostly focused on data analysis and visualization, the obtained results can be applied to improve Hearthstone Bots’ behaviour, e.g. predicting opponent’s actions after identifying a specific archetype in his/her deck.
... Lastly, it is possible that lay people involvement surpasses the conventional schemes proposed by the scientific literature (Callon, 1999), as in the case of impromptu citizen science projects, that are reported in the bottom right quadrant of Figure 3. These projects are peculiar in that they are independent "…from scholarly institutions, … are not organized by scientific teams, and their results are not published in academic venues" [Rughiniș, (2016), p.1]. Lay people are relatively independent in designing and implementing the research activities. The availability of public methodological guidelines directs the interactions between lay people, thus fostering their engagement in knowledge co-production. ...
Lay people involvement is arising as a viable alternative to conventional modes of knowledge generation and dissemination. However, little is known about the roles that lay people may play in scientific research; moreover, management and organizational challenges characterize the implementation of a socially-distributed mode of knowledge production. To shed light into this issue, we performed a realist literature review, that involved 81 papers. The study findings suggested that lay people may variously contribute in knowledge co-production; their role depends on the degree of autonomy they have and on the type of relationship they establish with expert scientists. The propensity of expert scientists and lay people to bring diverging inputs in knowledge co-production should be acknowledged and carefully addressed, in order to avoid drawbacks on the knowledge production ability of research institutions. The outcomes of citizen science projects should be assessed from both an economic and a societal point of view.
... Crowdsourcing platforms support collective action on a variety of tasks, such as problem-solving and innovations, data processing, knowledge aggregation and knowledge creation that require either simple or expert skill sets [4]. Citizen science project tasks, in particular, include crowd processing, collaborative problem-solving contributions [6] or creating knowledge [10]. There have been several studies designing workflows to help workers to increase the quality of their work and designing efficient voting algorithms to allocate the accurate score to each response [e.g., 1,8]. ...
Conference Paper
Contributors to online crowdsourcing systems generally work independently on pieces of a product. Task interdependencies may require collaboration, mostly implicitly, to develop a final product. Even individuals engaged in a group conversation may not stay with the same group for long, i.e., the group is an "occasional group". Further, occasional group interactions are often not well supported by the platforms. These factors particularly impact the process of group learning. I plan to develop a model of group learning in occasional groups and to test the model by using it to design a knowledge-based chatbot to improve learning in occasional groups.
The book emphasises the importance of implementing cybersecurity curricula in the university environment, in order to prepare future experts in the field. The book brings together the contributions of professors, PhD students, researchers, experts and trainers, for the development of a reference framework for education, research and cooperation in the field of cybersecurity.
Conference Paper
Full-text available
We propose the concept of 'impromptu crowd science' to analyze the Bechdel-Wallace movement. We argue that the practice of using this test to evaluate movies generates a form of digitally distributed knowledge that reveals a challenging conceptual impasse. The current organization of the movement, mainly through blog and newspaper articles, quantitative analyses and wiki syntheses, does not encourage the formulation of theoretically innovative answers, remaining captive in a rhetorical style of balancing pros-and-cons. We propose that hybridizations between such impromptu crowd science and academic inquiries could stimulate crowd theorizing, and we extend this challenge as an invitation for HCI researchers.
Full-text available
Context: The concept of citizen science (CS) is currently referred to by many actors inside and outside science and research. Several descriptions of this purportedly new approach of science are often heard in connection with large datasets and the possibilities of mobilizing crowds outside science to assists with observations and classifications. However, other accounts refer to CS as a way of democratizing science, aiding concerned communities in creating data to influence policy and as a way of promoting political decision processes involving environment and health. Objective: In this study we analyse two datasets (N = 1935, N = 633) retrieved from the Web of Science (WoS) with the aim of giving a scientometric description of what the concept of CS entails. We account for its development over time, and what strands of research that has adopted CS and give an assessment of what scientific output has been achieved in CS-related projects. To attain this, scientometric methods have been combined with qualitative approaches to render more precise search terms. Results: Results indicate that there are three main focal points of CS. The largest is composed of research on biology, conservation and ecology, and utilizes CS mainly as a methodology of collecting and classifying data. A second strand of research has emerged through geographic information research, where citizens participate in the collection of geographic data. Thirdly, there is a line of research relating to the social sciences and epidemiology, which studies and facilitates public participation in relation to environmental issues and health. In terms of scientific output, the largest body of articles are to be found in biology and conservation research. In absolute numbers, the amount of publications generated by CS is low (N = 1935), but over the past decade a new and very productive line of CS based on digital platforms has emerged for the collection and classification of data.
Full-text available
In this paper we argue that badge architectures are narrative, material and performative repertoires which can be meaningfully employed to provide university students with a coherent understanding of education. Badge architectures function as powerful interpretative tools through which students make sense of their immediate learning world. However, the efficiency of badges in education depends on associated structures of distribution, procurement and display that articulate modes of participation within a local community of practice. By considering our experience in the design, implementation and evaluation of the badge system ‘RL Hit List’, we claim that more insightful perspectives emerge from considering motivation as a socio-technical accomplishment than as a state of mind. This view might be read as an invitation for designers and researchers to reconsider the intrinsic - extrinsic dichotomy in assessing the value of badges to motivate students.
Full-text available
This article examines first impressions through a discursive and interactional lens. Until now, social psychologists have studied first impressions in laboratory conditions, in isolation from their natural environment, thus overseeing their discursive roles as devices for managing situated interactional concerns. I examine fragments of text and talk in which individuals spontaneously invoke first impressions of other persons as part of assessment activities in settings where the authenticity of speakers’ stances might be threatened: (1) in activities with inbuilt evaluative components and (2) in sequential contexts where recipients have been withholding affiliation to speakers’ actions. I discuss the relationship between authenticity, as a type of credibility issue related to intersubjective trouble, and the characteristics of first impression assessments, which render them useful for dealing with this specific credibility concern. I identify four features of first impression assessments which make them effective in enhancing authenticity: witness positioning (Potter, 1996, Representing reality: Discourse, rhetoric and social construction, Sage, London), (dis)location in time and space, automaticity, and extreme formulations (Edwards, 2003, Analyzing race talk: Multidisciplinary perspectives on the research interview, Cambridge University Press, New York).
The fourth and final chapter deals with how computer games portray the end of the First World War, both in regards to the actual ending and how and when the games themselves draw to a close. It is through understanding how both the war and games about it end that we can gain a greater understanding of how we are expected to remember it and the lessons that we are supposed to learn. It is the norm in computer games to allow the player a sense of triumph or a moment of success at the point of completion. How First World War games navigate this situation when portraying a war that is usually stripped of triumphalism is of particular interest. Whilst many of these games highlight the tragic nature of the war many of them also provide the player with a chance of triumphant victory.
This essay describes the popular Bechdel Test—a measure of women’s dialogue in films—in terms of social network analysis within fictional narrative. It argues that this form of vernacular criticism arrives at a productive convergence with contemporary academic critical methodologies in surface and postcritical reading practices, on the one hand, and digital humanities, on the other. The data-oriented character of the Bechdel Test, which a text rigidly passes or fails, stands in sharp contrast to identification- or recognition-based evaluations of a text’s feminist orientation, particularly because the former does not prescribe the content, but merely the social form, of women’s agency. This essay connects the Bechdel Test and a lineage of feminist and early queer theory to current work on social network analysis within literary texts, and it argues that the Bechdel Test offers the beginnings of a measured approach to understanding agency within actor networks.
Conference Paper
This paper examines video games that commemorate historical events, identifying ‘family resemblance’ features and specific rhetorical resources. We argue that the commemorative character of a game derives, typically, from four interrelated qualities: invoking a specific historical event, claiming a truthful representation, inviting empathic understanding, and offering players opportunities for reflection. Starting from the observation that time has an important role in achieving commemorative gameplay, we discuss several games in terms of narrative and procedural rhetoric, with focus on time-related mechanics. We propose a repertoire of design resources to assist the creation of meaningful games for remembrance.
This article examines the imagery and portrayal of First World War soldiers within modern computer games. The role of a soldier is a staple of computer games and the Second World War remains the most referenced and depicted conflict within the medium. Second World War games are highly influenced by cinematic portrayals of the conflict. The depiction of First World War soldiers, however, is a more recent and complicated development. Films focused on the First World War are not as populous as those depicting the Second and they also exist within different narrative conventions. By examining games such as Valiant Hearts: The Great War, Trenches 2, Verdun 1914–1918 and Trench Warfare, this article argues that soldiers fall into two major groups largely defined by trends in narrative portrayal: the Individuals and the Mass. However, because of the more morally complicated nature of the First World War and the audience’s expectations regarding the deadly nature of the conflict, figures within these groups come to represent competing visions of heroism and survival whilst in war time. By also considering the implications of these portrayals along national lines, this article highlights the complicated nature of media portrayals of the war and how computer games are evolving in a new direction to cope with it.
I examine current debates concerning digital badges in education, pointing to less remarked upon topics. By investigating badges as motivators, I conclude that a focus on badges as rewards has downplayed the importance of badge conditionalities ('tails') and entitlements ('antennae'), and their always situated effectiveness. Current discussions of badges as evidence-based credentials understate the interpretive work required to make sense of badge collections. I propose two heuristic definitions of badges as 'routes through an activity system' and as 'genres of hint-based multi-authored testimony of learning'. Alternative definitions are invited, as tools for thought.