Conference PaperPDF Available

Defining Human-Machine Micro-Task Workflows for Constitution Making


Abstract and Figures

This paper presents a novel task-oriented approach to crowdsource the drafting of a constitution. By considering micro-tasking as a particular form of crowdsourcing, it defines a workflow-based approach based on Onto2Flow, an ontology that models the basic concepts and roles to represent workflow-definitions. The approach is then applied to a prototype platform for constitution-making where human workers are requested to contribute to a set of tasks. The paper concludes by discussing previous approaches to participatory constitution-making and identifying areas for future work.
Content may be subject to copyright.
Defining Human-Machine Micro-Task Workflows for
Constitution Making
Nuno Luz1,2, Marta Poblet3, Nuno Silva1, Paulo Novais2
1 GECAD (Knowledge Engineering and Decision Support Group), Polytechnic of Porto
2 CCTC (Computer Science and Technology Center), University of Minho
3 Graduate School of Business and Law, RMIT University
Abstract. This paper presents a novel task-oriented approach to crowdsource
the drafting of a constitution. By considering micro-tasking as a particular form
of crowdsourcing, it defines a workflow-based approach based on Onto2Flow,
an ontology that models the basic concepts and roles to represent workflow-
definitions. The approach is then applied to a prototype platform for
constitution-making where human workers are requested to contribute to a set
of tasks. The paper concludes by discussing previous approaches to
participatory constitution-making and identifying areas for future work.
Keywords: Micro-tasking, micro-tasks, workflows, ontologies, political
crowdsourcing, legal crowdsourcing, constitution-making, participation.
1 Introduction
Constitution-making can be broadly defined as a set of activities intended to produce
a constitution, the highest law of a state. To the UN, constitution-making “covers both
the process of drafting and substance of a new constitution, or reforms of an existing
constitution” [1]. Klein and Sajo have also defined it as a “decision-making process
carried out by political actors, responsible for selecting, enforcing, implementing, and
evaluating societal choices” [2]. Given that constitution-making may only happen
once in a generation, it is often seen as a unique moment shaping both the present and
the future of a country. As Elster has put it, “if there is one task for which ‘wisdom’
would seem highly desirable, it is that of writing a constitution” [3].
This is a pre-press version of a paper with the same title in the proceedings volume of 15th International
Conference, GDN 2015, Warsaw, Poland (June 22-26, 2015) in Lecture Notes in Business Information
Processing 218: 333-344. © Springer International Publishing Switzerland. Personal use of this material is
permitted. However, permission to reprint/republish this material for advertising or promotional purposes
or for creating new collective works for resale or redistribution to servers or lists, or to reuse any
copyrighted component of this work in other works must be obtained from Springer International
This paper reviews a few examples of how the wisdom of the crowd has been
tapped in recent constitution-making processes across the world and proposes a new
approach to write a constitution based on micro-tasking, a particular form of
Section 2 provides definitions of crowdsourcing and micro-tasking and additional
background knowledge on recent examples of constitutional crowdsourcing. Section 3
briefly reviews ontology-based micro-tasking workflows and presents Onto2Flow, an
ontology designed to retrieve structured and semantically enriched data from micro-
tasks. Section 4 applies this framework to a prototype platform that enables the micro-
tasking of a constitutional text. Section 5 discusses both the potential and limitations
of this approach. The conclusion, finally, suggests future work in this area.
2 Background
The word crowdsourcing was coined by Jeff Howe and Mark Robinson in 2006 to
represent “the act of taking a job traditionally performed by a designated agent
(usually an employee) and outsourcing it to an undefined, generally large group of
people in the form of an open call” [4]. This broad conceptualization has been
followed by a myriad of definitions of crowdsourcing drawn from different but
connected approaches: collective intelligence (CI), human computation, social
intelligence, and social computing. It also has been noted that "while human
computation (HC) is a term that is mostly used by the scientific community,
crowdsourcing (CS) is a term highly employed in the business world [5]. Despite the
variety of perspectives, all approaches highlight three key elements in crowdsourcing:
crowds, tasks, and mediating technologies.
Micro-task crowdsourcing, in particular, is a special kind of human computation
where relatively complex tasks are divided into smaller and independent micro-tasks
[5]. These micro-tasks are then modelled and published through a computational
platform (e.g. Mechanical Turk and CrowdFlower), which distributes them through a
crowd of workers.
Micro-tasks are often employed for solving large-scale problems that are often too
complex for computers to solve on their own [6]. These problems usually require a
degree of creativity (or just common sense), plus some background knowledge [7, 8].
In our view, the drafting of a constitution: (i) can be represented as a large-scale
problem that can be divided into smaller tasks; (ii) these micro-tasks can be
completed by a crowd of heterogeneous citizens with different degrees of legal
expertise (from none to expert).
2.1 Crowdsourced constitution-making
In the political and legal domains, crowdsourcing methods and tools have been used
as a means to collect input from citizens on a variety of areas, such as legal drafting,
legal reform, legal education, policy-making and human rights advocacy [912].
Crowdsourced constitution-making, in particular, was famously displayed in Iceland
in 2011 with the use of social media to collect peoples’ views and opinions on the
constitutional draft [13]. Similar initiatives were taking place almost simultaneously
in Kenya (2010), Ghana (2010-2011), Somalia (2011), Egypt (2012), and Libya
(2012), among other countries [14]. Likewise, Morocco announced a constitutional
reform in early 2011 and, shortly after, a citizen-based initiative launched,
a dedicated crowdsourcing platform fully integrated with Facebook and Twitter where
citizens could like or dislike the proposed articles and comment on them [15].
In the effort to make constitution making as participatory as possible, these
initiatives have all taped on social media (and, in some cases, e-mail and text
messages) to elicit comments from the public. In all cases, and regardless of the final
number of participants, thousands of comments were posted and eventually collected.
The analysis of how these contributions were classified a posteriori and their eventual
impact on the final drafts would require a case-by-case approach. Yet, it seems clear
that in all mentioned examples the public was invited to comment, answer questions,
vote, or “like”, but not to “write” the constitution itself. To date, crowdsourced
constitution-making has heavily relied on online deliberation, but the impact of such
deliberative processes on the final outcome is yet to be fully assessed. While
deliberative processes are core to constitution-making, we aim at a complementary
approach where the constitutional draft is also the product of coordinated micro-
tasking via the participation of a large number of participants.
2.2 Ontologies in description logics
Our approach adds a new layer to constitution-making by considering a micro-task
workflow-based approach to the drafting and refinement of the document. Drafting
and refinement workflows are modelled using ontologies, which allow a formal,
explicit and shared conceptual representation while maintaining machine
interpretability. Ontologies are formal because they are supported by unambiguous
formal logics; explicit since they make domain assumptions explicit for reasoning and
understanding; and shared for its ability to provide consensus.
Ontologies “represent the best answer to the demand for intelligent systems that
operate closer to the human conceptual level” [16]. Thus they are an appropriate
representation mechanism for environments where both human and machine agents
must interpret the data and perform a particular set of actions. Furthermore, the
inherent extensibility of ontologies allows the growing set of domain ontologies in the
Semantic Web to be re-used in the representation of workflows.
3 Ontology-based Micro-Task Workflows
Micro-tasks (or simply “tasks” from now on) can be seen as atomic operations that
produce a specific set of data. These atomic operations occur within a specific domain
of operation involving certain domain knowledge. Given a task, its domain of
operation is defined by its input and output specifications.
Onto2Flow is an approach to the representation, instantiation and execution of
workflows that represents workflows of tasks as extensions of other domain
ontologies. These extensions are called workflow-definition ontologies. Workflow-
definition ontologies assemble two different data dimensions: (a) the static domain
dimension (corresponding to the domain ontology) and (b) the dynamic task and
workflow dimension (corresponding to the Onto2Flow ontology). In this perspective,
task-definitions (or task representations) are extensions of the domain ontology,
which add an operational dynamic dimension. Fig. 1 illustrates these two dimensions
and their assemblage.
Fig. 1. Static and dynamic dimension in a workflow-definition ontology.
3.1 The Onto2Flow process
Onto2Flow assumes that domain ontologies represent the structure and semantics of
the data presented to (and retrieved from) workers. Accordingly, the approach
considers two steps (outlined in Fig. 2): (i) task-definition and workflow-definition
(the ontology of the workflow), and (ii) the instantiation and execution of the
workflow on a particular input dataset.
Fig. 2. Overview of the Onto2Flow approach.
At the stage of workflow-definition (1), the requester must clearly define the activities
involved in the workflow through a semantic model of the input and output data and
create a workflow-definition. For workflow-definitions containing task-definitions
that human workers have to solve, crowd (user) interface templates must be supplied
along with the workflow-definition ontology. The interface templates present the task
data to the worker and retrieve the submitted response.
At stage 2 (instantiation and execution of the workflow-definition), workflow-
definition ontologies can be instantiated multiple times and executed by any workflow
engine that is able to interpret the Onto2Flow ontology and apply the ground rules
established by the proposed method. Furthermore, Onto2Flow-based workflow
engines may dispatch the execution of the tasks to external micro-task execution
communities such as Mechanical Turk and CrowdFlower, or provide their own task
resolution interfaces that may interact with external social networks.
3.2 The Onto2Flow ontology: concepts and roles
The Onto2Flow ontology defines the basic concepts and roles required to represent
workflow-definitions (see Fig. 3). It captures concepts and lessons learnt from
workflow-definition languages and approaches such as the XPDL (XML Process
Definition Language) and BPMN (Business Process Modelling Notation) [17].
Furthermore, it incorporates concepts that support the crowdsourcing, distribution,
and delivery of tasks.
Fig. 3. Overview of the Onto2Flow ontology.
The concept Job represents a workflow execution environment created by a
Requester, which may contain more than one Workflow.
Activities are the interconnected components that form a workflow. There are three
main types of activities: the Workflow, the Task and the Event. Among these,
Deliverables, which include Task and Event, represent a group of activities that
require worker or external interaction through some kind of Interface.
Two main types of actors are considered by the Onto2Flow ontology: the
Requester (the one requesting the execution of a workflow) and the Worker (the one
solving the tasks of the workflow), which are either Human or Machine.
Each Actor may belong to several ActorGroup. Actor groups allow requesters to
associate and filter groups of actors for participation in particular tasks. An
ActorGroup may include a wide set of attributes, including social network analysis
clustering measurements (e.g. clusterability), improving the control of the requester
over the selection of workers. Inclusively, each ActorGroupMembership may feature
a wide set of actor specific attributes and measurements (e.g. centrality and prestige).
Workflows are graphs of activities linked through transitions, which establish a
process that delivers a specific result dataset given an input dataset.
The flow of activities in a workflow is established through Transitions. There are
six types of transitions, depending on the set of (i) incoming activities
(BasicTransition, MergeTransition or SynchronizationTransition), (ii) outgoing
activities (ParallelTransition or DisjunctTransition), (iii) whether there is one or
more conditions to be fulfilled in order to continue its execution
An Event is an external occurrence that either triggers the continuation of a running
workflow (RunningEvent) or triggers the execution of a new workflow
A Task is a set of assignments and operations on top of input data, which must be
performed by workers. The representation of a task involves multiple concepts and
roles in the Onto2Flow ontology. These concepts are:
The Assignment concept, representing the actual operationalization of the task;
Input concepts:
The Unit concepts, which represent the input unit of work given to the worker;
The UnitContext concepts, which represent relevant contextual input data that
must be presented along with the unit (and possibly related to it);
Output concepts:
The Response concepts, which represent the top-level response or output given
by the worker;
The ResponseContext concepts, which represent additional output given by the
worker, usually related to the response.
Each work unit (represented by the Unit concept) is assigned to a worker through an
Assignment. The same unit may be assigned to different workers, resulting in different
solutions to the same problem.
The execution of a workflow requires interaction with external actors and services
during the execution of Event and Task activities. While an Event is typically listened
for, and arrives through an EventInterface, a Task must be delivered to and retrieved
from workers through a TaskInterface. Thus, interfaces represent logical and/or
physical components through which the interaction with workers (machine or human)
is performed (e.g. a Web service interface, a graphical user Web interface).
The ability to represent different types of interfaces enables the specification of
distinct interfaces, commonly used on user-centric environments [18]:
Simple, where a single medium or modality is used. For instance, tasks can be
delivered to workers through a visual interface, a sound interface, or simply
through a web interface (the common case for crowdsourcing applications);
Multi-modal, i.e. capable of merging and coordinating multiple mediums and
modalities as a single interface.
Accordingly, and of particular interest in the crowdsourcing scenario, different types
of user interface implementations, such as a game interface or a mobile interface, can
be used to distribute tasks through human workers.
4 Catalan Constitution-Making Scenario
The Catalan constitution-making scenario is a prototype of a micro-tasking platform
to crowdsource the elaboration of a constitutional text. This scenario uses the
Constitute project ontology as the static domain dimension. The Constitute project is a
database of constitutional texts to search and compare constitutions across the world
[19]. On top of the Constitute project ontology, a workflow-definition following the
Onto2Flow method was built. The resulting workflow-definition, as shown in Fig. 4,
aims to take the ontology-based representation of a proposed Catalan constitution and
crowdsource its elaboration, stemming from a basic initial text [20].
The process contemplates the following tasks, all performed by human workers:
T1 - evaluates sections of the current constitution document and is performed
by any worker;
T2 - revises and updates sections of the current constitution document marked
in the previous task and is performed by expert workers;
T3 - selects the best version of a section from the set of proposed sections in
the previous task and is performed by any worker.
Fig. 4. Overview of the Catalan constitution-making workflow-definition.
The Constitute project ontology represents the constitution document through
sections. A partial illustration of the Constitute project ontology is presented in Fig. 5.
An additional set of concepts was added to the static domain dimension in order to
represent the opinion and the assessment of the constitution sections.
Fig. 5. Partial Constitute project ontology and additional assessment concepts.
4.1 The Workflow-Definition
The constitution-making workflow-definition was built using both a construction
framework prototype implementation and the Protégé ontology editor. The Protégé
ontology editor was used to establish some common axioms that are not yet featured
by the construction framework, such as the union of input and output concepts, and
inverse roles. A detailed illustration of the workflow-definition is presented in Fig. 6.
Notice how each task-definition contains a complete representation of all the concepts
and relationships involved. Also, this representation is directly mapped to the
Constitute project ontology.
Fig. 6. Task-definitions in the constitution-making workflow-definition ( represents a
dependency relationship, which can be reduced to a subsumption).
In T1, the amount of assignments per unit will correspond to the amount of
evaluations given to each section. Thus, T1 must have an amount of assignments per
unit greater or equal to X, where X is the amount of evaluations that request an update
of the section. This amount (X) is used in T2 to assess which sections must be revised
and updated.
The use of the role transitive closure onto the parent role allows all descendant
sections of the unit section to be included in the assignment and shown to the worker.
Also, regular expressions may be used to restrict the value of data -type roles. Such is
the case of the value of the header role in T1 (Section_T1 header : “/^Article/”).
4.2 The Task-Definition UI Templates
In the Catalan constitution-making scenario all tasks are solved by human workers
(volunteers). Volunteers contribute by adopting two different profiles: non-experts or
experts. Non-experts are the large majority of citizens who sign into the platform to
complete tasks in T1; experts are those volunteers designated by the requesters with
an editing role of the outputs produced by non-experts (classification, collation,
amendments). In both cases, the workflow-definition includes an UI (User Interface)
template. The UI template of T1 presents the unit section, its parent section, and all its
descendant sections to the non-expert volunteer. The volunteer is then invited to
evaluate the contents of the section (an article of the constitution) and assess whether
it needs to be: (i) updated (rewritten), (ii) removed or (ii) accepted as it is. Volunteers
can access the complete initial constitutional draft at any time to situate their
assignment into the broader picture of the full text.
The UI template of T2 presents the unit section to expert workers in the same way
as T1, including any modifications of the constitutional text by non-experts in T1. The
expert volunteer is then asked to submit a new revised section with all outputs
collected T1 classified and, if necessary, edited and collated.
Finally, the UI template of T3 presents each of the previously submitted sections
(during T2), along with the original section. In T3, all volunteers are requested to
select the best version. Fig. 7 below offers an example of the UI template of T1 as
presented to non-expert volunteers.
Fig. 7. Example assignment with the UI template of T1.
5 Conclusions and Future Work
Crowdsourcing the writing of a constitution to a large number of citizens is a complex
task that can be addressed by subdividing it to smaller units (micro-tasks). While
there are a number of examples of participatory constitution-making that involve
online deliberation, none of them offers a platform for citizens to edit the articles of
the text. Rather, their focus on eliciting and collecting opinions from public
deliberation, generally via social media, makes crowdsourcing initiatives accessory to
the drafting process developed elsewhere (e.g. in constitutional commissions).
Ultimately, this contingent aspect of crowdsourcing makes it difficult to assess the
impact of online participation on both the drafting process and the final outcome.
In our approach, writing a constitution becomes the core task. We rely here on two
well-researched conditions in the literature on the “wisdom of the crowd effect”: (i)
independence of judgment and (ii) heterogeneity of the crowd [2123]. When these
two conditions are met, the crowd can perform better than individual experts.
To date, the platform has been tested by a reduced group of 8 experts who have
provided useful feedback. Future work involves expanding the testing to larger groups
of volunteers and refine the following issues: (i) identification of sub-topics within an
article and further division of micro-tasks; (ii) credentials and role of experts; (iii)
aggregation mechanisms in T3 (e.g. ratings, rankings) to avoid inconsistencies, and
(iv) generally, mechanisms to detect and resolve conflicts between different sections
in a constitution.
Beyond addressing these different issues dealing with coordination mechanisms,
further research will also be required to tackle substantive issues on how to coordinate
the crowd itself: (i) motivation; (ii) incentives to participate; (ii) relevance and quality
of the contributions; (iii) monitoring spam and sabotage attempts, etc. The ultimate
challenge is how to engage the crowds' collective wisdom in drafting such a high-
impact legal document as a national constitution.
Acknowledgments. This work is part-funded by FEDER Funds, by the ERDF
(European Regional Development Fund) through the COMPETE Programme
(Operational Programme for Competitiveness) and by National Funds through the
FCT (Portuguese Foundation for Science and Technology) within the project
FCOMP-01-0124-FEDER-028980 (PTDC/EEI-SII/1386/2012). The work of Nuno
Luz is supported by the doctoral grant SFRH/BD/70302/2010. The work of Marta
Poblet draws from previous research within the framework of the project
“Crowdsourcing: instrumentos semánticos para el desarrollo de la participación y la
mediación online” (DER 2012- 39492 -C02 -01) by the Spanish Ministry of Economy
and Competitiveness.
1. United Nations Rule of Law (2009)- Guidance Note of the Secretary-General:
United Nations Assistance to Constitution-making Processes.
2. Klein C, Sajo A (2012) Constitution-Making: Process and substance. The Oxford
Handbook of Comparative Constitutional Law 419.
3. Elster J (2012) The optimal design of a constituent assembly. Collective Wisdom:
Principles and Mechanisms 148172.
4. Howe J (2006) The rise of crowdsourcing. Wired magazine 14:14.
5. Luz N, Silva N, Novais P (2014) A survey of task-oriented crowdsourcing.
Artificial Intelligence Review 127.
6. Von Ahn L (2009) Human Computation. 46th ACM IEEE Design Automation
Conference. pp 418419
7. Chklovski T (2003) Learner: A System for Acquiring Commonsense Knowledge
by Analogy. Proceedings of the 2nd ACM International Conference on
Knowledge Capture. Sanibel Island, FL, USA, pp 412
8. Singh P, Lin T, Mueller ET, et al. (2002) Open Mind Common Sense: Knowledge
Acquisition from the General Public. On the Move to Meaningful Internet
Systems 2002: CoopIS, DOA, and ODBASE. Springer, pp 12231237
9. Orozco D (2014) Democratizing the Law: Legal Crowdsourcing (Lawsourcing) as
a Means to Achieve Legal, Regulatory and Policy Objectives. Regulatory and
Policy Objectives (November 7, 2014)
10. Poblet M (2013) Visualizing the law: crisis mapping as an open tool for legal
practice. Journal of Open Access to Law 1:
11. Aitamurto T (2012) Crowdsourcing for Democracy: New Era In PolicyMaking.
Committee for the Future, Parliament of Finland 1:2012.
12. Casanovas P (2012) Legal crowdsourcing and relational law: What the semantic
web can do for legal education. Journal of the Australasian Law Teachers
Association 5:159176.
13. Landemore H (2014) Inclusive Constitution-Making: The Icelandic Experiment.
Journal of Political Philosophy
14. Gluck J, Ballou B (2014) New Technologies in Constitution Making.
15. Deely S, Nesh-Nash T (2014) The Future of Democratic Participation: An Online
Constitution Making Platform. pp 4362
16. Obrst L, Liu H, Wray R (2003) Ontologies for Corporate Web Applications. AI
Magazine 24:49.
17. Hornung T, Koschmider A, Mendling J (2006) Integration of heterogeneous BPM
Schemas: The Case of XPDL and BPEL. CAiSE Forum 231:
18. Luz N, Pereira C, Silva N, et al. (2014) An Ontology for Human-Machine
Computation Workflow Specification. Lecture Notes in Artificial Intelligence
19. Constitute Project.
20. (2010) Constitucio de Catalunya.
21. Davis-Stober CP, Budescu DV, Dana J, Broomell SB (2014) When is a crowd
wise? Decision 1:79.
22. Levine SS, Prietula MJ (2013) The Hazards of Interaction: Why Isolation Can
Benefit Performance. In Academy of Management Proceedings 2013:10736.
23. Ober J (2013) Democracy’s Wisdom: An Aristotelian Middle Way for Collective
Judgment. American Political Science Review 107:104122.
... It also reminds of the Moroccan constitutional reform of 2011 that engaged more than 200,000 Facebook and Twitter users (although in this case the process was not led by a government or a parliament, but by grass-root activists who had launched the platform to collect popular input on the process). These earlier examples sparked a wave of crowdsourced constitution-making processes across the world (Gluck and Ballou 2014; Deely and Nesh-Nash 2014; Luz et al. 2015) with varied levels of engagement and success. ...
Full-text available
Available open access at This book shows the factors linking information flows, social intelligence, rights management and modelling with epistemic democracy, offering licensed linked data along with information about the rights involved. This model of democracy for the web of data brings new challenges for the social organisation of knowledge, collective innovation, and the coordination of actions. Licensed linked data, licensed linguistic linked data, right expression languages, semantic web regulatory models, electronic institutions, artificial socio-cognitive systems are examples of regulatory and institutional design (regulations by design).
... It also reminds of the Moroccan constitutional reform of 2011 that engaged more than 200,000 Facebook and Twitter users (although in this case the process was not led by a government or a parliament, but by grass-root activists who had launched the platform to collect popular input on the process). These earlier examples sparked a wave of crowdsourced constitution-making processes across the world (Gluck and Ballou 2014; Deely and Nesh-Nash 2014; Luz et al. 2015) with varied levels of engagement and success. ...
Full-text available
This chapter presents Linked Data, a new form of distributed data on the web which is especially suitable to be manipulated by machines and to share knowledge. By adopting the linked data publication paradigm, anybody can publish data on the web, relate it to data resources published by others and run artificial intelligence algorithms in a smooth manner. Open linked data resources may democratize the future access to knowledge by the mass of internet users, either directly or mediated through algorithms. Governments have enthusiastically adopted these ideas, which is in harmony with the broader open data movement.
... A growing literature based on case studies and empirical testing provides the basis for further refinement of methods: e.g. smart crowdsourcing (Noveck 2015), expert crowdsourcing (Kim et al. 2014, Griffith et al. 2017), microtasking (Luz et al. 2015). The exploration of new technologies and methods to harness the potential of crowdsourcing for civic action and politics, nevertheless, contrasts with the scarce attention given to the underlying assumptions about democracy, participation, equality, representation, and citizenship. ...
Full-text available
This paper aims at connecting democratic theory with civic technologies in order to highlight the links between some theoretical tensions and trilemmas and design trade-offs. First, it reviews some tensions and trilemmas raised by political philosophers and democratic theorists. Second, it considers both the role and the limitations of civic technologies in mitigating these tensions and trilemmas. Third, it proposes to adopt a meso-level approach, in between the macro-level of democratic theories and the micro-level of tools, to situate the interplay between people, digital technologies, and data.
... Open government and data initiatives by states, municipalities, and parliaments have encouraged designers to build digital experiences that encourage public engagement with policy and law. These crowd-civic systems blend digital technology, design, and public data (e.g., government budgets, town council meeting minutes, police reports) to educate the public [4,1], to inform decision-makers about public interests [2,3] and to help crowds of people craft policy proposals that affect change in their governments [7,5,6]. Crowdsourcing may also function as a powerful tool of civic engagement by promoting the generation and discovery of relevant local information about issues of public concern, by fostering consensus and buy-in amongst members of the community, and by providing mechanisms and tools for measuring and assessing policy impacts. ...
Conference Paper
Crowdsourcing technologies, strategies and methods offer new opportunities for bridging existing gaps among law, policymaking, and the lived experience of citizens. In recent years, a number of initiatives across the world have applied crowdsourcing to contexts including constitutional reform, drafting federal bills, and generating local policies. However, crowd-civic systems also come with challenges and risks such as socio-technical barriers, marginalization of specific groups, silencing of interests, etc. Using a design-thinking approach, this workshop will address both opportunities and challenges of crowd-civic systems to develop best practices for increasing public engagement with law and policy. The workshop organizers will suggest an initial framework explicitly intended to be criticized by participants and reconfigured through a series of iterative cooperative small-group activities focusing on ``diagnosing'' the failures of past crowd-civic system efforts and the successes of online action around social issues. While the ultimate objective of the workshop is to develop a best practices guide, we see iterations on the guide as a mechanism for fostering community and collaboration among policymakers, technologists, and researchers around crowd-civic systems for law and policy.
... Finally, Bcrowd as microtaskers^include people performing micro-tasked activities. Microtasking can be considered a special sub-type of human computation where tasks involving different degrees of complexity are divided into smaller and independent micro-tasks (Luz et al. 2015). Microtaskers can then generate structured, high-quality, and interpreted data by performing some specific tasks over raw data (e.g. ...
Full-text available
Mobile technologies, web-based platforms, and social media have transformed the landscape of disaster management by enabling a new generation of digital networks to produce, process, and analyse georeferenced data in real time. This unprecedented convergence of geomobile technologies and crowdsourcing methods is opening up multiple forms to participate in disaster management tasks. Based on empirical research, this paper first proposes a conceptualisation of crowdsourcing roles and then analyses methods and tools based on a combination of two variables: (i) types of data being processed; (ii) involvement of the crowds. The paper also surveys a number of existing platforms and mobile apps leveraging crowdsourcing in disaster and emergency management with the aim to contribute to the discussion on the advantages and limits of using crowdsourcing methods and tools in these areas. Available at
After almost one decade of active research into human computation and crowdsourcing, several approaches and business models based on crowdsourcing have emerged, managing and distributing work to the crowd. Dispute resolution approaches may incorporate crowdsourcing as a step to retrieve relevant data. The reverse relationship has also become a tendency, where crowdsourcing approaches are close to incorporate dispute resolution techniques to perform quality control and data aggregation or filtering. This chapter provides an introduction to crowdsourcing and its relationship with dispute resolution. A discussion regarding the apparent symbiotic relationship between these two research domains is also presented, along with an overview of several approaches and use cases of particular interest.
Although confidence in democracy to tackle societal problems is falling, new civic participation tools are appearing supported by modern ICT technologies. These tools implicitly assume different views on democracy and citizenship which have not been fully analysed, but their main fault is their isolated operation in non-communicated silos. We can conceive public knowledge, like in Karl Popper’s World 3, as distributed and connected in different layers and by different connectors, much as it happens with the information in the web or the data in the linked data cloud. The interaction between people, technology and data is still to be defined before alternative institutions are founded, but the so called linked democracy should rest on different layers of interaction: linked data, linked platforms and linked ecosystems; a robust connectivity between democratic institutions is fundamental in order to enhance the way knowledge circulates and collective decisions are made.
Full-text available
After almost one decade of active research into human computation and crowdsourcing, several approaches and business models based on crowdsourcing have emerged, managing and distributing work to the crowd. Dispute resolution approaches may incorporate crowdsourcing as a step to retrieve relevant data. The reverse relationship has also become a tendency, where crowdsourcing approaches are close to incorporate dispute resolution techniques to perform quality control and data aggregation or filtering. This chapter provides an introduction to crowdsourcing and its relationship with dispute resolution. A discussion regarding the apparent symbiotic relationship between these two research domains is also presented, along with an overview of several approaches and use cases of particular interest.
O trabalho explora perspectivas acerca da influência da interação humana nas redes sociais sobre a construção da ordem constitucional, seja em relação à produção de textos normativos, seja em relação à interpretação das disposições constitucionais vigentes. O objetivo da pesquisa é aferir a influência do uso das redes sociais sobre o poder constituinte e a hermenêutica constitucional, com o emprego de pesquisa bibliográfica, nacional e estrangeira, com o aporte de produções teóricas extrajurídicas. As conclusões do estudo sinalizam expressivo potencial participativo popular, em ordem a incrementar o suporte democrático da ordem constitucional.
This chapter reports how participatory processes and ICT tools can go against rule-driven bureaucratic approaches to political participation and public deliberation, trying to defy strict procedural norms in favor of more flexible formats for citizen mobilization, political co-thinking, and sustained social innovation in the area of constitutional building. After describing key theoretical issues on trends and perspectives of public participation in constitution building processes, we review arguments in favor of ICT and social media use in constitutional building and then discuss an ongoing Greek bottom-up experiment named Syntagma 2.0 that introduced an innovative co-creative process for the production of a new Constitution for Greece, to be delivered by the citizens for the citizens. Based on the results of the aforementioned project so far, we present success factors for such initiatives.
This article defines the emergent practice of lawsourcing, which is an extension of the increasingly popular crowdsourcing model. As defined in this article, lawsourcing occurs when a party releases an open call to online participants that requests their support to achieve a legal objective. This online call may involve private or governmental participants that leverage a crowd to achieve a legal objective. Various lawsourcing techniques fall under this umbrella and are described in this article. These practices fall under the following three broad categories: legal Q&A platforms, government participation forums, and strategic nonmarket practices. As discussed in the article, lawsourcing leverages the positive economic and social aspects of crowdsourcing. These qualities provide the foundation for lawsourcing to be a disruptive force in the current legal environment. The positive attributes of lawsourcing also indicate that it can be a powerful instrument to achieve legal reform, greater transparency with respect to the quality of legal services and improved access to the legal system.
This article considers the procedural and resulting legitimacy issues of constitution-making and fundamental constitutional amendment. These procedures are partly related to the different historical scenarios and substantive (material) factors that give rise to e-constitutions. It considers only those political and economic factors which contribute to specific constitution-making features. In this regard, revolutions, regime change, and state-building are particularly relevant. In the case of revolutions there is a deliberate departure from, a rupture with, the existing constitution and the processes of legal and therefore legitimate change. This raises a fundamental issue of legitimacy: What gives the right (authority) to enact a new constitution? The article refers to the process that is not based on pre-existing rules of procedure as one of creation and the related constitution-making is called creation ex nihilo. In the case of regime change or reform the procedural modalities of the existing constitution might be observed.
Open Mind Common Sense is a knowledge acquisition system designed to acquire commonsense knowledge from the general public over the web. We describe and evaluate our first fielded system, which enabled the construction of a 450,000 assertion commonsense knowledge base. We then discuss how our second-generation system addresses weaknesses discovered in the first. The new system acquires facts, descriptions, and stories by allowing participants to construct and fill in natural language templates. It employs word-sense disambiguation and methods of clarifying entered knowledge, analogical inference to provide feedback, and allows participants to validate knowledge and in turn each other.
During the Arab Spring revolutions protestors used mobile commu-nications technology and social media platforms to share information, mobilize supporters, and organize activities to bring about the political transformation of their countries. In each case the drafting of a new constitution was the next step adopted to continue that transformation. We ask whether the digital revolution that powered the overthrow of old regimes during the Arab Spring can also be used to facilitate a similar level of participation in the constitution making pro-cess and we present "my.con", an online platform allowing citizens to collabo-rate in constitutional drafting.
If there is one task for which “wisdom” would seem highly desirable, it is that of writing a constitution that is intended to last for the indefinite future. In fact, wisdom is not only desirable in light of the importance of the issues, but also necessary in light of their complexity. Writing to his son soon after the opening of the Federal Convention in Philadelphia, George Mason confessed that “to view through the calm, sedate medium of reason the influence which the establishment now proposed may have upon the happiness or misery of millions yet unborn, is an object of such magnitude, as absorbs, and in a manner suspends, the operations of the human understanding” (in Farrand 1966: vol. 3, 33). A good constitution is a complex piece of machinery, a set of interlocking parts that are finely adjusted to each other. A priori, one might think that the task of writing it is best entrusted to a single individual who can weigh all the relevant considerations without having to accept the compromises that are inevitable in any collective decision-making process. In stylized form, whereas both [A, B] and [A′, B′] might be viable combinations, the committee compromise [(A + A′)/2, (B + B′)/2] might not be. The first reasonably well documented constitution, that of Solon, was in fact the accomplishment of a single individual. Although details are shrouded in obscurity, it seems to have been remarkably successful. Other examples of constitutions essentially handed down by a single person include the French Charter of 1815, the “octroyed” Prussian constitution of 1848, and the constitution of the Fifth French Republic. These can hardly, however, be said to have been uncontroversially successful.
Interaction is central to organizational theory and strategic management, but scholars have differing views on the impact of interaction on performance. For some, interaction is essential for distilling good practices and spreading them. When people interact, good practices prevail and bad practices fade away, they assume. This assumption underlies practices in knowledge transfer, innovation, and education, such as best practices sessions and learning from peers. But social psychologists and institutional sociologists, among others, have shown that interaction can also disseminate useless or even harmful practices, thereby harming performance. To address the differing views, here we model how interaction affects people’s skills and ultimately – their performance. In the model, one’s performance depends on behavioral skills, and interactions spread skills and practices, whether valid or fallacious. We utilize recent findings in psychology to incorporate assumptions on how people assess their skills and how willing people are to learn and revise their skills. Building on these empirically grounded assumptions, we find that bad practices spread disproportionally faster than good ones. When people interact, those with good practices do not always prevail. Rather, interaction merely causes practices to become similar: some poor performers improve but good ones deteriorate. Therefore, isolation can benefit performance. When we split the population into teams, people interact less and performance variance is preserved. As they become more isolated, people can reach high performance. This may explain why some high-performance people are hermits, and why teams are often in-group biased.
Conference Paper
With the growing popularity of micro-task crowdsourcing platforms, a renewed interest in the resolution of complex tasks that require the cooperation of human and machine participants has emerged. This interest has led to workflow approaches that present new challenges at different dimensions of the human-machine computation process, namely in micro-task specification and human-computer interaction due to the unstructured nature of micro-tasks in terms of domain representation. In this sense, a semi-automatic generation environment for human-computer micro-task workflows from domain ontologies is proposed. The structure and semantics of the domain ontology provides a common ground for understanding and enhances human-computer cooperation.
Since the advent of artificial intelligence, researchers have been trying to create machines that emulate human behaviour. Back in the 1960s however, Licklider (IRE Trans Hum Factors Electron 4–11, 1960) believed that machines and computers were just part of a scale in which computers were on one side and humans on the other (human computation). After almost a decade of active research into human computation and crowdsourcing, this paper presents a survey of crowdsourcing human computation systems, with the focus being on solving micro-tasks and complex tasks. An analysis of the current state of the art is performed from a technical standpoint, which includes a systematized description of the terminologies used by crowdsourcing platforms and the relationships between each term. Furthermore, the similarities between task-oriented crowdsourcing platforms are described and presented in a process diagram according to a proposed classification. Using this analysis as a stepping stone, this paper concludes with a discussion of challenges and possible future research directions.