E-Voting System evaluation based on the Council of Europe recommendations: nVotes

Abstract

E-voting implantation has been facing important challenges in recent years. Several incidents, together with a lack of evaluation methodologies social and cultural customs hinder a broader application. In this work, the authors aim to contribute to a safer introduction of e-voting tools by applying a practical evaluation framework strongly based on the security requirements issued by the Council of Europe in 2017 to nvotes, a system that has been utilized to cast over 2 million votes over the last 6 years. The ultimate goal of the analysis is not to judge from a rigid, "infallible" but to contribute to a gradual and secure implementation of e-voting solutions in the democratic processes. The authors believe it can constitute a useful source of information for election officials, researchers and voters.

Full text

E-Voting System evaluation based on the Council of
Europe recommendations: nVotes
David Yeregui Marcos del Blanco1[0000-0001-7702-6602] David Duenas-Cid2,3 [0000-0002-0451-
4514] and Héctor Aláiz Moretón[0000-0001-6572-1261]
1 University of Leon, Campus de Vegazana, s/n, 24071 León
dmarcb01@estudiantes.unileon.es
hector.moreton@unileon.es
2 Tallinn University of Technology, Akadeemia tee 3, 12618 Tallinn, Estonia
3 Kozminski University, Jagiellonska 57/59, 03-301 Warsaw, Poland
david.duenas@taltech.ee / dduenas@kozminski.edu.pl
Abstract. E-voting implantation has been facing important challenges in recent
years. Several incidents, together with a lack of evaluation methodologies social
and cultural customs hinder a broader application. In this work, the authors aim
to contribute to a safer introduction of e-voting tools by applying a practical eval-
uation framework strongly based on the security requirements issued by the
Council of Europe in 2017 to nvotes, a system that has been utilized to cast over
2 million votes over the last 6 years.
The ultimate goal of the analysis is not to judge from a rigid, “infallible” but to
contribute to a gradual and secure implementation of e-voting solutions in the
democratic processes. The authors believe it can constitute a useful source of
information for election officials, researchers and voters.
Keywords: e-democracy, e-voting, system evaluation, nvotes.
1 Introduction
Since the first implementation of remote electronic voting in the 90s [4], the process of
dissemination of internet voting did not meet the initial and promised expectations.
Several countries experimented with the possibility of adding internet voting systems
to their elections1, but it just turned into a reality in a reduced number of them: Estonia,
Canada, Australia, Switzerland or Norway, amongst others. The Estonian case is the
most prominent success story, using Internet Voting uninterruptedly since 2005 in all
elections [1] an reaching high levels of acceptation [2] and cost efficiency [3, 4].
The dissemination of internet voting technologies is challenged by a complex set of
factors that affect different layers of administration, law, society and technology [5]
and that should be achieved in a constant dialogue between themselves: dealing with
1 For a better understanding, see International IDEA’s database on use of ICT in Elections:
https://www.idea.int/data-tools/data/icts-elections (last accessed 4 June 2020)

2
complexity in electoral management, reforming electoral laws, ensuring transparency,
neutrality and participation and ensuring secure and risk-free technological apparatus.
The latter factor, the security of the system, has been constantly labelled as an important
element not only for the correct functioning of the internet voting and its integration in
the electoral systems, but also as an element projecting trust in the society where the
system is being implemented [6,7,8].
Pursuing the same goal, the creation of trust as a key element for the adoption of
internet voting systems, the Council of Europe proposes a set of recommendations to
guide the process of implementation of electronic remote voting systems [9]. The
CM/Rec(2017)5 updates the previous Recommendations from 2004 and integrates les-
sons learned from previous experiences and developments in the electoral field to create
a useful and up-to-date document. Specifically, proposes a set of Principles, Standards
and Requirements that every electronic voting system should fulfil for the development
of elections and for reinforcing the democratic principles that are the common heritage
of its member states [10]: Elections should be Universal, Equal, Free and Secret, should
meet a set of regulatory and organizational requirements, should be transparent and
allow observation and should be accountable, and should use reliable and secure sys-
tems.
On the light of the aforementioned list, this paper presents an analysis on how the
system nVotes fits on the Council of Europe requirements. The ultimate goal of the
authors is not to judge from a rigid immovable or infallible point of view for the sake
of pin pointing shortcomings, but to establish a comprehensive, multi-faceted evalua-
tion in order to improve the knowledge and security level in the deployment of e-voting
systems
2 Related Works
The research work of Bräunlich, Grimm and Richter in 2013 [111] is considered one
of the most relevant to date. The authors presented the first interdisciplinary collabo-
ration which has transformed legal requirements into technical criteria. Specifically,
they established thirty Technical Design Goals (TDG), using the KORA methodology
(Konkretis-ierung Rechtlicher Inforderungen, Concretization of Legal Requirements)
[12]. This methodology had been used previously for mobile devices amongst others.
Neumann combined the previous methodology of Bräunlich, Grimm and Richter
with the Common Criteria for IT-Security Evaluation [13] and established sixteen tech-
nical requirements to relate the legal criteria to Bräunlich’s TDGs.
While Neumann’s work [14] has critically contributed to constructing a very valua-
ble framework, it still had room for improvement from a practical standpoint:
On the one hand, the security evaluation framework is aimed at schemes rather than
entire systems, with the author himself coming across an example of a structural flaw
that would not be identified using his evaluation scheme: “for instance, the Vote For-
warding Server and the Vote Storage Server of the Estonian Internet voting scheme are
developed and maintained by the same vendor” [14, p. 135].

3
Additionally, the security evaluation assumes that the voters will use the authentica-
tion tools sufficiently. Unfortunately, the tendency of the voters is not to verify: for
instance, one of the largest electoral e-voting initiatives which took place in New South
Wales in 2015, showed that only 1.7% of 283.669 votes were verified [15].
Furthermore, Neumann’s framework is based on probabilistic attack strategies (an
attacker may be able to make a certain impact or not), through Monte-Carlo simulations
[14]. It is an interesting approach indeed, although less useful for a practical evaluation
of the e-voting tool. As a result, the author concludes: “we therefore recommend to
incorporate the security evaluation framework into a larger decision-support system for
elections officials” [14, p. 138].
Following with the above recommendation, a decision-support system proposal was
developed by Marcos, et al. to present a decision-support system as a practical evalua-
tion framework [16]. It is in accordance with the guidelines from the 2017 Council of
Europe’s (“Guidelines on the implementation of the provisions of Recommendation
CM/Rec(2017)5 on standards for e-voting”), drawn up by the Directorate General of
Democracy and Political Affairs [17] and deals with the five key principles of a demo-
cratic election (universal, free, equal, direct and secret) detailed in the same document.
3 Evaluation Methodology
As previously stated, Bräunlich, Grimm and Richter greatly contributed to the evalua-
tion of e-voting systems against legal provisions by setting out the first draft of technical
design proposals (TDGs) by refining the election principles.
Consequently, Neumann [14] identified the imperfections of Bräunlich’s approach
and set out a series of technical requirements to bracket the legal criteria with the TDGs.
While Neumann’s work set out an irrefutable improvement, it constitutes a scheme
evaluation tool with probabilistic proofs as its core with Monte-Carlo simulations rather
than a practical evaluation framework tool for election officials and other stakeholders
involved in the democratic processes.
In 2018, Panizo et al. proposed an extended evaluation approach in the context of
the Spanish Constitution [18] and the Council of Europe’s latest e-voting recommen-
dations [17]. Their proposed methodology [19] involved the following steps:
1. Defining an homogeneous series of e-voting requirements with: the KORA method-
ology [12] as its basis, together with the CC and ISO 27001-IT Grundschutz guide-
line [13], their assimilation by Simic-Draws et al. [20], the Guidelines of the Council
of Europe [17] and Neumann’s methodology [14].
2. Formal conformity between point 1 and Bräunlich’s TDG’s [11], as in Figure 1.
3. Consultation with more than 30 international experts in e-voting (Research and In-
dustry Experts or RIE, selected using the snowball [21] and judgement [22] sampling
methodologies) to review the evaluation framework and add weighting factors.
4. Formal definition of the practical evaluation framework, including two sine-qua-non
requirements (E2Ev and Coercion Resistance) and 41 evaluation items.

4

Fig. 1. Integration of Panizo [19] and Bräunlich [11]
The work in [16] established for the first time a correlation between the end to end
verifiability (E2Ev) and coercion resistance (CR) to the legal requirements for a dem-
ocratic process and the Council of Europe:“The five key principles of electoral law are:
universal, equal, free, direct and secret suffrage and they are at the root of democracy”
(article 68 of the Spanish Constitution [18]).
Specifically, Marcos et al. Set out the equivalence of the aforementioned five key
principles, into a formal authentication of the E2Ev the universal, free, equal and direct
properties and its coercion resistance for the secrecy prerequisite.
The methodology presented to this point is solid from a legal point of view but still
lacks the technical and practical approach necessary for a complete evaluation.
In order to solve the shortcomings, five practical requisites were introduced, partially
based on the research by Benaloh, Rivest, Ryan and Volkamer [23], [24]. Subse-
quently, the requisites were codified, refined and subdivided into 73 specific items by
means of a partial application of Zissis and Lekkas [25] and New Zealand’s Department
of Internal Affair’s Communication on e-voting [26] 2.
As a final step, e-voting RIEs from Canada, France, Norway, Switzerland, Germany
and Spain among other countries were consulted again, with the purpose to assign a
weighting score to each requisite.
The following Figure 2 visually represents the complete evaluation methodology:
2
For a complete explanation of the previous process, please refer to the original work in [6], [8].

5
Fig. 2. Complete evaluation framework [16]
The sine-qua-non requirements (end-to-end verifiability and coercion resistance,
representing the five compulsory principles of a democratic election), which evaluation
is not a numerical value related to performance but instead in terms of “holds” (○) or
“does not hold” (⨯). There is a third possibility, when the property “stands under de-
termined, credible assumptions” (∆).
The second quantifiable and additional criteria, totaling 10 requirements, are evalu-
ated from 0 to 10. In order to obtain the numerical evaluation for each criterion, the
corresponding measurable sub-items are evaluated with three possible outcomes: non-
compliant (⨯), partially compliant (∆) and compliant (○).
The explanation of the evaluation framework design, implementation and constitu-
ent requirements and items has been simplified because of page extension constraints.
For a full version, the reader can refer to the original PhD thesis [27].
It is relevant to mention that this practical evaluation methodology has previously
been applied to Helios Voting and published by the IEEE [19].
4 nVotes Analysis
4.1 Introduction
nVotes [28] is a remote e-voting system developed by the Spanish company Agora
Voting SL in 2014. Its roots trace back to 2009 and the Internet Party, although the
developing team has since then dropped any political affiliation and nVotes is currently
an apolitical project. The key members are Lucas Cervera as CEO, Eduardo Robles as
CTO and David Ruescas as Head of R&D.
Until 2017, nVotes was known as Agora Voting and under such moniker it was one
of the 18 European start-ups to be accepted in the Impact Accelerator project, and
awarded with 100,000 EUR [29].
According to their website, nVotes has been used to cast over 2 million votes for
over 150 clients, including Public Administrations like the Barcelona Provincial Coun-
cil, Madrid City Council; Political Parties like Podemos, Ahora Madrid and Barcelona
en Comú, as well as Education Institutions like UNED University in Spain in processes
up to 150,000 voters.
4.2 Main characteristics
As previously mentioned, the methodology presented in Section 3 has been already
applied to other relevant e-voting tools, including Helios Voting [19] or iVote by Scytl
[30]; in both cases with numerous bibliography and research resources available:
 Helios Voting is a very well-known open source e-voting system [31], which has
been used as blueprint for several variations and improvements such as Helios KTV
[32] or Belenios [33].

6
 Scytl is probably the most widely used e-voting system at a global level, including
numerous legally-binding elections and pilots for a total of over 100,000 processes
managed and more than 200 employees. The information available ranges from re-
search papers to Government reports and corporate presentations.
In the case of of nVotes, the available bibliography is much more limited due to the
fact that they are neither a research standard tool, nor a global company. In order to
complement the publicly available information, the authors of this document got in
touch with nVotes and they key people have always been open and supporting in
providing all the available information and answers to the questions raised.
Additionally, the authors were provided with two documents named “Technical
Overview” and “Client Action Protocol”, which have been extremely useful for con-
ducting the analysis. They are at the reader’s disposal upon request to the authors since
they have not been published before.
nVotes scheme components and cryptographic primitives. According to the infor-
mation included in the “Technical Overview” and complemented with a Q/A with
nVotes technical team, the key elements are:
─ Registry: Comprised basically of the registration database programmed in Python. It
includes the SMS service platform Esendex [34], server certificate with TSL support,
Cloudfare [35] and Fail2ban [36] for protection against DDoS attacks and hardware
redundancy 1+1.
─ Virtual Polling Station: TLS server validation, cast-or-audit voting javascript (sim-
ilar to that of Helios Voting [31]), random number generator (not specified), HMAC
client authentication, Election Manager with Scala REST API, Postgresql database
and similar to the Registry case, Cloudfare and Fail2ban DDoS protection.
─ Electoral Authority: HTTP distributed queue, TLS client/server authentication,
mixnet library Verificatum [37] and tabulation library OpenSTV [38].
─ Election Verificator: a Python/Java
With regards to the main cryptographic primitives, they are the following:
─ El Gamal Homomorphic Encryption [39]
─ Pedersen Threshold Distributed Key Generator [40]
─ Verificatum verifiable mixnet [37]
─ Fiat-Shamir heuristic to convert Zero Knowledge Proofs into Non-Iteractive Zero
Knowledge Proofs [41]
─ Schnorr Signature [42] to make the ElGamal Encryption IND-CCA2.
nVotes voting sequence. As presented in the “Technical Review” and “Client Action
Protocol” documents, the voting procedure is as follows:
1. Authorities distributedly generate the Election’s Public Key with Pedersen [40].

7
2. Eve (voter) access the Registry site and provides the required personal information,
including a security code which has been sent independently by SMS
3. The Registry system compares the information provided with the census. If it is cor-
rect, Eve is forwarded to the Virtual Polling Station.
4. Eve fills her vote, encrypts it and sends it. Alternatively, she can audit it but in such
case, the cast vote is no longer valid and will not be tallied. This cast-or-audit ap-
proach is also implemented in Helios Voting [31].
5. Once the vote casting period ends, the authorities jointly proceed with the mix and
decryption of the ballots
6. The decrypted votes are tallied
7. The election results are published, together with the tally results, the vote’s cipher-
texts as well as the mixnet and decryption Zero Knowledge Proofs.
8. Voters and third parties can download and execute the election verificator
Once nVotes has been introduced, together with its associated scheme components,
cryptographic primitives and voting process, the practical evaluation methodology for
e-voting systems [16] can be applied.
The analysis is intended to be a sort of a guideline, which introduces strengths and
potential weaknesses in order to establish a safe range if utilization and to offer direc-
tions as to how to improve the voting system.
4.3 End to End Verifiability
Unfortunately, there is no formal, universal definition for end-to-end verifiability
(E2Ev). Additionally, symbolic analysis of security protocols still find associative and
commutative operators are out of reach. It is then not possible to analyze a homomor-
phic property [43] such as:
enc(pk; v1 )*enc(pk; v2 )=enc(pk; v1+v2) (1)
and therefore, a case by case analysis has to be conducted for each system.
Currently, probably the most widely accepted definition of E2Ev is the one by
Benaloh et al. in [23] and is comprised of the properties: “Cast as intended”, “Recorded
as cast” and “Tallied as recorded”
For the first and second items, nVotes presents a similar approach to that of Helios
Voting: the voter can audit her vote until she is convinced that it is trustable and once
cast, she receives a hash of the encrypted, which she can check on public bulletin board.
Finally, for the tallied as recorded condition, ElGamal (also present in Helios Voting)
together with Verificatum mixnet [37] and Schnorr [42] are implemented.
So regarding the question of nVotes being E2Ev or not and similar to the analysis in
[18] for Helios Voting giving the similarities, it can be considered end to end verifiable
assuming certain preconditions:
─ The cast and audit mechanism is used by a large enough number of voters so that
ballot alteration will not go unnoticed.
─ The Election Authorities and the Bulletin Board (BB) are honest
─ An attack which gains control of the Registry/Ballot is detected.

8
For the first precondition, Acemyan in [44] and the New South Wales case [15]
have shown that voters’ ballot verification percentage is quite low and they should not
be responsible of part of the security of an e-voting system.
In relation to the other two prerequisites, in a perfect scenario nVotes would comply
with both of them but in real elections, both the Election Authorities and/or the BB can
illegally introduce votes in what is known as “ballot stuffing”. For public, legally bind-
ing elections, that is something no to be taken lightly.
To sum up, if nVotes use is limited to elections with a low risk of corruption such
as student government bodies, local clubs, online groups, and other education-related
organizations, the pre-assumptions could be acceptable. For other, more demanding
types of elections, E2Ev cannot be asserted.
Evaluation: ∆. E2Ev holds if the preconditions set in nVotes’ Technical Overview
document are accepted and its use is limited to low corruption risk elections.
4.4 Coercion Resistance
Assuming probably the most accepted definition of privacy levels by Juels et al. [45]
and the proof by Hirt and Sako [46] that receipt-freeness is not enough for preserving
it in electronic elections, the level to reach is Coercion Resistance. It implies that a voter
cannot provide to an attacker any proof of her vote or even whether she voted or nor,
even if she is willing to cooperate.
As for nVotes, the voter receives a verification code after casting the ballot, therefore
she can prove it to a potential attacker.
Additionally, the Election Administrator of an Election can verify whether an spe-
cific person in the census has voted or not, which clearly compromises the privacy.
Evaluation: X. Does not hold.
4.5 Inviolability (I-n)
nVotes’ Technical Overview document includes an integrity, privacy and availability
analysis. The authors acknowledge the possibility of “ballot stuffing” if the Election
Administrators are corrupt and of DDoS attacks despite implementing specific tools
[35], [36].
There have also been questions raised about the census integrity used in consultative
referenda [47, 48] and the separation between the tally administrator and the census
administrator, which can be the same person and thus lead to potential collusions (I-4).
Safe authentication protocols, tracking tools, Risk Assessment and modularity prin-
ciples are partially compliant, with room for improvement.
Table 1. Inviolability in nVotes
I-n Definition Val

9
Evaluation: 4/10 points. The inviolability policy presents certain vulnerabilities
which, in the case of a private election, while being very serious, are ultimately up to
the organizer’s criteria whether to take the risk or not. For legally binding public elec-
tions, they are not acceptable and nVotes inviolability should be improved before being
used in such environment.
4.6 Usability (U-n)
nVotes presents a satisfactory performance in terms of simplicity and clarity in the
voting process (U-1, U-3) as well as in intuitiveness and lexicon choice both for the
voter and the administrators.
Concerning the aspects to be improved, there is no version adapted to collectives
with special needs, the SMS authentication might prove challenging for the elders and
the verification codes are too long and “imposing” voters with no technical background.
An intermediate usability layer might be advisable.
Overall, usability is satisfactory while it could be enhance with some simple, easy
to implement changes.
Table 2. Usability in nVotes
Evaluation: 6/10 points
4.7 Monitoring/Auditing (MA-n)
This aspect is especially relevant for nVotes due to the possibility of Ballot Stuff-
ing attacks through Registry and Eligibility if the Administrators are corrupt or col-
lide or DDoS hostile actions.
Probably due to the nature and scope of the elections managed, the Monitoring
and Auditing Protocol is based on Administrators training. According to the key team
I-1 Software and auxiliary system’s protection w/ safe authentication
protocols. Access via third-parties/vulnerable-servers not permitted. ∆
I-2 Action protocols in the event of compromised inviolability. X
I-3 Tracking tools and offline backup copies available. ∆
I-4 Distributed control in the critical nodes with division of responsibil-
ities to minimize collusion risks. X
I-5 Existence of Risk Assessment and Threat Modelling protocols. ∆
I-6 Modularity principles to confine potential attacks and coding bugs. ∆
I-7 Proper updating of items I-1…I-6 ∆
U-n Definition Val
U-1 Simplicity in the authentication, voting and verification O
U-2 Special attention to vulnerable groups pursuant to the Council of Eu-
rope and the United Nations’ resolutions on the matter. X
U-3 Transparency & clarity communicating the voter that the voting pro-
cess has successfully ended/vote has been received. O
U-4 Privacy and integrity preference over usability in a compromise. X
U-5 Intuitive/user-friendly admin interface for setup and management. O

10
members of nVotes, they are currently developing a unified protocol including all the
auditing-related activities.
Until it is available, nVotes generates retrievable logs, and provides reasonable
amounts of information and data in an easily understandable format. Even so, at this
point the Monitoring/Auditing Protocol is still largely to be developed and imple-
mented; therefore not satisfactory.
Table 3. Monitoring/Auditing in nVotes
Evaluation: 3/10 points
4.8 Software Development (SWD-n)
nVotes displays an overall solid Software Development (partly because of its open
source approach), with a satisfactory performance in usual software engineering prac-
tices (SWD-1), FAQ (SWD-4), impartiality (SWD-5), ballot cast termination (SWD-
8), compatibility (SWD-9), third party access (SWD-10), and protocolized application
(SWD-13).
Regarding the distributed approach (SWD-2), it has been correctly implemented for
key generation and encryption/decryption but there is no separation between the census
and the bulletin board. If the same person is responsible for both of them, there is an
important risk of collusion.
Finally, the primitives are well implemented but some of them have been already
been proven flawed and should be reviewed (SWD-11). Additionally, more frequent
updates would be preferable (SWD-14).
Table 4. Software Development in nVotes
SWD-n Definition Val
SWD-1 Usual software engineering requirements in terms of design, implemen-
tation and documentation.
O
MA-n Definition Val
MA-1 External, independent and distributed. X
MA-2 MA protocol from the design phase, to assure a correct development
throughout the whole lifecycle of the project. X
MA-3 Specific control on Risk Assess and Thread Modelling strategies. X
MA-4 Generation of periodical, tamper-proof, indelible logs; stored offline in
premises guarded by different personnel from other critical nodes. ∆
MA-5 Implementation from census collecting to post-electoral maintenance. ∆
MA-6 Well-documented, detailed information in the appropriate format. ∆
MA-7 Existence of a test bench to verify that the system is working correctly. X
MA-8 The members of the monitoring/auditing team must be independent from
the rest of authorities/administrators involved. X
MA-9 Auditing protocol for previous attacks and the MA protocol itself. X
MA-10 In the event of a successful attack, the system will give total priority to the
vote/voter’s privacy, even calling off the elections. X

11
SWD-2 Distributed approach on critical operations. No authority should have at-
tributions to single-handedly modify critical parameters.
∆
SWD-3 User-friendly approach. User’s guide and administrator’s guide well
documented and available well in advance.
∆
SWD-4 Secure and accessible website, with a well-documented FAQ. O
SWD-5 The voting options must be presented in a totally objective and unbiased
way, showing no preference whatsoever.
O
SWD-6 System must not provide the voter with evidence to proof her vote. X
SWD-7 The system must guarantee the voter’s privacy throughout the whole vot-
ing process, not being possible to rebuild the vote/voter link.
∆
SWD-8 The voting process must offer the possibility to be terminated at any
time, not saving any information compromising the voter’s privacy.
O
SWD-9 SW to be tested in every platform, operational system and browser with
a market share ≥ 1%.
O
SWD-10 Software must neither allow for third-party access (incl. social media)
nor include links to programs/sites outside the e-voting infrastructure.
O
SWD-11 The cryptographic primitives shall be tested in advance under conditions
more demanding than the ones expected during the elections in order to
avoid breakdowns and foresee shortages.
∆
SWD-12 Access to the source code by independent experts to reinforce security.
The code developer can demand an NDA to protect its IP.
∆
SWD-13 Use of protocolized systems/open standards to improve interoperability. O
SWD-14 Update policy, against new e-voting attacks as they are discovered. X
Evaluation: 7/10 points
4.9 Scalability (S-n)
nVotes has managed elections up to 150,000 votes in consultative referenda of po-
litical parties, although they didn’t managed many of the ex_software activities, which
were handled by the Party itself.
So far, the system has proved to be scalable to the amount of votes already man-
aged in private elections. The shortcomings related to monitoring, ex-software devel-
opment and potential collusion request a further in-depth improvement before being
considered for introduction in public binding elections.
Table 5. Scalability in nVotes
Evaluation: 5.5/10 points
S-n Definition Val
S-1 Maximum capacity tests both from a SW and a HW standpoint in environ-
ments more demanding than the elections to be managed.
∆
S-2 Ad-hoc performance tests for the most critical operations (authentication,
encryption/decryption, cryptographic primitives, tallying ...).
X
S-3 Existence of test benches more demanding than the actual elections. X
S-4 Clear indicators and metrics on the max manageable size and complexity
from a SW (cryptographic capabilities, number of voters) and ex_SW (in-
frastructure, costs, logistics, second channels etc.) standpoints.
∆
S-5 Clear definition of election which can be adequately handled by the e-voting
system (from consultative referenda to politically binding elections).
∆

12
4.10 Ex-Software Development (ESWD-n)
Ex_Software development is intimately related to the increased complexity of public
binding elections. The lower the score in this category, the less recommended it is for
the analyzed e-voting system to be implemented for such type of elections.
In the case of nVotes, it has been deployed only for private elections and referenda,
and therefore has not implemented ESWD1-4, ESWD6-7, and ESWD-10.
The aspects in which the development is satisfactory are: authentication by alterna-
tive channels (ESWD-11) and the master initialization protocol (ESWD-12).
As for the communication/problem solving/back up policy (ESWD5, 6, 8, 9, 14, 15),
nVotes stated that they offer different levels of services according to the needs and
budget of each election. They can even let the client handle most of the activities related
to back-up protocols, responsibilities attributions etc.
While that could make sense from a business perspective, the security implications
in case of a misuse or a scandal, and the potential impact in the reputation of nVotes,
advice against allowing the election organizer to handle such sensitive actions.
Table 6. Ex_Software Development in nVotes
ESWD-n Definition Val
ESWD-1 Design, development & update of SWD/ESWD protocols in parallel. N/A
ESWD-2 Safe protocol for credential, permission & responsib. distribution. N/A
ESWD-3 Automated access control and infrastructure surveillance. N/A
ESWD-4 Auditing and independent observers’ protocol. X
ESWD-5 Distributed back-up protocol. ∆
ESWD-6 Distribution of attributions and responsibilities throughout the whole
ex_sw development to minimize collusion risks.
X
ESWD-7 Availability of complementary, non e-voting systems. X
ESWD-8 Voters must be informed about the e-voting process in advance,
through websites, telephone, information stands…
∆
ESWD-9 If re-voting is permitted, provide a reinforced information campaign
to explain the prevalence of paper ballot.
∆
ESWD-10 Organize opinion polls on selected cohorts to gather reliable feed-
back on usability, tendencies and improvements.
X
ESWD-11 Authentication of credential submission by alternative channels. O
ESWD-12 Master initialization protocol to be executed right before the start of
the e-voting period to verify the correct operation/readiness.
O
ESWD-13 Implementation, to the extent possible, of protocolized and standard-
ized systems to improve interoperability.
∆
ESWD-14 Free assistance phone service available before/during the election.
ESWD-15 Complete PR strategy to promote e-voting and train voters, includ-
ing: webinars, stands, demos, open days etc.
∆
Evaluation: 4/10 points
4.11 Incidents and Attacks Protocol (IAP-n)
Elections imply a power transfer from the voters to the elected representatives.

13
The bigger the election or its reach, the more attention they conceal, also from potential
attackers. Hence the critical importance of a protocol to prevent and avoid attacks and
in case they occur, to isolate and mitigate the pernicious effects.
Due to the track record of elections managed by nVotes, they do not have a proper
such protocol in place, presenting only partial compliance in distributed/modular ap-
proach and actions taken towards limiting the risk of an attack with the introduction of
Cloudfare [35] and Fail2Ban services[36].
In conclusion, nVotes needs to develop a proper Incidents and Attacks Protocol be-
fore being used for legally binding, public elections.
Table 7. Incidents and Attacks protocol in nVotes
IAP-n Definition Val
IAP-1 Risk Assessment (RA), Privacy Impact Assessment (PIAS), Penetration
Testing (PT), Control Validation Plan (CVP) and Control Validation
Audit (CVA) protocols.
∆
IAP-2 Specific prevention protocols for each cryptographic scheme. X
IAP-3 All the information shall be kept to the extent possible in the country’s
National soil.
O
IAP-4 Implementation of protocols and reinforcement operations to minimize
the risk of permanent losses of information.
∆
IAP-5 Reinforced distributed approach to contribute to the absence of critical
nodes which undermine the e-voting system’s viability.
∆
IAP-6 Training and awareness campaigns to minimize the risk of voter-driven
attacks (phishing, social engineering, etc.).
X
IAP-7 Hackers/indep. experts to test and compromise the system beforehand. X
Evaluation: 4/10 points
4.12 Versatility (V-n)
nVotes can be used by the voter with a standard internet connection, hardware and
Operative System. While it works in most of the available browsers and devices, there
is no compatibility study available.
Regarding the existence of different versions depending on the type of election
(yes/no, 1/N, N/M, order etc.) there are no adapted versions but according to the data
in Verificatum [37], its performance is satisfactory enough to not require adapted ver-
sions. The authors believe that such statement is only partially true and largely depends
on the range of the election.
Finally, the score against the WCAG 2.0 standard was good but not brilliant (A).
Table 8. Versatility in nVotes
V-n Definition Val
V-1 Versions adapted to different election typologies (yes/no, 1/N...). ∆
V-2 Specific solutions for vulnerable groups (disabilities, illiterates etc.). X
V-3 The voter shall be able to vote using her personal device, through a standard
internet connection without installing any additional SW.
O
V-4 E-voting system tested in browsers/devices w/ a market share ≥ 1%. ∆
V-5 The interface is WCAG 2.0 AA compliant. ∆

14
Evaluation: 5/10 points
4.13 Cost (C-n)
Cost in a sensitive issue for e-voting systems. Most of them are not transparent in
their pricing policy. That is understandable to a certain point, but even the cheapest
option should offer a sufficient security level.
nVotes used to have a very clear, direct policy with 3 plans with a fix cost of 0.2
EUR per voter plus other associated costs. In its simplest version, it was possible to
organize a 1.000 voter election with all the required elements for a little over 1.000
EUR. Currently, the policy has changed and there is no clear indication of the cost for
the organization of an election.
While probably still an affordable option, the authors believe that the previous, more
transparent approach was better from a user’s point of view.
Table 9. Cost in nVotes
C-n Definition Val
C-1 Transparency and clarity in the cost breakdown.
C-2 System cost related to quality and performance. Comparison with other e-
voting solutions.
Evaluation: Review (6/10 points)
4.14 Maintenance (M-n)
Both from a software and ex-software perspective. On the software side, nVotes is
an open source project and therefore very open and verifiable. It is regularly updated.
Regarding the ex_software aspect, there is not much improvement and it would be very
advisable in order to extend the safe utilization range of the system.
As for everlasting privacy and post-quantum security, nVotes team is working on it
but there is no expected imminent announcement.
Finally, the maintenance cost is quite limited and performed internally.
Table 20. Maintenance in nVotes
M-n Definition Val
M-1 Covering both SW and ex_SW aspects. Frequency, thoroughness and ex-
istence of security logs to check the maintenance process are also evalu-
ated.
∆
M-2 Maintenance as everlasting privacy. N/A
M-3 Maintenance cost itself. ∆
Evaluation: 6.5/10 points

15
5 Final Results and Conclusion
nVotes [218] is a remote e-voting system developed by the Spanish company Agora
Voting SL and active since 2014. It has managed a total of 2 million votes with up to
150.000 votes in the same election.
In order to complement the relatively limited publicly available information for the
analysis in this article, they have been diligent and helpful and the authors with like to
extend their gratitude for their availability.
The ultimate goal of the analysis is not to judge from a rigid, “infallible” perspective
for the sake of it, but to try contribute to a gradual and secure implementation of e-
voting solutions in the democratic processes.
The formula and table below summarize the findings and scores of nVotes:
∑
∙⋯
∙


 ⋅
∑
∙⋯
∙


 (2)
Table 31. Practical Evaltuation Methodology [16] applied to nVotes
Due to the nature of the elections in which nVotes has been deployed, it is in an
intermediate position between Helios Voting and Scytl’s iVote systems. nVotes can
manage elections with a number of voters that Helios Voting has not been able to proof
so far but it presents undeniable shortcomings when it comes to legally binding elec-
tions, where a strong infrastructure, ex-software policies and monitoring/auditing pro-
tocols are a must.
Therefore, currently nVotes’ safe range of use is that of private elections.
Concerning the areas in which nVotes present a better performance, those are:
─ Open source approach, with good software engineering and possibility of review by
researchers/academia
─ Intuitive, simple and user-friendly interface for both the voter and the administrators.
─ Compatibility
─ Open standards, modularity
Requirement Code Weight nVotes
E2Ev E2Ev N.A. ∆
Coerc. Resistance CR N.A. X
Inviolability (I-n) 1.2 4 * 1.2 = 4.8
Usability (U-n) 0.8 6 * 0.8 = 4.8
Monitoring/Audit (MA-n) 1.2 3 * 1.2 = 3.6
Software Devel. (SWD-n) 1.2 7 * 1.2 = 8.4
Scalability (S-n) 0.8 5.5 * 0.8 = 4.4
Ex_Soft. Develop. (ESWD-n) 1.2 4 * 1.2 = 4.8
Incid./AttackProt. (IAP-n) 1.2 4 * 1.2 = 4.8
Versatility (V-n) 0.6 5 * 0.6 = 3
Cost (C-n) 1.0 7 * 1.0 = 7
Maintenance (M-n) 0.8 6.5 * 0.8 = 5.2
TOTAL 10 50.8

16
─ Support service during the elections
As for the aspects which should be improved:
─ No proper Audit/Monitoring or Incidents/Attacks protocols in place
─ Policy for credential, access and permit distribution. Currently allows for collusion
to happen between the census administrator and the election administrator
─ Ex_software development
─ Certain cryptographic primitives implemented are vulnerable [41]
─ No version for voters with special needs
Additionally, the election administrator can know whether a voter has voted or not and
a voter with a fake ID might be able to authenticate to vote. Even for private elections,
it should be an issue to be solved.
In short and considering all the points reviewed in the analysis, the authors estimate
that nVotes is currently not ready to be introduced for public, politically binding elec-
tions due to the limitations in auditing, monitoring, backup and potential collusion. Its
current secure rage is that of private elections, always taking into account the highly
recommended distribution of administrative roles.
The authors hope that this article can contribute, even if modestly, to improve the
knowledge and security level in the deployment of e-voting systems, through the com-
prehensive, multi-faceted evaluation presented.
Acknowledgements.The contribution of Dr. David Duenas-Cid is based upon work
supported by the Estonian Research Council grant (PUT 1361 “Internet Voting as
Additional Channel for Legally Binding Elections: Challenges to Voting Processes
Reengineering”, 2017–2020); and by the Polish National Research Center grant (Min-
iatura 3 - 2019/03/X/HS6/01688 “Zaufanie do technologii w e-administracji:
Powtórna analiza nieudanego wdrożenia elektronicznych maszyn do głosowania w
Holandii (2006-07)”).
References
1. Vinkel, P., Krimmer, R.: The how and why to internet voting an attempt to explain e-stonia.
In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial
Intelligence and Lecture Notes in Bioinformatics). pp. 178–191 (2017).
2. Solvak, M., Vassil, K.: Could Internet Voting Halt Declining Electoral Turnout? New
Evidence That E-Voting Is Habit Forming. Policy and Internet. 10, 1, 4–21 (2018).
3. Krimmer, R. et al.: How much does an e-vote cost? Compared Costs per Vote in
Multichannel Elections in Estonia. In: Krimmer, R. et al. (eds.) Electronic Voting. Third
International Joint Conference, E-Vote-ID 2018. pp. 117–132 Springer International
Publishing, Cham (2018). https://doi.org/10.1007/978-3-030-00419-4.
4. Krimmer, R. et al.: New methodology for calculating cost-efficiency of different ways of
voting: is internet voting cheaper? Public Money Manag. 0, 0, 1–10 (2020).
5. Trechsel, A.H. et al.: Potential and Challenges of E-Voting in the EU Study., Bruss (2016).

17
6. Gjøsteen, K.: Analysis of an internet voting protocol. IACR Cryptol. ePrint Arch. 1–16
(2010). https://doi.org/10.1007/978-3-642-32747-6_1.
7. Kulyk, O. et al.: Electronic Voting with Fully Distributed Trust and Maximized Flexibility
Regarding Ballot Design. In: Krimmer, R. and Volkamer, M. (eds.) Proceedings
EVOTE2014. pp. 139–149 TUT Press, Bregenz (2014).
8. Oostveen, A.-M., Van den Besselaar, P.: Security as belief User’s perceptions on the security
of electronic voting systems. Electron. Voting Eur. Technol. 47, May 2014, 73–82 (2004).
9. Council of Europe: Recommendation CM/Rec(2017)5 of the Committee of Ministers to
member States on standards for e-voting. (2017).
10. Driza Maurer, A.: Updated European standards for e-voting. The Council of Europe
recommendation Rec(2017)5 on standards for e-voting. In: Krimmer, R. et al. (eds.) E-Vote-
ID 2017. pp. 146–162 Springer, Bregenz (2017).
11. Bräunlich K., Grimm R., Richter P., “Sichere Internetwahlen Ein rechtswissenschaftlich-
informatisches Modell.” Nomos (2013).
12. Hammer V., Pordesch U., KORA (Konkretisierung Rechtlicher Inforderungen).
Betriebliche Telefon und ISDN-Anlagen rechtsgemäss gestaltet. (1993.)
13. The Common Criteria Recognition Agreement, “Common Criteria for Information
Technology Security Evaluation Part 1 : Introduction and general model July 2009 Revision
3 Final Foreword,” Nist, vol. 49, no. July, 93 (2009).
14. Neumann S.R., “Evaluation and Improvement of Internet Voting Schemes Based on
Legally-Founded Security Requirements,”, PhD Thesis. Technische Universität Darmstadt.
http://tuprints.ulb.tu-darmstadt.de/5375/1/Thesis.pdf (2016). Last accessed 2020/05/12.
15. “Electoral Commision New South Gales,”. [Online]. Available: http://www.elec-
tions.nsw.gov.au/voting/ivote Last accessed 2020/05/12
16. Marcos del Blanco D. Y. , Panizo Alonso L., and Hermida Alonso JA., “The need for
Harmonization in the online voting field: Towards an European Standard for edemocracy,”
First International Joint Conference, E-Vote-ID 2016, Bregenz, Austria, October 18-21,
2016, Proceedings, pp. 339–340 (2016).
17. Standards, “Guidelines on the implementation of the provisions of Recommendation
CM/Rec(2017)5 on standards for e-voting,” 1289 th Meet. , 14 June 2017 2 . 3 Ad hoc Comm.
Expert. Leg. , Oper. Tech. Stand. e- voting ( CAHVE ), no. June, pp. 1–19, (2017)
18. “Constitución Española” pp. 101931–101941, https://www.boe.es/legislacion/documen-
tos/ConstitucionCASTELLANO.pdf Last accessed 2020/05/12
19. Panizo Alonso L., Gasco M., Marcos del Blanco DY, Hermida Alonso JA, Alaiz Moreton
H. “E-voting system evaluation based on the Council of Europe recommendations: Helios
Voting”, IEEE Transactions on Emerging Topics in Computing, (2018).
20. D. Simić-Draws et al., “Holistic and Law Compatible IT Security Evaluation: Integration of
Common Criteria, ISO 27001/IT-Grundschutz and KORA,” Int. J. Inf. Secur. Priv., vol. 7,
no. 3, pp. 16–35, (2013).
21. Goodman, L.: Snowball Sampling. Ann. Math. Stat. 32, 148–170 (1961).
22. Kish, L.: Sample Design in business research. [American Statistical Association,Ltd.].
23. Benaloh, J.D.C., Rivest, R., Ryan, P.Y.A., Stark, P., Teague, V., Vora, P.: End-to-end veri-
fiability. arXiv e-prints. (2014).
24. Bernhard, D., Neumann, S., Volkamer, M.: Towards a Practical Cryptographic Voting
Scheme Based on Malleable Proofs. In: Heather, J., Schneider, S., and Teague, V. (eds.) E-
Voting and Identify: 4th International Conference, Vote-ID 2013, Guildford, UK, July 17-
19, 2013. Proceedings. pp. 176–192. Springer Berlin Heidelberg, Berlin, Heidelberg (2013).
25. Zissis, D., Lekkas, D.: Design, Development, and Use of Secure ElectronicVotingSys-
tems,http://services.igiglobal.com/resolvedoi/resolve.aspx? (2014).

18
26. Taiwhenua, T.T.: The Department of Internal Affairs - Online voting,
https://www.dia.govt.nz/online-voting. Last accessed 2020/05/12
27. Marcos del Blanco DY “Cybersecurity applied to e-democracy: Cryptographic analysis and
development of a practical evaluation methodology for remote electronic voting systems and
its application to the most relevant solutions” University of Leon, (2018) http://ri-
asc.unileon.es/archivos/documentos/tesis/Tesis_David_Y_Marcos.pdf
28. nVotes Homepage https://nvotes.com/ last accessed 2020/05/14
29. Impact Accelerator, https://www.impact-accelerator.com/ last accessed 2020/05/14
30. Marcos del Blanco DY., Gascó M. “A Protocolized, Comparative Study of Helios Voting
and Scytl/iVote” In: Teran L., Meier A., and Pincay J. (eds.) 2019 Sixth International Con-
ference on eDemocracy & eGovernment (ICEDEG), pp. 31-38. IEEE (2019).
31. Adida, B.: Helios: Web-based Open-audit Voting. In: Proceedings of the 17th Conference
on Security Symposium. pp. 335–348. USENIX Association, Berkeley, CA, USA (2008).
32. Kulyk, O., Teague, V., Volkamer, M.: Extending Helios Towards Private Eligibility Verifi-
ability. In: Haenni, R., Koenig, R., and Wikström, D. (eds.): 5th International Conference,
VoteID 2015, Bern, Switzerland, Proceedings. pp. 57–73. Springer (2015).
33. Cortier V., Gaudry P., Glondu S. “Belenios: a simple private and verifiable electronic voting
system”. Foundations of Security, Protocols, and Equational Reasoning, 2019, Fredericks-
burg, Virgina, United States. pp.214-238, DOI:10.1007/978-3-030-19052-1_14 (2019).
34. Esendex Homepage, https://www.esendex.es/ last accessed 2020/05/14
35. Cloudfare Homepage, https://www.cloudflare.com last accessed 2020/05/15
36. Fail2ban Homepage, https://www.fail2ban.org last accessed 2020/05/14
37. Verificatum Homepage, https://www.verificatum.org/ last accessed 2020/05/14
38. Open STV Homepage, https://www.opavote.com/?openstv=1 last accessed 2020/05/14
39. T. ElGamal. “A Public Key Cryptosystem and a Signature Scheme Based on Discrete Log-
arithms”. InProceedings of CRYPTO 84, vol 196 of LNCS, pp. 10-18, Springer, (1985).
40. Pedersen T. “A Threshold Cryptosystem without a Trusted Party”. Advances in Cryptology
– EUROCRYPT’91. D. Davies editor. Springer – Verlag LNCS series, (1991).
41. Fiat A., Shamir A. “How to prove yourself: Practical solutions to identification and signature
problems.” CRYPTO’86, 6th Annual International Cryptology Conference on Advances in
Cryptology, pp. 186-194. Santa Barbara, USA. (1986).
42. Schnorr. CP “Efficient Identification and Signatures for Smart Cards”. In Gilles Brassard,
editor, Advances in Cryptology - CRYPTO ’89, vol. 435 de Lecture Notes in Computer
Science, pp. 239–252. Springer, (1989).
43. V. Cortier, “Formal Verification of e-Voting: Solutions and Challenges,” ACM SIGLOG
News, vol. 2, no. 1, pp. 25–34, (2015)
44. Acemyan CZ, Kortum P., et al. “From Error to Error: Why Voters Could not Cast a Ballot
and Verify Their Vote with Helios, Pret a Voter and Scantegrity II”. Rice University. In The
Usenix Journal of Election Technology and Systems, (2015)
45. Juels A., Catalano D., Jakobsson M. Coercion-Resistant Electronic Elections. In: Chaum D.
et al. (eds) Lecture Notes in Computer Science, vol 6000. Springer, (2010)
46. Hirt, M., Sako, K.: Efficient Receipt-free Voting Based on Homomorphic Encryption. In:
Proceedings of the 19th International Conference on Theory and Application of Crypto-
graphic Techniques. pp. 539–556. Springer (2000).
47. El Español Homepage, https://www.elespanol.com/espana/20160511/123987880_0.html
last accessed 2020/05/15
48. 20 minutos Homepage, https://www.20minutos.es/noticia/2419700/0/podemos-defiende-
fiabilidad/sistema-votacion-acusaciones/primarias/ 2020/05/15