Conference Paper

A Risk-Limiting Audit in Denmark: A Pilot

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

The theory of risk-limiting audits is well-understood, at least mathematically. Such audits serve to create confidence in the reported election outcome by checking the evidence created during the election. When election officials introduce election technologies into the voting process, it is best to do this after the appropriate auditing framework has been implemented. In this paper, we describe our experiences with piloting a risk-limiting audit of a referendum that was held in Denmark on December 3, 2015. At the time of the publication of this paper, Denmark’s election law did not permit electronic voting technologies to be used during voting allowing us to study auditing in isolation. Our findings are that (1) risk-limiting audits also apply to paper and pencil elections; (2) election officials usually support risk-limiting audits even if no voting technologies are used because these audits can improve the efficiency of the manual count; (3) that practical and organizational challenges must be overcome to keep audits repeatable, in particular it must be possible to identify individual ballots repeatedly and reliably; (4) it is possible to arrange an audit for the result of an earlier stage in a count during a later stage, for example, an audit of the rough count results fine count; and (5) that whenever the electronic voting technologies are considered, auditing should be considered as part of feasibility study.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... It will automatically trigger a full recount if enough statistical evidence is not found to support the election outcome, and it can be adopted to several electoral systems, including first-past the post [6], d'Hondt based systems [15], and instant runoff elections [1]. One would expect RLAs to strengthen public confidence in the election, first, because they can be integrated into existing election processes, for example, in Denmark, where the result of the first (rough) count can be verified during the second (fine) count [10]. Second, some of the ceremonies surrounding RLAs can be turned into public events, such as the dice-rolling ceremony used to create entropy to select a random sample (see the public notice of the Colorado Secretary of State [5]). ...
... The statutory pilot programs are run in Georgia, Indiana, and Nevada whereas the use of RLAs is optional in California, Ohio, Oregon and Washington. In contrast, Michigan and New Jersey are running only administrative pilot programs, similar to the one, we conducted in Denmark [10]. For a detailed description of risk-limiting audits and its application in other countries, see [11]. ...
Preprint
Full-text available
Risk-limiting audits (RLAs) are expected to strengthen the public confidence in the correctness of an election outcome. We hypothesize that this is not always the case, in part because for large margins between the winner and the runner-up, the number of ballots to be drawn can be so small that voters lose confidence. We conduct a user study with 105 participants resident in the US. Our findings confirm the hypothesis, showing that our study participants felt less confident when they were told the number of ballots audited for RLAs. We elaborate on our findings and propose recommendations for future use of RLAs.
... Finally, statistics is a somewhat different approach, which has to do with verification of the ballot count. The idea is that a small random sample of ballots can be used post-tallying to do so-called 'risk-limiting audits' which can mathematically verify the full count (see Schürmann 2016). The major problem is that this solution does not seem trustworthy to many non-experts, who have a hard time understanding how small samples can be statistically sufficient to verify the whole set of votes. ...
... Some states require them by law, while others are in more preliminary stages of their implementation. Specific RLA methods were also designed for a number of elections in Europe [41] and Australia [7], but to the best of our knowledge they have yet to be implemented, with the sole exception of a preliminary pilot conducted in Denmark [33]. ...
Preprint
Full-text available
Risk-limiting audits (RLAs) are a significant tool in increasing confidence in the accuracy of elections. They consist of randomized algorithms which check that an election's vote tally, as reported by a vote tabulation system, corresponds to the correct candidates winning. If an initial vote count leads to the wrong election winner, an RLA guarantees to identify the error with high probability over its own randomness. These audits operate by sequentially sampling and examining ballots until they can either confirm the reported winner or identify the true winner. The first part of this work suggests a new generic method, called ``Batchcomp", for converting classical (ballot-level) RLAs into ones that operate on batches. As a concrete application of the suggested method, we develop the first ballot-level RLA for the Israeli Knesset elections, and convert it to one which operates on batches. We ran the suggested ``Batchcomp" procedure on the results of 22nd, 23rd and 24th Knesset elections, both with and without errors. The second part of this work suggests a new use-case for RLAs: verifying that a population census leads to the correct allocation of political power to a nation's districts or federal-states. We present an adaptation of ALPHA, an existing RLA method, to a method which applies to censuses. Our census-RLA is applicable in nations where parliament seats are allocated to geographical regions in proportion to their population according to a certain class of functions (highest averages). It relies on data from both the census and from an additional procedure which is already conducted in many countries today, called a post-enumeration survey.
... While the full information assumption is not entirely realistic, we note that in a district-based model both parties only need to know the vote counts in each district rather than individual votes, and one can get fairly accurate district-level information from independent polls. Also, verifying whether the votes in a district have been tampered with is possible using risk-limiting audits Lindeman and Stark [2012]; Schürmann [2016]. ...
Preprint
Full-text available
Complexity of voting manipulation is a prominent topic in computational social choice. In this work, we consider a two-stage voting manipulation scenario. First, a malicious party (an attacker) attempts to manipulate the election outcome in favor of a preferred candidate by changing the vote counts in some of the voting districts. Afterwards, another party (a defender), which cares about the voters' wishes, demands a recount in a subset of the manipulated districts, restoring their vote counts to their original values. We investigate the resulting Stackelberg game for the case where votes are aggregated using two variants of the Plurality rule, and obtain an almost complete picture of the complexity landscape, both from the attacker's and from the defender's perspective.
Book
Full-text available
This volume contains papers presented at E-Vote-ID 2021, the Sixth International Joint Conference on Electronic Voting, held during October 5-8, 2021. Due to the extraordinary situation provoked by Covid-19 Pandemic, the conference is held online for second consecutive edition, instead of in the traditional venue in Bregenz, Austria. E-Vote-ID Conference resulted from the merging of EVOTE and Vote-ID and counting up to 17 years since the �rst E-Vote conference in Austria. Since that conference in 2004, over 1000 experts have attended the venue, including scholars, practitioners, authorities, electoral managers, vendors, and PhD Students. The conference collected the most relevant debates on the development of Electronic Voting, from aspects relating to security and usability through to practical experiences and applications of voting systems, also including legal, social or political aspects, amongst others; turning out to be an important global referent in relation to this issue. Also, this year, the conference consisted of: { Security, Usability and Technical Issues Track { Administrative, Legal, Political and Social Issues Track { Election and Practical Experiences Track { PhD Colloquium, Poster and Demo Session on the day before the conference E-VOTE-ID 2021 received 49 submissions, being, each of them, reviewed by 3 to 5 program committee members, using a double blind review process. As a result, 27 papers were accepted for its presentation in the conference. The selected papers cover a wide range of topics connected with electronic voting, including experiences and revisions of the real uses of E-voting systems and corresponding processes in elections.
Article
Complexity of voting manipulation is a prominent topic in computational social choice. In this work, we consider a two-stage voting manipulation scenario. First, a malicious party (an attacker) attempts to manipulate the election outcome in favor of a preferred candidate by changing the vote counts in some of the voting districts. Afterwards, another party (a defender), which cares about the voters' wishes, demands a recount in a subset of the manipulated districts, restoring their vote counts to their original values. We investigate the resulting Stackelberg game for the case where votes are aggregated using two variants of the Plurality rule, and obtain an almost complete picture of the complexity landscape, both from the attacker's and from the defender's perspective.
Chapter
Electronic Voting Machines (EVMs) used in the 2019 General Elections in India were fitted with printers to produce Voter-Verifiable Paper Audit Trails (VVPATs). VVPATs allow voters to check whether their votes were recorded as they intended. However, confidence in election results requires more: VVPATs must be preserved inviolate and then actually used to check the reported election result in a trustworthy way that the public can verify. A full manual tally from the VVPATs could be prohibitively expensive and time-consuming; moreover, it is difficult for the public to determine whether a full hand count was conducted accurately. We show how Risk-Limiting Audits (RLAs) can provide high confidence in Indian election results. Compared to full hand recounts, RLAs typically require manually inspecting far fewer VVPATs when the outcome is correct, and are much easier for the electorate to observe in adequate detail to determine whether the result is trustworthy.We show how to apply two RLA strategies, ballot-level comparison and ballot polling, to General Elections in India. Our main result is a novel method for combining RLAs in constituencies to obtain an RLA of the overall parliamentary election result. KeywordsRisk-limiting auditBallot-polling auditBallot-level comparison auditTransitive auditMulti-level election auditFisher’s combining function
Article
Full-text available
Risk-limiting audits provide statistical assurance that election outcomes are correct by manually examining portions of the audit trail-paper ballots or voter-verifiable paper records. This article sketches two types of risk-limiting audits, ballot-polling audits and comparison audits, and gives example computations. These audits do not require in-house statistical expertise.
Article
Full-text available
The MACRO method of Stark (9) can be used to derive an ex- tremely simple risk-limiting audit procedure based on sampling individual ballots at random. There is an easy rule to determine how many ballots must be audited at random to confirm, at a given risk limit, the outcomes of all the contests on those ballots for which the reported margin in votes is suciently large—provided the percentage of ballots the audit finds to have overstated any margin is suciently low. For instance, suppose there is a collection of contests we wish to audit at simultaneous risk limit no larger than 10%: If any of the outcomes is wrong, we want there to be at least a 90% chance of a full hand count to set the record straight. Suppose that the smallest margin among the contests is V votes, and that N ballots were cast in those contests combined. We would like to be able to stop the audit if the percentage of ballots in the sample with errors that overstated the margin of some winner over some loser by one vote is at most M/2V (half the smallest margin, as a percentage of the total number of ballots), and no ballot in the sample has an error that overstated the margin of a winner over a loser by two votes. Then we may start by auditing a random sample of n = 15.2N/V ballots. Concretely, suppose that 100,000 ballots were cast in all and that the smallest margin among the contests was 2,000 ballots. We draw a random sample of 15.2/2% = 761 ballots. If at most 7 of those ballots (1% of the sample, half the diluted margin of 2%) shows one or more errors that overstated a margin by one vote and none shows an error that overstated any margin by 2 votes, the audit can stop: All contests can be certified at a risk limit of 6.7% (< 10%). If the audit reveals more errors, the sample might need to be expanded; in that case, the MACRO computations of Stark (9) determine when sampling can stop. This paper gives general rules for selecting the initial sample size as a function of the risk limit, the smallest margin, and the percentage of one-vote margin over- statements we would like to tolerate among ballots in the sample without having to expand the initial sample.
State of colorado, risk-limiting audit final report, post-election audit initiative grant no. eac110150e
  • N Mcburnett