Risk-limiting audit

A risk-limiting audit (RLA) is one way of checking whether computers tallied an election accurately. It involves (1) storing paper ballots securely until they can be checked,[1] (2) manually comparing a statistical sample of paper ballots to the computers' records for those same ballots,[2] and (3) checking whether all the computer records of ballots in the election were totalled correctly.[3]

Advantages of an RLA include: samples can be small and inexpensive if the margin of victory is large;[3] there are options for the public to watch and verify each step;[1] and errors found in any step lead to corrective actions, including larger samples, up to a 100% hand count if needed. Disadvantages include: the sample needs to be a large fraction of all ballots to minimize the chance of missing mistakes, if any contest is close; and it is hard to check computer totals publicly, except by releasing computer records to the public.[4] If examining sampled ballots shows flaws in ballot storage, the usual approach cannot recover correct results,[5] and researchers recommend a re-vote if the number of ballots held in flawed storage is enough to change winners.[6] An alternative to re-votes is to create and verify backups of the paper ballots soon after they are voted, so there is an alternative to flawed storage of the original ballots.

As with other election audits, the goal is to identify not only intentional alterations of ballots and tallies, but also bugs in election machines, such as software errors, scanners with blocked sensors[7] or scanners skipping some ballots. The approach does not assume that all ballots, contests or machines were handled the same way, in which case spot checks could suffice. The sample sizes are designed to have a high chance of catching even a brief period when a scratch or fleck of paper blocks one sensor of one scanner, or a bug or hack switches votes in one precinct or one contest, if these problems affect enough ballots to change the result.

Comparisons can be done ballot-by-ballot or precinct-by-precinct, though the latter is more expensive.[8]

Sample sizes depend on: winning margin, number of ballots voted, confidence level, and type of audit.

Categories of audits

There are three types of risk-limiting audits.[8] The three approaches are listed in order of increasing numbers of ballots which need to be hand-checked. For example in a jurisdiction with 64,000 ballots, an 8% margin of victory, and allowing 10% of any mistakes to go undetected, method 1, ballot comparison, on average, needs 80 ballots, method 2, ballot polling, needs 700 ballots, and method 3, batch comparison, needs 13,000 ballots (in 26 batches).[8] The methods are usually used to check computer counts, but methods 2 and 3 can also be used to check accuracy when the original results were hand-counted.[1] The steps in each type of risk-limiting audit are:

  1. Ballot comparison. Election computers provide their interpretation of each ballot ("cast vote record"); humans check computers' "cast vote records" against stored physical ballots in a random sample of ballots; an independent computer tabulates all "cast vote records" independently of earlier tabulations to get new totals;[3] humans report any differences in interpretations and total tallies.
  2. Ballot polling. Humans count a random sample of ballots; humans report any difference between manual percentage for the sample and computer percentage for the election.
  3. Batch comparison. Election results provide total for each batch of ballots (e.g. precinct); in a random sample of batches humans hand-count all ballots; for 100% of batches humans check by manual addition or independent computer if the election's initial summation of batches was correct; humans report any difference between original tallies and audit tallies.

All methods require:

  • Procedure to re-count all paper ballots more accurately if errors are detected. This is usually planned as a 100% manual count, but could involve fixing or replacing erroneous computers, doing a new computer count, and auditing that, until an audit shows no problem.
  • Auditing all types of ballots, including military, absentee, provisional, etc.
  • Clarifying which contests were audited and which were not, or auditing all contests or a large enough random sample of contests so the chance of missing erroneous results is acceptably low.
  • Auditing a large enough random sample of ballots so the chance of missing mistakes is acceptably low.
  • Selecting a random sample after initial results are public, because telling hackers in advance which contests and ballots will be in the sample, lets them freely hack other contests and ballots.
  • Selecting the random sample before results are final, so errors can be fixed.
  • Doing the manual check immediately when the sample is selected; if insiders have altered computer files, they could use any delay to change sampled ballots to match the erroneous computer files, thus hiding the errors.
  • Having enough security on the ballots during transportation and storage, so neither insiders nor outsiders can change them.
  • Having enough independent participants select different digits of the random number seed, so no one can control the seed and hence the random number series which selects the random sample.[2]
  • Having the public see all steps, including the content of ballots and computer records while officials examine them, to know they are counted accurately.[1]

The last three items are hard in one-party states, where all participants may be swayed by the ruling party.

Hand-checking ballots (method 1) identifies bugs and hacks in how election computers interpret each ballot, so computer processing can be improved for future elections. Hand-counting ballots (methods 2 and 3) bypasses bugs and hacks in computer counts, so it does not identify exactly what mistakes were made. Independently totaling cast vote records (method 1) or batch totals (method 3) identifies bugs and hacks in how election computers calculate totals. Method 2 does not need this independent totaling step, since it has a large enough sample to identify winners directly.

Colorado uses method 1 in most counties, and method 2 in a few counties which use election machines which do not record and store "cast vote records." Colorado uses no audit method in two counties which hand-count ballots in the first place.[9]

Risk-limiting audits are a results audit to determine if votes were tabulated accurately, not a process audit, to determine if good procedures were followed.[10]

Implementation

The process starts by selecting a ‘risk limit,’ such as 9% in Colorado,[11] meaning that if there are any erroneous winners in the initial results, the audit will catch at least 91% of them and let up to 9% stay undetected and take office.[8][3] Another initial step is to decide whether to audit: all contests; a random sample of contests, allowing a known risk that erroneous winners will take office; or a non-random sample, so no statistical confidence is available on the non-audited contests. Based on a formula, a sample size is determined for each contest being audited.[12][8][13] The size of the sample depends primarily on the margin of victory in the targeted contest.

A random starting point (seed) is chosen by combining information from multiple independent people,[2] to create a series of random numbers identifying specific ballots to pull from storage, such as the 23rd, 189th, 338th, 480th ballots in precinct 1, and other random numbers in other precincts.

When storage is opened, records are checked to see if each sampled precinct still has the same number of ballots recorded during the election, if correct numbers appear on seals, if machines or containers have been tampered with in any way, and/or other methods to check if ballots have avoided intrusion.[14][8][3] If ballots have not been stored successfully, advocates of risk-limiting audits say there should be a re-vote,[6][8] or no result should be declared,[15] which usually requires a re-vote, or results can be declared if "the number of questionable or missing audit records is small enough that they cannot alter the outcome of the contest."[14][8] However if storage or records are flawed, laws may require initial results to be accepted without audit.[16] To provide an alternative to a re-vote, seven Florida counties back up the paper ballots by copying them the day after they are voted, with machines independent of election machines.[17] While any copy can have flaws, comparing cast vote records to these independent backup copies would give an alternative to re-voting or skipping the audit when storage is not trustworthy. Florida does not hand-check this backup, which would be required by a risk-limiting audit. Instead Florida machine-audits 100% of votes and contests. They have found discrepancies of 1-2 ballots from official machines.[18] Maryland has a less safe alternate approach. Maryland's election machines create and store ballot images during the election, separate from the cast vote records. Most election machines do so.[19] Maryland compares cast vote records to these ballot images from the same election machines.[20] Unlike Florida, this approach is not an independent backup or check. A hack or bug in the election machine can alter, skip, or double-count both image and cast vote record simultaneously.[21] Maryland's semi-independent checking is better than no checking, since it has found and resolved discrepancies, such as folded ballots leaving fold lines on the images, which computers interpreted as write-in votes; sensor flaws which left lines on the images, interpreted as overvotes; and double-feeds where two ballots overlap in the scanner, and one is uncounted.[22]

When an audit produces the same result as initial election results, the outcome is confirmed, subject to the risk limit, and the audit is complete. If the audit sample shows enough discrepancies to call the outcome into question, a larger sample is selected and counted. This process can continue until the sample confirms the original winner, or a different winner is determined by hand-counting all ballots.[8]

Issues
Colorado elections in 2017, sample sizes needed for risk-limiting audits

Sample size

Sample sizes rise rapidly for narrow margins of victory, with all methods. In a small city or county, with 4,000 ballots, method 1, ballot comparison, would need 300 ballots (300–600 minutes, as discussed in Cost below) for a contest with a 2% margin of victory. It would need 3,000 ballots (50-100 staff hours in the city or county) for a 0.1% margin of victory. Method 2 or 3, ballot polling or batch comparison, would need a full hand count of the 4,000 ballots (70-130 staff hours). Margins under 0.1% occur in one in sixty to one in 460 contests. Large numbers of contests on a ballot raise the chances that these small margins and large samples will occur in a jurisdiction, which is why no place does risk-limiting audits on all contests, leaving most local government races unaudited, though millions of dollars are at stake in local spending[23] and land use decisions.[24] Colorado picks contests with wider margins to avoid large samples.[12]

When sample sizes are limited by budget, they still have some likelihood of catching errors.

When Maryland evaluated audit methods, it noted that local boards of elections could not budget, or plan staffing, for risk-limiting audits, since the sample "is highly dependent on the margin of victory in any given audited contest... A very close margin of victory could... require days of staff work, possibly compromising the local certification deadline."[20]

An alternative to large samples is to audit an affordable sample size, and let the risk limit vary instead of the sample size.[21] For a fixed sample, closer margins of victory would have more risk of letting erroneous winners take office, but any substantial sample would still have a known substantial chance of catching errors. Election managers would announce the level of confidence provided by the sample, and would have procedures to follow up if the sample finds one or more errors.[21]

The sample sizes presented will be enough to confirm a result, subject to the risk limit, when the apparent winner is the actual winner. If the sample does not confirm the win, more ballots are sampled, up to a 100% hand count to confirm a different winner.[8]

Ballot transport and storage

Ballots are at risk when being transported from polling places to central storage, and may be accompanied by guards and/or a convoy of the public.[25]

Auditing in the US is done several days after the election, so paper ballots and computer files need to be stored securely. Even insiders have breached security.[26][27] Physical security has its own large challenges. No US state has adequate laws on physical security of the ballots.[28] 2008 guidelines for risk-limiting audits included starting audits as soon as possible after the election.[29] 2018 guidelines recommend waiting until all ballots are counted, which in the US means one or more weeks until provisional ballots are adjudicated.[30] A faster start for the audit makes it feasible for independent parties to guard storage sites.[31] Security recommendations include preventing access by anyone alone,[1] which would typically require two hard-to-pick locks, and having keys held by independent officials if such officials exist in the jurisdiction; having storage risks identified by people other than those who design or manage the system; and using background checks on staff.[32]

Britain,[33] Canada,[34] France,[35] Germany,[36] Spain[37] and other countries count all ballots on election night, including final decisions on questionable ballots, so final counts happen before ballots go into storage. Canada[38] and France[39] sometimes use election machines with paper trails. In these cases risk-limiting audits could be done as soon as results are announced on election night.

If audits cannot be done on election night, creating and verifying a paper or electronic backup of the paper ballots on election night, or the next day as in Florida,[17] would provide an alternative benchmark if flaws are later discovered or suspected in the primary storage of paper ballots. Electronic copies in particular can be stored in safes or safe deposit boxes and/or distributed widely, with hash codes to ensure reliability.[40][4]

Election storage often uses tamper-evident seals,[32][8] although seals can typically be removed and reapplied without damage, especially in the first 48 hours.[41] Photos taken when the seal is applied can be compared to photos taken when the seal is opened.[5] Detecting subtle tampering requires substantial training.[41][42][43] Election officials usually take too little time to examine seals, and the public is too far away to check seal numbers, though they could compare old and new photos projected on a screen. Seal numbers and photos would need their own secure storage.

Experienced testers can usually bypass all physical security systems.[44] Locks[45] and cameras[46] are vulnerable before and after delivery.[44] Insider threats and the difficulty of following all security procedures are usually under-appreciated, and most organizations do not want to learn their vulnerabilities.[44]

Method 1 requires the ballots to be kept in strict order so one can compare the computer interpretations of sampled ballots with those exact physical ballots.[47] If the correct ballots are present, but out of order, method 2 can be used. Maryland, like other states, randomizes the order of paper ballots and cast vote records to protect ballot secrecy, so method 1 cannot be done there, since paper ballots and cast vote records cannot be compared.[20]

Public monitoring

All the methods, when done for a state-wide election, involve manual work throughout the state, wherever ballots are stored, so the public and candidates need observers at every location to be sure procedures are followed.[1] However, in Colorado and most states the law does not require any of the audit work to be done in public.[48]

Independent totals

Method 1, ballot comparison, includes a second step, besides checking the sample of ballots: 100% of the computer interpretations of ballots ("cast vote records") need to be re-tabulated by computers independent of the original election computers.[3] This re-tabulation checks whether election computers tallied the cast vote records correctly. Like any computer step this independent tally is subject to hacks and bugs, especially when voting rules are complex, such as variations in the number of candidates from different districts to vote for. The reason for the re-tabulation step is that independently programming a different kind of machine provides an independent check on official election machines.

While all methods require physical security on the paper ballots, method 1 also requires enough security on the cast vote records so no one can change them. This can be accomplished by computer-calculating, storing and comparing a hash code for each file of cast vote records:[49] (a) right after the election, (b) when independent tabulation is done, and (c) when ballot comparison is done.[21]

Colorado says it has a system to do the independent count of cast vote records, but it is not yet publicly documented,[50] so the chance of bugs or hacks affecting this independent computer at the Secretary of State's office along with one or more of the election machines is unknown.

Cost

Cost depends on pay levels and staff time needed, recognizing that staff generally work in teams of two or three (one to read and one or two to record votes). Teams of four, with two to read and two to record are more secure[51] and would increase costs.

Staff Time Needed for Hand Counts
Location Total Staff Time, Minutes per Vote Checked Number of Contests Checked per Ballot Total Staff Time, Minutes per Ballot Checked Full Precincts / Batches, or Random Ballots Number of Ballots Checked Team Sizes Year and Source Ballots on Rolls or Sheets of Paper? Overheads Excluded?
Carroll County, MD236no ballots checked, just images24722016 [20]SheetsOrganizing ballot images for review, training, legal, supervision
Montgomery County MD235no ballots checked, just images8222016 [20]SheetsOrganizing ballot images for review, training, legal, supervision
Humboldt County, CA6318random143unknown2011 [52]SheetsIndependent count, done by graduate student on university computer
Merced County, CA224random198unknown2011 [52]SheetsIndependent count, done by graduate student on university computer
Orange County, CA222mostly full467unknown2011 [52]SheetsIndependent count, done by graduate student on university computer
Bibb County, GA0.53921full59232006 [53]Rolls
Camden County, GA0.33411full47032006 [53]Rolls
Cobb County, GA0.64225full97632006 [53]Rolls
Clark County, NV0.72115full1,268unknown2004 [54]Rolls
Washington State1.311.3full1,886,490unknown2004 [54]Sheets

One to two minutes per vote checked means 25-50 cents per vote at $15/hour, or $2,500-$5,000 per ten thousand votes. Checking full precincts is relevant to method 3 and 100% hand-counts, when needed. Checking random ballots can take more time: pulling individual ballots from boxes and returning them to the same spot. It is relevant to methods 1 and 2.

State variations

As of early 2017, about half the states require some form of results audit.[48] Typically, these states prescribe audits that check only a small flat percentage, such as 1%, of voting machines. As a result, few jurisdictions have samples large or timely enough to detect and correct tabulation errors before election results are declared final.[55][56]

In 2017, Colorado[57] became the first state to implement ballot comparison audits, auditing one contest, not randomly chosen, in each of 50 of its 64 counties,[58] several days after the election. Following the 2018 General Election, Colorado will conduct audits in the 62 of its 64 counties that use automated vote counting equipment (the two remaining counties hand count the ballots).

Rhode Island passed legislation[59] requiring that state's Board of Elections to implement risk-limiting audits beginning in 2018. Individual jurisdictions elsewhere may be using the method on the local election clerks' initiative.

Endorsements

In 2018 the American Statistical Association, Brennan Center for Justice, Common Cause, Public Citizen and several election integrity groups endorsed all three methods of risk-limited audits. Their first five criteria are:[1]

  1. EXAMINATION OF VOTER-VERIFIABLE PAPER BALLOTS: Audits require human examination of voter-marked paper ballots — the ground truth of the election. Voter-marked paper ballots may be marked by hand or by ballot marking device. Audits cannot rely on scanned images or machine interpretations of the ballots to accurately reflect voter intent.
  2. TRANSPARENCY: Elections belong to the public. The public must be able to observe the audit and verify that it has been conducted correctly, without interfering with the process.
  3. SEPARATION OF RESPONSIBILITIES: Neither the policy and regulation setting for the audit, nor the authority to judge whether an audit has satisfied those regulations, shall be solely in the hands of any entity directly involved with the tabulation of the ballots or the examination of ballots during the audit.
  4. BALLOT PROTECTION: All the ballots being tabulated and audited must be verifiably protected from loss, substitution, alteration or addition.
  5. COMPREHENSIVENESS: All jurisdictions and all validly cast ballots, including absentee, mail-in and accepted provisional ballots, must be taken into account. No contest should be excluded a priori from auditing, although some contests may be prioritized.

In 2014, the Presidential Commission on Election Administration recommended the methods in broad terms:

  • "Commission endorses both risk-limiting audits that ensure the correct winner has been determined according to a sample of votes cast, and performance audits that evaluate whether the voting technology performs as promised and expected."[60]

By selecting samples of varying sizes dictated by statistical risk, risk-limiting audits eliminate the need to count all the ballots to obtain a rapid test of the outcome (that, is, who won?), while providing some level of statistical confidence.

In 2011, the federal Election Assistance Commission initiated grants for pilot projects to test and demonstrate the method in actual elections.[61]

Professor Phillip Stark of the University of California at Berkeley has posted tools for the conduct of risk-limiting audits on the university's website.[62]

See also

References
  1. "Principles and Best Practices for Post-Election Tabulation Audits" (PDF). ElectionAudits.org. 2018. Retrieved August 12, 2019.
  2. Cyrus Farivar (July 25, 2012). "Saving throw: securing democracy with stats, spreadsheets, and 10-sided dice". Ars Technica. Retrieved July 27, 2012.
  3. Stark, Philip (March 16, 2012). "Gentle Introduction to Risk-limiting Audits" (PDF). IEEE Security and Privacy – via University of California at Berkeley, pages 1, 3.
  4. Trachtenberg, Mitch (2013-07-29). "The Humboldt County Election Transparency Project and TEVS" (PDF). Report for Elections Advisory Commission. Retrieved 2018-04-05.
  5. Stark, Philip (July 26, 2018). "An Introduction to Risk-Limiting Audits and Evidence-Based Elections Prepared for the Little Hoover Commission" (PDF). University of California, Berkeley. Retrieved August 16, 2019.
  6. Benaloh, Josh, Douglas Jones, Eric L. Lazarus, Mark Lindeman, Philip B. Stark (September 9, 2011). "SOBA: Secrecy-preserving Observable Ballot-level Audit (slides)" (PDF). USENIX EVT/WOTE. Retrieved August 14, 2019.
  7. Examples of blocked or faulty scanner sensors include:
  8. Lindeman, Mark (executive editor), Jennie Bretschneider, Sean Flaherty, Susannah Goodman, Mark Halvorson, Roger Johnston, Ronald L. Rivest, Pam Smith, Philip B. Stark (October 1, 2012). "Risk-Limiting Post-Election Audits: Why and How" (PDF). University of California at Berkeley. pp. 3, 16. Retrieved April 9, 2018.
  9. Volkosh, Dan (January 4, 2018). "Voting Systems Public". Colorado Secretary of State. Retrieved April 9, 2018.
  10. "Report on Election Auditing by the Election Audits Task Force of the League of Women Voters of the United States" (PDF). League of Women Voters. 2009. Retrieved April 7, 2017.
  11. "Round #1 State Report". www.sos.state.co.us/pubs/elections/auditCenter.html. November 20, 2017. Retrieved April 9, 2018.
  12. Colorado Secretary of State (July 5, 2018). "Target contests for risk‐limiting audits per Election Rule 25.2.2(i)" (PDF). sos.state.co.us. Retrieved June 29, 2019.
  13. Formulae are presented and explained in:
  14. Stark, PB and DA Wagner (May 8, 2012). "Evidence-Based Elections" (PDF). IEEE Security and Privacy.
  15. Benaloh, Josh; Jones, Douglas; Lazarus, Eric; Lindeman, Mark; Stark, Philip B. (2011). "SOBA: Secrecy-preserving Observable Ballot-level Audit". arXiv:1105.5803 [stat.AP].
  16. Kurth, Joel (December 13, 2016). "Records: Too many votes in 37% of Detroit's precincts". Retrieved August 17, 2019. and North Carolina law: "2015 North Carolina General Statutes: Principles and rules for counting official ballots. § 163-182.1 - 1(b)(1)". Justia Law. Retrieved August 17, 2019.
  17. Stofan, Jake (April 25, 2018). "Leon County among Florida precincts to implement Clear Audit for elections". WCTV. Retrieved August 17, 2019.
  18. Dailey, Aikens, Maddox (November 21, 2016). "Voting System Post-Election Audit Report" (PDF). Leon County, Florida. Retrieved August 17, 2019.CS1 maint: multiple names: authors list (link)
  19. Harris, Bev (November 2, 2016). "Ballot Images – A new way to verify that results are true". BlackBoxVoting.org. Retrieved August 14, 2019.
  20. Maryland State Board of Elections (October 21, 2016). "Post-Election Tabulation Audit Pilot Program Report" (PDF). elections.maryland.gov. Retrieved June 29, 2019.
  21. Vora, Poorvi (November 6, 2016). ""Exhibit B. Pages 20-23 in Lamone, Linda H. (December 22, 2016). "Joint Chairman's Report on the 2016 Post-Election Tabulation Audit" (PDF). Maryland State Board of Elections. Retrieved August 12, 2019.
  22. Walker, Natasha (February 16, 2017). "2016 Post Election Audits in Maryland" (PDF). National Institute of Standards and Technology. Retrieved August 12, 2019.
  23. "State and Local Expenditures". Urban Institute. March 5, 2015. Retrieved August 17, 2019.
  24. "Land Use Decisions and Decision Making Processes". University of Oregon. Retrieved August 17, 2019.
  25. Experts recommend letting different party members ride in the ballot transport vehicle, letting other observers accompany the vehicle, and using guards when needed: "Securing Ballot Boxes During Transportation —". aceproject.org. Retrieved August 12, 2019. Ballots have fallen from trucks during transport: "South Africa: Most Mpumalanga Ballot Papers Recovered". AllAfrica. April 22, 2009. Retrieved August 15, 2019. King, Colbert (September 9, 2006). "A Ballad of the Ballot Follies in D.C." Washington Post. ISSN 0190-8286. Retrieved August 15, 2019.
  26. Turner, Karl (November 5, 2007). "Elections board workers take plea deal". Cleveland Plain Dealer. Retrieved August 17, 2019.
  27. Recount Now (January 11, 2017). "Report on the 2016 Presidential Recount in Clark County, Nevada. Page 20" (PDF). Internet Archive. Retrieved August 17, 2019.
  28. Benaloh, Public Evidence from Secret Ballots; et al. (2017). Electronic voting : second International Joint Conference, E-Vote-ID 2017, Bregenz, Austria, October 24–27, 2017, proceedings. Cham, Switzerland. p. 122. ISBN 9783319686875. OCLC 1006721597.
  29. "as soon as possible after the initial tallies" Lindeman, Mark, Mark Halvorson, Pamela Smith, Lynn Garland, Vittorio Addona, Dan McCrea (September 1, 2008). "Best Practices: Chain of Custody and Ballot Accounting, ElectionAudits.org" (PDF). Brennan Center for Justice et al. Retrieved April 9, 2018.
  30. "Starting to audit only when all the audit units have already been counted is the most straightforward method." "Principles and Best Practices for Post-Election Tabulation Audits" (PDF). ElectionAudits.org. 2018. Retrieved August 12, 2019.
  31. Organisation for Security and Cooperation in Europe (OSCE) policy calls for independent foreign officials to sleep with ballots, and allows parties to do so: Turkish opposition sleeping with balllots: Republicans guarded warehouse of voted ballots in 2000 in Florida and had state police lock up ballots in New Mexico:
  32. "Chapter 3. PHYSICAL SECURITY" (PDF). US Election Assistance Commission. Retrieved April 24, 2018.
  33. Game, Chris (May 7, 2015). "Explainer: how Britain counts its votes". The Conversation. Retrieved August 16, 2019. Keaveney, Paula (June 8, 2017). "How votes are counted on election night". The Conversation. Retrieved August 16, 2019.
  34. "Elections, Our Country, Our Parliament". lop.parl.ca. Retrieved August 16, 2019.
  35. "Qu'est-ce qu'un dépouillement ? - Comment se déroule une journée dans un bureau de vote ? Découverte des institutions - Repères - vie-publique.fr" (in French). January 14, 2018. Retrieved August 16, 2019.
  36. "Stimmenauszählung". Mülheim an der Ruhr (in German). 2019. Retrieved August 17, 2019.
  37. "¿Qué es el escrutinio y cómo se cuentan los votos en las elecciones generales 2019?". El Confidencial (in Spanish). April 28, 2019. Retrieved August 17, 2019. and Section 14 of the law:"Ley Orgánica 5/1985, de 19 de Junio, del régimen electoral general. SECCIÓN 14.ª ESCRUTINIO EN LAS MESAS ELECTORALES". www.juntaelectoralcentral.es. Retrieved August 17, 2019.
  38. Casey, Liam (May 9, 2018). "Electronic voting machines expected to speed up voting, ballot counting: Elections Ontario". Global News. Retrieved August 16, 2019.
  39. "Quelles sont les règles applicables dans les bureaux de vote dotés d'une machine à voter ? - Conseil constitutionnel présidentielle 2017". presidentielle2017.conseil-constitutionnel.fr. Retrieved August 16, 2019.
  40. Burke, Paul (2018). "Scanners, Hashes, and Election Security" (PDF). J of Physical Security. 11: 1–19 – via RBSekurity.
  41. Johnston, Roger G., and Jon S. Warner (July 31, 2012). "How to Choose and Use Seals". Army Sustainment. Retrieved May 4, 2018.
  42. Coherent Cyber, Freeman, Craft McGregor Group (August 28, 2017). "Security Test Report ES&S Electionware 5.2.1.0" (PDF): 9 – via California Secretary of State. Cite journal requires |journal= (help)CS1 maint: multiple names: authors list (link)
  43. Stauffer, Jacob (November 4, 2016). "Vulnerability & Security Assessment Report Election Systems &Software's Unity 3.4.1.0" (PDF) – via Freeman, Craft, MacGregor Group for California Secretary of State. Cite journal requires |journal= (help)
  44. Seivold, Garett (April 2, 2018). "Physical Security Threats and Vulnerabilities - LPM". losspreventionmedia.com. Retrieved April 24, 2018.
  45. Lockpicking is widely taught and practiced: Vanderbilt, Tom (March 12, 2013). "The Strange Things That Happen at a Lock-picking Convention". Retrieved August 15, 2019. Different techniques apply to electronic locks: Menn, Joseph (August 6, 2019). "Exclusive: High-security locks for government and banks hacked by researcher". Reuters. Retrieved August 16, 2019. Greenberg, Andy (August 29, 2017). "Inside an Epic Hotel Room Hacking Spree". Wired. ISSN 1059-1028. Retrieved August 16, 2019. There are no statistics on how often criminals enter rooms undetected, but law enforcement often does so, so ability to enter rooms undetected is widespread: Tien, Lee (October 26, 2014). "Peekaboo, I See You: Government Authority Intended for Terrorism is Used for Other Purposes". Electronic Frontier Foundation. Retrieved August 15, 2019.
  46. Security camera flaws have been covered extensively: Bannister, Adam (October 7, 2016). "How to hack a security camera. It's alarmingly simple". IFSEC Global, Security and Fire News and Resources. Retrieved August 16, 2019. Doffman, Zak. "Official Cybersecurity Review Finds U.S. Military Buying High-Risk Chinese Tech (Updated)". Forbes. Retrieved August 14, 2019. Schneier, Bruce (October 8, 2007). "Hacking Security Cameras - Schneier on Security". www.schneier.com. Retrieved August 16, 2019. Dunn, John (June 11, 2019). "Critical flaws found in Amcrest security cameras". Naked Security. Retrieved August 16, 2019.
  47. Lovato, Jerome, Danny Casias, and Jessi Romero. "Colorado Risk-Limiting Audit:Conception to Application" (PDF). Bowen center for Public Affairs and Colorado Secretary of State. Retrieved April 5, 2018.
  48. "State Audit Laws". Verified Voting. February 10, 2017. Retrieved April 2, 2018.
  49. "Colorado Secretary of State Election Rules [8 CCR 1505-1], Rule 25. Post-election audit" (PDF). Colorado Secretary of State. August 23, 2018. Retrieved August 12, 2019.
  50. Lindeman, Mark, Ronald L. Rivest, Philip B. Stark and Neal McBurnett (January 3, 2018). "Comments re statistics of auditing the 2018 Colorado elections" (PDF). Colorado Secretary of State. Retrieved April 9, 2018.
  51. Goggin, Stephen N., Michael D. Byrne, and Juan E. Gilbert (2012). "Post-Election Auditing: Effects of Procedure and Ballot Type on Manual Counting Accuracy, Efficiency, and Auditor Satisfaction and Confidence" (PDF). Election Law Journal.
  52. California Secretary of State (July 30, 2014). "Post-Election Risk-Limiting Audit Pilot Program 2011-2013, Final Report to the United States Election Assistance Commission" (PDF). Internet Archive. and California Secretary of State (July 30, 2014). "Appendices, Post-Election Risk-Limiting Audit Pilot Program 2011-2013 Final Report to the United States Election Assistance Commission." Pages 81-90" (PDF). Internet Archive. and Overview. The time estimates of other California counties in the study included time to scan ballots to enable ballot comparison audits, so their costs were not comparable. None of the 11 California counties doing audits chose a close race or needed a 100% hand-count.
  53. Georgia Secretary of State, Elections Division. (2007-04). " (April 1, 2007). "Voter Verified Paper Audit Trail Pilot Project Report. Pages 18-22, 42-63" (PDF). web.archive.org. Retrieved August 17, 2019.
  54. Theisen, Ellen (2004). "Cost Estimate for Hand Counting 2% of the Precincts in the U.S." (PDF). votersunite.org. Retrieved May 4, 2018.
  55. Counting Votes 2012: A State by State Look at Voting Technology Preparedness, Pamela Smith, Verified Voting Foundation; Michelle Mulder, Rutgers School of Law-Newark; Susannah Goodman, Common Cause Education Fund, 2012. Accessed May 12, 2017
  56. Trautgott, Michael W.; Conrad, Frederick G. (2012). "Confidence in the Electoral System: Why We Do Auditing". In Alvarex, R. Michael; Atkeson, Lonna Rae; Hall, Thad E. (eds.). Confirming Elections: Creating Confidence and Integrity Through Election Auditing. Palgrave MacMillan. ISBN 978-0-230-34121-0.
  57. Paul, Jesse (November 22, 2017). "Colorado's first-of-its-kind election audit is complete, with all participating counties passing". The Denver Post. Retrieved April 10, 2018.
  58. Casias, Danny (November 8, 2017). "Audited contests in each county for 2017 RLA (XLSX)". www.sos.state.co.us/pubs/elections/auditCenter.html. Retrieved April 15, 2018.
  59. "Passing Your Vote Through Security: The Rise of Risk Limiting Audits in Rhode Island". State of Elections. January 29, 2018. Retrieved June 30, 2019.
  60. The American Voting Experience: Report and Recommendations of the Presidential Commission on Election Administration, January 2014
  61. Post-Election Risk-Limiting Audit Pilot Program 2011-2013, Final Report to the United States Election Assistance Commission, California Secretary of State 2013. Accessed May 12, 2017
  62. "Tools for Comparison Risk-Limiting Election Audits". www.stat.berkeley.edu. Retrieved May 17, 2017.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.