Today, ACCURATE submitted comments on the first such pilot program under the new system, geared towards UOCAVA voters. This pilot program is a joint collaboration between FVAP, NIST and EAC, under the MOVE Act, that seeks to provide “kiosk” voting systems for a federal election for UOCAVA voters.

It’s an ambitious undertaking, and the draft standard reflects a great deal of work towards setting requirements to which voting systems can be tested and certified to provide UOCAVA voting capacity. ACCURATE’s comments break down like so:

• The focus on controlled, supervised voting system architectures is appropriate. Many of the fundamental problems with forms of Internet voting are associated with uncontrolled platforms–users PCs, mobile devices, etc.–in unsupervised environments–i.e., at home instead of a dedicated polling place-like environment. The requirements restrict voting systems to dedicated platforms in supervised environments, short-circuiting this concern with broader efforts at Internet voting.
• The requirement for a Voter-Verified Paper Record (VVPR) is warranted. ACCURATE strongly believes that auditability achieved through an independent, indelible audit trail that the voter has an opportunity to correct is an essential part of computerized voting system integrity. The Draft calls for such a record, in the form of a paper record. However, we feel the need to point out that VVPRs are not terribly useful unless audits are conducted using these records to provide regular checks on the correct functioning of the voting system.
• The usability and accessibility requirements need work. ACCURATE noted that there are no accessibility requirements in the Draft and the usability requirements seem hastily assembled from a previous standards effort. In our comments, we discuss how attention to usability and accessibility is key during the development stages of new technology and go on to recommend that some additional usability testing and requirements be added to the draft.
• There have been significant improvements in security specification and testing. The Draft does a good job at improving upon some of the security specifications and testing that we have seen in the past. We are encouraged to see threat modeling and penetration testing adopted in the draft requirements and we recommend a few changes that would make them even stronger.

The EAC made this manual available for a 15-day public comment period that ended today and we submitted comments (In 2006, ACCURATE submitted public comments on the original manual for the larger testing and certification program).

From our comment submitted today:

The Draft Manual does an admirable job of incorporating some of the features of a feedback-rich pilot testing process, but we believe that it can and should go further. Our recommendations fall into four categories. First, the EAC should amend the Draft Manual to provide more details about what separates pilot certification from certification under the current, VVSG-based certification program. Specifically, the EAC should clarify what qualifies as a voting system pilot program, how it will decide whether to allow a manufacturer to pursue pilot certification for a given system, and what conditions are attached to pilot certification. Second, the pilot certification program should accept feedback from, and establish a systematic process for responding to, voters. Third, the EAC should strengthen the Draft Manual’s provisions for engaging with manufacturers at the system design stage and feeding data from pilot elections back to the design stage. Finally, the EAC should address the question of balance between piloting relatively mature systems and permitting pilots to force potentially major changes in pilot system design. This involves questions of the time and expense involved in pilot certification.

Our comment goes into detail about what we think could be improved in the VSPPTC Manual and how the unique nature of pilot voting systems provide opportunities and pose risks different from more mature voting technology.

The due date is April 16, 2010, 11:59 p.m. PDT… send in your best work!

Cryptographer Ben Adida, who is unaffiliated with the Scantegrity project or any other party in the election, has agreed to act as an independent auditor of the election. Working from nothing but the public specifications of how the system works, he’s independently verifying that the results are correct.

It’s important to note that, for this particular election technology, the votes are being cast on traditional paper ballots that could always be counted, recounted, or otherwise inspected manually. That’s not strictly necessary for election security — our own VoteBox system works more like a paperless electronic voting system and has the same security guarantees as Scantegrity — but it’s essential when rolling out a new technology where a real election with real politicians’ careers is at stake. We need to know that real elections can be really verified, and we need a fallback position if the crypto somehow goes wrong.

Of course, for these technologies to truly get out of the lab and into the field, we can’t expect Ben Adida to personally verify every election, worldwide, nor should we trust him to. What we can expect is that tools that Adida and others like him build will be picked up and used by local election watchers, party officials, news outlets, and the like. We’re not there yet, but we’re on our way.

(Note: Truly, the first ever binding e2e election was a web-based election for the president of a Belgian university, based on Adida’s Helios system (full paper). This used similar cryptographic mechanisms, but no web-based election system can ever have the coercion resistance or privacy guarantees of voting in a classical voting booth.

Edit: The University of Ottawa Graduate Students Association had a binding e2e election in 2007 using PunchScan, a predecessor to Scantegrity.)

ACCURATE’s comments criticize the new draft v1.1 for claiming to include only incremental, modest changes when, in fact, many of the requirements would necessitate substantial revisions and re-engineering of voting systems. We argue that the VVSG II, the previous more-ambitious overhaul of the VVSG, is a better backdrop upon which to require substantial revisions of voting systems than the draft VVSG v1.1.

Most notably omitted is any requirement for software independence, which would require systems to be designed so that undetected flaws in the voting system software could not cause undetectable changes in the vote count. In fact, many of the more onerous, detailed requirements in the VVSG v1.1 bear the full burden, now, of security assurance, when requiring software independent system architectures would bypass the need to rely on, for example, detailed coding requirements. ACCURATE fully supports requiring software independence as the backbone of a robust and comprehensive next-generation voting system certification regime, and we were disappointed to see no evidence of it in the VVSG v1.1. The commentary goes on to emphasize that most of the changes relevant for cryptography, structured vote data and security are not nearly as powerful as they would be in a regime that required software independence.

ACCURATE is more optimistic about the benefits from the testing-related changes, such as those proposed changes to the accuracy and reliability testing framework, the usability and accessibility requirements (and the requirement that manufacturer’s must conduct usability testing) and the required voting system documentation. However, there is a conspicuous absence of volume testing and adversarial vulnerability testing, two types of testing that have shown great promise in state-based testing efforts. Finally, the novel usability benchmarks and benchmark testing are also missing, despite evidence that the general usability and pollworker documentation benchmarks are close to finished if not actually finished.

ACCURATE welcomes the opportunity to participate further as the draft VVSG is modified and readied for adoption.

Download the ACCURATE comments: PDF

If you would like to demonstrate a system, please email the following details to the EVT/WOTE chairs (evtwote09chairs@usenix.org):

1. Name of the system
2. Short abstract
3. Demo requirements (e.g., space, power, network, etc.)

There is no hard deadline for proposals, however we would appreciate hearing from you as soon as possible so that we know how much space we’ll need (if we cannot accommodate all the demos, earlier proposals will receive preference).

Best wishes, the EVT/WOTE Chairs:

David Jefferson
Lawrence Livermore National Laboratory
d_jefferson@yahoo.com

Joseph Lorenzo Hall
UC Berkeley/Princeton
joehall@berkeley.edu

Tal Moran
Harvard University
talm@seas.harvard.edu

This year, the organizers of the USENIX/ACCURATE Electronic Voting Technology Workshop (EVT) have merged EVT with the IAVoSS Workshop on Trustworthy Elections (WOTE) to create a joint two-day workshop (EVT/WOTE ’09). EVT/WOTE seeks to bring together researchers from a variety of disciplines, ranging from computer science and human-computer interaction experts through political scientists, legal experts, election administrators, and voting equipment vendors.

The program features a keynote address by Lawrence Norden, Brennan Center for Justice at NYU School of Law; system demonstrations; and sessions on usability, security, trustworthy elections, forensics, and more.

The full program can be found at http://www.usenix.org/events/evtwote09/tech/

Don’t miss out on this opportunity to engage in dynamic discussion on key topics in the e-voting community. Register today at http://www.usenix.org/evtwote09/proga

EVT/WOTE ’09 will be co-located with the 18th USENIX Security Symposium (USENIX Security ’09), August 10-14, 2009: http://www.usenix.org/sec09

We look forward to seeing you in Montreal!

David Jefferson, Lawrence Livermore National Laboratory

Joseph Lorenzo Hall, University of California, Berkeley/Princeton University

Tal Moran, Harvard University EVT/WOTE ’09 Program Chairs

evtwote09chairs@usenix.org

This year, the USENIX/ACCURATE Electronic Voting Technology (EVT) workshop and the IAVoSS Workshop on Trustworthy Elections (WOTE) have merged into a single two-day workshop, co-located with USENIX Security ’09 in Montreal. The combined Workshop will feature distinct EVT and WOTE sessions; accepted papers will appear in a joint Proceedings (grouped by session). Please see the EVT/WOTE ’09 page for more information.

In the news, we’re seeing a number of reports concerning “vote flipping.” The story, as typically reported, is that voters are attempting to vote for one candidate but observe that the machine “flipped” their vote to the other candidate. For touch-screen voting machines, the most likely cause of this issue is miscalibration of the screens (or, perhaps, a voter who is significantly taller or shorter than the person who did the calibration, since different angles of view require different calibrations). I wrote a detailed explanation of this issue two years ago.

UPDATE: Barbara Ballard, a usability expert with Little Spring Designs offers some excellent advice about how to configure touch screen button layouts to minimize or eliminate the parallax issues that seem to induce or exacerbate vote flipping.

UPDATE 2: Matt Blaze writes how miscalibration of touch-screen voting machines could be used as a mechanism to disenfranchise voters.

A related issue concerns reports of vote flipping on the Hart InterCivic eSlate voting machine. These machines do not have touch-sensitive screens, so therefore poor calibration cannot explain the voter confusion. Since my home county uses eSlates, I went to vote early, this morning, and paid careful attention to how the user interface works. For those unfamiliar with eSlates, the voter’s primary interface to the machine is a dial-wheel and an “Enter” button, which operates in a manner that would be quite familiar to users of Apple’s iPod. You turn the wheel and it highlights successive entries. You press the Enter button and it indicates your selection graphically. Using some HTML tables, I’ve attempted to recreate the salient details below.

First, here’s an approximation of what a normal voter will see when they start off. Note that I’ve simplified a few things, and I’ve replaced the actual instructions with some fake text. It’s sadly safe to assume that most voters won’t bother to read the instructions, anyway. For purposes of explaining the user interface, I’ve left out our Libertarian candidate for U.S. Senate, Yvonne Adams Schick, and any write-in boxes. My apologies, but this is already pretty complicated. (Also, I didn’t debug this against every web browser. If you’re using Firefox or Safari, you should see everything as intended. Hart also has an online Flash demo, slide show, and videos that broadly show how an eSlate works, although they don’t include straight-ticket voting.)

The blue background highlighting tells you where you are, and when you start, you’re on the instructions. As you turn the wheel, it highlights “straight ticket” and then “Republican party” and so forth. Let’s say our voter wants to vote straight-ticket Libertarian. They would need to scroll until they saw this:

Then our voter would hit the “Enter” button and would next see this:

Several things just changed. Every race with a Libertarian candidate now has that candidate selected with a dark red box, and the other candidates have their visual contrast reduced. Also, the blue highlight block has now advanced to the header for the presidential race. Also of note, if there’s a race without a Libertarian candidate, such as my simplified U.S. Senate race, those candidates will still have the full black&white contrast. At this point, the voter would most commonly scroll past the presidential race and indicate preferences in other races. If the voter decides to “split the ticket” and change their presidential preference, perhaps by selecting John McCain, the whole screen goes blue and displays a message in large, white letters indicating that they just split their ticket. (“Press Enter to Continue.”)

Okay, now that we have a basic understanding of the eSlate UI, how could this be causing voter confusion? The Houston Chronicle reported:

In response to reports from other locations that electronic voting machines were reversing votes cast in the presidential election, [County Clerk Beverly Kaufman] said, “We tested those aspects of the system in a formal way with party officials and that did not occur. I still cannot urge people enough to check your work and keep your focus on what you’re doing.”

Her office was informed early today that some of the first voters had cast straight-ticket Democratic ballots and then discovered that the electronic machines listed them as voting for John McCain in the presidential election.

Hector de Leon, spokesman for the County Clerk’s Office, said problems such as those reported with the identification scanning system likely reflected poll workers’ inexperience in operating the new equipment.

Ahh, inexperience. Let’s try to sort out what might have happened. Let’s assume that some of the voters in question didn’t read the instructions. Let’s further assume that these voters are not regular computer users, so the visual cues to indicate choice selection (the dark red square, the low-contrast colors for non-selected candidates) do not speak to their experience. After our example voter cast a straight-ticket Libertarian vote, they might not understand that they had selected Barr already and would miss the graphical cues that confirm this. Instead, they would scroll down to Barr and press the Enter button. This will generate the full-screen warning message, deselect Barr, and leave the blue highlight on Barr for the voter to make a subsequent choice, including re-selecting him. This sort of confusion is entirely feasible on an eSlate.

Two years ago, we heard concerns about voters “bouncing” on the Enter button, and problems which may have resulted, so I paid careful attention this time. After the voter makes a selection, the highlighted area advances to the subsequent race title (“President”, “U.S. Senate”, etc.), which cannot actually be selected. This, at least, gives us some confidence that a voter who accidentally presses the Enter button multiple time will not accidentally indicate multiple selections in a series of races.

As an aside, some eSlate units support a “Disability Access Unit,” adding headphones that work in a relatively intuitive fashion. The voting machine reads out whatever text is highlighted. In this way, a visually impaired voter will always know “where they are,” and navigation is strictly one-dimensional, going forward or backward from one area to the next. It’s reasonable to recommend that voters with perfectly good vision but who have limited computer experience also use the headphones. It may well help reinforce the navigation and selection model of the eSlate and reduce voter concerns about “vote flipping.”

Of course, we could also implore voters to pay attention to the summary screen. For my ballot, which I cast this morning, the summary screen was actually three separate pages listing my selections. Races in which I abstained to offer a selection were highlighted in red text. Will this help voters detect mistakes? It will for some voters, but many voters won’t carefully read their summary screen. At Rice, we did some experiments on this, building a voting machine that introduced deliberate errors into the summary screen. Would voters notice? Unfortunately, our research shows that as many as 63% of voters fail to notice errors on the summary screen.

In the future, there are number of ways Hart InterCivic could improve this system. They could replace the dark red square, as an indication of a voter’s selection, with a symbol that is clearer to non-computer users, perhaps a large, green check-mark. The EAC’s Effective Election Design document has a number of good recommendations along these lines.

Likewise, many voters will simply not understand the concept of straight ticket voting. We encourage states that require straight ticket voting, such as Texas, to eliminate the practice altogether. This would be particularly helpful in a state like North Carolina where, by law, the straight ticket voting option affects every race except the presidential race. That’s not at all intuitive.

In conclusion, while we have no reason to believe that voting machines are actually flipping votes, the issues underlying these concerns are very real. We should not blame voters when the real problem lies with poor usability engineering of electronic voting machines, whether that means calibration and angle-of-view issues on touch-screens, or non-intuitive navigation and selection models on eSlates.

[The author thanks Mike Byrne, Marti Hearst, Doug Jones, Sharon Laskowski, Peter Neumann, Whitney Quesenbery, Dan Sandler, Pamela Smith, and Dave Wagner for their comments on drafts of this article.]