RE: Riverside Voting Machine Verification Process Lacking

["Paul Tiger" <tigerp@xxxxxxxxxxxx>]

An interesting observation, but all of the systems that I am familiar with
provide a 'test deck" to be run before and after elections. This is to be
done after the county has programmed the DREs, so we know that we are
testing for real issues and candidates and not some faked stuff.

A few of the vendors showed tests that could be run during an open poll.
Meaning that the veracity of a DRE could be tested during the course of the
balloting, not just before or after.

However, also as a systems engineer (18 years), I think that I can safely
say that a test mode can easily be differentiated from a working mode.
Therefore, I have serious doubts about what is being tested. Simple
diagnostics to test which lights blink, switches are toggled, or gates
cycled has little to do with real-time operations.
For me a test would be done in elections mode and the ballot serial number
could be manually recorded and set aside by the operators at the time of
totalization.

Paul Tiger

-----Original Message-----
From: Christian Rudolph [mailto:reindeer@xxxxxxxxxx]
Sent: Tuesday, October 21, 2003 1:07 AM
To: Moon Lee; Boulder County Voting Email List
Subject: Re: Riverside Voting Machine Verification Process Lacking

Moon quoted Akin when he wrote:

"The only thing that I've heard about this is that they needed the
pre-election mode to make sure that the test didn't leave any votes in
the system that would be counted during the election.
And this seems  really weird to me ...
I can see why a company might have a testing mode,
but I do not understand why a company wouldn't test something in
production mode. And you can ask any QA person about this and they're
going to tell you the same thing, that it's not a thorough test if it's
not tested in production mode."

http://www.salon.com/tech/feature/2003/10/15/riverside_voting_machines/print
.html
End quote.


Moon Lee, et al.,
In my twenty-three years of experience, eight as a software quality
assurance engineer, I can tell you that if no testing is done in
production mode, then this system has NOT BEEN TESTED.
If Akin's claim is accurate, then Sequoia should not be selected.

-Christian Rudolph









On Wed, 2003-10-15 at 13:09, Moon Lee wrote:
> http://www.salon.com/tech/feature/2003/10/15/riverside_voting_machines/
> print.html
>
> Bad grades for a voting-machine exam
>
> Riverside County, Calif., invited citizens to observe a test of its
> computerized voting systems. One participant was not impressed.
>
> By Farhad Manjoo
>
> Oct. 15, 2003  |             Over the past several years, as computer
> scientists began expressing concerns about the security of touch-screen
> electronic voting machines, elections officials across the nation have
> reassured the public with a simple answer: Testing.
>
> Elections officials maintain that before they are ever used in an
> election, electronic voting systems are put through a battery of tests,
> the culmination of which is the "logic and accuracy" test that counties
> perform a few weeks or days prior to an election. This examination is
> billed as a simple, straightforward way of telling whether a machine is
> working as it should. A predetermined number of ballots are fed into
> the machine, and then the votes are tabulated. If the system spits out
> the results you expect, the system is deemed fully functional.
>
> But Jeremiah Akin, a 28-year-old computer programmer who recently
> observed one of these tests in Riverside County, Calif., says that what
> he saw did nothing to mitigate his concerns about electronic voting --
> indeed, the whole thing made him more worried than ever.
>
> Akin, who observed the test as a representative of the Peace and
> Freedom Party, says that representatives of other parties who were
> there signed off on the test without waiting to see the complete
> results. In fact, he says, nobody else seemed concerned that anything
> could go wrong with touch-screen machines. In a 22-page report Akin
> wrote recording his observations of the test, he says that "statements
> made by the Registrar of Voters indicated to me that she is not
> qualified to assess the reliability and security of such systems, and
> that she misunderstands some essentials of computer programming and
> operation. Her deputies refused to answer some important questions.
> Some statements made by officials at the Registrar's office, and found
> on the contractor's Web site, I learned on the test day were misleading
> or inaccurate. Further research after the test day has turned up
> several other reasons to doubt the reliability, security and accuracy
> of the system."
>
> Riverside County was one of the first places in the nation to employ
> touch-screen machines -- the county used them in the 2000 election,
> before anyone had ever heard of the problems with older, punch-card
> machines. After that race, Mischelle Townsend, the county's registrar
> of voters, was celebrated in the national media, held up as a visionary
> who'd seen the promise of voting with computers.
>
> Since then, however, some of the world's most respected computer
> scientists have highlighted serious problems with electronic voting
> machines. In July, scientists at Johns Hopkins and Rice found alarming
> security holes in voting machines made by Diebold, which provides
> election systems in 37 states. On its Web site, Sequoia Voting Systems,
> which makes the machines used in Riverside County, insists that its
> machines are safer than Diebold's. But technologists say that because
> Sequoia's systems don't produce a voter-verifiable paper trail -- some
> physical evidence that the voter's choice has been accurately rendered
> -- they're no better than most of the other electronic machines on the
> market.
>
> Now that such machines are under fire, Akin says that Townsend went out
> of her way to defend electronic voting during the logic-and-accuracy
> test, which was held on Sept. 9, in anticipation of California's Oct. 7
> gubernatorial recall race.
>
> The story Akin tells of that test indicate serious shortcomings with
> the machines as well as the process used to verify them. He spoke to
> Salon on Tuesday from Riverside.
>
>  >> From what you write about what happened at this test, it seems that
> the elections officials were trying to reassure everyone about
> electronic voting machines. Was this whole exercise to prove that these
> machines work? <<
>
> Well, before an election there's a legal requirement to run
> logic-and-accuracy testing, so that's what the purpose of the test was,
> and we were brought in to observe that. But it did have more of the
> feel of a sales pitch than of a test.
>
>  >> You write that the elections officials were specifically addressing
> some of the concerns that people have with these machines. <<
>
> Yes, they were, and before and after the meeting Mischelle was going
> after people who don't support these types of machines, especially Bev
> Harris and also the computer science community. She was saying computer
> scientists don't know about how elections are run -- she didn't name
> any specific computer scientists. But just lumping everybody together
> and labeling them as ignorant isn't very convincing.
>
>  >> Can you tell me about the other people who were there? You say that
> it seemed to you that most of the people who were observing didn't
> understand or didn't have the same questions you did about the
> technology. <<
>
> Yeah, that's true. There was one who said, "This is like 'Star Trek'!"
> He was talking about how one day voting over the Internet would be
> possible. He really was not technically proficient. He didn't
> understand some of the limitations of working with computers. People
> who work with computers a lot know that they have bugs and know that
> they crash and there's no real way to get all the software bugs out of
> a complex system. These people didn't have any experience with that and
> so they could be easily convinced otherwise.
>
> There was somebody from the Libertarian party who was there, and
> somebody from the Republican Party who showed up late and played with
> his phone a lot. So I don't know how technically proficient he was.
>
>  >> Describe to me how the test was run. <<
>
> Well, I was picturing that people would go up and touch the touch
> screen and verify that what they had pressed was registered as a vote.
> But the way it's run is, they have a test cartridge that they pop into
> the back of the machine, and it runs a script -- it runs several
> hundred different voters, like some type of emulation.
>
>  >> Sort of a simulation of what would happen during a day of voting. <<
>
> Yeah. But the touch screens themselves weren't actually pressed. Nobody
> got to touch those. So we didn't see what was on them, and we didn't
> see the input that was put into the machine. All that we saw was the
> output that came out later. And, I mean -- that's like telling somebody
> that your calculator can add 2 plus 2, then pressing some buttons
> behind a screen, and then showing them that it says 4.
>
> The votes were put in and they started to run, and at that point we
> were told that it would take several hours ... [A couple of hours
> later] we went back to see if the test scripts had finished running,
> but they hadn't. So at that point we broke for lunch and we were going
> to come back to get the results.
>
> I saw several people standing with Mischelle Townsend, and they were
> signing this piece of paper. At that point, I didn't know what was on
> the piece of paper; otherwise I would have said something right then
> and there. I thought it was a type of roll sheet. [Actually, the sheet
> certified that the observers had seen the test and believed that
> everything looked aboveboard. The form, which Akin refused to sign,
> said: "We the undersigned declare that we observed the process of logic
> and accuracy testing of voting equipment performed by the Riverside
> County Registrar of Voters, as required by law and that all tests
> performed resulted in accurate voting of all units tested, including
> both touch screen and absentee systems."]
>
>  >> Mischelle Townsend told Salon that she disagreed with you about
> people signing off on these tests before they'd seen the results. She
> said that people saw the results and they had hard copies of the
> results, and they signed off on it after that. <<
>
> I saw people signing this paper before that.
>
>  >> Townsend also said that the document was not any kind of "rigorous
> legal" form; it's just an official roster to say you were there. <<
>
> But the document doesn't say it's a roll call. It says that the people
> watched the entire test and observed the results.
>
> So at that point I left, went home for lunch, and came back around
> 2:30. Nobody else was there. Brian Foss [the county's information
> technology manager] and I went to the machines to take the cartridges
> out -- but we found that they had already been taken out and the
> machines had been sealed shut.
>
>  >> OK, so you went to the machines, and the cartridges that store the
> ballots weren't there in the machines? <<
>
> Yeah. And Brian Foss asked the guy from Sequoia who was there what
> happened to the cartridges, and he said that they'd been pulled out.
>
> [Eventually], Brian Foss pulled up one -- but I'm not sure if it was
> part of the test or not. He went over to the software that tallies the
> votes. He puts the card into the card-reading slot and starts up the
> software, which I think is called WinEDS. And I immediately notice it's
> running on Windows XP. This caught me off guard -- I'm like, hold on a
> second, because throughout the day we were told by Brian Foss and
> Mischelle Townsend that Sequoia Voting Systems did not use Microsoft
> software.
>
> But it turns out that if you go to Sequoia's Web site they're very
> careful about how they phrase it: They say that the Sequoia voting
> kiosks don't use Microsoft Windows but the tallying machine does. But
> the way Mischelle Townsend and Brian Foss said it, they said the whole
> thing didn't use Microsoft Windows. So I asked Brian Foss how come he
> didn't mention that the software uses Microsoft Windows, when earlier
> in the day they talked about how systems based on Microsoft Windows are
> not secure.
>
> And he didn't answer me. I asked him several times, and each time I
> asked him I tried to make eye contact with him and he wouldn't make eye
> contact with me.
>
> Now, since the WinEDS program runs on top of Microsoft, there's room
> for -- well, basically it calls some Microsoft APIs [application
> programming interfaces], and if those are modified in any way a
> modification of the system could happen that wouldn't be detectable in
> the type of code review and security tests that Sequoia's software is
> subject to.
>
>  >> So you mean people could just make changes to the computer on which
> the voting software is running -- and the Sequoia system could be
> altered in some way? <<
>
> Yes ... Well, so he printed out the results from one card that he'd put
> in there, and he also printed out results from four other cartridges
> that he said had been part of the test, but I don't even know where
> they came from.
>
>  >> He gave you this printout of what came out of the machines, but did
> he show you what was on the script that was run on the machines? So you
> could sort of compare what votes were put in to what came out? <<
>
> No, at the beginning we were given sheets of paper and we were told,
> this is what the script is going to test for. But we have no idea what
> was on the card that they stuck in.
>
> Also, this is very important -- I noticed that the vote-tallying
> software has three different modes. A pre-election mode, an election
> mode, and what they call the post-election "verification" mode. These
> tests were only run in the pre-election mode. And to a programmer who's
> had software go through QA testing that seems really weird, because the
> software wasn't tested in production mode.
>
> There are times when you need a testing mode for software. For example,
> if you're doing credit-card-processing software you need a testing
> mode, because you don't want to constantly use your actual card to test
> the system. But testing always goes through a production test, too [in
> which real cards are used].
>
>  >> Is there any indication to you what the difference is between the
> different modes? <<
>
> The only thing that I've heard about this is that they needed the
> pre-election mode to make sure that the test didn't leave any votes in
> the system that would be counted during the election. And this seems
> really weird to me ... I can see why a company might have a testing
> mode, but I do not understand why a company wouldn't test something in
> production mode. And you can ask any QA person about this and they're
> going to tell you the same thing, that it's not a thorough test if it's
> not tested in production mode.
>
>  >> I also wanted to talk to you about this paper-trail issue. The
> elections officials told you that the machines they used produced a
> paper trail, but they didn't mean the same thing that the critics mean
> when they ask for a paper trail.  <<
>
> Yeah. The way Mischelle said, it was that the computer science
> community asked for paper trails but they don't understand that there
> already are paper trails. But what people in the computer science
> community asked for was a voter-verified paper trail. What's demanded
> is a piece of physical evidence that the voter verifies after the vote.
> [The Sequoia machines used in Riverside print out a record of all the
> votes cast on each machine during an election. This "paper trail"
> doesn't address concerns that the machines might incorrectly record the
> votes in the first place.]
>
> Mischelle also stated that there's no real reason for a voter-verified
> paper trail. She said that the paper trail wasn't possible because
> printers would jam up during an election. The woman from the
> Libertarian party said that she gets receipts all the time when she
> goes to stores so she didn't understand why they couldn't make a
> printer that would work reliably. Then Mischelle said there's no reason
> to make a paper trail because it would do exactly what the machines are
> doing. It would be wasted effort. Which is a kind of a silly argument.
> There's a bunch of different reasons why you need a paper trail.
>
>  >> Did you vote in Riverside County on Oct. 7? <<
>
> Yeah, I voted absentee. Because even though no system is perfect at
> least absentee has a paper trail.
>
>  >> Do you have any thoughts on how the election went that day? <<
>
> Well, apparently there was a pretty large margin, so I don't think
> people are going to pay attention to it. [In Riverside County 70
> percent of the voters were in favor of recalling Gray Davis, and 61
> percent voted for Arnold Schwarzenegger.]
>
>  >> But if something bad happened, people might not know about it? <<
>
> Yeah, exactly: Without a paper trail there's no way to know that. They
> say there's no known instance of fraud, but they make it so you can't
> tell if there's an instance of fraud, so that claim doesn't say very
> much.
>
>  >> Mischelle Townsend told Salon that you were "a young man who had a
> chip on his shoulder when he came in here." She said that you came into
> the test with a "closed-minded" attitude and that you didn't want to
> "listen to the facts." <<
>
> This is exactly what I expect from her. Instead of responding to my
> arguments she'll just try to paint me in a bad light. I'm sure she
> didn't say anything about the fact that the voting kiosks rely on a
> Windows operating system in order for the results to be read, even
> though Sequoia and Mischelle Townsend said that one of the benefits of
> their system is that the results don't rely on Windows.
>
> I fully expect Mischelle to not really address anything I say in my
> report. I expect her to say I have a chip on my shoulder or I don't
> listen to facts. I've never seen her deal with anyone who's technically
> literate in any other way.
>