Harvie,
Pasted below is a copy of the note I sent
you re “test mode”
The key is that
the LAT statutes and rules do not currently specify against the use of
test mode, although the certification rules do specify against test
mode. –
see 45.6.2.3.2.
45.6.2.3.2 All tests shall be conducted as described in this
section 45.6.2.3 in regular election mode. At no point shall testing be
conducted in any form of test mode
I have many years of professional experience
testing complex systems including operating systems and banking systems.
RE the challenge – has somebody
verified that this is not a hoax?
Al
From: Al Kolwicz [mailto:alkolwicz@xxxxxxxxx]
Sent: Wednesday, October 04, 2006
10:09 AM
To: 'Harvie Branscomb'; 'neal
McBurnett'; Dr. Charles Corry (ccorry@xxxxxxxx)
Cc: 'ken@xxxxxxxxxxxxx'; 'Teak
Simonton'; 'helen Lindow'
Subject: RE: spare memory cards;
election vs. test modes
Harvie,
We believe that the LAT
should be conducted in live mode. This means to us, (1) no “test mode”
type switch or indicator, (2) live election definition, not a test definition,
and (3) live election database for collecting votes and audit logs.
Before running the test, the database must be securely
“zeroed” After running the test, a copy of the database
should be made and secured, and the database again zeroed.
Following the election, a
copy of the election database should be made and secured, and the database
zeroed for the repeat of the test.
Test election definitions
can (accidentally) be different then the production definition, and cannot be
trusted to represent the live data. Even a one-bit difference matters.
Test mode indicators
cause the software to follow a different path than the live election, and
cannot be trusted.
Test databases can
(accidentally) be different from the production, and cannot be trusted.
The
rule for certification tests seems to recognize this
– see 45.6.2.3.2.
45.6.2.3.2 All tests shall be conducted as described in this
section 45.6.2.3 in regular election mode. At no point shall testing be
conducted in any form of test mode
The statutes
direct the SOS to establish rules, but they are not adequate.
1-7-509.(6) The secretary of state
shall promulgate rules in accordance with article 4 of title 24, C.R.S.,
prescribing the manner of performing the logic and accuracy
testing required by this section.
The rule is vague
and does not really test anything of consequence.
11.5.3
Logic and Accuracy Test
The designated election
official shall conduct a Logic and Accuracy Test according to the following
requirements.
11.5.3.1
The designated election official shall create a Testing Board consisting of at
least two persons, one from each major political party.
11.5.3.2
Prior to the commencement of voting, the designated election official shall
conduct the public Logic and Accuracy Test.
11.5.3.3
The Logic and Accuracy test shall be open to representatives of the press and
the public to the extent allowable and pursuant to section 1-7-509(2)(b),
C.R.S. The designated election official may limit the number of representatives
from each group to accommodate for space limitations and other considerations.
11.5.3.4
Testing Board Test Ballots – In preparation for the Logic and Accuracy
Test, the designated election official shall provide to each member of the
Testing
24
Board, at least twenty-five (25) ballots that are clearly
marked as test ballots to be used for the Logic and Accuracy Test.
11.5.3.5 The members of the Testing Board shall secretly vote
their position and retain a record of the tally of their test votes. The test
ballots shall have a known predetermined outcome by the members of the Testing
Board’s secret vote and tally. Of the twenty-five test ballots, two shall
be tested as audio ballots where applicable.
11.5.3.6 County Test-Ballots – In preparation for the
Logic and Accuracy Test, the designated election official shall prepare a
sufficient number of test ballots that represent every precinct which shall
include every ballot style, allow for a
sufficient number of ballots to mark every vote position for every candidate on
every race including write-in candidates, allow for situations where a race may
permit an elector to vote for two or more positions, and include overvotes and
undervotes for each race.
11.5.3.7 The test ballots shall be tested on each type of
voting device utilized in a given election and each method of counting. The
tests shall include testing of absentee counting methods, election day counting
methods, provisional ballot counting methods, early voting counting methods and
audio ballots, if applicable.
11.5.3.8 Conducting the Test
11.5.3.8.1 The designated election official and Testing Board
shall observe the tabulation of all test ballots by means of the voting device
and compare the tabulation with the previously retained records of the test
vote count. The cause of any discrepancies shall be corrected prior to the
start of vote tabulation.
11.5.3.8.2 Prior to the start of testing, all devices used
will have the public counter reset to zero, and presented to the testing board
for verification.
11.5.3.8.3 An appropriate number of voting devices will be
available and the testing board may witness the necessary programming and/or downloading
of memory devices necessary to test the specific precincts.
11.5.3.8.4 The Testing Board and designated election official
or his or her designated deputized clerks, as necessary, shall count the test
ballots as follows:
(a) Absentee Ballots:
(1) All county test ballots shall be counted on at least one,
but not more than three, absentee vote counting devices and have the
predetermined total verified to the machine total.
25
(2) All Testing Board Member test ballots shall be counted individually
with reports generated to verify the machine count to the predetermined hand
tally.
(b) Precinct Count Ballots (Optical Scan and DRE):
(1) The Testing Board shall randomly select 20% but not more
than 10 ballots representing unique precincts from the Testing Board’s
test ballots.
(2) In the event a selected precinct contains a combination
of DRE and Optical Scan voting devices, the Testing Board shall decide on the
percentage of ballots to be counted on each type of device used for that precinct.
(3) The precinct specific county test ballots will be added
to the testing board test ballots to be counted on the specific precinct
device. The testing board shall manually verify the ballots to be counted prior
to any machine count.
(4) The Testing Board shall verify the manual count to the
voting device count.
(c) Vote Center
Count Ballots – Optical Scan:
(1) All testing board test ballots shall be counted on at
least one, but not more than 5 voting devices designated for Vote Center
Counting and have the predetermined total verified to the machine total.
(2) All test ballots shall be counted individually with
reports generated to verify the machine count to the predetermined tally of the
test ballots.
(3) The testing board shall randomly select the machines to
be tested.
(d) Vote Center
Count Ballots – DREs:
(1) All testing board test ballots shall be counted on at
least one, but not more than 5 DREs designated for Vote Center Counting and
have the predetermined total verified to the machine total.
(2) All test ballots shall be counted individually with
reports generated to verify the machine count to the predetermined tally of the
test ballots.
(3) The testing board shall randomly select the machines to
be tested.
26
(e) Early Voting and Provisional Ballots Counted on Optical
Scan Devices:
(1) All test ballots shall be counted on at least one, but
not more than five, optical scan devices designated for Early Voting or
Provisional Ballot Counting and have the predetermined total verified to the
machine total.
(2) All test ballots shall be counted individually with
reports generated to verify the machine count to the predetermined tally of the
test ballots.
(f) Early Voting and Provisional Ballots Counted on DREs:
(1) All test ballots shall be counted on at least one, but
not more than five, DREs designated for Early Voting or Provisional Ballot
Counting and have the predetermined total verified to the machine total.
(2) All Testing Board Member test ballots shall be counted
individually with reports generated to verify the machine count to the
predetermined tally of the Testing Board test ballots.
11.5.3.8.5 DREs equipped with V-VPAT devices shall be
manually verified (by hand) to determine that the pre-determined total of the
testing board ballots, matches the V-VPAT total, which in turn matches the
machine total.
11.5.3.8.6 At least two of the testing board ballots shall be
identified as Audio Ballots to be tested as such, and included with the count.
11.5.3.8.7 All test materials, when not in use, shall be kept
in a metal box with individual seals for each member of the Testing Board. The
designated election official may affix his or her own seal in addition to those
of the Testing Board. The designated election official shall be the custodian
of the box or boxes but shall not open and/or use the test materials outside of
the presence of the Testing Board.
11.5.3.8.8 The Testing Board and the designated election
official shall sign a written statement attesting to the qualification of each
device that was successfully tested, the number of the seal attached to the
voting device at the end of the test, any problems discovered, and provide any
other documentation as necessary to provide a full and accurate account of the
condition of a given device.
11.5.3.8.9 Upon completion of the testing, the Testing Board
shall witness the resetting and sealing of each tested voting device.
From: Harvie Branscomb
[mailto:harvie@xxxxxxxxxxxxx]
Sent: Friday, October 06, 2006
8:44 AM
To: Al
Kolwicz; neal McBurnett
Cc: ken@xxxxxxxxxxxxx; Tom Morris; Carolyn Bninski; Geof Cahoon; Ivan C.
Meek; Joe Pezzillo; Kellen Carey; Margit Johansson; Mary Eberle; Myriah Conway; Neal McBurnett; Paul Walmsley; Peter Richards; Ralph
Shnelvar; Scott A. Morris; Stith Bennett;
Claudia Kuhns; cvv-discuss@xxxxxxxxxxxxxxxxx; cfvi@xxxxxxx
Subject: spare memory cards;
election vs. test modes
Al and Neal and other
election evangelists:
I did not receive any comments on
this important email- does anyone have any?
I need some backup to make this important argument to the Clerk in Eagle County.
(Green and black are
from harvie, blue
is from Teak Simonton)
FYI this morning I found at least one "white hat
hacker" to take on the recent challenge by Jefferson
County to try to crack the security on their iVotronics machinery. There is an
association of "white hat hackers" called the Hackers
Conference. "White hat hackers" are not dangerous like
"black hat" but probably as close as we can get in
"laboratory" instead of field conditions.
Today you may be able to help me get a handle on how to clarify this discussion
I am having regarding choice of memory cards and method of programming for the
LAT and election with Teak, the Clerk of Eagle County.
I am looking for your comments.
This discussion of test mode vs election mode is crucial to testing
effectiveness and I think a requirement not to use test mode is either in the
statute or in the rules or both. Do either of you know the exact location
of this?
Al, I did not get a reply from you
on this... did you find the provision? I have also asked Ken Gordon -
mainly to see if he is paying attention.
Then there is the point of whether the individual memory cards
get tested. So far they
have never all been tested and apparently are not required to be tested
individually. They are swapped during the election without concern for their
individual condition. Is this a safe assumption? Probably not since
they are flash memory and flash memory actually fails over time. I am not aware
of a test for the performance of the flash memory cards. In future rules
there should be a provision for this testing.
Teak says that it is the contents of the memory card which gets tested.
If so, then that same
tested contents in any memory card will have been considered to have been
tested, presumably. I think this is the status quo.
And therefore it would not be a problem to test a subset of identical memory
cards while saving others for the election.
All memory cards for test and
election could be programmed in the same operation to be sure they are
identically programmed. This seems likely to the best procedure insuring that
we are testing under election conditions (ideally the date would also be
programmed for the election date during the test).
It is the manufacturer who poses the extra levels of security
which impose the difference between election and test mode, such that in
election mode the cards may be used only once, and so forth. This is not a reason to use test mode for
the test.
In my observations, the Clerk is frequently required to
work-around this security restriction to accomplish the election by either
reprogramming the cards or using pre-programmed spares, therefore the security
measure of making the cards work "only in the election" and
"only once in the election" is actually moot in common practice.
On the other hand the added security of separating a special "test
mode" which is more flexible actually increases the potential for fraud,
substantially, by injuring the effectiveness of the tests if they are being
performed in "test mode" rather than under the conditions of
"election". This is, simply put, because all of the bugs in the
software, and intentional fraud, if any, will therefore be found only in
"election mode" which is
unfortunately not tested.
In view of the ease and nonchalance and frequency with which
memory cards are supplemented with spares during the election, it seems
to me that there is no point in giving any attention to the "security
measures" which would have prevented them from being supplemented and
swapped or replaced (because the
work-around defeats this security).
Instead the cards should be kept under effectively maintained identification
and seal and either one of following two procedures followed (depending on how many memory cards are available for use).
1) the identical card is used for test and election, with a re-zeroing or
re-downloading of the card done after the LAT and before the election. This is
not the preferred method, but required if there are an insufficient number of
cards available to accomplish #2.
2) a duplicate set of cards is prepared in advance, enough for test and for
election. The cards are chosen at random for the test so that the
identity as the one for the election is determined by chance just prior to
testing. During the test these specific cards are clearly marked as dedicated
to the test so these cards will never be tallied in an election. I think this
is the best solution.
where? In this case, the cards chosen at random for tests are sealed and
separated so that they can not be included in the election tally (along with
any other memory cards in inventory).
Has this been worked out in election rules elsewhere?
But Teak says : [Teak Simonton] It is my understanding that we are not required to test each
memory card used, but to test the programming of the election that the cards
hold – and testing some of the cards suffices. I am definitely
uncomfortable programming extra Diebold cards and using some for test and some
for election mode – this would make the security people go crazy.
Any of these test cards could conceivably be uploaded on election night in
replacement of the real cards – we can’t do this.
Teak, what you are
describing as "we can't do this", is exactly what you have regularly
been doing... creating extra memory cards as spares for use in emergencies.
This is just as dangerous or more dangerous than programming extra memory cards
for the test. And it might indeed make the security people go crazy. Are
you going to stop programming spare memory cards? You must be consistent
on this point. Either you adhere to the security restrictions imposed by
the manufacturer and live within those restrictions, or you provide physical
security to be sure that test and spare memory cards are never tallied. In
which case it should be no problem to use extra memory cards for the testing.
It is inconsistent to go both ways on this.
The production of "spare" memory cards must be
addressed by the rules and procedures in the future, particularly in terms of
security. The failure of the manufacturer's own equipment makes the spares
essential. Anyway, it seems to me the existence of the spares makes the
distinction between "test" and "election" mode meaningless,
and therefore the use of any "test" mode should be abandoned, as I believe
Colorado has
already decided.
Looking forward to your comments
Harvie Branscomb