[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Comments on Election Rule 45 (fwd)
I was feeling unusually masochistic last week, so, read Rule 45, and then
somewhat reluctantly, chose to file these comments. I feel a little
foolish about being drawn once again into commenting on (and perhaps
therefore implicitly legitimizing) a process that seems to be intended to
avoid significant public input, made particularly blatant by the fact that
the Rule has been in force for almost a month. Anyway, I'm forwarding the
comments on to the CVV list anyway in case anyone happens to be
interested - I do not plan to attend the public hearing.
Hope everyone still reading the list is doing well, and enjoying the
happier things in life,
- Paul
---------- Forwarded message ----------
Date: Sun, 30 Oct 2005 00:49:16 -0600 (MDT)
From: Paul Walmsley <paul@xxxxxxxxxxx>
To: john.gardner@xxxxxxxxxxxxxxx
Subject: Comments on Election Rule 45
Mr. Gardner,
Please accept these comments on Election Rule 45 for the upcoming public
hearing on Monday, October 31. I request that these comments be entered into
the public record for this hearing.
Regards,
- Paul
Comments on Colorado Secretary of State Election Rule 45
29 October 2005
Paul Walmsley <paul@xxxxxxxxx>
A brief reading of Colorado Secretary of State Election Rule 45
regarding Colorado's Voting System Certification Standards reveals
several significant problems that should be resolved. The comments
below describe some of these problems and proposes solutions in the
form of an improved standards drafting process.
Several different versions of Rule 45 exist on the Secretary of
State's web site. The following comments refer to what is apparently
the most recent version of Rule 45, labeled "RULE 45 - 10-03-05" [1].
Furthermore, the comments below should not be considered a
comprehensive review of Rule 45.
1 The State's voting system certification standards must be
precisely written, and many sections of Rule 45 are not. This
imprecision make it impossible to convert those into verifiable,
testable metrics.
1.1 One example of this imprecision is section 45.5.2.1.1, which
states that "the voting system shall exhibit an evolution towards new
technologies," an abstract requirement which is never concretely
elaborated. Another is section 45.5.2.3.8, which states that "the
environment in which all databases in the subsystem are maintained
shall include all necessary provisions for security and access
control," without describing the actual security and access control
provisions required. The practical requirements of these and other
sections are unclear to the point of being meaningless. While
statements of general guiding principles are important, they must be
concretely defined with specific, testable criteria in standards
documents.
1.2 Rule 45's performance standards also are very imprecisely defined.
They make references to operational terms which have no
publicly-accepted meaning, and which are not defined in the Rule's
glossary, and extend the meanings of other terms, apparently to
include other processes. For example, section 45.5.2.2.1 refers to
"counting ballots", but does not define whether this process includes
the process of casting ballots, as implied by the inclusion of the DRE
requirement in subsection (b), or whether it includes the process of
ballot scanning, as implied by the optical scan subsections (a) and
(c), or whether it includes the process of vote tabulation, which on
many systems is separate from the vote casting and ballot scanning
processes. Section 45.5.2.2.2 refers to terms like "election media
download" and "ballot style assignment" which are not defined - it's
not clear whether these terms refer to pre-election preparations, or
to the actual ballot casting process. It is also unclear whether
these performance requirements refer to individual DRE or vote
scanning machines, as implied by section 45.5.2.2.1; or whether these
metrics refer to the voting system as a whole, including all of the
DRE or vote scanning machines. Testing conformance to the performance
requirements in Rule 45 is practically impossible unless the processes
under test are rigorously defined.
1.3 The above examples represent only a portion of unclear
requirements in Rule 45. These types of imprecision in a
certification standards document is undesirable. It creates loopholes
by which vendors may certify equipment to the letter of the Rule that
does not meet the intention of the Rule. It also fosters confusion as
to the Rule's true requirements.
2 The State's voting system certification standards should define
performance requirements that specify the maximum amount of time
required to perform real-world tasks required in an election, rather
than requiring metrics which are not useful for Colorado elections, as
are defined in Rule 45. For example, section 45.5.2.2.1 requires a
minimum ballot counting rate requirement of 100 ballots per minute for
central count optical scan ballots. Such a minimum requirement may be
useful for Hinsdale County, Colorado, with only a few hundred voters;
but is not useful for Denver County, Colorado, with hundreds of
thousands of potential voters. Vendors should instead be required to
demonstrate that their system is capable of completing the specified
election action in a reasonable election time frame. For example, a
more meaningful version of section 45.5.2.2.1 would require voting
systems to complete the vote count in a specified maximum number of
hours.
3 The parts of the State's voting system certification standards that
pertain to computer security and cryptography should be written and
reviewed by experts in those fields. This is not the case for at least
the information security sections of Rule 45, which were clearly not
written or reviewed by experts in the field. For example, section
45.5.2.7.2 refers to "a minimum encryption requirement of 40-bit
encryption." Presumably Rule 45's authors are trying to ensure the
privacy and integrity of election records that are transmitted over
public networks. But section 45.5.2.7.2 does not do this - in fact,
it is technically meaningless. For it to be meaningful, the vendor
must provide full cryptographic protocol and implementation details,
including details on key generation, entropy collection, message
authentication, and sender and recipient authentication. These
details must be reviewed by experts in cryptography and information
security.
4 The State's voting system certification standards should require
mandatory compliance with the EAC's Voluntary Voting System Standards
[2] - Rule 45 does not require this. These standards are intended to
represent the state of the art in voting system standards, and unlike
Rule 45, have received nationwide public review. If Colorado's
standards are to live up to the Secretary of State's description of
them as "one of the most challenging and thorough programs in the
country," they should at least require compliance with the latest
standards guidance developed by the Federal Government.
5 The State's voting system certification standards should explicitly
forbid vendors from submitting "confidential" or "trade secret"
information for certification as documentation or application
responses to ensure maximum openness and transparency, and Rule 45
does not do this. A vendor could conceivably claim "trade secret"
status for significant portions of the documentation required to
comply to the rule. This would prevent the public from engaging in
any meaningful review of the voting systems, and would conflict with
the Secretary of State's commitment to an "open and transparent"
voting system certification process [3]. The Colorado voting system
standards should forbid the vendor from claiming "trade secret" status
for any documentation provided to the State examiners, given the
special public trust requirements for voting systems.
6 The State's voting system certification standards should include a
glossary which includes all of the terms in the standards which do not
have clear public usages, and should build on glossaries already
created in the federal voting system standards. But as noted
previously, Rule 45's glossary does not contain entries for terms like
"election media download," which do not have commonly-accepted
unambiguous meanings in elections. Additionally, terms like "Ballot
Image" are ambiguously defined. For example, it is unclear whether a
"ballot image" refers to cast vote records, or to graphics files
depicting the scanned paper ballots from optical scan systems.
7 The State's standards should require vendor systems to support live
auditing procedures [4], which Rule 45 does not require. Live
auditing is the process of continuously evaluating the accuracy and
functionality of election systems during an election using live
ballots. Without mandating vendor support for live audit techniques,
the accuracy of the election system can only be tested before or after
an election, and such testing is inadequate to assure election accuracy.
Many of the principles that motivate Colorado's voting system
standards seem well-intentioned. However, the specific manifestation
of these standards in the present Rule 45 does not measure up to these
principles. It is my belief from a cursory examination of Rule 45
that the problems cited above are not simply endemic to the sections
quoted above, but exist throughout the document. This suggests that
the public would best be served by committing to a different
standards-making process than was used for the Rule under discussion.
A better standards-making process would include the following principles:
- It should be continuously edited by staff experienced with the
precise language necessary to define standards.
- It should involve independent subject matter experts outside the
Secretary of State's area of expertise.
- All drafts should be released to the public for ongoing comment
throughout the drafting process.
- The document should strive to avoid duplicating existing Federal
standards, and should draw on existing work, both at the Federal
level and standards work by other states. In particular, the State
should mandate compliance with the existing EAC Voluntary Voting
System Guidelines.
- It should first be articulated in a general set of requirements for
voting system performance that are then translated into specific,
testable, concrete requirements. For example, a general requirement
that "election voting system records and data must be secure from
tampering and unauthorized interception" can be translated into
specific technical requirements that serve that general principle.
Similar general principles and concrete requirements should be
articulated for accessibility, reliability, live auditability,
speed, and other desirable aspects for Colorado's voting systems.
It is also troubling that Rule 45 was adopted in emergency, before any
opportunity for comment or review by independent subject matter
experts or the public. Rule 45 seems rushed, and it is a matter of
some concern that voting systems may be certified under the emergency
rule that would not be certifiable under a more deliberate Rule,
either revised from the current rule or rewritten. Therefore, I
encourage the Colorado Secretary of State to:
- Revoke the emergency adoption of Rule 45,
- Commit to an open public comment and revision process for a revised
voting system standard before it is adopted, and for all future
revisions, and
- When revised standards are complete, require immediate
re-certification of all equipment certified under the emergency
adoption of Rule 45.
Sincerely,
Paul Walmsley
Boulder, Colo.
Footnotes
1. http://www.elections.colorado.gov/DesktopModules/Downloads/download.aspx?tid=501&_iid=193
2. http://guidelines.kennesaw.edu/vvsg/guide_toc.asp
3. http://www.elections.colorado.gov/DDefault.aspx?tid=501
4. http://www.booyaka.com/~paul/ea/eac-20050930/interpretation-live-audit.txt
http://www.booyaka.com/~paul/ea/eac-20050930/tabulation-live-audit.txt
http://www.booyaka.com/~paul/ea/eac-20050930/live-audit-overview.txt