Part 4 Summary and Conclusion

4.2 Testing, Accuracy and Secrecy

Testing
The principal issues identified by the Commission in relation to the testing of the system are as follows:

• The software has been updated many times since the pilot elections in 2002 and since the full desk review of the source code:
  
  
  • The original desk reviews of earlier versions of the software continue to be relied upon as the baseline for evaluating the ongoing changes to the system that give rise to new versions,
  • There have been a large number of new versions of the software since the original desk reviews and tests took place,
  • As changes are made to the system, each new software version needs to be reviewed and tested in full before it can be relied upon for use in real elections,
  • It has not been possible for the Commission to review the impact of the changes made in successive versions of the software in time for inclusion in this report,
  • The fact that new versions of the software continue to be issued in the run-up to the June elections is unsatisfactory,
  • There is not sufficient time before the June elections for full testing of the final version of the software which would be essential before the software could be run in these elections;
    
    
• It has not been possible for the Commission to obtain access to the full source code of the system:
  
  
  • it has therefore not been possible to carry out the preliminary review of the full source code that might have been possible within the timeframe of this report,
  • A comprehensive review of the full source code of the system is necessary to establish its trustworthiness to a level compatible with the critical importance of voting at elections: such a comprehensive code review is outside the timeframe of this report,
  • There is not sufficient time before the June elections to allow a full code review of the final version of the software that would actually run in these elections;
    
    
• Some components of the system have not been tested, in particular those at the interface between tested components;
  
  
• The tests of the system carried out to date are insufficient to establish its reliability for use at elections in Ireland in June:
  
  
  • There has been very limited “end-to-end” testing of the full system in its entirety as it would run in June, and none has been carried out independently: significant in this context is that the system as a whole will be required to register, combine, disaggregate, mix and count votes for up to four different polls being held at the same time,
  • There has been no parallel testing of the system in a real election, either against the traditional manual system of voting or against an alternative electronic means; such parallel testing is very important for such a critical system as voting at elections: although the system was deployed on a pilot basis in 2002, these elections were not run in parallel with a paper ballot, and the software has been modified many times since then;
    
    
• The system has not been tested as a whole or certified as being suitable for use in an Irish electoral context by an accredited testing and certification authority.
  

Accuracy
The principal issues identified by the Commission in relation to the accuracy of the proposed system largely follow from the Commission’s concerns about testing:

• As the software version proposed for use at the forthcoming elections is not as yet finalised, it is impossible for anyone to certify its accuracy;
  
  
• The issues set out above in relation to the testing of the system make it impossible to determine its accuracy in the context of this report;
  
  
• Certain of the tests performed at the request of the Commission identified an error in the count software which could lead to incorrect distributions of surpluses; there is a possibility that further testing will uncover further software errors;
  
  
• While eliminating the possibility of certain types of inadvertent voter error, the chosen system introduces the possibility of new types of error in the use by electors of the voting machine, particularly in the context of a number of simultaneous polls;
  
  
• There is a possibility of interference with the voting machine, ballot module and hardened PC:
  
  
  • In particular, experts retained by the Commission found it very easy to bypass electronic security measures and gain complete control of the “hardened” PC, overwrite the software, and thereby in theory to gain complete control over the count in a given constituency,
  • The examinations carried out by the Commission’s experts suggests that these “hardened” PCs are the weakest link in the security of the proposed system and it is significant that there appears to have been no systematic testing and certification of the “hardening” of the PCs notwithstanding their susceptibility to either inadvertent error or deliberate manipulation by those with access to them;
    
    
• The system allows the inadvertent use of outdated versions of the software, as well as the overwriting of the software with replacement software;
  
  
• Attention is required to procedural issues and controls regarding the storage, handling, deployment and use of the equipment by election personnel as contained in the documentation issued to returning officers.

Furthermore, in the context of the June elections, in which each elector would be asked to use the same voting machine to vote simultaneously on a number of different matters, it is important to note that accuracy in the translocation and counting of votes critically involves the system for the aggregation of votes from many different polling machines, followed by their subsequent disaggregation, then separate mixing and counting in local and European elections, as well as the proposed referendum.

Secrecy
The principal issues identified by the Commission in relation to the secrecy of the system are as follows:

• The voting machine “beeps” as preferences are being selected, and to signal voter errors; this allows limited inferences to be drawn by those outside the polling booth about the number of preferences cast: in particular a voter voting for a single candidate would be easy to identify by those in the vicinity of the machine;
  
  
• There is reduced voting secrecy for persons with certain disabilities (although this is not a legal issue in the sense that, in McMahon v Attorney General the Court held that the right to secrecy is not an absolute one) as well as for persons who are unfamiliar with technology and who may need third-party assistance in using the machine;
  
  
• Publication of ballot results in full is a valuable aid in checking the accuracy of the results but this can in theory reveal deliberate voter “signatures” of low-preference votes which could allow voters to identify themselves in a context of corruption or intimidation;
  
  
• It may be possible for an insider to overcome the randomness of the method used for the storage of votes in the ballot module.

< Back to table of 'Report Contents'