Pictured above is ISAlliance Board Chair Ken Silva (VeriSign) with the Co-Director of Carnegie Mellon CyLab, Pradeep Khosla.  Carnegie Mellon partners with the ISAlliance in producing information security deliverables like Software Assurance and Privacy Policy Trends.

These deliverables may be in the form of a white paper, such as the Privacy Policy Trends, an assessment tool like T-SQUARE and the SQUARE methodology, or other material designed for operational improvements in corporate security. 

The state of the art for software assurance is a broad topic. No single document can hope to cover the entire scope of efforts ongoing in this area. Rather, this document is the first in what is expected to be at least a two part series that focuses on a particular means of achieving software assurance. This first document identifies the state of the art with regards to process means of achieving software assurance. However, we believe that process alone cannot hope to achieve a high enough level of quality for all types of applications. Furthermore, we believe that the cost/benefit (or sometimes just plain benefit) function of quality efforts can be greatly improved by the addition of technology and tools to quality regimes.

We expect to come back at a later time and discuss other tools and technology that help close this quality gap and improve the cost/benefit function of quality efforts.
 

This report is exclusive to members of the Internet Security Alliance.

Following is the Executive Summary of our first paper:

    Both tools and process are needed to achieve critical level of acceptance for software quality. This paper focuses on the process elements that contribute to greater assurance that our software does what it is supposed to do, is free of exploitable vulnerabilities, and does not exhibit unexpected behaviors. 

    Process efforts can be very effective at reducing the defect densities of software released to the field. The Capability Maturity Model (CMM) identifies levels of maturity for the software process. Data shows that higher maturity processes achieve higher quality and the difference between CMM level 1 and CMM level 5 is roughly an order of magnitude. More evolved process definitions, particularly those that take the motivation of individual developers teams into account and encourage early defect removal, like the Team Software Process (TSP), can achieve an additional order of magnitude improvement over generic CMM level 5 processes.

    These efforts work because of the economic advantages of removing defects early and the synergistic effects of providing appropriate feedback loops for the situation at hand. Testing is currently our primary means of achieving high assurance but just as in manufacturing, the best processes attempt to use testing only as a measure on the effectiveness of the earlier process steps rather than a means of removing large volumes of defects. Reviews and inspections are the primary process based means of removing defects early.

    We see growing adoption rates of high maturity processes as more packaged process definitions like the TSP become available. Other process feedback loop emphasis shifts are being explored. Agile methods tend to favor feedback from the customer to the design and feedback of the information learned while coding back to the design and other parts of the code. Synchronize and stabilize processes tend to emphasize any feedback that would be an indicator of problems in the application programming interface (API) of larger systems. Test driven development and other process steps introduced by agile methods are experiencing emergent adoption even with more traditional process definitions.

    The integration of new tools to enhance software assurance into existing processes and in particular the changes that will imply to the process feedback loops is identified as a critical area of further study. As these tools start to mature and see wide spread adoption it will be important to understand how there introduction influences the other aspects of the process definition.