Following a series of fatal accidents in the mid-1990s, a formal investigation was conducted with the Therac-25 radiotherapy machine. Led by Nancy Leveson of the University of Washington, the investigation resulted in a set of recommendations on how to create safety-critical software solutions in an objective manner. Since then, industries as disparate as aerospace, automotive and industrial control have encapsulated the practices and processes for creating safety- and/or security-critical systems in an objective manner into industry standards.
Although subtly different in wording and emphasis, the standards across industries follow a similar approach to ensuring the development of safe and/or secure systems. This common approach includes ten phases:
1. Perform a system safety or security assessment
2. Determine a target system failure rate
3. Use the system target failure rate to determine the appropriate level of development rigor
4. Use a formal requirements capture process
5. Create software that adheres to an appropriate coding standard
6. Trace all code back to their source requirements
7. Develop all software and system test cases based on requirements
8. Trace test cases to requirements
9. Use coverage analysis to assess test completeness against both requirements and code
10. For certification, collect and collate the process artifacts required to demonstrate that an appropriate level of rigor has been maintained.