Experimental study of software dependability

Abstract

There is trend of increasing demand for highly dependable software systems. The factors that influence the dependability of software systems include software faults and hardware faults. To improve software dependability, it is necessary to understand the characteristics of these faults and how they affect software systems. In this study, a distributed fault injection and monitoring environment (DEFINE) has been developed. It consists of a target system, a fault injector, a software monitor, a workload generator, a controller, and several analysis utilities. DEFINE can inject software faults as well as hardware faults, can trace fault propagation in software systems and among machines, can monitor whether faults are activated and when the faults are activated, and has accurate time control. The fault models used are extracted from the results of field error data analyses and fault simulations. Fault injection experiments on the UNIX kernel (SunOS 4.1.2) and the Sun Network File System are conducted to study fault impact and to investigate fault propagation. Three kinds of fault injections are conducted: uniform fault injection, biased fault injection, and path-based fault injection. Based on the experimental results, fault propagation models have been developed for both hardware and software faults, and transient Markov reward analysis has been performed to evaluate the loss of performance after a fault is injected.

Experimental results show that the majority of no-impact faults are latent. Memory faults and software faults usually have a very long latency, while bus faults and CPU faults tend to crash the system immediately. About half of the detected errors are data faults, and they are detected while the system is trying to access a memory location it has no privilege to access. Only about 8% of faults propagate to other UNIX subsystems. Fault propagation from servers to clients occurs more frequently than from clients to servers. The fault impact depends on the workload. Transient Markov reward analysis shows that the performance losses incurred by bus faults and CPU faults are much higher than those incurred by software and memory faults. Among software faults, the impact of pointer faults is higher than that of non-pointer faults.