Cyber-Risk Informatics

9 METRICS FOR SOFTWARE RELIABILITY FAILURE-COUNT MODELS IN CYBER-RISK

LEARNING OBJECTIVES

• Present an overview of failure-count models and the state of the art and science of their predictive accuracy for comparison purposes.
• Review traditional models and recently discovered advances in Bayesian modeling to form a basis of comparison through metrics.
• Show examples to illustrate how these methods work through a software program.

9.1 INTRODUCTION AND METHODOLOGY ON FAILURE-COUNT ESTIMATION IN SOFTWARE RELIABILITY

This chapter is related to the general problem of ranking usual means discussed in the literature and others as a background to the 2001 publication by Sahinoglu et al. focusing on statistical measures for comparing the predictive merits of software reliability models [1–7]. There is increasing pressure to develop and quantify measures of computer software reliability. With the ascent of software reliability models as detailed previously in Chapter 8, there is now even more pressure on assessment of the predictive quality of these measures, both in the sense of their goodness of fit and their pairwise comparisons [8, 9]. However, current methods used to compare these models use deterministic measures, and hence their results do not reflect the uncertainty inherent in these observations. In particular, the predictive accuracy of various methods is compared through measures such as average absolute (to avoid a zero average) relative error (ARE) and mean squared ...