SlideShare a Scribd company logo
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 1
Software Testing and Quality Assurance
Theory and Practice
Chapter 15
Software Reliability
Prof. Abraham Paul
Assistant Prof. VIT
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 2
Outline of the Chapter
• What is Reliability?
• Definitions of Software Reliability
• Factors Influencing Software Reliability
• Applications of Software Reliability
• Operational Profiles
• Reliability Models
• Summary
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 3
What is Reliability?
• Reliability is a broad concept.
– It is applied whenever we expect something to behave in a certain way.
• Reliability is one of the metrics that are used to measure quality.
• It is a user-oriented quality factor relating to system operation.
– Intuitively, if the users of a system rarely experience failure, the system is
considered to be more reliable than one that fails more often.
• A system without faults is considered to be highly reliable.
– Constructing a correct system is a difficult task.
– Even an incorrect system may be considered to be reliable if the frequency of
failure is “acceptable.”
• Key concepts in discussing reliability:
– Fault
– Failure
– Time
– Three kinds of time intervals: MTTR, MTTF, MTBF
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 4
What is Reliability?
• Failure
– A failure is said to occur if the observable outcome of a program execution is
different from the expected outcome.
• Fault
– The adjudged cause of failure is called a fault.
– Example: A failure may be cause by a defective block of code.
• Time
– Time is a key concept in the formulation of reliability. If the time gap between
two successive failures is short, we say that the system is less reliable.
– Two forms of time are considered.
• Execution time (τ)
• Calendar time (t)
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 5
What is Reliability?
• MTTF: Mean Time To Failure
• MTTR: Mean Time To Repair
• MTBF: Mean Time Between Failures (= MTTF + MTTR)
Figure 15.1: Relationship between MTTR, MTTF, and MTBF.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 6
What is Reliability?
• Two ways to measure reliability
– Counting failures in periodic intervals
• Observer the trend of cumulative failure count - µ(τ).
– Failure intensity
• Observe the trend of number of failures per unit time – λ(τ).
• µ(τ)
– This denotes the total number of failures observed until execution time τ
from the beginning of system execution.
• λ(τ)
– This denotes the number of failures observed per unit time after τ time units
of executing the system from the beginning. This is also called the failure
intensity at time τ.
• Relationship between λ(τ) and µ(τ)
– λ(τ) = dµ(τ)/dτ
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 7
Definitions of Software Reliability
• First definition
– Software reliability is defined as the probability of failure-free operation of a
software system for a specified time in a specified environment.
• Key elements of the above definition
– Probability of failure-free operation
– Length of time of failure-free operation
– A given execution environment
• Example
– The probability that a PC in a store is up and running for eight hours without
crash is 0.99.
• Second definition
– Failure intensity is a measure of the reliability of a software system operating
in a given environment.
• Example: An air traffic control system fails once in two years.
• Comparing the two
– The first puts emphasis on MTTF, whereas the second on count.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 8
Factors Influencing Software Reliability
• A user’s perception of the reliability of a software depends upon two
categories of information.
– The number of faults present in the software.
– The ways users operate the system.
• This is known as the operational profile.
• The fault count in a system is influenced by the following.
– Size and complexity of code
– Characteristics of the development process used
– Education, experience, and training of development personnel
– Operational environment
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 9
Applications of Software Reliability
• Comparison of software engineering technologies
– What is the cost of adopting a technology?
– What is the return from the technology -- in terms of cost and quality?
• Measuring the progress of system testing
– Key question: How of testing has been done?
– The failure intensity measure tells us about the present quality of the system:
high intensity means more tests are to be performed.
• Controlling the system in operation
– The amount of change to a software for maintenance affects its reliability. Thus
the amount of change to be effected in one go is determined by how much
reliability we are ready to potentially lose.
• Better insight into software development processes
– Quantification of quality gives us a better insight into the development
processes.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 10
Operational Profiles
• Developed at AT&T Bell Labs.
• An OP describes how actual users
operate a system.
– An OP is a quantitative
characterization of how a system
will be used.
• Two ways to represent
operational profiles
– Tabular
– Graphical
Table 15.1: An example of operational
profile of a library information
system.
Figure 15.2: Graphical representation of
operational profile of a library information
system.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 11
Operational Profiles
• Use of operational profiles
– For accurate estimation of the reliability of a system, test the system in the
same way it will be actually used in the field.
• Other uses of operational profiles
– Use an OP as a guiding document in designing user interfaces.
• The more frequently used operations should be easy to use.
– Use an OP to design an early version of a software for release.
• This contains the more frequently used operations.
– Use an OP to determine where to put more resources.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 12
Reliability Models
• Main idea
– We develop mathematical models for λ(τ) and µ(τ).
• Basic assumptions in developing a reliability model
– Faults in the program are independent.
– Execution time between failures is large w.r.t. instruction execution time.
– Potential test space covers its use space.
– The set of inputs per test run is randomly chosen.
– The fault causing a failure is immediately fixed or else its re-occurrence is not
counted again.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 13
Reliability Models
• Intuitive idea
– As we observe another system failure and the corresponding fault is fixed,
there will be fewer number of faults remaining in the system and the failure
intensity will be smaller with each fault fixed.
– In other words, as the cumulative failure count increases, the failure intensity
decreases.
• Two decrement processes
– Decrement process 1
• The decrease in failure intensity after observing a failure and fixing the
corresponding fault is constant.
– This gives us the Basic model.
– Decrement process 2
• The decrease in failure intensity after observing a failure and fixing the
corresponding fault is smaller than the previous decrease.
– This gives us the Logarithmic model.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 14
Reliability Models
• Parameters of the models
– λ0: The initial failure intensity
observed at the beginning of
system testing.
– v0: The total number of
system failures that we expect
to observe over infinite time
starting from the beginning of
system testing.
θ: A parameter representing
n0n-linear drop in failure
intensity in the Logarithmic
model.
Figure 15.3: Failure intensity λ as a function of
cumulative failures µ.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 15
Reliability Models
• Basic model
Assumption: λ(µ) = λ0 (1 - µ/v0)
dµ(τ)/dτ = λ0 (1 - µ(τ)/v0)
µ(τ) = λ0(1 - µ/v0)
λ(τ) = λ0.e -λ0 τ/v0
• Logarithmic model
Assumption: λ(µ) = λ0e-θµ
dµ(τ)/dτ = λ0e-θµ(τ)
µ(τ) = ln(λ0θτ + 1)/θ
λ(τ) = λ0/(λ0θτ + 1)
Figure 15.4: Failure intensity λ as a function of
execution time τ (λ0= 9 failures/unit time,v0=
500 failures, θ = 0.0075).
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 16
Reliability Models
Figure 15.4: Cumulative failure µ as a function of execution time τ (λ0=
9 failures/unit time,v0 = 500 failures, θ = 0.0075).
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 17
Reliability Models
• Example
Assume that a software system is undergoing system level testing. The initial
failure intensity of the system was 25 failures/CPU hours, and the current
failure intensity is 5 failures/CPU hour. It has been decided by the project
manager that the system will be released only after the system reaches a
reliability level of at most 0.001 failures/CPU hour. From their experience the
management team estimates that the system will experience a total of 1200
failures over infinite time. Calculate the additional length of system testing
required before the system can be released.
– The system will experience a total of 1200 failures over infinite time. Thus, we
use the Basic model.
– λc and λr are the current failure intensity and the failure intensity at the time of
release.
– Assume that the current failure intensity has been achieved after executing the
system for τc hours.
– Let λr be achieved after testing the system for a total of τr hours.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 18
Reliability Models
• (Example continued)
– (τr - τc) denotes the additional execution time requires to achieve λr.
We can write λc and λr as follows.
λc = λ0.e -λ0 τc/v0
λr = λ0.e -λ0 τr/v0
λc/ λr = (λ0.e -λ0τc/v0
)/(λ0.e -λ0 τr/v0
)
= e (τr - τc) λ0/v0
ln(λc/ λr) = (τr - τc) λ0/v0
(τr - τc)= (v0/ λ0)ln(λc/ λr)
= (1200/25)ln(5/0.001)
= 408.825 hours
It is required to test the system for more time so that the CPU runs for another 408.825
hours to achieve the reliability level of 0.001 failures/hour.
Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 19
Summary
• Reliability is a user-oriented quality
factor relating to system operation.
• The chapter introduced the following.
– Fault and failure
– Execution and calendar time
– Time interval between failures
– Failures in periodic intervals
– Failure intensity
• Software reliability was defined in
two ways.
– The probability of failure-free
operation of a system for a specified
time in a given environment.
– Failure intensity is a measure of
reliability.
• User’s perception of reliability:
– The number of faults in a system.
– How a user operates a system.
• The number of faults in a system is
influenced by the following:
– Size and complexity of code.
– Development process.
– Personnel quality.
– Operational environment
• Operational profile
– A quantitative characterization of how
actual users operate a system.
– Tabular and graphical representation
• Applications of reliability metric
• Reliability models
– Six assumptions
– Two models
• Basic
• Logarithmic

More Related Content

PPT
Software reliability
Anand Kumar
 
PPTX
Software Reliability
Gurkamal Rakhra
 
PPT
Software Reliability
ranapoonam1
 
PPTX
Software testing and process
gouravkalbalia
 
PPT
The Software Development Process
Cesar Augusto Nogueira
 
PPTX
Software reliability engineering process
Himanshu
 
PPTX
Eleven step of software testing process
Himanshu
 
PPT
Software Testing Fundamentals
Chankey Pathak
 
Software reliability
Anand Kumar
 
Software Reliability
Gurkamal Rakhra
 
Software Reliability
ranapoonam1
 
Software testing and process
gouravkalbalia
 
The Software Development Process
Cesar Augusto Nogueira
 
Software reliability engineering process
Himanshu
 
Eleven step of software testing process
Himanshu
 
Software Testing Fundamentals
Chankey Pathak
 

What's hot (20)

PPTX
Software engineering 23 software reliability
Vaibhav Khanna
 
PPT
Black box & white-box testing technique
SivaprasanthRentala1975
 
PPT
Risk management in software engineering
deep sharma
 
PPTX
Software Quality Assurance
Saqib Raza
 
PPTX
Software quality assurance
Er. Nancy
 
PPTX
The Art of Debugging.pptx
KarthigaiSelviS3
 
PPTX
Ch 3 software quality factor
Kittitouch Suteeca
 
PPT
Risk management(software engineering)
Priya Tomar
 
PPTX
Software testing
balamurugan.k Kalibalamurugan
 
PPTX
Unit 5
Jignesh Kariya
 
PPT
Chapter 15 software product metrics
SHREEHARI WADAWADAGI
 
PPT
Improving software economics
deep sharma
 
PDF
Software process
Dr. Loganathan R
 
PPT
Risk Management by Roger Pressman
Rogerio P C do Nascimento
 
PPT
Manual testing concepts course 1
Raghu Kiran
 
PDF
Types of software testing
Prachi Sasankar
 
PDF
Software Reliability and Safety.pdf
Dr. Ajeet Kumar Pandey
 
PPTX
Introduction to software testing
Hadi Fadlallah
 
PPTX
source code metrics and other maintenance tools and techniques
Siva Priya
 
PPTX
Quality management
MunibaNoureen
 
Software engineering 23 software reliability
Vaibhav Khanna
 
Black box & white-box testing technique
SivaprasanthRentala1975
 
Risk management in software engineering
deep sharma
 
Software Quality Assurance
Saqib Raza
 
Software quality assurance
Er. Nancy
 
The Art of Debugging.pptx
KarthigaiSelviS3
 
Ch 3 software quality factor
Kittitouch Suteeca
 
Risk management(software engineering)
Priya Tomar
 
Chapter 15 software product metrics
SHREEHARI WADAWADAGI
 
Improving software economics
deep sharma
 
Software process
Dr. Loganathan R
 
Risk Management by Roger Pressman
Rogerio P C do Nascimento
 
Manual testing concepts course 1
Raghu Kiran
 
Types of software testing
Prachi Sasankar
 
Software Reliability and Safety.pdf
Dr. Ajeet Kumar Pandey
 
Introduction to software testing
Hadi Fadlallah
 
source code metrics and other maintenance tools and techniques
Siva Priya
 
Quality management
MunibaNoureen
 
Ad

Similar to Ch15 software reliability (20)

PPTX
Software Reliability_CS-3059_VISHAL_PADME.pptx
VishalPadme2
 
PDF
Software Testing and Quality Assurance Assignment 2
Gurpreet singh
 
PPT
Ch13 system testexecution
abcxyz_abc
 
PDF
chapter-09.pdf software metrics Bahir dar university
ethiobahirdarhotel
 
PPTX
Software reliability & quality
Nur Islam
 
DOCX
Some Commonly Asked Question For Software Testing
Kumari Warsha Goel
 
PDF
Optimal Selection of Software Reliability Growth Model-A Study
IJEEE
 
DOCX
Performance testing interview questions and answers
Garuda Trainings
 
PPTX
Software engineering quality assurance and testing
Bipul Roy Bpl
 
PDF
Manual Tester Interview Questions(1).pdf
SupriyaDongare
 
PDF
CS-438 COMPUTER SYSTEMS MODELING WK1LEC1-2.pdf
MUHAMMADUSMANYOUSUF1
 
PPT
software Engineering process
Raheel Aslam
 
PPT
08-Performence_Testing Project Explain.ppt
pspc139
 
PDF
What is Test Matrix?
QA InfoTech
 
PPT
Performance testing and rpt
SivaprasanthRentala1975
 
PDF
An Introduction to Software Testing Life Cycle
KMSSolutionsMarketin
 
PPTX
Manual testing interview questions and answers
Rajnish Sharma
 
PPT
Sech1920 1200112979886874-3
Mateti Anilraja
 
PDF
What is Performance Testing?
QA InfoTech
 
PPT
Non Functional Testing
Nishant Worah
 
Software Reliability_CS-3059_VISHAL_PADME.pptx
VishalPadme2
 
Software Testing and Quality Assurance Assignment 2
Gurpreet singh
 
Ch13 system testexecution
abcxyz_abc
 
chapter-09.pdf software metrics Bahir dar university
ethiobahirdarhotel
 
Software reliability & quality
Nur Islam
 
Some Commonly Asked Question For Software Testing
Kumari Warsha Goel
 
Optimal Selection of Software Reliability Growth Model-A Study
IJEEE
 
Performance testing interview questions and answers
Garuda Trainings
 
Software engineering quality assurance and testing
Bipul Roy Bpl
 
Manual Tester Interview Questions(1).pdf
SupriyaDongare
 
CS-438 COMPUTER SYSTEMS MODELING WK1LEC1-2.pdf
MUHAMMADUSMANYOUSUF1
 
software Engineering process
Raheel Aslam
 
08-Performence_Testing Project Explain.ppt
pspc139
 
What is Test Matrix?
QA InfoTech
 
Performance testing and rpt
SivaprasanthRentala1975
 
An Introduction to Software Testing Life Cycle
KMSSolutionsMarketin
 
Manual testing interview questions and answers
Rajnish Sharma
 
Sech1920 1200112979886874-3
Mateti Anilraja
 
What is Performance Testing?
QA InfoTech
 
Non Functional Testing
Nishant Worah
 
Ad

Recently uploaded (20)

PDF
AI-Driven IoT-Enabled UAV Inspection Framework for Predictive Maintenance and...
ijcncjournal019
 
PPTX
business incubation centre aaaaaaaaaaaaaa
hodeeesite4
 
PDF
The Effect of Artifact Removal from EEG Signals on the Detection of Epileptic...
Partho Prosad
 
PPTX
IoT_Smart_Agriculture_Presentations.pptx
poojakumari696707
 
PDF
Construction of a Thermal Vacuum Chamber for Environment Test of Triple CubeS...
2208441
 
PPT
Understanding the Key Components and Parts of a Drone System.ppt
Siva Reddy
 
PPTX
sunil mishra pptmmmmmmmmmmmmmmmmmmmmmmmmm
singhamit111
 
PDF
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
PPTX
22PCOAM21 Session 1 Data Management.pptx
Guru Nanak Technical Institutions
 
PPTX
Chapter_Seven_Construction_Reliability_Elective_III_Msc CM
SubashKumarBhattarai
 
PPTX
22PCOAM21 Session 2 Understanding Data Source.pptx
Guru Nanak Technical Institutions
 
PDF
Machine Learning All topics Covers In This Single Slides
AmritTiwari19
 
PDF
2010_Book_EnvironmentalBioengineering (1).pdf
EmilianoRodriguezTll
 
PDF
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
PPTX
Module2 Data Base Design- ER and NF.pptx
gomathisankariv2
 
PDF
All chapters of Strength of materials.ppt
girmabiniyam1234
 
PPTX
MSME 4.0 Template idea hackathon pdf to understand
alaudeenaarish
 
PPTX
Online Cab Booking and Management System.pptx
diptipaneri80
 
PDF
2025 Laurence Sigler - Advancing Decision Support. Content Management Ecommer...
Francisco Javier Mora Serrano
 
PDF
settlement FOR FOUNDATION ENGINEERS.pdf
Endalkazene
 
AI-Driven IoT-Enabled UAV Inspection Framework for Predictive Maintenance and...
ijcncjournal019
 
business incubation centre aaaaaaaaaaaaaa
hodeeesite4
 
The Effect of Artifact Removal from EEG Signals on the Detection of Epileptic...
Partho Prosad
 
IoT_Smart_Agriculture_Presentations.pptx
poojakumari696707
 
Construction of a Thermal Vacuum Chamber for Environment Test of Triple CubeS...
2208441
 
Understanding the Key Components and Parts of a Drone System.ppt
Siva Reddy
 
sunil mishra pptmmmmmmmmmmmmmmmmmmmmmmmmm
singhamit111
 
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
22PCOAM21 Session 1 Data Management.pptx
Guru Nanak Technical Institutions
 
Chapter_Seven_Construction_Reliability_Elective_III_Msc CM
SubashKumarBhattarai
 
22PCOAM21 Session 2 Understanding Data Source.pptx
Guru Nanak Technical Institutions
 
Machine Learning All topics Covers In This Single Slides
AmritTiwari19
 
2010_Book_EnvironmentalBioengineering (1).pdf
EmilianoRodriguezTll
 
67243-Cooling and Heating & Calculation.pdf
DHAKA POLYTECHNIC
 
Module2 Data Base Design- ER and NF.pptx
gomathisankariv2
 
All chapters of Strength of materials.ppt
girmabiniyam1234
 
MSME 4.0 Template idea hackathon pdf to understand
alaudeenaarish
 
Online Cab Booking and Management System.pptx
diptipaneri80
 
2025 Laurence Sigler - Advancing Decision Support. Content Management Ecommer...
Francisco Javier Mora Serrano
 
settlement FOR FOUNDATION ENGINEERS.pdf
Endalkazene
 

Ch15 software reliability

  • 1. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 1 Software Testing and Quality Assurance Theory and Practice Chapter 15 Software Reliability Prof. Abraham Paul Assistant Prof. VIT
  • 2. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 2 Outline of the Chapter • What is Reliability? • Definitions of Software Reliability • Factors Influencing Software Reliability • Applications of Software Reliability • Operational Profiles • Reliability Models • Summary
  • 3. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 3 What is Reliability? • Reliability is a broad concept. – It is applied whenever we expect something to behave in a certain way. • Reliability is one of the metrics that are used to measure quality. • It is a user-oriented quality factor relating to system operation. – Intuitively, if the users of a system rarely experience failure, the system is considered to be more reliable than one that fails more often. • A system without faults is considered to be highly reliable. – Constructing a correct system is a difficult task. – Even an incorrect system may be considered to be reliable if the frequency of failure is “acceptable.” • Key concepts in discussing reliability: – Fault – Failure – Time – Three kinds of time intervals: MTTR, MTTF, MTBF
  • 4. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 4 What is Reliability? • Failure – A failure is said to occur if the observable outcome of a program execution is different from the expected outcome. • Fault – The adjudged cause of failure is called a fault. – Example: A failure may be cause by a defective block of code. • Time – Time is a key concept in the formulation of reliability. If the time gap between two successive failures is short, we say that the system is less reliable. – Two forms of time are considered. • Execution time (τ) • Calendar time (t)
  • 5. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 5 What is Reliability? • MTTF: Mean Time To Failure • MTTR: Mean Time To Repair • MTBF: Mean Time Between Failures (= MTTF + MTTR) Figure 15.1: Relationship between MTTR, MTTF, and MTBF.
  • 6. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 6 What is Reliability? • Two ways to measure reliability – Counting failures in periodic intervals • Observer the trend of cumulative failure count - µ(τ). – Failure intensity • Observe the trend of number of failures per unit time – λ(τ). • µ(τ) – This denotes the total number of failures observed until execution time τ from the beginning of system execution. • λ(τ) – This denotes the number of failures observed per unit time after τ time units of executing the system from the beginning. This is also called the failure intensity at time τ. • Relationship between λ(τ) and µ(τ) – λ(τ) = dµ(τ)/dτ
  • 7. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 7 Definitions of Software Reliability • First definition – Software reliability is defined as the probability of failure-free operation of a software system for a specified time in a specified environment. • Key elements of the above definition – Probability of failure-free operation – Length of time of failure-free operation – A given execution environment • Example – The probability that a PC in a store is up and running for eight hours without crash is 0.99. • Second definition – Failure intensity is a measure of the reliability of a software system operating in a given environment. • Example: An air traffic control system fails once in two years. • Comparing the two – The first puts emphasis on MTTF, whereas the second on count.
  • 8. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 8 Factors Influencing Software Reliability • A user’s perception of the reliability of a software depends upon two categories of information. – The number of faults present in the software. – The ways users operate the system. • This is known as the operational profile. • The fault count in a system is influenced by the following. – Size and complexity of code – Characteristics of the development process used – Education, experience, and training of development personnel – Operational environment
  • 9. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 9 Applications of Software Reliability • Comparison of software engineering technologies – What is the cost of adopting a technology? – What is the return from the technology -- in terms of cost and quality? • Measuring the progress of system testing – Key question: How of testing has been done? – The failure intensity measure tells us about the present quality of the system: high intensity means more tests are to be performed. • Controlling the system in operation – The amount of change to a software for maintenance affects its reliability. Thus the amount of change to be effected in one go is determined by how much reliability we are ready to potentially lose. • Better insight into software development processes – Quantification of quality gives us a better insight into the development processes.
  • 10. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 10 Operational Profiles • Developed at AT&T Bell Labs. • An OP describes how actual users operate a system. – An OP is a quantitative characterization of how a system will be used. • Two ways to represent operational profiles – Tabular – Graphical Table 15.1: An example of operational profile of a library information system. Figure 15.2: Graphical representation of operational profile of a library information system.
  • 11. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 11 Operational Profiles • Use of operational profiles – For accurate estimation of the reliability of a system, test the system in the same way it will be actually used in the field. • Other uses of operational profiles – Use an OP as a guiding document in designing user interfaces. • The more frequently used operations should be easy to use. – Use an OP to design an early version of a software for release. • This contains the more frequently used operations. – Use an OP to determine where to put more resources.
  • 12. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 12 Reliability Models • Main idea – We develop mathematical models for λ(τ) and µ(τ). • Basic assumptions in developing a reliability model – Faults in the program are independent. – Execution time between failures is large w.r.t. instruction execution time. – Potential test space covers its use space. – The set of inputs per test run is randomly chosen. – The fault causing a failure is immediately fixed or else its re-occurrence is not counted again.
  • 13. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 13 Reliability Models • Intuitive idea – As we observe another system failure and the corresponding fault is fixed, there will be fewer number of faults remaining in the system and the failure intensity will be smaller with each fault fixed. – In other words, as the cumulative failure count increases, the failure intensity decreases. • Two decrement processes – Decrement process 1 • The decrease in failure intensity after observing a failure and fixing the corresponding fault is constant. – This gives us the Basic model. – Decrement process 2 • The decrease in failure intensity after observing a failure and fixing the corresponding fault is smaller than the previous decrease. – This gives us the Logarithmic model.
  • 14. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 14 Reliability Models • Parameters of the models – λ0: The initial failure intensity observed at the beginning of system testing. – v0: The total number of system failures that we expect to observe over infinite time starting from the beginning of system testing. θ: A parameter representing n0n-linear drop in failure intensity in the Logarithmic model. Figure 15.3: Failure intensity λ as a function of cumulative failures µ.
  • 15. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 15 Reliability Models • Basic model Assumption: λ(µ) = λ0 (1 - µ/v0) dµ(τ)/dτ = λ0 (1 - µ(τ)/v0) µ(τ) = λ0(1 - µ/v0) λ(τ) = λ0.e -λ0 τ/v0 • Logarithmic model Assumption: λ(µ) = λ0e-θµ dµ(τ)/dτ = λ0e-θµ(τ) µ(τ) = ln(λ0θτ + 1)/θ λ(τ) = λ0/(λ0θτ + 1) Figure 15.4: Failure intensity λ as a function of execution time τ (λ0= 9 failures/unit time,v0= 500 failures, θ = 0.0075).
  • 16. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 16 Reliability Models Figure 15.4: Cumulative failure µ as a function of execution time τ (λ0= 9 failures/unit time,v0 = 500 failures, θ = 0.0075).
  • 17. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 17 Reliability Models • Example Assume that a software system is undergoing system level testing. The initial failure intensity of the system was 25 failures/CPU hours, and the current failure intensity is 5 failures/CPU hour. It has been decided by the project manager that the system will be released only after the system reaches a reliability level of at most 0.001 failures/CPU hour. From their experience the management team estimates that the system will experience a total of 1200 failures over infinite time. Calculate the additional length of system testing required before the system can be released. – The system will experience a total of 1200 failures over infinite time. Thus, we use the Basic model. – λc and λr are the current failure intensity and the failure intensity at the time of release. – Assume that the current failure intensity has been achieved after executing the system for τc hours. – Let λr be achieved after testing the system for a total of τr hours.
  • 18. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 18 Reliability Models • (Example continued) – (τr - τc) denotes the additional execution time requires to achieve λr. We can write λc and λr as follows. λc = λ0.e -λ0 τc/v0 λr = λ0.e -λ0 τr/v0 λc/ λr = (λ0.e -λ0τc/v0 )/(λ0.e -λ0 τr/v0 ) = e (τr - τc) λ0/v0 ln(λc/ λr) = (τr - τc) λ0/v0 (τr - τc)= (v0/ λ0)ln(λc/ λr) = (1200/25)ln(5/0.001) = 408.825 hours It is required to test the system for more time so that the CPU runs for another 408.825 hours to achieve the reliability level of 0.001 failures/hour.
  • 19. Software Testing and QA Theory and Practice (Chapter 15: Software Reliability) © Naik & Tripathy 19 Summary • Reliability is a user-oriented quality factor relating to system operation. • The chapter introduced the following. – Fault and failure – Execution and calendar time – Time interval between failures – Failures in periodic intervals – Failure intensity • Software reliability was defined in two ways. – The probability of failure-free operation of a system for a specified time in a given environment. – Failure intensity is a measure of reliability. • User’s perception of reliability: – The number of faults in a system. – How a user operates a system. • The number of faults in a system is influenced by the following: – Size and complexity of code. – Development process. – Personnel quality. – Operational environment • Operational profile – A quantitative characterization of how actual users operate a system. – Tabular and graphical representation • Applications of reliability metric • Reliability models – Six assumptions – Two models • Basic • Logarithmic

Editor's Notes

  • #2: ------------------------------------------------------------------ ------------------------------------------------------------------ ------------------------------------------------------------------- --------------------------------------------------------------------