Code Complete by Steve McConnell : Chapter 25

The author focuses on unit-testing in this chapter. Unit-testing is a hard activity for developers to swallow since its outcome, to find errors, is completely reverse of what other development activities try to achieve. The author then discuss a general approach to unit testing:

• Test for each relevant requirement.
• Test for each relevant design concern.
• Add test cases to those that tests the requirements and the design and data flow tests.

The author describes that it is impossible to prove that a program is correct by testing it since the number of test cases to exhaustively test even a simple program is too large.

The author suggests to use coverage monitors to make sure complete code path is covered. There are several methods to determine the set of test cases that cover complete code path including:

1. Structured Basis Testing: This involves calculating cyclomatic complexity and creating test cases based on it.
2. Data-flow Testing: The author defined different states in which data can be classified and then enlists anomalous sequences of data states.
3. Equivalence Partitioning: This involves dividing the data in partitions such that any one partition achieves what all combined achieves and redundancy is removed.
4. Error Guessing: It involves using heuristics to guess which part of the program has errors and try to create test cases for it. The author then provides kinds of errors that can be guessed.

• Boundary Analysis: Off-by-one errors are the most common bugs.
• Classes of bad data: Typical bad data set include
  
• Too much data
• Too little data
• Uninitialized data
• Invalid data
• Classes of good data: Errors can be present in processing of nominal data set too. Other good data sets worth checking are:
• Minimum normal configuration
• Maximum normal configuration
• Compatibility with old data
  

The author then provides some statistics on distribution of errors which says that 80 percent of the errors are found in 20 percent of the routines while 50 percent of the errors are found in 10 percent of the routines. The author cites a study to provide classification of errors that were detected and some suggestions:

• The scope of most errors is limited
• Many errors are outside the domain of construction
• Most implementation errors are programmer's fault
• Clerical errors are a common source of problems
• Misunderstanding the design is a common cause of programmer errors
• Avoiding errors in assignment statements is a key to quality
• Most errors are easy to fix

As mentioned previously, the proportion of construction errors in software decreases as the number of lines of code increases. The author then provides some data related to industry average figures for number of defects found in software.

The author then provides some guidelines to tackle the errors in testing itself.

The author then provides some list of tools to help in testing:

• Scaffolding
• Result comparators
• Test-data generators
• Coverage monitors
• Symbolic debuggers
• System perturbers
• Error databases
  

The author then talks about points to improve testing including planning for test, regression testing and keeping test records.